Problem: Installation of Python Modules via requirements.txt doesnt seem to work as intended about semaphore

[tboerger]

#2083

[Fabl0s]

Just tried the current develop docker build - works now, thank you

[pushpinderbal]

@fiftin
Same issue when running from the Web UI. Maybe it has something to do with all requirements.txt modules being installed in /home/semaphore and not in the venv

8a04df013a1c:~$ cat /etc/semaphore/requirements.txt
pywinrm
pymssql
netaddr
passlib

8a04df013a1c:~$ /opt/semaphore/venv/bin/pip3 list -v
Package            Version     Location                                            Installer
------------------ ----------- --------------------------------------------------- ---------
ansible            9.4.0       /opt/semaphore/venv/lib/python3.11/site-packages    pip
ansible-core       2.16.7      /opt/semaphore/venv/lib/python3.11/site-packages    pip
boto3              1.34.115    /opt/semaphore/venv/lib/python3.11/site-packages    pip
botocore           1.34.115    /opt/semaphore/venv/lib/python3.11/site-packages    pip
certifi            2024.2.2    /opt/semaphore/venv/lib/python3.11/site-packages    pip
cffi               1.16.0      /opt/semaphore/venv/lib/python3.11/site-packages    pip
charset-normalizer 3.3.2       /opt/semaphore/venv/lib/python3.11/site-packages    pip
cryptography       42.0.7      /opt/semaphore/venv/lib/python3.11/site-packages    pip
idna               3.7         /opt/semaphore/venv/lib/python3.11/site-packages    pip
Jinja2             3.1.4       /opt/semaphore/venv/lib/python3.11/site-packages    pip
jmespath           1.0.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
MarkupSafe         2.1.5       /opt/semaphore/venv/lib/python3.11/site-packages    pip
netaddr            1.3.0       /home/semaphore/.local/lib/python3.11/site-packages pip
packaging          23.2        /usr/lib/python3.11/site-packages
passlib            1.7.4       /home/semaphore/.local/lib/python3.11/site-packages pip
pip                24.0        /opt/semaphore/venv/lib/python3.11/site-packages    pip
pycparser          2.22        /opt/semaphore/venv/lib/python3.11/site-packages    pip
pymssql            2.3.0       /home/semaphore/.local/lib/python3.11/site-packages pip
pyparsing          3.1.1       /usr/lib/python3.11/site-packages
pyspnego           0.10.2      /home/semaphore/.local/lib/python3.11/site-packages pip
python-dateutil    2.9.0.post0 /opt/semaphore/venv/lib/python3.11/site-packages    pip
pywinrm            0.4.3       /home/semaphore/.local/lib/python3.11/site-packages pip
PyYAML             6.0.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
requests           2.32.3      /opt/semaphore/venv/lib/python3.11/site-packages    pip
requests-ntlm      1.2.0       /home/semaphore/.local/lib/python3.11/site-packages pip
resolvelib         1.0.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
s3transfer         0.10.1      /opt/semaphore/venv/lib/python3.11/site-packages    pip
setuptools         65.5.0      /opt/semaphore/venv/lib/python3.11/site-packages    pip
six                1.16.0      /opt/semaphore/venv/lib/python3.11/site-packages    pip
urllib3            2.2.1       /opt/semaphore/venv/lib/python3.11/site-packages    pip
xmltodict          0.13.0      /home/semaphore/.local/lib/python3.11/site-packages pip

The UI reports the error: msg: 'winrm or requests is not installed: No module named ''winrm'''
However, if I run the ansible-playbook command logged into the container

• docker exec -it docker-semaphore-1 bash
• /opt/semaphore/venv/bin/python3 /opt/semaphore/venv/bin/ansible-playbook -i inventory.yml --user svc-ansible --ask-pass --extra-vars '{"ANSIBLE_HOST_KEY_CHECKING":false,"ansible_connection":"winrm","ansible_port":"5986","ansible_winrm_server_cert_validation":"ignore","semaphore_vars":{"task_details":{"id":2,"url":null,"username":"admin"}}}' --fork 30 ping.yml

The above works OK and does not report the same module missing error

8a04df013a1c:~$ semaphore version
v2.9.112-4f95ac8-1717065276

maybe related to the fix implemented on: #2060 and the install command

semaphore/deployment/docker/server/server-wrapper

Line 193 in 80e2c8e

pip3 install --upgrade --user \

looks like the --user flag for pip is not recommended/supported within venv

[tboerger]

A PR which removes the --user flag from the server/runner wrapper is incoming, I'm on it. Hopefully this will resolve the remaining issue related to requirements.txt