Python 3.10 compatiblity about pyq

[b-nln]

There are no plans to release 5.0.1, as there is no full python 3.10 support yet.

[sashkab]

After first problem fixed, there is second problem.

$ python3.10 -mpip list
Package    Version
---------- -------
attrs      21.2.0
coverage   6.0
iniconfig  1.1.1
numpy      1.21.2
packaging  21.0
pip        21.2.4
pluggy     1.0.0
py         1.10.0
pyparsing  2.4.7
pyq        5.0.0
pytest     6.2.5
pytest-cov 3.0.0
pytest-pyq 1.2.0
setuptools 58.2.0
toml       0.10.2
tomli      1.2.1
wheel      0.37.0
$ pyq -mpip list
/Users/user/.virtualenvs/310/bin/pyq: No module named pip

I believe we saw this kind of issues before, but can't recall if it was addressed. It was related to the virutalenv and venv modules, how they are now implemented... which breaks logic PyQ relies on.

[sashkab]

Interestingly:

$ QHOME=$VIRTUAL_ENV/q $VIRTUAL_ENV/q/m64/q
...
q)p)import numpy
q)p)import pytest
q)p)import pip
q)p)print(pip.__file__)
/Users/user/.virtualenvs/310/lib/python3.10/site-packages/pip/__init__.py
q)p)print(sys.executable)
/Users/user/.virtualenvs/310/bin/pyq
q)p)print(sys.path)
['/usr/local/Cellar/python@3.10/3.10.0_1/Frameworks/Python.framework/Versions/3.10/lib/python310.zip', '/usr/local/Cellar/python@3.10/3.10.0_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10', '/usr/local/Cellar/python@3.10/3.10.0_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/lib-dynload', '/Users/user/.virtualenvs/310/lib/python3.10/site-packages']
q)p)print(sys.prefix)
/Users/user/.virtualenvs/310
q)p)print(sys.exec_prefix)
/Users/user/.virtualenvs/310

but

$ pyq
Python 3.10.0 (default, Oct  6 2021, 01:11:32) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.executable
'/Users/user/.virtualenvs/310/bin/pyq'
>>> sys.path
['', '/usr/local/Cellar/[email protected]/3.10.0_1/Frameworks/Python.framework/Versions/3.10/lib/python310.zip', '/usr/local/Cellar/[email protected]/3.10.0_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10', '/usr/local/Cellar/[email protected]/3.10.0_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/lib-dynload']
>>> sys.prefix
'/usr/local/Cellar/python@3.10/3.10.0_1/Frameworks/Python.framework/Versions/3.10'
>>> sys.exec_prefix
'/usr/local/Cellar/python@3.10/3.10.0_1/Frameworks/Python.framework/Versions/3.10'

[sashkab]

NB: mapping import was fixed in the internal MR 746, which is planned to be released as 5.0.1...

[MurrMack]

Hi Aleks - is there a plan for 5.0.1 to be released any time soon? the default version of python on ubuntu 22.04 LTS is 3.10 so others may start running into this issue as upgrades begin to occur. Cheers!

[MurrMack]

Is there anyway to force pyq to use a different version of python? My default version of python is 3.9 using update-alternatives however when I run pyq it tries to still use 3.10

CPUS=24 pyq                                                                                                                                                                         
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/murray/.local/lib/python3.10/site-packages/pyq/__init__.py", line 4, in <module>
    from collections import Mapping as _Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)
'2022.08.15T08:58:06.931 python
  [5]  /home/murray/q/p.k:7: p)from pyq import q
                             ^
  [2]  /home/murray/q/python.q:1: \l p.k

python --version  
Python 3.9.13

python3 --version
Python 3.9.13

[b-nln]

Use virtualenv with python3.9 and install PyQ in there as described in the installation guide.

[sashkab]

After spending hours earlier this week in order to rewrite Python initialization for 3.10 and failing, I came up with a quick workaround for the importing modules installed in the virtual environment when using Python 3.10 and PyQ:

$ pyq
Python 3.10.5 (main, Jul 25 2022, 15:52:08) [GCC 11.2.1 20220219] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import markdown
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.virtualenvs/3/lib/python3.10/site-packages/pyq/__init__.py", line 820, in __import__
    m = _imp(name, globals, locals, fromlist, level)
ModuleNotFoundError: No module named 'markdown'

Unfortunately, due to Python Initialization changes in 3.10, sys.path is not properly initialized when Python is loaded from kdb+. Dirty workaround is to extend sys.path to what it should have been:

>>> import sys
>>> sys.path.extend(['/home/user/.virtualenvs/3/lib/python3.10/site-packages', '/usr/lib/python3.10/site-packages'])
>>> import markdown
>>> markdown.__version__
'3.3.4'

Basically, first path is your $VIRTUAL_ENV/lib/python3.10/site-packages, and second path is where system python 3 packages are installed.