[RFE] Support Python 3.11 in the C extension about python-frozendict

[lurch]

#67 (comment)

[mcepl]

Even copying 3_10 to 3_11 directory doesn’t help:

[   13s] running build_ext
[   13s] building 'frozendict._frozendict' extension
[   13s] creating build/temp.linux-x86_64-cpython-311
[   13s] creating build/temp.linux-x86_64-cpython-311/frozendict
[   13s] creating build/temp.linux-x86_64-cpython-311/frozendict/src
[   13s] creating build/temp.linux-x86_64-cpython-311/frozendict/src/3_11
[   13s] gcc -Wsign-compare -DNDEBUG -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -fPIC -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/Include -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/stringlib -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/clinic -I/home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src -I/usr/include/python3.11 -c frozendict/src/3_11/frozendictobject.c -o build/temp.linux-x86_64-cpython-311/frozendict/src/3_11/frozendictobject.o -DPY_SSIZE_T_CLEAN
[   13s] In file included from frozendict/src/3_11/frozendictobject.c:6:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘_PyDict_CheckConsistency’:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:491:36: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]   491 |                 CHECK(mp->ma_values[i] != NULL);
[   13s]       |                                    ^
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:434:16: note: in definition of macro ‘CHECK’
[   13s]   434 |     do { if (!(expr)) { _PyObject_ASSERT_FAILED_MSG(op, Py_STRINGIFY(expr)); } } while (0)
[   13s]       |                ^~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:491:23: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]   491 |                 CHECK(mp->ma_values[i] != NULL);
[   13s]       |                       ^~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:434:16: note: in definition of macro ‘CHECK’
[   13s]   434 |     do { if (!(expr)) { _PyObject_ASSERT_FAILED_MSG(op, Py_STRINGIFY(expr)); } } while (0)
[   13s]       |                ^~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘_d_PyDict_Next’:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:799:30: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]   799 |         value = mp->ma_values[i];
[   13s]       |                              ^
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:799:17: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]   799 |         value = mp->ma_values[i];
[   13s]       |                 ^~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘dict_dealloc’:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:830:25: warning: initialization of ‘PyObject **’ {aka ‘struct _object **’} from incompatible pointer type ‘PyDictValues *’ {aka ‘struct _dictvalues *’} [-Wincompatible-pointer-types]
[   13s]   830 |     PyObject **values = mp->ma_values;
[   13s]       |                         ^~
[   13s] In file included from /usr/include/python3.11/Python.h:45,
[   13s]                  from frozendict/src/3_11/frozendictobject.c:1:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: In function ‘dict_traverse’:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:39: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                                       ^
[   13s] /usr/include/python3.11/objimpl.h:199:13: note: in definition of macro ‘Py_VISIT’
[   13s]   199 |         if (op) {                                                       \
[   13s]       |             ^~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:26: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                          ^~
[   13s] /usr/include/python3.11/objimpl.h:199:13: note: in definition of macro ‘Py_VISIT’
[   13s]   199 |         if (op) {                                                       \
[   13s]       |             ^~
[   13s] In file included from /usr/include/python3.11/Python.h:38:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:39: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                                       ^
[   13s] /usr/include/python3.11/pyport.h:24:38: note: in definition of macro ‘_Py_CAST’
[   13s]    24 | #define _Py_CAST(type, expr) ((type)(expr))
[   13s]       |                                      ^~~~
[   13s] /usr/include/python3.11/objimpl.h:200:30: note: in expansion of macro ‘_PyObject_CAST’
[   13s]   200 |             int vret = visit(_PyObject_CAST(op), arg);                  \
[   13s]       |                              ^~~~~~~~~~~~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:17: note: in expansion of macro ‘Py_VISIT’
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                 ^~~~~~~~
[   13s] /usr/include/python3.11/pyport.h:24:37: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]    24 | #define _Py_CAST(type, expr) ((type)(expr))
[   13s]       |                                     ^
[   13s] /usr/include/python3.11/object.h:107:28: note: in expansion of macro ‘_Py_CAST’
[   13s]   107 | #define _PyObject_CAST(op) _Py_CAST(PyObject*, (op))
[   13s]       |                            ^~~~~~~~
[   13s] /usr/include/python3.11/objimpl.h:200:30: note: in expansion of macro ‘_PyObject_CAST’
[   13s]   200 |             int vret = visit(_PyObject_CAST(op), arg);                  \
[   13s]       |                              ^~~~~~~~~~~~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:1090:17: note: in expansion of macro ‘Py_VISIT’
[   13s]  1090 |                 Py_VISIT(mp->ma_values[i]);
[   13s]       |                 ^~~~~~~~
[   13s] frozendict/src/3_11/frozendictobject.c: In function ‘frozendict_merge’:
[   13s] frozendict/src/3_11/frozendictobject.c:453:41: error: invalid use of undefined type ‘struct _dictvalues’
[   13s]   453 |                 value = other->ma_values[i];
[   13s]       |                                         ^
[   13s] frozendict/src/3_11/frozendictobject.c:453:25: error: invalid use of incomplete typedef ‘PyDictValues’ {aka ‘struct _dictvalues’}
[   13s]   453 |                 value = other->ma_values[i];
[   13s]       |                         ^~~~~
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c: At top level:
[   13s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_11/cpython_src/Objects/dictobject.c:431:1: warning: ‘_PyDict_CheckConsistency’ defined but not used [-Wunused-function]
[   13s]   431 | _PyDict_CheckConsistency(PyObject *op, int check_content)
[   13s]       | ^~~~~~~~~~~~~~~~~~~~~~~~
[   13s] warning: build_ext: building extension "frozendict._frozendict" failed: command '/usr/bin/gcc' failed with exit code 1

BTW with all versions of Python I get this warning:

[   12s] In file included from frozendict/src/3_9/frozendictobject.c:6:
[   12s] /home/abuild/rpmbuild/BUILD/frozendict-2.3.5/frozendict/src/3_9/cpython_src/Objects/dictobject.c:443:1: warning: ‘_PyDict_CheckConsistency’ defined but not used [-Wunused-function]
[   12s]   443 | _PyDict_CheckConsistency(PyObject *op, int check_content)
[   12s]       | ^~~~~~~~~~~~~~~~~~~~~~~~

[Marco-Sulla]

@mcepl Well, dict in CPython 3.11 is changed a lot. I started to incorporate the new code into frozendict in the cpython_3_11 branch.

[PhorstenkampFuzzy]

For me a pipline just broke on because the newer version where not usable on 3.11. But the older versions build just fine for me on 3.11 using poetry and the python:3.11-slim image. Can it lead to problems?

[Marco-Sulla]

@PhorstenkampFuzzy

This is quite strange.

frozendict setup now builds a pure python wheel for archs and/or Python versions not (yet) supported by the C Extension.

Can you post:

• the frozendict version
• the pipeline
• the error

?

[PhorstenkampFuzzy]

It does work fine for me and builds on the fly.
I am using:
python 3.11.0 on windows64 bit and the current python:3.11-slim docker image.
I am installing it as a poetry dependency and as a dependency when installing a wheel.
I am currently using 2.3.5 and 2.3.4. (Where there is a *.tar.gz published).

I have no problem at all so long as i do not increase the version to something where there is no *.tar.gz to build from.

[Marco-Sulla]

@PhorstenkampFuzzy Yes, this is fixed in 2.3.7. Sorry for the confusion.