BUILD: Pandas 1.2.5 build no longer works about pandas

[lithomas1]

Can you try upgrading your Cython?

This looks like aio-libs/aiohttp#6600, which someone reports was fixed in Cython 0.29.5

[adrianonobre]

potentially caused by this:
numpy/numpy@b3f9fc0

[myriky]

same issue

[lithomas1]

We are not planning on adding numpy 2 support to pandas 1.2.5 since numpy 2 compat is a pretty involved process and pandas 1.2.5 is already 3 years old at this point.

Please upgrade to pandas 2.2.2 as that is the first pandas version to support numpy 2.

[adrianonobre]

Hi @lithomas1, I think there's a misunderstanding (I'll take numpy 2.0 mention from the title)
I'm not trying to use numpy 2.
The issue is rather that pandas 1.2.5 build no longer works (with numpy 1.26, see notes in the "installation logs" section).

I was just pointing out that a change done for numpy 2 seems to have bled into numpy 1.26 as a breaking change, potentially affecting projects that depend on numpy 1.26

Thanks for looking into this

[lithomas1]

Sorry for the misunderstanding.

Did you mention that this started failing after numpy 2.0 was released?
(If so, this might because the build of pandas 2.0 is pulling the newest numpy, not the one you have installed)

[adrianonobre]

No worries! (and thanks again for your time, @lithomas1 )

Correct.
We noticed the pandas build started failing yesterday "out of the blue".
We've got dependencies versions pinned in our requirements file. We didn't make any changes to these versions. Here's a couple of them:
numpy==1.26.4
pandas==1.2.5

Investigating a bit we noticed that the NumPy project did a release release this week (2.0) and we found this change which seems to align with the error message we're getting in the pandas build (i.e. a missing struct attribute elsize):

      pandas/_libs/src/ujson/python/JSONtoObj.c:260:33: error: no member named 'elsize' in 'struct _PyArray_Descr'
              npyarr->elsize = dtype->elsize;
                               ~~~~~  ^

The easiest way a colleague found to repro this is as follows:

# make virtual env (make sure python is 3.11)
python -m venv ./sample-venv
# activate virtual env
. ./sample-venv/bin/activate
# install same packages we do in the project (relevant to pandas/numpy)
pip install --no-cache six==1.16.0
pip install --no-cache pytz==2024.1
pip install --no-cache python-dateutil==2.9.0.post0
pip install --no-cache numpy==1.26.4
pip install --no-cache cython==0.29.21
# blows up
pip install --no-cache pandas==1.2.5 

[lithomas1]

Can you try installing pandas with --no-build-isolation?

(You'll need all dependencies pre-installed, but this should force pip to use your numpy, and not pull its own numpy)

[mttr]

For what it's worth, this problem seems to be due to this line in pyproject.toml and can be fixed by changing it to "numpy<2; python_version>='3.9'"

[lithomas1]

Yeah, this is maybe something to consider in the future, but for now the --no-build-isolation step should fix it.

[adrianonobre]

Can you try installing pandas with --no-build-isolation

NEWEST EDIT:

Workaround:
We were able to get it working by using "--no-build-isolation" + bumping a cython version to 0.29.37. So:

# make virtual env (make sure python is 3.11)
python -m venv ./sample-venv
# activate virtual env
. ./sample-venv/bin/activate
# install same packages we do in the project (relevant to pandas/numpy)
pip install --no-cache six==1.16.0
pip install --no-cache pytz==2024.1
pip install --no-cache python-dateutil==2.9.0.post0
pip install --no-cache numpy==1.26.4
pip install --no-cache cython==0.29.37  <--------------- IMPORTANT: version
pip install --no-cache pandas==1.2.5 --no-build-isolation <------- IMPORTANT: no-build-isolation flag
# WORKS NOW!

---

OLD:

I tried pip install pandas==1.2.5 --no-build-isolation and got a different error:

(EDIT: fwiw I got Python 3.11 when I got the the error below, someone reported that it worked while they were using Python 3.9)

      pandas/_libs/algos.c:235:12: fatal error: 'longintrepr.h' file not found
        #include "longintrepr.h"
                 ^~~~~~~~~~~~~~~

[adrianonobre]

Thanks for your help @lithomas1 (and @mttr ) ! 🙏