Install fails because of metadata version on Mac OS about osqp-python

[sunilshah]

@vineetbansal

For some reason, subsequent install from source using git clone --recurse-submodules https://github.com/osqp/osqp-python failed to install a useable version. The suggested git fetch did not help.

pip3 install https://files.pythonhosted.org/packages/5a/61/7b94eb7de2b7c9d12d8757ef9d0f5dfe3c5dc0ba0e88f08d4ae929996a34/osqp-0.6.2.post8.tar.gz  

generates a useable osqp version in python 3.10.8.

However, the version tag is still 0.0.0. It does complain that cvxpy requirements are not met. However, I am able to run osqp solver from within cvxpy.

I am switching back to the older version pip installs (0.6.2.post0) which does work. I see some comments about cvxopt in the git log. By the way, on Apple silicon, cvxopt does have an glpk related unresolved issue.

[vineetbansal]

Hi @sunilshah - can you try to run pip uninstall osqp followed by a pip install --upgrade osqp again and see if it works? We had a transient version of osqp up on pyPI for a few hours yesterday which is no longer available, and my guess is that this is a pip cache issue that should go away with these steps.

[sunilshah]

@vineetbansal

I tried your suggestion but it did not help. I had built from source per instructions on the website but it had version 0.0.0 tagged. I rebuilt from source afresh just now. It installs but gives version 0.0.0. This is on arm64.

git clone --recurse-submodules https://github.com/osqp/osqp-python
cd osqp-python
pip3 setup.py install 
python3 setup.py install

..

Using /opt/homebrew/lib/python3.10/site-packages
Finished processing dependencies for osqp==0.0.0

python3
Python 3.10.8 (main, Oct 13 2022, 09:48:40) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import osqp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/lib/python3.10/site-packages/osqp-0.0.0-py3.10-macosx-12-arm64.egg/osqp/__init__.py", line 3, in <module>
    from ._version import version as __version__
ModuleNotFoundError: No module named 'osqp._version'

[vineetbansal]

@sunilshah - thanks for the details. I think we might have outdated instructions on our website somewhere. python setup.py install is no longer the correct way to build from source, but it's pip install . Can you try that, and maybe tell me where you found those instructions and I'll fix them.

[sunilshah]

@vineetbansal

Do you mean once you cd into the directory osqp-python run pip install ? That fails as well, asking for more requirements on the pip3 command. Please post the correct instructions for me to try.

By the way, to be complete

pip3 uninstall osqp

pip3 install -U osqp
Collecting osqp
  Downloading osqp-0.6.2.post8.tar.gz (227 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 227.6/227.6 kB 2.8 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Discarding https://files.pythonhosted.org/packages/5a/61/7b94eb7de2b7c9d12d8757ef9d0f5dfe3c5dc0ba0e88f08d4ae929996a34/osqp-0.6.2.post8.tar.gz (from https://pypi.org/simple/osqp/): Requested osqp from https://files.pythonhosted.org/packages/5a/61/7b94eb7de2b7c9d12d8757ef9d0f5dfe3c5dc0ba0e88f08d4ae929996a34/osqp-0.6.2.post8.tar.gz has inconsistent version: expected '0.6.2.post8', but metadata has '0.0.0'
  Using cached osqp-0.6.2.post5.tar.gz (226 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Discarding https://files.pythonhosted.org/packages/b5/1c/cc191ce88cb3f0d1aa6eee99553df0b1cc3659e3c01ca6414c3675add8ad/osqp-0.6.2.post5.tar.gz (from https://pypi.org/simple/osqp/): Requested osqp from https://files.pythonhosted.org/packages/b5/1c/cc191ce88cb3f0d1aa6eee99553df0b1cc3659e3c01ca6414c3675add8ad/osqp-0.6.2.post5.tar.gz has inconsistent version: expected '0.6.2.post5', but metadata has '0.0.0'
  Using cached osqp-0.6.2.post4.tar.gz (226 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Discarding https://files.pythonhosted.org/packages/17/d2/ab54ca05546939c72e5b410ab6d2d079f542c2d39f3d31e67fd3f2fe64f8/osqp-0.6.2.post4.tar.gz (from https://pypi.org/simple/osqp/): Requested osqp from https://files.pythonhosted.org/packages/17/d2/ab54ca05546939c72e5b410ab6d2d079f542c2d39f3d31e67fd3f2fe64f8/osqp-0.6.2.post4.tar.gz has inconsistent version: expected '0.6.2.post4', but metadata has '0.0.0'
  Using cached osqp-0.6.2.post0.tar.gz (219 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: qdldl in /opt/homebrew/lib/python3.10/site-packages (from osqp) (0.1.5.post2)
Requirement already satisfied: scipy>=0.13.2 in /opt/homebrew/lib/python3.10/site-packages (from osqp) (1.9.3)
Requirement already satisfied: numpy>=1.7 in /opt/homebrew/lib/python3.10/site-packages (from osqp) (1.23.5)
Building wheels for collected packages: osqp
  Building wheel for osqp (pyproject.toml) ... done
  Created wheel for osqp: filename=osqp-0.6.2.post0-cp310-cp310-macosx_12_0_arm64.whl size=156755 sha256=93652844ddc26aaa7691ea2a329c2cd131eedc71b4884cdd1b99625e5e3c5be8
  Stored in directory: /Users/sunilshah/Library/Caches/pip/wheels/58/15/b5/622a37c3e654078934cec9b518894a9f595c73cc8aefd6a4c2
Successfully built osqp
Installing collected packages: osqp
Successfully installed osqp-0.6.2.post0
sunilshah@Sunils-Mac-mini invest % python3
Python 3.10.8 (main, Oct 13 2022, 09:48:40) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import osqp
>>> exit()

This installs osqp-0.6.2.post0 and works correctly, but it does show as an outdated module in pip3 list.

[vineetbansal]

To install from source:
first upgrade pip by doing pip install --upgrade pip.
Then you have a few options:
pip install . (inside the cloned folder for the repository, master branch)
Or copy the source tar.gz link from pypi and run pip install <http://path/to/source.gz>:

pip install https://files.pythonhosted.org/packages/5a/61/7b94eb7de2b7c9d12d8757ef9d0f5dfe3c5dc0ba0e88f08d4ae929996a34/osqp-0.6.2.post8.tar.gz

Incidentally, I think the output above you're getting about Discarding.. etc:

Discarding https://files.pythonhosted.org/packages/5a/61/7b94eb7de2b7c9d12d8757ef9d0f5dfe3c5dc0ba0e88f08d4ae929996a34/osqp-0.6.2.post8.tar.gz (from https://pypi.org/simple/osqp/): Requested osqp from https://files.pythonhosted.org/packages/5a/61/7b94eb7de2b7c9d12d8757ef9d0f5dfe3c5dc0ba0e88f08d4ae929996a34/osqp-0.6.2.post8.tar.gz has inconsistent version: expected '0.6.2.post8', but metadata has '0.0.0'
...

might also be related to using a very old version of pip. So even if the source install works (and pip list shows you the correct version), you might want to then try the uninstall + pip install -U osqp step again and see if you're able to install v0.6.2.post8 that way.

[sunilshah]

@vineetbansal

pip version is current 22.3.1

pip3 --version
pip 22.3.1 from /opt/homebrew/lib/python3.10/site-packages/pip (python 3.10)

git clone --recurse-submodules https://github.com/osqp/osqp-python
cd osqp-python
pip3 install .
Processing /Users/sunilshah/osqp-python
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: qdldl in /opt/homebrew/lib/python3.10/site-packages (from osqp==0.0.0) (0.1.5.post2)
Requirement already satisfied: scipy>=0.13.2 in /opt/homebrew/lib/python3.10/site-packages (from osqp==0.0.0) (1.9.3)
Requirement already satisfied: numpy>=1.7 in /opt/homebrew/lib/python3.10/site-packages (from osqp==0.0.0) (1.23.5)
Building wheels for collected packages: osqp
  Building wheel for osqp (pyproject.toml) ... done
  Created wheel for osqp: filename=osqp-0.0.0-cp310-cp310-macosx_12_0_arm64.whl size=181950 sha256=d8aad1e3a83ff7af6f79e12d4816636a981b884580f4dbd7a440661f346843ad
  Stored in directory: /Users/sunilshah/Library/Caches/pip/wheels/2d/b4/5b/63a670afcfb1309f5321a6234985d1dfecdfa6b5c599bb8572
Successfully built osqp
Installing collected packages: osqp
  Attempting uninstall: osqp
    Found existing installation: osqp 0.6.2.post0
    Uninstalling osqp-0.6.2.post0:
      Successfully uninstalled osqp-0.6.2.post0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cvxpy 1.2.2 requires osqp>=0.4.1, but you have osqp 0.0.0 which is incompatible.
Successfully installed osqp-0.0.0

[vineetbansal]

@sunilshah - yes, I see the problem now and have fixed it on our end. An osqp version is not official till we push the tags upstream, which is why you're getting osqp-0.0.0 in the message above (and in pip list). Can you try the git clone and pip install steps again? Sorry for the trouble, and thanks for your patience..

[sunilshah]

@vineetbansal
Here it is.

git clone --recurse-submodules https://github.com/osqp/osqp-python
Cloning into 'osqp-python'...
remote: Enumerating objects: 7144, done.
remote: Counting objects: 100% (1522/1522), done.
remote: Compressing objects: 100% (513/513), done.
remote: Total 7144 (delta 980), reused 1430 (delta 913), pack-reused 5622
Receiving objects: 100% (7144/7144), 28.39 MiB | 7.04 MiB/s, done.
Resolving deltas: 100% (4460/4460), done.
Submodule 'osqp_sources' (https://github.com/oxfordcontrol/osqp) registered for path 'osqp_sources'
Cloning into '/Users/sunilshah/osqp-python/osqp_sources'...
remote: Enumerating objects: 25166, done.        
remote: Counting objects: 100% (1029/1029), done.        
remote: Compressing objects: 100% (344/344), done.        
remote: Total 25166 (delta 678), reused 968 (delta 651), pack-reused 24137        
Receiving objects: 100% (25166/25166), 45.31 MiB | 5.77 MiB/s, done.
Resolving deltas: 100% (17129/17129), done.
Submodule path 'osqp_sources': checked out 'f9fc23d3436e4b17dd2cb95f70cfa1f37d122c24'
Submodule 'lin_sys/direct/qdldl/qdldl_sources' (https://github.com/oxfordcontrol/qdldl.git) registered for path 'osqp_sources/lin_sys/direct/qdldl/qdldl_sources'
Cloning into '/Users/sunilshah/osqp-python/osqp_sources/lin_sys/direct/qdldl/qdldl_sources'...
remote: Enumerating objects: 770, done.        
remote: Counting objects: 100% (219/219), done.        
remote: Compressing objects: 100% (20/20), done.        
remote: Total 770 (delta 201), reused 202 (delta 199), pack-reused 551        
Receiving objects: 100% (770/770), 142.78 KiB | 1.59 MiB/s, done.
Resolving deltas: 100% (401/401), done.
Submodule path 'osqp_sources/lin_sys/direct/qdldl/qdldl_sources': checked out '7d16b70a10a152682204d745d814b6eb63dc5cd2'
sunilshah@Sunils-Mac-mini ~ % cd osqp-python 
sunilshah@Sunils-Mac-mini osqp-python % pip3 install .
Processing /Users/sunilshah/osqp-python
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: qdldl in /opt/homebrew/lib/python3.10/site-packages (from osqp==0.0.0) (0.1.5.post2)
Requirement already satisfied: numpy>=1.7 in /opt/homebrew/lib/python3.10/site-packages (from osqp==0.0.0) (1.23.5)
Requirement already satisfied: scipy>=0.13.2 in /opt/homebrew/lib/python3.10/site-packages (from osqp==0.0.0) (1.9.3)
Building wheels for collected packages: osqp
  Building wheel for osqp (pyproject.toml) ... done
  Created wheel for osqp: filename=osqp-0.0.0-cp310-cp310-macosx_12_0_arm64.whl size=182247 sha256=38f447fa0a95bd3fb215b4a14607bda5cc6a43e926c7169ce58ed8cc14addfcc
  Stored in directory: /Users/sunilshah/Library/Caches/pip/wheels/2d/b4/5b/63a670afcfb1309f5321a6234985d1dfecdfa6b5c599bb8572
Successfully built osqp
Installing collected packages: osqp
  Attempting uninstall: osqp
    Found existing installation: osqp 0.6.2.post0
    Uninstalling osqp-0.6.2.post0:
      Successfully uninstalled osqp-0.6.2.post0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cvxpy 1.2.2 requires osqp>=0.4.1, but you have osqp 0.0.0 which is incompatible.
Successfully installed osqp-0.0.0

By the way, it installed version labeled 0.0.0 and seems to work correctly from within cvxpy. I have only run one example on it.

[vineetbansal]

Since you're installing from source, you have to checkout the tag corresponding to the version that you wish to install. So in your case git checkout v0.6.2.post8 (a git fetch may be required first to make sure you have the tags fetched), before trying a pip install .. The version you have will work fine too, it's just that it's not the official released version corresponding to the binary wheels on pyPI, hence the 0.0.0.

I realize that none of this is intuitive and I'm giving you all this information piecemeal. I think we'll write up a detailed README/FAQ on this very soon.

[imciner2]

Hi @sunilshah and @vineetbansal, what is the status of the pip installation versions? I see that we have 0.6.3 on PyPi right now, so did that fix this installation problem?

[vineetbansal]

@imciner2 - this is fixed going forward on account of the following line in the build_default workflow (added a couple of weeks ago):

python -m build --sdist --outdir wheelhouse

python setup.py sdist used before wouldn't have picked up the version correctly.

This is also fixed as a one-off in v0.6.3 on pypi (I just verified by looking at the PKG-INFO file for that .tar.gz as well by actually installing it on a Mac M2.)

[sunilshah]

@imciner2

Yes, osqp version 6.3.0 does install and run using pip install -U osqp. I tested on M1 Mac (running native mode) OS version 13.4, python version 3.11.4 and pip version 23.1.2. It also runs and installs fine under Rosetta2.

[imciner2]

Great, thanks both for letting me know. I'll close this issue then.