Comments (8)
commented on May 27, 2024
1
Yeap your solution works at least on linux platforms 🙂 As the use of ninja is internal of cmake-build-extension and the end user have no idea of what is going on in this tricky case, may I suggest to add this code in the project? For instance here:
diff --git a/src/cmake_build_extension/build_extension.py b/src/cmake_build_extension/build_extension.py
index a27d144..c4667c7 100644
--- a/src/cmake_build_extension/build_extension.py
+++ b/src/cmake_build_extension/build_extension.py
@@ -148,6 +148,7 @@ class BuildExtension(build_ext):
"-GNinja",
f"-DCMAKE_BUILD_TYPE={ext.cmake_build_type}",
f"-DCMAKE_INSTALL_PREFIX:PATH={cmake_install_prefix}",
+ f"-DCMAKE_MAKE_PROGRAM={shutil.which('ninja')}",
]
# Extend the configure arguments with those passed from the extensionfrom cmake-build-extension.
commented on May 27, 2024
1
I agree about Windows compatibility, it should be ok.
from cmake-build-extension.
diegoferigo
commented on May 27, 2024
Hi @remy-phelipot, sorry for the late reply, I missed this issue somehow. ninja is a dependency, and it should provide ootb the missing command.
cmake-build-extension/setup.cfg
Line 50 in bb836e4
Can you provide more info on your use case?
from cmake-build-extension.
commented on May 27, 2024
Hello @diegoferigo.
I tested again and indeed ninja seems to be correctly installed, so my original message do not correctly reflect the actual problem.
I'm encountering the following error when building the example module (I added ninja as build dependency in pyproject.toml to be sure it is installed, but is not necessary):
$ python -m build --wheel
* Creating venv isolated environment...
* Installing packages in isolated environment... (cmake_build_extension, ninja, numpy, pybind11, setuptools>=45, setuptools_scm[toml]>=6.0, wheel)
* Getting dependencies for wheel...
running egg_info
writing src/mymath.egg-info/PKG-INFO
writing dependency_links to src/mymath.egg-info/dependency_links.txt
writing entry points to src/mymath.egg-info/entry_points.txt
writing requirements to src/mymath.egg-info/requires.txt
writing top-level names to src/mymath.egg-info/top_level.txt
writing manifest file 'src/mymath.egg-info/SOURCES.txt'
* Installing packages in isolated environment... (wheel)
* Building wheel...
running bdist_wheel
running build
running build_py
running build_ext
==> Configuring:
$ cmake -S /home/phelipot/projects/cmake-build-extension/example -B /home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig -DPython3_ROOT_DIR=/tmp/build-env-j2geah8k -DCALL_FROM_SETUP_PY:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=OFF -DEXAMPLE_WITH_SWIG:BOOL=ON -DEXAMPLE_WITH_PYBIND11:BOOL=OFF
==> Building:
$ cmake --build /home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings --config Release
==> Installing:
$ cmake --install /home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings
-- Found Python3: /tmp/build-env-j2geah8k/bin/python3 (found version "3.8.10") found components: Interpreter Development.Module NumPy
-- Configuring done
-- Generating done
-- Build files have been written to: /home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings
[4/4] Linking CXX shared module bindings_swig/mymath/_swig_bindings.so
-- Install configuration: "Release"
-- Installing: /home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/libmymath.a
-- Up-to-date: /home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/include/mymath.h
-- Up-to-date: /home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/bin/print_answer_swig
-- Up-to-date: /home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathTargets.cmake
-- Up-to-date: /home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathTargets-release.cmake
-- Up-to-date: /home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathConfig.cmake
-- Up-to-date: /home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathConfigVersion.cmake
-- Installing: /home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/bindings.py
-- Installing: /home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/_swig_bindings.so
==> Configuring:
$ cmake -S /home/phelipot/projects/cmake-build-extension/example -B /home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_Pybind11Bindings -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11 -DPython3_ROOT_DIR=/tmp/build-env-j2geah8k -DCALL_FROM_SETUP_PY:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=OFF -DEXAMPLE_WITH_SWIG:BOOL=OFF -DEXAMPLE_WITH_PYBIND11:BOOL=ON
==> Building:
$ cmake --build /home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_Pybind11Bindings --config Release
==> Installing:
$ cmake --install /home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_Pybind11Bindings
CMake Error at CMakeLists.txt:2 (project):
Running
'/tmp/pip-build-env-rr5z5dv1/overlay/bin/ninja' '--version'
failed with:
No such file or directory
-- Configuring incomplete, errors occurred!
See also "/home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_Pybind11Bindings/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
File "/home/phelipot/projects/cmake-build-extension/example/.venv/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/home/phelipot/projects/cmake-build-extension/example/.venv/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/phelipot/projects/cmake-build-extension/example/.venv/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 261, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/tmp/build-env-j2geah8k/lib/python3.8/site-packages/setuptools/build_meta.py", line 221, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/tmp/build-env-j2geah8k/lib/python3.8/site-packages/setuptools/build_meta.py", line 207, in _build_with_temp_dir
self.run_setup()
File "/tmp/build-env-j2geah8k/lib/python3.8/site-packages/setuptools/build_meta.py", line 150, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 43, in <module>
setuptools.setup(
File "/tmp/build-env-j2geah8k/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/build-env-j2geah8k/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
self.run_command('build')
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/build-env-j2geah8k/lib/python3.8/site-packages/cmake_build_extension/build_extension.py", line 110, in run
self.build_extension(ext)
File "/tmp/build-env-j2geah8k/lib/python3.8/site-packages/cmake_build_extension/build_extension.py", line 224, in build_extension
subprocess.check_call(configure_command)
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-S', '/home/phelipot/projects/cmake-build-extension/example', '-B', '/home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_Pybind11Bindings', '-GNinja', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_INSTALL_PREFIX:PATH=/home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11', '-DPython3_ROOT_DIR=/tmp/build-env-j2geah8k', '-DCALL_FROM_SETUP_PY:BOOL=ON', '-DBUILD_SHARED_LIBS:BOOL=OFF', '-DEXAMPLE_WITH_SWIG:BOOL=OFF', '-DEXAMPLE_WITH_PYBIND11:BOOL=ON']' returned non-zero exit status 1.
ERROR Backend subproccess exited when trying to invoke build_wheel
If I produce the sdist and the wheel using python -m build, then the build process is executed successfully.
Do you know what is going on and if there is any solution?
from cmake-build-extension.
diegoferigo
commented on May 27, 2024
Calling python -m build triggers the generation of the sdist and, from it, the wheel. Passing the --wheel option skips the sdist generation and the wheel is built directly from the sources. In your case, there's a mismatch in the two behaviors.
I couldn't reproduce your problem on my PC (see details below). The example has two CMakeExtensions, one for SWIG and the other for pybind11. What's strange is that the former works, but the process fails when the latter is configured. It's odd that it complains about ninja, since it has already successfully built the first extension in the very same isolated environment.
Let's keep investigating, I suspect that is something specific to your setup since it's not happening in mine nor in CI that builds installs and tests in all major OSs.
python -m build --wheel
cmake-build-extension/example on master via △ v3.21.3 via 🐍 v3.8.10 🅒 /conda
❯ python -m build --wheel
* Creating venv isolated environment...
* Installing packages in isolated environment... (cmake_build_extension, numpy, pybind11, setuptools>=45, setuptools_scm[toml]>=6.0, wheel)
* Getting dependencies for wheel...
running egg_info
writing src/mymath.egg-info/PKG-INFO
writing dependency_links to src/mymath.egg-info/dependency_links.txt
writing entry points to src/mymath.egg-info/entry_points.txt
writing requirements to src/mymath.egg-info/requires.txt
writing top-level names to src/mymath.egg-info/top_level.txt
writing manifest file 'src/mymath.egg-info/SOURCES.txt'
* Installing packages in isolated environment... (wheel)
* Building wheel...
running bdist_wheel
running build
running build_ext
==> Configuring:
$ cmake -S /home/dferigo/git/cmake-build-extension/example -B /home/dferigo/git/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig -DPython3_ROOT_DIR=/tmp/build-env-hlr9nns_ -DCALL_FROM_SETUP_PY:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=OFF -DEXAMPLE_WITH_SWIG:BOOL=ON -DEXAMPLE_WITH_PYBIND11:BOOL=OFF
==> Building:
$ cmake --build /home/dferigo/git/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings --config Release
==> Installing:
$ cmake --install /home/dferigo/git/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /conda/bin/x86_64-conda-linux-gnu-cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /conda/bin/x86_64-conda-linux-gnu-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python3: /tmp/build-env-hlr9nns_/bin/python3 (found version "3.8.10") found components: Interpreter Development.Module NumPy
-- Found SWIG: /conda/bin/swig (found suitable version "4.0.2", minimum required is "4.0")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/dferigo/git/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings
[7/7] Linking CXX shared module bindings_swig/mymath/_swig_bindings.so
-- Install configuration: "Release"
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/libmymath.a
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/include/mymath.h
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/bin/print_answer_swig
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathTargets.cmake
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathTargets-release.cmake
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathConfig.cmake
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathConfigVersion.cmake
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/bindings.py
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig/_swig_bindings.so
==> Configuring:
$ cmake -S /home/dferigo/git/cmake-build-extension/example -B /home/dferigo/git/cmake-build-extension/example/build/temp.linux-x86_64-3.8_Pybind11Bindings -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX:PATH=/home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11 -DPython3_ROOT_DIR=/tmp/build-env-hlr9nns_ -DCALL_FROM_SETUP_PY:BOOL=ON -DBUILD_SHARED_LIBS:BOOL=OFF -DEXAMPLE_WITH_SWIG:BOOL=OFF -DEXAMPLE_WITH_PYBIND11:BOOL=ON
==> Building:
$ cmake --build /home/dferigo/git/cmake-build-extension/example/build/temp.linux-x86_64-3.8_Pybind11Bindings --config Release
==> Installing:
$ cmake --install /home/dferigo/git/cmake-build-extension/example/build/temp.linux-x86_64-3.8_Pybind11Bindings
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /conda/bin/x86_64-conda-linux-gnu-cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /conda/bin/x86_64-conda-linux-gnu-c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Python3: /tmp/build-env-hlr9nns_/bin/python3 (found version "3.8.10") found components: Interpreter Development.Module NumPy
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found pybind11: /tmp/build-env-hlr9nns_/lib/python3.8/site-packages/pybind11/include (found version "2.8.0" )
-- Configuring done
-- Generating done
-- Build files have been written to: /home/dferigo/git/cmake-build-extension/example/build/temp.linux-x86_64-3.8_Pybind11Bindings
[6/6] Linking CXX shared module bindings_pybind11/mymath/bindings.cpython-38-x86_64-linux-gnu.so
-- Install configuration: "Release"
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11/lib/libmymath.a
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11/include/mymath.h
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11/bin/print_answer_pybind11
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11/lib/cmake/MyMath/MyMathTargets.cmake
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11/lib/cmake/MyMath/MyMathTargets-release.cmake
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11/lib/cmake/MyMath/MyMathConfig.cmake
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11/lib/cmake/MyMath/MyMathConfigVersion.cmake
-- Installing: /home/dferigo/git/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_pybind11/bindings.cpython-38-x86_64-linux-gnu.so
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/mymath_swig
creating build/bdist.linux-x86_64/wheel/mymath_swig/lib
copying build/lib.linux-x86_64-3.8/mymath_swig/lib/libmymath.a -> build/bdist.linux-x86_64/wheel/mymath_swig/lib
creating build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake
creating build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
copying build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathTargets.cmake -> build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
copying build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathTargets-release.cmake -> build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
copying build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathConfigVersion.cmake -> build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
copying build/lib.linux-x86_64-3.8/mymath_swig/lib/cmake/MyMath/MyMathConfig.cmake -> build/bdist.linux-x86_64/wheel/mymath_swig/lib/cmake/MyMath
creating build/bdist.linux-x86_64/wheel/mymath_swig/bin
copying build/lib.linux-x86_64-3.8/mymath_swig/bin/print_answer_swig -> build/bdist.linux-x86_64/wheel/mymath_swig/bin
copying build/lib.linux-x86_64-3.8/mymath_swig/bin/__main__.py -> build/bdist.linux-x86_64/wheel/mymath_swig/bin
copying build/lib.linux-x86_64-3.8/mymath_swig/__init__.py -> build/bdist.linux-x86_64/wheel/mymath_swig
creating build/bdist.linux-x86_64/wheel/mymath_swig/include
copying build/lib.linux-x86_64-3.8/mymath_swig/include/mymath.h -> build/bdist.linux-x86_64/wheel/mymath_swig/include
copying build/lib.linux-x86_64-3.8/mymath_swig/bindings.py -> build/bdist.linux-x86_64/wheel/mymath_swig
copying build/lib.linux-x86_64-3.8/mymath_swig/_swig_bindings.so -> build/bdist.linux-x86_64/wheel/mymath_swig
creating build/bdist.linux-x86_64/wheel/mymath_pybind11
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/lib
copying build/lib.linux-x86_64-3.8/mymath_pybind11/lib/libmymath.a -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
copying build/lib.linux-x86_64-3.8/mymath_pybind11/lib/cmake/MyMath/MyMathTargets.cmake -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
copying build/lib.linux-x86_64-3.8/mymath_pybind11/lib/cmake/MyMath/MyMathTargets-release.cmake -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
copying build/lib.linux-x86_64-3.8/mymath_pybind11/lib/cmake/MyMath/MyMathConfigVersion.cmake -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
copying build/lib.linux-x86_64-3.8/mymath_pybind11/lib/cmake/MyMath/MyMathConfig.cmake -> build/bdist.linux-x86_64/wheel/mymath_pybind11/lib/cmake/MyMath
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/bin
copying build/lib.linux-x86_64-3.8/mymath_pybind11/bin/print_answer_pybind11 -> build/bdist.linux-x86_64/wheel/mymath_pybind11/bin
copying build/lib.linux-x86_64-3.8/mymath_pybind11/bin/__main__.py -> build/bdist.linux-x86_64/wheel/mymath_pybind11/bin
copying build/lib.linux-x86_64-3.8/mymath_pybind11/__init__.py -> build/bdist.linux-x86_64/wheel/mymath_pybind11
creating build/bdist.linux-x86_64/wheel/mymath_pybind11/include
copying build/lib.linux-x86_64-3.8/mymath_pybind11/include/mymath.h -> build/bdist.linux-x86_64/wheel/mymath_pybind11/include
copying build/lib.linux-x86_64-3.8/mymath_pybind11/bindings.cpython-38-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel/mymath_pybind11
running install_egg_info
running egg_info
writing src/mymath.egg-info/PKG-INFO
writing dependency_links to src/mymath.egg-info/dependency_links.txt
writing entry points to src/mymath.egg-info/entry_points.txt
writing requirements to src/mymath.egg-info/requires.txt
writing top-level names to src/mymath.egg-info/top_level.txt
writing manifest file 'src/mymath.egg-info/SOURCES.txt'
Copying src/mymath.egg-info to build/bdist.linux-x86_64/wheel/mymath-0.0.0-py3.8.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/mymath-0.0.0.dist-info/WHEEL
creating '/home/dferigo/git/cmake-build-extension/example/dist/tmpj8dg_w_n/mymath-0.0.0-cp38-cp38-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'mymath_pybind11/__init__.py'
adding 'mymath_pybind11/bindings.cpython-38-x86_64-linux-gnu.so'
adding 'mymath_pybind11/bin/__main__.py'
adding 'mymath_pybind11/bin/print_answer_pybind11'
adding 'mymath_pybind11/include/mymath.h'
adding 'mymath_pybind11/lib/libmymath.a'
adding 'mymath_pybind11/lib/cmake/MyMath/MyMathConfig.cmake'
adding 'mymath_pybind11/lib/cmake/MyMath/MyMathConfigVersion.cmake'
adding 'mymath_pybind11/lib/cmake/MyMath/MyMathTargets-release.cmake'
adding 'mymath_pybind11/lib/cmake/MyMath/MyMathTargets.cmake'
adding 'mymath_swig/__init__.py'
adding 'mymath_swig/_swig_bindings.so'
adding 'mymath_swig/bindings.py'
adding 'mymath_swig/bin/__main__.py'
adding 'mymath_swig/bin/print_answer_swig'
adding 'mymath_swig/include/mymath.h'
adding 'mymath_swig/lib/libmymath.a'
adding 'mymath_swig/lib/cmake/MyMath/MyMathConfig.cmake'
adding 'mymath_swig/lib/cmake/MyMath/MyMathConfigVersion.cmake'
adding 'mymath_swig/lib/cmake/MyMath/MyMathTargets-release.cmake'
adding 'mymath_swig/lib/cmake/MyMath/MyMathTargets.cmake'
adding 'mymath-0.0.0.dist-info/METADATA'
adding 'mymath-0.0.0.dist-info/WHEEL'
adding 'mymath-0.0.0.dist-info/entry_points.txt'
adding 'mymath-0.0.0.dist-info/top_level.txt'
adding 'mymath-0.0.0.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built mymath-0.0.0-cp38-cp38-linux_x86_64.whl
from cmake-build-extension.
commented on May 27, 2024
I reset my environment and do:
$ virtualenv -p python3 .venv
$ . .venv/bin/activate
(.venv) $ pip install --upgrade pip build
(.venv) $ python -m build --wheel
* Creating venv isolated environment...
* Installing packages in isolated environment... (cmake_build_extension, ninja, numpy, pybind11, setuptools>=45,
...
Successfully built mymath-0.0.0-cp38-cp38-linux_x86_64.whl
So far so good. But if I execute the command again it fails:
$ python -m build --wheel
* Creating venv isolated environment...
* Installing packages in isolated environment... (cmake_build_extension, ninja, numpy, pybind11, setuptools>=45,
...
==> Installing:
$ cmake --install /home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings
CMake Error at CMakeLists.txt:2 (project):
Running
'/tmp/build-env-y54elrvx/bin/ninja' '--version'
failed with:
No such file or directory
-- Configuring incomplete, errors occurred!
See also "/home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
File "/home/phelipot/projects/cmake-build-extension/example/.venv/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/home/phelipot/projects/cmake-build-extension/example/.venv/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/phelipot/projects/cmake-build-extension/example/.venv/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 261, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/tmp/build-env-bil93mdf/lib/python3.8/site-packages/setuptools/build_meta.py", line 221, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/tmp/build-env-bil93mdf/lib/python3.8/site-packages/setuptools/build_meta.py", line 207, in _build_with_temp_dir
self.run_setup()
File "/tmp/build-env-bil93mdf/lib/python3.8/site-packages/setuptools/build_meta.py", line 150, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 43, in <module>
setuptools.setup(
File "/tmp/build-env-bil93mdf/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/build-env-bil93mdf/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
self.run_command('build')
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/build-env-bil93mdf/lib/python3.8/site-packages/cmake_build_extension/build_extension.py", line 110, in run
self.build_extension(ext)
File "/tmp/build-env-bil93mdf/lib/python3.8/site-packages/cmake_build_extension/build_extension.py", line 224, in build_extension
subprocess.check_call(configure_command)
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '-S', '/home/phelipot/projects/cmake-build-extension/example', '-B', '/home/phelipot/projects/cmake-build-extension/example/build/temp.linux-x86_64-3.8_SwigBindings', '-GNinja', '-DCMAKE_BUILD_TYPE=Release', '-DCMAKE_INSTALL_PREFIX:PATH=/home/phelipot/projects/cmake-build-extension/example/build/lib.linux-x86_64-3.8/mymath_swig', '-DPython3_ROOT_DIR=/tmp/build-env-bil93mdf', '-DCALL_FROM_SETUP_PY:BOOL=ON', '-DBUILD_SHARED_LIBS:BOOL=OFF', '-DEXAMPLE_WITH_SWIG:BOOL=ON', '-DEXAMPLE_WITH_PYBIND11:BOOL=OFF']' returned non-zero exit status 1.
ERROR Backend subproccess exited when trying to invoke build_wheel
It seems temporary files are produced in the build directory that interfere between each consecutive build. If I delete the directory, then it works again like a charm.
(.venv) $ rm -rf build && python -m build --wheel
...
Successfully built mymath-0.0.0-cp38-cp38-linux_x86_64.whl
from cmake-build-extension.
diegoferigo
commented on May 27, 2024
Ow ok I finally understood the problem after your last comment! The CMakeCache.txt file stored in the build folder contains the path to the ninja command, which is installed in a virtualenv in /tmp. The virtualenv is deleted after the first time the wheel is built, but the build folder, being stored in-source, is kept. The second time you build the wheel, the CMakeCache.txt entry does not get updated with the new virtualenv folder in /tmp, therefore it fails.
I suspect that a workaround is passing in your setup.py the following option: f"-DCMAKE_MAKE_PROGRAM={shutil.which('ninja')}". Can you please try to check if it solves the problem?
from cmake-build-extension.
diegoferigo
commented on May 27, 2024
Yes this was the idea, thanks for trying! Do you have specific reasons to say that this works only on linux platforms? The shutil module should work also on Windows and macOS.
Feel free to open a PR with this change, otherwise I'll take care of it in the next days.
from cmake-build-extension.
Related Issues (18)
- Expose binaries of the CMake project
- Extend the example with also pybind11 bindings HOT 1
- Publish to conda-forge HOT 5
- Use gitignore when packaging with custom sdist command
- Allow disabling `CMakeExtension`s through environment variable
- Using cmake extension modules alongside other extension modules HOT 5
- linking against imported library HOT 4
- Adjust supported Python versions HOT 1
- Python 3.7 on Windows fails in CI
- Downstream packages inherit setuptools_scm behavior HOT 2
- sdist doesn't package source_dir HOT 7
- write __init__.py into a wrong folder
- Should always check ninja in run() of BuildExtension? HOT 2
- Would SWIG example `normalize_numpy` leak memory?
- Evaluate if recommend `~=` usage for dependencies versions HOT 1
- Update `README.md` to use the more common `src` code folder HOT 3
- Example cannot be imported in Windows if python >= 3.8 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cmake-build-extension.