Install fails on Python 3.12 due to missing distutils about whisper-diarization

[MahmoudAshraf97]

which package exactly causes the failure?

[telotortium]

I believe it's numpy 1.23.5:

$ source venv3.12/bin/activate
$ pip install -r requirements.txt
Collecting git+https://github.com/m-bain/whisperX.git@78dcfaab51005aa703ee21375f81ed31bc248560 (from -r requirements.txt (line 3))
  Cloning https://github.com/m-bain/whisperX.git (to revision 78dcfaab51005aa703ee21375f81ed31bc248560) to /private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-req-build-eej96ayq
  Running command git clone --filter=blob:none --quiet https://github.com/m-bain/whisperX.git /private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-req-build-eej96ayq
  Running command git rev-parse -q --verify 'sha^78dcfaab51005aa703ee21375f81ed31bc248560'
  Running command git fetch -q https://github.com/m-bain/whisperX.git 78dcfaab51005aa703ee21375f81ed31bc248560
  Running command git checkout -q 78dcfaab51005aa703ee21375f81ed31bc248560
  Resolved https://github.com/m-bain/whisperX.git to commit 78dcfaab51005aa703ee21375f81ed31bc248560
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting demucs (from -r requirements.txt (line 4))
  Cloning https://github.com/facebookresearch/demucs to /private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-install-gardeugs/demucs_ae033499040946c8be07b6a285bd9cbe
  Running command git clone --filter=blob:none --quiet https://github.com/facebookresearch/demucs /private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-install-gardeugs/demucs_ae033499040946c8be07b6a285bd9cbe
  Resolved https://github.com/facebookresearch/demucs to commit e976d93ecc3865e5757426930257e200846a520a
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting wget (from -r requirements.txt (line 1))
  Using cached wget-3.2.zip (10 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting nemo_toolkit==1.22.0 (from nemo_toolkit[asr]==1.22.0->-r requirements.txt (line 2))
  Using cached nemo_toolkit-1.22.0-py3-none-any.whl.metadata (15 kB)
Collecting deepmultilingualpunctuation (from -r requirements.txt (line 5))
  Using cached deepmultilingualpunctuation-1.0.1-py3-none-any.whl.metadata (4.0 kB)
Collecting huggingface-hub (from nemo_toolkit==1.22.0->nemo_toolkit[asr]==1.22.0->-r requirements.txt (line 2))
  Downloading huggingface_hub-0.23.0-py3-none-any.whl.metadata (12 kB)
Collecting numba (from nemo_toolkit==1.22.0->nemo_toolkit[asr]==1.22.0->-r requirements.txt (line 2))
  Using cached numba-0.59.1-cp312-cp312-macosx_11_0_arm64.whl.metadata (2.7 kB)
Collecting numpy<1.24,>=1.22 (from nemo_toolkit==1.22.0->nemo_toolkit[asr]==1.22.0->-r requirements.txt (line 2))
  Using cached numpy-1.23.5.tar.gz (10.7 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/commands/install.py", line 377, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 95, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 427, in resolve
    failure_causes = self._attempt_to_pin_criterion(name)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 239, in _attempt_to_pin_criterion
    criteria = self._get_updated_criteria(candidate)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 230, in _get_updated_criteria
    self._add_to_criteria(criteria, requirement, parent=candidate)
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/resolvelib/structs.py", line 156, in __bool__
    return bool(self._sequence)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
           ^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
    candidate = func()
                ^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 182, in _make_candidate_from_link
    base: Optional[BaseCandidate] = self._make_base_candidate_from_link(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 228, in _make_base_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 290, in __init__
    super().__init__(
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
    self.dist = self._prepare()
                ^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 222, in _prepare
    dist = self._prepare_distribution()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 301, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 525, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 640, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/operations/prepare.py", line 71, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 54, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 124, in _install_build_reqs
    build_reqs = self._get_build_requires_wheel()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/distributions/sdist.py", line 101, in _get_build_requires_wheel
    return backend.get_requires_for_build_wheel()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_internal/utils/misc.py", line 745, in get_requires_for_build_wheel
    return super().get_requires_for_build_wheel(config_settings=cs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "/Users/me/misc/build/whisper-diarization/venv3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/private/var/folders/wn/0jmk5twd7j756pcyqwtnfd4r0000gn/T/pip-build-env-4ezcfsc3/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 10, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils'

[transcriptionstream]

Have you tried using pip to install numpy and distutils in the venv prior to running the install?

[MahmoudAshraf97]

I guess the final solution to this dependency hell is to fork nemo and remove all the bloated requirements that aren't related to this project