pangmaoshashou
### Branch main branch (1.x version, such as `v1.0.0`, or `dev-1.x` branch) ### Prerequisite - [x] I have searched [Issues](https://github.com/open-mmlab/mmaction2/issues) and [Discussions](https://github.com/open-mmlab/mmaction2/discussions) but cannot get the expected help. - [x] I have read the [documentation](https://mmaction2.readthedocs.io/en/latest/) but cannot get the expected help. - [x] The bug has not been fixed in the [latest version](https://github.com/open-mmlab/mmaction2). ### Environment OS: macOS (arm64, Apple Silicon M-series chip) Python Version: 3.8 (managed via Conda) Target Project: MMAction2 Problematic Dependency: decord ### Describe the bug Hi MMAction2 Team and Community, I recently went through the process of installing MMAction2 on a macOS arm64 (Apple Silicon) machine and encountered significant difficulties with the `decord` dependency. Standard installation methods via pip or conda for `decord` failed on this platform. After extensive troubleshooting, I found a viable solution that involves using the `eva-decord` fork and specific versions of dependencies. I'm sharing this detailed process here to help other users facing similar issues and potentially to inform future documentation Verbesserungen. **Problem Summary:** The primary issue was that `pip` could not find a suitable distribution for `decord` because: 1. The PyPI JSON Simple API for `decord` did not list any source distributions (.tar.gz). 2. No pre-compiled `decord` wheels for `osx-arm64` were available on PyPI. 3. Mainstream Conda channels (dmlc, conda-forge) also lacked `osx-arm64` packages for `decord`. 4. Compiling the original `dmlc/decord` from source failed due to API incompatibilities with the latest FFmpeg (7.x). --- **Environment:** * OS: macOS (arm64, Apple Silicon M-series chip) * Python Version: 3.8 (managed via Conda) * Target Project: MMAction2 * Problematic Dependency: `decord` **Problem Description:** Attempting to install MMAction2 on a macOS arm64 system via `pip install -v -e .` failed because its dependency `decord>=0.4.1` could not be satisfied. The error message was `ERROR: Could not find a version that satisfies the requirement decord>=0.4.1 (from mmaction2) (from versions: none)`. **Troubleshooting and Resolution Process:** 1. **Initial Diagnosis: `pip` Fails to Find `decord`** * Attempts with `pip install "decord>=0.4.1" -v` and `pip index versions decord` both reported no matching distribution. * `curl -v https://pypi.org/simple/decord/` confirmed that network connectivity to PyPI Simple Index (HTML format) was working and contained entries for `decord`. * However, `curl -H "Accept: application/vnd.pypi.simple.v1+json" https://pypi.org/simple/decord/` revealed that **the JSON Simple API response from PyPI did not list any source distributions (`.tar.gz`) for `decord`**; it only listed wheel files. This was identified as the root cause for `pip` not finding a source package to compile, even with the `--no-binary :all:` flag. 2. **Attempting Conda Installation of `decord`** * `conda install -c dmlc decord` failed with `PackagesNotFoundError` (no osx-arm64 package in the dmlc channel). * `conda install -c conda-forge decord` also failed with `PackagesNotFoundError` (no osx-arm64 package in the conda-forge channel). * Conclusion: At the time, mainstream Conda channels did not provide pre-compiled `decord` packages for `osx-arm64`. 3. **Attempting to Build Original `dmlc/decord` from Source** * Cloned the repository: `git clone --recursive https://github.com/dmlc/decord.git`. * Installed build dependencies: Xcode Command Line Tools, Homebrew, and then `brew install cmake ffmpeg libomp`. * An initial network issue (`Could not resolve host: github.com`) was resolved by fixing local network/DNS settings. * **Build Failure:** When attempting to compile with the latest FFmpeg installed via Homebrew (version 7.x at the time), `cmake` could find FFmpeg, but the `make` stage failed with compilation errors related to symbols like `AVBSFContext` and `av_bsf_free`, which had changed in newer FFmpeg APIs. This indicated an incompatibility between the original `decord` source code and FFmpeg 7.x. 4. **Switching to the `eva-decord` Fork** * Based on a suggestion, switched to the `georgia-tech-db/eva-decord` fork ([https://github.com/georgia-tech-db/eva-decord](https://github.com/georgia-tech-db/eva-decord)), which claimed support for Apple Silicon and newer FFmpeg versions (5.x, 6.x). * Cloned the fork: `git clone --recursive https://github.com/georgia-tech-db/eva-decord.git`. 5. **Adjusting FFmpeg Version for `eva-decord` Compatibility** * An initial attempt to compile `eva-decord` with FFmpeg 7.x resulted in `cmake` success, but `make` failed during the compilation of `src/audio/audio_reader.cc`. Errors indicated missing members like `channels` in `AVCodecParameters` and `channel_layout` in `AVFrame`, pointing to an API incompatibility between `eva-decord` (primarily supporting up to FFmpeg 6.x) and FFmpeg 7.x. * **Solution: Downgrade FFmpeg.** * Uninstalled FFmpeg 7.x: `brew uninstall ffmpeg` * Installed FFmpeg 6.x (Homebrew provided an `ffmpeg@6` package): `brew install ffmpeg@6` (which installed version 6.1.2_10). * Updated the `PKG_CONFIG_PATH` environment variable to point to the `pkgconfig` directory of the newly installed `ffmpeg@6`: ```bash unset PKG_CONFIG_PATH export PKG_CONFIG_PATH="$(brew --prefix ffmpeg@6)/lib/pkgconfig:$PKG_CONFIG_PATH" ``` 6. **Building and Installing `eva-decord`** * In the `eva-decord` source directory, compiled using the downgraded FFmpeg 6.x: ```bash cd eva-decord rm -rf build mkdir build && cd build cmake .. -DUSE_CUDA=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=${CONDA_PREFIX} make -j8 # or make ``` The compilation process showed several **warnings** about deprecated FFmpeg APIs but no **errors**, eventually succeeding with `[100%] Built target decord`. The C++ core library was successfully built. * **Installing Python Bindings - Crucial Step:** * An initial `pip install .` from the `eva-decord` root directory failed, as `setup.py` was not present there. * **Identified Issue:** After successfully building and installing `eva-decord` (from its `python/` subdirectory), `pip list` showed the package name as `eva-decord`, not `decord`. This would cause MMAction2's dependency resolver to fail. * **Solution:** 1. Navigate to the `eva-decord/python/` directory. 2. Edit the `setup.py` file. Modify the `name` parameter in the `setup()` function call from `name='eva-decord'` to `name='decord'`. 3. Save `setup.py`. 4. Uninstall any previously installed versions of `eva-decord` or `decord`: ```bash pip uninstall eva-decord -y pip uninstall decord -y ``` 5. In the `eva-decord/python/` directory, reinstall using the modified `setup.py`: ```bash pip install . -vvv ``` * **Verification:** `pip list | grep decord` then showed `decord 0.6.0`. In Python, `import decord; print(decord.__name__, decord.__version__)` confirmed `decord 0.6.0`. 7. **Installing MMAction2** * Navigated back to the `mmaction2` source directory. * Executed `pip install -v -e .`. * **Success!** The MMAction2 dependency resolver found the installed `decord 0.6.0` package and proceeded to install MMAction2 and its other dependencies successfully. **Summary and Recommendations:** * Resolving the `decord` dependency for projects like MMAction2 on macOS arm64 can be challenging due to the lack of official source distributions (sdist) on PyPI and limited arm64 support in Conda channels for `decord`. * The `georgia-tech-db/eva-decord` fork serves as a viable alternative, offering better support for Apple Silicon and more recent FFmpeg versions. * Ensuring FFmpeg version compatibility is crucial when building `eva-decord` (or the original `decord`) from source. Current experience suggests FFmpeg 6.x (e.g., `ffmpeg@6` from Homebrew) works well with `eva-decord`. * **Key Fix:** If using a `decord` fork that uses a different package name in its `setup.py` (like `eva-decord`), it's necessary to modify its `name` parameter to `decord`. This allows projects like MMAction2, which depend on the standard `decord` package name, to correctly identify the dependency. * Correctly setting the `PKG_CONFIG_PATH` environment variable to point to the `lib/pkgconfig` directory of the chosen FFmpeg version is essential for CMake to locate the FFmpeg libraries. Hopefully, this report will assist other developers facing similar challenges on macOS arm64. --- ### Reproduces the problem - code sample _No response_ ### Reproduces the problem - command or script _No response_ ### Reproduces the problem - error message _No response_ ### Additional information _No response_