How to successfully run a demo about tianshou

[opcode81]

We need to improve the developer docs in general and provide instructions that enable an installation of an environment that supports the full range of tasks.

For instance, in order to be able to install a Python env that supports Atari, one may need to install system-level depenendencies cmake and zlib1g prior to the Poetry install (on Ubuntu, apt-get install cmake zlib1g-dev).
Afterwards, I had to run pip install gymnasium[atari] and, for the game ROMs to be pullable, I had to run pip install gymnasium[accept-rom-license]. The poetry extra atari is not sufficient, unfortunately.

[Trinkle23897]

Please use source code installation, i.e. pip install -e .
We did a major refactor few months ago, and didn’t update the pypi wheel. cc @MischaPanch

[MischaPanch]

FYI: Release of the wheel in next version is planned for late January or early February

[MischaPanch]

If you clone the repo, the surest say to install is with poetry, e.g. using poetry install

[dantp-ai]

My system is:

• macOS 13.6.3
• Python

  >>> import sys
  >>> print(sys.version, sys.platform)
  3.11.7 (main, Dec 27 2023, 17:48:28) [Clang 15.0.0 (clang-1500.1.0.2.5)] darwin
  

---

• I tried running the same demo from above, after having installed the dependencies with poetry like so: poetry install --all-extras --with dev (without the extras, opencv-python and atari-py are not being installed).

• For running this demo, I had to additionally install manually pip install gymnasium[atari] and pip install gymnasium[accept-rom-license]. I also had to manually install pip install cmake

• envpool fails to install:

  ERROR: Could not find a version that satisfies the requirement envpool (from versions: none)
  ERROR: No matching distribution found for envpool
  

  • This seems to be an issue with envpool reported also on its Issues board here.

• After all this, when running python atari/atari_rainbow.py the program seems to be stuck because I get intermittently the message that ALE is loaded

  A.L.E: Arcade Learning Environment (version 0.8.1+53f58b7)
  [Powered by Stella]
  

[MischaPanch]

Thank you a lot for going through the installation and looking into this! I will check the atari example in particular and address the installation details, such that everything is installed at once with the extras.

Could you please also try mujoco or one of the other environments? Atari is a particularly complicated one, the problems might be due to gymnasium (on mac) and not due to tianshou

[dantp-ai]

Yes. I'll try it on a different environment.

Regarding envpool issue, meanwhile I found out that it does not work because there is no support for Mac currently (here and [Feature Request] Py3.10, Windows and MacOS support).

[Trinkle23897]

It should be configured properly as if it doesn't find a successful envpool import, it would fall back to gymnasium implementation in tianshou's atari example.

[dantp-ai]

I was able to run on macOS the example under box2d/acrobot_dualdqn.py.

I tried also to run mujoco/mujoco_sac.py, but so far no luck on macOS Ventura. It seems that gcc@9 doesn't compile on macOS versions higher than Monterey:

$ brew install gcc@9
...
gcc@9: This formula either does not compile or function as expected on macOS
versions newer than Monterey due to an upstream incompatibility.
Error: gcc@9: An unsatisfied requirement failed this build.

[dantp-ai]

On a Ubuntu-based machine I was more successful.

OS

Distributor ID: Ubuntu
Description:    Ubuntu 20.04.6 LTS
Release:        20.04
Codename:       focal

Python

>>> import sys
>>> print(sys.version, sys.platform)
3.11.7 (main, Dec 29 2023, 19:47:35) [GCC 9.4.0] linux

I successfully ran box2d/acrobot_dualdqn.py, python atari_dqn.py --task "PongNoFrameskip-v4", python mujoco_sac.py.

After having followed the steps in the Contributions section, I had to additionally do:

• poetry install --all-extras
• apt-get update && apt-get install -y python3-opencv

[MischaPanch]

As I suspected, the problem lies mainly in gymnasium and not in tianshou. I'll extend the readme and the pyproject to better deal with macos. Weirdly, the tests in CI on macos run through without envpool problems. Maybe we should extend them as well