salesforce / ai-economist Goto Github PK

Foundation is a flexible, modular, and composable framework to model socio-economic behaviors and dynamics with both agents and governments. This framework can be used in conjunction with reinforcement learning to learn optimal economic policies, as done by the AI Economist (https://www.einstein.ai/the-ai-economist).

Home Page: https://www.einstein.ai/the-ai-economist

License: BSD 3-Clause "New" or "Revised" License

Python 61.93% Shell 0.11% Jupyter Notebook 31.17% Cuda 6.79%

ai deep-reinforcement-learning economics ml multi-agent multi-agent-reinforcement-learning python-3 reinforcement-learning-environments simulation-framework

ai-economist's People

Contributors

Stargazers

Watchers

Forkers

founding-zero wedosideprojects deyh2020 fardinabbasi nitainijholt wwwwei stephanzheng doncat99

ai-economist's Issues

Error on "covid19_and_economic_simulation.ipynb"

Thanks for the great works!

I tried to run the code, "tutorials/covid19_and_economic_simulation.ipynb", but the below error is revealed.
How could I deal with it?

File "/home/aithlab/OneDrive/0.PhD/1.Study/0.RL/10.AI-economist/main_covid19.py", line 188, in
action_type="real_world"

File "/home/aithlab/OneDrive/0.PhD/1.Study/0.RL/10.AI-economist/main_covid19.py", line 173, in fetch_env_dense_log
env.step(action_seq[t]);

File "/home/aithlab/OneDrive/0.PhD/1.Study/0.RL/10.AI-economist/ai-economist-master/ai_economist/foundation/base/base_env.py", line 1013, in step
component.component_step()

File "/home/aithlab/OneDrive/0.PhD/1.Study/0.RL/10.AI-economist/ai-economist-master/ai_economist/foundation/components/covid19_components.py", line 133, in component_step
if self.world.use_cuda:

AttributeError: 'World' object has no attribute 'use_cuda'

The "self.world" instance does not have "use_cuda" attribute.

Disabling Planner Agent

Hello team,

Thank you for your wonderful work!

I may have overseen this in the environment setup, but I am not sure how to disable the social planner. In the tutorial, it reads "the Scenario class implements an economic simulation with multiple agents and (optionally) a social planner. " However, I can not find any environmental config to remove the social planner.

Moreover, the Planner agent seems assumed to exist in a lot of places in the code such as in

ai_economist/foundation/base_env.py line 227

# Foundation assumes there's only a single planner
n_planners = 1
self.num_agents = (
    n_agents + n_planners
)  # used in the warp_drive env wrapper (+ 1 for the planner)

Thank you,

Michael

Tried the basic simulation on colab and could not even complete install the ai economist package on the first cell

Installing build dependencies ... done
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

COVID19 Training Script Simulation

I am trying to train the covid_and_economy training script using
python training_script.py --env covid_and_economy_environment with my environment initialized in this dockerfile.

However, I am getting the following error:

Inside training_script.py: 1 GPUs are available.
Inside covid19_components.py: 1 GPUs are available.
Inside covid19_env.py: 1 GPUs are available.
Inside env_wrapper.py: 1 GPUs are available.
/home/miniconda/lib/python3.7/site-packages/torch/cuda/__init__.py:120: UserWarning:
    Found GPU%d %s which is of cuda capability %d.%d.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability supported by this library is %d.%d.

  warnings.warn(old_gpu_warn.format(d, name, major, minor, min_arch // 10, min_arch % 10))
In order to run this simulation, you will need an activation code.
Please fill out the form at https://forms.gle/dJ2gKDBqLDko1g7m7 and we will send you an activation code to the provided email address.

***

Saving the activation code in '/home/miniconda/lib/python3.7/site-packages/ai_economist/foundation/activation_code.txt' for future use.
Using the real-world data to only initialize the env, and using the fitted models to step through the env.
Loading real-world data from /home/miniconda/lib/python3.7/site-packages/ai_economist/foundation/scenarios/covid19/../../../datasets/covid19_datasets/data_and_fitted_params
Loading fit parameters from /home/miniconda/lib/python3.7/site-packages/ai_economist/foundation/scenarios/covid19/../../../datasets/covid19_datasets/data_and_fitted_params
Using external action inputs.
Initializing the CUDA data manager...
Initializing the CUDA function manager...
WARNING:root:the destination header file /home/miniconda/lib/python3.7/site-packages/warp_drive/cuda_includes/env_config.h already exists; remove and rebuild.
WARNING:root:the destination runner file /home/miniconda/lib/python3.7/site-packages/warp_drive/cuda_includes/env_runner.cu already exists; remove and rebuild.
WARNING:root:CUDADataManager casts the data 'action_in_cooldown_until' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'default_agent_action_mask' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'no_op_agent_action_mask' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'max_daily_subsidy_per_state' from type float64 to float32
WARNING:root:CUDADataManager casts the data 'default_planner_action_mask' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'no_op_planner_action_mask' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'num_vaccines_available_t' from type float64 to float32
Traceback (most recent call last):
  File "training_script.py", line 125, in <module>
    obs_dim_corresponding_to_num_agents=obs_dim_corresponding_to_num_agents,
  File "/home/miniconda/lib/python3.7/site-packages/warp_drive/training/trainer.py", line 220, in __init__
    self.training_batch_size_per_env,
  File "/home/miniconda/lib/python3.7/site-packages/warp_drive/training/utils/data_loader.py", line 66, in create_and_push_data_placeholders
    env_wrapper, policy_tag_to_agent_id_map
  File "/home/miniconda/lib/python3.7/site-packages/warp_drive/training/utils/data_loader.py", line 111, in _validate_policy_tag_to_agent_id_map
    obs = env_wrapper.obs_at_reset()
AttributeError: 'FoundationEnvWrapper' object has no attribute 'obs_at_reset'
root@b2de45be54f8:/home/ai-economist/ai_economist/training# cd ..
root@b2de45be54f8:/home/ai-economist/ai_economist# cd ..
root@b2de45be54f8:/home/ai-economist# python ai_economist/training/example_training_script.py
# --env covid_and_economy_environment
python: can't open file 'ai_economist/training/example_training_script.py': [Errno 2] No such file or directory
root@b2de45be54f8:/home/ai-economist# python ai_economist/training/example_training_script.py  --env covid_and_economy_environment
python: can't open file 'ai_economist/training/example_training_script.py': [Errno 2] No such file or directory
root@b2de45be54f8:/home/ai-economist# python ai_economist/training/training_script.py  --env covid_and_economy_environme
nt
Inside training_script.py: 1 GPUs are available.
Inside covid19_components.py: 1 GPUs are available.
Inside covid19_env.py: 1 GPUs are available.
Inside env_wrapper.py: 1 GPUs are available.
/home/miniconda/lib/python3.7/site-packages/torch/cuda/__init__.py:120: UserWarning:
    Found GPU%d %s which is of cuda capability %d.%d.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability supported by this library is %d.%d.

  warnings.warn(old_gpu_warn.format(d, name, major, minor, min_arch // 10, min_arch % 10))
Using the activation code already present in '/home/miniconda/lib/python3.7/site-packages/ai_economist/foundation/activation_code.txt'
Using the real-world data to only initialize the env, and using the fitted models to step through the env.
Loading real-world data from /home/miniconda/lib/python3.7/site-packages/ai_economist/foundation/scenarios/covid19/../../../datasets/covid19_datasets/data_and_fitted_params
Loading fit parameters from /home/miniconda/lib/python3.7/site-packages/ai_economist/foundation/scenarios/covid19/../../../datasets/covid19_datasets/data_and_fitted_params
Using external action inputs.
Initializing the CUDA data manager...
Initializing the CUDA function manager...
WARNING:root:the destination header file /home/miniconda/lib/python3.7/site-packages/warp_drive/cuda_includes/env_config.h already exists; remove and rebuild.
WARNING:root:the destination runner file /home/miniconda/lib/python3.7/site-packages/warp_drive/cuda_includes/env_runner.cu already exists; remove and rebuild.
WARNING:root:CUDADataManager casts the data 'action_in_cooldown_until' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'default_agent_action_mask' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'no_op_agent_action_mask' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'max_daily_subsidy_per_state' from type float64 to float32
WARNING:root:CUDADataManager casts the data 'default_planner_action_mask' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'no_op_planner_action_mask' from type int64 to int32
WARNING:root:CUDADataManager casts the data 'num_vaccines_available_t' from type float64 to float32
Traceback (most recent call last):
  File "ai_economist/training/training_script.py", line 125, in <module>
    obs_dim_corresponding_to_num_agents=obs_dim_corresponding_to_num_agents,
  File "/home/miniconda/lib/python3.7/site-packages/warp_drive/training/trainer.py", line 220, in __init__
    self.training_batch_size_per_env,
  File "/home/miniconda/lib/python3.7/site-packages/warp_drive/training/utils/data_loader.py", line 66, in create_and_push_data_placeholders
    env_wrapper, policy_tag_to_agent_id_map
  File "/home/miniconda/lib/python3.7/site-packages/warp_drive/training/utils/data_loader.py", line 111, in _validate_policy_tag_to_agent_id_map
    obs = env_wrapper.obs_at_reset()
AttributeError: 'FoundationEnvWrapper' object has no attribute 'obs_at_reset'

Any ideas what is wrong? I just git clone the repo without any modifications.

from utils import plotting - error

which utils package we have to use to import plotting.

Where are the actions (performed)?

Hey,

I sadly cannot find in all the code framework exactly where the agent's actions are "executed". I find it also difficult to follow the "action framework" (would be glad to read such things within the tutorial). What I mean is, I want to check which action is actually executed by the single agent? The default is also, that the agent makes actions for every each component? I cannot see where the agent choose which component to use for which action.

The only interesting part I found related to this is the following method of base_env:

set_agent_component_action()

But this methods is not used anywhere?

Can you help me! ray_results is empty

I have run the two_level_curriculum_training_with_rllib code, phase 1 and phase 2. But the ray_results is empty.
Thanks a lot !

Problem of running "economic_simulation_basic.ipynb" colab

I am trying to run the file in the colab. However, it doesn't seem to work:

When I tried to run the following:

import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
from IPython import display
from utils import plotting # plotting utilities for visualizing env. state

Only error result:

RuntimeError Traceback (most recent call last)
RuntimeError: module compiled against API version 0xf but this version of numpy is 0xe

ImportError Traceback (most recent call last)
in
1 import numpy as np
----> 2 get_ipython().run_line_magic('matplotlib', 'inline')
3 import matplotlib.pyplot as plt
4 from IPython import display
5 from utils import plotting # plotting utilities for visualizing env. state

8 frames
/usr/local/lib/python3.9/dist-packages/matplotlib/init.py in _check_versions()
190 # Quickfix to ensure Microsoft Visual C++ redistributable
191 # DLLs are loaded before importing kiwisolver
--> 192 from . import ft2font
193
194 for modname, minver in [

ImportError: numpy.core.multiarray failed to import

NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.

https://colab.research.google.com/github/salesforce/ai-economist/blob/master/tutorials/economic_simulation_basic.ipynb#scrollTo=w4ROkAd60Gve&line=1&uniqifier=1

Is there anyone facing the same problem or just me?

Instalation problem

Hello, I'm having a problem when I try to install it. When I run the pip install command, it returns me the error:

Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py egg_info did not run successfully.
│ exit code: 1
:
:
:
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

multi_agent_training_with_rllib.ipynb colab install dependency error!

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.6.0 requires numpy~=1.19.2, but you have numpy 1.20.3 which is incompatible.
tensorflow 2.6.0 requires six~=1.15.0, but you have six 1.16.0 which is incompatible.
tensorflow 2.6.0 requires typing-extensions~=3.7.4, but you have typing-extensions 3.10.0.0 which is incompatible.
moviepy 0.2.3.5 requires decorator<5.0,>=4.0.2, but you have decorator 5.0.9 which is incompatible.
google-colab 1.0.0 requires ipykernel~=4.10, but you have ipykernel 5.5.5 which is incompatible.
google-colab 1.0.0 requires ipython~=5.5.0, but you have ipython 7.23.1 which is incompatible.
google-colab 1.0.0 requires notebook~=5.3.0; python_version >= "3.0", but you have notebook 6.4.1 which is incompatible.
google-colab 1.0.0 requires pandas~=1.1.0; python_version >= "3.0", but you have pandas 1.2.4 which is incompatible.
google-colab 1.0.0 requires requests~=2.23.0, but you have requests 2.25.1 which is incompatible.
google-colab 1.0.0 requires six~=1.15.0, but you have six 1.16.0 which is incompatible.
google-colab 1.0.0 requires tornado~=5.1.0; python_version >= "3.0", but you have tornado 6.1 which is incompatible.
flask 1.1.4 requires click<8.0,>=5.1, but you have click 8.0.1 which is incompatible.
flask 1.1.4 requires Jinja2<3.0,>=2.10.1, but you have jinja2 3.0.1 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
Successfully installed GPUtil-1.4.0 Jinja2-3.0.1 Pillow-8.3.2 Pygments-2.9.0 QtPy-1.9.0 Send2Trash-1.5.0 ai-economist-1.4 appnope-0.1.2 argon2-cffi-20.1.0 astroid-2.5.6 async-generator-1.10 beautifulsoup4-4.9.3 black-21.5b1 bleach-3.3.0 certifi-2020.12.5 cffi-1.14.5 chardet-4.0.0 click-8.0.1 decorator-5.0.9 flake8-3.9.2 ipykernel-5.5.5 ipython-7.23.1 ipywidgets-7.6.3 isort-5.8.0 jsonschema-3.2.0 jupyter-client-6.1.12 jupyter-console-6.4.0 jupyter-core-4.7.1 jupyterlab-widgets-1.0.0 kiwisolver-1.3.1 lazy-object-proxy-1.6.0 lz4-3.1.3 matplotlib-3.2.1 matplotlib-inline-0.1.2 mccabe-0.6.1 mypy-extensions-0.4.3 nbclient-0.5.3 nbconvert-6.0.7 notebook-6.4.1 numpy-1.20.3 openpyxl-3.0.7 packaging-20.9 pandas-1.2.4 pandocfilters-1.4.3 pathspec-0.8.1 pluggy-0.13.1 prometheus-client-0.10.1 prompt-toolkit-3.0.18 pycodestyle-2.7.0 pycryptodome-3.10.1 pyflakes-2.3.1 pylint-2.8.2 pyrsistent-0.17.3 pytest-6.2.4 python-dateutil-2.8.1 pytz-2021.1 pyyaml-5.4.1 pyzmq-22.0.3 qtconsole-5.1.0 regex-2021.4.4 requests-2.25.1 scipy-1.6.3 six-1.16.0 soupsieve-2.2.1 terminado-0.10.0 tornado-6.1 tqdm-4.60.0 traitlets-5.0.5 typed-ast-1.4.3 typing-extensions-3.10.0.0 urllib3-1.26.5
WARNING: Upgrading ipython, ipykernel, tornado, prompt-toolkit or pyzmq can
cause your runtime to repeatedly crash or behave in unexpected ways and is not
recommended. If your runtime won't connect or execute code, you can reset it
with "Factory reset runtime" from the "Runtime" menu.
WARNING: The following packages were previously imported in this runtime:
[IPython,PIL,cffi,dateutil,decorator,ipykernel,ipywidgets,jupyter_client,jupyter_core,kiwisolver,matplotlib,mpl_toolkits,numpy,pandas,prompt_toolkit,pygments,pytz,six,tornado,traitlets,zmq]
You must restart the runtime in order to use newly installed versions.

dependency version conflicts

Unable to pip install ai-economist or pip install -e . in a fresh conda environment on windows, I have not tested on Mac or Linux as of today.

versions: `python 3.9, pip 21.2.4`

Issue: matplotlib dependency conflict:

I was able to resolve the issue by commenting out matplotlib==3.2.1 in requirements.txt then doing a pip install matplotlib after installing ai-economist

verions: `python 3.10, pip 21.2.4`

issue: scipy dependency conflict

I'm running into similar requirements.txt issues with poppygen, a python package I've created that generates demographically accurate human agents. I'll update here if I find a more robust solution.

An issue when running "multi_agent_training_with_rllib" notebook

Hello all!

I could install AI-Economist with all its dependencies in my laptop as it is was explained in your Github. I could also run successfully most of the notebooks without any issues. However, now I am running "multi_agent_training_with_rllib" notebook and it gives me an error in the following part:

Below, we fetch the dense logs for each rollout worker and environment within

dense_logs = {}

Note: worker 0 is reserved for the trainer actor

for worker in range((trainer_config["num_workers"] > 0), trainer_config["num_workers"] + 1):
for env_id in range(trainer_config["num_envs_per_worker"]):
dense_logs["worker={};env_id={}".format(worker, env_id)] =
trainer.workers.foreach_worker(lambda w: w.async_env)[worker].envs[env_id].env.previous_episode_dense_log

Here is the error:

RayTaskError(TypeError) Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_2452\4135580017.py in
6 for env_id in range(trainer_config["num_envs_per_worker"]):
7 dense_logs["worker={};env_id={}".format(worker, env_id)] =
----> 8 trainer.workers.foreach_worker(lambda w: w.async_env)[worker].envs[env_id].env.previous_episode_dense_log

~\anaconda3\envs\ai-economist\lib\site-packages\ray\rllib\evaluation\worker_set.py in foreach_worker(self, func)
351 if self.local_worker() is not None:
352 local_result = [func(self.local_worker())]
--> 353 remote_results = ray.get([w.apply.remote(func) for w in self.remote_workers()])
354 return local_result + remote_results
355

~\anaconda3\envs\ai-economist\lib\site-packages\ray_private\client_mode_hook.py in wrapper(*args, **kwargs)
103 if func.name != "init" or is_client_mode_enabled_by_default:
104 return getattr(ray, func.name)(*args, **kwargs)
--> 105 return func(*args, **kwargs)
106
107 return wrapper

~\anaconda3\envs\ai-economist\lib\site-packages\ray\worker.py in get(object_refs, timeout)
1829 worker.core_worker.dump_object_store_memory_usage()
1830 if isinstance(value, RayTaskError):
-> 1831 raise value.as_instanceof_cause()
1832 else:
1833 raise value

RayTaskError(TypeError): ray::RolloutWorker.apply() (pid=4748, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x00000213920F8848>)
File "python\ray_raylet.pyx", line 728, in ray._raylet.execute_task
File "python\ray_raylet.pyx", line 729, in ray._raylet.execute_task
File "python\ray_raylet.pyx", line 2028, in ray._raylet.CoreWorker.store_task_outputs
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\serialization.py", line 421, in serialize
return self._serialize_to_msgpack(value)
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\serialization.py", line 400, in _serialize_to_msgpack
metadata, python_objects
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\serialization.py", line 361, in _serialize_to_pickle5
raise e
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\serialization.py", line 357, in _serialize_to_pickle5
value, protocol=5, buffer_callback=writer.buffer_callback
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\cloudpickle\cloudpickle_fast.py", line 74, in dumps
cp.dump(obj)
File "C:\Users\Aslan\anaconda3\envs\ai-economist\lib\site-packages\ray\cloudpickle\cloudpickle_fast.py", line 621, in dump
return Pickler.dump(self, obj)
TypeError: can't pickle generator objects

I think this error is related to the compatibility issues of different modules. Here is the details of my operating system:

Processor Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz 1.99 GHz
Installed RAM 16.0 GB (15.9 GB usable)
System type 64-bit operating system, x64-based processor

Edition Windows 10 Pro
Version 21H1
Experience Windows Feature Experience Pack 120.2212.4180.0

Here is the installed modules in my ai-economist environment:

I was wondering if you could hint any possible solution. Many thanks in advance!

RL-Algorithm

Hey,

I wonder if I did oversee it, but where exactly is the implementation of the RL-Model described in the paper?

rl_warp_drive not in requirements.txt ?

Hello,
As today, it doesn't seem possible to import ai-economist without first installing rl_warp_drive.

pip3 install -r requirements.txt

alant@Nitro:~/repos/ai-economist$ python3 -c "import ai_economist"
1 GPUs are available.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/alant/repos/ai-economist/ai_economist/__init__.py", line 7, in <module>
    from ai_economist import foundation
  File "/home/alant/repos/ai-economist/ai_economist/foundation/__init__.py", line 7, in <module>
    from ai_economist.foundation import utils
  File "/home/alant/repos/ai-economist/ai_economist/foundation/utils.py", line 15, in <module>
    from ai_economist.foundation.base.base_env import BaseEnvironment
  File "/home/alant/repos/ai-economist/ai_economist/foundation/base/base_env.py", line 16, in <module>
    from ai_economist.foundation.components import component_registry
  File "/home/alant/repos/ai-economist/ai_economist/foundation/components/__init__.py", line 9, in <module>
    from . import (
  File "/home/alant/repos/ai-economist/ai_economist/foundation/components/covid19_components.py", line 23, in <module>
    from warp_drive.utils.constants import Constants
ModuleNotFoundError: No module named 'warp_drive'

warp_drive doesn't seem to be in the current requirements.txt
Is it expected/intended ?
Best,
A

Be asked for an activation code in the demo of the COVID-19 case study

In the demo of warp_drive, covid 19 data is used. When running the demo, we are prompted to need an activation code. We are asked to fill in "covid-19 simulation code access request". We have filled in and explained that the use of data sets is only for academic research, and respect data privacy. But a day has passed without receiving any reply, prompting that we only need to wait a few minutes for the above.

https://forms.gle/dJ2gKDBqLDko1g7m7
https://github.com/salesforce/ai-economist/blob/master/tutorials/multi_agent_gpu_training_with_warp_drive.ipynb

The following is the reason for applying. If it's convenient, please send the activation code to my email [email protected]
Sincerely thank you！

Hello, I'm a researcher from Zhejiang laboratory in China. Our research topic is a simulation of social governance in Hangzhou city, including the spread of disease & government tax policy research. I have been using AI economists recently, and a very important part of it is the combination with warp drive. Covid 19 data is used in the demo for users. When running the demo, we are prompted to need activation code. Is it convenient to send the activation code to us? The purpose of our use is to learn the reinforcement learning knowledge of AI economists. In addition, we promise to respect the privacy of covid 19 data and never use it for any commercial purposes.

pip install rl_warp_drive fails

Trying to run the covid example code locally, it seems to require warp-drive package.

When I attempt to install it I get:

Requirement already satisfied: MarkupSafe>=0.9.2 in /home/andriy/miniconda3/envs/ai-economist/lib/python3.7/site-packages (from mako->pycuda==2021.1->rl_warp_drive) (2.0.1)
Building wheels for collected packages: pycuda, gym, pytools
  Building wheel for pycuda (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/andriy/miniconda3/envs/ai-economist/bin/python /home/andriy/miniconda3/envs/ai-economist/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpmu6wv6qu
       cwd: /tmp/pip-install-8x2mqudw/pycuda_92c7dba3b9f041d696c54cadb59fb0ef
  Complete output (117 lines):
  ***************************************************************
  *** WARNING: nvcc not in path.
  *** May need to set CUDA_INC_DIR for installation to succeed.
  ***************************************************************
  *************************************************************
  *** I have detected that you have not run configure.py.
  *************************************************************
  *** Additionally, no global config files were found.
  *** I will go ahead with the default configuration.
  *** In all likelihood, this will not work out.
  ***
  *** See README_SETUP.txt for more information.
  ***
  *** If the build does fail, just re-run configure.py with the
  *** correct arguments, and then retry. Good luck!
  *************************************************************
  *** HIT Ctrl-C NOW IF THIS IS NOT WHAT YOU WANT
  *************************************************************
  Continuing in 10 seconds...
  Continuing in 9 seconds...
  Continuing in 8 seconds...
  Continuing in 7 seconds...
  Continuing in 6 seconds...
  Continuing in 5 seconds...
  Continuing in 4 seconds...
  Continuing in 3 seconds...
  Continuing in 2 seconds...
  Continuing in 1 seconds...
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/_mymako.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/debug.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/_cluda.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/compiler.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/elementwise.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/reduction.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/gpuarray.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/cumath.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/autoinit.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/scan.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/curandom.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/driver.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/tools.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/characterize.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/autoprimaryctx.py -> build/lib.linux-x86_64-3.7/pycuda
  copying pycuda/__init__.py -> build/lib.linux-x86_64-3.7/pycuda
  creating build/lib.linux-x86_64-3.7/pycuda/gl
  copying pycuda/gl/autoinit.py -> build/lib.linux-x86_64-3.7/pycuda/gl
  copying pycuda/gl/__init__.py -> build/lib.linux-x86_64-3.7/pycuda/gl
  creating build/lib.linux-x86_64-3.7/pycuda/sparse
  copying pycuda/sparse/operator.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
  copying pycuda/sparse/pkt_build.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
  copying pycuda/sparse/coordinate.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
  copying pycuda/sparse/packeted.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
  copying pycuda/sparse/cg.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
  copying pycuda/sparse/inner.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
  copying pycuda/sparse/__init__.py -> build/lib.linux-x86_64-3.7/pycuda/sparse
  creating build/lib.linux-x86_64-3.7/pycuda/compyte
  copying pycuda/compyte/dtypes.py -> build/lib.linux-x86_64-3.7/pycuda/compyte
  copying pycuda/compyte/array.py -> build/lib.linux-x86_64-3.7/pycuda/compyte
  copying pycuda/compyte/__init__.py -> build/lib.linux-x86_64-3.7/pycuda/compyte
  running egg_info
  writing pycuda.egg-info/PKG-INFO
  writing dependency_links to pycuda.egg-info/dependency_links.txt
  writing requirements to pycuda.egg-info/requires.txt
  writing top-level names to pycuda.egg-info/top_level.txt
  reading manifest file 'pycuda.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  /home/andriy/miniconda3/envs/ai-economist/lib/python3.7/distutils/dist.py:274: UserWarning: Unknown distribution option: 'test_requires'
    warnings.warn(msg)
  warning: no files found matching 'doc/source/_static/*.css'
  warning: no files found matching 'doc/source/_templates/*.html'
  warning: no files found matching '*.cpp' under directory 'bpl-subset/bpl_subset/boost'
  warning: no files found matching '*.html' under directory 'bpl-subset/bpl_subset/boost'
  warning: no files found matching '*.inl' under directory 'bpl-subset/bpl_subset/boost'
  warning: no files found matching '*.txt' under directory 'bpl-subset/bpl_subset/boost'
  warning: no files found matching '*.h' under directory 'bpl-subset/bpl_subset/libs'
  warning: no files found matching '*.ipp' under directory 'bpl-subset/bpl_subset/libs'
  warning: no files found matching '*.pl' under directory 'bpl-subset/bpl_subset/libs'
  adding license file 'LICENSE'
  writing manifest file 'pycuda.egg-info/SOURCES.txt'
  creating build/lib.linux-x86_64-3.7/pycuda/cuda
  copying pycuda/cuda/pycuda-complex-impl.hpp -> build/lib.linux-x86_64-3.7/pycuda/cuda
  copying pycuda/cuda/pycuda-complex.hpp -> build/lib.linux-x86_64-3.7/pycuda/cuda
  copying pycuda/cuda/pycuda-helpers.hpp -> build/lib.linux-x86_64-3.7/pycuda/cuda
  copying pycuda/sparse/pkt_build_cython.pyx -> build/lib.linux-x86_64-3.7/pycuda/sparse
  running build_ext
  building '_driver' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/src
  creating build/temp.linux-x86_64-3.7/src/cpp
  creating build/temp.linux-x86_64-3.7/src/wrapper
  creating build/temp.linux-x86_64-3.7/bpl-subset
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/python
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/python/src
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/python/src/converter
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/python/src/object
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/smart_ptr
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/smart_ptr/src
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/system
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/system/src
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/thread
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/thread/src
  creating build/temp.linux-x86_64-3.7/bpl-subset/bpl_subset/libs/thread/src/pthread
  gcc -pthread -B /home/andriy/miniconda3/envs/ai-economist/compiler_compat -Wl,--sysroot=/ -Wsign-compare -fwrapv -Wall -O3 -DNDEBUG -fPIC -DBOOST_ALL_NO_LIB=1 -DBOOST_THREAD_BUILD_DLL=1 -DBOOST_MULTI_INDEX_DISABLE_SERIALIZATION=1 -DBOOST_PYTHON_SOURCE=1 -Dboost=pycudaboost -DBOOST_THREAD_DONT_USE_CHRONO=1 -DPYGPU_PACKAGE=pycuda -DPYGPU_PYCUDA=1 -DHAVE_CURAND=1 -Isrc/cpp -Ibpl-subset/bpl_subset -I/tmp/pip-build-env-1z7_9r8k/overlay/lib/python3.7/site-packages/numpy/core/include -I/home/andriy/miniconda3/envs/ai-economist/include/python3.7m -c src/cpp/cuda.cpp -o build/temp.linux-x86_64-3.7/src/cpp/cuda.o
  In file included from src/cpp/cuda.cpp:4:
  src/cpp/cuda.hpp:14:10: fatal error: cuda.h: No such file or directory
     14 | #include <cuda.h>
        |          ^~~~~~~~
  compilation terminated.
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for pycuda
  Building wheel for gym (setup.py) ... done
  Created wheel for gym: filename=gym-0.20.0-py3-none-any.whl size=1650478 sha256=4a09ea29ec95f5ba0acc80a6442f0712481b2c82865a6ec8af6baa342d058f92
  Stored in directory: /home/andriy/.cache/pip/wheels/0d/08/46/9393ca20304119b40cbb5794479b77658b42e3db6d3bdd343e
  Building wheel for pytools (setup.py) ... done
  Created wheel for pytools: filename=pytools-2021.2.8-py2.py3-none-any.whl size=60725 sha256=b22e3446ad253626d55e874cad4b9abb14156b3095521fbe7bb0039bc0629858
  Stored in directory: /home/andriy/.cache/pip/wheels/bd/2d/ef/0127a17bafa44971f11d05d0e38d7947144cf9e33313bf12a7
Successfully built gym pytools
Failed to build pycuda
ERROR: Could not build wheels for pycuda which use PEP 517 and cannot be installed directly

Question - about the simulation and relation with Entity Component System concepts

Hi,

I'm curious to know how you guys have implemented the game/simulation (which technologies mostly) that is available at https://youtu.be/4iQUcGyQhdA, and given some similarities with Entity Component System concepts in gaming development, could you please share some thoughts about how you are going to complexify the simulation ?

Thank you in advance

windows os paths

@sunil-s thanks for checking that out, I switch to python 3.7 (as noted in the setup.py) I almost found success there, installed via pip install -e ., however now I get this error, unable to find the map file:

Note: this works without issue in google colab

How to get started means how to start the simulation and how shall we change the experiment parameters

two different types of mobile agents

hello, does ai-economist allow for creating and using two different types of mobile agents? I am trying to run a simulation on the simple-wood-and-stone env and need two different mobile agents classes. is this achievable if so can you please provide me with some pointers?

@alextrott16 could you perhaps please give me some directions on this? I would really appreciate it!

rllib recurrent_tf_modelv2 ModuleNotFoundError

I'm having issues running python3 training_script.py --run-dir phase1:
from ray.rllib.models.tf.recurrent_tf_modelv2 import ( ModuleNotFoundError: No module named 'ray.rllib.models.tf.recurrent_tf_modelv2'

I've tested with multiple versions of ray[rllib], the error still persists.

I can append my pip freeze if needed (running Conda with Python 3.7)

Issues using the custom neural network models in AI-Economist

Hello,

I am developing AI-Economist and I am trying to run my own modified version of AI-Economist with your provided custom neural network models: KerasConvLSTM and RandomAction.

Here is my system properties:

Model Name: MacBook Pro
Model Identifier: MacBookPro17,1
Chip: Apple M1
Total Number of Cores: 8 (4 performance and 4 efficiency)
Memory: 16 GB

Here is the list of modules in my modified ai-economist environment:

Name Version Build Channel

absl-py 1.2.0 pypi_0 pypi
aiosignal 1.2.0 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
appnope 0.1.2 py310hca03da5_1001 anaconda
argon2-cffi 20.1.0 pypi_0 pypi
astroid 2.5.6 pypi_0 pypi
asttokens 2.0.5 pyhd3eb1b0_0 anaconda
astunparse 1.6.3 pypi_0 pypi
async-generator 1.10 pypi_0 pypi
attrs 21.2.0 pypi_0 pypi
backcall 0.2.0 pyhd3eb1b0_0 anaconda
beautifulsoup4 4.9.3 pypi_0 pypi
black 22.8.0 pypi_0 pypi
bleach 3.3.0 pypi_0 pypi
bs4 0.0.1 pypi_0 pypi
bzip2 1.0.8 h620ffc9_4 anaconda
c-ares 1.18.1 h3422bc3_0 conda-forge
ca-certificates 2022.07.19 hca03da5_0
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cachetools 5.2.0 pypi_0 pypi
certifi 2020.12.5 pypi_0 pypi
cffi 1.14.5 pypi_0 pypi
chardet 4.0.0 pypi_0 pypi
click 8.0.1 pypi_0 pypi
cloudpickle 2.1.0 pypi_0 pypi
cycler 0.10.0 pypi_0 pypi
debugpy 1.5.1 py310hc377ac9_0 anaconda
decorator 5.0.9 pypi_0 pypi
defusedxml 0.7.1 pypi_0 pypi
distlib 0.3.6 pypi_0 pypi
dm-tree 0.1.7 pypi_0 pypi
entrypoints 0.3 pypi_0 pypi
et-xmlfile 1.1.0 pypi_0 pypi
executing 0.8.3 pyhd3eb1b0_0 anaconda
filelock 3.8.0 pypi_0 pypi
flake8 3.9.2 pypi_0 pypi
flatbuffers 1.12 pypi_0 pypi
fonttools 4.37.1 pypi_0 pypi
frozenlist 1.3.1 pypi_0 pypi
gast 0.4.0 pypi_0 pypi
google-auth 2.11.0 pypi_0 pypi
google-auth-oauthlib 0.4.6 pypi_0 pypi
google-pasta 0.2.0 pypi_0 pypi
gputil 1.4.0 pypi_0 pypi
grpcio 1.43.0 py310h00ca444_0 conda-forge
gym 0.21.0 pypi_0 pypi
h5py 3.6.0 nompi_py310hb8bbf05_100 conda-forge
hdf5 1.12.1 nompi_h829dc4f_103 conda-forge
idna 2.10 pypi_0 pypi
imageio 2.21.2 pypi_0 pypi
iniconfig 1.1.1 pypi_0 pypi
ipykernel 5.5.5 pypi_0 pypi
ipython 7.31.1 pypi_0 pypi
ipython-genutils 0.2.0 pypi_0 pypi
ipywidgets 7.6.3 pypi_0 pypi
isort 5.8.0 pypi_0 pypi
jedi 0.18.0 pypi_0 pypi
jinja2 3.0.1 pypi_0 pypi
jsonschema 3.2.0 pypi_0 pypi
jupyter 1.0.0 pypi_0 pypi
jupyter-client 6.1.12 pypi_0 pypi
jupyter-console 6.4.0 pypi_0 pypi
jupyter-core 4.7.1 pypi_0 pypi
jupyter_client 7.2.2 py310hca03da5_0 anaconda
jupyter_core 4.10.0 py310hca03da5_0 anaconda
jupyterlab-pygments 0.1.2 pypi_0 pypi
jupyterlab-widgets 1.0.0 pypi_0 pypi
keras 2.9.0 pypi_0 pypi
keras-preprocessing 1.1.2 pypi_0 pypi
kiwisolver 1.3.1 pypi_0 pypi
krb5 1.19.2 hfff258c_3 conda-forge
lazy-object-proxy 1.6.0 pypi_0 pypi
libblas 3.9.0 16_osxarm64_openblas conda-forge
libcblas 3.9.0 16_osxarm64_openblas conda-forge
libclang 14.0.6 pypi_0 pypi
libcurl 7.81.0 h8d07098_0 conda-forge
libcxx 12.0.0 hf6beb65_1 anaconda
libedit 3.1.20191231 hc8eb9b7_2 conda-forge
libev 4.33 h642e427_1 conda-forge
libffi 3.4.2 hc377ac9_4 anaconda
libgfortran 5.0.0 11_3_0_hd922786_25 conda-forge
libgfortran5 11.3.0 hdaf2cc0_25 conda-forge
liblapack 3.9.0 16_osxarm64_openblas conda-forge
libnghttp2 1.46.0 h62a264d_0 conda-forge
libopenblas 0.3.21 openmp_hc731615_3 conda-forge
libsodium 1.0.18 h1a28f6b_0 anaconda
libssh2 1.10.0 h7a5bd25_3 conda-forge
libzlib 1.2.12 ha287fd2_2 conda-forge
llvm-openmp 14.0.4 hd125106_0 conda-forge
lz4 3.1.3 pypi_0 pypi
markdown 3.4.1 pypi_0 pypi
markupsafe 2.1.1 pypi_0 pypi
matplotlib 3.5.3 pypi_0 pypi
matplotlib-inline 0.1.6 pypi_0 pypi
mccabe 0.6.1 pypi_0 pypi
mistune 0.8.4 pypi_0 pypi
modified-ai-economist 1.1.0 dev_0
msgpack 1.0.4 pypi_0 pypi
mypy-extensions 0.4.3 pypi_0 pypi
nbclient 0.5.3 pypi_0 pypi
nbconvert 6.0.7 pypi_0 pypi
nbformat 5.1.3 pypi_0 pypi
ncurses 6.3 h1a28f6b_2 anaconda
nest-asyncio 1.5.1 pypi_0 pypi
networkx 2.8.6 pypi_0 pypi
notebook 6.4.1 pypi_0 pypi
numpy 1.21.0 pypi_0 pypi
oauthlib 3.2.0 pypi_0 pypi
openpyxl 3.0.7 pypi_0 pypi
openssl 3.0.5 h03a7124_2 conda-forge
opt-einsum 3.3.0 pypi_0 pypi
packaging 20.9 pypi_0 pypi
pandas 1.2.4 pypi_0 pypi
pandocfilters 1.4.3 pypi_0 pypi
parso 0.8.2 pypi_0 pypi
pathspec 0.10.1 pypi_0 pypi
pexpect 4.8.0 pyhd3eb1b0_3 anaconda
pickleshare 0.7.5 pyhd3eb1b0_1003 anaconda
pillow 9.0.1 pypi_0 pypi
pip 21.2.4 py310hca03da5_0 anaconda
platformdirs 2.5.2 pypi_0 pypi
pluggy 0.13.1 pypi_0 pypi
prometheus-client 0.10.1 pypi_0 pypi
prompt-toolkit 3.0.18 pypi_0 pypi
protobuf 3.19.4 pypi_0 pypi
ptyprocess 0.7.0 pyhd3eb1b0_2 anaconda
pure_eval 0.2.2 pyhd3eb1b0_0 anaconda
py 1.10.0 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pycodestyle 2.7.0 pypi_0 pypi
pycparser 2.20 pypi_0 pypi
pycryptodome 3.10.1 pypi_0 pypi
pyflakes 2.3.1 pypi_0 pypi
pygments 2.9.0 pypi_0 pypi
pylint 2.8.2 pypi_0 pypi
pyparsing 2.4.7 pypi_0 pypi
pyrsistent 0.17.3 pypi_0 pypi
pytest 6.2.4 pypi_0 pypi
python 3.10.5 h4eee789_0_cpython conda-forge
python-dateutil 2.8.1 pypi_0 pypi
python_abi 3.10 2_cp310 conda-forge
pytz 2021.1 pypi_0 pypi
pywavelets 1.3.0 pypi_0 pypi
pyyaml 5.4.1 pypi_0 pypi
pyzmq 22.0.3 pypi_0 pypi
qtconsole 5.1.0 pypi_0 pypi
qtpy 1.9.0 pypi_0 pypi
ray 2.0.0 pypi_0 pypi
readline 8.1.2 h1a28f6b_1 anaconda
regex 2021.4.4 pypi_0 pypi
requests 2.25.1 pypi_0 pypi
requests-oauthlib 1.3.1 pypi_0 pypi
rsa 4.9 pypi_0 pypi
scikit-image 0.19.3 pypi_0 pypi
scipy 1.9.1 pypi_0 pypi
seaborn 0.12.0 pypi_0 pypi
send2trash 1.5.0 pypi_0 pypi
setuptools 61.2.0 py310hca03da5_0 anaconda
six 1.16.0 pyh6c4a22f_0 conda-forge
soupsieve 2.2.1 pypi_0 pypi
sqlite 3.38.5 h1058600_0 anaconda
stack_data 0.2.0 pyhd3eb1b0_0 anaconda
tabulate 0.8.10 pypi_0 pypi
tensorboard 2.9.1 pypi_0 pypi
tensorboard-data-server 0.6.1 pypi_0 pypi
tensorboard-plugin-wit 1.8.1 pypi_0 pypi
tensorboardx 2.5.1 pypi_0 pypi
tensorflow-deps 2.8.0 0 apple
tensorflow-estimator 2.9.0 pypi_0 pypi
tensorflow-macos 2.9.2 pypi_0 pypi
tensorflow-metal 0.5.1 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
terminado 0.10.0 pypi_0 pypi
testpath 0.5.0 pypi_0 pypi
tifffile 2022.8.12 pypi_0 pypi
tk 8.6.12 hb8d0fd4_0 anaconda
toml 0.10.2 pypi_0 pypi
tomli 2.0.1 pypi_0 pypi
tornado 6.1 py310h1a28f6b_0 anaconda
tqdm 4.60.0 pypi_0 pypi
traitlets 5.0.5 pypi_0 pypi
typing-extensions 3.10.0.0 pypi_0 pypi
tzdata 2022a hda174b7_0 anaconda
urllib3 1.26.5 pypi_0 pypi
virtualenv 20.16.4 pypi_0 pypi
wcwidth 0.2.5 pyhd3eb1b0_0 anaconda
webencodings 0.5.1 pypi_0 pypi
werkzeug 2.2.2 pypi_0 pypi
wheel 0.37.1 pyhd3eb1b0_0 anaconda
widgetsnbextension 3.5.1 pypi_0 pypi
wrapt 1.12.1 pypi_0 pypi
xz 5.2.5 h1a28f6b_1 anaconda
zeromq 4.3.4 hc377ac9_0 anaconda
zlib 1.2.12 h5a0b063_2 anaconda

As it is clear from the above list, there are some extra modules beside the required modules of the original AI-Economist and also some modules have different versions than what are mentioned in the original AI-Economist. For example, the ray version here is 2.0.0. I think with this version of ray, the parent class of any tensorflow recurrent custom model should be set RecurrentNetwork instead of RecurrentTFModelV2, so I made modifications in tf_models as required.

Then, I ran the following commands similar to what are brought in your provided notebook: multi_agent_training_with_rllib

from rllib.env_wrapper import RLlibEnvWrapper
env_obj = RLlibEnvWrapper({"env_config_dict": env_config_dict}, verbose=True)

import ray
from ray.rllib.agents.ppo import PPOTrainer

from ray.rllib.models.catalog import ModelCatalog
from rllib.tf_models import KerasConvLSTM, RandomAction

ModelCatalog.register_custom_model(KerasConvLSTM.custom_name, KerasConvLSTM)
ModelCatalog.register_custom_model(RandomAction.custom_name, RandomAction)

policies = {
"a": (
None, # uses default policy
env_obj.observation_space,
env_obj.action_space,
{'clip_param': 0.3,
'entropy_coeff': 0.025,
'entropy_coeff_schedule': None,
'gamma': 0.998,
'grad_clip': 10.0,
'kl_coeff': 0.0,
'kl_target': 0.01,
'lambda': 0.98,
'lr': 0.0003,
'lr_schedule': None,
'model': {'custom_model': 'keras_conv_lstm',
'custom_model_config': {'fc_dim': 128,
'idx_emb_dim': 4,
'input_emb_vocab': 100,
'lstm_cell_size': 128,
'num_conv': 2,
'num_fc': 2},
'max_seq_len': 25},
'use_gae': True,
'vf_clip_param': 50.0,
'vf_loss_coeff': 0.05,
'vf_share_layers': False} # define a custom agent policy configuration.
),
"p": (
None, # uses default policy
env_obj.observation_space_pl,
env_obj.action_space_pl,
{'clip_param': 0.3,
'entropy_coeff': 0.125,
'entropy_coeff_schedule': [[0, 2.0], [50000000, 0.125]],
'gamma': 0.998,
'grad_clip': 10.0,
'kl_coeff': 0.0,
'kl_target': 0.01,
'lambda': 0.98,
'lr': 0.0001,
'lr_schedule': None,
'model': {'custom_model': 'keras_conv_lstm',
'custom_model_config': {'fc_dim': 256,
'idx_emb_dim': 4,
'input_emb_vocab': 100,
'lstm_cell_size': 256,
'num_conv': 2,
'num_fc': 2},
'max_seq_len': 25},
'use_gae': True,
'vf_clip_param': 50.0,
'vf_loss_coeff': 0.05,
'vf_share_layers': False} # define a custom planner policy configuration.
)
}

policy_mapping_fun = lambda i: "a" if str(i).isdigit() else "p"

policies_to_train = ["a", "p"]

trainer_config = {
"multiagent": {
"policies": policies,
"policies_to_train": policies_to_train,
"policy_mapping_fn": policy_mapping_fun,
}
}

trainer_config.update(
{
"num_workers": 5,
"num_envs_per_worker": 1,
# Other training parameters
"train_batch_size": 4000,
"sgd_minibatch_size": 4000,
"num_sgd_iter": 1
}
)

env_config = {
"env_config_dict": env_config_dict,
"num_envs_per_worker": trainer_config.get('num_envs_per_worker'),
}

trainer_config.update(
{
"env_config": env_config
}
)

ray.init()

trainer = PPOTrainer(env=RLlibEnvWrapper, config=trainer_config )

However, the following error is generated:

RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.init() (pid=1353, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x16a373790>)
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/evaluation/rollout_worker.py", line 613, in init
self._build_policy_map(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/evaluation/rollout_worker.py", line 1784, in _build_policy_map
self.policy_map.create_policy(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/policy/policy_map.py", line 123, in create_policy
self[policy_id] = create_policy_for_framework(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/utils/policy.py", line 71, in create_policy_for_framework
return policy_class(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/algorithms/ppo/ppo_tf_policy.py", line 83, in init
base.init(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 81, in init
self.model = self.make_model()
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/policy/dynamic_tf_policy_v2.py", line 221, in make_model
return ModelCatalog.get_model_v2(
File "/Users/asataryd/miniforge3/envs/modified-ai-economist/lib/python3.10/site-packages/ray/rllib/models/catalog.py", line 587, in get_model_v2
raise ValueError(
ValueError: It looks like you are still using <rllib.tf_models.KerasConvLSTM object at 0x16a6f2ec0>.register_variables() to register your model's weights. This is no longer required, but if you are still calling this method at least once, you must make sure to register all created variables properly. The missing variables are {<Reference wrapping <tf.Variable 'a_wk4/conv2D_2_pol/kernel:0' shape=(3, 3, 16, 32) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/value/bias:0' shape=(1,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_pol/bias:0' shape=(16,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_pol/beta:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_pol/kernel:0' shape=(3, 3, 26, 16) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_pol/lstm_cell/bias:0' shape=(512,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/bias:0' shape=(512,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/value/kernel:0' shape=(128, 1) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_pol/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/logits/bias:0' shape=(80,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_2_val/kernel:0' shape=(3, 3, 16, 32) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_val/beta:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_pol/lstm_cell/recurrent_kernel:0' shape=(128, 512) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/logits/kernel:0' shape=(128, 80) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_val/kernel:0' shape=(234, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_val/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/embedding_pol/embeddings:0' shape=(100, 4) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_pol/gamma:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_2_val/bias:0' shape=(32,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_2_pol/bias:0' shape=(32,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_val/bias:0' shape=(16,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/recurrent_kernel:0' shape=(128, 512) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_pol/kernel:0' shape=(128, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_pol/lstm_cell/kernel:0' shape=(128, 512) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/conv2D_1_val/kernel:0' shape=(3, 3, 26, 16) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_val/kernel:0' shape=(128, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense2_val/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_pol/kernel:0' shape=(234, 128) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/dense1_pol/bias:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/embedding_val/embeddings:0' shape=(100, 4) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/layer_norm_val/gamma:0' shape=(128,) dtype=float32>>, <Reference wrapping <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/kernel:0' shape=(128, 512) dtype=float32>>}, and you only registered {<tf.Variable 'a_wk4/conv2D_2_pol/kernel:0' shape=(3, 3, 16, 32) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_pol/bias:0' shape=(16,) dtype=float32>, <tf.Variable 'a_wk4/value/bias:0' shape=(1,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_pol/kernel:0' shape=(3, 3, 26, 16) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_pol/beta:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/lstm_pol/lstm_cell/bias:0' shape=(512,) dtype=float32>, <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/bias:0' shape=(512,) dtype=float32>, <tf.Variable 'a_wk4/value/kernel:0' shape=(128, 1) dtype=float32>, <tf.Variable 'a_wk4/dense2_pol/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_2_val/kernel:0' shape=(3, 3, 16, 32) dtype=float32>, <tf.Variable 'a_wk4/logits/bias:0' shape=(80,) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_val/beta:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/lstm_pol/lstm_cell/recurrent_kernel:0' shape=(128, 512) dtype=float32>, <tf.Variable 'a_wk4/logits/kernel:0' shape=(128, 80) dtype=float32>, <tf.Variable 'a_wk4/dense1_val/kernel:0' shape=(234, 128) dtype=float32>, <tf.Variable 'a_wk4/dense1_val/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/embedding_pol/embeddings:0' shape=(100, 4) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_pol/gamma:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_2_val/bias:0' shape=(32,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_2_pol/bias:0' shape=(32,) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_val/bias:0' shape=(16,) dtype=float32>, <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/recurrent_kernel:0' shape=(128, 512) dtype=float32>, <tf.Variable 'a_wk4/dense2_pol/kernel:0' shape=(128, 128) dtype=float32>, <tf.Variable 'a_wk4/lstm_pol/lstm_cell/kernel:0' shape=(128, 512) dtype=float32>, <tf.Variable 'a_wk4/conv2D_1_val/kernel:0' shape=(3, 3, 26, 16) dtype=float32>, <tf.Variable 'a_wk4/dense2_val/kernel:0' shape=(128, 128) dtype=float32>, <tf.Variable 'a_wk4/dense2_val/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/dense1_pol/kernel:0' shape=(234, 128) dtype=float32>, <tf.Variable 'a_wk4/dense1_pol/bias:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/embedding_val/embeddings:0' shape=(100, 4) dtype=float32>, <tf.Variable 'a_wk4/layer_norm_val/gamma:0' shape=(128,) dtype=float32>, <tf.Variable 'a_wk4/lstm_val/lstm_cell_1/kernel:0' shape=(128, 512) dtype=float32>}. Did you forget to call register_variables() on some of the variables in question?

This happen even if I use RandomAction network. Also, in your provided notebook, you have not used these custom modelsI. Moreover, I think this is most probably not related to the changes that I made in the AI-Economist but it might be related to the compatibility issues of different versions of the modules that I have in my environment. I greatly appreciate any comments.

By the way, please let me know if you need further information. If it is required I would be happy to share the GitHub repository of my own version of AI-Economist with you. Many thanks in advance!

Simulation Card Link Broken

In the README the link https://www.github.com/salesforce/ai-economist/Simulation_Card_Foundation_Economic_Simulation_Framework.pdf is broken, there is no file in that address.

Trt tutorials/economic_simulation_advanced.ipynb Error

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.6.0 requires numpy~=1.19.2, but you have numpy 1.20.3 which is incompatible.
tensorflow 2.6.0 requires six~=1.15.0, but you have six 1.16.0 which is incompatible.
tensorflow 2.6.0 requires typing-extensions~=3.7.4, but you have typing-extensions 3.10.0.0 which is incompatible.
moviepy 0.2.3.5 requires decorator<5.0,>=4.0.2, but you have decorator 5.0.9 which is incompatible.
google-colab 1.0.0 requires ipykernel~=4.10, but you have ipykernel 5.5.5 which is incompatible.
google-colab 1.0.0 requires ipython~=5.5.0, but you have ipython 7.23.1 which is incompatible.
google-colab 1.0.0 requires notebook~=5.3.0; python_version >= "3.0", but you have notebook 6.4.0 which is incompatible.
google-colab 1.0.0 requires pandas~=1.1.0; python_version >= "3.0", but you have pandas 1.2.4 which is incompatible.
google-colab 1.0.0 requires requests~=2.23.0, but you have requests 2.25.1 which is incompatible.
google-colab 1.0.0 requires six~=1.15.0, but you have six 1.16.0 which is incompatible.
google-colab 1.0.0 requires tornado~=5.1.0; python_version >= "3.0", but you have tornado 6.1 which is incompatible.
flask 1.1.4 requires click<8.0,>=5.1, but you have click 8.0.1 which is incompatible.
flask 1.1.4 requires Jinja2<3.0,>=2.10.1, but you have jinja2 3.0.1 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
Successfully installed Jinja2-3.0.1 Pillow-8.2.0 Pygments-2.9.0 QtPy-1.9.0 Send2Trash-1.5.0 ai-economist-1.2.3 appnope-0.1.2 argon2-cffi-20.1.0 astroid-2.5.6 async-generator-1.10 beautifulsoup4-4.9.3 black-21.5b1 bleach-3.3.0 certifi-2020.12.5 cffi-1.14.5 chardet-4.0.0 click-8.0.1 decorator-5.0.9 flake8-3.9.2 ipykernel-5.5.5 ipython-7.23.1 ipywidgets-7.6.3 isort-5.8.0 jsonschema-3.2.0 jupyter-client-6.1.12 jupyter-console-6.4.0 jupyterlab-widgets-1.0.0 kiwisolver-1.3.1 lazy-object-proxy-1.6.0 lz4-3.1.3 matplotlib-3.2.1 matplotlib-inline-0.1.2 mccabe-0.6.1 mypy-extensions-0.4.3 nbclient-0.5.3 nbconvert-6.0.7 notebook-6.4.0 numpy-1.20.3 openpyxl-3.0.7 packaging-20.9 pandas-1.2.4 pandocfilters-1.4.3 pathspec-0.8.1 pluggy-0.13.1 prometheus-client-0.10.1 prompt-toolkit-3.0.18 pycodestyle-2.7.0 pycryptodome-3.10.1 pyflakes-2.3.1 pylint-2.8.2 pyrsistent-0.17.3 pytest-6.2.4 python-dateutil-2.8.1 pytz-2021.1 pyzmq-22.0.3 qtconsole-5.1.0 regex-2021.4.4 requests-2.25.1 scipy-1.6.3 six-1.16.0 soupsieve-2.2.1 terminado-0.10.0 tornado-6.1 tqdm-4.60.0 traitlets-5.0.5 typed-ast-1.4.3 typing-extensions-3.10.0.0 urllib3-1.26.4
WARNING: Upgrading ipython, ipykernel, tornado, prompt-toolkit or pyzmq can
cause your runtime to repeatedly crash or behave in unexpected ways and is not
recommended. If your runtime won't connect or execute code, you can reset it
with "Factory reset runtime" from the "Runtime" menu.
WARNING: The following packages were previously imported in this runtime:
[IPython,PIL,cffi,dateutil,decorator,ipykernel,ipywidgets,jupyter_client,kiwisolver,matplotlib,mpl_toolkits,numpy,pandas,prompt_toolkit,pygments,pytz,six,tornado,traitlets,zmq]
You must restart the runtime in order to use newly installed versions.

tf_models KerasConvLSTM seq_lens tensor

Hi, I'm detaching the trainer from rllib (need of custom stuff incompatible with it). I'm having problems using the model.forward method:
what should I put in seq_lens? I couldn't find any documentation about it.

Last error:

ValueError: Input 0 of layer "permute_1" is incompatible with the layer: expected ndim=5, found ndim=4. 
Full shape received: (Dimension(1), Dimension(2), Dimension(11), Dimension(11))

Code context:

from keras_model import build_model
from env_wrapper import RLlibEnvWrapper
from tf_models import KerasConvLSTM, get_flat_obs_size
import tensorflow as tf
from tensorflow.python.framework.ops import enable_eager_execution
# enable_eager_execution()

# Model config and env config as in /tutorials/rllib/phase1/config.yaml
model_config = {
    'custom_model': "keras_conv_lstm",
    'custom_options': {
        'fc_dim': 128,
        'idx_emb_dim': 4,
        'input_emb_vocab': 100,
        'lstm_cell_size': 128,
        'num_conv': 2,
        'num_fc': 2,
    },
    'max_seq_len': 25,

}

env_config = {'env_config_dict': {
    # ===== SCENARIO CLASS =====
    # Which Scenario class to use: the class's name in the Scenario Registry (foundation.scenarios).
    # The environment object will be an instance of the Scenario class.
    'scenario_name': 'layout_from_file/simple_wood_and_stone',

    # ===== COMPONENTS =====
    # Which components to use (specified as list of ("component_name", {component_kwargs}) tuples).
    #   "component_name" refers to the Component class's name in the Component Registry (foundation.components)
    #   {component_kwargs} is a dictionary of kwargs passed to the Component class
    # The order in which components reset, step, and generate obs follows their listed order below.
    'components': [
        # (1) Building houses
        ('Build', {
            'skill_dist':                   'pareto',
            'payment_max_skill_multiplier': 3,
            'build_labor':                  10,
            'payment':                      10
        }),
        # (2) Trading collectible resources
        ('ContinuousDoubleAuction', {
            'max_bid_ask':    10,
            'order_labor':    0.25,
            'max_num_orders': 5,
            'order_duration': 50
        }),
        # (3) Movement and resource collection
        ('Gather', {
            'move_labor':    1,
            'collect_labor': 1,
            'skill_dist':    'pareto'
        }),
        # (4) Planner
        ('PeriodicBracketTax', {
            'period':          100,
            'bracket_spacing': 'us-federal',
            'usd_scaling':     1000,
            'disable_taxes':   False
        })
    ],

    # ===== SCENARIO CLASS ARGUMENTS =====
    # (optional) kwargs that are added by the Scenario class (i.e. not defined in BaseEnvironment)
    'env_layout_file': 'quadrant_25x25_20each_30clump.txt',
    'starting_agent_coin': 10,
    'fixed_four_skill_and_loc': True,

    # ===== STANDARD ARGUMENTS ======
    # kwargs that are used by every Scenario class (i.e. defined in BaseEnvironment)
    'n_agents': 4,          # Number of non-planner agents (must be > 1)
    'world_size': [25, 25],  # [Height, Width] of the env world
    'episode_length': 1000,  # Number of timesteps per episode

    # In multi-action-mode, the policy selects an action for each action subspace (defined in component code).
    # Otherwise, the policy selects only 1 action.
    'multi_action_mode_agents': False,
    'multi_action_mode_planner': True,

    # When flattening observations, concatenate scalar & vector observations before output.
    # Otherwise, return observations with minimal processing.
    'flatten_observations': True,
    # When Flattening masks, concatenate each action subspace mask into a single array.
    # Note: flatten_masks = True is required for masking action logits in the code below.
    'flatten_masks': True,

    # How often to save the dense logs
    'dense_log_frequency': 1
}}

env = RLlibEnvWrapper(env_config)
obs = env.reset()

# so num_outputs must be equal to env.action_space's value 
model = KerasConvLSTM(env.observation_space,
                      env.action_space, num_outputs=50, model_config=model_config, name=None)
state = model.get_initial_state()

# probably the issue is here
# rank_1_tensor = tf.constant([(50,),(136,),(1,),(2,11,11),(7,11,11)], shape=(1,5,1))
rank_1_tensor = tf.constant([1])

flat_obs_space = get_flat_obs_size(env.observation_space)


def dict_to_tensor_dict(a_dict: dict):
    """
    pass a single agent obs, returns it's tensor_dict
    """
    tensor_dict = {}
    for key, value in a_dict.items():
        tensor_dict[key] = tf.convert_to_tensor(value, name=key)

    return tensor_dict

obs_tensor_dict = dict_to_tensor_dict(obs['0'])

# from /python3.7/site-packages/ray/rllib/models/tf/tf_modelv2.py
# `input_dict` (dict): dictionary of input tensors, including `"obs", "obs_flat", "prev_action", "prev_reward", "is_training"`

input_dict = {
    'obs': obs_tensor_dict,
    # 'obs_flat': ,
    'prev_action': None, 
    'prev_reward': None, 
    'is_training': True
}

output, new_state = model.forward(input_dict, state, rank_1_tensor)

Thank you for the help!

Add citation file

Recently (yesterday) Github announced support for citation of software, directly from Github:

https://twitter.com/natfriedman/status/1420122675813441540

After reading your blog I thought it would be a great addition to add citation information for the code itself.

salesforce / ai-economist Goto Github PK

ai-economist's People

Contributors

Stargazers

Watchers

Forkers

ai-economist's Issues

Only error result:

To view examples of installing some common dependencies, click the "Open Examples" button below.

versions: python 3.9, pip 21.2.4

verions: python 3.10, pip 21.2.4

Below, we fetch the dense logs for each rollout worker and environment within

Note: worker 0 is reserved for the trainer actor

Name Version Build Channel

Recommend Projects

Recommend Topics

Recommend Org

Jobs

To view examples of installing some common dependencies, click the
"Open Examples" button below.

versions: `python 3.9, pip 21.2.4`

verions: `python 3.10, pip 21.2.4`