danfoa / commons_game

A custom implementation of DeepMind's "the commons game"

The Commons Game

This repository presents a custom implementation of game in the paper A multi-agent reinforcement learning model of common-pool resource appropriation by Deep Mind, presented at Advances in Neural Information Processing Systems 30 (NIPS 2017).

The main goal of the paper is to introduce Reinforcement Learning based simulated environments as a way of addressing the modeling of common-pool resource dynamics. This, because abstract models based on non-cooperative game theory fail to predict deal world dynamics of these scenarios.

Details

In this implementation we replaced the original DQN algorithm of the original paper with DDQN with replay buffer. Furthermore, we dont use a MLP architecture but rather a light weight CNN that could theoretically process the agents observations easily. More information on the implementation can be found in the following example Colab.

See details in Google Colab

View source on GitHub

Requirements

The only requirements of the project are

• Tensoflow2.x
• OpenGym python package

Additionally, if you want to go ahead and see the code functionality first, you can check the example Colab notebook.

TODOs

• Use and compare novel RL value-based and policy-gradient based algorithms.

Acknowledgements

This repository is based on the code of:

• sequential_social_dilemma_games
• TensorFlow2.0-for-Deep-Reinforcement-Learning

commons_game's Issues

Learning interruption problem

Hi,

Thanks for this great implementation.

I run the learning model of'commons_game(train_utils)' on my laptop.
However, my learning was stopped due to insufficient laptop memory.
For example, it took about 3 days to train a single agent for 2800 episodes.
In this case, is there a way to resume learning from where it left off?
If I inevitably stop learning, what is the way to resume it?

My laptop specifications are as follows:
Window 10 Home
Intel core i7-10750H
16 GB RAM
NVIDIA GeForce RTX 2060max-q(vram: 6GB)
Python version 3.8.5
tensorflow-gpu version 2.2.0.

And, please let me know if there is a more efficient way to use'commons_game' in this notebook spec.

I'm still investigating but any help would be appreciated :)