markhaoxiang / city-learn

city-learn rl

A baseline multiagent reinforcement learning experiment on https://github.com/intelligent-environments-lab/CityLearn with learning

• Custom implementation of PPO, DDPG, TD3 on a CommNet Architecture
• Infrastructure for running MLOPS on the CityLearn environment

Setup

1. pip3 install virtualenv
2. virtualenv env
3. source venv/bin/activate
4. pip3 install -r requirements.txt

Example

1. Run train.py example_experiment -f

city-learn's Issues

Environment Generator

Description

Module to create sample training environments to create train, dev, test sets and a more robust training / validation framework.

Requirements

1. Generate JSON of environment and constants
2. Generate building features, variable number of buildings
3. Alter time series for weather, pricing, carbon content etc.
4. Automate as pipeline, minimal manual work and inputs. Provide API to interface with training and visualizer.

Episode Viewer

Description

Visualize episode replays. Something similar to https://jmerle.github.io/koreye-2022/visualizer?input=36310051 would be a good example. To open source

Requirements

1. High priority Store and load episodes. Select storage format that will be conducive to future imitation learning and asynchronous learning efforts.
2. Platform independent.
3. Playback with step forward and step backwards.
4. Section for graphs and section for statistical data.

Please submit simple GUI design on task assignment!

Algorithm Implementation

Description

Select paper. Implement an algorithm. Note: Please try to set input feature space and other variables as a constant! We will standardize this section before phase 2. The training framework, module setup, and parameter selection pipeline will largely depend on the first few attempts, complete before 7 August.

Requirements

1. Select paper. Create issue and project notice on Github.
2. Implement.
3. Experiment.
4. Prepare short presentation with team on impact, and optimizations.

Documentation

1. Cleanup
2. Open-source release.