pwdemars / rl4uc

Reinforcement learning for unit commitment

Hi there 👋

I am a data scientist at UK Power Networks, working on forecasting, power systems modelling and other data-driven projects to support the energy transition.

My PhD at UCL focused on reinforcement learning for the unit commitment problem; you can find the environment I designed to simulate this problem in rl4uc, and the methods I used to solve it in ts4uc.

More recently, I've been working on indoor temperature prediction for residential buildings and optimisation of heating (public repo coming soon!)

Another project I'm proud of is my work on the Western Power Distribution data science competition, forecasting demand and solar PV to optimise a grid-scale battery. This project has its own website.

Feel free to get in touch with questions or opportunities to collaborate. 🌱

rl4uc's Issues

Data retrieval doesn't return the same dates for test

In data_retrieval.py the SEED variable is defined but not used to set a random seed. The subset of days held out for testing therefore differs each time the script is run.

Probably easier to just define the test days manually to be honest. Those used in publications are:

['2016-01-12',
 '2016-07-25',
 '2016-11-21',
 '2017-03-18',
 '2017-04-07',
 '2017-05-12',
 '2017-05-26',
 '2017-06-25',
 '2017-12-18',
 '2017-12-30',
 '2018-01-15',
 '2018-03-08',
 '2018-03-18',
 '2018-05-30',
 '2018-09-11',
 '2018-11-13',
 '2019-04-03',
 '2019-05-22',
 '2019-10-30',
 '2019-11-09']

Add some tutorial notebooks

Rename heat rates

What is currently called 'heat rate' in env.gen_info is not really heat rate but maximum economic efficiency.

Full OpenAI Gym Integration

It would be great to integrate fully with OpenAI Gym in order to:

• Increase accessibility
• Allow for simple benchmarking with stablebaselines and other RL libraries

This would primarily involve the following changes:

• Correct implementation of observation and action spaces.
• Addition of the info return from env.step()

Write tests

There are currently no tests for rl4uc. It would be good to have a few including:

• Successful make_env for training environment
• Successful make_env for testing environment
• Assert correct fuel costs for a given transitions

Integration with deterministic UC solver for benchmarking

In my research I have been using the pglib-uc repo to benchmark against MILP solutions. You can see my fork of this repo here.

It might be helpful to integrate this more completely into rl4uc, such that RL solutions can be easily compared with the deterministic solutions with reserve constraints.