pwdemars / ts4uc

Tree Search for the UC Problem

Hi there 👋

I am a data scientist at UK Power Networks, working on forecasting, power systems modelling and other data-driven projects to support the energy transition.

My PhD at UCL focused on reinforcement learning for the unit commitment problem; you can find the environment I designed to simulate this problem in rl4uc, and the methods I used to solve it in ts4uc.

More recently, I've been working on indoor temperature prediction for residential buildings and optimisation of heating (public repo coming soon!)

Another project I'm proud of is my work on the Western Power Distribution data science competition, forecasting demand and solar PV to optimise a grid-scale battery. This project has its own website.

Feel free to get in touch with questions or opportunities to collaborate. 🌱

ts4uc's Issues

Track depth in anytime IDA*

How does the search depth vary when using IDA*? Are there certain times of day when search is particularly shallow or deep? Does this help explain performance at all?

Implement node count

Count the number of times each node has been visited in search. needed for anytime algorithms.

Implement lowest path cost (best value)

Track the lowest path cost of trajectories going through each node. Needed for some anytime algos

Learned heuristic

In relation issue #8, it may be worth considering a heuristic that is learned. To do this, it would be necessary to run the uniform cost search algo for a load of problems, then fit the model $y = f(X)$ where $y$ is the path cost or step cost, and $X$ is the state vector.

Track time taken per period for A* search

One of the motivations for investigating anytime IDA* was that runtime can be very unpredictable for A* search. Some episodes run very quickly, others much more slowly. It would be really useful to measure this, as it's an important result in the context of motivating IDA*.

TODO:

• Record time taken per period
• Plot distribution
• Plot distribution of episode run times too

Create a parameters folder

Parameters to use to train the policy networks are not included in this repo - they should. Relevant for experiments 101 and 102.

Develop more complex priority list heuristic

At the moment, the priority list heuristic does not consider any generator constraints. As a result, it does not distinguish between nodes of the same generation: they all have the same heuristic costs.

One idea for improving the performance of A* and RTA* is to improve the heuristic. The goal would effectively be to reduce the error between the actual costs and the heuristic estimate, while keeping it (maybe roughly) admissible.

This would have two potential benefits: it could improve the speed of the informed methods relative to uniform cost search, by pruning subtrees more aggressively. Also, it could allow RTA* to exceed the performance of A*, which currently has not been achieved. Nodes at the search horizon currently all have the same h(n): for A* h(n) = 0 for all n and for RTA* h(n) = c > 0 for all n. A better heuristic would distinguish nodes at the search horizon when using RTA*.

Compare performance of heuristic methods

Compare A* and RTA* with uniform cost search.

The key metrics to consider are:

• Mean cost
• Mean time

Ultimately, we are hoping that A* methods will run quicker than uninformed methods (uniform-cost search). The mean cost is likely to be quite similar I expect.

If there is little difference, it may be worth considering if the heuristic can be improved, and if this is likely to improve performance.

Of course, one option is to apply a ML approach: predicting cost given state using some kind of regression method. One issue here is that the loss function will have to be carefully designed to encourage admissibility (i.e. it will be non-symmetric). Might be worth opening an issue for this.

Add uniform cost search to experiment 201

It would be worth comparing uniform cost search (as invesigated in chapter 1) to experiment 201, comparing heuristic methods.

Add HPC parameter files to TS4UC

I haven't moved over the parameter files properly from AISO to ts4uc. It would be good to do this, and preferably to allow these files to be generated locally on my machine, referring to the $HOME environment variable.

A new directory data/hpc_params would be a good start.

Rolling horizon problem

Similar to day_ahead.py, have a rolling_horizon.py for solving the rolling horizon problem formulation.

In this context, the same methods can still be applied but with the following changes:

• The search tree cannot be retained at each timestep
• Scenarios should be generated afresh at each timestep, with the ARMA processes seeded by the observed forecast errors.

In addition, there may be the chance to apply some other search methods that use And-Or trees.

Heat rates have been renamed in rl4uc

What was gen_info.heat_rates is now more sensibly named gen_info.min_fuel_cost. Ensure that there is no reference to heat_rates anywhere.

Qualitative comparison of anytime versus other methods

How does anytime compare with other methods in terms of schedules (similar analysis to chapter 1)?

• Startups
• Average up time
• Number of simultaneous switches
• Plot schedules

Experiment 101: training the policy networks

Need to write the bash script for training the policy networks to be used in the later experiments e.g. 103, 104, 105 and 201.

This should train policies for 5--10, 20 and 30 generator systems. Ideally it will be fully seeded.

Anytime RTA* algorithm

Implement an anytime version of the RTA* algorithm, using principles from MCTS.

I would like to investigate at least 2 of the following methods:

• Choosing by visit count
• Choosing by mean cost
• Choosing by min cost

Iterative Deepening A* (IDA*)

This is perhaps a more obvious approach to writing an anytime algorithm. Simply solve with A* up to depth 1, 2,... etc. until the time budget is spent, then take the most recent solution.

Admissible Heuristics

I need to investigate why the priority list heuristic is not admissible. In general, it does underestimate costs as it should do, but sometimes (notably in the middle of the day) it doesn't:

It could be that this is due to some kind of bug, such as:

• Wrong horizon is set
• Economic dispatch is wrong

It is also possible that this is indeed accurate, which would be true if the distribution of operating costs w.r.t. scenarios would skewed towards low values, such that in practice, operating costs were lower on average than the perfect forecast. However, my feeling is that that is surely wrong, since my research previously concluded that guided tree search was a couple of % off the perfect forecast costs.

Add step sizes

A possible extension that I've played with before is to include non-uniform step sizes in the tree search. For instance, you could consider making decisions every 2 steps. This would allow for deeper search.

Add a `none` option to A* and variants

This is really just refactoring as it would effectively revert A* to uniform cost search, without having to rely on a separate algorithm.

Implement mean value

Mean value of trajectories passing through each node. Needed for some anytime algorithms

solve_and_test.py file missing

Hi, your 'local_scripts' seems to require the 'pglib-uc/solve_and_test.py' file to run correctly, but this file is neither included in this repository nor pglib-uc. Can you add this file to this repository, please? Thank you.

for example (local_scripts/106_milp.sh):

for g in 10 20 30 ;
do python $HOME/pglib-uc/solve_and_test.py --save_dir ../results/${date}_106/milp_g${g}_perfect \
						 --env_params_fn $HOME/ts4uc/data/day_ahead/${g}gen/30min/env_params.json \
						 --test_data_dir $HOME/ts4uc/data/day_ahead/${g}gen/30min \
						 --num_samples 1000 \
						 --perfect_forecast true ;
done