ai4opt / opfgenerator

Instance generator for OPF problems

Copyright Georgia Tech 2022-2024

OPFGenerator

Instance generator for various OPF problems (ACOPF & DCOPF currently supported)

• OPFGenerator
  • Installation instructions
    • Using HSL solvers (ma27, ma57)
  • Quick start
    • Generating random instances
    • Building and solving OPF problems
  • Generating datasets
  • Solution format
    • ACPPowerModel
    • DCPPowerModel
    • SOCWRPowerModel
  • Datasets
    • Format
    • Loading from julia
    • Loading from python
  • Loading and saving JSON files

Installation instructions

This repository is a non-registered Julia package.

• Option 1: install as a Julia package. You can use it, but not modify the code

  using Pkg
  Pkg.add("[email protected]:AI4OPT/OPFGenerator.git")

• Option 2: clone the repository. Use this if you want to change the code

  git clone [email protected]:AI4OPT/OPFGenerator.git

  To use the package after cloning the repo

  $ cd OPFGenerator
  $ julia --project=.
  julia> using OPFGenerator

  If you are modifying the source code, it is recommened to use the package Revise.jl so that you can use the changes without having to start Julia. Make sure you load Revise before loading OPFGenerator in your julia session.

  using Revise
  using OPFGenerator

Using HSL solvers (ma27, ma57)

Please refer to Ipopt.jl installation instructions for how to install non-default linear solvers, e.g., HSL, Pardiso, etc... Note that a specific license may be required.

To use HSL linear solvers when solving OPF instances, set the parameter "linear_solver" to "ma27" or "ma57" in the config file. The recommended solver for Ipopt is ma27.

solver.name = "Ipopt"
solver.attributes.linear_solver = "ma27"

Quick start

Generating random instances

using Random, PGLib, PowerModels
using OPFGenerator
PowerModels.silence()

using StableRNGs
rng = StableRNG(42)

old_data = make_basic_network(pglib("3_lmbd"))

# Load scaler using global scaling + uncorrelated LogNormal noise
config = Dict(
    "load" => Dict(
        "noise_type" => "ScaledLogNormal",
        "l" => 0.8,
        "u" => 1.2,
        "sigma" => 0.05,        
    )
)
opf_sampler  = SimpleOPFSampler(old_data, config)

# Generate a new instance
new_data = rand(rng, opf_sampler)

old_data["load"]["1"]["pd"]  # old 
1.1

new_data["load"]["1"]["pd"]  # new
1.1596456429775048

To generate multiple instances, run the above code in a loop

dataset = [
    OPFGenerator.rand(rng, opf_sampler)
    for i in 1:100
]

Building and solving OPF problems

OPFGenerator supports multiple OPF formulations, based on PowerModels.

using PowerModels
using PGLib

using JuMP
using Ipopt

using OPFGenerator

data = make_basic_network(pglib("14_ieee"))
acopf = OPFGenerator.build_opf(PowerModels.ACPPowerModel, data, Ipopt.Optimizer)
optimize!(acopf.model)
res = OPFGenerator.extract_result(acopf)

res["objective"]  # should be close to 2178.08041

Generating datasets

A script for generating multiple ACOPF instances is given in exp/sampler.jl.

It is called from the command-line as follows:

julia --project=. exp/sampler.jl <path/to/config.toml> <seed_min> <seed_max>

where

• <path/to/config.toml> is a path to a valid configuration file in TOML format (see exp/config.toml for an example)
• <seed_min> and <seed_max> are minimum and maximum values for the random seed. Must be integer values. The script will generate instances for values smin, smin+1, ..., smax-1, smax.

Solution format

Individual solutions are stored in a dictionary that follows the PowerModels result data format. As a convention, dual variables of equality constraints are named lam_<constraint_ref>, dual variables of (scalar) inequality constraints are named mu_<constraint_ref>, and dual variables of conic constraints are named nu_<constraint_ref>_<i> where i denotes the coordinate index.

Collections of instances & solutions are stored in a compact, array-based HDF5 file (see Datasets/Format below.)

ACPPowerModel

See PowerModels documentation.

Primal variables

Component	Key	Description
bus	"vm"	Nodal voltage magnitude
	"va"	Nodal voltage angle
generator	"pg"	Active power generation
	"qg"	Reactive power generation
branch	"pf"	Branch active power flow (fr)
	"pt"	Branch active power flow (to)
	"qf"	Branch reactive power flow (fr)
	"qt"	Branch reactive power flow (to)

Dual variables

Component	Key	Constraint
bus	"mu_vm_lb"	Nodal voltage magnitude lower bound
	"mu_vm_ub"	Nodal voltage magnitude upper bound
	"lam_kirchhoff_active"	Nodal active power balance
	"lam_kirchhoff_reactive"	Nodal reactive power balance
generator	"mu_pg_lb"	Active power generation lower bound
	"mu_pg_ub"	Active power generation upper bound
	"mu_qg_lb"	Reactive power generation lower bound
	"mu_qg_ub"	Reactive power generation upper bound
branch	"mu_sm_fr"	Thermal limit (fr)
	"mu_sm_to"	Thermal limit (to)
	"lam_ohm_active_fr"	Ohm's law; active power (fr)
	"lam_ohm_active_to"	Ohm's law; active power (to)
	"lam_ohm_reactive_fr"	Ohm's law; reactive power (fr)
	"lam_ohm_reactive_to"	Ohm's law; reactive power (to)
	"mu_va_diff"	Voltage angle difference

DCPPowerModel

See PowerModels documentation.

Primal variables

Component	Key	Description
bus	"va"	Voltage angle
generator	"pg"	Power generation
branch	"pf"	Power flow

Dual variables

Component	Key	Constraint
bus	"lam_kirchhoff"	Power balance
generator	"mu_pg_lb"	Power generation lower bound
	"mu_pg_ub"	Power generation upper bound
branch	"lam_ohm"	Ohm's law
	"mu_sm_lb"	Thermal limit (lower bound)
	"mu_sm_ub"	Thermal limit (upper bound)
	"mu_va_diff"	Voltage angle difference

SOCWRPowerModel

See PowerModels documentation.

Primal variables

Component	Key	Description
bus	"w"	Squared voltage magnitude
generator	"pg"	Active power generation
	"qg"	Reactive power generation
branch	"pf"	Branch active power flow (fr)
	"pt"	Branch active power flow (to)
	"qf"	Branch reactive power flow (fr)
	"qt"	Branch reactive power flow (to)
	"wr"	Real part of voltage product
	"wi"	Imaginary part of voltage product

Dual variables depend on whether a quadratic (SOCWRPowerModel) or conic (SOCWRConicPowerModel) formulation is considered. The former are identified with (quad), the latter with (cone) in the table below. Only the dual variables of quadratic constraints are affected by this distinction. Note that conic dual are high-dimensional variables, and separate coordinates are stored separately.

Component	Key	Constraint
bus	"mu_vm_lb"	Nodal voltage magnitude lower bound
	"mu_vm_ub"	Nodal voltage magnitude upper bound
	"lam_kirchhoff_active"	Nodal active power balance
	"lam_kirchhoff_reactive"	Nodal reactive power balance
generator	"mu_pg_lb"	Active power generation lower bound
	"mu_pg_ub"	Active power generation upper bound
	"mu_qg_lb"	Reactive power generation lower bound
	"mu_qg_ub"	Reactive power generation upper bound
branch	"lam_ohm_active_fr"	Ohm's law; active power (fr)
	"lam_ohm_active_to"	Ohm's law; active power (to)
	"lam_ohm_reactive_fr"	Ohm's law; reactive power (fr)
	"lam_ohm_reactive_to"	Ohm's law; reactive power (to)
	"mu_va_diff_lb"	Voltage angle difference lower bound
	"mu_va_diff_ub"	Voltage angle difference upper bound
	"mu_sm_fr"	(quad) Thermal limit (fr)
	"mu_sm_to"	(quad) Thermal limit (to)
	"mu_voltage_prod_quad"	(quad) Voltage product relaxation (Jabr)
	"nu_voltage_prod_soc_1"	(cone) Voltage product relaxation (Jabr)
	"nu_voltage_prod_soc_2"	(cone) Voltage product relaxation (Jabr)
	"nu_voltage_prod_soc_3"	(cone) Voltage product relaxation (Jabr)
	"nu_voltage_prod_soc_4"	(cone) Voltage product relaxation (Jabr)
	"nu_sm_fr_1"	(cone) Thermal limit (fr)
	"nu_sm_fr_2"	(cone) Thermal limit (fr)
	"nu_sm_fr_3"	(cone) Thermal limit (fr)
	"nu_sm_to_1"	(cone) Thermal limit (to)
	"nu_sm_to_2"	(cone) Thermal limit (to)
	"nu_sm_to_3"	(cone) Thermal limit (to)

Datasets

Format

Each dataset is stored in an .h5 file, organized as follows. See Solution format for a list of each formulation's primal and dual variables.

/
|-- meta
|   |-- ref
|   |-- config
|-- input
|   |-- seed
|   |-- pd
|   |-- qd
|   |-- br_status
|-- ACPPowerModel
|   |-- meta
|   |   |-- termination_status
|   |   |-- primal_status
|   |   |-- dual_status
|   |   |-- solve_time
|   |-- primal
|   |   |-- pg
|   |   |-- qg
|   |   |-- ...
|   |-- dual
|       |-- lam_kirchhoff_active
|       |-- lam_kirchhoff_reactive
|       |-- ...
|-- DCPPowerModel
|   |-- meta
|   |-- primal
|   |-- dual
|-- SOCWRConicPowerModel
    |-- meta
    |-- primal
    |-- dual

Loading from julia

using HDF5

D = h5read("dataset.h5", "/")  # read all the dataset into a dictionary

Loading from python

The following code provides a starting point to load h5 datasets in python

import h5py
import numpy as np


def parse_hdf5(path: str, preserve_shape: bool=False):
    dat = dict()

    def read_direct(dataset: h5py.Dataset):
        arr = np.empty(dataset.shape, dtype=dataset.dtype)
        dataset.read_direct(arr)
        return arr

    def store(name: str, obj):
        if isinstance(obj, h5py.Group):
            return
        elif isinstance(obj, h5py.Dataset):
            dat[name] = read_direct(obj)
        else:
            raise ValueError(f"Unexcepted type: {type(obj)} under name {name}. Expected h5py.Group or h5py.Dataset.")

    with h5py.File(path, "r") as f:
        f.visititems(store)

    if preserve_shape:
        # recursively correct the shape of the dictionary
        ret = dict()

        def r_correct_shape(d: dict, ret: dict):
            for k in list(d.keys()):
                if "/" in k:
                    k1, k2 = k.split("/", 1)
                    if k1 not in ret:
                        ret[k1] = dict()
                    r_correct_shape({k2: d[k]}, ret[k1])
                    del d[k]
                else:
                    ret[k] = d[k]

        r_correct_shape(dat, ret)

        return ret
    else:
        return dat

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant No. 2112533 NSF AI Institute for Advances in Optimization (AI4OPT). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

opfgenerator's Issues

Provide option to skip sysimage

The current automated slurm workflow consists of 4 steps, each one in its own job and each job depending on the previous one:

1. sysimage
2. ref
3. sampler
4. extract

We currently do not support running multiple systems (eg 118_ieee, 300_ieee, 1354_pegase...) at the same time. Therefore, a user needs to run the above workflow multiple times, one for each system.
This can create conflicts for the sysimage job, since different jobs would attempt to save a sysimage at the same location (currently app/julia.so).

In addition, if a user wants to skip the sysimage step (e.g. they have already compiled one), they need to manually modify the submit.sh script in order to remove the --dependency that the ref job has on the sysimage job.

Supporting multiple systems in the same config would be more work than we can afford at the moment.
Instead, I would suggest we separate the sysimage creation from the other 3 jobs. This would add one manual step (creating the sysimage) before one can submit the ref/sampler/extract series of jobs.

ACOPF formulation mixes old and new nonlinear interface

Flows are still set using the old @NLconstraint macro:

OPFGenerator/src/opf/acp.jl

Lines 104 to 110 in 3211699

	# From side of the branch flow
	model[:ohm_active_fr][i] = JuMP.@NLconstraint(model, p_fr == (g+g_fr)/ttm*vm_fr^2 + (-g*tr+b*ti)/ttm*(vm_fr*vm_to*cos(va_fr-va_to)) + (-b*tr-g*ti)/ttm*(vm_fr*vm_to*sin(va_fr-va_to)) )
	model[:ohm_reactive_fr][i] = JuMP.@NLconstraint(model, q_fr == -(b+b_fr)/ttm*vm_fr^2 - (-b*tr-g*ti)/ttm*(vm_fr*vm_to*cos(va_fr-va_to)) + (-g*tr+b*ti)/ttm*(vm_fr*vm_to*sin(va_fr-va_to)) )
	# To side of the branch flow
	model[:ohm_active_to][i] = JuMP.@NLconstraint(model, p_to == (g+g_to)*vm_to^2 + (-g*tr-b*ti)/ttm*(vm_to*vm_fr*cos(va_to-va_fr)) + (-b*tr+g*ti)/ttm*(vm_to*vm_fr*sin(va_to-va_fr)) )
	model[:ohm_reactive_to][i] = JuMP.@NLconstraint(model, q_to == -(b+b_to)*vm_to^2 - (-b*tr+g*ti)/ttm*(vm_to*vm_fr*cos(va_to-va_fr)) + (-g*tr-b*ti)/ttm*(vm_to*vm_fr*sin(va_to-va_fr)) )

Termination / primal / dual status checks when extracting results

We don't seem to have systematic checks of termination / primal / dual statuses before querying results from solvers.
Somehow we haven't had any issue so far, but we should have additional checks.

module load (lmod) statements in slurm template files

Users may run into issues when running slurm jobs using the automated workflow because of missing / not properly loaded modules.

For instance, none of the template files include module load julia (nor its dependencies).
The sampler job template contains module load parallel, and the sysimage job template contains module load gcc.

It's worth noting that not all platforms / HPC clusters may use lmod to manage modules, and these may not be consistent across clusters.

The current worfklow works well for someone who has module load julia in their .bashrc (or any statement that lets the OS know julia). It will throw an error if this is not the case though.

Suggested fix:

• mention this in the documentation, and be clear about which dependencies we use (OS-wise).
  At the moment I count:
  • julia
  • gcc (for the sysimage)
  • parallel (for the sampler job)
  • any cluster-wide solver install (e.g. Gurobi or CPLEX) if such solver is used
• we could add more explicit module load julia statements etc. in all the template job files (which, in our case, would force us to use the PACE-installed julia)
• we could add a small test/diagnostic job, to be executed before / separately from the automated slurm workflow, just to make sure that the user has everything setup correctly. Something that, for instance, just checks whether certain libraries are available etc...

Update SLURM pipeline to new quotas

It seems PACE has updated the max job length in embers queue to 8 hours

Unit tests

Update to PGLib v23.07

The PGLib collection was updated last summer. The new version v23.07 contains additional instances.
I do not know whether OPF instances that were in v21.07 were modified.

So far, we have been using PGLib instances from v21.07, because that's the version that was linked in the PGLib.jl package.
Since v0.2, PGLib.jl now loads instances from the v23.07 collection.
We have a compat requirement for PGLib.jl that has prevented the update to PGLib.jl v0.2 / pglib v23.07.

Fix name inconsistency betwee JSON and h5 format

Fool-proofing automated slurm workflow

I ran into an issue when using the new slurm automated workflow: my sysimage job failed, which prevented the rest of the jobs from ever running.
I don't know how, but having a failsafe would be nice.

I did not see anything in slurm that allows something like "if job B depends on job A and job A failed, then fail job B" rather than "if job B depends on job A and job A failed, then job B will wait forever" (the latter is slurm's current behavior).

Some possibilities:

• something magic in slurm that would do the above one?
• use --dependency=afterany and track job progress differently?

TBH, this would be more of a convenience, not the highest priority.

Export reserve data in input H5

Economic Dispatch exports the reserve data as part of the solution even though it's really an input, since the schema for the input H5 in exp/sampler.jl doesn't include reserve data:

OPFGenerator/exp/sampler.jl

Lines 128 to 135 in 4a1f117

	D["input"] = Dict{String,Any}(
	"meta" => Dict{String,Any}("seed" => Int[]),
	"data" => Dict{String,Any}(
	"pd" => Vector{Float64}[],
	"qd" => Vector{Float64}[],
	"br_status" => Vector{Bool}[],
	)
	)

Re-organize `test/sampler.jl`

We are starting to saturate test/sampler.jl, which currently includes unit tests for the data sampling code, and functional tests for the data-generation (OPF sampling and solving).

It would be good to separate those.
On top of my head:

• tests for update! of OPF models should go with their respective formulations
• tests for data distributions and sampling should be isolated, and preferably moved to a test/sampler folder (mimicking the structure of the source code)
• tests for data generation (sampling and solving OPF instances) should be isolated too. It's less clear to me what this should look like.
  I'm thinking we might want to move code currently in exp/sampler.jl to source code.

Data-generation distribution with bounded support

The current default data-generation distribution has unbounded support for individual loads.
This makes it challenging to run verification tasks on the same distribution (or support thereof) as was used for training.

Alternatives could be:

• use uniform white noise instead of log-normal --> simple
• use truncated log-normal noise -> stays somewhat lognormal, but we would potentially need to introduce another parameter for truncation

Write proper unit tests for HDF5 utilities

We are enforcing stricter conventions than what HDF5 supports.
We should have proper tests to:

• check that invalid values will throw an error
• check that invalid-but-can-be-converted types throw a warning
• saving / reloading an HDF5 file gives us the data we expect. This is particularly important for non bitstype (especially Complex numbers) which may store real/imaginary parts separately.

Arrow files?

We currently use JSON and HDF5 file formats.
Arrow files might be something worth considering in the future.

For pointers:

• https://arrow.apache.org/
• https://arrow.apache.org/docs/python/index.html
• https://arrow.apache.org/julia/stable/

Not critical at this point

Point users to Ipopt.jl for HSL installation instructions

We have our own instructions in the README but I think it's safer to trust Ipopt.jl and direct users there instead since the steps may change in the future. For example there are already some specific steps for macOS due to some security features which are not documented in our README, but are in Ipopt.jl's.

Avoid re-building models

In sampler.jl, we build a new JuMP model for each formulation for each seed. Instead, we should build one model per formulation and use POI to change the loads for each seed. This should reduce build times substantially after the first one.

In the package code, this will also require updating all of the formulations to make the loads parameters, wrapping the optimizer with POI.Optimizer, and adding ParametricOptInterface to the dependencies. Since we add the dependency, the code for changing the parameter values can go in the package as well.

In the SLURM pipeline, this would still build one model per formulation per minibatch. There may be a way to create each formulation's model once in the make_sysimage or ref job, then reuse it in each minibatch.

Missing bus solution for IEEE 300

Symptom:
After parsing one of the solution for IEEE 300 (pglib_opf_case300_ieee_9.json.gz) as a dictionary structure (dict):

• Length of dict['data']['bus'] = 300
• Length of dict['sol']['bus'] = 201
  which are not matching.

Use App instead of Sysimage?

It could be worth investigating whether an app would be more suitable than a sysimage in our case.

See PackageCompiler documentation:

• App: https://julialang.github.io/PackageCompiler.jl/stable/apps.html
• Sysimage: https://julialang.github.io/PackageCompiler.jl/stable/sysimages.html

Increase memory for extract job

We currently set a default memory of 16gb for the extract job.
That's not enough for large datasets, because the current post-processing code loads everything into memory, then saves a single HDF5 file.

Good solution: make that code more memory efficient 😎
Easy solution: increase the memory requirement to ~168gb

Exporting dual solution

PowerModels doesn't export duals by default, which we need.
Unless PM implements that functionality, the fastest way for us to have it is to re-implement the ACOPF model ourselves.

Tracker: lanl-ansi/PowerModels.jl#409

ACP, DCP do not export dual for slack bus constraint

Noticed this when seeing if we have a convention for where to put singleton (not per-bus/branch) variables/constraints in the solution JSON

Export solver's primal/dual objective value

Mismatch between model formulation and exported dual solution

There is a mismatch between the way we build certain OPF models and how we export the dual solution.
This is particularly true for the SOCWRConicPowerModel.

The mismatch stems from a discrepancy between the SOC formulation (Model 2 in the DCP paper) that include the Jabr inequality (7k) for each branch

whereas the actual JuMP model has one Jabr inequality for each bus-pair:

OPFGenerator/src/opf/socwr.jl

Lines 129 to 140 in 11199a4

	# Voltage product relaxation (quadratic form)
	if OPF == PM.SOCWRPowerModel
	JuMP.@constraint(model,
	voltage_prod_quadratic[(i, j) in keys(ref[:buspairs])],
	wr[(i,j)]^2 + wi[(i,j)]^2 <= w[i]*w[j]
	)
	elseif OPF == PM.SOCWRConicPowerModel
	JuMP.@constraint(model,
	voltage_prod_conic[(i, j) in keys(ref[:buspairs])],
	[w[i] / sqrt(2), w[j] / sqrt(2), wr[(i,j)], wi[(i,j)]] in JuMP.RotatedSecondOrderCone()
	)
	end

This doesn't change anything... unless there are parallel branches in the system (which is typically true for large systems).
On the primal side, writing Jabr per branch or per bus-pair doesn't change the feasible set.
On the dual side though, writing Jabr for each bus-pair changes the dual-feasible set, and we didn't account for that when exporting the dual solution.

I'd need to double-check the math, but I think that a quick fix would be to equally split the dual variable across parallel branches 🤔
cc @ivanightingale whose work led to this realization.

Total scaling + LogNormal noise

Implement a new load scaler as follows:

• first scale all loads by a common factor alpha (which models the change in total load), drawn from a uniform distribution whose min/max values are provided by the user
• then add independent multiplicative noise to each load, drawn from a LogNormal distribution with mean 1.0 and standard deviation provided by the user.