frederikgeth / powermodelsprivacypreserving.jl Goto Github PK

convert it to a paper after OEFY

• clean up runtests.jl
• license.md
• readme.md
• installation notes
• documentation
• travis for unit testing, doc deployment?

Not sure how we can deal with non-Guassian distributions and up with computationally feasible chance constraints.

Key literature I've encountered:

• Nemirovski, A., & Shapiro, A. (2007). Convex approximations of chance constrained programs. SIAM Journal on Optimization, 17(4), 969–996. https://doi.org/10.1137/050622328
• Lubin, M., Bienstock, D., & Pablo, J. (2016). Two-sided linear chance constraints and extensions, 1–23.
• Roald, L., & Andersson, G. (2017). Chance-constrained AC optimal power flow: reformulations and efficient algorithms, 1–8. Retrieved from http://arxiv.org/abs/1706.03241
• Venzke, A., Halilbasic, L., Markovic, U., Hug, G., & Chatzivasileiadis, S. (2017). Convex relaxations of chance constrained ac optimal power flow, 1–8. Retrieved from https://arxiv.org/pdf/1702.08372.pdf

The idea is to wrap up a first version of the code with the following features:

• an re-implementation of [1]
• a variant with loss faithfulness instead of cost faithfulness
• obfuscation of the branch shunt admittance #1

The open TODOs are:

• add variables for shunt admittance
• make code more amenable to running on multiple data sets. i.e. extract parts of the scripts and make it part of the core of the module (i.e. add to /src)
• decide on transformer tap/shift anonymisation

[1] Fioretto, F., Mak, T. W. K., & Van Hentenryck, P. (2019). Differential Privacy for Power Grid Obfuscation, 1–9. Retrieved from http://arxiv.org/abs/1901.06949

• Which parameters are they trying to obfuscate?
• Which data do they need? Is it the same?
• Guassian vs Lapacian? What are the implications?
• Shunt admittance obfuscated?
• Methodology? How does it compare?

Todo by the next telco (30/4)

see
[1] Dvorkin, V., Fioretto, F., Van Hentenryck, P., Kazempour, J., & Pinson, P. (2020). Differentially Private Optimal Power Flow for Distribution Grids, 1, 1–9. Retrieved from http://arxiv.org/abs/2004.03921

extract parts of the scripts and make it part of the core of the module, to automate the testing on various data sets

Progress this week:

• branch shunts merged
• solved issue with zero resistance, nonzero reactance branches (Elliot/Fred)
• ran bigger test cases (Elliot)
• noise on series/shunt admittance (Fred)

Agenda

• discussion Data61 organization on Github.com
• chance constraints on laplace distributions are SOC-representable yay!

Action items

• Fred arranges annual github subscription for the team (preferably 7)
• David puts report structure on teams, so that we can start pulling things together
• Run the whole of PG lib test cases (Elliott)
• Export to 'm' files through https://github.com/lanl-ansi/PowerModels.jl/blob/2818685d9d5f24cea8d3a1c57f0e511578cfddaa/src/io/matpower.jl#L656

Action items previous meeting:

• Branch with variable shunt admittance
• Take a look at code https://github.com/wdvorkin/DP_CC_OPF, supplement to [1]
• Documentation? -> Fred
• Github Actions - try to transfer to D61, try to prepay Github actions for 1 year (David)
• Fred has to take a look at noise on series/shunt admittance -> they are not independent (#1)

Mosek cookbook:
https://docs.mosek.com/modeling-cookbook/index.html
https://www.juliaopt.org/JuMP.jl/v0.21.1/constraints/#Quadratic-constraints-1

(see #17)

This makes no sense physically, so we should take a look. We may need to change how the noise is being generated. Furthermore, it raises the questions if this may happen for r and x?

Add unit tests for Milestone V 0.2.

Series resistance is proportional to length and cross section
Series reactance is a function of the voltage level
Shunt susceptance is proportional to length
Current rating is a function of series resistance

At the very least shunt susceptance should be obfuscated as well.

Previous meeting: #20

Action items previous meeting:

• Fred arranges annual github subscription for the team (preferably 7)
• David puts report structure on teams, so that we can start pulling things together
• Run the whole of PG lib test cases (Elliott)
• Export to 'm' files through https://github.com/lanl-ansi/PowerModels.jl/blob/2818685d9d5f24cea8d3a1c57f0e511578cfddaa/src/io/matpower.jl#L656

Progress:

• testing on PG Lib #24
• results from PG Lib: all cases working up to 6515 bus, exceptions: pglib_opf_case10000_tamu, pglib_opf_case13659_pegase, pglib_opf_case2736sp_k, pglib_opf_case2737sop_k, pglib_opf_case2746wop_k.m, pglib_opf_case500_tamu.m_unsolved, pglib_opf_case9241_pegase
• report draft

Action items:

• Fred arranges annual github subscription for the team (preferably 7)
• Fred talks through how PowerModelsDistribution works in one of the upcoming meetings.

Paper on PowerModelsDistribution: https://arxiv.org/pdf/2004.10081.pdf

In https://github.com/lanl-ansi/PowerModelsDistribution.jl, you find a generalization of PowerModels where branch series/shunt impedance are (3x3) matrix parameters. Soon, (4x4) will be supported. Anyway, a potential research path is to support that case, this would allow us to obfuscate power distribution data sets.

In principle, you can install this code through
(v1.3) pkg> add https://github.com/frederikgeth/PowerModelsPrivacyPreserving.git

However, at least on my computer, I don't manage to log in, as it keeps repeatedly asking for authentification. It may have something to do with JuliaLang/Pkg.jl#1733

@afeutrill @davidsmith2020 , can you install the package with
(v1.3) pkg> add https://github.com/frederikgeth/PowerModelsPrivacyPreserving.git?
if that works, you should be able to do this next:
(v1.3) pkg> develop PowerModelsPrivacyPreserving

and then you can go in github desktop, 'add local repository' and point to ˜/.julia/dev/PowerModelsPrivactyPreserving.

transformers are branches an additional complex tap ratio parameter. We commonly use polar coordinates for this parameter

• the magnitude part is called tap,
• the angle part is called shift.

Generally, tap is already public information, as it can be derived from voltage level data (e.g. AREMI already has made geographical and voltage level data available for Australia). Generally shifts are 0, and if they're not it's likely to be public information as well. (e.g. phase shifting transformer are much more expensive, and subject to regulatory scrutiny)

I'm fine ignoring this for now, but it is a nuance to track

Previous meeting see #8

• branch with variable shunt admittance - Andrew to open Pull Request for discussion
• Discuss new paper [1]
• Discuss and compare with code https://github.com/wdvorkin/DP_CC_OPF, supplement to [1] -> next telco
• restructured test scripts (thanks Elliott!)
• documentation? -> Fred
• Github Actions - try to transfer to D61, try to prepay Github actions for 1 year (David)
• Fred has to take a look at noise on series/shunt admittance (is it independent?)

next meeting see #20

currently we can obfuscate the impedance of branches, however, there are more component data and problem types to support long-term

• transformer parameters
• load power consumption (and ZIP parameterization)
• generator set points (clearing volumes) in the context of market clearing
• HVDC point-to-point (matlab-style) or HVDC grid models (PMACDC)
• switches, storage systems, bus shunts (capacitor banks)

A list of things to discuss tomorrow in the meeting

• Show how to use github watch functionality
• Show how to install this package and how to put it in development mode #2
• Show how to run the unit tests, and add unit tests
• Show how to branch the code, do pull request from branch to master, and code review #7
• Cleaning up the code #3
• We should run the code on more power flow cases #6
• Supporting more components to anonymise #4
• Larger scale tests: sanity checks and validation?
• Discuss new paper [1] -> next telco
• Discuss and compare with code https://github.com/wdvorkin/DP_CC_OPF, supplement to [1] -> next telco
• Plan for milestones and writing a paper
• documentation? -> next Fred todo
• Review Andrew's approach

Suggested milestones

• V 0.2
• V 0.3

Please add your agenda items to this thread :)

next meeting #17

[1] Dvorkin, V., Fioretto, F., Van Hentenryck, P., Kazempour, J., & Pinson, P. (2020). Differentially Private Optimal Power Flow for Distribution Grids, 1, 1–9. Retrieved from http://arxiv.org/abs/2004.03921

A way to test the scalability of the code is to run it on all test cases in the PG Lib OPF test cases, specifically OPF and OPF with HVDC.

This testing should not be done on Github Actions as it is too time consuming

Please add papers you found in this thread:

• Fioretto, F., Mak, T. W. K., & Van Hentenryck, P. (2019). Differential Privacy for Power Grid Obfuscation, 1–9. Retrieved from http://arxiv.org/abs/1901.06949
• Dvorkin, V., Fioretto, F., Van Hentenryck, P., Kazempour, J., & Pinson, P. (2020). Differentially Private Optimal Power Flow for Distribution Grids, 1, 1–9. Retrieved from http://arxiv.org/abs/2004.03921
• Dvorkin, V., Hentenryck, P. Van, Kazempour, J., & Pinson, P. (2019). Differentially Private Distributed Optimal Power Flow. [Math.OC].
• Gonçalves, C., Bessa, R. J., & Pinson, P. (2020). Critical overview of privacy-preserving learning in vector autoregressive models for energy forecasting. Retrieved from http://arxiv.org/abs/2004.09612
• Jia, M., Wang, Y., Shen, C., & Hug, G. (2020). Privacy-preserving Distributed Probabilistic Load Flow, 1–8. Retrieved from http://arxiv.org/abs/2004.09131

The current export still exports the original branch impedance/susceptance values