alsetlab / rapid

RaPId (a recursive acronym for "Rapid Parameter Identification") utilizes different optimization and simulation technologies to provide a framework for model validation and calibration of any kind of dynamical systems, but specifically catered to power systems.

RaPId

Abstract

RaPId (a recursive acronym for "Rapid Parameter Identification") utilizes different optimization and simulation technologies to provide a framework for model validation and calibration of any kind of dynamical systems, but specifically catered to power systems. A quick overview of RaPId can be found here.

Contents

1. Abstract
2. Contents
3. Quick Download: Release
4. Contributing
5. RaPId Introduction
6. Cite us in your Publications
7. RaPId - Quick Start
8. Demo videos
9. Software requirements
10. Installation
11. Using the toolbox
12. Creating FMUs
13. Creating FMUs for Power System Models
14. Acknowledgements
15. Dedication
16. No Warranty

Quick Download: Release

• If you are not familiar using GIT to work with repositories, and all you want is to check out RaPId, please go directly to the Release Tab.

Contributing

• Reporting Bugs: please use the Issues tab to report bugs.
• If you are familiar with GIT, and want to be involved in the development (from bug fixes, to anything else!), please clone the repository and use GIT to contribute.
• Contributions: Via Pull Requests.

RaPId - Introduction

RaPId (a recursive acronym for "Rapid Parameter Identification") utilizes different optimization and simulation technologies to provide a framework for model validation and calibration of any kind of dynamical systems, but specifically catered to power systems. The foundation of RaPId is MATLAB/Simulink which is necessary for processing the available measurements and for the simulation of the Modelica model within the MATLAB/Simulink platform. Simulation of the Modelica model is carried out using the FMI Toolbox for MATLAB. Within the MATLAB environment, RaPId provides the user with different functionalities, such as the set-up of an optimization problem, access to different solvers to simulate the model, and automates a parameter optimization process (or “calibration”) so that the model is not only validated (i.e. satisfies the error allowed in an objective function) but also calibrated (i.e. the error of the objective function is minimized by finding optimal parameters).

The user is provided with a Graphical User Interface (GUI) which he can use to set-up the model validation and calibration problem. More advanced users, such as researchers, can take benefit of the modular design of RaPId which allows the use of the command line interface via MATLAB scripts, the inclusion of dedicated optimization routines (by extending RaPId's source code), and the formulation of different optimization problems.

RaPId has been designed with the researcher in mind: to give maximum flexibility and efficiency to develop methods and derive tools for model validation, calibration and correction. The spirit of RaPId is that other researchers will use it and contribute to its features – as such, RaPId is distributed as an Open Source Software.

What RaPId becomes in the future is up to you, the researcher or engineer that needs freedom and flexibility to put your creativity in action, and if RaPId makes this happen, please share it with all of us by contributing with your developments!

RaPId - Cite us in your Publications

If you use RaPId in your work or research, all we ask you in exchange is that you cite the following papers. You are also welcome to submit your contributions (modifications to RaPId or examples) using GIT.

If you only have space for one publication, our prefered citation is the following:

• Luigi Vanfretti, Maxime Baudette, Achour Amazouz, Tetiana Bogodorova, Tin Rabuzin, Jan Lavenius, Francisco José Goméz-López, RaPId: A modular and extensible toolbox for parameter estimation of Modelica and FMI compliant models, SoftwareX, Available online 25 August 2016, ISSN 2352-7110, http://dx.doi.org/10.1016/j.softx.2016.07.004. Download paper: here

If you only have space for additional publications, you can also cite the following papers:

• L. Vanfretti, T. Bogodorova, and M. Baudette, “Power System Model Identification Exploiting the Modelica Language and FMI Technologies,” 2014 IEEE International Conference on Intelligent Energy and Power Systems, June 2-6, 2014, Kyiv, Ukraine. Download paper here.
• L. Vanfretti, T. Bogodorova, and M. Baudette, “A Modelica Power System Component Library for Model Validation and Parameter Identification,” 10th International Modelica Conference 2014, Lund, Sweden, Mar. 10 – 12, 2014. Download paper here.

RaPId - Quick Start

Demo Videos

To see RaPId in action, you can find demonstration videos on Youtube:

1. Demonstrating basic usage of RaPId:

• Using the GUI
• Using the CLI.

2. Using RaPId to identify Power System Parameters

• Using RaPId to identify exciter parameters in the MOSTAR power plant:

3. Advanced features (Future! Release: TBA)

• Extended functionalities of RaPId for small-signal (linearized model) and time-domain (non-linear time domain) power system inter-area electromechanical mode model validation and calibration: Inter-Area Mode Model-Validation

Note: RaPId is continuously evolving, thus the GUI and source code is subject to change (usually improvements) compared to what is depicted in the videos.

Software requirements (dependencies):

Here is compiled a list of the required software packages to run the toolbox.

• Matlab R2016b (MathWorks). R2018b will be supported and replace R2016b when the FMI Toolbox adds support for that version.

• Simulink (MathWorks)

• Matlab Toolboxes:

  • Optimization Toolbox
  • Global Optimization Toolbox
  • Statistics Toolbox
  • Signal Processing Toolbox

• FMI Toolbox v2.6.x and above (Modelon)

• Note on Operating Systems:

  • Testing for the current version of RaPId has only been carried out using Windows 10 (64-bit)
  • While the software should work in other operating systems supporting MATLAB/Simulink, the FMI Toolbox and with FMUs exported for that platform, this variants have not been tested.
  • In the future, testing will be carried for Ubuntu (LTS versions 16 and 18), and specific versions of Red Hat (TBD).
  • MAC OS cannot be supported natively, as the FMI Toolbox is not supported in Mac OS.

Installation:

Provided that all the software required have been installed on your machine, the installation has been automated with an installation script.

The first step of the installation is to procure the RaPId toolbox:

• You can download the .zip file from the GitHub homepage and unzip the file in any directory on the computer
• You can clone the repository on your local machine

The rest of the installation will be executed in Matlab:

1. Launch Matlab and change the working directory to the ./rapid/ folder of the RaPId toolbox.
2. Run the 'setupRapid.m' script located in this sub-folder.
3. Check that the dependency check didn't prompt any warning.
4. The GUI of the toolbox will be launched upon successful installation.

Using the toolbox:

The best way to get familiar with the toolbox is to refer to the examples provided. Each example is delivered in a separate folder in the ./Examples/ folder, containing the necessary elements to use the toolbox:

• the FMU containing the model to optimize
• the Simulink model providing a wrapper for the FMU
• a .mat file containing the reference measurements
• a .mat file (typically named 'container.mdl'), containing all the settings of the experiments

To load an example, click on the 'load container' button and select an example's container. After successful loading, you can simply click on 'Run Optimization' and the toolbox will run the optimization process.

To build your own experiments, the best practice is to work in the ./Examples/work_dir/ directory (it is ignored by GIT). You can start by copying the files of one of the examples, replace with your own FMU, load the copied container and modify the appropriate settings to suite your experiment.

Don't forget to save your container, after your modifications!

Creating FMUs

In principle, RaPId be used with any FMU generated from an FMI Compliant tool. However, because RaPId depends on the FMI Toolbox for MATLAB/Simulink, the user needs to check if their tool has been supported and/or tested by the FMI Toolbox developers, Modelon.

Creating FMUs for Power System Models

The development of RaPId was carried out in parallel to the development of a Modelica library for power networks to be able to validate and calibrate power system models. Currently, only the Modelica OpenIPSL library is recommended: https://openipsl.org

To create FMUs for power system models you can use a Modelica tool of your choice capable of generating FMUs. FMU generation with RaPId has been tested using JModelica and Dymola.

If you would like to use Modelica to generate FMUs containing power system models, you can use the OpenIPSL. Several of the examples in RaPId where developed using the OpenIPSL.

Acknowledgements

The initial development of RaPId was possible through the iTesla project funded by the European Commission through the FP7 program.

Dedication

To the memory of Jan Ove Gjerde.

No Warranty

Copyright 2015-2018 Luigi Vanfretti, Achour Amazouz, Maxime Baudette, Tetiana Bogodorova, Jan Lavenius, Tin Rabuzin, Giuseppe Laera, Francisco Gomez-Lopez

The authors can be contacted by email: [email protected]

This package is part of Rapid Parameter Identification ("RaPId") .

RaPId is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

RaPId is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with RaPId. If not, see http://www.gnu.org/licenses/.

rapid's Issues

Remove Dependency on Image Processing Library

When we implemented the first version of RaPId, I wanted to display the logo.
At that time (and still now) the only way that I knew how to do it was through some functions of the Image Processing library.

I think we can replace with the following function: http://se.mathworks.com/help/matlab/ref/imagesc.html

Otherwise, I guess there is not much point in keeping the image...

Don't develop prioritizing the GUI: Always update all script Examples!

For some reason, my old examples are not available, and more importantly, non of the new examples have any scripts to execute RaPId without a GUI.

I hate GUIs!!! We designed the software from the beginning to be independent from it.
Now the software has evolved, and I can't run my examples... it seems like @janlav and @GiuseppeLaera have done most of the development focusing on the use of the GUI. This is really great, but we should focus on NOT only developing the functionalities and updating the GUI, but most importantly, preserving and updating the script examples.

It is very important we have all examples with scripts.

I have asked @tinrabuzin that my old SecondOrder example should be automatically launched and tested at installation, so this would allow to check through scripts that everything is working fine.

We also need to develop a testing script that runs through all of the examples in the Examples folder and makes sure they are still compatible with the changes made to the source.

To summarize:

• @tinrabuzin use scripting example to be executed together with installation
• @janlav document all the information contained in RapidObject and what is needed to run a script. Also, build the automatic testing code to run through all the examples to check if they are working. Set the examples to be executed for a couple of iterations...

@janlav we latter need to discuss how to build a test code to check that all the optimizes are working...

Textbook example

The Rafael's textbook example should be fixed.

RaPId cmd

An absolute path is hard-coded in the script

containerFile='C:\Users\janlav\Documents\rapid_release\source\RaPId\examples\myTestIO64bit\variableIO64PSOtestContainer.mat';
contentOfContainer=who('-file',containerFile);
load(containerFile);

It should probably be passed as an argument, or else that is personalizable by the user.

Installation script: run a simple test example at the end

Suggestion From Luigi:

• The set-up script should test the installation by running the simple example I developed for Rafael. It is the text book example of finding the parameters of a 2nd order system. rum_cmd_rafael.m
• The code should close the Simulink diagram automatically, which with the current code is left opened.

Note @lvanfretti : I'm adding this new issue, to close the previous one, as the solution implemented by @tinrabuzin is working.

Implement relative path

The toolbox should implement support for relative path.
I think the default behavior should be to use relative path for model, FMU, and in and out data. This behaviour could be toggled off with a debug setting somewhere in the toolbox's advanced settings.

A good thing would be to include this code in the toolbox:
http://www.mathworks.com/matlabcentral/fileexchange/41253-improved-relativepath-m

Move run_rapid_cmd

I suggest we

1.Make a new folder called Templates and
2. move and rename it rapid_CLI_template.m or something and
3. Improve documentation for rapid() with an Example help pointing to this template as mentioned in Issue #34

Update of Example folder ( adding scripts)

From @lvanfretti

For some reason, my old examples are not available, and more importantly, non of the new examples have any scripts to execute RaPId without a GUI.

All the following examples should be updated to include a script running them:

• Cauer Lowpass Analog
• Generator Aggregation
• Line Equivalent
• Mostar
• Second Order

Note: When preparing the release on Github some examples were discarded because we needed to regenerate the FMU and couldn't do so.

Adjustable timeout setting

Currently, if the simulation of the model runs longer than 1s, the whole simulation is discarded. I propose this setting is integrated so that the timeout can be adjusted.

Cost weights are not being updated

I was trying to assign different weights to the output signals of the FMU. I noticed that for some reason those weights are not updated in the RaPId object. My guess is that there is a bug in the GUI.

Calibrate parameter inside Blocks of Simulink Model

We need to modify RaPId so that the parameters inside of Simulink can also be changed, and not only the parameters of the Model in the FMU.

The use case is control design using Simulink controller, while the plant is in the FMU.

@tinrabuzin maybe we can ask Ravi to spend some time on this, please explain him more in detail.

Integrating MVMO Algorithm as option for optimization

@janlav @MaximeBaudette

I have listened to a talk today regarding the MVMO algorithm by Jose Luis Rueda.

He has applied it to several power system problems with some large dimensions, and their results seem good.

I was wondering if we can integrate it into RaPId.

Let me know what you think, here are the related links:
https://www.uni-due.de/mvmo/download

List of RaPId dependancies

I created a few of the functions that go through the structure of the RaPId directory and check for the toolbox that is required by each of the files.

Currently, the Matlab function that checks the dependancies of the particular .m files cannot recognise that the toolbox is missing (it only returns the toolboxes that are used AND installed on the computer where it is executed).

That is why I created 'dependancy_list.mat' by running my script on my computer with everything installed. dependancy_list.mat is created by running check_dependancies.m.

What I propose now is that the end user should run check_installed.m which loads dependancy_list.mat, compares it with installed toolboxes and gives a warning if there is something missing. I also added this check in the setup_rapid.m.

Let me know what you think! @MaximeBaudette @janlav @lvanfretti

General Settings window

When I have to change something in my experiment, this window is one of the main reasons why I don't feel like doing that:

Is there any way that we can make the cells at least resizable so that you can see what parameters you're setting without having to click on the cell?

Avoiding the Simulink Container

During the initial design of RaPId, my intention was to avoid completely the Simulink "container".
However, we didn't really know what would be the use of RaPId with Achour, and we though the container would give users flexibility. But now that some people is using RaPId for problems that require > 1,000 simulations, I think the Simulink container is making things slow and therefore we are sacrificing too much performance for the sake of flexibility.

So, I think we can set this issue for someone to take care of it - the idea is to "add" a functionality that allows to define the inputs and outputs via script (we already can define parameters programmatically) - and in this way utilize the function from the FMI toolbox that allows to call the FMU directly.

The reason is that it adds a lot of overhead, as each time that a simulation is executed, the Simulink model is opened, simulated, and then closed. Instead, it is possible to avoid the Simulink container if we could instead find a way of specifying inputs and outputs for a given FMU directly in a script.

This functionality would be of great value once someone has done some work and knows exactly what outputs to be used. I think we would decrease the execution time at least by 1/2 for the models.

The container is also used for visualization of the iterative process, but this could be enabled through a simple plot when the inputs and outputs have been defined through scripting instead of the FMI Block.

Constraints implementation in all algorithm

Using the CG algorithm I noticed that the constraints setup in the configuration don't seem to be taken into account when running the algorithm.

I'm creating this issue so we remember to check that it is correclty implemented in all methods.

Wiki User Guide - Clean up and Move to Wiki

• - User Documentation

The user documentation should include:

• - Introduction (as in "RaPId_quickStart.docx")
• - Required Dependencies
• - Installation (explaining the automated script and displaying the prompts)
• - Functionalities (Deeper Description in "RaPId_quickStart.docx")
• - Appendixes:
• - FMU Generation using JModelica (as in "RaPId_quickStart.docx")
• - FMU Generation using MapleSIM ( @tinrabuzin you can populate this in the Wiki after you do the job for Mike)

@MaximeBaudette the idea of the Wiki I think is the best approach so people can provide input and examples. The examples should not go in the "user manual", but instead in an "Application Guide". @tinrabuzin and I will work in the application guide (similar to the one done for iPSL) in the coming weeks after the workshop and we have completed the paper for RaPId and iPSL.

There are several files in the ./Documentation that need to be cleaned up.
I propose the following is done:

• Delete .pdf files, as they can't be edited by users.
• Keep the original .pptx (2012 and 2013) in a folder with name and the RaPId_quickStart.docx to "./pre_release_documentation_materials"
• If User Manual.docx has been updated, we can keep it where it is, but it should be replaced by the wiki. The User Manual.docx should include the list of authors, and track changes. It should also have the iTesla logo and information about the project that is missing in "RaPId_quickStart.docx". If doing the wiki is easier than editing the Word file, I suggest we drop the word file and put everything in the wiki. In such case, we just need to put a file in the folder with a link to the wiki, and to mention that in the "./pre_release_documentation_materials" there are materials that can be useful to users but might be outdated (we keep them for the time being though).

Result Visualization

In current version, I haven't been able to use the "Last Result", "Display Result" buttons.

I propose the following implementation:

• Display Result: the Scope should be opened, the solution should be loaded in the FMU to get the optimal parameter values, and the simulation run to display the solution
• Last Result: A message should be printed in the Matlab command listing all parameters and their optimal values using the names obtained from the general settings, and the values from the solution.

Additional a message could be printed, telling the user whether the solution could be considered optimal or not. If the optimization has been stopped because of the fitnessStopRatio, the solution can be considered optimal. If the optimization has been stopped because the max number of iteration has been reached, then the solution is not optimal.

A step towards Continuous Integration

The idea is to start building a Matlab script with a battery of tests that should be run on the toolbox to check everything is working properly.

This is a first step for implementing continuous integration for the toolbox, which could be later automated via diverse tools (we are not yet at the point where we master this). But we can still start reflecting on what kinds of test should be run on the toolbox to check changes made.

From @lvanfretti for a first idea of what these tests could be:

Also, build the automatic testing code to run through all the examples to check if they are working. Set the examples to be executed for a couple of iterations...
@janlav we latter need to discuss how to build a test code to check that all the optimizes are working...

Setup script cannot be run

I get the following error message when I try to run the setup script.

BTW: Jan, that was after your last PR.

Dynamic Optimization (Parameter Estimation and Correction over time)

We need to figure out how we could support optimization over time, meaning that in the time-horizon during the simulation, we can calibrate the parameters of the model or the control actions.
This is known as dynamic optimization, but control people uses in NMPC.

Some of this work has been done in OpenModelica: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.278.7832&rep=rep1&type=pdf

So I am not sure we need to support this in full or if even possible given that for some methods we would need a linearized model and I think that is supported only partially in FMI 2.

However, I think that for some problems such approach, or similar such as trajectory sensitivities can be useful.

32bits and 64bits FMUs for all examples

All examples should be delivered with binaries for both 32 bits and 64 bits.

Fix help and custom documentation in Matlab for all functions and classes

Issues:

• Content.m must be correctly filled in and Sources-folder should be renamed
• Help consists of the Copyright section instead of function description (fix for all functions/classes)
• Help Report issue: copyright outdated (fix for all functions/classes)
• Help Report issue: no See also (fix for relevant functions/classes)
• Create custom documentation including HTML and other links to supplemental documentation.

Restructuration of the Rapid folder

Current situation:

The code of rapid is splitted in different folders dedicated to the core of the toolbox, the algorithms, the GUI elements.

Goal:

Reorganize the folders to clearly separate the core functions from the optimization algorithms. Have only one folder for all the algorithms files

Unix issues

Hello. Thank you for the awesome software you developed.

However, I have several issues installing and using it, that I would like to resolve with your preferable assistance. I am trying to install RaPId on a unix machine using last version in Releases section, Matlab 2014b and FMI Toolbox 2.6.4.

First of all, RaPId has dependency on Matlab up to 2014b and also Modelon FMI Toolkit, which is (according to Modelon) incompatible with Matlab 2014b.

Second, RaPId messes paths on unix. I was able to resolve it during the installation process by simply modifying the script. However, the problem seems to go very much under the hood to the extent that I can not fix it. It shows up when I try to run any example. It produces a warning like "FMI Toolbox was not able to load FMU into Simulink, because FMU was not found at relative path", which shows a corrupted path. I am also not able to open FMUme block in Simulink.

I would be very grateful for assistance.

Rebuild the installation package

The installation script and files should be checked to verify that all necessary files are present and working.

• check the script is running
• check no error after cloning the repo

Wiki Technical Documentation

Create the content in the Techical Documentation of the Toolbox for the Wiki

Optimization algorithms - technical description + input,output, & parameter descriptions

• PSO
• GA
• NAIVE
• CG
• PF
• NM
• Knitro
• fmincon
• COMBI

  Objects and methods used by core RAPID

• RAPIDCLASS
• RAPID-function
• func
• rapid_Simusystem
• rapid_ODESolve
• rapid_objectivefunction

GUI

• - GUI

NaNs generated after running RaPId

I created a new simulink and script file for 64 bit version of the Mostar FMU. When I run the script, there's a bunch of NaNs output in the command window after the rapidObject.runIdentification() line (line 78) is run. This is in the examples/mostar/mostar.m script

Generate FMU Menu not updated

The menu requires to load the values stored in dataFmu.mat to open.
I suspect it has not been updated as the other menus to use default values if nothing is found in the RaPIdobject.
BTW: maybe these settings shouldn't be in the RaPIdobject but somewhere else, like store on the user'd machine in a system folder?

Rewrite of run_GUI and run_CMD

I'm not very familiar with MatLab programming, but I think both interface can be rebuild around common core functions that both interface should use.
This way we could ensure a better maintenance of the code.

Also It seems that run_cmd is implemented in such a way that the user should modify the script for setting the parameters of his choice. I don't really like that and think it should be implemented in such way that everything is put in argument (using objects defining the plugins parameter)

GUI Issues with Matlab > 2012a

I sat down with Ahsan and found that there is an issue with the "General Settings" tab of the GUI when using Matlab 2013a, which probably also happens with newer versions of Matlab.

The tab is displayed and immediately hidden.

In "./gui/generalSettings.m" I took off the comment "uiwait(handles.figure1)" under "%UIWAIT ...", this made the tab show up, but then there were a lot of warnings and some errors in the command window.

So, what I did was not right, but it means we could probably fix this easily.

bogus issue

just to test how to close and reopen an issue