utwente-energy / demkit

DEMKit Toolkit Core

DEMKit

DEMKit is short for Decentralized Energy Management toolKit. It is designed with a cyber-physcial systems approach in mind to support both the simulation and real-world demonstration experiments of smart grid technhologies. It contains various algorithms to optimize and coordinate the use of energy flexbility to ensure the energy system can facilitate the energy transition towards a sustainable society. Furthermore, it models the behaviour of devices in individual components, such that simulation code can be replaces easily with API calls for real-world experiments on real hardware. Lastly, load-flow simulations are included to assess the performance of these algorithms within power grids.

DEMKit is originally developed at the University of Twente, Enschede, The Netherlands by the Energy Management Research at the University of Twente group. For more information on our research, please visit: https://www.utwente.nl/en/eemcs/energy/

We believe that we can only transition to a sustainable society through transparency and openness. Hence we decided to make our efforts open source, such that society can fully benefit from and contribute to science. We invite everybody to join us in this cause.

Knowledge resources

DEMKit is developed based on over 15 years of experience in the energy management research field. Therefore, knowledge resulting from various research projects and scientific publications is included. A list of publications by the responsible research group can be found here: https://www.utwente.nl/en/eemcs/energy/publications/

Furthermore, the following PhD dissertations form the foundation of DEMKit regarding respectively the core principles, optimization and heating models:

• G. Hoogsteen, "A Cyber-Physical Systems Perspective on Decentralized Energy Management", PhD Thesis, University of Twente, Enschede, the Netherlands. Available: https://research.utwente.nl/en/publications/a-cyber-physical-systems-perspective-on-decentralized-energy-mana

• T. van der Klauw, "Decentralized Energy Management with Profile Steering: Resource Allocation Problems in Energy Management", PhD Thesis, University of Twente, Enschede, the Netherlands. Available: https://research.utwente.nl/en/publications/decentralized-energy-management-with-profile-steering-resource-al

• R.P. van Leeuwen, "Towards 100% renewable energy supply for urban areas and the role of smart control", PhD Thesis, University of Twente, Enschede, the Netherlands. Available: https://research.utwente.nl/en/publications/towards-100-renewable-energy-supply-for-urban-areas-and-the-role-

How to cite

If you use DEMKit for publications, please cite either:

• G. Hoogsteen, "A Cyber-Physical Systems Perspective on Decentralized Energy Management", PhD Thesis, University of Twente, Enschede, the Netherlands. Available: https://research.utwente.nl/en/publications/a-cyber-physical-systems-perspective-on-decentralized-energy-mana
• G. Hoogsteen, J. L. Hurink, and G. J. M. Smit, "DEMKit: a Decentralized energy management simulation and demonstration toolkit," In 2019 IEEE PES Innovative Smart Grid Technologies Europe, Bucharest, 5 pages, October 2019. https://doi.org/10.1109/ISGTEurope.2019.8905439

Installation

The software requires Python 3.x (3.11 is tested)(https://www.python.org/) and depends on the Python libraries found in the requirements.txt file. Furthermore, it utilizes InfluxDB 1.x (1.8.10 is tested, 2.x is incompatible!)(https://www.influxdata.com/) to store simulation results and data. Optionally, Grafana is often used for visualization (https://grafana.com/).

DEMKit consists of two parts:

1. The core DEMKit code with all components (this repository)
2. A workspace with so-called simulation scenarios (models), utilizing the components and code of DEMKit. An example can be obtained here: https://github.com/utwente-energy/demkit-example

Various methods to install DEMKit are possible, provided that the requirements are met. However, we recommned to use one of the following two approaches. For a more detailed description, refer to the setup guide that can be found in the "doc" folder.

Docker-based setup

Setup

The easiest method is to have a setup utilizing Docker (https://www.docker.com/). Provided Docker and Git are installed on your system and you have copied your public SSH key to your Github account settings (https://github.com/settings/keys):

1. Download the setup file from the setup-folder (*.bat for Windows, *.sh for UNIX-based systems) and place it in the folder where you wish to install DEMKit.
2. Execute the file, which should install all software and build the DEMKit Docker image automatically. In the end, a demonstration simulation is executed.

To verify if your setup is functioning, browse to http://localhost:3000, login using username=admin password=admin and select the Example dashboard. Now you should see the results of your first simulation.

Running

You can run simulations by executing the "rundemkit.bat" file (Windows) or execute "docker compose up" on the command-line in the DEMKit folder. The "docker-compose.yaml" file contains the configuration, which you can modify to run another simulation scenario by modifying::

  - DEMKIT_FOLDER=example
  - DEMKIT_MODEL=demohouse

An example scenario can be found in the workspace folder. Note that DEMKIT_MODEL reflects the scenario to be loaded, which is the Python file without the "*.py" extension. To create your own simulation scenario, it may be wise to clone/fork our example into a new folder (don't forget to adjust your configuration) as starting base from https://github.com/utwente-energy/demkit-example .

Custom setup

Installation

Experts can install DEMKit on their own, which generally results in better simulation performance, but requires more steps. Assuming this repository is cloned to your system (depending on your operating system and/or specific setups, you may need to substitute "python" with "python3"):

1. Install the dependencies: python -m pip install -r requirements.txt
2. Install Grafana and InfluxDB 1.x if not yet installed. Optionally, you may run Grafana and InfluxDB using Docker, for which you can simply execute: docker compose up -d services/docker-compose.yml
3. Copy and rename the "conf/usrconf.py.misc" file to "conf/usrconf.py"
4. Modify the configuration file "conf/usrconf.py" to reflect your system's variables and paths.

Refer to the stup guide found in the "doc" folder for further instructions on how to set-up e.g. Grafana to connnect to the correct database.

Running

You can run simulations by executing demkit.py with 2 arguments::

python demkit.py -f FOLDER -m MODEL

Here FOLDER and MODEL represent the location and Python file (without the "*.py" extension) of the simulation scenario to be executed. An example scenario (which can be used as base) may be obtained from https://github.com/utwente-energy/demkit-example .

License

This software is made available under the Apache version 2.0 license: https://www.apache.org/licenses/LICENSE-2.0

The software depends on external software (e.g. Python, Grafana, InfluxDB and optionally Docker) and libraries. These external packages are likely to contain other software which may be licenced under other licenses. It is the user's responsibility to ensure that the use of external software and libraries complies with any relevant licenses. This also applies to created Docker images by the user through the usage and/or execution of the Dockerfile, Docker-compose files, and setup scripts provided with this software. A list of used Python libraries can be found in the requirements.txt file.

Contact

In case you need assistance, please contact:

Gerwin Hoogsteen:

• https://people.utwente.nl/g.hoogsteen
• g.hoogsteen [at] utwente [dot] nl
• demgroup-eemcs [at] utwente [dot] nl

demkit's Issues

Exception at live re-start

Hi Gerwin,

I get the following exception at start related to sunData, and sometimes also later. I merged the old ORTEP scripts using the sst sensor and model workspace with the most recent Demkit. In the ORTEP model the sunData is used to predict PV on both 3-phase and single-line inverters:

• pvc = LivePvCtrl("PVCTRL_3P_L"+str(i+1), pv, ctrl, sun, sim)
• pvc = LivePvCtrl("PVCTRL_SP_L3", pv, ctrl, sun, sim)

09:24:43 | MESSAGE: Starting
09:24:50 | MESSAGE: Simulating at time: 09-08-2023 09:24:50 CEST+0200
Exception in thread Thread-3 (zCallSingle):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/dennis/work/ortep/demkit-github/components/core/core.py", line 682, in zCallSingle
    return getattr(recv, func)(*args)
  File "/home/dennis/work/ortep/demkit-github/components/ctrl/groupCtrl.py", line 225, in initiatePlanning
    self.doInitialPlanning(s)
  File "/home/dennis/work/ortep/demkit-github/components/ctrl/groupCtrl.py", line 261, in doInitialPlanning
    results = self.zCall(self.children, 'doInitialPlanning', s, list(parents))
  File "/home/dennis/work/ortep/demkit-github/components/core/entity.py", line 143, in zCall
    return self.host.zCall(receivers, func, *args)
  File "/home/dennis/work/ortep/demkit-github/components/core/core.py", line 654, in zCall
    return self.zCallList(receivers, func, *args)
  File "/home/dennis/work/ortep/demkit-github/components/core/core.py", line 671, in zCallList
    result[recv] = getattr(recv, func)(*args)
  File "/home/dennis/work/ortep/demkit-github/components/ctrl/devCtrl.py", line 158, in doInitialPlanning
    result = copy.deepcopy(self.doPlanning(signal,  False))
  File "/home/dennis/work/ortep/demkit-github/components/ctrl/loadCtrl.py", line 133, in doPlanning
    p[c] = self.doPrediction(time-(time%timeBase),  time-(time%timeBase)+timeBase*len(signal.desired[c]))
  File "/home/dennis/work/ortep/demkit-github/components/ctrl/live/livePvCtrl.py", line 82, in doPrediction
    result = self.predictProduction(sunPrediction, startTime,  endTime)
  File "/home/dennis/work/ortep/demkit-github/components/ctrl/live/livePvCtrl.py", line 171, in predictProduction
    result.append(min(0, max(sunData[idx]['GHI']*self.model[b][0] + sunData[idx]['DNI']*self.model[b][1], self.maxProduction[b]) ) )
IndexError: list index out of range

Kind regards,
Dennis

crash with forwardLogging and real/imag mixup in devCtrl.py

When forwardLogging is enabled (seems to be default), there seems to be a mixup with commodities across phases that can cause a crash. I turned this crash in to a warning by checking if the commodity is planned or realized, e.g.:

@@ -137,14 +137,17 @@ class DevCtrl(OptCtrl):
 		if self.forwardLogging and self.host.logControllers:
 			for c in signal.commodities:
 				for i in range(0,  signal.planHorizon):
-					self.logValue("W-power.plan.real.c."+c,  self.plan[c][int(signal.time + i*signal.timeBase)].real, int(signal.time + i*signal.timeBase))
-					if self.host.extendedLogging:
-						self.logValue("W-power.plan.imag.c." + c, self.plan[c][int(signal.time + i*signal.timeBase)].imag, int(signal.time + i * signal.timeBase))
+					if c in self.plan:
+						self.logValue("W-power.plan.real.c."+c,  self.plan[c][int(signal.time + i*signal.timeBase)].real, int(signal.time + i*signal.timeBase))
+						if self.host.extendedLogging:
+							self.logValue("W-power.plan.imag.c." + c, self.plan[c][int(signal.time + i*signal.timeBase)].imag, int(signal.time + i * signal.timeBase))
+					else:
+						self.logWarning("W-power.plan.%s, unavailable for %s"%(c, self.name))

-					if self.useEventControl:
-						self.logValue("W-power.realized.imag.c." + c,self.realized[c][int(signal.time + i * signal.timeBase)].imag,int(signal.time + i * signal.timeBase))
+					if self.useEventControl and c in self.realized:
+						self.logValue("W-power.realized.real.c." + c,self.realized[c][int(signal.time + i * signal.timeBase)].real,int(signal.time + i * signal.timeBase))
 						if self.host.extendedLogging:
-							self.logValue("W-power.realized.real.c." + c,self.realized[c][int(signal.time + i * signal.timeBase)].real,int(signal.time + i * signal.timeBase))
+							self.logValue("W-power.realized.imag.c." + c,self.realized[c][int(signal.time + i * signal.timeBase)].imag,int(signal.time + i * signal.timeBase))

 		self.lockPlanning.release()
 		return dict(self.realized)

15:35:25 | WARNING: [LOADCTRL_L1] W-power.plan.EL2, unavailable for LOADCTRL_L1
15:35:27 | WARNING: [LOADCTRL_L1] W-power.plan.EL3, unavailable for LOADCTRL_L1
15:35:27 | WARNING: [LOADCTRL_L2] W-power.plan.EL1, unavailable for LOADCTRL_L2
15:35:27 | WARNING: [LOADCTRL_L2] W-power.plan.EL3, unavailable for LOADCTRL_L2
15:35:27 | WARNING: [LOADCTRL_L3] W-power.plan.EL1, unavailable for LOADCTRL_L3
15:35:27 | WARNING: [LOADCTRL_L3] W-power.plan.EL2, unavailable for LOADCTRL_L3
15:35:27 | WARNING: [PVCTRL_3P_L1] W-power.plan.EL2, unavailable for PVCTRL_3P_L1
15:35:27 | WARNING: [PVCTRL_3P_L1] W-power.plan.EL3, unavailable for PVCTRL_3P_L1
15:35:27 | WARNING: [PVCTRL_3P_L2] W-power.plan.EL1, unavailable for PVCTRL_3P_L2
15:35:27 | WARNING: [PVCTRL_3P_L2] W-power.plan.EL3, unavailable for PVCTRL_3P_L2
...

Furthermore, there seems to be a mixup for real and imaginary logging in the end of the code above and lower down in the same file:

@@ -239,9 +242,9 @@ class DevCtrl(OptCtrl):
 					self.realized[c][t] = plan[c][i]
 
 					if self.forwardLogging:
-						self.logValue("W-power.realized.imag.c." + c,self.realized[c][t].real,t)
+						self.logValue("W-power.realized.real.c." + c,self.realized[c][t].real,t)
 						if self.host.extendedLogging:
-							self.logValue("W-power.realized.real.c." + c,self.realized[c][t].imag,t)
+							self.logValue("W-power.realized.imag.c." + c,self.realized[c][t].imag,t)

Kind regards,
Dennis

startup logo gets skewed when terminal is not exactly 120 characters

Apparently, the startup ascii-art logo gets skewed when the (linux) terminal width is not exactly 120 characters wide, probably because it doesn't use newlines in the sys.stderr.write() statements.