robertobucher / pysimcoder

Block diagram editor and real time code generator for Python

pysimCoder

pysimCoder is a Rapid Prototyping application, that can be used to generate realtime code for different targets.

At present it is possible to generate code for Linux (with or without preempt-rt) on PC and Raspberry PI, for the BRIKI card (partially on the SAMD21 uC, see www.meteca.org) and for the NuttX RTOS.

The behaviour is similar to other RPC applications like Simulink and XCos.

pysimCoder can run in any Linux environment or in Windows using WSL. (at the bottom of this README is explained how to install pysimCoder in WSL)

Despite the fact that the main purpose of pysimCoder is to generate RT control code, it is also possible to perfom simple simulations mixing continous time and discrete time blocks.

Since 07.09.2022 pysimCoder implements subsystems (more than one block, right mouse click) and a new save format, based on json, which replaces the previous format of the files based on XML. Old files can be loaded and saved in the new format. The new format is better readable, because it can be simply opened by a python script like this:

import json
import pprint

fname = 'example.dgm'

f = open(fname,'r')
msg = f.read()
f.close()
fileDict = json.loads(msg)
pprint.pprint(fileDict)

A Virtual Machine (Virtual Box) is available. Please write to me to get the address and password to download it.

It is also possible to get a docker image with pysimCoder using

docker pull robertobucher/pysimcoder

More info at the end of this file.

http://robertobucher.dti.supsi.ch/pysimcoder/

23.01.2019 [email protected]

Installation

Project requires Python 3.9 runtime to be present on the system. Installation instructions and scripts expect Debian derived distibution and may require adjustements on other systems.

Superuser privileges are necessary for installation of the compiled drivers.

In cases when environment doesn't fulfill the aforementioned conditions, a containerized deployment with one of the provided images is preferrable.

Install system dependencies and python libraries

sudo apt install gcc gfortran git cmake binutils emacs python3.9 python3-numpy \
    python3-scipy python3-sympy python3-matplotlib python3-pip jupyter-qtconsole \
    python3-lxml python3-pyqtgraph libopenblas-dev liblapack-dev libxml2-dev \
    libcomedi-dev python3-pyqt5

sudo pip install -r requirements.txt

Alternatively the dependencies can be installed with provided scripts:

sudo python3 ubuntu_dependency_installer.py
sudo python3 python_libs_install.py

Build and install the code as a superuser with:

sudo make

Set environment variables in user .bashrc

make user

You may also install modules for different targets as a normal user with:

make <target>

Finally you launch the application from the command line. The .bashrc may have to be reloaded beforehand.

pysimCoder

pysimCoder as docker image

It is now possible to pull a docker image to run pysimCoder in a container.

Install docker (see https://docs.docker.com/engine/install/ubuntu/).

Download a pysimCoder image directly from the Docker Container page:

docker pull robertobucher/pysimcoder:latest

The image can be launched with

$ docker run --rm --env="DISPLAY" --net=host -v $XAUTHORITY:/tmp/.XAuthority -e XAUTHORITY=/tmp/.XAuthority robertobucher/pysimcoder:latest

At the prompt launch

psc

More info and the Dockerfile can be found at https://github.com/robertobucher/pysimCoder-Docker

pysimCoder in Windows

pysimCoder can run in Windows under WSL.

Simply install some additional packages download pysimCoder and istall it.

    sudo apt-get update
    sudo apt-det dist-upgrade

and

sudo apt-get install \
	gcc \
	gfortran \
	git \
	cmake \
	binutils \
	emacs \
	python3 \
	python3-numpy \
	python3-scipy \
	python3-sympy \
	python3-matplotlib \
	python3-pip \
	jupyter-qtconsole \
	python3-lxml \
	python3-pyqtgraph \
	libopenblas-dev \
	liblapack-dev \
	libxml2-dev \
	libcomedi-dev \
	python3-pyqt5 \
	python3-qwt \
	python3-serial

In Debian the following package should be installed for pyQt6

sudo apt-get install qt6-svg-dev

In Ubuntu the following packages should be installed for pyQt6

sudo apt-get install \
    qt6-base-dev \
    libqt6svg6-dev

Then move to a directory where to install pysimCoder and do

git clone https://github.com/robertobucher/pysimCoder
cd pysimCoder
make addfiles
make modules
make fmu
sudo make link
make user
make alias
make full_lib

pysimcoder's Issues

Support for multiple NuttX targets

Improve export by adding NuttX current target (arch, chip, board) commit to NuttX mainline adds NUTTX_ARCH, NUTTX_ARCH_CHIP and NUTTX_BOARD variables to NuttX export. We could use this to identify the target in export file and even to support multiple exported targets as a subdirectory of nuttx-targets directory.

Ideally the build for specific target would be selected from GUI, which would also take care of blocks' source code compilation. This way the user would not have to go to CodeGen/nuttx/devices and run make manually. Similar approach could be used for other targets including Linux.

No module named 'supsisim'

I have made a model in pysimCoder, when I click on the "simulate" button and then when the simulation is finished I click on the plot block but the result from the plot does not appear and I got the following message: "This plot is stored under the tmp folder and can be opened with a double click". I am not sure why the plot window does not appear? If any of you have any problem would be nice to share any useful information :)

Second problem I have is, when I run the code from the tmp.py file , I got the following error:
ModuleNotFoundError: No module named 'supsisim'
Any suggestion how can I solve this? What should I install or do to be able to run the code?
I also attached the model.

Thank you in advance.
print.pdf

Suggestion to use single main function for all Linux RT targets and conditionalize /dev/nrt open attempt

/dev/nrt and iopl(3) is not available non-x86 targets and even on x86 ones requires modification
of the system kernel installation by local or laboratory systems administrator.

I suggest to use single Linux linux_main_rt.c for all targets and conditionalize /dev/nrt and iopl(3)
sequence by some define which can be provided in template. Then it would be easy to have one
template without iopl access (i.e. rt.tmf) and another with it (i.e. rt_iopl.tmf).

I prepare pull request, only question is the C processor define name. Some options

1. CG_WITH_NRT or IOPL
2. PYSIM_WITH_NRT or IOPL
3. PYSIM_CG_WITH_NRT or IOPL
4. WITH_NRT or IOPL
5. WITH_RAW_IO_ACCESS

I suggest to consider/discuss even some other minor changes.

• when parameter priority (-p) is set to -1 then do not attempt to switch to FIFO RT policy. It is clear that no real-time or latency guaranties can be provided in such case, but it is great to check RT code build path and generated executable by unprivileged users
• add support to select value for priority parameter in target selection dialog. One option is "task priority" filed another is to provide there option to specify additional parameters for executable in free form. In the fact both would worth to be provided
• consider how to specify upload method for the run command, this is for longer discussion but worth to be done to make running of the generated code on RPi, MZ_APO and NuttX targets easy

Raspi ADC/IMU blocks

Dear Roberto:

I have been testing the new code for raspberry pi crosscompiling with pysimcoder. Great work. I am wondering if the spi devices to connect with the ADC/IMU blocks are particular ones or the code is generic and must be tuned depending on the selected chip.

Thanks

pysimCoder on other platforms

I've begun to work with docker, in order to produce docker images and try the SW on other platforms (Mac and Windows).

I've tested the generated image under Linux and it correctly works. The file for the generation of the image are now provided in the disctro. The README.md file contains a short description of the steps required to generate the finale image.

Anaconda compatibility

Could it be possible to install pysimCoder inside an anaconda environement? This module could benefit from the Scipy, Numpy and Control modules already available in that Python distribution.

If it is not possible, could you explain what could I modify during the installation for such purpose?

How to make the block like ±|u|^v

I'm trying to make the block like the block like ±|u|^v.

Under folders "/pysimCoder-master/resources/blocks/blocks/",
with reference to ".xblk blocks",
under folders "/pysimCoder-master/BlockEditor/",
using defBlocks.py or xblk2Blk.py, I'm trying to make the block like the block like ±|u|^v
but I haven't been able to do it.

So I would appreciate it if you could provide samples targeting ±|u|^v.

Best regard.

MZ_APO CTU education kit support

The MZ_APO is Xilinx Zynq 7010 based education board used for Computer Architectures and Real-time Systems Programming courses at Czech technical University in Prague.

The VHDL FPGA design implemented for the curse provides multiple peripherals including support to connect two DC motor kits with incremental encoder feedback. Another advanced design includes PMSM motor control peripherals which allows to connect kit to PMSM motor power electronics as well.

The plan is use kits as another target to demonstrate pysimCoder.

@beqirdio (our thesis student) started to work on the inclusion of the MZ_APO support. Due to HW education goals, there is no driver on the Linux kernel side but peripherals are directly accessed by mmap. We decided to use MZ_APO specific MMIO registers sets directly for pysimCoder target. Support for three (red green blue) rotary encoder knobs has been implemented by @beqirdio as the initial demonstration. DC motor peripherals will follow. The pull request is in preparation.

There is initial version to include MZ_APO support provided for the discussion if it is acceptable

https://github.com/beqirdio/pysimCoder/tree/devel

There is one change in the generic code part as well which helps with diagnostic message print when
there is syntactic error in included RCPblk, it is very difficult to to find cause of failure to locate block
generation function import failure is silent. Limitation to *.py files has been required because else there
is reported huge amount of failures to import help files as the Python sources

beqirdio@8123c8e

Comments welcomed.

Clone block command crashes PysimCoder editor

Every time 'Clone block' is invoked using GUI in editor, the program crashes completely.

Traceback shown below:

Traceback (most recent call last):
  File "/home.nfs/beqirdio/Documents/PYTHON/toolbox/supsisim/editor.py", line 165, in cloneBlock
    item.clone(QPointF(DP, DP))
  File "/home.nfs/beqirdio/Documents/PYTHON/toolbox/supsisim/block.py", line 141, in clone
    b = Block(None, self.scene, self.name, self.inp, self.outp,
  File "/home.nfs/beqirdio/Documents/PYTHON/toolbox/supsisim/block.py", line 36, in __init__
    raise ValueError('Needs 12 or 3 arguments; received %i.' % len(args))
ValueError: Needs 12 or 3 arguments; received 11.
Aborted

PdServ integration

Hello Roberto,

I am member of the EtherLab team (etherlab.org) and the creator of the EtherCos package (Scicos, EtherLab, EtherCAT) many years ago.

I am quite happy to see your new project.

I have started a new fork

https://github.com/igh-ab/pysimCoder/tree/master

to test several pysimcoder extension

• Integration of pdserv library, to use of GUI testmanager, DLS data logging system, and pdcom and qt-pdwidgets library
• Extend the code generation without using a precompiled library, code generation generates one single C file with all needed code included. This will simplify compilation and cross compilation.
• Extend the codegeneration to be able to use non function based blocks, like fieldbus IO-systems. Try to port the Matlab TLC methods to the code generation. Blocks add code to includes, linked libraries, CFlags, predefinition, function, prerun, run and postrun sections. Example implemenation sineBlk, constBlk, mxmultBlk, NullBlk.
• Extend codegeneration by derivated classes of RCPblk, to be able to handle families of blocks.
• Integration of first EtherCat blocks

Please feel free to comment or discuss my ideas.

Best regards

Andreas

Conflicting and vague installation instructions complicate deployment

Installation procedure described in README.md doesn't specify versions of individual dependencies, duplicates installation of certain system packages, like python-scipy, in ubuntu_dependency_installer.py and in manual procedure, while leaving others, like gfortran in only one.

Several of the packages, like already mentioned python-scipy, were at some point renamed to python3-*, reflecting new version of the python runtime used, and are no longer available under the original name in many distros[0], including recent releases of popular Fedora and Ubuntu distros. Additional, older README file, with different instructions is also an issue.

All things considered, current state of the instructions leads to unnecessary confusion and loss of time on the side of new users.
Potentially deterring contributors that would be otherwise interested in the project.

To improve the situation I'd propose one of the following changes.

1. Rewrite instructions to have a single list of dependencies and ensure that all are installed in minimum number of commands.
2. Specify, at least, minimum viable version of the dependencies and OS.
3. Create requirements.txt file, listing packages, constraints and licenses for packages installed using pip.
4. (Slightly more complicated) rewrite the repo to use poetry[1] packaging and publish the package on PiPy repo.

I'd be willing to tackle points 1. trough 4. personally. With a view that rewriting the repo to use poetry might prove not prove worthwhile, depending on exploration of the option.

[0] https://pkgs.org/search/?q=python-scipy
[1] https://python-poetry.org/

New file format in pysimCoder and subsystems

The last commit contains support for subsystems. There i s probably a lot of test to do, in order to validate this new feature.

To facilitate the storing of files containing subsystem, I changed the format used to store the ".dgm" files. Instead of the previous XML format, the new files are now saved in json format. This allows to better read the files using for example pprint.

import json
import pprint

fname = 'example.dgm'

f = open(fname,'r')
msg = f.read()
f.close()
fileDict = json.loads(msg)
pprint.pprint(fileDict)

The old file format can still be read and imported in pysimCoder.

In order to get the subsystems, simply chose the blocks (more than 1!) and create it using the right mouse click on one of the blocks.
Subsystems can be reopen with a double click. Modifications should be manually reloaded after closing the superblock window.
It is not possible to add port to superblocks yet.

Issues using Tests

Hello! I am trying to run your tests to try generating C-code, but there are errors when I try to run any test file. They are different depending on the test. I would appreciate any help in resolving these issues!
This is using the docker image with the web-based NVC viewer.

Here are a few of the errors. I have a suspicion that I am doing something wrong.

/pysimCoder/Tests/ControlDesign/BallOnWheel

BoWRT.dgm

BoWRTfix.dgm

BoWsim.dgm

/pysimCoder/Tests/ControlDesign/Disks

disk_sim.dgm

diskRT1.dgm

scopy.py and RTPlot.py

I've been asked to modify the 2 plots system in order to have the execution/simulation time on the X-axis instead of the sample value.

I've implemented a temporary solution (not yet in repository) where i have this solution:

1. for the RT-scope in block diagram I send an additional parameter when I open the scope.py application with the sampling time: then I use this value to modify the X-axis on the plot
2. For RTScope.py I put a new editor field with the sampling time in dialog and this value is used to recalculate the time value on the X-axis

A better solution is to send the time value in the data exchanged with the block diagram, but in this case there are a lot of blocks to be changed (serial, TCP, and UDP output).

Discussion is open!

New branch "subsystems"

This new branch is an attempt to implement "subsystems" in pysimCoder. There are a lot of modifications compared to the previous versions:

• dgm files are now saved into a json format instead of the previous xml format. Files are in this cases more readable because the saved file is a Python dictionary.
• The files are saved in the new format and they can no more be opened by the previous versions of pysimCoder!
• Subsystems can be created by selecting more than one block and using the popup menu of the right mouse button
• It is possible to reopen a sysbsystem and it is possible to modify it internally. Modifications should be reloaded explicitly!
• It is not possible to add input or output ports to the subsystems yet!

A lot of tests are now required (I've implemented the code in this last 2 days...) !

Moving from PyQt5 to PyQt6

I've modified the files to work with both PyQt5 and PyQt6.

If the try to import he PyQt6 packages fails, pyimCoder starts with the packages of PyQt5.
Due to the new approach of PyQt6 in handling the enum values, it can happens that some errors appears.
Please, let me inform for every problem related to this issue.

pysimCoder support for Silicon-Heaven infrastructure

Silicon-Heaven (SHV) is an open source communication infrastructure that implements an open protocol for data serialization. Support of SHV in pysimCoder allows the runtime tuning of model parameters and runtime block diagram introspection. This issue is an introduction to upcoming commits bringing SHV support to pysimCoder mainline. The complete manual will be presented on pysimCoder wiki.

Code Generation process changes

The SHV support brings following major changes to code generation files:

• SHVgen.py: new file taking care of SHV related C code generation. Implements generation of required SHV headers, env variables, entry point function or static tree generation if selected. The generation functions are called from RCPgen.py.
• RCPgen.py: added SHVgen.py functions calls.
• scene.py: Function generateCCode newly defines additional parameters like block name or parameters' names, also sets environmental variables with SHV setting.
• dialg.py, pyEdit.py: added GUI window for SHV parameters setting.

ExtLibs

New folder ExtLibs was added to pysimCoder root directory. Additional external libraries are downloaded into this folder if required.

Template Makefile

LinuxRT and NuttX template makefiles were changed to include header files from SHV related external libraries if libraries are downloaded.

CodeGen/Common

SHV related code is located in new folder CodeGen/Common/shv. The source code consists of four files:

• shv_com.c: implements TCP communication interface with the server.
• shv_tree.c: common functions with tree operations (search, add...)
• shv_methods.c: implementation of supported methods (get, set, ls, dir) that can search in the tree or change model's parameters
• shv_pysim.c: contains entry point function shv_tree_init called from generated C code. Takes care of tree initialization if dynamic tree selected.

pyblock.h

Two additional structures added to pyblock.h for SHV support.

Documentation

Documentation describing pysimCoder setting for SHV support and compilation of required additional applications can be found on pysimCoder wiki.

Future Improvements

SHV can currently only read and write double parameters, integer parameters and lists are not supported.

CC @robertobucher @ppisa

Flipping the block in the GUI should not flip the block's picture

Make the block flip such that the svg won't flip with it.