Using SBFspot to read data from SMA Energy Meter about sbfspot

[SBFspot]

[SBF@25/08/2014]
If you like this feature request, please vote

[SBFspot]

[Willimeter@25/08/2014]
That would be a good feature. best regards!

[SBFspot]

[tutu81@8/09/2014]
Voted as well. This feature would be awesome, allowing us to know energy that I produce, sell and buy in only one piece of software! What else?

[SBFspot]

[tutu81@17/10/2014]
Hi all,

It is something new about this great feature? Tried to contact via the provided link but without success.

Best regards,
Vince

[SBFspot]

[SBF@17/10/2014]
Still working on it...

[SBFspot]

[Arlen@22/10/2014]
Just red there
http://www.photovoltaikforum.com/datenlogger-f5/download-smaspot-t100965-s10.html
that ur working (again) at ur software.

My Last try was with smaspot.
During the next weeks i should take my raspberry back into operation and test ur software.

Thumbs up for ur effords!

[SBFspot]

[krubik@30/12/2014]
Hi Luc,

are you still working on this?

Klaus

[SBFspot]

[SBF@30/12/2014]
sure, but I had a lot of other things to do... I'll try to finish it asap

[SBFspot]

[tutu81@9/02/2015]
Hi Luc,

May we help you? For exemple testing some piece of code?

Regards,
Vince

[SBFspot]

[DonConto@13/02/2015]
Hi, i am also interested to the energy meter support. I wrote some C code to listen to the multicast packets on port 9522. During my tests in noticed, that SBFspot opened a port UDP 9522. Does this mean, that there is already some kind of code implemented?

[SBFspot]

[SBF@13/02/2015]
No, there is no EM code in the official release...
SBFspot uses UDP9522 to talk to your inverter

[SBFspot]

[haggan@23/06/2015]
There is a python program here.
Also exist in C code
Could be used as a base maybe?

https://github.com/datenschuft/SMA-EM/commits/master

Whats current status of this issue?

[SBFspot]

[SBF@23/06/2015]
Currently the EM part is a bit "on hold" - Try to resume it in the next days.
I have it working for SQLite, but I have to test if it doesn't overloads the db as there is a lot of data to process (every second)

[SBFspot]

[haggan@23/06/2015]
Cool
Can I maybe test the code I have the Energy meter and a spare RaspPI to play with.

[SBFspot]

[SBF@21/07/2015]
Short status update: The EM datastream is overloading the SQLite DB (more or less like expected)
Writing a record every second is a bit too much, so SBFspot gets a lot of 'database locked" errors.
I have to review the design of the EM reader and I'm also reviewing the SQLite code to avoid these DB locks as much as possible.
So it's a work in progress ;-)

[SBFspot]

[course66@24/08/2015]
I'm very interested too. If SBFspot will implement SMA Energy Meter, I'll buy it, otherwise not.
ciao

[SBFspot]

[craigshaffer@22/09/2015]
I'd like to see your code so that I can use it as an example of how to populate the consumption data in the database. I've got a $25 meter design that with a $50 current clamp can read the consumption data and put it in a response to an HTTP request. I don't understand, however, how to modify the database in sbfspot to get it to upload that data into pvoutput.

[SBFspot]

[tutu81@21/10/2015]
Hi SBF,

Something new for this feature ?

[broke23]

Is this still ongoing SBF ?

[mahadi]

[SBF@13/02/2015]
No, there is no EM code in the official release...
SBFspot uses UDP9522 to talk to your inverter

@sbf-: Is it possible to change the port number or is this predefine by SMA? Background: I am getting conflicts while reading multicast messages from my Sunny Home Manager on port 9522 and the usage of SBF Spot in parallel.

[sbf-]

Port 9522 is reserved for SMA by IANA
sma-spw | 9522 | udp | SMA Speedwire

[rucksman]

Is this still on your roadmap? I am experimenting on using both SBFspot and a PHP script to read data from the Energy Meter, but this seems not to be possible at the moment. If I run SBFspot, I cannot connect with my PHP script, and running my PHP script does not allow SBFspot to connect. Funny enough it is possbible for a very short time to use both, if I first start my PHP script and then run SBFspot. If your time does not permit implementing this feature, probably you might be able to investigate on the problem of reading out data in parallel via SBFspot and from EM? I did some research, and the closest I got was this post here: https://stackoverflow.com/questions/5106674/error-address-already-in-use-while-binding-socket-with-address-but-the-port-num
Basically what it says that probably only the connection to the socket is closed but not the socket itself. The socket goes into a TIME_WAIT state and it takes some time (some minute) until it is relased. Maybe a solution can be to use SO_REUSEADDR (https://linux.die.net/man/3/setsockopt).

[sbf-]

Is this still on your roadmap?

Well, the biggest problem is I don't have a ethernet connected inverter and an EM neither. This code is written by someone else. So I'm afraid I can't really help here.

I read the article on StackOverflow and also here and this paragraph isn't very promising:

Oddly, using SO_REUSEADDR can actually lead to more difficult "address already in use" errors. SO_REUSADDR permits you to use a port that is stuck in TIME_WAIT, but you still can not use that port to establish a connection to the last place it connected to. What? Suppose I pick local port 1010, and connect to foobar.com port 300, and then close locally, leaving that port in TIME_WAIT. I can reuse local port 1010 right away to connect to anywhere except for foobar.com port 300.

But maybe you can help and test the proposed SO_REUSEADDR to see if it helps?

[rucksman]

Theoretically I can do that of course. But I have never done a single line of code in C++, so I have no idea about syntax etc. If you could point me in the right direction I would be very happy to test and give some feedback.

[notreallybob]

@Sbf: It's been a long time since I've done socket programming, so this might be wrong: Is the cause of the SO_REUSEADDR due to using "connected UDP"? ie. The UDP socket is bound to a specific address and port. The only reason I've ever needed to use bound port/address for UDP was for reducing connection setup and teardown for each datagram send. Since sbfspot is really only fired off once every 5 minutes via cron, perhaps you could live with using "unconnected UDP" and not bind the port and address?

[mincequi]

I can do that within the scope of #439

[rucksman]

I can do that within the scope of #439

Having data from the Energy Meter as well would be awesome! Is your fork (already now) a fully functional replacement for the "original" SBFspot?

[mincequi]

My fork tries to stay as close as possible to the original SBFspot and it is planned to be a drop-in replacement (until it potentially gets merged back to the original repo).
However, changes of my fork are rather drastic:

• It shall run in daemon mode (instead of a cronjob)
• Data import and export intervals are now decoupled (so, live and non-persisted data can be updated more often than persisted data)
• As of now, PVUpload is not respected. This should be fixed (or integrated in SBFspot itself).

Currently, I am working on a MQTT based protocol to export live and historical data (to be shown on an iOS/Android App). Once this protocol is proved to work, I will integrate exporting consumption data from an Energy Meter.

I will keep you updated, once I get this in a proper state.

[rucksman]

I will keep you updated, once I get this in a proper state.

@mincequi Do you have a rough idea when this could be the case?

[mincequi]

@rucksman
There is an implementation to read the SMA Energy Meters: https://github.com/RalfOGit/sma-emeter-and-inverter-to-influxdb-cpp
I will investigate that one and then we could integrate this one into SBFspot.

What are your exact requirements? Do you need database storage?

[rucksman]

@mincequi Not necessarily. Would be sufficient to have access to the data via MQTT to integrate measurments into my Smarthome (Homematic + Node Red). Database storage could be done through MQTT/Node Red as well if it would be necessary. But of course it would be a nice bonus if you would integrate that directly from the start, unless it would considerably delay the release.

Are your sure that the implementation you mentioned has the same measurements as SBFspot and on top everything out of the Energy Meter? At a first glimpse I did not see any information concerning the data that this implementation produces.

Edit: It seems as if this implementation you mentioned is not usable for me at the moment. I would have to change a lot in the code as it requires InfluxDB which I do not have and a lot of code editing because there is no config file. When trying the compile I always get an error that complains about missing /home/admin/Desktop/influxdb-cxx/build/lib/libInfluxDB.so. As I would run it on a linux server, there is no Desktop directory, and I would have to compile the influxdb-cxx although I do not need it. So all in all this implementation is not suitable fpr my use case.

[mincequi]

@rucksman
Ok, i will implement the MQTT export using MsgPack as a first shot.

The library seems to do what we want. I tried it and at least i can successfully read SMA Energy Meter data with an interval of 1 second.
I already did an initial integration of that library into SBFspot and removed the dependency to InfluxDB.
Checkout my fork of this lib for now: https://github.com/mincequi/sma-emeter-and-inverter-to-influxdb-cpp.

However, i will soon be ready with my integration in a few days.

[mincequi]

@rucksman et al.
MQTT export of energy meter data is working. I only export the totalAc Power, as of now.
Please checkout my fork. Installation instructions in short:

sudo apt install libmosquittopp-dev libmsgpack-dev
git clone --recurse-submodules https://github.com/mincequi/SBFspot.git
cd SBFspot/SBFspot && mkdir build && cd build
cmake ..
make

Please check your SBFspot.cfg and set MQTT_ItemFormat to MSGPACK.
The data will be exported to sbfspot_/live
The format is a MsgPack serialized key-value map. I hope, you can interpret this.
Please let me know, if there are issues.

[rucksman]

@mincequi Unfortunately it did not work. First I had to compile cmake which took hours on my raspberry. I installed libmosquittopp-dev and libmsgpack-dev and cloned the repo you mentioned. When doing
cmake ..
I get

-- Checking for module 'msgpack'
--   No package 'msgpack' found
-- Checking for module 'libmosquittopp'
--   No package 'libmosquittopp' found
CMake Warning at thirdparty/sma-emeter-and-inverter-to-influxdb-cpp/CMakeLists.txt:7 (find_package):
  By not providing "FindInfluxDB.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "InfluxDB",
  but CMake did not find one.

  Could not find a package configuration file provided by "InfluxDB" with any
  of the following names:

    InfluxDBConfig.cmake
    influxdb-config.cmake

  Add the installation prefix of "InfluxDB" to CMAKE_PREFIX_PATH or set
  "InfluxDB_DIR" to a directory containing one of the above files.  If
  "InfluxDB" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring done
-- Generating done
-- Build files have been written to: /home/pi/mincequi/SBFspot/SBFspot/build

Trying to ignore the warnings and doing
make
gets me this:

[  1%] Building CXX object thirdparty/sma-emeter-and-inverter-to-influxdb-cpp/CMakeFiles/emeter-grabber.dir/src/DataProcessor.cpp.o
In file included from /home/pi/mincequi/SBFspot/SBFspot/thirdparty/sma-emeter-and-inverter-to-influxdb-cpp/src/DataProcessor.cpp:2:0:
/home/pi/mincequi/SBFspot/SBFspot/thirdparty/sma-emeter-and-inverter-to-influxdb-cpp/include/DataProcessor.hpp:6:84: fatal error: string_view: Datei oder Verzeichnis nicht gefunden
 #include <string_view>          // for std::string_view (forgotten in <InfluxDB.h>)
                                                                                    ^
compilation terminated.
thirdparty/sma-emeter-and-inverter-to-influxdb-cpp/CMakeFiles/emeter-grabber.dir/build.make:75: recipe for target 'thirdparty/sma-emeter-and-inverter-to-influxdb-cpp/CMakeFiles/emeter-grabber.dir/src/DataProcessor.cpp.o' failed
make[2]: *** [thirdparty/sma-emeter-and-inverter-to-influxdb-cpp/CMakeFiles/emeter-grabber.dir/src/DataProcessor.cpp.o] Error 1
CMakeFiles/Makefile2:197: recipe for target 'thirdparty/sma-emeter-and-inverter-to-influxdb-cpp/CMakeFiles/emeter-grabber.dir/all' failed
make[1]: *** [thirdparty/sma-emeter-and-inverter-to-influxdb-cpp/CMakeFiles/emeter-grabber.dir/all] Error 2
Makefile:135: recipe for target 'all' failed
make: *** [all] Error 2

Would you prefer to continue this conversation in your repository as it is not really related to this one?

[mincequi]

@rucksman
Hmm, compiling cmake by yourself does not seem to be the right approach. You can get cmake in the official apt repository of raspbian. That's probably also the reason, why cmake does not find the installation of msgpack and mosquittopp.

Yes, we can continue that discussion within my fork.

[rucksman]

@mincequi OK, I will continue there.

[rucksman]

@mincequi Sorry, no way to continue there... no "Issues" tab.

Had to compile cmake because your CMakeLists.txt states
cmake_minimum_required(VERSION 3.10)
In the raspbian repo only version 3.6.2 is available (I am on jessie).

Probably it is a naming problem? Because looking at my installed packages I see

pi@raspberrypi:~/mincequi/SBFspot/SBFspot/build $ dpkg -l | grep 'msgpack'
ii  libmsgpack-dev                         0.5.7-3                          armhf        binary-based efficient object serialization library
ii  libmsgpack3                            0.5.7-3                          armhf        binary-based efficient object serialization library
ii  libmsgpackc2                           0.5.7-3                          armhf        binary-based efficient object serialization library
pi@raspberrypi:~/mincequi/SBFspot/SBFspot/build $ dpkg -l | grep 'libmosquittopp'
ii  libmosquittopp-dev                     1.3.4-2+deb8u4                   all          MQTT version 3.1 client C++ library, development files
ii  libmosquittopp1                        1.3.4-2+deb8u4                   armhf        MQTT version 3.1 client C++ library

So there is no package "msgpack" (but libmsgpack3) and also no package "libmosquittopp" (but libmosquittopp1).

[rucksman]

@mincequi OK, moved now.

[hevilp]

Do we have something new here? Is it possible to get the consumption?

[wimleers]

Echoing this question. Between https://github.com/RalfOGit/sma-emeter-and-inverter-to-influxdb-cpp and the https://github.com/mincequi/SBFspot fork, it seems possible, but based on the conversation here and how few people succeeded in using it, it seems extraordinarily finnicky?

EDIT: based on #420, it looks like consumption data would be recorded by SBFspot today if your inverter has been configured to use the energy meter (which also means SMA's Home Manager … I think?) as a "data source". Where/how would one do that in the SMA inverter configuration UI? I cannot find any such info in the manuals.

P.S.: Related feature request in sibling project: codyc1515/homebridge-sma-inverter#41

[gfdsa]

docker run --net=host -v /mnt/sbf-tri.cfg:/usr/bin/sbfspot.3/SBFspot.cfg --rm --name sbfspot1 sbfspot /usr/bin/sbfspot.3/SBFspot -nosql -nocsv -v5 | grep Grid
Grid Power Out : 0W
Grid Power In  : 159W
GridRelay Status:      Closed
00464B01 28 Wed Nov  9 07:49:47 2022 'Grid voltage phase L1 against L2' 37769 37769 37769 37769
00464C01 28 Wed Nov  9 07:49:47 2022 'Grid voltage phase L2 against L3' 37841 37841 37841 37841
00464D01 28 Wed Nov  9 07:49:47 2022 'Grid voltage phase L3 against L1' 37758 37758 37758 37758
Grid Freq. : 50.00Hz

This is from inverter

[wimleers]

P.S.: Related feature request in sibling project: codyc1515/homebridge-sma-inverter#41

Update: since then I've created https://github.com/wimleers/homebridge-sma-home-manager 😊

[raimund-hdr]

EMETER Protokoll.pdf
This is an official SMA Dokument describing the energy-meter protocol. Maybe later some tags have been added...

[raimund-hdr]

em.c.zip
Some sample code...