Discrepancies between app and integration totals about ha-myenergi

[lozzd]

The pattern of the graph looks pretty normal by the way, so it's just like there's some multiplier on the actual value that's distorting the numbers - this is "generated today" - the app shows the total ended yesterday at 13.1kWh, versus less than 10kWh from the integration.

Since the "today" numbers are used in the HA Energy dashboard, this does affect price calculation etc of grid usage.

[G6EJD]

I’ve just checked mine, the alignment between systems is good:
SolarEdge Generation: 11.06kWh
MyEnergi Home Generated Today: 11.52kWh
Shelly Import: 11.44kWh
GivEnergy: 10.93kWh

So all near perfect alignment given that a CT has an accuracy of +/-3%

Is your system Single Phase?
Are you sure the Grid CT is on the incoming Grid feed and not a spur?

[lozzd]

Yes, the CT is correctly located.
I'm more worried that the data from the same source (myEnergi) is totally different between their app and this integration.
Maybe it's a question for the underlying python library?

[G6EJD]

When Johan created the integration I ran the Python query for a whole day and got the same result, so I think it’s ok.

You can run the script from the HA command line, the results are in Joules-sec so for a day the result needs to be divided by 60*60 to convert to Wh then 1000 for kWh.

Your Smart meter is the reference and your readings are at 30-50% higher, so something strange is going on.

Most MyEnergi users get good correlation to about 3%, so I think that confirms their system is good.

I s just checked mine again:
SolarEdge 13.1kWh
MyEnergi 13.1kWh

How does the MyEnergi App totals compare with the Integration?

[lozzd]

Sorry to clarify the numbers in my first post are myEnergi app versus the integration... so totally different. Disregard the single smart meter reading for now...

[G6EJD]

OK, so do you think your python scripts are up-to-date?

[lozzd]

I believe so. The const.py here does stil say 0.0.18 but it's running 0.0.20.

When Johan created the integration I ran the Python query for a whole day and got the same result, so I think it’s ok.

Can you elaborate how to do that so I can verify on the CLI?

As of 16:39, the myenergi app reports generation of 17.1kWh and the integration reports 12.25kWh!

[G6EJD]

See here: https://github.com/cjne/pymyenergi

noting the ratio is not constant between the two readings, it’s getting close to 1:1

[lozzd]

Thanks!

bash-5.1# myenergi --password <redacted> --username <redacted> overview | grep generated
Energy generated : 13.55kWh

Unsurprisingly, that matches what HA says. but the myenergi app says 17.7kWh!

So I assume this issue is for cjne/pymyenergi then?

[lozzd]

I'll have to take a meter reading on my generation meter tonight and compare to tomorrow night to see what's actually true here. Now I'm very confused..

[G6EJD]

Do you see a full response from 00:00, the API returns from 00:00 to the current time.

The question is, how come mine is perfectly aligned between GA, App and SolarEdge results?

[lozzd]

How can you tell?
Certainly the HA graphs all increment from midnight (UK time) so I should expect so.

I was starting to think maybe an app bug, but if yours align... hmm.

Just checked the clock on the zappi too just in case... all good.

[lozzd]

Results of today (21st, mostly cloudy)
Certified generation meter: 12.83kWh
Myenergi app: 12.9kWh (rounds to one decimal place)
Home Assistant/pymyenergi: 9.65kWh

Well. at least the CTs are working as expected.

That leaves us looking at pymyenergi and wondering why it's so different to the official app....

[G6EJD]

I definitely think your Python code is out of date, my results:
SolarEdge 14.0 kWh
MyEnergi App: 14.1 kWh
HA: 14.1 kWh
Shelly: 14.0 kWh

Try removing the Integration, then delete the Python folder and reinstall.

[lozzd]

I'll do that tomorrow.
I've just noticed the value in HA is now 10.02kWh... so increased 0.37kWh despite the "now" (obviously) being flat for hours since the sun went down...

[lozzd]

Removed, HACS deleted the directory, restarted HA, re-installed repo from HACS, re-added the integration - showing 1.18kWh in HA/python, app is showing 2kWh.

So there is still a difference between pymyenergi and the app

[G6EJD]

All I can observe is there are probably 000’s using this integration and if there was an endemic fault then it would have emerged by now and others would be reporting the same.

The question is what is different with your environment?

I’m using the entity: myenergi_yourname_generated_today
My is actually
myenergi_home_generated_today

Have you removed old entities from HA it tends to retain everything, my system has 43 entities and 4 devices for Myenergi

I’ve just checked my 4 systems (SolarEdge, myenergi App and HA and a Shelly monitor) they are all giving identical readings.

[lozzd]

All I can observe is there are probably 000’s using this integration and if there was an endemic fault then it would have emerged by now and others would be reporting the same.

Yes, I tend to agree and if I could figure out what was happening I wouldn't have opened this!
I do wonder if it's not affecting more people but they haven't noticed because the numbers seem very feasible - they're off 10-20% after all.

The biggest giveaway should have been the fact the HA number continues to increment after the sun set, but I hadn't noticed that until now.

I haven't bothered touching the entities as we now know this is not a HA problem - pymyenergi shows it without HA's involvement (yes the generated number from that output matches exactly the entity myenergi_<home>_generated_today)

So given both pymyenergi and the Myenergi app use the same "API", I'm struggling to figure out what's different here.

A snapshot of today so far:

bash-5.1# myenergi --password <redacted> --username <redacted> zappi energy
generated: 3.03kWh

App:

HA:

[lozzd]

Can you do a debug run -d on your end to see if it matches this? It's a shame debug mode doesn't output raw data, I might put a PR in to add that so we can inspect it...

bash-5.1# myenergi -d --password <redacted> --username <redacted> zappi energy
DEBUG:pymyenergi.client:Refreshing data for all myenergi devices
DEBUG:pymyenergi.connection:Get Myenergi base url from director
DEBUG:httpx._client:HTTP Request: GET https://director.myenergi.net/cgi-jstatus-E "HTTP/1.1 401 Unauthorized"
DEBUG:httpx._client:HTTP Request: GET https://director.myenergi.net/cgi-jstatus-E "HTTP/1.1 200 OK"
INFO:pymyenergi.connection:Updated myenergi active server to https://s18.myenergi.net
DEBUG:pymyenergi.connection:GET /cgi-get-app-key- https://s18.myenergi.net/cgi-get-app-key-
DEBUG:httpx._client:HTTP Request: GET https://s18.myenergi.net/cgi-get-app-key- "HTTP/1.1 401 Unauthorized"
DEBUG:httpx._client:HTTP Request: GET https://s18.myenergi.net/cgi-get-app-key- "HTTP/1.1 200 OK"
DEBUG:pymyenergi.connection:GET status 200
DEBUG:pymyenergi.connection:GET /cgi-jstatus-* https://s18.myenergi.net/cgi-jstatus-*
DEBUG:httpx._client:HTTP Request: GET https://s18.myenergi.net/cgi-jstatus-* "HTTP/1.1 401 Unauthorized"
DEBUG:httpx._client:HTTP Request: GET https://s18.myenergi.net/cgi-jstatus-* "HTTP/1.1 200 OK"
DEBUG:pymyenergi.connection:GET status 200
DEBUG:pymyenergi.client:Adding zappi zappi-<redacted>
DEBUG:pymyenergi.client:Adding harvi harvi-<redacted>
DEBUG:pymyenergi.base_device:Fetching hour history data for zappi
DEBUG:pymyenergi.connection:GET /cgi-jdayhour-Z<zappi SN>-2022-8-23-0-24 https://s18.myenergi.net/cgi-jdayhour-Z<zappi SN>-2022-8-23-0-24
DEBUG:httpx._client:HTTP Request: GET https://s18.myenergi.net/cgi-jdayhour-Z<zappi SN>-2022-8-23-0-24 "HTTP/1.1 401 Unauthorized"
DEBUG:httpx._client:HTTP Request: GET https://s18.myenergi.net/cgi-jdayhour-Z<zappi SN>-2022-8-23-0-24 "HTTP/1.1 200 OK"
DEBUG:pymyenergi.connection:GET status 200
generated: 2.22kWh

[lozzd]

I finally got around to play some more. Does this output look sane?

[{'dom': 25, 'dow': 'Thu', 'gen': 3240, 'imp': 2340840, 'mon': 8, 'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'gen': 3180,
  'hr': 1,
  'imp': 2189400,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'gen': 2700,
  'hr': 2,
  'imp': 2206620,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'exp': 900,
  'gen': 2520,
  'hr': 3,
  'imp': 1835100,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'gen': 2700,
  'hr': 4,
  'imp': 1881660,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'gen': 2460,
  'hr': 5,
  'imp': 1637040,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'gen': 3120,
  'gep': 1080,
  'hr': 6,
  'imp': 1529700,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'gen': 2160,
  'gep': 113760,
  'hr': 7,
  'imp': 2882760,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'gep': 405780,
  'hr': 8,
  'imp': 1629600,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'gep': 717540,
  'hr': 9,
  'imp': 1552860,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'exp': 12960,
  'gep': 1543740,
  'hr': 10,
  'imp': 1620900,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'exp': 1135080,
  'gen': 1260,
  'gep': 3363960,
  'hr': 11,
  'imp': 144900,
  'mon': 8,
  'yr': 2022},
 {'dom': 25,
  'dow': 'Thu',
  'exp': 34560,
  'gep': 589080,
  'hr': 12,
  'imp': 938640,
  'mon': 8,
  'yr': 2022}]

I assume this is in UTC which is why I only have entries up to hr 12, is the first row that misses hr 0? so 12-1am?

I did also check and there's no difference between if I switch between the history_energy_minutes and history_energy_hours calls.

[lozzd]

I'm losing my mind here.
I just checked "imported" on the myenergi website for yesterday and it reports 24.31kW (not even close) but their app reports 15.3kWh! Their own website and app dont' agree let alone HA (11.43kWh)

I think I'm going to go to myenergi support at this stage......

[sddgit]

I installed this integration just a couple of hours ago, and am seeing the same problem:

HA integration Generated Today: 38.53
Myenergi app Generation: 47.9
HA Energy component Solar Production: 46.5 (driven by Enphase Envoy feed)
Enphase web site 46.6

HA integration Imported Today: 15.53
Myenergi app Imported: 17.9
Myenergi web site: 17.43
HA Energy component Imported: 17.2
Enphase web site 16.8

I also see the great disparity between the app and web site for yesterday:
Myenergi app imported yesterday: 50.8
Myenergi web site imported yesterday: 68.3
HA Energy component Imported: 50.1
Enphase web site 49.8

Do the “today” sensors come directly from the API, or are there calculations involved? I wonder if the definitions of today, yesterday etc are wrong because of some time zone problem between the API, app and web site.

[G6EJD]

All the data comes from the MyEnergi servers via the API.

I note on the Myenergi forum that some customers report inaccuracies during the day but these get corrected the next day, so if you check yesterday against a known reading there should be parity. The problem is therefore in the day.

All data is recorded in UTC.

I don’t see any difference in readings, suggesting the code here is OK.

[lozzd]

I'm glad I'm not on my own here @sddgit. I've not heard anything back from my support case with myenergi. I'll let you know when I do.

[sddgit]

@G6EJD Regardless of any intra-day discrepancies that might exist, the fact that, for example, today’s generation is wildly different between the integration and the app would indicate something’s amiss somewhere. They are using, presumably, the same API, so why would that be the case? Do you know if the “today” value comes directly from the API, or is it a calculation that is performed? Perhaps it comes back as some sort of value that must be corrected based on time zone? Clutching as straws here! As I said, “yesterday” values are different between the app and the web site, so that’s another problem, but outside the scope of the integration.

@lozzd What time zone are you in, out of interest?

[lozzd]

I'm in the UK so currently BST (UTC+1). Reading the python code everything is set to use UTC, which makes sense. i don't think it's timezone related because I can see the "generation today" increment at, say, 10pm when I've been at 0 watts generated (due to the sun being down...) for many hours, so 1 hour offset is enough to disprove that (I think)

Btw if you want to see the API examples they're in my output above. If you get CLI access to your HA instance you can run the myenergi command to see the numbers (which exactly match what's in HA). If you run with debug mode, you can see the URLs, which make sense to me - they're constructed of the serial of the zappi, the date (today), the hour to start (0, e.g. midnight) and the number of hours (24, makes sense).
I've tried understanding the JSON response but I don't get it. The numbers even for 1am-2am show generation.
The app doesn't though!
So maybe they switched the app to use a different API and some people's original API are broken based on their account? My account is only a couple of months old, I assume yours is also new? (I only got my zappi/hub/harvi in July)

[sddgit]

Yes, my Zappi is only a couple of months old (a V2.1 one). Do those “today” numbers come through as a number that’s been calculated server side, do you know?

[lozzd]

Yes, although there is a bit of calculation to convert it to kWh.
See post #213 (comment) and you'll see the JSON response, it's hourly so the gen numbers are added up and then turned in to kWh.

So the actual totalling of power generated by hour is done on the server side, yes.

[sddgit]

Right. But the integration adds up all the hours and converts to kWh?

[lozzd]

Yes. Or rather technically the pymyenergi library does, and the integration uses that library.

[sddgit]

Understand. Let’s hope you get a response from myenergi regarding this. I suppose it’s feasible there is something going astray in the Python library that doesn’t affect everyone somehow. All very strange.

[lozzd]

Does anyone know the difference between gep and gen in the API response? Sometimes it has one or the other and sometimes both

[yacoob]

I think I'm seeing similar problem here. Much simpler instalation, zappi with internal hub, no solar panels. The charger is in use right now, and a charging session has started roughly a minute past midnight. With 01:20AM on the clock, both myenergi app and myenergi website show that the charge is proceeding at 7.7kW, with 10.4kWh already imported from the grid. sensor.myenergi_hub_grid_import_today in HA shows 4.41kWh. sensor.myenergi_hub_power_import looks sensible, hovering at 8kW, which is correct, entailing 7.7kW for the charger and ~300 kW for the house. sensor.myenergi_zappi_energy_used_today is also at 4.41kWH.

Happy to debug more here, if you let me know what you'd like me to check :)

[lozzd]

Welcome to the club. myenergi has yet to respond to my support ticket - still open. if it goes on much longer I'll ping "Dr Chris", their CTO who is very active on their forum. It sounds like at least a few people have got totally random unreliable numbers across their platform which is concerning.

I was starting to wonder the other day if my installer didn't do something quite right. Maybe others can chime in if they have a similar setup?
I have the Zappi, installed on a circuit not via the consumer unit. I then have a Harvi with 2x CT's - one on the incoming mains (before both Zappi and consumer unit, right after meter) and one on the generation. The Zappi is the "master" and configured to read the CT's from the Harvi.

It seems that whenever the Zappi reboots (e.g. I did a firmware update the other day, but just a reboot will do) the Harvi comes unpaired with the Hub (light goes red - and if I reboot the Hub, it goes out completely!). During the time the Harvi light is red or off, the app still shows the live power stats just fine - I guess the data flow is going Harvi -> Zappi -> Hub - but the API we're using stops updating anything under the "Harvi" section (timestamp halts too), so no live metering is done at all at all. This has led me to believe that the app is using a totally different API than the cgi- endpoint used by the myenergi python code. I have to then re-pair the Harvi with the hub, and when I do the numbers start updating in the API again and thus in Home Assistant. Very odd.

I haven't even broched the subject of why there's a (much much smaller) discrepancy between my smart meter IHD and the myenergi app. Honestly I don't trust the Smart Meter at all - it's one of the Landis & Gyr SMETS1 models that's only just been "re-activated" to be smart, and the IHD only updates the live readings every 2-3 minutes. I can turn the kettle on, watch the myenergi app instantly show it, and the IHD for the smart meter can be blissfully unaware still showing 300 watts and the kettle finishes before it updates... Useless!

[G6EJD]

The Harvi only sends data to the Master which sends data via the Hub, the Master is the hub not the Hub.
My data is still in prefect alignment as is most users.
The Smart meter records all usage and updates the display and providers servers periodically, but they are accurate.

[yacoob]

Two questions:

• for the users that do see discrepancies; do we have a confirmation whether it's the data coming out of myenergi API (that ha-myenergi uses) that is invalid, or is that data correct and it gets miscounted within the lib or HA integration code?
• I wonder what is the hardware that people in this thread are using; maybe it's related to the v2 of zappi having an integrated hub?

[lozzd]

I will try and figure out item 1 today. It's not a complicated piece of maths from the look of the code, so I am pretty sure it's the API endpoint. As I say, I don't think the app is using that endpoint any more. Need to find someone who wants to decompile the Android app and poke around maybe.

I have a Zappi that was manufactured earlier this year, it's a v2 (as indicated by the firmware it runs) but I don't think it has a hub. I have a separate hub (and the harvi)

[DReffects]

Hey Guys, I've just installed Home Assisant and this awesome integration today, so everything is shiny and new.

I am noticing the very same deveations:

For today:

App:
Generated total: 24.7 kWh
Green energy used: 11.8 kWh
Green energy exported: 12.7 kWh
Imported energy: 9.1 kWh

Total energy used: 20.7 kWh
Car charged: 2.72kWh
Usage of our home:: 18.3 kWh

These numbers also add up with the readings at myaccount.myenergi.com

Home Assistant integration reports:

sensor.myenergi_xxx_generated_today: 20.63 kWh (still rising despite being night atm)
sensor.myenergi_xxx_green_energy_today: 2.3 kWh (is this the amount of green energy to my car? I have set a value of 80% green energy with eco+ mode)
sensor.myenergi_xxx_3_grid_export_today 10,79kWh (still rising despite being dark at the moment)
sensor.myenergi_xxx_3_grid_import_today: 7.73 kWh

Any idea what is going on here?

[lozzd]

That sounds like the exact problem yep. It's the generation raising past sunset that's the giveaway.
It does seem like they fixed the website to match the app.
I haven't heard back from support, despite also PMing Dr Chris.
Getting a bit frustrating now. I've got an Eddi coming next week and I'm somewhat worrying if I got in to the wrong ecosystem

[DReffects]

I am rather new to the whole myenergi system, Dr Chris was very helpful in the past since my installer sold me a pre-used hub that had another zappi linked.

I've opened up a topic on their forums: https://myenergi.info/reported-api-values-are-extremely-different-from-a-t6655.html

Perhaps you could chime in there.

I absolutely love the "true solar surplus charging" with the zappi.

[lozzd]

I found it quite interesting that my stats updated in HA totally fine the entire time the app was down this morning.

Dr Chris replied to my support ticket and suggested that it could be data loss, since there is some "data infill" that happens on the app to mitigate loss of transmission. However, I pointed out that I have no yellow (!) and asked for clarification on whether that happens server side or client side - since if it's on server side the numbers should still match, right?

[G6EJD]

How does their system know to flag a yellow if no data was ever sent, are they counting samples expected out of 3600 or 60 or whatever and if the answer doesn’t match flag data not received? Who knows. I have never had a different result between API and App and I’m sure it’s a poor radio link performance for those that have the issue. My radio link is rock solid.

[lozzd]

How does their system know to flag a yellow if no data was ever sent, are they counting samples expected out of 3600 or 60 or whatever and if the answer doesn’t match flag data not received?

Yes.

My radio link is rock solid.

My equipment is 2 meters apart with no obstructions. So pretty sure mine is too.

I pointed this out and haven't had a response in a few days. Will update when I hear anything back

[sddgit]

How does their system know to flag a yellow if no data was ever sent, are they counting samples expected out of 3600 or 60 or whatever and if the answer doesn’t match flag data not received? Who knows. I have never had a different result between API and App and I’m sure it’s a poor radio link performance for those that have the issue. My radio link is rock solid.

I have a Zappi with built-in hub, so no radio link involved, and no warning triangles. And if it were a radio link problem, wouldn’t it affect api, app and web about equally?

[lozzd]

By the way - after my Eddi installation yesterday I will note that my "green energy today" number is totally matching the app. So at least one metric is correct between the app and API/HA!

[lozzd]

First full day of having the Eddi paired - still using the same Hub, Zappi is still the master, Harvi still has both CTs attached........
And now my numbers are correct???

App Gen / HA Gen: 12.8kWh / 12.75kWh
App Import / HA import: 8.4kWh / 8.21kWh
App Export / HA export: 3.6kWh / 3.58kWh

So... has adding the Eddi fixed an issue in pymyenergi, or has it fixed something on the server? 🤔

[sddgit]

Too much to ask to remove/replace the Eddi to see if anything changes?

[lozzd]

I think that'll be quite hard to do. It just automatically adds to your account as soon as the Hub sees it.

So two things happened on the morning I had the Eddi installed. They also had that long maintenance.

When I looked at my data in the app a few hours later, all of my history for my grid and generation, but not the house had been wiped and started fresh. The forum could not tell me if that was due to the Eddi installation or the maintenance.

If everyone else's is still wrong/out of sync and they have their historical data, then it was the Eddi and not the maintenance...

[sddgit]

I have my history, no Eddi, and my “today” numbers don’t match between the app and HA.

[pannlov]

I have also the exact same problem. I can take my grid import yesterday as an example. MyEnergi app, MyEnergi webportal, my solar power meter and my Energy Power Company web portal all reports about 32 Kwh. myEnergi HA integrations reports 23 kwh. Sp for sure the HA integration is incorrect. It would be a really good integration if it works with correct values, but with that much discrepancies it is of no use. I hope this will ne fixed and keep up the good work!

[G6EJD]

Out of interest what do you use the grid import total for?

[pannlov]

I use it on the Energy Dashboard, to see how much I consumed and the cost for it, which is also incorrect then ofc.

[markruys]

I think this plugin has two issues:

• It doesn't take into account your local time zone. The myenergy API uses UTC, pymyenergi too, so it returns wrong data if you're not in UTC.
• pymyenergi uses hourly data from the API. The app/site uses minute resolution. These numbers don't match.

I've written a Python script to query the API, just to understand what's going on, this is what I get:

Minute resolution, date: 2022-10-14, exported:  0.6 kWh, imported: 23.6 kWh, diversion: 10.5, boosted:  0.0
Hour   resolution, date: 2022-10-14, exported:  0.6 kWh, imported: 23.2 kWh, diversion: 10.5, boosted:  0.0
Minute resolution, date: 2022-10-15, exported:  1.0 kWh, imported: 11.5 kWh, diversion:  0.0, boosted:  0.0
Hour   resolution, date: 2022-10-15, exported:  1.0 kWh, imported:  9.5 kWh, diversion:  0.0, boosted:  0.0
Minute resolution, date: 2022-10-16, exported: 13.6 kWh, imported:  8.2 kWh, diversion:  0.0, boosted:  0.0
Hour   resolution, date: 2022-10-16, exported: 13.6 kWh, imported:  7.2 kWh, diversion:  0.0, boosted:  0.0
Minute resolution, date: 2022-10-17, exported:  0.6 kWh, imported:  9.2 kWh, diversion:  0.0, boosted:  0.0
Hour   resolution, date: 2022-10-17, exported:  0.6 kWh, imported:  7.8 kWh, diversion:  0.0, boosted:  0.0

[markruys]

The top line is correct (that is, equals to what the app tells me):

[lozzd]

• pymyenergi uses hourly data from the API. The app/site uses minute resolution. These numbers don't match.

Wow. That's pretty huge and almost certainly the issue then? The script you wrote is fairly damning that there's an issue there.
However, I wonder how that explains how mine have matched perfectly 1:1 since getting an Eddi.
Can you gist a copy of the script and I'll run it against my data to double check?

I suspect Dr Chris at myenergi would be very interested to see that, since it points squarly at their own API, so they have something to look in to.

[G6EJD]

The strange aspect of this issue, is some users like me, have never seen a discrepancy, so it could be the server end is calculating differently depending on equipment age or fit, perhaps only later units are showing the discrepancy. Probably a mismatch of the server scripts.

[markruys]

A piece of code I wrote to do some testing (zappi only, easy to adapt): https://goonlinetools.com/snapshot/code/#u5c88c06peqek62ena1kv7

The minute sample rate is exactly what I get in the myenergy app. Code (and app) takes into account time zone. Difference between hourly and minute sample rate is based on gaps in the minute data. This script (and the app) interpolated missing data, the hourly API download doesn't.

[G6EJD]

I also observe that the difference is not deterministic or constant between other users, and if a software/script error, I’d expect a constant result. One constant though is it always under reads/reports. I still maintain it’s a poor radio link between device and Master and it’s dropping packets and thereby is reporting correct data assuming it actually arrived!

[sddgit]

I have a V2.1 Zappi, built in hub, no radio link involved, and still get discrepancies. If it’s true the plug-in doesn’t take into account time zone, could it be that the further you are from UTC the worse it gets?

[G6EJD]

MyEnergi store all their data in UTC the timezone has to be handled locally.

[G6EJD]

This is what they say:
‘The API currently reports the time in UTC. That's simple but means that the graphs on the app are out by one hour in the UK (BST) and further adrift elsewhere. It also has an impact for anyone using the API.
To display the data correctly on the app we need to know either the timezone + UTC, or simply the time in Local Time. The Timezone will come from the settings in the zappi / Eddi’
And:
Anyone using the API needs to recalculate the time to display (and they need access to the timezone set in the eddi/zappi, NOT the timezone reported locally for example by a phone.

[lozzd]

Minute resolution, date: 2022-10-19, exported:  2.0 kWh, imported: 11.0 kWh, diversion:  0.0, boosted:  0.0
Hour   resolution, date: 2022-10-19, exported:  1.5 kWh, imported:  7.8 kWh, diversion:  0.0, boosted:  0.0

App displays 11kWh import, so minute is correct.
But ha-myenergi is also showing 11kWh (well, 10.98 to be precise)

I do think it's time to approach myenergi again on their data aggregation tactics when it comes to minutely -> hourly rollups.

When I was talking to Dr Chris, he specifically said that any data infill that happened to correct missing packets is done on the client. So missing packets or not, I'm not sure why you'd expect those two API calls to return different data when summed.

(also to be answered - why mine fixed itself? Did it start using hourly and flip to using minutely data? When I installed the Eddi? What?)

[lozzd]

It supports either: https://github.com/CJNE/pymyenergi/blob/2e68ce0efc1267761cef89ecdaf1c1b08814b656/pymyenergi/base_device.py#L134-L137

Need to look at what would adjust that resolution variable

[markruys]

I do think the myenergy hourly cgi-jdayhour API call has a bug. I'm fine with interpolating the minute data at the callers site, but when the caller gets hourly data, the interpolation must be done by the API.

Indeed this integration already has means to switch to minutes. But that's only part of the fix. It needs to interpolate missing data too. Also ha-myenergy should take time zone info into account. So I think we face three different issues/fixes here.

I do consider to write a PR, but I'm short on time, so let's see when I can find some spare time...

[lozzd]

I'm not really sure what we can glean from this but to add to "adding an Eddi fixed my stats" confusion - this evening my Eddi crashed (the arrows were moving on the display but not responsive to any button press) and during that time period, the "consumption" numbers stopped working in the official app and in Home Assistant.

For clarity:

• The ME app had yellow exclamation points on every element, despite only the Eddi being offline
• The ME graphs are all empty for the time period the Eddi was crashed
• The HA live graphs are all absolutely fine, so instantaneous readings were operating correctly, but the totals graphs stopped updating
• The Zappi was operating fine (and was, and is still is the master)
• The Hub showed the Eddi only offline (red LED)
• Everything came back instantly the second I power cycled the Eddi

So despite neither the master device nor the Hub in my setup, the Eddi is playing some part in my total consumption calculations. It must take priority somehow.

[POHakansson]

Hi @lozzd and @markruys

I'm not using this repo, but I have followed the discussions since I'm trying to use the API to calculate my energy costs by matching hourly consumption with the Nordpool market prices. I have a two year old three phase Zappi V2, two Harvis with 3 clamps each. The app usually matches readings from my Fronius inverter and the energy company's numbers for bought/sold kWh.

For me too, the generation figures are way off compared to the app and the myenergi dashboard. If I pull the minute samples for my example day, I get 1438 samples so almost perfect. I don't care about my time offset in this case, since the generation is all during daytime.

I opened the debug panel in Chromium and had a look at what's happening in the myenergi dashboard. I could grab the data the webpage pulled for my example date, and it was identical to what I get through the API. Still their calculations for app and dashboard show

Imported 9,6 kWh, Exported 36,4, Consumed 11,1 -> Generated = 47,5

Calcluated from the minute samples from the API I get

Imported 9,6 kWh, Exported 36,4, Generated 42,3

I don't get a gen field in my data.

I just can't understand how they come to their figures. Looking at the source code for the dashboard, there are various comments about in which order to do things to match the app, but I'm not fluent in .ts/.js files so I don't understand exactly what they are doing. The most interesting file is probably Mapper.ts if someone wants to have a look. I tried setting a breakpoint to inspect some data, but I suppose the files are hosted in the cloud somewhere and I'm not allowed to debug them :-)

BR,
P-O

[ChuckMountain]

Did anything come of this, just installed the HomeAssistant integration via HACS and noticed that my MyEnergi import was well down in HA compared to the actual App\Website.

Will experiment a bit when I get more time