Timeout requesting chart for year or all about solectrus

[unclej84]

The new version 0.7.0 works great with the cache

[unclej84]

version 0.6.0 shows the same error in the console but not on the page as 0.6.1

[ledermann]

Thanks for this bug report. Yes, it appears to be a timeout when querying InfluxDB.

Currently, there is no setting to define an individual timeout, so the default of the InfluxDB client library is used, which is 10 seconds.

I can add a setting for this, but I'm uncertain if this will be a suitable solution. Waiting 10 seconds or more for a query result is far too long, IMHO. What setup do you use? On what machine does your InfluxDB run?

By the way, there is a difference whether the current year or the previous years are queried: Previous years are cached after the first hit, the current year is never cached. So, there can be just two slow queries: Current year and all data

In my setup (https://solectrus.ledermann.dev), querying "all data" needs 2.5 seconds - which IMHO is already too slow. The reason is, that currently there is no still data down-sampling, so my InfluxDB contains data for each 5 seconds of 2 years, so the database contains about 12Mio records. Data down-sampling is on my todo-list (#644).

About 0.6.0 vs. 0.6.1: The latter contains only updated dependencies, so I'm a bit confused why it behaves differently.

[unclej84]

The setup is simular to the in-house hosting config. It is running in a virtual machine (Ubuntu 22.04) with total of 5GB RAM and 4 cores. (influx has assigned 2.5GB RAM and Dashboard 400MB) All are running as docker container with bridge network on the same VM. SENEC_INTERVAL is set to 7 seconds.
The result was for the current year (set up was April 2022)

[ledermann]

Strange - if the query for a time range of just 7 months exceeds the timeout of 10 seconds, then there are other issues with the setup. Have you tried a reboot? Do you have installed a recent version of InfluxDB? Is the machine busy due to other processes?

I doubt that rising the timeout limit would solve the issue. The problem is not the timeout, the problem is why InfluxDB is so slow.

[unclej84]

Have managed to speed up the system a bit (removed docker limits for dashboard and influx) so that year and all are displaying again for now. The influx version was 2.4 and is now 2.5.1. Hope to keep it running when additional data is collected.

[ledermann]

BTW: The memory usage of my Solectrus containers is quite low, InfluxDB uses around 250 MB, the dashboard requires around 150 MB:

$ docker stats --no-stream | grep solectrus

CONTAINER ID   NAME                                                                          CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O         PIDS
d39e35b6ac08   solectrus_influxdb.1.nztmq42h49g7jhrn5imnc2n4p                                0.13%     223.8MiB / 15.26GiB   1.43%     972kB / 348kB     9.95MB / 39.3MB   18
4f2a5a118d59   solectrus_app.1.uomnrygwar80s21q7iu9hn6v4                                     0.01%     149.5MiB / 15.26GiB   0.96%     195kB / 361kB     90.1kB / 4.76MB   13
ca0bd1869347   solectrus_forecast-collector.1.4eqxu2ebq3hzhs9gp3kufzfb8                      0.00%     28.16MiB / 15.26GiB   0.18%     419kB / 183kB     2.18MB / 0B       1
a740a468a169   solectrus_db.1.lybaw8nl16w8qznf2ner7ulwq                                      0.00%     27.4MiB / 15.26GiB    0.18%     700kB / 1.39MB    7.82MB / 977MB    7
61449eaba26c   solectrus_fluxbackup.1.wh4t1jbt8jxec0pbifqkenv7q                              0.01%     3.852MiB / 15.26GiB   0.02%     2.24kB / 0B       5.16MB / 8.19kB   3
77d19104c88a   solectrus_redis.1.l62n7jgsjr77hcu5m1ng7ava4                                   0.21%     8.445MiB / 15.26GiB   0.05%     5.57MB / 6.72MB   434kB / 69.2MB    5

The memory usage of the InfluxDB container changes only slightly while performing queries:

The size of my InfluxDB database (containing 2 years of data for each 5 seconds) is about 150 MB:

$ du my-volumes/solectrus/influxdb/ -hs
156M	my-volumes/solectrus/influxdb/

I'm using InfluxDB 2.5.1, too.