Force update option for binary sensor about feature-requests

[OttoWinter]

I never understood why people would use the force_update option. It just increases the DB size considerably and floods the HA state machine - and most graph frontends (like grafana) have options to not have to rely on force update.

[HugoVonHory]

I understand that it is standard behaviour and it saves data and processing. Can anyone advise, how to solve it? If there are no data in time range, grafana displays nothing (or graph only on part of time range, where are values). Still can't find any option, or procedure for correct display, force_update is only thing I currently see to make diplay consistent.

[casperklein]

@OttoWinter When not every event is logged to a database (because the value haven't changed), how do you distinguish later, wether for example the temperature was the same or the device just failed and stopped sending sensor data?

[brandond]

If you're querying a DB, you could look at the timestamp (age) of the most recent reading to decide whether or not to use it. If you're using HA automations, you could check the device status to see if it's online or unavailable. Just a couple suggestions; I'm sure you can come up with more solutions that are specific to your use case.

[OttoWinter]

If there are no data in time range, grafana displays nothing

There's a fill option just for that

wether for example the temperature was the same or the device just failed

The state of the sensor is unavailable when HA can't connect to it.

It's not that much work to add force_update to ESPHome, but I'm against it for several reasons (mostly because it's really bad for HA performance). And for all the reasons I've heard so far there was a simple solution.

If someone can convince me that force_update is required and there's no simple other way, I can add it.

[casperklein]

I fully understand your point of view and respect it. And as you said, for most use-cases there are alternative ways to get it done.

For me however it means, when migrating a device which previously used a component (MQTT) that supported force_update, to esphome native api, I have to adjust my automation (node-red) and it's getting more complex.

One last example, then I'll be quiet :-)

I get a notification, when my washing machine has finished. This is done by checking, if power usage is 3mins constant at a fixed low value (2 Watts).

(Power sensor updates every 60s.)

With force_update: Received 2W three times in a row? --> if yes --> notify

Without force_update: Received 2W? --> sleep 1min --> query HA if value is still 2W --> if yes --> sleep 1min --> query HA if value is still 2W --> if yes --> notify

[HugoVonHory]

There's a fill option just for that
It works only for future from last value in time frame (e.g. last value to now), but not for history to last value before time frame, or even if no value is there.
In general force update is optional, so it could be enabled only realy required and with appropriate update interval.

[dainman]

I asked this same question in discord about a month ago and I have been playing with settings in Grafana, however my graphs are not as consistant as the were when using MQTT and the force_update. Here are a few comments to your view points

1 - Fill previous only works if there is a measurement in the timeframe that you have selected. Example I have alot of my graphs that are set to 6 hour resolution and in my home some time it can go 7-8 hours without a change so then the sensor disappears unless I widen the time frame out to say 12 hours

2 - as for unavailable that only shows in home assistant. if you look in Grafana and again there is a measurement in the time frame it will fill previous as we told it to or if not it will show a blank

3 - I saw no difference in performance of my HA installation from using MQTT with force_update enabled to now using the API. I may not have a huge install but I have about 6 ESP32s with 2 temps and a humidity sensor each

It would be nice to have a force_update with a variable interval, however parity with MQTT force_update would be acceptable as well

[messier433]

Hi,

there is a simple way to override the force_update flag inside the python code:
e.g. For ESP home sensors, cou can add following lines to /srv/homeassistant/lib/python3.7/site-packages/homeassistant/components/esphome/sensor.py

class EsphomeSensor(EsphomeEntity):
    ...
    """Force update fix."""
    @property
    def force_update(self) -> bool:
        """Return True if state updates should be forced.
        If True, a state change will be triggered anytime the state property is
        updated, not just when the value changes.
        """
        return True

This forces updates for all ESP home sensors. So its an ugly hack but it does the job for now...

@OttoWinter: Grafanas / InfluxDB fill(previous) has issues when there is no previous data in the current time window. Also after grouping mean values, a 0 power value is not shown as zero anymore but as the average of 0 and the neighboaring samples. And in addition it would be nice to differentiate if a sensor is sending no data or the same data

[corneyl]

Another reason for allowing the force_update option is if you want to count totals. I have e.g. a pulse sensor measuring the rotations of a disk in a power meter. If for two periods of time this is the same the state will not be updated and if you then sum the total of the pulses (using the statistics sensor in HA), the amount of pulses in the second window are not counted because there was no state update.

[OttoWinter]

@corneyl So why not use the pulse_counter + integration sensor in esphome then?

[corneyl]

That would make the problem smaller, but it's still possible that two 5-minute windows have the same amount of pulses. Making the integration window longer would solve this, but then you lose data when updating the configuration/firmware.

[OttoWinter]

@corneyl That's if you do the integral on the HA side, but if you would use the integration sensor in esphome you would not have that issue.

[messier433]

@OttoWinter: Just a side note: the integration sensor in esphome is still not integrating correctly: esphome/issues#533 ;)

[haufes]

@OttoWinter I also would like to see force_update as an option for sensors. I am as well monitoring power usage (on my house) using a Pulse Counter and pushing it into influx. I've considered using fill(previous), but this issue with that, it isn't possible to distinguish between the metric not changing and no metric at all.

EG. When the power is out the graph would display the last metric and would show more consumption than actually happened.

Other idea that maybe options that fill some of the need and wouldn't fill the db

• only force if the measured value is over a unit, in a range or not a unit.
• if the esp is determined to be offline emit a value for the sensor (But this wouldn't help if HA was also offline)

PS: Love ESPHome, great job!