Create new firmware branch with 'edge' processing that significantly reduces telemetry about openeew-sensor

[johnwalicki]

Vaclav, RUWO and John met on 12/09/2020 to discuss the STA/LTA algorithm implementation on how to detect quakes.
Here are some minutes of that discussion:
Attendees: @vaclav Kuna @RUWO @john Walicki

• Vaclav reviewed his Python STA/LTA algorithm implementations (there is a loop technique and a Matrix math technique)
• The board needs to store 10 seconds of data [x,y,z] in some array.
• Run the algorithm on new readings (32 samples from the last second) and the prior 10 seconds. So its really 11 seconds of data.
• Better to run the algorithm on each of the 32 new values against the prior array of values.
• John suggested that we don't think of the prior 10 seconds of data in terms of seconds, just a queue of 320 values. There are a variety of example First In First Out queue implementations on StackOverflow.
• If the result is greater than a threshold ( 3 is a good compromise but might be different for different sensors), send the prior 10 seconds to the cloud and start sending real time accelerometer data for the next 5 minutes
• We discussed being able to adjust the Threshold via a MQTT topic on a per sensor basis. If one sensor is located in a "noisy" environment and is throwing lots of false positives, we could tell it to up its threshold via a message sent down from the cloud. Tabled as an advanced feature.
• Since John received a new board from Andy today, he could possibly send his old board to Belgium. The old board has a working ESP32 and ADXL355 but other components are flakey. Wout only needs the ESP32 and ADXL355.
• Split the project into several steps.

1. • Implement a FIFO queue that stores/manages the prior 320 readings (10 seconds)
2. • Implement a function which sends this bigger block of data via MQTT to the cloud. (Current implement is sending 1 second at a time and would need to be modified with larger buffers)
3. • Iterate on the STA/LTA algorithm
4. • When the 10 second block of data arrives in the cloud, store it somewhere, start the alarm process?

[johnwalicki]

Firmware version 1.3.0 branch is open. I am working on the STA/LTA algorithm in v1.3.0
https://github.com/openeew/openeew-sensor/tree/version130

[johnwalicki]

Sections of code required to implement this feature are landing in the version130 branch
6edafb8
abc14a9

[johnwalicki]

When sending 10 seconds of accelerometer data to the cloud, it should really include timestamps. Currently the two patches in this feature just store the prior 10 seconds of accelerometer X,Y,Z data in a FIFO queue of 320 readings ( 32 readings/second x 10 seconds ). Time is relative to "now", there's no absolute timestamp. When we transmit and subsequently store the data in the cloud database for later analysis, there's no reference point. I need to add timestamps to the FIFO queue.

Note that the 32 readings are sampled once per second so the timestamp accuracy is going to be slightly off. Since the order of samples is preserved, the analysis could spread the samples "across" that trailing second.

Note its a trailing second. the program will record the timestamp when it reads the ADXL355 buffer (which contained the prior second's worth of data) That timestamp will be the prior 32 samples.

[andygrillo]

To be discussed with Vacalv, but potentially we can omit the device timestamp and add a timestamp when its been ingested in the cloud. As device>cloud latency is typically less than 1/2s, it should be ok.

…

On Wed, 23 Dec 2020 at 15:34, John Walicki ***@***.***> wrote: When sending 10 seconds of accelerometer data to the cloud, it should really include timestamps. Currently the two patches in this feature just store the prior 10 seconds of accelerometer X,Y,Z data in a FIFO queue of 320 readings ( 32 readings/second x 10 seconds ). Time is relative to "now", there's no absolute timestamp. When we transmit and subsequently store the data in the cloud database for later analysis, there's no reference point. I need to add timestamps to the FIFO queue. Note that the 32 readings are sampled once per second so the timestamp accuracy is going to be slightly off. Since the order of samples is preserved, the analysis could spread the samples "across" that trailing second. Note its a trailing second. the program will record the timestamp when it reads the ADXL355 buffer (which contained the prior second's worth of data) That timestamp will be the prior 32 samples. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABP5GHNTNRVKFHCZ44WU4CTSWJO5PANCNFSM4P2F44HA> .

-- Andres Meira *m* [phone] *skype *andresmeira

[johnwalicki]

Firmware v1.3.0 has been published
This Issue has been closed via #61