openagriculturefoundation / openag-device-software Goto Github PK

flash latest image onto BBB and ./update.sh on EDU-B-006 using the v0.1.1 tag
go to http://:8000 and see this:

TypeError at /dashboard/
'NoneType' object is not subscriptable
Request Method:	GET
Request URL:	http://172.17.2.103:8000/dashboard/
Django Version:	1.11.4
Exception Type:	TypeError
Exception Value:	
'NoneType' object is not subscriptable
Exception Location:	/home/debian/openag-device-software/app/viewers.py in get_environment_summary, line 228

app.log shows this:

[05/Jul/2018 12:58:25.733] DEBUG app.viewers: Getting environment summary
[05/Jul/2018 12:58:26.072] INFO app.common: Unable to get sensor variable info, `spectrum_key` does not exist

This thread will be spawned by the device manager thread. It will check the local db tables for entries without the is_synced flag set, build payloads and publish them over MQTT so they end up in our big query database.

and add a 'capture' event
will be quite helpful to help debug lighting issues

@rbaynes

• Change how pytest works for the project.
• Move the existing tests from the top level tests/ dir to the same folder as the code that is being tested.
• add 90% more tests for full coverage.

Currently device config is set in device.py and after uncommenting a new config uuid the database state must be cleared by running with --no-device options and clearing the state model from the django admin console...This is not ideal...

It would be nice to be able to select the device config from a drop down on the dashboard and have that restart the device. This will be needed once we have users changing the device config without software contributor supervision.

Not highly pressing right now but is definitely on the horizon and will make development more convenient.

e.g. check relative humidity within 0-100% else report value as none and report event as a debug message

• In boot message, send the IP of wlan0 or eth0, whichever is active (see the getMAC() code)
• Publish a regular status message with:
  • All Device Modes: NORMAL or the ERROR string from any peripheral.
  • Recipe:
    • Time Elapsed: 0 Days 2 Hours 6 Minutes
    • Time Remaining: 0 Days 9 Hours 54 Minutes
  • resources available
  • IoT status: dis / connected, UTC
  • internet connection status: dis / connected, UTC

YES!! Tests!!!

Is there a way to do it on the GH side, so all commits are formatted the same?
In the old days, you could do a pre-commit hook, but that relies on each developer to set it up.

Jake likes: https://github.com/ambv/black

Manu likes: YAPF (black is an extension of YAPF, so let's use black)

• Don't let unvalidated recipes make it into the database.
• Compare recipe version to about.json recipe_version. Ignore if they don't match and log an error.

We are currently using Django 1.11. The big change in Django 2.0 is python3 is now used under the hood. There are a few minor changes to the routing system (e.g. removed regex in url path names) but migration should be pretty easy.

For easy diagnostics.

1. create a debian package server in gcloud
2. write scripts to make a debian package from the jbrain code with install and upgrade scripts
3. change our default BBB image to have our package server in its sources.list
4. push an updated jbrain apt package to the pkg. svr.
5. change device update script to do apt-get install -y openag-device-software

• Think about the process of notifying a user that an update for their device is available (in the cloud UI).
• Only if a recipe is not running?
• How does the cloud UI tell the device to do the upgrade?
  • pub sub message to brain
  • brain pauses all operations
  • brain does the apt-get, writing output to a log
  • brain runs a delayed script IN THE BACKGROUND, that will run sudo service rc.local restart
  • brain exits

Rob, back this out. There is a PeripheralEvents mixin class that handles it for all subclasses of PM.

This is the device side of the larger problem of handling images in our system.

See Carol Willing's Talk at PyCon 2018 for guidance.

This thread will by spawned by device manager. It will frequently share out 'real-time' state information. Ideally this is a web socket via a django channel but that might not be v1.0, need to think through this. For v1, it might be best to make frequent uploads to a low capacity, high access rate database.

For I2C devices, this will replace the file stream and have 3 functions (read, write, read_write) and a byte queue.

• Read will pull bytes out of the queue
• Write will go to a conditional "lookup table" for adding bytes into queue
• Read/write will write then return read response

Simulator is an object that can hold state (e.g. if a register bit enabled or not, unit reporting specifications, etc.)

Simulator will run from i2c.py but get passed in at the driver layer if simulation mode is enabled.

device side of UI issue - https://github.com/OpenAgInitiative/openag-cloud-v1/issues/81

Do this for a temperature controlled fan. Abstract controllers properly (e.g. common PID class, etc.). Will need to rework actuator concept a bit...controllers should be linked to specific actuator instances, not types. So controller will read the global temperature variable then send commands to Fan-1.

• Clear out old entries from database when disk space gets low.
• Delete oldest images from disk when disk space gets low.
• Add offline data storage support. When device reconnects, then send stored data.

Currently, the approximation just puts the individual channel spectral power distributions (SPDs) into a matrix (e.g. A = [[ch1], [ch2]]), and the desired SPD into a vector (e.g. b = [des]). Then solves Ax = b for x, by using a non-negative least squares approximation. This lends to an uneven distribution of power supplied from the channels once the spectrum is filled. For example, with a very low granularity of spectrum calibration for a red, blue, green, and white channel. The algorithm will only use the red, green, and blue channels to fill the spectrum. This can be improved by increasing the granularity of the calibration but in the case of repeated channels, the spectrum is satisfied with a single channel. This should be able to be improved by weighting the channels...just need to think through all of this a bit more...

Error should have status 400 if bad request or 500 if something internal failed (e.g. io error).

See Carl Meyer's Talk at PyCon 2018 on Type Checked Python in the Real World for guidance.

Create an abstraction that will cleanly perform the try until health failure function...

images are already stored in a local folder and will be uploaded when a connection to the backend is available.

What about other data?
Is this part of the ResourceManager #37 ?

These endpoints became out of date with the migration to the /events based endpoint so we could have individual peripheral based events. Just need to modify the view the route to events instead of directly to the recipe manager

When bringing up new hardware it would be nice to have a standardized method to validate all hardware works as expected...once we have the i2c r/w simulator, automated tests should work seamlessly for testing on real hardware too...but!..not all devices have the same hardware config so we will need to setup config files for the tests specific to each device and include a link to them in the device config file (similar to what we do with the peripheral setup files)...that way at the end of the day, we will be able to run ./test.sh to run all automated tests (what gets integrated with CI) ...and ALSO be able to run ./test.sh --edu1 or ./test.sh --smhz1. Not a first priority issue as we get the core infrastructure up but is a HUGE win from a device reliability perspective.

test IoTPubSub and IoTManager

class needs to provide a simulated response

if so, what should we show?

• last ping time
• if we are currently connected
• last command received?
• last published value?

What has to change in the IotManager class to expose this status? Just add some properties?

• take django out of debug mode
• change anything for postgres (admin account?)
• change anything for google iot?
• we run from /etc/rc.local as root. Keep this as is.