quickbird-uk / quickbirduwpdashboard

Quickbird UWP App and Server

Quickbird IoT Dashboard

A Windows 10 app that will act as display / control interface for industrial control systems. The app syncs with the cloud, so you can monitor what's happening remotely. It is meant for developers/makers who are building or prototyping industrial control systems based on MCU/Arduino or PLCs and want to give them a touch-screen interface like this: The result is that "Normal people" can interact with your system, which means makers and hackers can get their industrial prototypes and custom-built systems into sellable state 3 times faster.

Functionality

• Multi-protocol IoT gateway will capture data from MQTT, BLE, serial and sync it to the cloud
• Dashboard will show the data and controls on a touch-schreen device
• Offline-first brings "IoT cloud" to you and runs on your local network, which means low latency and reliability regardless of your internet connection. It will record the data while offline and sync later.
• Idiot-proof It's one package that you download from the store, so you can't screw it up. You don't need to know linux commands or SSH, It requires no special skills beyong setting up a normal windows computer and running an app.

How do I use it?

Grab an Arduino, and write your sensor / control software like you usually would, make it collect readings and control motors / pumps / whaver. Take a look at our guide for making Arduino's work realiably. Your goal is to connect your Arduinos to the local application and the applciation will take care of syncing to the cloud, You will have somethign like this: Next, head over to our integrationGuide and choose the mode of communication that suits you. Take your time to read the considerations we give to the way you should design the system for best results. It should take you no more than a day to make your device talk to the app. The app will detect your device and display all the readings immediately.

Use in Industrial Setting

For Industrial and semi-industrial settings, the app is meant to be run in Assigned Access mode. When the applciaiton is run in this manner, user cannot quit the applicaiton unless they have administrator password. If the application crashes, it will be restarted. To make sure the applicaiton is running at all times (and collecting data), set the computer to LogOn automatically, and to restart after power failure. We did this on a touch-screen device and as a result you get a really nice control-panel type device.

Security model

• The server API is secured with TLS and JWT (pertty standard stuff)
• The LAN is somewhat secure (AES keys Individually provisioned for each device)
• Some areas of the app can be protected with a pin to stop accidental and foolish stuff
• Whoever has admin access to the PC is god
• Whoever can reprogram your hardware / arduino is god

Connectivity

The app is meant to harvest data from different sources, the more the marrier. Currently only LAN is implemented, but we are working on more of them:

• LAN Networking ✓
• USB HID ⧗
• BLE ⧗
• Serial-JSON ⧗
• Ethernet-enabled PLCs ⧗
• Yoctopuse Kit ⧗
• KNX protocol ⧗

Data Model

Check our data model here

Networking

The application runs an embedded MQTT server, GnatMQ, on local port 1883 It is meant to be used in conjunction with an arduino or simular device that's sending MQTT messages over the network. It expects to recieve readings on the topic "readings".

It will also signal presence of the broken on the local network by broadcasting UDP packets on port 44000 The packets contain the work 'sekret" in unicode (yes, I know it's stupid). The idea is for any network device that requires a display to detect this mesage and connect to the app. If the app detects that there is another broker on the network, broadcasting UDP messages, it will notify the user of the conflict. If the "Local Device Management" functionality is disabled, UDP broadcasting and MQTT server are shut down.

Future plans:

• We are considering AllJoin and other LAN networking standards.
• We are considering Implementing network Fail-over
• We are planning to implement AES encryption

BLE

Enable the application to collect data frin BLE sensors. We have not yet created a complete plan around how we want to implement BLE, but currently we belive that the best way to implement BLE sensors is through the use of broadcasts. However, doing so introduces a problem - several applciaitons will recieve the BLE broadcast simultaneously

Serial

In our current plans, serial port is a tricky one and we are debating wether we should include it. The chief problem with serial ports is that:

• Serial ports include no identifying information. One day 'COM1' is an arduino, the next day it's a printer.
• Opening a serial port to an arduino causes it to restart.
• When one opens a serial port, you do not know if you are catching a data frame from the start or in the middle of a transmission. In our current thinking, serial port functionality only makes sense if you have an embedded board, such as raspberi pi, and have an MCU /Arduino permanently connected to the hardware serial port, as opposed to one emulated over USB. However Windows 10 is not the best OS when it comes to support of single-board computers. So we find ourselves in a bit of a dialemma, and consider the plan here to need some refinement.

USB-HID

The application aims to collect data form Arduino / MCU devices connected through USB-HID. This repository includes a PDF that describes HID descriptors used for common sensors and other stuffs. Work on this functionality has not yet begun. We will aim to provide examples with Arduino Due and Leonardo

Supported Deployments

We suppport use of the app on the following

• Windows 10 Pro on a generic computer
• Latte Panda a board with embedded Arduino Leonardo Controller. An RTC must be added for any serious work
• UP Mini PC which can make sense if you add an Arduino Hat
• Windows 10 IoT Core on a Raspberri Pi (Testing only)
• Windows 10 IoT Core on Dragonboard 410c (has no RTC, so is only of limited use).
• One day we will have a .Net core deployment with Avalonia as UI, so you'll be bale ot deploy this on any linux machine. But that is faaar down the road.

Virtual Device

To test the app and server components, use the virtual device option in setting. However, activating this option in more than one instance of the applicaiton at the same time will result in weired sync issues, as they will be producing data for the same device, and the server won;t know which data to choose. .

This app is not meant to

Run in the background

We do not accept dealing with problems that will arise from running this app in the background while the user is torrenting porn or installing yet another virus. If you are not prepared to dedicate at least one of those cheap set-top boxes, then this isn't for you.

Support headless mode

Headless linux-boxes are cheaper, but there is always something wrong with them - they run out of disk space, your forgot the IP address, someone stepped on them, etc. Whatever the case - average dude at the factory can't see what's happening, and can't fix a problem by clicking buttons.

Run the actual control loop

The app will never contain the actual control loop, i.e. it will never respond to readings from a sensor to change setting on one of the actuators. The reason for this is simple - first the app should not be a safety-critical component of hte system, it's crash and restart should have no effect. second is that your system has failure modes known only to you. For isntance, we've seen systems where a disconnected PH probe will report a reading of 14, and activating pumps based on this information would cause disaster. Only you as developer of the hardware can know of these details and should manage them.

Server

The server is currently written on ASP.net 4.6 and is due for replacement with ASP.net core. The REST API is due to be replaced with a Websocket-based system

Authentication

Uses Azure API APP in-built twitter authentication. The Server grabs a token from twitter, and then creates it's own token. The token is avaliable in app in Settings tab under Twitter Authentication, click the button called "CopyAuthToken".

You need to pass this token when making an API request using the header X-ZUMO-AUTH

Development

DASHBOARD: https://waffle.io/quickbird-uk/QuickbirdUWP

Websocket Endpoint: {serveruri}/Websocket

StorePage: https://www.microsoft.com/en-us/store/apps/quickbird/9nblggh6cclt

API Docs: https://greenhouseapi.azurewebsites.net/swagger/ui/index

Arduino Code: Check Examples Folder

Libraries and Dependecies

• Syncfusion.SfChart
• Newtonsoft Json.net
• SQLLite (to be replaced by Dbreeze)
• GnatMQ (modified)
• EntityFrameworkCore

Milestones

2.0.0

Alerts + core server complete.

1.9.0 (planning)

Port to .Net core

• Replace twitter
  • Use Identity
  • Require phone text confirmation
  • Open signup
  • Organizational linking
• Fat Sync API
  • All dependant data should be downloaded in a single request (mult-table download).
• Billing ?
• Alerts
• Control

1.7.0 (WIP)

Maintenance:

• Public Release on store.
• Bug fixes
• Stable App
• Welcome page
• Documentation
• Make the system testable and usable

Change Log (Released Milestones)

1.6.0 (released on store)

• Removed all non working features to make it clean.

1.5.0 (released on store)

• API breaking update to sync code
• Sensor histories now use UploadedAt instead of UpdatedAt

Our Commitment

• Keep the app open source
• We may or may not keep the server component open source

quickbirduwpdashboard's Issues

Properly document MQTT operation

Document how the Arduino is suppose to work with this shit - what format to use, what JSON structure, etc.

read: https://msdn.microsoft.com/en-gb/magazine/ff796231.aspx?f=255&MSPPError=-2147217396

Disable Toasts

Let the user Disable toasts, they are annoying

Look at https://github.com/Azure/connectthedots

Making data avaliable publically

I know we haven't discussed an account system that supports multiple users to keep down complexity of this thing.
But let's say we will have two test users. Is there a good way to share this data publically? I am considering making all non-paying user's data public.
Do you @t3hmun have any thoughts on this matter?

Table Missing

Weired bug, not sure WTF is this

Move In-App DB to DBreeze

SQLite is a piece of crap and can't do migrations properly.
Nuke it from the orbit and use DBreeze.

DocumentationFile.xml not generated

DocumentationFile.xml is not being correctly generated on the server side when a deployment from visual studio is attempted

Put Main Menu Into Proper shape!

Half of the design complexity is because I agonised between selecting Location and Device as the main anchor for all the data, and essentially the Top-level anchor for all the data.
Need to make a choice and that's it.

• That's basically the menu on the left of the app - what does the "main menu" display? Devices? Crops? Locations?
• A grouped list can display any two of those. Crops grouped by Location could make sense.
• Then there needs to be an icon or some indication for whether that device is attached locally or through cloud.
• Then there needs to be some way of seeing which device is managing that, the name of the device.
• Jesus! And then there is still an Icon to play with, and whether that shit is OK or Alerted!
• And lastly, it will no longer be called crop, it will be called "something you are monitoring" because the app will not be specific to greenhouses any more!

Move the APP to Template 10

I have experimented with Template10 and it;s amazing. Will need to update this app with template 10, coming soon. Will update this when I get round to it.

Flashing Arduino Firmware

Display Free Disk-space

display free disk-space and enable user to ommit old data from the pc

Device Type Interface

Allow a device Type to be created, a device type will be owned by a user and have some characteristics that will be set in the app. Amongst them:

• Custom commands the device should have
• whether the device has a clock
• Whether the device does historical data storage
• whether the device has actuators and stuff
• Settings for the device
• It's update rate
• Min / Max alarms
• Min / Max readings sensors can provide
• Behaviours

Concept of computer and locality

Make the user aware of Which devices are on lan, and which are connected remotely.
Some operations should be possible to perform only on the local devices.

Devices with changing sensors

Currently app has no support for adding or removal of sensors from a device, but it would be trivial to support this - have a button for hiding inactive sensors, and logic for creating them

For now, we have a placeholder:

catch (ArgumentNullException)
{
     //TODO add a new sensor to the device! 
     Util.Toast.NotifyUserOfError($"Device has a new sensor with typeID {message.SensorTypeID}, however adding sensors is not supported yet.");
}

Location Management Page

For the moment there is no Location Management, we just give each device a bogus location. There needs to be UI area where location can be created.
Since locations are not deleted, only marked as deleted, the locations that are "deleted" must be hidden.

Embedded DHCP Server

Having an embedded DHCP server could be useful, and make router-less Ethernet connections easier to manage.
Whether we'll actually do it depends on how complex this turns out to be.

read: http://www.codeproject.com/Articles/988606/Storing-time-series-in-Microsoft-Azure-Table-Stora

Computer Awareness

For the app to work, there must be awareness of which computer the device is actually connected to.
There is currently no data model for that, and no computer identity.

HttpBaseProtocolFilter.ClearAuthenticationCache on logout

windows.web.http likes to cache credentials for the entire run of the app (disposing the httpclient doesn't affect this) and magically apply the credentials, ignoring new credentials.

The logout must call ClearAuthenticationCache in order to remove old credentials and not apply the new credentials.

Device Inspection Page

An area where we can inspect a device - it's name, last time we received a message from it, etc.

Online Indicator

When our app is in assigned access mode, there is no way to know if the app is syncing successfully or not. You can't quit the app and there is no visual indicator.
We need some indicator really badly. I could be liked either to success of the sync operation or to the "connected" property on websocket.
Without it, I was wondering why an unconnected computer doesn't sync. This is especially bad with WiFi which can be intermittent.

LocalID for Sensors

Give sensors localID so that we can have several identical sensors.

Re-design the graph page

• Graphs should use new Syncfusion package
• the selector for Sensors / legend should not be so huge
• Cleanup the code, it's a total mess
• loading indicator while loading would be nice.
• Meed to show missing values
  

Improve Loading Speed of the Graphs

At the moment graphs take a long time to load on crappy computers, like my laptop. It's going to become more of a problem as we accumulate more data.
You should add a spinning circle for hte duration of load operation, and see if you can improve loading speed. In our previous app i have tested a few ways of doing that, see here:
http://stackoverflow.com/questions/35854862/how-to-read-a-thousand-objects-with-ef7-and-sqlite

UDP Message is non-sensical

Currently the UDP message is just the word sekret which makes no freaking sense. Replace it with something that does make at least a bit of sense.

Add "Credits" section

Add a section that says "credits" and mentions the people. that worked on the app

Publically Create-able SensorTypes

Allow the general public to create sensor types, subsystems and placements in accordance with their needs

Integrate with Flower Power

Flower power is a cool BLE sensor, it would be good to integrate with it

Websocket API

Alerts configuration UI

We always meant to have alerts and their configuration. However we never managed to design them properly.
We do however know that proper alerts need to:

• Set Alert level, or no alert level
• Set it individually for each sensor
• Mute Alerts on a Specific Sensor
• Actually raise the damn Alert, probably as a toast.
• The alert stays until it's dismissed

Read about websockets

Make the names neutral

Instead of having all names specific to greenhouses - Crops, etc, we could make them neutral, so they fit any application.

New Login System

We'de had it with Twitter Auth.
Get a proper Authentication !

New CropCycle always creates new CropType, Crash if alreadyExists

This code re 78f3ce0 isn;t quite right, will check exact reason later.
This is probably trivial.

LAN Sync for Historical Data

Expand protocol to allow the device to upload data from the past which it might have stored on the SD card or simular

Raspberry Pi Desployment

Make sure the app works on Raspberri PI / WIndows IoT Core
There are going to be numerous people that don't have a computer they can dedicate to this, but have a raspberri Pi.
Make sure the app works there.

Scrap roaming settings/credentials.

• Roaming credentials are difficult to debug and may result in confusing behavior.
• It doesn't help us with stability.
• It binds an account to the user's microsoft account (not fun for debugging and doing admin on multiple accounts).
• Have to write code to check if the new credentials are for the same account.
• Roaming sync is unreliable and random.

Using roaming settings buys us very little of any use at the moment, especially as we already have our own DB.

Add Arduino client examples

Add well-written examples for Arduino

Accept JSON in MQTT

Accepting JSON in MQTT messages will help address usability of the applicaiton

Invalid Sensor ID cause the App to crash

If a IoT device sends messages with invalid SensorID to the app, the app will accept the data and try to save it.
This will cause violation of the foreign key and exception to be thrown form the SQLite. The app will then crash. This must be addressed, with a check and a Toast thrown to the user.

Serial Port Interface

Serial Port makes it really easy to test what is happening - although you should not use it in production, it will earn the app adoption

https://developer.microsoft.com/en-us/windows/iot/win10/unavailableapis add to wiki

User Action for device creation

Make new devices show up in UI, but don't create database entries for them unless user clicks the button

VirtualDevice

In many cases, users will download the app and have no idea what it does. They may also have no actual hardware to test the system with.
The app needs to provide a way to simulate a device and readings, so that users can appreciate the UI and the app functionality.

Cloud Sync issue

It appears that our much-hated and struggled cloud sync algorithm is flawed, and leaves gaps in the data
This is what graphs on the source computer look like

And this is what graphs on the destination computer look like

In addition, the app is currently in a consistent refuse-to-sync state, meaning that for 5 days now no new data made it to the server. However the serve is displaying successful REST requests with no errors.
Furthermore, streaming of real-time data has failed as well, somehow. We appear to be in a 'shit's fucked' situation.

Replace Relays with Actuators

Struggled with a name idiotically. They are actuators. Design around them.

Toast invalid MQTT messages

Toast when invalid message is received over MQTT

Humidity sensor name text does not fit on card

Get the app in the store, publically

Get the application publicly in the store, so that anyone can download it