nubeio / rubix-bacnet-server

Rubix BACnet server

Running in development

• Use poetry to manage dependencies

• Simple script to install

  ./setup.sh

• Join venv

  poetry shell

• Build local binary

  poetry poetry run pyinstaller run.py -n rubix-bacnet --clean --onefile \
  --add-data pyproject.toml:. \
  --add-data VERSION:. \
  --add-data config:config \
  --add-data migrations:migrations

  The output is: dist/rubix-bacnet

Docker build

Build

./docker.sh

The output image is: rubix-bacnet:dev

Run

docker volume create rubix-bacnet-data
docker run --rm -it -p 1919:1919 -v rubix-bacnet-data:/data --name rubix-bacnet rubix-bacnet:dev

Deploy on Production

• Download release artifact
• Review help and start

$ rubix-bacnet -h
Usage: rubix-bacnet [OPTIONS]

Options:
  -p, --port INTEGER              Port  [default: 1717]
  -g, --global-dir PATH           Global dir
  -d, --data-dir PATH             Application data dir
  -c, --conf-dir PATH             Application config dir
  -i, --identifier TEXT           Identifier  [default: bacnet]
  --prod                          Production mode
  -s, --setting-file TEXT         Rubix BACnet: setting json file
  -l, --logging-conf TEXT         Rubix BACnet: logging config file
  --workers INTEGER               Gunicorn: The number of worker processes for handling requests.
  --gunicorn-config TEXT          Gunicorn: config file(gunicorn.conf.py)
  --log-level [FATAL|ERROR|WARN|INFO|DEBUG]
                                  Logging level
  -h, --help                      Show this message and exit.

MQTT client

Topic structure

Publish value topic

<client_id>/<client_name>/<site_id>/<site_name>/<device_id>/<device_name>/rubix/bacnet_server/points/<type>/<object_identifier>/<object_name>

Debug topic

<client_id>/<client_name>/<site_id>/<site_name>/<device_id>/<device_name>/rubix/bacnet_server/debug

Example debug topic

+/+/+/+/+/+/rubix/bacnet_server/debug

How to test using bacnet-stack (here, 2508 as device_id)

Pre-requisite

• Run this app on our local PC
• Configure IP to your local PC IP or can configure by enabling enable_ip_by_nic_name & ip_by_nic_name
• Run this app
• Install bacnet-stack v1.0.0 on BBB or other device on the same network
• Now, it will make below commands available

To get available device

> ./bacwi
;Device   MAC (hex)            SNET  SADR (hex)           APDU
;-------- -------------------- ----- -------------------- ----
  2508    0A:00:00:06:BA:C0    0     00                   1024
;
; Total Devices: 1

To test for a BO

- Read present value
> ./bacrp 2508 4 1 85

- Read priority array
> ./bacrp 2508 4 1 87

- Write a value to @16 of 1
> ./bacwp 2508 4 1 85 16 -1 9 1

- Write a value to @16 of null
> ./bacwp 2508 4 1 85 16 -1 0 0

To test for an AO

- Read present value
> ./bacrp 2508 1 1 85

- Read priority array
> ./bacrp 2508 1 1 87

- Write a value to @16 of 1
> ./bacwp 2508 1 1 85 16 -1 4 1

- Write a value to @16 of null
> ./bacwp 2508 1 1 85 16 -1 0 0

To read device/point info

Point info

- Point Name
> ./bacrp 2508 1 1 77

- Point Discription
> ./bacrp 2508 1 1 28

- Point Units
> ./bacrp 2508 1 1 117

- Point Event State
> ./bacrp 2508 1 1 36

Device info

> ./bacrp 2508 8 2508 77
"nube-io"

> ./bacrp 2508 8 2508 75
(device, 2508)

> ./bacrp 2508 8 2508 112
operational-read-only

> ./bacrp 2508 8 2508 121
"NUBE-IO-IO vendor_name"

> ./bacrp 2508 8 2508 120
1173

Writing string

Write device name

./bacwp 202 8 202 77 0 -1 7 test2

./bacrp 2508 8 2508 77

Write point name

./bacwp 202 8 202 77 0 -1 7 test2

./bacrp 2508 8 2508 77

rubix-bacnet-server's Issues

Return object instance

I really need to use this one in wires

https://github.com/NubeDev/bacnet-flask/blob/3672053c853d05498979bc87fc3e1a0386ce342f/src/bacnet_server/models/point_store.py#L7

script.bash

update script.bash to match this one:
https://github.com/NubeIO/rubix-point-server/blob/master/script.bash

NOTE: make sure to edit the help section and any other relevant fields that relate to this project specifically

Add HTTP PATCH (point update)

Add HTTP PATCH (point update) via the point_name same as update by point object id

Update point store

On update of the bacnet server (from a 3rd party BACnet device or our CRUD API )the point_store table will need to be updated

https://github.com/NubeDev/bacnet-flask/blob/master/src/__init__.py#L56

Stop log's from production

logging.conf

change logging.conf to logging.example.conf

https://github.com/NubeIO/rubix-bacnet-server/blob/master/src/__init__.py#L8

if os.environ.get("data_dir") is None:
    logging_file = 'logging/logging.conf'
else:
    logging_file = os.path.join(os.environ.get("data_dir"), 'logging.conf')
try:
    logging.config.fileConfig(logging_file)
except Exception as e:
    raise Exception(f'Failed to load logging config file {logging_file}. '
                    f'Assure the example config is cloned as logging.conf')

Duplicate MQTT Loop Threads

https://github.com/NubeDev/bacnet-flask/blob/master/src/bacnet_server/mqtt_connection.py#L29
MqttConnection.__client.loop_start() kicks off its own thread
and
MqttConnection.__client.loop_forever() is running it's own blocking loop in called thread
Only need one or the other

Error when adding points

/home/aidan/code/python/bacnet-flask/venv/bin/python /home/aidan/code/python/bacnet-flask/run.py
2020-12-02 08:01:26,964 INFO: src.background> Running Background Task...
 * Serving Flask app "src" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
2020-12-02 08:01:26,966 INFO: werkzeug>  * Running on http://0.0.0.0:1717/ (Press CTRL+C to quit)
2020-12-02 08:01:46,621 INFO: werkzeug> 127.0.0.1 - - [02/Dec/2020 08:01:46] "GET /api/bacnet/points HTTP/1.1" 200 -
2020-12-02 08:01:46,748 INFO: werkzeug> 127.0.0.1 - - [02/Dec/2020 08:01:46] "GET /favicon.ico HTTP/1.1" 404 -
2020-12-02 08:01:54,330 INFO: werkzeug> 127.0.0.1 - - [02/Dec/2020 08:01:54] "GET /api/bacnet/points HTTP/1.1" 200 -
2020-12-02 08:01:54,910 INFO: werkzeug> 127.0.0.1 - - [02/Dec/2020 08:01:54] "GET /api/bacnet/points HTTP/1.1" 200 -
2020-12-02 08:01:55,388 INFO: werkzeug> 127.0.0.1 - - [02/Dec/2020 08:01:55] "GET /api/bacnet/points HTTP/1.1" 200 -
2020-12-02 08:02:24,424 ERROR: src> Exception on /api/bacnet/points [POST]
Traceback (most recent call last):
  File "/home/aidan/code/python/bacnet-flask/src/bacnet_server/resources/point/point_base.py", line 45, in add_point
    BACServer.get_instance().add_point(point)
  File "/home/aidan/code/python/bacnet-flask/src/bacnet_server/bac_server.py", line 109, in add_point
    self.__bacnet.this_application.add_object(ao)
AttributeError: 'NoneType' object has no attribute 'this_application'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 468, in wrapper
    resp = resource(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 583, in dispatch_request
    resp = meth(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 675, in wrapper
    resp = f(*args, **kwargs)
  File "/home/aidan/code/python/bacnet-flask/src/bacnet_server/resources/point/point_plural.py", line 20, in post
    return self.add_point(data, _uuid)
  File "/home/aidan/code/python/bacnet-flask/src/bacnet_server/resources/point/point_base.py", line 48, in add_point
    abort(500, message=str(e))
  File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 32, in abort
    original_flask_abort(http_status_code)
  File "/usr/local/lib/python3.8/dist-packages/werkzeug/exceptions.py", line 822, in abort
    return _aborter(status, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/werkzeug/exceptions.py", line 807, in __call__
    raise self.mapping[code](*args, **kwargs)
werkzeug.exceptions.InternalServerError: 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
2020-12-02 08:02:24,425 INFO: werkzeug> 127.0.0.1 - - [02/Dec/2020 08:02:24] "POST /api/bacnet/points HTTP/1.1" 500 -
2020-12-02 08:02:24,433 ERROR: src> Exception on /api/bacnet/points [POST]
Traceback (most recent call last):
  File "/home/aidan/code/python/bacnet-flask/src/bacnet_server/resources/point/point_base.py", line 45, in add_point
    BACServer.get_instance().add_point(point)
  File "/home/aidan/code/python/bacnet-flask/src/bacnet_server/bac_server.py", line 109, in add_point
    self.__bacnet.this_application.add_object(ao)
AttributeError: 'NoneType' object has no attribute 'this_application'

During handling of the above exception, another exception occurred:

VERSION

add VERSION file with only contents as the version

1.0.0

Add version to /system endpoint
https://github.com/NubeIO/rubix-bacnet-server/blob/master/src/system/resources/ping.py

with open('VERSION') as version_file:
    version = version_file.read().strip()
...

        return {
            'version': version,
            ...

Add priority to MQTT message

Related to: #362

Currently we are sending numeric present_value only on the topic.

Now send it as:

{"value": "<present_value>", "priority": "<priority>"}

Add migration setup

BACnet server issues

• on rename of point the bacnet system cant see the change untill the service is restarted
• on change of the IP address the bacnet server will not start

Add delete all points endpoint

Boolean Points should only have allowed values of 1 or 0

Currently BACnet binaryOutput and binaryValue can be set to values other than 0 or 1. binary point values should be restricted.

MQTT to HTTP CLIENT

import json
import time
import paho.mqtt.client as paho
import requests

# MQTT broker setting
broker_url = "localhost"
broker_port = 1883
broker_timeout = 30
broker_username = "user"
broker_password = "password"


def on_connect(mqttc, userdata, flags, rc):
    if rc == 0:
        print("Connected With Result Code: {}".format(rc))
    else:
        print("Error! Not connected to broker.")


def on_disconnect(mqttc, userdata, rc):
    print("Client Got Disconnected")


# MQTT handler
mqttc = paho.Client()
mqttc.on_connect = on_connect
mqttc.username_pw_set(username=broker_username, password=broker_password)
mqttc.connect(broker_url, broker_port, broker_timeout)


# Topic 2 - device2/topic2
def on_message_topic2(mosq, obj, msg):
    print("-------------------------")
    print("-------------------------")
    print("-------------------------")
    print("Topic: " + msg.topic)
    print("Message payload: ", msg.payload)
    payload = msg.topic

    service = 'modbus_rtu'
    match = ['rubix', 'points', service]
    t_parts = payload.split("/")
    point_name = None
    point_val = None
    point_fault = None
    if t_parts[0:3] == match and len(t_parts) == 10:
        try:
            point_name = t_parts[8]
            x = json.loads(msg.payload.decode('utf-8'))
            point_val = str("%s" % x['value'])
            point_fault = str("%s" % x['fault'])
        except Exception as e:
            print(e)
            pass
        # point_fault = msg.payload['fault']

    #
    print("-------------------------")
    print(point_name)
    print(point_val)
    print(point_fault)
    print("-------------------------")

    ip = "0.0.0.0"
    port = 1717


    if point_name is not None:
        url = f'http://{ip}:{port}/api/bacnet/points/name/{point_name}'
        body = {
            "priority_array_write": {
                "_16": point_val
            },

        }
        print(url)
        print(body)
        headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
        r_p = requests.patch(f'{url}', data=json.dumps(body), headers=headers)
        r_json = r_p.json()
        print(r_json)





main_topic = f'rubix/points/modbus_rtu/+/+/+/+/+/+/data'
mqttc.subscribe("#")
mqttc.message_callback_add(main_topic, on_message_topic2)

# Run endless loop
mqttc.loop_forever()

present_value & priority_array_write

https://github.com/NubeDev/bacnet-flask/blob/master/src/__init__.py#L89
present_value: will be the result of the highest priority_array. so for example as below the present_value: 22.2

     priority_array_write: {
    "_1": 22.2,
    "_2": "None",
    "_3": "None",
    "_4": "None",
    "_5": "None",
    "_6": "None",
    "_7": "None",
    "_8": "None",
    "_9": "None",
    "_10": "None",
    "_11": "None",
    "_12": "None",
    "_13": "None",
    "_14": "None", None is send no data but dont change the priority array
    "_15": "NULL", THIS WILL LET THE USER SEND A null to realise the value from @15
    "_16": 99
},

In MQTT publish value add point topic

Create an additional field source on point table

Create that additional field source with enum values on point table:

• OWN
• MAPPING

And make our endpoint returns those value with source filter. For example: ?source=OWN, ?source=OWN&MAPPING.

Reason of doing this task: we want to filter only source with type OWN when we do import/export. And recreate other Mappings points from Mapping side.

REST endpoints changes

REST endpoint changes to conform with other projects

ALL:

api_prefix = 'api/bacnet'

SERVER:

• server endpoints need /server/... in all
• other changes

server_api_prefix = f'{api_prefix}/server'
api.add_resource(BACnetServer, f'/{server_api_prefix}')
api.add_resource(BACnetPointPlural, f'/{server_api_prefix}/points')
api.add_resource(BACnetPointSingular, f'/{server_api_prefix}/points/uuid/<string:uuid>')
api.add_resource(BACnetPointObject, f'/{server_api_prefix}/points/object/<string:object_type>/<string:address>')
api.add_resource(BACnetPointName, f'/{server_api_prefix}/points/name/<string:object_name>')

MASTER:

• master endpoints need /master/... in all
• other changes

master_api_prefix = f'{api_prefix}/master/'
api.add_resource(NetworkList, f'/{master_api_prefix}/networks')
api.add_resource(Network, f'/{master_api_prefix}/networks/uuid/<string:uuid>')
api.add_resource(NetworksIds, f'/{master_api_prefix}/networks/ids')
api.add_resource(DeviceList, f'/{master_api_prefix}/devices')
api.add_resource(Device, f'/{master_api_prefix}/devices/uuid/<string:uuid>')
api.add_resource(DevicePoints, f'/{master_api_prefix}/devices/uuid/<string:uuid>/points')
api.add_resource(DevicePoint, f'/{master_api_prefix}/devices/uuid/<string:uuid>/points/object/<string:obj>/<string:obj_instance>/<string:prop>')

api.add_resource(PointWritePresentValue, f'/{master_api_prefix}/devices/uuid/<string:uuid>/points/write/object/<string:obj>/<string:obj_instance>/<string:value>')

to add:

api.add_resource(Point, f'/{master_api_prefix}/points')
api.add_resource(Point, f'/{master_api_prefix}/points/uuid/<string:uuid>')

SYSTEM:

• /system/ping -> /system (leave old one in for now)

system_api_prefix = f'{api_prefix}/system'
api.add_resource(Ping, f'/{system_api_prefix}', f'/{system_api_prefix}/ping')

BACnet point addresses should only be unique grouped by objectType

Currently the BACnet service requires BACnet points to have a unique address regardless of BACnet objectType.

It should allow analogOutput-1 and binaryOutput-1.

Impove the amout of HTTP point patches

The bacnet server is getting locked up when there are to many HTTP point updates over the rest api

MQTT topic needs to be based of the rubix-service plat details

This topic is for cloud services

Workflow

• User will update the deatils from the UI
• rubix-service will publish the updated details over MQTT
• each service like for example lora-raw will subscribe to the changes and update as required

https://github.com/NubeIO/rubix-service/blob/master/src/platform/model_wires_plat.py#L5

On restart of stack please publish all the values again on MQTT

On CRUD of a point we need to update bacpypes

When a point is updated over the API we need to update bacpypes server

Example:

• A point address is changed
• A point name is changed
• A point name is added/deleted

Update eventState via the API

Payload can be normal or fault

{
  eventState: normal || fault
}

• object: eventState
• bacnet id 36
• data type BACnetEventState ENUMERATED: normal(0)fault(1)

https://github.com/NubeDev/bacnet-flask/blob/master/src/bacnet_server/bac_server.py#L49

Startings of bash install

wget https://github.com/NubeDev/bacnet-flask/archive/v1.1.1.zip && unzip v1.1.1.zip && rm v1.1.1.zip && cd bacnet-flask-1.1.1
sudo apt-get install python3-venv -y && python3 -m venv venv && source venv/bin/activate
pip install --upgrade pip && pip install -r requirements.txt && pip install -r requirements.txt
# rename the dir
mv $PWD/bacnet-flask-1.1.1/ $PWD/bacnet-flask
# update ip and device ID, use the device ip for the device ID
cp $PWD/settings/config.example.ini  $PWD/settings/config.ini
ip addr | grep eth0 | grep inet
nano settings/config.ini
# test the bacnet stack starts
python run.py
# install the service
sudo cp systemd/nubeio-bacnet-server.service /etc/systemd/system/ && sudo systemctl daemon-reload && sudo systemctl enable nubeio-bacnet-server.service && sudo journalctl -f -u nubeio-bacnet-server.service

git clone --depth 1 https://github.com/NubeDev/bacnet-flask
cd bacnet-flask/
sudo apt-get install python3-venv -y && python3 -m venv venv && source venv/bin/activate
pip install --upgrade pip && pip install -r requirements.txt && pip install -r requirements.txt
# update ip and device ID, use the device ip for the device ID
cp $PWD/settings/config.example.ini  $PWD/settings/config.ini
ip addr | grep eth0 | grep inet
nano settings/config.ini
# test the bacnet stack starts
python run.py
# install the service
sudo cp systemd/nubeio-bacnet-server.service /etc/systemd/system/ && sudo systemctl daemon-reload && sudo systemctl enable nubeio-bacnet-server.service && sudo journalctl -f -u nubeio-bacnet-server.service
# check its running
sudo systemctl restart nubeio-bacnet-server.service
sudo journalctl -f -u nubeio-bacnet-server.service

Shorten the UUID length

Use shortuuid as in this pull request: NubeIO/rubix-point-server#370

Brief Description: it uses the uuid4 with base57 URL encodable characters.

Move data.db into production dir

Issue on CRUD bacnet points

The DB was deleted and service was restart

delete and edit a point not working

use next address isnt working

On create point next available address

In the end point for create new point have an option to use the next avalibe address https://github.com/NubeIO/rubix-bacnet-server/blob/master/src/bacnet_server/models/model_point.py#L16

use_next_available_address = db.Column(db.Boolean(), nullable=False, default=False)
https://github.com/NubeIO/rubix-bacnet-server/blob/master/src/bacnet_server/models/model_point.py#L11

This will make it easier for the UI to auto create points

So if use_next_available_address then address needs to be null

On update of API brodcast the MQTT

Same as here

https://github.com/NubeDev/bacnet-flask/blob/master/src/bacnet_server/feedbacks/analog_output.py#L32

Add an endpoint for BACnet server

This is to replace the ini file
We can only have a max of 1 bacnet server for now and probs forever

• name
• ip
• port
• bacnet_id

On update the BACnet server will need to be reinitialised

Update name and units

Make it so rest can update the point name and point units

object_name
units

Point name

point name should not be the UUID it must be the object_name

debian@beaglebone:~/bacnet-stack-0.8.6/bin$ ./bacrp 123 1 1 77
"d7510848-6901-461e-8b15-39e0876683e0"

Units

units we need to be able to update the units

debian@beaglebone:~/bacnet-stack-0.8.6/bin$ ./bacrp 123 1 1 117
milliseconds

Discription

Also this need to work over the REST
https://github.com/NubeDev/bacnet-flask/blob/master/src/bacnet_server/models/model_point.py#L21

Create objectIdentifier

object_identifier: str, unique, feedback from bacpypes, This is made up of a combination of the object_type and address, Example: analogOutput-1

https://github.com/NubeDev/bacnet-flask/blob/master/src/__init__.py#L91