zmoooooritz / stapy

An easy to use SensorThings API Client written in Python

stapy

This is an easy to use SensorThings API Client written in Python

Important

This project is no longer being actively maintained. Since the STA standard keeps evolving, the feature set of this client implementation will keep falling behind. However the project Frost-Python-Client is actively maintained and provides a seemingly complete API.

The SensorThings API (STA) is an OGC standard for IoT device sensing data.
A server implementation of the STA can be found here FROST.
For more (advanced) information about the SensorThings API see the official OGC documentation.

Installation

stapy is compatible with Python 3.7+ and the recommended way to install is via pip

pip install stapy

Usage

stapy can be used both form the command line (in two different ways) and as a Python library.

STA_URL

At first it is necessary to set the URL of the STA. Optionally it is also possible to set credentials for a basic authentication. Within the CLI this can be achieved as follows:

stapy -u {URL}
stapy -c {USR} {PWD}

stapy can of course be also run as a Python module(python -m stapy)

The URL and credentials can also be set from within a Python script:

>>> import stapy

>>> stapy.set_sta_url({URL})
>>> stapy.set_credentials({USR}, {PWD})

CLI - Interactive

The interactive mode is the easiest one to use but can not be used programmatically. Therefore it is probably a good starting point to get familiar with the STA. The interactive mode can be invoked in the following way.

stapy -i

Currently it does support POST, PATCH and DELETE requests.

CLI - Normal

The normal command line mode is a bit more difficult to use but can be automated.

stapy --help

Should give sufficient information on how two use it properly. As the interactive mode it does support POST, PATCH and DELETE requests.

Library

This is the Python interface to the SensorThings API (stapy is meant to be used as Python library). Therefore it supports all requests (POST, PATCH, DELETE and GET).

The relevant classes can be found within the files entity.py, delete.py, post.py, patch.py and query.py in the sta sub-module.

The following syntax can be used to create new entities:

>>> from stapy import Post

>>> Post.observed_property("TestProp", "TestProp-Desc", "TestProp-Def")

To understand which arguments are available and mandatory it is advisable to have a look at the STA-Docs and/or use the interactive mode of stapy.

Following is one example of a GET request:

>>> from stapy import Query, Entity

>>> results, times = Query(Entity.Observation).select("result", "phenomenonTime").order("result").get_data_sets()

results afterwards contains all results of the present Observations in ascending order.
times contains the respective times for the results.

stapy does support all GET query options that are available for the STA. Some examples are select, filter, orderby and skip. These can be chained together as seen in the example above.

Further use-cases and applications can be found in the examples.

Development

To build this project, run python setup.py build. To execute the unit tests, run python setup.py test.

stapy's Issues

Post Datastream error

Hey there,
thx for this library, looks promising.
I just did some testing and ran into an error when posting a Datastream:

Python 3.8.10
stapy 0.2.4

import  stapy as sta

sta.set_api_url('https://top-secret-frost-server.de/v1.1')

iotid_thing = sta.Post.thing("rpi-new", "Rpi4")
iotid_sensor = sta.Post.sensor("Temperature", "On-Board temperature sensor", "text/html", "https://abc.de")

iotid_datastream = sta.Post.datastream(
  "Rpi-4 temperature",
  "Rpi4 temperature datastream",
  "DegC",
  "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
  iotid_thing, 2, iotid_sensor
)

Posting the Datastream fails with this error:

```text
Traceback (most recent call last):   
  File "main.py", line 8, in <module>
    iotid_datastream = sta.Post.datastream(
  File "/home/bruno/.local/lib/python3.8/site-packages/stapy/sta/post.py", line 32, in datastream
    return Post.entity(Entity.Datastream, **params)
  File "/home/bruno/.local/lib/python3.8/site-packages/stapy/sta/post.py", line 195, in entity
    ent.set_param(**params)
  File "/home/bruno/.local/lib/python3.8/site-packages/stapy/sta/abstract_entity.py", line 50, in set_param
    self.json = self._update_json(self.entry_map, self.json, **data)
  File "/home/bruno/.local/lib/python3.8/site-packages/stapy/sta/abstract_entity.py", line 112, in _update_json
    val_is = cast(dict, val_is)
  File "/home/bruno/.local/lib/python3.8/site-packages/stapy/common/util.py", line 28, in cast
    return switch.get(typ)(value)
  File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Any idea what I am doing wrong? I tired with setting a dict as properties as well.

Add CLI support

Allow for CLI arguments that are directly mapped onto requests (post / query)

Documentation

Document what this application can do and how to use it properly

Add some examples for the different usecases

Add for each way of usage some examples in the examples directory

Basic auth example

Does stapy support using a SensorThingsAPI protected by BasicAuth?
Can you give an example on how to use it? As requests library is use in the back, this should be possible.

python-Levenshtein causes issues

As this module is written in C it needs to be complied and therefore needs the according compiler which may not be available everywhere
Therefore out could / should replace this package

Clean-Up request error handling

The current implementation is cumbersome and does not provide sufficient feedback in case an error occurs

Add support for PATCH requests

• Could be build upon Post.py or from ground up
• Add the cli option to patch
• Add the patch functionality to the interactive mode
• Update the documentation accordingly

Refine the entities

Refine the format of the different entities and add more entry_checks where needed

Store the config.ini at an appropriate location

Stapy 0.3.0 not available on Python 3.10

I am trying to POST an Observation using this script. This has been working fine, but (I assume) since an update to Python 3.10 I'm getting an error.

Versions:

Python 3.10.6 

Thx in advance!

`python script.py "https://my.api.url.de/v1.1"`

script.py:
```python

import stapy as sta
from gpiozero import CPUTemperature
import datetime
import sys

if len(sys.argv) > 1:
  api_URL = sys.argv[1]
  # print("Using API URL: " + api_URL + "\n")
else:
  raise ValueError('No API URL passed to script!')

sta.set_api_url(api_URL)

datastream_id = 168

#phenomenonTime = datetime.datetime.utcnow()
phenomenonTime = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%S.%fZ")

cpu = CPUTemperature()
result = cpu.temperature

obs = sta.Post.observation(
  phenomenon_time = phenomenonTime,
  result = result,
  datastream_id = datastream_id
)

print("Posted Observation %d: %s   %6.2f" % (obs, phenomenonTime, result))

Error msg:

Request was not successful (Incorrect request: Failed to parse TimeInterval 2022-10-10T00:00:00-2022-10-28T00:00:00)
Posted Observation -1: 2022-10-28T09:24:10.553088Z    42.35

Error messages when using STA without BasicAuth

When running queries against STA without BasicAuth, I get error messages like this. Everything works as expected, but there is this console output:

import  stapy as sta
import datetime
import random
import sys

if len(sys.argv) > 1:
  api_URL = sys.argv[1]
  print("Using API URL: " + api_URL + "\n")
else:
  raise ValueError('No API URL passed to script!')

sta.set_sta_url(api_URL)

datastream_id = 169
phenomenonTime = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%S,%fZ")
result = random.randrange(0, 20)

obs = sta.Post.observation(
  phenomenonTime,
  result,
  datastream_id = datastream_id
)

results, times = sta.Query(sta.Entity.Observation).entity_id(obs).select("result", "phenomenonTime").get_data_sets()
print('Observation queried from server:\n\tphenomenonTime: %s\n\tresult: %d' % (times, results))

Console output:

CRITICAL:root:The provided key (STA_USR) does not exist in the config file
CRITICAL:root:The provided key (STA_PWD) does not exist in the config file
CRITICAL:root:The provided key (STA_USR) does not exist in the config file
CRITICAL:root:The provided key (STA_PWD) does not exist in the config file
Observation queried from server:
        phenomenonTime: 2022-03-11T09:45:49.321Z
        result: 16

Setting credentials should be optional, as it is not required for every STA. Hence, I think this should not be displayed, when no BasicAuth credentials set using:
stapy.set_credentials({USR}, {PWD}).

Day and month switched when posting Observations

Hey there,

I noticed some stange behavior when trying to POST Observations. I am not sure if this is my fault or if it is a bug.
It appears to be the case, that somehow day and month are switched under certain circumstances when posting Observations.
See the script below:

• phenomenonTime = datetime.datetime.utcnow().strftime("%Y%m%dT%H%M%S.%fZ"): When I use this as timestamp, the Observations ends up in the server with switched month and day.

• phenomenonTime = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%S.%fZ"): This should work, right? It is the ISO date format that FROST accepts. Well, it does not. I get an error message :

  Request was not successful (Failed to store data.)
  

• phenomenonTime = datetime.datetime.utcnow().strftime("%Y%d%mT%H%M%S.%fZ"): this appears to be working fine.
  However, I switched month and day in the timestamp.

I there something I'm missing or is this a bug?

Here is the script I used for testing:

python3 script.py 'https://my-frost-server.de/v1.1'

script.py

import  stapy as sta
import datetime
import random
import sys

if len(sys.argv) > 1:
  api_URL = sys.argv[1]
  print("Using API URL: " + api_URL + "\n")
else:
  raise ValueError('No API URL passed to script!')

sta.set_api_url(api_URL)

datastream_id = 169

# This leads to switched month - day on the server
phenomenonTime = datetime.datetime.utcnow().strftime("%Y%m%dT%H%M%S.%fZ")

# This leads to Request was not successful (Failed to store data.)
# phenomenonTime = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%S.%fZ")

# This works correctly, Note: Day and month are switched in the timestamp
# phenomenonTime = datetime.datetime.utcnow().strftime("%Y%d%mT%H%M%S.%fZ")

result = random.randrange(0, 20)

print('\nObservation before POST to server:\n\tphenomenonTime: %s\n\tresult: %d' % (phenomenonTime, result))
print('\nPOST Observation...')

obs = sta.Post.observation(
  phenomenonTime,
  result,
  datastream_id = datastream_id
)

print("Posted Observation %d: %s %d" % (obs, phenomenonTime, result))
print('POST Observation...done!\n')

results, times = sta.Query(sta.Entity.Observation).entity_id(obs).select("result", "phenomenonTime").get_data_sets()
print('Observation queried from server:\n\tphenomenonTime: %s\n\tresult: %d' % (times, results))

inquirer issue python 3.10

Thanks for creating/packaging/idea!

Thought to test it on my Debian desktop (python 3.10) but got:

$ stapy -u http://localhost:8080/v1.0
/home/richard/venv/lib/python3.10/site-packages/thefuzz/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Traceback (most recent call last):
  File "/home/richard/venv/bin/stapy", line 5, in <module>
    from stapy.cli.main import main
  File "/home/richard/venv/lib/python3.10/site-packages/stapy/cli/main.py", line 3, in <module>
    from stapy.cli.parser import Parser
  File "/home/richard/venv/lib/python3.10/site-packages/stapy/cli/parser.py", line 10, in <module>
    from stapy.cli.cli import CLI
  File "/home/richard/venv/lib/python3.10/site-packages/stapy/cli/cli.py", line 1, in <module>
    from PyInquirer import prompt
  File "/home/richard/venv/lib/python3.10/site-packages/PyInquirer/__init__.py", line 6, in <module>
    from prompt_toolkit.token import Token
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/__init__.py", line 16, in <module>
    from .interface import CommandLineInterface
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/interface.py", line 19, in <module>
    from .application import Application, AbortAction
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/application.py", line 8, in <module>
    from .key_binding.bindings.basic import load_basic_bindings
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/key_binding/bindings/basic.py", line 9, in <module>
    from prompt_toolkit.renderer import HeightIsUnknownError
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/renderer.py", line 11, in <module>
    from prompt_toolkit.styles import Style
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/styles/__init__.py", line 8, in <module>
    from .from_dict import *
  File "/home/richard/venv/lib/python3.10/site-packages/prompt_toolkit/styles/from_dict.py", line 9, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.py)

Tried to update 'inquirer' to latest version... but that dit not help yet...