stellio-hub / pyngsild

NGSI-LD library

PyNgsild is a library to communicate/integrate simply with a FIWARE Context Broker using the NGSI-LD API.

Concepts

The library defines Classes for each element of the NGSI-LD data model, namely:

• Entity,
• Property,
• Relationship.

Then, a ContextBroker Class implements the NGSI-LD operations (e.g. Create Entity). The current implementation supports:

• Create Entity,
• Update Entity Attributes,
• Append Entity Attributes,
• Delete Entity,
• Retrieve Entity,
• Query Entities

How To

Create an entity

A simple entity can be created with:

from pyngsild.entity import Entity

entity_1 = Entity(id='uri:entity_id', type='entity_type')

Then, the NGSI-LD context of this entity can be set with:

entity_1.at_context = 'https://my/context'

Create a property

A simple property could be created with:

from pyngsild.proprel import Property

property_1 = Property(name='property_name', value='property_value')

A complete property could be created with:

property_2 = Property(name='property_name', value='property_value',
                       observed_at='some_date_time', unit_code='a_unit_code',
                       datasetid='uri:prop:datatsetid')

GeoProperty are supported and can be created with:

from pyngsild.proprel import GeoProperty

geo_property = GeoProperty(name='location',
                           value='{'type': 'Point',
                                   'coordinates': [46.969047065728226, 19.649525998191066]}')

It is then easy to add a property to an entity or to another property:

property_1.add(geo_property)
entity_1.add(property_1)

Create a relationship

It really follows the same principle as Property (with obviously different type of attributes). For instance, to create relationship:

from pyngsild.proprel import Relationship

relationship_1 = Relationship(name='relationship_name', object_='to:uri:object',
                         observed_at='some_date_time',
                         datasetid='uri:rel:datasetid')

a property can be added to a relationship:

relationship_1.add(property_2)

And a relationship can be added to a property:

property_1.add(relationship_1)

At this stage, the entity_1 has a property property_1, which has a geo_property and a relationship_1, which itself has a property property_2.

The NGSI-LD json representation of this entity can be simply displayed with:

print(entity_1.to_ngsild())

Context Broker

Operations to a NGSI-LD Context Broker are supported through the ContextBroker Class. For instance, creating an entity into a Context Broker is simply done with:

from pyngsild.ctxbroker import ContextBroker

ctxb = ContextBroker()
response = ctxb.create_entity(entity=entity_1)

ContextBroker manage authorisation tokens to access a Context Broker, that is getting and renewing an access token. For this, ContextBroker() requires the following environment variables:

• PYNGSILD_SSO_SERVER_URL: Environment variable for the SSO server URL
• PYNGSILD_SSO_CLIENT_ID: Environment variable for the client_id
• PYNGSILD_SSO_CLIENT_SECRET = Environment variable for the client_secret

Also, the following environment variable is required for the URL of the Context Broker:

• PYNGSILD_CB_HOST: Environment variable for Context Broker host URL

Installation

pip install git+https://{token}@github.com/stellio-hub/pyngsild

Or alternatively:

pip install git+ssh://[email protected]/stellio-hub/pyngsild

pyngsild's Issues

unit tests, comparing classes

assert == for classes is not the correct way to compare two instances of a Class.
Look for the correct way and add to tests.
That causes a significant number of tests to fail.
Or tests to succeed whereas they should fail.

Add support for query entities

Refactoring similar code

There are similar code across the different classes, Entity, Relationship, Property.
To refactor (using class inheritance ?)

Adding several datasetid to a property (likely) not supported

• On creation of the property, if a datasetid is provided, should be created as list []
• And so adding a new instance of property would add a new item in the list (with a different datasetid obviously)
• updating an existing instance will update the attribute(s) of the appropriate instance identified by datasetid

Update implementation of the methods of ContextBroker class

The methods of the ContextBroker class must follow more closely the NGSI-LD specification (https://www.etsi.org/deliver/etsi_gs/CIM/001_099/009/01.04.01_60/gs_CIM009v010401p.pdf section 5 & 6), typically how the Class methods are named or which HTTP methods is used.

Missing dependencies in setup.py

When "pip installing" the pyngsild package, dependencies (such as requests package) are not installed.
This is likely due to dependencies not listed in setup.py, i.e. setup.py should contain a install_requires keyword listing the dependencies, e.g:
setup(
install_requires=[
"tzlocal==2.1",
"requests==2.25.1"]
)

(note to self: investigate diff with find_packages() )

CtxBroker operation append_entity_attributes only add one fragment at a time

append_entity_attributes operation only accepting fragment as of type Property or Relationship implies that it cannot append several properties/relationships in a single call.
The operation should also support a list of (Property, Relationship) as input so that multiple append of (Property, Relationship) in single call is possible.

Add oauthlib, requests_oauthlib to requirements.txt

GeoProperty is not supported

It is currently not possible to create a geoproperty. Trying to create a GeoProperty , for e.g. a geometry of type point ends up with an unsupported JSON, that is:

{
    "@context": "https://mycontext.jsonld",
    "id": "urn:ngsi-ld:AgriParcel:pilze:1os",
    "type": "AgriParcel",
    "location": {
        "type": "Property",
        "value": {
            "type": "Point",
            "coordinates": [39.2753478, 16.4077153]
        }
    }
}

Which triggers the following error from Stellio Context Broker:

{
    "detail": "Property https://uri.etsi.org/ngsi-ld/location has an instance without a value",
    "type": "https://uri.etsi.org/ngsi-ld/errors/BadRequestData",
    "title": "The request includes input data which does not meet the requirements of the operation"
}

The correct JSON should be: ("type": "GeoProperty"):

{
    "@context": "https://mycontext.jsonld",
    "id": "urn:ngsi-ld:AgriParcel:pilze:1os",
    "type": "AgriParcel",
    "location": {
        "type": "GeoProperty",
        "value": {
            "type": "Point",
            "coordinates": [39.2753478, 16.4077153]
        }
    }
}

Update README

Not up to date anymore with the different recent changes.

improve __repr__ of Classes

repr for all classes are minimal, i.e. basically only valid after creation of an instance.
Shall be improved to contain full representation of the instance when called, e.g. for Entity instance -> all properties.

Update property versus add property

The update_property() method of the ContextBroker class is currently doing an HTTP PATCH, i.e. doing exactly what it says, updating a property.
There's currently no ContextBroker method to ADD a property.
Trying to ADD a property using ContextBroker.update_property() fails, as the property does not exist.
It seems using an HTTP POST instead of the HTTP PATCH, could serve both purpose, i.e. adding not yet existing property and updating it when it exists.

Is this a correct way of doing it ?
Should we replace PATCH by POST in the ContextBroker.update_property() ?

Add unittest for Entity containing relationships

As the title says ...

ContextBroker returns

ContextBroker shall return the appropriate object as opposed to a requests.Response object.
Typically, a ContextBroker.get_entity() shall return an Entity instance (or at least as a first step a JSON representation of an Entity insstance)

Evaluate implementation change of properties in Property

attribute properties is identical in Property and Entity classes, but different implementation. The implementation in Entity is more pythonic, we must evaluate if both are identical and correct, in which case implementation in class Property must be updated to follow the one in class Entity.

Manage authentication internally through contextBroker

Currently, the authentication process has to be done externally through other means (token retrieval and refresh). It would be great if the ContextBroker class managed this internally, by providing the authentication information on instantiation of the class(clientId and clientSecret) as parameters and managing the token retrieval and refresh internally as needed.
Thanks!!!

Automatic python datetime to observedAt field format conversion

It would be great if the library automatically converted datetime objects to the format required by the observedAt field.

For example:

property = Property(name='test', value='')
property.observedAt = datetime.now()

would convert to

{
    "test": {
        "type": "Property",
        "value": "",
        "observedAt": "2021-07-16T12:22:00Z"
    },
}

Add tests for ctxbroker module

For testing all methods, in different configuration.