fiware / context.orion-ld Goto Github PK

(spec has to be updated)

instead a JSON-LD Link header shall be provided as part of the HTTP response

Entity:

'id': 'urn:ngsi-ld:T_Query:EntityForQuery2345',
    'type': 'T_Query',
    'P100': {
      'type': 'Property',
      'value': 12,
      'observedAt': '2018-12-04T12:00:00',
      'P1_R1': {
        'type': 'Relationship',
        'object': 'urn:ngsi-ld:T2:6789'
      },
      'P1_P1': {
        'type': 'Property',
        'value': 0.79
      }
    },
    'R100': {
      'type': 'Relationship',
      'object': 'urn:ngsi-ld:T2:6789',
      'R1_R1': {
        'type': 'Relationship',
        'object': 'urn:ngsi-ld:T3:A2345'
      },
      'R1_P1': {
        'type': 'Property',
        'value': false
      }
    },
    'location': {
      'type': 'GeoProperty',
      'value': {
        'type': 'Point',
        'coordinates': [-30.01, 75.01]
      }
    }
  };

When retrieved, coordinates are

'value': {
'type': 'Point',
'coordinates': [-30.01, 75.010002]
}

The steps 06 and 07 in the functest ngsild_content_type_and_context.test shows that the HTTP Link header comes back with TWO initial start brackets:

Link: <<https://fiware.github.io/NGSI-LD_TestSuite/ldContext/testContext.jsonld>; rel=http://www.w3.org/ns/json-ld#context; type=application/ld+json>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"

Also, rel="http://www.w3.org/ns/json-ld#context" is there TWICE ...

Something is wrong here ...

The kjson library has a memleak for JSON buffers that present a JSON Parse Error.
It's a bit tricky, but MUST be fixed

Right now, URI Expansion is only used in orionldPostEntities.
The functions that implement the URI expansion are static functions "tailor made" for that specific request.
Soon other requests will need URI Expansion as well and the implementation will need to get more generic.

Hi,
I think that it exists a problem with the timestamps in format ISO 8601 when receiving the notification data related to a subscription.
When I post an entity like the one in https://github.com/FIWARE/NGSI-LD_Experimental/blob/master/doc/example-code.md

curl http://localhost:1026/ngsi-ld/v1/entities/ -H 'Content-Type:application/ld+json' -iX  POST -d '{
   "id": "urn:ngsi-ld:Vehicle:A4568",
   "type": "Vehicle",   
   "brandName": {
      "type": "Property",   
      "value": "Mercedes"
    },
   "isParked": {
      "type": "Relationship",
      "object": "urn:ngsi-ld:OffStreetParking:Downtown1",
      "observedAt":"2018-12-04T12:00:00Z",
      "providedBy": {
            "type": "Relationship",
            "object": "urn:ngsi-ld:Person:Bob"
       }
    },
   "@context": ["http://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld"]
}'

I get the field observedAt correctly with a GET request:
curl "http://localhost:1026/ngsi-ld/v1/entities?type=Vehicle" -H "Accept: application/ld+json"

But when I create a subscription for getting that entity

curl http://localhost:1026/ngsi-ld/v1/subscriptions/ -H 'Content-Type:application/ld+json' -iX POST -d '{
 "id": "urn:ngsi-ld:Subscription:01",
 "type": "Subscription",
 "entities": [{"type": "Vehicle"}],
 "@context": "http://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld", 
 "description": "Notify me of all product vehicles changes", 
 "notification": {
     "format": "normalized",
     "endpoint": {
         "uri": "http://mydomain/",
         "accept": "application/ld+json"
       }
   }
 }'

I get the following response:

{
 "id":"urn:ngsi-ld:Notification:5ce59a3631178d6ca8eefbe6",
 "type":"Notification",
 "subscriptionId":"urn:ngsi-ld:Subscription:01",
 "notifiedAt":"2019-05-22T18:51:34Z",
 "data":[
     {
       "id":"urn:ngsi-ld:Vehicle:A4568",
       "type":"Vehicle",
       "brandName":{
            "type":"Property",
            "value":"Mercedes"
        },
       "isParked":{
            "type":"Relationship",
            "value":"urn:ngsi-ld:OffStreetParking:Downtown1",
            "observedAt":"",
            "providedBy":"urn:ngsi-ld:Person:Bob"
        }
     }
  ]
}

With content-type: application/json

As it can be seen, in my subscription I specify that I want my notifications to have content in ld+json format instead of json and the field observedAt is empty in the notification.
I am using docker-compose.

Furthermore, I was wondering if I could get in the notifications the creDate and modDate fields of the objects saved in Orion with a subscription.

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextConsumption/retrieve_entity_test.js#L10

The official Core @context is at

http://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld

Please modify your code to point to it and remove the old forge.etsi....

Thanks

POST /ngsi-ld/v1/subscriptions/

{
  "id": "urn:ngsi-ld:Subscription:9000",
  "type": "Subscription",
  "entities": [
    {
      "id": "urn:ngsi-ld:T:T1234",
      "type": "T"
    }
  ],
  "notification": {
    "endpoint": {
      "uri": "http://localhost:1234"
    }
  }
}

it fails with

{
    "type": "http://uri.etsi.org/ngsi-ld/errors/BadRequestData",
    "title": "Either 'timeInterval' or 'watchedAttributes' must be present. But not both of them",
    "details": "None of them"
}

But the spec says that watchedAttributes or entities shall be present, so in that case all attributes shall be watched and the subscription shall be created.

Using

https://raw.githubusercontent.com/FIWARE/dataModels/master/specs/AgriFood/AgriCrop/example-normalized-ld.jsonld

an error is raised

{"type":"http://uri.etsi.org/ngsi-ld/errors/BadRequestData","title":"Attribute must be a JSON object","details":"createdAt"}

createdAt / modifiedAt are not reified properties, thus they are directly the date as string in ISO8601 format

Example:

GET /ngsi-ld/v1/entities/urn:ngsi-ld:Vehicle:V1234?attrs=nonPresentAttribute

Response is

{"body": {"details": "no details", "title": "No context element found", "type": "http://example.org/ngsi-ld/errors/ResourceNotFound"}, "headers": {"connection": "close", "content-length": "119", "content-type": "application/json", "date": "Thu, 22 Nov 2018 16:11:21 GMT"}, "request": {"headers": {"accept": "application/json"}, "method": "GET", "uri": {"auth": null, "hash": null, "host": "localhost:1030", "hostname": "localhost", "href": "http://localhost:1030/ngsi-ld/v1/entities/urn%3Angsi-ld%3AT%3A1542903081782?attrs=notFoundAttr", "path": "/ngsi-ld/v1/entities/urn%3Angsi-ld%3AT%3A1542903081782?attrs=notFoundAttr", "pathname": "/ngsi-ld/v1/entities/urn%3Angsi-ld%3AT%3A1542903081782", "port": "1030", "protocol": "http:", "query": "attrs=notFoundAttr", "search": "?attrs=notFoundAttr", "slashes": true}}, "statusCode": 404}

And it should be (as it is currently in Orion):

200 OK

{
"id": "urn:ngsi-ld:Vehicle:V1234",
"type": "Vehicle"
}

We need to support cases

{
  "@context":  {
    "foaf": "http://xmlns.com/foaf/0.1/"
  },
  "id": "http://example.org/people#joebob",
  "type": "foaf:Person",
  "foaf:name":  {
     "type": "Property",
     "value": "Jose"
  }
}

See test at

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextProvision/append_entity_attrs_test.js#L83

I have noticed that sometimes error payloads are generated as follows:

"type": "http://example.org/ngsi-ld/errors/ResourceNotFound" 

Error payloads "type" member shall always contain ETSI-defined URIs:

Hi,
I am trying to get a subscription working for new added data sources. For this I add the following subscription:
{ "id": "urn:ngsi-ld:Subscription:testsubscription", "type": "Subscription", "entities": [ { "type": ".*" } ], "notification": { "format": "keyValues", "endpoint": { "uri": "http://callback_computer:8080/callback" } }, "@context": "https://fiware.github.io/NGSI-LD_TestSuite/ldContext/testFullContext.jsonld" }

From time to time I get a 201 header back with the subscription id as location. Besides the fact that I don't get any callback for new context (don't know why) the orion-ld server, provided as the latest docker image, constantly crashes.
When trying to add the subscription the connection seems to hang up. After a while (ca. 1 minute) the broker has crashed with a "std::bad_alloc" in its log.
Even if I am doing something wrong with the subscription a crash should not happen? Is there any help/tutorial showing ho to get a subscription working with NGSI-LD? I just found the test suite where I copied the subscription example and changed it to mine (as shown above).
Thanks and kind regards,
tank

Error:

{"body": {"details": "https://fiware.github.io/NGSI-LD_Tests/ldContext/testFullContext.jsonld", "title": "failure to create context from URL", "type": "http://example.org/ngsi-ld/errors/BadRequestData"}, "headers": {"connection": "close", "content-length": "188", "content-type": "application/json", "date": "Tue, 20 Nov 2018 16:50:38 GMT"}, "request": {"headers": {"Content-Type": "application/ld+json", "accept": "application/json", "content-length": 307}, "method": "POST", "uri": {"auth": null, "hash": null, "host": "localhost:1030", "hostname": "localhost", "href": "http://localhost:1030/ngsi-ld/v1/entities/", "path": "/ngsi-ld/v1/entities/", "pathname": "/ngsi-ld/v1/entities/", "port": "1030", "protocol": "http:", "query": null, "search": null, "slashes": true}}, "statusCode": 400}

The default content to be provided in the absence of an Accept header is application/json

by condition over object

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextConsumption/query_entities_test.js#L98

by condition over observedAt

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextConsumption/query_entities_test.js#L108

over property of a property

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextConsumption/query_entities_test.js#L118

etc.

Section 6.3.4 of the specification

From NGSI-LD Test:

1. created entity successfully
   {
   'id': 'urn:ngsi-ld:T:001',
   'type': 'T',
   'P1': {
   'type': 'Property',
   'value': 'Hola'
   }
   }
2. update fails with:
   {
   'P1': {
   'type': 'Property',
   'value': 'Ciao'
   }
   }

Orion-LD returns:
{type: 'http://uri.etsi.org/ngsi-ld/errors/BadRequestData',
title: 'Attribute does not exist',
details: 'P1' }
}
Used docker version. Endpoint : http://loalhost:1026/ngsi-ld/v1/entities/urn:ngsi-ld:T:001
HTTP Method: PATCH
Header: Content-type: application/json

There is something wrong?
Thank you.
Cris

In some (all?) error responses, the @context member is missing in the payload.
See for example step 2+3 in functest ngsild_entity_creation.test

See test at

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextProvision/create_entity_test.js#L239

const entity = {
      'id': 'urn:ngsi-ld:T:' + new Date().getTime(),
      'type': 'T',
      'AR1': {
        'type': 'Relationship',
        'object': ['urn:ngsi-ld:T:1234', 'urn:ngsi-ld:T:5678']
      }
    };

When rendering timestamp data, and as per the latest version of the spec the trailing timezone (always 'Z', has to be rendered). This applies for observedAt, createdAt, modifiedAt or any other DateTime attribute.

Hi,
besides other problems with the NGSI-LD subscriptions (see #60 ) it seems there are some additional problems.
If I create a subscription
{ "id": "urn:ngsi-ld:Subscription:testsubscription", "type": "Subscription", "entities": [ { "type": "vehicle" } ], "notification": { "format": "keyValues", "endpoint": { "uri": "http://mycallback:8080/callback" } }, "@context": "https://fiware.github.io/NGSI-LD_TestSuite/ldContext/testFullContext.jsonld" }
I get a 201 Created back with location /ngsi-ld/v1/subscriptions/urn:ngsi-ld:Subscription:testsubscription. (So if I don't create it twice I get the right response without a crash.)
If I try to get all subscriptions I it seems the interface is not implemented yet:
{ "type": "http://uri.etsi.org/ngsi-ld/errors/BadRequestData", "title": "not implemented - GET /ngsi-ld/v1/subscriptions", "details": "no details" }
If I now want to avoid adding the subscription again I tried to delete it with
DELETE http://ourorion:1026/ngsi-ld/v1/subscriptions/urn:ngsi-ld:Subscription:testsubscription
This results in
{ "type": "http://uri.etsi.org/ngsi-ld/errors/BadRequestData", "title": "The requested subscription has not been found. Check id", "details": "urn:ngsi-ld:Subscription:testsubscription" }
This is strange as this is the same ID as used in the first step where I received the 201 Created.

String Filters in ngsild need to URI expand the attribute.
E.g. for the string filter ?q=A1.i==12 A1 will be URI expanded to http:/xxx.yyy.zzz/A1 (or any other URI depending on the @context).

But, what happens if an attribute is created with APIv2?
Then its name is actually "A1" in the database and after URI expansion, A1 will not be found.
To fix this, we'd need to know that en entity/attribute has been created with APIv2 and not ngsild and no URI expansion should be done for this entity/attribute.

Hi, and congratulations for the great work with OrionLD!

I've been testing the new broker and have found a really odd sequence that utterly crashes the broker. It even won't go back up after restarting the docker container.

The sequence is easy: with a mint new container (won't work with a previously existing container which has answered queries) launch two queries in sequence:

1. GET http://{{orion}}/ngsi-ld/ex/v1/version
2. POST http://{{orion}}/ngsi-ld/v1/entities with the following payload and headers:

{
    "@context": [
            "http://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld",
            "http://schema.lab.fiware.org/ld/jsonldcontext.jsonld"
        ],
    "id": "urn:ngsi-ld:Room:Room1",
    "type": "Room",
    "temperature": {
        "value": 23,
        "type": "Property",
        "unitCode": "C"
    },
    "pressure": {
        "value": 720,
        "type": "Property",
        "unitCode": "mm Hg"
    }
}

and for headers in the second query Content-Type: application/ld+json

Hope this mystery unveils something worth fixing back there ;)

Best regards,
Pedro.

Hi,
I just updated the docker container. Now broker is not starting due to some connection problems to mongo db.
time=Wednesday 15 May 08:30:35 2019.273Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=mongoConnectionPool.cpp[152]:mongoConnect | msg=Database Startup Error (cannot connect to mongo - doing 100 retries with a 1000 microsecond interval)
As far as I can see the mongo db is running. Maybe there is something wrong with the new docker image? System is started via docker-compose.

See

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextConsumption/query_entities_test.js

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextConsumption/query_entities_with_ld_context_test.js

Something seems to have gone terribly wrong about mongo query types.
Many of the unit tests in mongoQueryTypes_test.cpp crash and I don't know why.
This is not the priority right now so I just disable the failing tests and continue.

See description of the crash in mongoQueryTypes_test.cpp, line 219:

FIXME: Test disabled as the call to mongoEntityTypes causes the unit test to crash, in the line:
...

To be revisited

Subscription creation has just been implemented.
However, it works only for Content-Type "application/ld+json", i.e. with the @context in the payload.
Need to accept "application/json" as wel (i.e. @context in the Link HTTP Header).

Now default MIME type is application/ld+json (in the absence of an Accept header).

but to the Default @context or to the corresponding @context

Test failing:

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextSubscription/create_subscription_errors_test.js#L24

The function urnCheck() in src/lib/orionld/common/urnCheck.cpp only checks that the URN starts with urn:, nothing more.
The rest of the function needs to be implemented.

The keyValues option is only implemented for GET operations.
Do we want it for POST/PATCH as well?

UPDATE, Jan-29 2022:

This issue just got more relevant.
There is a push from the FIWARE Foundation for the NGSI-LD API to support key-values for update and perhaps even create operations.

A proposal has been prepared for ETSI ISG CIM and will be presented abd discussed in the NGSIO.-LD API call on Feb 1 2022.
Also, a first "pilot" for the request "PATCH /ngsi-ld/v1/entities/{entityId}/attrs/{attributeName}" is under implementation.

and not the core NGSI-LD @context

https://forge.etsi.org/gitlab/NGSI-LD/NGSI-LD/blob/master/defaultContext/defaultContext.jsonld

POST /ngsi-ld/v1/entities/

{
    "id": "urn:ngsi-ld:ParkingSpot:santander:daoiz_velarde_1_5:3",
    "type": "ParkingSpot",
    "status": {
        "type": "Property",
        "value": "free",
        "observedAt": "2018-09-21T12:00:00Z"
    },
    "category": {
        "type": "Property",
        "value": [
            "onstreet"
        ]
    },
    "refParkingSite": {
        "type": "Relationship",
        "object": "urn:ngsi-ld:ParkingSite:santander:daoiz_velarde_1_5"
    },
    "name": {
        "type": "Property",
        "value": "A-13"
    },
    "location": {
        "type": "GeoProperty",
        "value": {
            "type": "Point",
            "coordinates": [
                -3.80356167695194,
                43.46296641666926
            ]
        }
    },
    "@context": [
        "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld",
        "https://schema.lab.fiware.org/ld/context"
    ]
}

Please check the input format at

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextConsumption/retrieve_entity_with_ldcontext_test.js#L102

A subscription is created with the following application/ld+json payload:

{"type":"Subscription","@context":"https://schema.lab.fiware.org/ld/context","entities":[{"type":"ParkingSpot"}],"watchedAttributes":["status"],"notification":{"endpoint":{"uri":"http://localhost:1880/notif-LD","accept":"application/ld+json"},"format":"normalized","attributes":["status"]}}

Then a GET /ngsi-ld/v1/subscriptions is issued as follows:

curl -X GET \
  http://localhost:1030/ngsi-ld/v1/subscriptions/ \
  -H 'Accept: application/json' \
  -H 'Cache-Control: no-cache' \
  -H 'Connection: keep-alive' \
  -H 'Host: localhost:1030' \
  -H 'Link: <https://schema.lab.fiware.org/ld/context>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \
  -H 'Postman-Token: 3a91b721-797b-4e7f-8f1a-e07261cfd8c8,053e36cc-29bb-443f-8627-20b8adef1179' \
  -H 'User-Agent: PostmanRuntime/7.13.0' \
  -H 'accept-encoding: gzip, deflate' \
  -H 'cache-control: no-cache'

{
        "id": "urn:ngsi-ld:Subscription:5d13537c8cae7ee10f60ea97",
        "type": "Subscription",
        "entities": [
            {
                "type": "https://uri.fiware.org/ns/datamodels/ParkingSpot"
            }
        ],
        "watchedAttributes": [
            "https://uri.fiware.org/ns/datamodels/status"
        ],
        "notification": {
            "attributes": [
                "https://uri.fiware.org/ns/datamodels/status"
            ],
            "format": "normalized",
            "endpoint": {
                "uri": "http://localhost:1880/notif-LD",
                "accept": "application/ld+json"
            }
        },
        "expires": "2038-01-19T03:14:07Z"
    }

• expires should not be there
• ldContext Mapping should have happened

It should be, for instance,

Link: <https://fiware.github.io/NGSI-LD_Tests/ldContext/testFullContext.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"'

See

https://github.com/Fiware/NGSI-LD_Tests/blob/master/contextProvision/append_entity_attrs_test.js#L76

To support the following String Filter ...:

GET /ngsi-ld/entities?q=A1.observedAt>2018-09-01T12:00:00

... we will need to modify the current implementation.
observedAt is a metadata and not part of the compound value for A1.
With the current implementation, this would work:

GET /ngsi-ld/entities?mq=A1.observedAt>2018-09-01T12:00:00

But, that doesn't follow the spec of ngsi-ld.

What needs to be done with the current implementation is to treat observedAt, modifiedAt etc as special words and do the lookup in the metadata vector for 'q' String Filters.