amwa-tv / is-13

AMWA IS-13 NMOS Annotation Specification [Work In Progress]

[Work In Progress] AMWA IS-13 NMOS Annotation Specification

This repository holds the source for this Specification, part of the family of Networked Media Open Specifications from the Advanced Media Workflow Association

What does it do?

• Allows control and monitoring applications to update resource labels, descriptions and tags

Why does it matter?

• Operators need to be able to distinguish between different resources
• Often this is aided by user-provided information (beyond what is provided by the vendor through BCP-002-02)

How does it work?

• Media Nodes advertise it in their Node services

Getting started

There is more information about the NMOS Specifications and their GitHub repos at https://specs.amwa.tv/nmos.

is-13's Issues

Add Overview and other common specification sections

Needs doing :-)

Add section around change persistance

We need a section which describes the requirements around persistance of changes.

I think nodes MUST or SHOULD persist changes issued through this API across reboots but these may be lost if a node goes through a factory reset workflow. Also need to discuss if these changes persist after firmware updates (I personally think yes).

Entire resources or just ID/path lists under resource types?

Under /node/{resourceType}/ we currently have an array of whole resources like Node API.
But now that we're not slavishly following the Node API, since we have that /node/ base path, would it be cleaner to just have just ID/path list responses like from e.g. Connection API /single/senders/?

I.e.

GET /x-nmos/annotation/v1.0/node/senders/

would return

[
  "c72cca5b-01db-47aa-bb00-03893defbfae/",
  "171d5c80-7fff-4c23-9383-46503eb1c63e/",
  "a2655c48-8a46-4c82-b9bc-98760d59d7f8/"
]

instead of

[
  {
    "id": "c72cca5b-01db-47aa-bb00-03893defbfae",
    "label": "Foo",
    ...
  },
  {
    "id": "171d5c80-7fff-4c23-9383-46503eb1c63e",
    "label": "Bar",
    ...
  },
  {
    "id": "a2655c48-8a46-4c82-b9bc-98760d59d7f8"
    "label": "Baz",
    ...
  }
]

Better status codes for PATCH requests

The current API uses the 501 Internal Server Error status for conditions that are not errors of the server or unexpected errors but known conditions that may arise in the use of the API.

I would propose using the following status code for the PATCH method for conditions that are allowed by the API and expected from some implementations.

405 Method Not Allowed
Returned when the implementation does not support the annotation API for a specific resource. Client shall not retry the request.

422 Unprocessable Content
Returned when the implementation does not support the annotation API request: string too long, read-only tag, too many tags values, too many tags, etc. Client shall not retry the request. It shall modify the request based on the detailed status message before retrying.

507 Insufficient Storage
Returned when the implementation does not have more space for storing annotations. Client shall not retry the request.

Join the dots on updating the Registry after updating annotations

The Version Increments section talks about keeping the properties in sync with the Node API and specifically the version attribute. It could probably do with an explicit link to https://specs.amwa.tv/is-04/releases/v1.3.2/docs/APIs_-_Common_Keys.html#version and an informative note highlighting that this means in registered operation, the changes are registered with the Registration API and thus Controllers can receive notifications of the changes via Query API WebSocket subscriptions.

State that tag keys need to be unique

Discussion 2023-05-04: We would benefit from text stating that the tag keys need to be unique. They are currently URNs but there's nothing in theory stopping us having duplicates.

Constraints

Can we define a minimum level of support for changing label, description and tags?

E.g.

• whether the spec requires all of these are editable
• whether there is a guaranteed minimum supported string length
• whether there is a guaranteed minimum number of tags that can be added, and values per tag

And then just accept that 500 responses may occur if these are exceeded...

Or

Do we need some kind of /constraints endpoint that allows each Node to indicate its supported string lengths, maximum number of tags and tag values, and which tags are editable, etc.

Allow all tags to be reset in one operation?

Should it be possible to remove/reset to default all tags on a resource in one request?

PATCH /self

{
  "tags": null
}

This would need to succeed even when the resource has one or more read-only tags and leave them unchanged.
This begs the question about whether a request like the following should succeed (and leave the value(s) unchanged) or fail?

PATCH /self

{
  "tags": {
    "read-only-tag": null
  }
}

Thoughts?

---

FWIW, I think it's clear that a request to actually change the value(s) associated with a read-only tag must fail with 500 response.

PATCH /self

{
  "tags": {
    "read-only-tag": [...]
  }
}

Compare behaviour with read/write tags...

1. Replace the value(s)

   PATCH /self

   {
     "tags": {
       "read-write-tag": [...]
     }
   }

2. Remove the tag or reset to its default/underlying value(s)

   PATCH /self

   {
     "tags": {
       "read-write-tag": null
     }
   }

409 Conflict or 412 Precondition Failed

https://github.com/garethsb/nmos-rwnode/blob/5e0fd19ec0ec9d38e30bc1f988b05af33a4350f1/APIs/ReadWriteNodeAPI.raml#L57-L58

412 Precondition Failed is normally used with ETag

Persistence and resource lifecycle/lifetime

Currently the wording in the Behaviour: Persistence of Updates section is:

"The API implementation MUST persist updates to the core information properties for the lifetime of a resource uniquely identified by its ID, including consistency over reboots, power cycles, and software upgrades."

I think traditional devices which have fairly static UUID allocations are well covered by the statement.

In terms of more dynamic devices I think we might need to better capture the relationship between persistence requirements and lifecycle/lifetime of resources. I think at the very least we need to define what is the lifetime of a resource.

Some options are:

• A. Does the lifetime of a resource end when the host Node expects to never use its UUID again? (e.g. a Node which has a remove resource workflow that permanently destroys the use of that UUID and future newly created resources are not expected to reuse that UUID).
• B. Does the lifetime of a resource end when its UUID disappears from the APIs hosted by the Node/Device?

For a Node/Device which has a limited number of operating profiles it can swap between, options A and B may result in different persistence outcomes.

For A the Node/Device is required to persist information for every operating profile even if it is currently offline because their lifetime never really ends.

For B the Node/Device only needs to persist the information for the current online operating profile. Switching away from the current operating profile could clear all associated information as the lifetime of resources ends.

Ultimately I believe our goal is to specify what is the minimum guarantee implementers can rely on.

Addition or removal of specific tag values

Currently API allows a tag to be removed or for the array of values to be replaced.
It does not provide a way to append a tag value or remove one or more values from the existing list.

Relative merits of Node-based vs Registry-based Annotation API

We've discussed on group calls, the relative merits of Node-based vs Registry-based Annotation API.

Some of the things we thought of...

	Node-based	Registry-based
Discovery mechanism	Node services	Not clear... DNS-SD?
Interop with Node API	Well defined	Not provided
Interop with Query API	Well defined	Provided but complicated a little by version
Annotation limits	Requirements based on low-powered devices	Could be relaxed?
Annotation lifetime	Well defined	Probably needs configurable garbage collection?
Annotation mobility	Intrinsic	No

Minimums for additional limitations

https://github.com/AMWA-TV/is-13/blob/374c934932232e6cdf96a7dae7cb3558b45beaca/docs/Behaviour.md#additional-limitations identifies possible ways an implementation might be limited. Also discussed in #23 (comment).

In Slack, @rbgodwin-nt gave a strawman for some minimum limitations:

from the standpoint of wanting to do some deployment on the NMOS Testbed on AWS and adding labels via Ansible it would be very helpful if I knew that all the NMOS resources supported some minimum set of annotations. For example, all NMOS Resoueces (devices, nodes, senders, receivers, flows etc) SHALL support a minimum of 5 user custom tags with at least 512byte strings each. Then I can do my ansible in a way that meets this and nto have to worry about some device chocking on my annotations.

@alabou responded:

such requirements may be too much form small devices (over 2KB per resource) of persistent storage. I think we must be very careful for such minimum requirement as it may prevent devices form exposing the annotation API at all.

Can we reach consensus on appropriate minimum limitations, e.g.

• there MUST be no limits on characters in strings (i.e. all valid JSON string characters, escapes)
• label and description MUST be read-write
• a minimum label and description length that MUST be supported
• a minimum number of read-write user (urn:x-nmos:tag:user: namespace) tags per resource (and limits on the names of those tags) that MUST be supported
• a minimum number of values per user tag that MUST be supported
• a minimum length of such tag values that MUST be supported
• reset of description, label and user tags MUST be supported

Restrict use of API to certain resources?

Discussion 2023-05-04 assumption was that we only use this API on nodes, devices, senders and receivers, whereas sources and flows might be more dynamic so this API is less useful for them.

Note that senders and receivers could also be dynamic and it also needs to be allowed for those not to be user-labelled (but they will have factory labels)

Should the above be formalised in the specification?

Changes in this API MUST trigger IS-04 version changes

which can be followed through the notification mechanism of a registry.

Decide whether to remove the version check mechanism

The group currently doesn't feel there is a specific use case that needs addressing with this mechanism and it also introduces a concept which doesn't exist in the other specifications.

The notification mechanism provides a way for systems to stay in sync with the recent changes and re-try.

Originally posted by @cristian-recoseanu in #1 (comment)

How to handle external changes

If the label, description & tags are changed via some external mechanism then updating the Version Timestamp is enough?

Bulk changes

Do we need a bulk API?

Making changes for many resources of one type could be done by supporting PATCH (or POST) on the collection endpoints /devices, /sources, etc. with an array of objects - each object would need id property in addition to the current resource_core_patch.json properties.

Obvs, could also come up with a single request endpoint to do resources of several types.

These approaches would still need an object per resource. If we want a way to have one change applied to multiple resources, that needs a different kind of API.

Use with APIs besides IS-04

IS-11 introduces Input and Output resources, which both have a /properties endpoint that uses the same IS-04 resource_core.json schema. Like IS-04, the label, description and tags of these resources are currently read-only.

Should IS-13 support these IS-11 resources or future additions to the shared NMOS data model somehow?

One difference that would currently make updating these Input/Output resources less useful is that they aren't registered and discoverable via the Query API.

In any case, another possibility would be that IS-11 directly adds support for an IS-13-like PATCH since it's not published yet...

(IS-08 also introduced Input and Output resources though I think these seem second tier, acting as they do as the interface between the Channel Map and Sources, Receivers or non-network I/O interfaces. IS-08 Inputs and Outputs do also have a read-only /properties endpoint, but it doesn't follow the same schema, having name (cf. label) and description but no tags. IS-08 also uses a more general identifier format rather than UUIDs.)