eclipse-uprotocol / up-spec

uProtocol Specifications

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in IETF BCP14 (RFC2119 & RFC8174)

SPDX-FileCopyrightText: 2023 Contributors to the Eclipse Foundation

See the NOTICE file(s) distributed with this work for additional
information regarding copyright ownership.

This program and the accompanying materials are made available under
the terms of the Apache License Version 2.0 which is available at
https://www.apache.org/licenses/LICENSE-2.0

SPDX-FileType: DOCUMENTATION
SPDX-License-Identifier: Apache-2.0

Table of Contents

• 1. Overview
• 2. How to get started?

1. Overview

uProtocol is a communication protocol that enables software (uEs) to easily communicate with each other running in distributed systems. The protocol is designed to be lightweight, efficient, easy to use, and run on top of any other communication middleware (transport) such as SOME/IP, MQTT, zenoh, HTTP, etc…​

To achieve the above mentioned objectives, we have split the specifications into sections (folders) that are:

• The Basics: Foundational principles and concepts of uProtocol (ex. such as addressing, IDs, message header definitions, etc…​).

• Transport layer (uP-L1): Common interface for sending and receiving of messages intended to be implemented employing a particular communication middleware technology (ex. MQTT, SOME/IP, HTTP, Zenoh, etc…​).

• Communication Layer (uP-L2): The common transport agnostic, programming language specific, API for the publisher/subscriber, notification, and RPC messaging patterns used by application/service developers to communicate with each other.

• Application Layer (uP-L3): Service contracts for subscription management, service/device discovery, and event caching that is common across all deployments (implementations) of uProtocol.

2. How to get started?

It is always best to start with the basics to familiarize yourself with common terminology used with uProtocol.

If you’re a developer who would like to use uProtocol in your application or service, please refer to the appropriate language library (ex. up-cpp, up-java, up-rust, etc..) README.adoc.

If you would like to contribute to uProtocol to develop a new language library or transport implementation, please checkout the CONTRIBUTING.adoc.

up-spec's Issues

uSubscription as a part of Language SDK / uLink

Was reading through the uSubscription section and viewing this diagram on uProtocol SDK and had some questions.

1. Is it intended to have the uSubscription interface be exposed as a part of the Language SDK or have that interface and implementation be solely within the uLink library?
2. I guess this is really just an expansion of the first question to flesh out my thoughts. Thinking of a couple of examples, could there be both, say, a uprotocol-rust-ulink-zenoh and uprotocol-rust-ulink-http2 and they could potentially have different traits (i.e. interfaces) for uSubscription, so long as they both had implemented all the required functions, e.g. rpc Subscribe(SubscriptionRequest) returns (SubscriptionResponse)?

If that is the intention, I'm curious to hear your thoughts on the benefits of doing so.

I definitely agree that there will be different underlying implementations of uSubscription based on what transport it's being used over depending on which uprotocol-rust-ulink-* crate it's a part of. But I'm less sure on the benefits of not consolidating around an interface for it in uprotocol-rust.

I could see similar arguments being made for uTwin and uDiscovery having standardized interfaces in the Language SDKs.

Or perhaps the spec has drifted out of sync with current thoughts? I can see that in the uprotocol-rust implementation, the .protos for uSubscription, uTwin, and uDiscovery have been code-genned into Rust, but marked as unused yet.

TL;DR: Is it in the scope of the uprotocol-foo libraries to standardize interfaces for uSubscription, uTwin, and uDiscovery or is that completely up to each uprotocol-foo-ulink-bar library?

RpcClient::invokeMethod() passing CallOptions

Align the spec with up-java and up-rust where we pass CallOptions in lieu of UAttributes to the method as the only attribute we need is the ttl and token as the other UAttributes can be auto-generated in the implementation of invokeMethod().

Should up-l2 API consist of up-l1?

The up-l1 API includes send and register_listener, while the up-l2 API includes invoke_method and register_rpc_listener.
We can find the description in the up-l2 rpcclient spec.

Implementations communicate to their underlining transport through the uTransport Interface.

It means we should implement invoke_method with the up-l1 API.

However, there is no similar description in the rpcserver spec (register_rpc_listener).
@PLeVasseur and I discussed this while supporting uStreamer for upc-zenoh-rust.
I assume that all the up-l2 consists of up-l1, but just want to make sure that assumption is correct.

uDiscovery Database Access Permissions

Reading the Database Access section on uDiscovery, I was wondering the mechanism by which the uDiscovery service can arbitrate and ensure which requests to modify nodes are legitimate.

It seemed like perhaps this could be done by looking in the uMessage's uAttributes at the UUID at the random portion which is generated once per uE.

Is that the mechanism by which the uLDS can confirm access permissions?

I thought that perhaps a malicious actor may be able to listen for events from the uE they want to spoof to get access to the uAttributes containing the uE's randomly generated ID. It may then be possible to craft a uMessage with the spoofed UUID to be able to modify nodes the malicious actor should not have permissions for.

Wondering if I'm missing something in the uDiscovery mechanism to validate access.

Add missing Legal Items

Need to add missing files and tags in each file to adhere to https://www.eclipse.org/projects/handbook/#legaldoc

uDiscovery::FindNodes() - accepted wildcard locations / formats

I can understand from the examples given in the .proto for uDiscovery::FindNodes() and in the UURI spec on wildcards that wildcards may fall anywhere within a UURI. Can there also be multiple sections represented as wildcards?

Based on the following example from the .proto

uDomain: `//*.device/`

could we also have UURIs like the following?

1. All doors on local vehicle, whatever version, learn what methods we can call on them: up:/*.vehicle/body.access/*/door.*
2. Learn what topics are being published by chimes: up:/*.vehicle/body.notification/*/chime#*

uP-L1: Android Binder Transport Specific Requirements

Android binder (AIDL) requirements for the definition of Parcable object model. based on uProtocol 1.5

Fix Copyright header to Eclipse mandatory template

per https://www.eclipse.org/projects/handbook/#ip-copyright-headers, Eclipse requires that the committer puts in copyright section to match that of the committer, in our case GM. This issue will track making that change in all the projects.

Conceptual mismatch in uSubscription service definition

Hi,

while reading through the uSubscription service spec I stumbled across sections 3.1. and 3.2 which seem to restrict the topics that the subscribers and publishers are allowed to receive change notifications for.

However, at the beginning of section 3 I found

The uSubscription service notifies observers when there is a change in a subscription states by publishing notification events to the topic up:/core.usubscription/2/subscriptions#Update.

In typical pub/sub implementations (like MQTT) you would expect that a subscriber to that topic will actually receive all messages being published to that topic. However, the spec seems to restrict the messages that subscribers are allowed to receive based on (internal) state like the list of topics that the client has created/subscribed to. I do not see how this kind of subscription filtering could be implemented using existing MQTT broker implementations.

In general, to me it seems that the way that the uSubscription service's behavior is defined in the spec does not match what is generally considered pub/sub but instead calls for a dedicated notification mechanism based on subscriber/producer specific topics.

Clarification of Micro URI requirements

Hi there 👋

Wanted a bit of help clarifying some of the language around Micro URIs:

1. Do all the fields of a Micro URI need to contain valid (nonzero) bits and be the exact length specified for each type? I'm struggling to reconcile the two following requirements:
   a. "All fields of micro UUri MUST be populated"
   b. "Wildcard is is represented in a long or short URI only when the segment is blank."

Does this mean that it is not possible to have wildcards due to requirement a? Or is requirement a combined with b saying something different?

2. It seems required to serialize to Micro form when transporting "over the wire". This means between uDevices? If so, I'm unsure how we can send Topics, RpcRequest, or RpcResponse between uDevices. The requirements I'm trying to make sense together:
   1. "MUST use the Miro Uri serialization format "on the wire""
   2. The formats of the Micro formats don't seem to include necessary pieces:
      • RPC seem to require at a minimum the UResource name and method in addition to id: "MUST return UStatus::Failed if the UUri.resource.name != "rpc" || UUri.resource.instance != "response" || UUri.resource.id != 0"
      • Topics seem to require UResource message / fragment: "MUST contain UE_NAME, UE_VERSION, RESOURCE, and MESSAGE"

Maybe I've missed how this is accomplished. Could you point me in the right direction?

Usage of uSubscription to get messages for topic subscribed to

I was reading through more of the uSubscription spec, and wanted to check my understanding.

Is it the case that by using uSubscription::Subscribe(), we then use the topic (which is a UURI) contained within the SubscriptionResponse with the uP-L1 uTransport::registerListener() to actually register a UListener to handled when a new message is available?

Similarly, a publisher after calling uSubscribe::CreateTopic() would then go use the uP-L1 uTransport::send()?

Or worded differently, is the uSubscription more a means of ensuring that the topic we'd like to receive is active and healthy, and potentially being multicated into our uDevice? It doesn't handle actually publishing / receiving and that's handled by using uP-L1 in an application. Is that correct?

Definition of UMessageType identifiers

The CloudEvents Common Attributes section defines string values which must be used to encode UMessageType.

FMPOV these values should be defined in the corresponding proto file so that we can refer to them programmatically.

uP-L1: SOME/IP Transport Requirements

Detailed requirements for mapping the SOME/IP protocol to uProtocol (uTransport requirements)

UUri spec uses inconsistent data types/fields

The uProtocol 1.5 UUri spec contains a class diagram illustrating the UUri structure and related entities.

The UAuthority interface in the diagram defines an id property of type bytes. The text in section 2.1 does not specify such a property at all.
Section 2.1 defines properies domain, deviceand address properties which are not contained in the diagram.

The UEntity interface in the diagram defines a version property of type optional string. The text in section 2.2 specifies this property as an optional integer (should rather be an unsigned integer FMPOV).
Section 2.2 specifies the type of the id property as uint16 whereas the diagram defines it as uint32.

Section 2.3 specifies the type of UResource's id property as uint16 whereas the diagram defines it as uint32.

4.2 Notification Message example seems to be wrong, seems to be copy of 4.4 Response Message

See https://github.com/eclipse-uprotocol/uprotocol-spec/blob/main/up-l2/messages/v1/README.adoc#42-notification-message

4.2. Notification Message
{
    "specversion": "1.0",
    "id": "5b9fe861-8c1c-4899-9b07-ad1cde652e10",
    "source": "up://VCU.VIN.veh.up.gm.com/body.access/1/rpc.UpdateDoor",
    "sink": "up://VCU.VIN.veh.up.gm.com/MyAppp/1/rpc.response",
    "type": "res.v1",
    "datacontenttype": "application/x-protobuf",
    "dataschema": "/google.rpc.Status",
    "priority": "CS4",
    "reqid": "cf8b1bcd-30bd-43be-a8d3-ad1cde652e10",
    "ttl": 50000,
    "data": /* Protobuf serialized google.rpc.Status Message*/
}

This is an exact copy of https://github.com/eclipse-uprotocol/uprotocol-spec/blob/main/up-l2/messages/v1/README.adoc#44-response-message

4.4. Response Message
{
    "specversion": "1.0",
    "id": "5b9fe861-8c1c-4899-9b07-ad1cde652e10",
    "source": "up://VCU.VIN.veh.up.gm.com/body.access/1/rpc.UpdateDoor",
    "sink": "up://VCU.VIN.veh.up.gm.com/MyAppp/1/rpc.response",
    "type": "res.v1",
    "datacontenttype": "application/x-protobuf",
    "dataschema": "/google.rpc.Status",
    "priority": "CS4",
    "reqid": "cf8b1bcd-30bd-43be-a8d3-ad1cde652e10",
    "ttl": 50000,
    "data": /* Protobuf serialized google.rpc.Status Message*/
}

But I would expect that a Notification Message had different content, e.g. the type should probably be different, and there should probably be no reqid?

uP-L2: UURI requirements clarification

I'm a bit unclear on if the RpcClient, RpcServer should have stronger requirements on the UURIs they are given so that they are resolved.

It appears like it should be the case for uP-L1 and uP-L3 based on my reading of the UURI spec.

However uP-L2 is not mentioned in the same section.

Further when I check on the section about implementing RpcClient, the requirements look a bit looser on the UURI:

MUST return INVALID_PARAMETER for the following errors:
* UUri is null or empty

So I'm a bit unclear.

Is uP-L2 communication somehow different and can use unresolved UURIs? Is the uLink library supposed to fill in any missing details with uDiscovery? Or should the UURI also need to be resolved for uP-L2 comms?

Dispatcher responsibilities towards UMessageType UMESSAGE_TYPE_PUBLISH and UMESSAGE_TYPE_RESPONSE CEs

Another question as I implement a uStreamer.

There's a section in the Dispatcher spec that reads:

Note: These communication layer requirements are still for point-2-point uE communication to and from a dispatcher

There's a section that reads:

• MUST support At-least-once delivery policy, this means that the dispatcher will make every attempt to dispatch the CE to the intended Receiver
  • MUST queue CEs not successfully acknowledged (transport level at-least-once delivery confirmation described above)
  • MUST attempt to retry transmission of the CE. Retry policy is specific to the dispatcher implementation
  • Dispatcher MUST NOT discard CEs unless either CE has expired (CE.ttl), or the egress queue is full. CEs that cannot be delivered are sent to a Dead Letter Office Topic

I think these requirements make a lot of sense for CEs with UMessageType of UMESSAGE_TYPE_REQUEST as there is some expectation of an acknowledgement.

My question is regarding UMessageType UMESSAGE_TYPE_PUBLISH and UMESSAGE_TYPE_RESPONSE CEs.

These are, in my mind, more of a "fire and forget" style of messages.

Are the above requirements intended to hold to to them as well? If so, is there an intent to have a receiver of UMessageType UMESSAGE_TYPE_PUBLISH and UMESSAGE_TYPE_RESPONSE CEs provide some sort of acknowledgement? Is that noted somewhere in the spec?

Replace google.rpc.Status with org.eclipse.uprotocol.v1.UStatus

Due to the dependency on google.rpc.Status we have had to pull in googleapis which are not required. We've introduced UStatus to the uTransport API so next logical thing to do is to bring it up to the core-api and deprecate the dependency on googleapis so that we only depend on protobuf.

Name & Number Registry

Hi Everyone,

uProtocol (like many other internet protocols) defines names and numbers that are used to address things like topics, services, devices, etc.. per the URI specification. At GM we saw the need to create a central registry in order to not have uEs reuse the name name/numbers (ex. a service from VW shouldn't have the name "com.gm" etc...
The Internet Address and Name Authority (IANA) takes care of registering and management of name/number assignments, maybe it is something we need to ensure no overlap in name/number assignments with COVESA (vehicle), uProtocol, cloud, and other uServices. thoughts?

Dispatcher: Mechanism by which we bridge to other transports

Hey there 👋

As I'm fleshing out the uStreamer implementation, I wanted to make sure I'm understanding the overall flow.

1. Receive CE on our registered UListener for up-client-bar-rust which will give us all CEs which have an Authority set
2. Peek at the CE's UAttributes's sink (a UUri)
3. Route as below:

• If the sink is our uDevice, then:
  • If the UMessageType is UMESSAGE_TYPE_PUBLISH we translate into up-client-zenoh-rust UTransport::send() to dispatch locally
  • If the UMessageType is UMESSAGE_TYPE_RESPONSE we look up a stored Zenoh Query to attempt to reply to it
  • If the UMessageType is UMESSAGE_TYPE_REQUEST we then call the up-client-zenoh-rust RpcClient::invoke_method(), wait on the response locally, and then send back on the same up-client-bar-rust we received the CE on
• If the sink is not our uDevice, then:
  • Broadcast over every up-client-foo-rust we have with uP-L1 UTransport::send()
    • (because from my limited reading I did not see a mechanism by which to determine from the CE the intended target transport & we could potentially transit multiple intermediate transports)
    • I think if we're just an intermediary we don't have any special behavior based on the UMessageType

Does that sound about right?

What would we do in the case receiving a CE with UMessageType of UMESSAGE_TYPE_UNSPECIFIED? Drop it?

Typical types of URIs, perhaps useful to make builder(s) for common types

Trying to gather some feedback on an idea.

I've been thinking about writing some kind of builder(s) to ensure a user can readily build typical URIs with all the relevant fields filled out and in resolved state without needing to reference the spec or validation code.

What are the most often used types of URIs it would be useful to have builder for?

Off the top of my head, I'm imagining:

• Topic
• RpcRequest
• RpcResponse
• NonRpc (some kind of catch all for other resolved kinds of URIs)

What do y'all think?

Clarifying the bits and bytes of Uri must fit as a condition of `isMicroForm(UUri): boolean`

Hey folks 👋

A little while back I added some stricter checking into up-rust to ensure that the UUri that we are passed into isMicroForm() will fit into the available bits and bytes.

Call it the paranoia of someone that's worked in embedded systems 🙃

Would we be open to me formalizing this in the appropriate place in the spec, for example in the Uri Validators section?

Could be as simple as changing to:

API	Description
isMicroForm(UUri): boolean	Returns true if the Uri part contains the required ids to serialized to Micro format and the fields of the Uri can fit within the specified number of bits and bytes. Both sets of details can be obtained under [Micro Formats]

What do y'all think?

CAP & TAP Specifications

Migrate the CAP & TAP definitions to permissions.adoc

uP-L3 & uP-L2: Throttling publication rate in the streamer

Often there is the case where a topic is published at a high frequency within a device but we don't require (in fact prefer) to receive the published event less frequently between devices. Example situation requiring this throttling is say a service publishing current GPS location every 500ms however sending at that frequency would be too costly to the cloud so we would like to sent every 3 seconds.

The CR shall be to cover adding to the SubscribeAttribute, an optional attribute that will be interpreted by the streamers for throttling published events.

uTransport Interfaces

uProtocol 1.3.7 has been implemented by GM and open sourced to Eclipse. Version 1.4 of the protocol included a set of APIs for the application layer interfaces of uSubscription, uDiscovery, uTwin defined in proto files. The initial deployment of uProtocol served QM applications and services running in Android and the cloud using a single programming language Java. Phase 2 of SDV will require additional programming language support (C++ & RUST), additional deployments (Linux, MCUs, ), ASIL-B requirements, as well as advanced transports that must perform zero copy, etc....

For phase 1 of uProtocol, CloudEvent was chosen as the PDU (protocol data unit, a.k.a the message envelope/object that contains the data & metadata) for 2 reasons:

• Specification: Definition for how to describe data in a common way

• Implementation: Easy of use (the SDKs, mental model that allows extensibility), support in the Cloud, Existing transport bindings (ex. HTTP, MQTT, etc...)

To use the analogy of the post office and sending and receiving mail; You send can send mail in many different shape & size packages, using different label formats (hand written, bar codes, etc...), with different shipping means (air, ground, etc...) however nothing will be delivered without a common attributes for a destination address (name, street, citiy, etc...). CloudEvent provided that common attribute defintion but it unfortunately also meant sending your package in only one label & packaging format.

So in retrospect, and while evaluating the use of CloudEvent as the PDU for other transports like Zenoh, DDS, SOME/IP, etc.. (transports that require zero copy, small header formats, inability to use strings for address, etc...), a number of problems have been identified with CloudEvents (the implementation), they are:

• Fixed Attribute Types: Although we can add additional attributes of any type, the core (specification) attributes types cannot change (ex. MUST be a string)

• Extension Attribute Map: Adding additional attributes require adding a map of attribute names (as a string) to the attribute value thus requiring the name to also be sent on the wire (bloats the header)

• Not Extend-able: CloudEvent Classes are object that is generated from protobuf. The interface can be expanded by adding additional atributes (mentioned above) however the class itself (CloudEvent), cannot nor should not be extended (reference)

• Pass by Value, not Reference: The immutable CloudEvents object contain the data and does not support passing the data by reference, this is needed for zerocopy of the data unless the entire CloudEvent object is written to the transport

Additional problem identified was that there is no common interface definition for the transport layer, this has led developers writing their own that sometimes mixes up uProtocol concepts/layers or are duplicated in each uE.

The following changeset that will span all the projects shall attempt to address the above mentioned problems by introducing a programming language agnostic uTransport interface, and a data model for the required contracts required for uE-2-uE and uDevice-2-uDevice communication.

IMPORTANT NOTE 1: Fundamental assumption with this proposal is that the transport "knows best" how to write the passed UUri (topic), uPayload, and UAttributes to the underlining transport PDU. Given this assumption, the developer facing interfaces (uTransport) no longer works with a generic PDU implementation (CloudEvent). The onus is on the Transports to ensure that the contents of Uuri, UPayload, and UAttributes are carried throughout the network from source to the destination intact.

Given that not all transports send the exact same message format, streamers MUST either know (at compile time) the next-hop UTransport, or it will require storing the uTransport (type) in uDiscovery.

IMPORTANT NOTE 2: Transports that already use CloudEvents as their PDUs (ex. Binder, MQTT) can continue to do so as this proposal no longer dictates the underlining transport wire format

RpcClient InvokeMethod() returning uMessage in lieu of UPayload

What we would like to implement is a pluggable streamer that we can easily implement the functionality using only a communication middleware's up-client library that implements UTransport, RpcClient, and RpcServer interfaces.
UTransport allows for simple sending and receiving of messages of any kind while Rpc interfaces are there to simplify client & server side RPC requests/responses correlation.

When a communication middleware (transport) uses a different implementation for rpc vs pub/sub (ex. zenoh uses queriables for rpc and publisher & subscriber APIs for pub/sub)), we cannot simply call UTransport APIs to wire up the RPC flows.

What is missing for the streamer to be able to use RpcClient (to proxy a clients request to invoke a method) is the interface returning UAttributes, otherwise I cannot call send() to send the response on a different transport.

If we limited uStreamer to only using uTransport APIs, it means we cnanot use zenoh queriables and we have to implement the request/response coralation inside of the streamer (in lieu of inside of the up-client libraries that know how to do it anyways for the given implementation).

this issue is to change invokeMethod() to return UMessage in lieu of just the UPayload.

Does UTransport::register_listener match on source or sink address?

The RegisterListener() spec does not explicitly define the (type of) address to match on.

Register a UListener to receive message(s) for a given UUri (topic).

seems to suggest that the given topic should be matched on a message's destination address (sink attribute). However, that would mean that publish messages cannot be matched because they only contain an origin address (source attribute).

Clarification around uP-L1 source and sink

Hi there 👋

Was in the process of implementing a uStreamer that lead to a question about the uP-L1 docs.

In the Dispatcher spec it says:

Like IP packets, uProtocol messages (CE) have a source attribute (originator of the message) and sink attribute (where should the message be sent to). These attributes are used to route the CEs from one uE to the next if the destination is not the receiving uE.

There's a diagram at the bottom of the page with source and sink marked as I'd expect based on the above description.

If I look at the uP-L1 spec and the Rust implementation:

    async fn send(
        &self,
        topic: UUri,
        payload: UPayload,
        attributes: UAttributes,
    ) -> Result<(), UStatus>;

    async fn register_listener(
        &self,
        topic: UUri,
        listener: Box<dyn Fn(Result<UMessage, UStatus>) + Send + Sync + 'static>,
    ) -> Result<String, UStatus>;

I had assumed that the topic UUri of UTransport::send() gets mapped into the UMessage's source UUri by the up-client library to enable the Dispatcher to use UTransport::register_listener in order to listen for and react to all incoming CEs to route them over the other transports and then allow us to subsequently route them back to their source (for example in the case of RPC Request / Response).

However, when I read the spec for uP-L1 UTransport::send, the UUri parameter is described as the destination. For me that didn't make sense together. There is already a sink UUri contained within the UAttributes, contained within the UMessage.

Can you help me square my understanding?

uP-L1: Zenoh Transport Specific Requirements

The following will capture the requirements for mapping UUri, UAttributes, and UPayload to zenoh defined header and payload for consistencies across implementations & languages of the zenoh transport.

UTransport::RegisterListener - listen to > 1 topic

Hey there 👋

Was working through the logic of implementing a uStreamer.

From what I understand the uStreamer would want to register one UListener with UTransport::RegisterListener() for each transport the uStreamer needs to switch over, then within the callback route to the other transports it can speak.

Reading through the spec on UTransport::RegisterListener, it appears that should be possible:

MUST support registering more than one topic per listener

But I'm unclear on how that mechanism would work: do we pass in a UURI to UTransport::RegisterListener() which has all wildcards? How would that work for a transport that only supports micro form?

uSubscription::RegisterForNotifications mechanism clarification

When I read the spec and the service definition for uSubscription, I was having a hard time understanding how those uEs that are interested in learning about subscription changes receive them other than calling the uP-L1 uTransport::registerListener() API on the topic for subscription updates: up:/core.usubscription/2/subscriptions#Update

Reading the service definition for uSubscription::RegisterForNotifications() it appears to return a uStatus. It doesn't look like the API accommodates being given a UListener.

So I'm unclear on how the following is accomplished from the Subscription Change Notifications section:

Subscribers are automatically registered to receive subscription change notifications for topics they have subscribed too through the Subscribe() API.

How to integrate our own transport layer

Hi,
After studying uprocotol-spec, I still can't fully understand some points about uProtocol.

1. Is there any example application using uProtocol now? I'm just curious how the uProtocol API looks like when I want to do discovery/subscription or something else. It would be good to have a reference.
2. If I want to implement our own transport with uProtocol, then what should we do? Implement ulink by ourselves? I saw uprotocol-python-ulink-zenoh. Does that mean you're already working on putting Zenoh into uProtocol?
3. I saw there is a branch about SOME/IP bridge. I'm curious about the architecture here.
   
   I'm a little confused. Does that mean uProtocol bridge is used to bridge uDevice to other devices using SOME/IP natively? If there are two uDevice and I want them to communicate with each other, is it possible to use SOME/IP Bridge on both sides and make them talk to each other? Something like this:

flowchart LR
    uD1["uDevice 1"]
    uD2["uDevice 2"]
    b1["SOME/IP bridge"]
    b2["SOME/IP bridge"]
    uD1 -->|uProtocol| b1
    b1 -->|SOME/IP| b2
    b2 -->|uProtocol| uD2

Hope you don't think these questions are a little bit silly. They bothered me for quite a while.

How to invoke the Discovery API

Hi,

I was wondering how a uE (e.g. a client application) would interact with the Local Discovery Service? Given that there is only a proto definition, there seem to be multiple possibilities to implement the LDS, e.g. exposing a gRPC interface via HTTP2. Is the client supposed to simply know how to interact with the LDS or is the LDS to be invoked transparently by a uBus component instead?

SOME/IP Specifications

Initial commit of the uProtocol specifications

Does RpcClient::invoke_method() require a payload or not?

The RpcClient spec defines the UUri IN parameter to be mandatory.
However, there is no mentioning of the payload parameter which seems to indicate that payload is optional. This would make a lot of sense to me because there will probably be methods that do not require any input.

In any case, IMHO we should make it more explicit if parameters are mandatory or optional.

Validation of UUri when sending Publish, Request, Response type -- builders, send(), invokeMethod()

In the UTransport::send() spec we can see that there's these lines:

• SHOULD pass resolved UUris (contains both names and numbers)
• SHOULD use the Miro Uri serialization format

And in the RpcClient::invokeMethod() spec it's written:

Implementations communicate to their underlining transport through the uTransport Interface.

I think in-practice we've strayed away from the above requirement that RpcClient::invokeMethod() be using UTransport::send(), particularly in the Zenoh implementation, due to the ability to use Zenoh Queryables.

I see SHOULD is used to specify that resolved UUris are what should be passed in and also that it SHOULD use Micro Uri serialization format.

A few points I think I'd like to see addressed:

1. Should up-foolang client libraries be opinionated about checking the UUris fit Micro Uri format or are resolved Uris when they are making builders? I'm looking at the Source section on the languages spec page where there's a discussion that builders can be provided.
   • In particular, there's some work being done over in up-rust to update to add a UMessageBuilder and reviewing the code, I was reminded of requirements from the spec. Feedback here would help.
2. Should up-client-bartransport-foolang UTransport::send() implementations be checking one or both of the above SHOULD lines from the spec?
3. Should up-client-bartransport-foolang RpcClient::invokeMethod() implementations be forced to use UTransport::send()? If not we should probably:
   • update the spec to clarify they can have a completely separate implementation of RpcClient::invokeMethod() which does not depend on UTransport::send() and also
   • add any UUri requirements we come up with for the 2. item above

Tagging @sophokles73, @stevenhartley for visibility (for some odd reason Daniel's handle doesn't come up with I "@" in this issue)

Drop uTransport authenticate()

This PR will cover the following changes:

• uTransport:

  • Remove authenticate() as it is not a client facing API
  • Use UMessage in lieu of individual parts (UUri, UAtributes, UPayload) given we moved the data model to proto and teh justifications for keeping them separate (for passing payload by reference) will not be an issue when we use our own proto compiler for C++ (and possibly rust)

• RpcServer: New interface that will be used for the server-side communication of RPC. Servers need to register listeners for request messages and then send the response

Definition/usage of message priority is ambiguous

Based on the definition of mapping common attributes to CloudEvents, my understanding is that request messages MUST contain a priority of at least CS4.

It is unclear, though, if response messages also need to contain a priority (of the same priority). This would make sense to me, because if I am interested in getting a response quickly (indicated by CS4 on the request), it seems reasonable to also require the response to be delivered high-prio, or am I mistaken?

Also, the default priority to assume when no priority is specified is defined inconsistently across the spec:

• The source mentioned above defines the default to be CS0

• The QoS specification defines the default value to be CS1, which also seems to be in line with the wording used in the UAttributes proto where CS1 is described as

  Standard, undifferentiated application such as General (unclassified)

FMPOV we need to adapt the CloudEvent attributes mapping section accordingly.

Specification Cleanup

The following issue will be used to track the following changes/initiatives:

• Rebranding of ulink library to uPClient
• Renaming of projects
• Dropping the term uPlatform (we don't need yet another term to just mean it is an implementation of uProtocol)
• Dropping the term "SDK" and more simply explaining the protocol libraries and their purpose
• Adding the RpcServer interface
• Cleanup of uTransport & RpcClient interfaces to match language libraries
• Other minor cosmetic fixes and issues found.

Specification of transport protocol bindings

I appreciate the latest additions to the spec made in #14 👍

While reading through the uTransport's Send() definition I was wondering, if the proposed way of letting implementations decide if and how to map payload, attributes and URI to the underlying transport's messaging primitives will actually provide for the required interoperability.

Let's assume the mentioned HartleyTransport protocol existed and it would provide means to send messages that contain a byte array as payload and would also allow to set generic string based attributes on a message.

The first one to implement the uTransport for HartleyTransport in Java will now decide, how the URI and attributes passed in to the Send() method will be mapped to HartleyTransport message attributes.

Several months later, I want to implement the uTranport for HartleyTransport in my fantasy programming language BUMLUX. How do I find out, which attributes of a HartleyTransport message contain the URI and the message properties provided by the original sender?

If the sender of the message used the Java implementation, then I will need to dig into the Java code to find out where the information is to be found in the message. However, what if somebody else had implemented a Python based implementation of the HartleyTransport and has mapped the attributes differently, e.g. using a certain attribute name prefix?

I guess what I am saying is that IMHO we need transport protocol specific bindings for the message payload, attributes and URI instead of leaving this up to the (first/each?) implementer of a transport. We might be able to simply refer to the existing CloudEvent bindings for obvious transport protocols and add our own bindings for more exotic protocols like Zenoh.

WDYT?

uTransport is inconsistent with uMessage definition

The uTransport's send() method is defined to accept three parameters. The first one being a UUri which is supposed to represent the Destination for the UPayload.

The uMessage protobuf file defines the uMessage correspondingly:

message UMessage {
    // The source (address) of the message. For published events this is the producers publish topic
    // for requests it is the calling uE respose topic (who sent the request). For responses this is the
    // method URI from the uService
    UUri source = 1;

    // uProtocol mandatory and optional attributes
    UAttributes attributes = 2;

    // Optional message payload containing the data to be sent
    UPayload payload = 3;
}

However, the first (UUri typed) property is defined as the source (address) instead of the destination. The destination address of the message seems to be contained in the UAttributes' sink property.

uP-L2: Adding NOT_FOUND to commstatus

The following change shall expand the list of dispatcher returned commstatus errors to include NOT_FOUND.

This specifically relates to situations where the request sink URL (the destination) is not found. For instance, in cases where a vehicle has not been provisioned yet, the dispatcher cannot ascertain the destination.
This distinction is needed to separate it from the "UNAVAILABLE" case where the vehicle has been provisioned but not connected.

This affects https://github.com/eclipse-uprotocol/uprotocol-spec/blob/main/up-l2/dispatchers/README.adoc

Need to specify expected behavior for UUID generation

The UUID spec defines that within each millisecond a maximum of 4095 distinct UUIDs can be generated. However, it is unclear what the expected behavior is when that limit is reached, e.g. is a uEntity supposed to be able to send messages having the same UUID at that point? Or does this mean that each uEntity can send at most 4095 messages per millisecond and once this limit is reached, it has to wait for the clock progressing to the next millisecond before it can send additional messages?

Even if this question seems rather hypothetical due to the (usually) limited processing power available in a vehicle, it would still make implementing an SDK much more robust and predictable.

uprotocol-spec: Specifications Cleanup

Update the following specifications:

• UUri
• uTransport
• RpcClient
• Languages

And add the following new spec:

• RpcServer

Initial commit

Initial commit of the specifications

Move UAttributes data model definition to match uprotocol-core-api

We have defined UAttributes in proto in lieu of per language library, this allows for consistent mental model across all languages. Need to update spec to reflect this as well.

Discovery for SDV 2.0 needs Concept Requirements to define what it should be

Per meeting with workgroup prior:

uDiscovery 2.0 Concept for In-Vehicle clarifications - 1 pager needed to help answer below details.

• what is the scope - how it will be used by uEs on CCU, uEs on CHM

• what data does it store - calibrations, configurations, PII? mechatronics properties? [Halim may have thoughts]

• what is supported in non connected regions for the scope mentioned above? *vehicle should be considered source of truth

• how does it receive that data - at flash time? run time APIs from services themselves? discovery queries HW itself?

• role of CHM discovery vs CCU discovery

• interaction between BO and vehicle ?

uP-L1: SOME/IP Transport Requirements

Specification for how to translate uProtocol to SOME/IP protocol that contains functionality at all three layers of the stack.

Define how `UTransport::registerListener()` shall map which `UUri` (i.e. source vs sink) onto Zenoh KeyExpr for the various `Notification`, `Publish`, `Request`, and `Response` message types

Hey @evshary 👋 --

We were having an internal discussion about which UUri to map (source vs sink) and how for up-client-zenoh-cpp.

I also saw the discussion pivot over on #73 via @sophokles73's latest message.

I think it would be very good to document similar to how @sophokles73 did, but for Zenoh so that we have consistency across all up-client-zenoh-foo:

@evshary: Would you like to take the ball and write-up what this should be in the spec here so that we're all on the same page?

I'd like to tag Michael on here as well, but his name doesn't show up when I @. I'll forward this issue to him.

Distributed Tracing

From Jeff:
Define optional uAttribute (traceparent) that is compatible with the CloudEvents Distributed Tracing extension and industry norms to support proper end-to-end observability. This is especially valuable for more complex situations. One example could be where a UE may raise an event as a result of another event.

Example:

Vehicle's session service publishes an trip started event (id=abc123)
A driver app receives the event, looks up driver's identity, and D.O.T. log to see first drive of the day. Driver app publishes shift_started event (id=xyz456, traceparent=abc123)
When an issue arises, the operational team can quickly and effectively see the correlated chain of events
For discussion: would we also want to include an optional tracestate to be more complete?