cbor-wg / cddl Goto Github PK

Since they're an extension point, do we need a registry to list extensions?

https://tools.ietf.org/html/draft-ietf-cbor-cddl-02#section-3.7 includes:

 basic-header = {
     field1: int,
     field2: text,
   }

   advanced-header = {
     ~basic-header,
     field3: bytes,
     field4: ~time,
   }

   (Note that leaving out the first unwrap operator in the latter
   example would lead to nesting the basic-header in its own map inside
   the advanced-header,…

However, maps require a key name, so

advanced-header = { basic-header, … }

is a syntax error.

Maybe the example should be an array?

https://mailarchive.ietf.org/arch/msg/cbor/CycDwteihq6hPZkMMtaLvv_98b4

In section 2, the description of the _table_ composition model was confusing on first read.

Perhaps it is because the first example involves words that can be interpreted in several ways, such as tag, and language, or that it uses words like key without prior definition. For people with a JavaScript background, perhaps describing table as being like a Map (rather than Object) would help.

In the YAML specification and related map tag definition, the word mapping is used for the representation style with name-value pairs, while map is used for what I think you are calling _table_.

Jim: "When writing down a CDDL
grammar, keep it as simple as possible while still expressing the needed
language. "

Carsten: "Just because CDDL makes it easier to handle complexity in a specification, that does not make that complexity somehow less bad (except on the level of the humans having to grasp the complex structure while reading the spec)."

(Two different issues, but both probably worth a mention.)

https://mailarchive.ietf.org/arch/msg/cbor/xp1-0QT3MQFxAZ6C3v1vAfgzUzs

If spec authors rely on CDDL checking to verify that structures have an expected shape, the lack of general cuts could cause malicious data to get to places that don't expect it. For example, if someone were to define an extensible array:

arr = [*elem]
elem /= { type: "foo", val: tstr .size 16 }
elem /= { type: "bar", val: uint }
elem /= { type: tstr, val: any }  ; For future extensions.

then the array [{type: "foo", val: "0123456789ABCDEFbuffer overflow here"}] is a valid instance of arr. We should warn specification authors and implementers to be careful to avoid that.

Considering this example from "Figure 4: Using a group for factorization":

dog = {
    identity,
    leash-length: float,
}

identity = (
    age: int,
    name: tstr,
)

Is it equivalent to?

dog = {
    age: int,
    name: tstr,
    identity,
    leash-length: float,
}

{...} being a map, I would think so. But the syntax is a bit confusing. I find the (...) a bit confusing. It can represent a group, choice/enum, substitution (in the case above).

It would have been more logic to define identity as:

identity = {
    age: int,
    name: tstr,
}

A CDDL tool could check if we could only substitute map inside a map. Or group inside a group.

ABNF needs to be fixed to allow the usual (C99) syntax for hex floats:

0x4711.0815p+42
0xdead.beefp-53

Now that CBOR allows protocols to specify that CBOR values that are distinct in the basic generic data model are equivalent in the protocol's specific data model, CDDL should specify which of those data models .eq and.ne use, or explicitly allow CBOR-based protocols to decide.

Given the RFC 8610 status of CDDL, it may be prudent to track active implementations. Not sure if it is best to list CDDL implementations at http://cbor.io/impls.html or somewhere else, but figured here would be a good first step.

Will also contribute my own attempt at implementing the spec in Rust as a start :)

Rust: https://github.com/anweiss/cddl (Crates.io) (as of 08/12/2019, partial implementation)
Ruby: https://rubygems.org/gems/cddl/

The below cddl spec will throw an error when run with the cddl tool. Maybe I am using the language wrong but I cannot work out from the error message where my mistake lies.

What confuses me is that if b is an anonymous group this works fine.

a = {
  b
}

b = (
  (x: int // y: int), z: int
)

The command and its output:

λ cddl test.cddl generate
/Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1031:in `grpent': [[:grpchoice, [[:member, 1, 1, [:text, "x"], [:type1, [:prim, 0], [:prim, 1]]]], [[:member, 1, 1, [:text, "y"], [:type1, [:prim, 0], [:prim, 1]]]]], [:member, 1, 1, [:text, "z"], [:type1, [:prim, 0], [:prim, 1]]]] (RuntimeError)
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1239:in `block (2 levels) in group'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1239:in `each'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1239:in `flat_map'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1239:in `block in group'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1238:in `map'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1238:in `group'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1189:in `block in type1'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1189:in `each'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1189:in `flat_map'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1189:in `type1'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:933:in `block in r_process'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:922:in `map'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:922:in `r_process'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:225:in `rules'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:254:in `generate'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/bin/cddl:68:in `block in <top (required)>'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/bin/cddl:67:in `times'
        from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/bin/cddl:67:in `<top (required)>'
        from /usr/local/bin/cddl:22:in `load'
        from /usr/local/bin/cddl:22:in `<main>'

Looks like the discussions feature is not turned on for this repository or I would use it instead of making an issue.

Trying to gather my bearings as I get back up to speed on modeling

I'm curious how those working on CDDL might compare CDDL to CUE.
https://cuelang.org/docs/usecases/datadef/

@mjkoster @beriberikix @vitprajzler @alvarowolfx

https://tools.ietf.org/html/draft-ietf-cbor-cddl-02#section-2.1.2 says

An entry consists of a keytype and a valuetype

But the grammar says

grpent = [occur S] [memberkey S] type optcom
/ [occur S] groupname [genericarg] optcom ; preempted by above
/ [occur S] "(" S group S ")" optcom

which allows entries that aren't just a key and a value.

e.g. RTC (RoT-C)

The current CDDL specification uses in "Figure 5: Personal Data: Example for extensibility" this example:

NameComponents = (
	? firstName: tstr,
	? familyName: tstr,
)

How do you know when parsing a CBOR buffer which tstr is which one if the CBOR buffer only contains one tstr?

The group as it does not use key, there is no way to identify which tstr is represented.

What I am looking at is mapping the CDDL to my source code. Choices can currently be expressed in different forms with different syntax.
And it looks to me CDDL is more orienting towards CBOR validation rather than CBOR description. Am I right? Is it intented?

Here are examples from the current CDDL specification:

attire = "bow tie" / "necktie" / "Internet attire"
protocol = 6 / 17
delivery = (
	street: tstr, ? number: uint, city //
	po-box: uint, city //
	per-pickup: true
)

attire /= "swimwear"

delivery //= (
	lat: float, long: float, drone-type: tstr
)

basecolors = (
                black: 0, red: 1,  green: 2,  yellow: 3,
                blue: 4,  magenta: 5,  cyan: 6,  white: 7,
              )
extended-color = &(
                basecolors,
                orange: 8,  pink: 9,  purple: 10,  brown: 11,
              )

To clean it, first I will remove the choice define as a group (used by basecolors in the examples). The CDDL spec defines groups in the context of map or array. While the example basecolors would not have a sense at all in the context of map or array. If you want to keep the syntax, we could use =&(...) to always define "complex" choice.

While the example attire and protocol are only defined for CBOR data validation, the other examples are more used to structure CBOR data.

My suggestion would be to redefine the "choice" definition and have only two types of choices:

• the (validation) choice mainly uses for CBOR data validation. The syntax choice1 / choice2 / ... could be kept
• the named choice mainly uses for CBOR data representation. The named choice would cover all other choices and would require an annoted name for each entry. The name would not be used in the CBOR representation. The syntax |name1: choice1 | name2: choice2 | ... | could be used to represent named choice (| is often used to represent or operation in programming language).

So the examples above would be converted into:

delivery = | full-address: ( street: tstr, ? number: uint, city)
           | po-address: (po-box: uint, city)
           | pickup: per-pickup: true

delivery =| coordonate: (lat: float, long: float, drone-type: tstr)


basecolors =| black: 0 | red: 1 |  green: 2 |  yellow: 3 | blue: 4 |  magenta: 5 |  cyan: 6 |  white: 7
extended-color =| basecolors | orange: 8 | pink: 9 | purple: 10 | brown: 11

Using this syntax I would easily link these definitions to my source code. For instance in C, I would have:

enum basecolors { BLACK = 0, RED = 1, GREEN = 2, ...}
enum extended_color { BLACK = 0, RED = 1, GREEN = 2, ..., ORANGE = 8, ...}
struct delivery {
	enum { FULL_ADDRESS, PO_ADDRESS, PICKUP, DRONE } type;
	union {
		struct { char* street; int number, char* city; } full_address;
		struct { int po_box; char* city; } po_address;
		struct { bool per_pickup } pickup;
		struct { float lat; float long; char* drone_type; } drone;
	}
}

Using the current CDDL syntax, it is impossible to link group choice to source code. And it makes hard to maintain the changes.

I don't currently see any support for co-occurrence constraints in the spec. Would be great to allow for values on the right-hand side of control operators to be cross-references to other map/group entry types in the same instance document.

https://mailarchive.ietf.org/arch/msg/cbor/RzUEpK5pRMNX4aKRZrsbfSd4MaI

See: https://github.com/cbor-wg/cddl/blob/master/CONTRIBUTING.md#other-resources

Is cbor mailing-list used instead? I have just seen some CDDL related messages on it: https://mailarchive.ietf.org/arch/browse/cbor/

(As long as an extended diagnostic notation is being developed as part of the CDDL spec:)

It would be nice to have a notation for byte strings that are really encoded CBOR items or sequences of such (matching what .cbor and .cborseq do at the CDDL level).

One motivating example:
core-wg/oscore#100

Today, the spec is written for CBOR, and just implies that CDDL can also be used with JSON.
Since that is happening, add an appendix that makes the information explicit which parts of CDDL remain unused with JSON.

Initial attempt of a list:

• CBOR tags
• byte-string constants (h'beef' etc.)
• prelude elements that based on major types #2 and #6 (bstr/bytes, all the tags from tdate to cbor-any)

When mapping the JSON data model (which is not distinguishing integer from floating-point numbers) to that of CBOR, numbers with a zero fractional part become integers. Details of this may also need some additional information. Probably also point to I-JSON (RFC7493).

Perhaps some spec-specific definitions in section 1.2 - Terminology

...and clarify Bert's role.

As far as I can see the specification does not specify how to include other CDDL files into a CDDL file. Is it expected? It could be expected if the user is responsible to concatenate the CDDL files before processing it.

Complex CDDL description might be easier to read/manage when split into multiple file.

Does the CDDL type 1 include CBOR floating values equal to 1.0? Does the range 1..2 include 1.5? Does the CDDL type 1.0 include the CBOR integer value 1?

This seems likely to cause interoperability problems if it's not specified, especially now that the CBOR spec is clear that integer 1 is a different value from floating 1.0.

The group matching rule suggested in https://www.ietf.org/mail-archive/web/cbor/current/msg00380.html is wrong, but it's not obvious to me what the right one is. If there's a normative definition of PEG, referring to that is sufficient as long as this specification says how to turn a group into a PEG grammar.

It may be necessary to specify array-group matching separately from map-group matching, because of the ordering differences.

Can a range of sizes be applied to an integral type?, e.g. uint .size (3..5)? Probably not.

Is it meaningful to apply a size to a floating type? e.g. float .size 4? Maybe that would mean that values that can't be expressed in a single-precision float aren't allowed?

Does [*uint] .size 7 mean that the array has to be represented in exactly 7 bytes?