cbor-wg / cddl Goto Github PK
View Code? Open in Web Editor NEWThe draft leading up to RFC8610
Home Page: https://tools.ietf.org/html/rfc8610
License: Other
The draft leading up to RFC8610
Home Page: https://tools.ietf.org/html/rfc8610
License: Other
Since they're an extension point, do we need a registry to list extensions?
https://tools.ietf.org/html/draft-ietf-cbor-cddl-02#section-3.7 includes:
basic-header = {
field1: int,
field2: text,
}
advanced-header = {
~basic-header,
field3: bytes,
field4: ~time,
}
(Note that leaving out the first unwrap operator in the latter
example would lead to nesting the basic-header in its own map inside
the advanced-header,…
However, maps require a key name, so
advanced-header = { basic-header, … }
is a syntax error.
Maybe the example should be an array?
In section 2, the description of the _table_
composition model was confusing on first read.
Perhaps it is because the first example involves words that can be interpreted in several ways, such as tag
, and language
, or that it uses words like key
without prior definition. For people with a JavaScript background, perhaps describing table as being like a Map
(rather than Object
) would help.
In the YAML specification and related map tag definition, the word mapping
is used for the representation style with name-value pairs, while map
is used for what I think you are calling _table_
.
Jim: "When writing down a CDDL
grammar, keep it as simple as possible while still expressing the needed
language. "
Carsten: "Just because CDDL makes it easier to handle complexity in a specification, that does not make that complexity somehow less bad (except on the level of the humans having to grasp the complex structure while reading the spec)."
(Two different issues, but both probably worth a mention.)
https://mailarchive.ietf.org/arch/msg/cbor/xp1-0QT3MQFxAZ6C3v1vAfgzUzs
If spec authors rely on CDDL checking to verify that structures have an expected shape, the lack of general cuts could cause malicious data to get to places that don't expect it. For example, if someone were to define an extensible array:
arr = [*elem]
elem /= { type: "foo", val: tstr .size 16 }
elem /= { type: "bar", val: uint }
elem /= { type: tstr, val: any } ; For future extensions.
then the array [{type: "foo", val: "0123456789ABCDEFbuffer overflow here"}]
is a valid instance of arr
. We should warn specification authors and implementers to be careful to avoid that.
Considering this example from "Figure 4: Using a group for factorization":
dog = {
identity,
leash-length: float,
}
identity = (
age: int,
name: tstr,
)
Is it equivalent to?
dog = {
age: int,
name: tstr,
identity,
leash-length: float,
}
{...}
being a map, I would think so. But the syntax is a bit confusing. I find the (...)
a bit confusing. It can represent a group, choice/enum, substitution (in the case above).
It would have been more logic to define identity as:
identity = {
age: int,
name: tstr,
}
A CDDL tool could check if we could only substitute map inside a map. Or group inside a group.
ABNF needs to be fixed to allow the usual (C99) syntax for hex floats:
0x4711.0815p+42
0xdead.beefp-53
Now that CBOR allows protocols to specify that CBOR values that are distinct in the basic generic data model are equivalent in the protocol's specific data model, CDDL should specify which of those data models .eq
and.ne
use, or explicitly allow CBOR-based protocols to decide.
Given the RFC 8610 status of CDDL, it may be prudent to track active implementations. Not sure if it is best to list CDDL implementations at http://cbor.io/impls.html or somewhere else, but figured here would be a good first step.
Will also contribute my own attempt at implementing the spec in Rust as a start :)
Rust: https://github.com/anweiss/cddl (Crates.io) (as of 08/12/2019, partial implementation)
Ruby: https://rubygems.org/gems/cddl/
The below cddl spec will throw an error when run with the cddl tool. Maybe I am using the language wrong but I cannot work out from the error message where my mistake lies.
What confuses me is that if b is an anonymous group this works fine.
a = {
b
}
b = (
(x: int // y: int), z: int
)
The command and its output:
λ cddl test.cddl generate
/Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1031:in `grpent': [[:grpchoice, [[:member, 1, 1, [:text, "x"], [:type1, [:prim, 0], [:prim, 1]]]], [[:member, 1, 1, [:text, "y"], [:type1, [:prim, 0], [:prim, 1]]]]], [:member, 1, 1, [:text, "z"], [:type1, [:prim, 0], [:prim, 1]]]] (RuntimeError)
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1239:in `block (2 levels) in group'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1239:in `each'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1239:in `flat_map'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1239:in `block in group'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1238:in `map'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1238:in `group'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1189:in `block in type1'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1189:in `each'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1189:in `flat_map'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:1189:in `type1'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:933:in `block in r_process'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:922:in `map'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:922:in `r_process'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:225:in `rules'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/lib/cddl.rb:254:in `generate'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/bin/cddl:68:in `block in <top (required)>'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/bin/cddl:67:in `times'
from /Library/Ruby/Gems/2.3.0/gems/cddl-0.8.5/bin/cddl:67:in `<top (required)>'
from /usr/local/bin/cddl:22:in `load'
from /usr/local/bin/cddl:22:in `<main>'
Looks like the discussions feature is not turned on for this repository or I would use it instead of making an issue.
Trying to gather my bearings as I get back up to speed on modeling
I'm curious how those working on CDDL might compare CDDL to CUE.
https://cuelang.org/docs/usecases/datadef/
https://tools.ietf.org/html/draft-ietf-cbor-cddl-02#section-2.1.2 says
An entry consists of a keytype and a valuetype
But the grammar says
grpent = [occur S] [memberkey S] type optcom
/ [occur S] groupname [genericarg] optcom ; preempted by above
/ [occur S] "(" S group S ")" optcom
which allows entries that aren't just a key and a value.
e.g. RTC (RoT-C)
The current CDDL specification uses in "Figure 5: Personal Data: Example for extensibility" this example:
NameComponents = (
? firstName: tstr,
? familyName: tstr,
)
How do you know when parsing a CBOR buffer which tstr
is which one if the CBOR buffer only contains one tstr
?
The group as it does not use key, there is no way to identify which tstr
is represented.
What I am looking at is mapping the CDDL to my source code. Choices can currently be expressed in different forms with different syntax.
And it looks to me CDDL is more orienting towards CBOR validation rather than CBOR description. Am I right? Is it intented?
Here are examples from the current CDDL specification:
attire = "bow tie" / "necktie" / "Internet attire"
protocol = 6 / 17
delivery = (
street: tstr, ? number: uint, city //
po-box: uint, city //
per-pickup: true
)
attire /= "swimwear"
delivery //= (
lat: float, long: float, drone-type: tstr
)
basecolors = (
black: 0, red: 1, green: 2, yellow: 3,
blue: 4, magenta: 5, cyan: 6, white: 7,
)
extended-color = &(
basecolors,
orange: 8, pink: 9, purple: 10, brown: 11,
)
To clean it, first I will remove the choice define as a group (used by basecolors
in the examples). The CDDL spec defines groups
in the context of map
or array
. While the example basecolors
would not have a sense at all in the context of map
or array
. If you want to keep the syntax, we could use =&(...)
to always define "complex" choice.
While the example attire
and protocol
are only defined for CBOR data validation, the other examples are more used to structure CBOR data.
My suggestion would be to redefine the "choice" definition and have only two types of choices:
choice1 / choice2 / ...
could be kept|name1: choice1 | name2: choice2 | ... |
could be used to represent named choice (|
is often used to represent or
operation in programming language).So the examples above would be converted into:
delivery = | full-address: ( street: tstr, ? number: uint, city)
| po-address: (po-box: uint, city)
| pickup: per-pickup: true
delivery =| coordonate: (lat: float, long: float, drone-type: tstr)
basecolors =| black: 0 | red: 1 | green: 2 | yellow: 3 | blue: 4 | magenta: 5 | cyan: 6 | white: 7
extended-color =| basecolors | orange: 8 | pink: 9 | purple: 10 | brown: 11
Using this syntax I would easily link these definitions to my source code. For instance in C, I would have:
enum basecolors { BLACK = 0, RED = 1, GREEN = 2, ...}
enum extended_color { BLACK = 0, RED = 1, GREEN = 2, ..., ORANGE = 8, ...}
struct delivery {
enum { FULL_ADDRESS, PO_ADDRESS, PICKUP, DRONE } type;
union {
struct { char* street; int number, char* city; } full_address;
struct { int po_box; char* city; } po_address;
struct { bool per_pickup } pickup;
struct { float lat; float long; char* drone_type; } drone;
}
}
Using the current CDDL syntax, it is impossible to link group choice to source code. And it makes hard to maintain the changes.
I don't currently see any support for co-occurrence constraints in the spec. Would be great to allow for values on the right-hand side of control operators to be cross-references to other map/group entry types in the same instance document.
See: https://github.com/cbor-wg/cddl/blob/master/CONTRIBUTING.md#other-resources
Is cbor
mailing-list used instead? I have just seen some CDDL related messages on it: https://mailarchive.ietf.org/arch/browse/cbor/
(As long as an extended diagnostic notation is being developed as part of the CDDL spec:)
It would be nice to have a notation for byte strings that are really encoded CBOR items or sequences of such (matching what .cbor and .cborseq do at the CDDL level).
One motivating example:
core-wg/oscore#100
Today, the spec is written for CBOR, and just implies that CDDL can also be used with JSON.
Since that is happening, add an appendix that makes the information explicit which parts of CDDL remain unused with JSON.
Initial attempt of a list:
When mapping the JSON data model (which is not distinguishing integer from floating-point numbers) to that of CBOR, numbers with a zero fractional part become integers. Details of this may also need some additional information. Probably also point to I-JSON (RFC7493).
Perhaps some spec-specific definitions in section 1.2 - Terminology
...and clarify Bert's role.
As far as I can see the specification does not specify how to include other CDDL files into a CDDL file. Is it expected? It could be expected if the user is responsible to concatenate the CDDL files before processing it.
Complex CDDL description might be easier to read/manage when split into multiple file.
Does the CDDL type 1
include CBOR floating values equal to 1.0
? Does the range 1..2
include 1.5
? Does the CDDL type 1.0
include the CBOR integer value 1
?
This seems likely to cause interoperability problems if it's not specified, especially now that the CBOR spec is clear that integer 1
is a different value from floating 1.0
.
The group matching rule suggested in https://www.ietf.org/mail-archive/web/cbor/current/msg00380.html is wrong, but it's not obvious to me what the right one is. If there's a normative definition of PEG, referring to that is sufficient as long as this specification says how to turn a group into a PEG grammar.
It may be necessary to specify array-group matching separately from map-group matching, because of the ordering differences.
Can a range of sizes be applied to an integral type?, e.g. uint .size (3..5)
? Probably not.
Is it meaningful to apply a size to a floating type? e.g. float .size 4
? Maybe that would mean that values that can't be expressed in a single-precision float aren't allowed?
Does [*uint] .size 7
mean that the array has to be represented in exactly 7 bytes?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.