cbor / cbor.github.io Goto Github PK

View Code? Open in Web Editor NEW

74.0 12.0 33.0 410 KB

cbor.io web site

HTML 100.00%

cbor.github.io's Introduction

cbor.github.io

cbor.io web site

cbor.github.io's People

Contributors

Stargazers

Watchers

cbor.github.io's Issues

Indicate what implementations support canonical

It would be really handy to be able to know how to use and know which implementations support canonical encoding.

List of companies using CBOR

Hey there! I'm interested in compiling a (small) list of companies using CBOR in production for research purposes, and Google is not helping much. Are you aware of a bunch companies using the format?

cbor.me decoder internal error on bignum inputs

The following sequence is not correctly handled:

a2 9b 80 00 00 00 00 00 00 00 00 00 00 00 00 00

(Also see PJK/libcbor#16)

JSON->CBOR using decimal fractions?

I have long arrays of short numbers in JSON. To give a small example:

[3.1, 15.2, -80.6, 40.7]

cbor.me converts it to 37 bytes (more than in the input!):

84                     # array(4)
   FB 4008CCCCCCCCCCCD # primitive(4614162998222441677)
   FB 402E666666666666 # primitive(4624746457346762342)
   FB C054266666666666 # primitive(13858744174572365414)
   FB 404459999999999A # primitive(4630924833085561242)

I think that with decimal fractions the result would be more concise.
Is there any ready-to-use converter that automatically uses decimal fractions where it makes sense?

As part of my MSc dissertation at University of Oxford, I wrote and published two papers covering the characteristics of various binary serialization formats, including CBOR and performing a space-efficiency benchmark, respectively.

Sharing them here in case anybody finds them interesting! The first paper explains how CBOR works including an annotated hexadecimal example and the second compares CBOR to various other popular serialization formats.

A Survey of JSON-compatible Binary Serialization Specifications: https://arxiv.org/abs/2201.02089
A Benchmark of JSON-compatible Binary Serialization Specifications: https://arxiv.org/abs/2201.03051

All the best!

discuss JSON <--> CBOR roundtripping on the website.

a major reason to adopt CBOR is as a way to represent JSON on the wire + on storage. it would be nice to very clearly see what the problems with roundtripping CBOR -> JSON -> CBOR and JSON -> CBOR -> JSON might be.

Is the root element required to be a map or an array?

In JSON, it seems to be expected that the root element is a map or an array. From reading the RFC for CBOR, I can’t find any mention of a requirement like this; ie, that a map or array is the first object which then holds other objects. Is this required and I just missed it? If not, what is the method to convert a CBOR stream without a map or array as the root object to JSON?

unclear when indefinite length could be bad

there's some subtle hints about indefinite length items not being desirable in some circumstances, but it's very unclear why.
Actually i'm unsure why i would ever use definite lengths at all.

Using indefinite-length encoding allows
an encoder to not need to marshal all the data for counting, but it
requires a decoder to allocate increasing amounts of memory while
waiting for the end of the item.

for unconstrained devices: why even bother. server memory is basically infinite. json works this way as well.
for constrained devices: knowing the array is too large earlier in the byte-stream doesn't give you more options to deal with the problem. It's still too large. Also the parsed tree's size is vaguely related to the byte-streams size, so the problem already hit you before parsing the cbor.

someone also complained on reddit (cant find the link anymore) that indefinite length items are a design mistake in cbor, but i can't figure out why. They seem fine.

spec: ambiguity with tags before `ff` breaks

Putting 9f c0 ff into http://cbor.me/ diagnostic tool says "break stop code outside indefinite length item", but that error message doesn't sound right: the 9f is an indefinite length item. The problem is that the c0 tag should precede a data item and the ff break stop code isn't a data item.

That's assuming that it's actually an error to precede ff with tags. I don't think the spec explicitly says this, other than section 2.4's "a data item can optionally be preceded by a tag". It doesn't explicitly reject the converse: tags that don't precede a data item.

Tangentially, that quote ends with "a tag" not "tags" plural, and tags are also not a data item, so according to a literal intepretation of rejecting the converse, the c1 in c1 c2 03 is invalid. On the other hand, section 2.4 goes on to say "if tag A is followed by tag B, which is followed by data item C, tag A applies to the result of applying tag B on data item C", so it sounds like tags (plural) is valid. The test-vectors don't cover this but http://cbor.me/ accepts c1 c2 03. Is it that, when spec language lawyering, c2 03 combined should be considered a data item, which is preceded by c1?

float pointer error

I use [http://cbor.me/] to verify my cbor f/w , and i found a issue about float point to cbor...

the pattern is:

BF455465737432FA3FB5E354FF

the ideal output might be:

{h'5465737432': 1.421000}

but actually output as:

{h'5465737432': 1.4210000038146973}

the float point to hex data is reference this link:
[http://gregstoll.dyndns.org/~gregstoll/floattohex/]

Please add to implementations - Revived `cbor-js` as `cbor-redux`

The implementation for browser cbor-js has been unmaintained since 2016, and all other implementations had specific dependencies on the Node.js runtime when I needed it for both browser and Deno.

I have forked cbor-js and created cbor-redux. All outstanding reported bugs on the original repo are fixed, additional tests have been written to cover code 100%, it's been rewritten it in TypeScript, and released for browser, Deno, and Node.

Plan is to maintain this and get it even with node-cbor and updates to the CBOR spec over the next couple of months.

Links to releases:
https://www.npmjs.com/package/cbor-redux
https://deno.land/x/cbor_redux
https://www.skypack.dev/npm/cbor-redux

Request to include jsoncons in C++ implementations page

jsoncons is a modern C++, header-only library for JSON construction that supports encode to/decode from CBOR.

Anchors and links within a CBOR object?

I'm not sure if this is the right place for this; please tell me where to report this if it isn't.

For one of the projects I'm working on I need to be able to serialize an object graph. This is a generic directed graph that cannot be converted into a tree (something that CBOR can easily represent natively). I know that CBOR can be extended relatively easily; are there any plans to add anchors and links to the CBOR specification? Not ones that reach outside of the single CBOR-encoded byte string, but only within it, kind of like how pointers within a single flat address space only reference objects within the same address space.

I know that I can create my own tags to do this, but if someone else is already doing something along these lines I'd rather reuse their work.

EDIT

I should have dug around more. In the IANA registry, there is reference to tags 28 & 29, which are what I was looking for. Closing this issue now.

Skip-scan Java implementation

Hey guys,

please add my skip-scan parser implementation and fluent data builder to the implementation list. It is build in Java https://github.com/noctarius/borabora.

Documentation and first EA upcoming :)

Thanks,
Chris

Issue while encoding in Swift (IOS)

There is no function available for encoding a string into CBOR in SWIFT(IOS). Only decoder is available in Swift.

List cyborg as a Java library

I've been working on a Java library called cyborg, at https://github.com/dwaite/cyborg.

Additional information on the library:

The initial layer 'electrode' works as a tokenizer and supports stream-based parsing and generation. I am using it in multiple related projects to read and write data with simple schemas. I wanted more flexibility than you typically would get with a library-dictated object model such as with XML or JSON Java tooling, as I thought these didn't pair well with CBOR's richer core data type or with extensible semantic tagging.

I hope to eventually have layers on top for translation to and from other formats, as well as a potential eventual object mapper to allow one to convert Java objects to and from CBOR.

I currently consider the library as in alpha, and am targeting an initial beta release (with binary builds submitted to the maven central repository) on June 10th.

Please add Minicbor C implementation

Minicbor is a small C library for reading and writing CBOR. Uses callbacks for streamed decoding and encoding with no internal memory allocation.

https://minicbor.readthedocs.io/

Add flunn as Python implementation

I've forked flynn and and fixes some compatibility issues and have made some refactoring.
https://pypi.python.org/pypi/flunn

Kotlin know supports CBOR

Kotlin know supports CBOR as part of kotlinx-serialization:

https://kotlin.github.io/kotlinx.serialization/kotlinx-serialization-cbor/kotlinx-serialization-cbor/kotlinx.serialization.cbor/index.html

C, C++ implementation

I think you published a C implementation on github, and there are also an extra couple of good one.
Whould be nice to point to them, because the lack of opensource C implementation can be a show stopper for potential adopters

New Perl CBOR modules: CBOR::Free and CBOR::PP

Hello,

I’ve written a two new Perl CBOR implementations:
https://metacpan.org/pod/CBOR::Free
https://metacpan.org/pod/CBOR::PP

CBOR::Free is in XS. It’s a bit more restricted in scope relative to CBOR::XS, but it’s got a more permissive license and (unlike CBOR::XS) supports newer Perl versions.

It’s also, in my benchmarks, faster yet than CBOR::XS. :)

CBOR::PP was my first attempt at a CBOR library. I’m releasing it in case it’s of use to someone but anticipate that CBOR::Free will be what pretty much anyone prefers to use.

Very small typo

In the paragraph titled Extensible, the first sentence should start out with "To be able >>>to<<< grow..." not "To be able grow..." as it does now. Minor point, admittedly.

Does Cbor support schema evolution?

Can a writer and reader have different schema ?

CBOR implementation for Swift

I have written a library for CBOR on Swift and I have published it on my GitHub. Alongside with more than 30 different automated unit tests, it is being used practically in our project at the moment. It is working very well, and it is very easy to use for iOS projects (it is available and accessible on Cocoapods too).

Vancosys Data Security Inc.: www.vancosys.com
CBORSwift on GitHub: https://github.com/Hassaniiii/CBORSwift

A zero-memory-footprint C decoder

Here's a parser in use on some microcontroller/IoT deployments with no memory allocation:

https://github.com/quartzjer/cb0r

Implementation: (Swift) PotentCodables has a fully featured CBOR implementation

Qt İmplementation

Please add link to Qt İmplementation
https://github.com/anton-dutov/cbor-qt

Yet another one c++ implementation

Not sure if this: https://github.com/serge-klim/bobl C++ implementation would be useful to somebody. It's not exactly coder/decoder per se. It is more type based compile time encoders/decoders generator which uses CBOR as one of underlying protocols and its API is similar to recommended:

encoded = CBOR.encode(data) ➔ data = CBOR.decode(encoded) ( auto encoded = bobl::cbor::encode(data); ➔ auto data = bobl::cbor::decode(begin(encoded), end(encoded));)

New C# implementation

Hi,

Please add a CBOR implementation for .Net (C#):

Dahomey.Cbor
https://github.com/dahomey-technologies/Dahomey.Cbor

Thank you,

Michaël

cbor.io is accessible via HTTPS but the cert isn't right (github.io issue)

Chrome throws up a security warning because the SSL certificate for cbor.io is registered to a.ssl.fastly.net.

Official Qt implementation

Cbor support is added to official Qt repository. Please add/replace Qt link with link to official documentation, like https://doc.qt.io/qt-5/qcborvalue.html

Is there any diagnostic CBOR tools that can analyse defective CBOR payloads?

Most cbor tools just simply throws a non informative error message (e.g. cbor.me ).

Is there a CBOR tool that can spit out a diagnostic message that can work out and point out defective parts of the CBOR message?

The ruby cbor diagnostic tool just crashes if it encounter an error.

Update the recommended JS Browser implementation

The recommended JS browser implementation has not been maintained as it should. Please consider this fork which is more up-to-date:
https://github.com/aaronhuggins/cbor-redux
(I am not associated with this project in any way - I just want the best CBOR implementation.)

JSON for Modern C++ 2.0.9 supports CBOR

The latest release of JSON for Modern C++ - a header-only C++ library that aims to promote JSON to a first-class data type in C++11 - now supports CBOR as (de)serialization format.

Examples for the usage:

It would be great if you could list us in the implementation section of your website. Furthermore, any feedback is greatly appreciated.

cbor.me off by one in comments for negative ints

For negative ints (major type 1) the diagnostic output is correct, but the comments on the right hand side look wrong. It looks like cbor.me is failing to subtract the -1 implicit in the negative int type.

For example, for hex 0x26 I see a comment of negative(6) and for 0x37 negative(23) (screenshots below).

If I'm just misunderstanding the comments then I think some clarifying message on the page would be helpful.

bad links again ...

many of the IETF and IANA (and a few other) links are missing the "https://" also, the "cbor.me" link is bad

Elixir CBOR

I've forked and updated your excbor package and was wondering if you could add it to the list of elixir implementations on cbor.io.

https://github.com/scalpel-software/cbor

Also, I think you may want to add additional test data for big integers. This area seems to be neglected in the appendix of the RFC.

Cross-collaboration with JSON5

Recently there has been a new JSON spec being drafted, might be a good idea to discuss in regards to binary data encodings.
json5/json5#190

GoldFish - Extremely fast CBOR (and JSON) lib

https://github.com/OneNoteDev/GoldFish - "JSON/CBOR streaming library, without using memory, like a GoldFish"

I tested in my machine, CBOR parsing performance is 968MB/s (compiled with GCC 7.1). It is 2.5 times faster than tinycbor in deserialize cbor.

I hope you can add it into http://cbor.io/impls.html

Link needs an update

C, C++
A CBOR implementation in C is part of the RIOT operating system for constrained nodes

Structure of the RIOT repository changed and link is not working.

A range of tags for private use

I'm currently designing an inter-process method invocation system, where CBOR will be the main message encoding. I need to tag references to remote objects in the messages, but it seems silly to get a tag assigned in https://docs.google.com/spreadsheets/d/1UajnKeIx_piJsAGRz5YJprvfwrBUeSVykqwG_JQL9rk/edit?usp=sharing. Since the tag for object references will only be used in my system, it seems a waste of the tag code space if I try to register my tag in the IANA registry.

In the case like mine, I think it's a common practice to reserve a range of tag values for private use. I think I might write a spec and reserve a range of tag values from the registry, but it's better to have it specified in the main CBOR spec, IMHO.

What do you think?

another implementation: lua-ConciseSerialization

lua-ConciseSerialization is a pure Lua implementation of CBOR.

Install via luarocks

See http://fperrad.github.io/lua-ConciseSerialization/

bad links again ...

many of the IETF and IANA (and a few other) links are missing the "https://"

Different Byte strings decodes into same UTF string, is that correct?

Hello! We at Radix are using CBOR. I just found strange behavior in encoding and decoding of this UTF string:
"radix.particles.message"

The Cbor playground cbor.me encodes that into the byte string

77
72616469782e7061727469636c65732e6d657373616765

Which also both SwiftCBOR and node-cbor do.

But Java Jacksson Cbor encodes it to:

7817
72616469782e7061727469636c65732e6d657373616765

Notice the difference, the prefix 77 versus 7817.

The interesting part is that both Cbor.me and SwiftCBOR decodes 781772616469782e7061727469636c65732e6d657373616765 into the same UTF string: "radix.particles.message".

In other words, two different byte strings decode into the same UTF string.

Is that correct?

According to the RFC 7049, the major type is 3, i.e. 0b011 followed by the length of that string, which is 23 is decimal, which results in 0b10111 in binary, and 01110111 in hex is 77.

So where does 7817 come from? And why does it correctly decode into the same UTF string?

cddl-gen project from Nordic

It'd be great to add the cddl-gen project from Nordic:

https://github.com/NordicSemiconductor/cddl-gen

OCaml has CBOR implementations

The OCaml programming language has a couple of notable CBOR implementations.

The cbor package is a minimal and lightweight implementation of just the core CBOR data structures. It encodes and decodes to and from OCaml strings.
The orsetto package is a serialization framework that includes streaming encoders and decoders for both JSON and CBOR.

Implemented one more CBOR parser in C

https://github.com/vi/simple_cbor_stream_parse

Although hacky and sketchy, it should be embedded-friendly, as it does not use memory allocations and can limit all numbers to 16 or 32 bits.

Not sure if it is good enough to be listed on cbor.io, but I don't see any other place to submit CBOR implementations to.

Java implementation : libcbor

libcbor

Hi,

I have made a fully featured CBOR stream serializer/parser with some unique functionality
that makes this library particularly suitable for network processing.

https://github.com/Marlinski/libcbor/

please add this a CBOR implementation for Java.