GithubHelp home page GithubHelp logo

bergzand / nanocbor Goto Github PK

View Code? Open in Web Editor NEW
45.0 4.0 23.0 372 KB

CBOR library aimed at heavily constrained devices

License: Creative Commons Zero v1.0 Universal

C 94.51% Meson 2.01% Python 3.48%
cbor embedded c iot

nanocbor's Introduction

NanoCBOR

NanoCBOR is a tiny CBOR library aimed at embedded and heavily constrained devices. It is optimized for 32-bit architectures but should run fine on 8-bit and 16-bit architectures. NanoCBOR is optimized for decoding known CBOR structures while optimizing the flash footprint of both NanoCBOR and the code using NanoCBOR.

The decoder of NanoCBOR should compile to 600-800 bytes on a Cortex-M0+ MCU, depending on whether floating point decoding is required.

Compiling

Compiling NanoCBOR as is from this repository requires:

Furthermore, the tests make use of CUnit as test framework.

All of these can usually be found in the package repository of your Linux distribution.

Building NanoCBOR is a two-step process. First a build directory has to be created with the necessary Ninja build files:

meson build

Second step is to compile NanoCBOR:

ninja -C build

This results in a libnanocbor.so file inside the build directory and binaries for the examples and tests in their respective directories inside the build directory.

When including NanoCBOR into a custom project, it is usually sufficient to only include the source and header files into the project, the meson build system used in the repo is not mandatory to use.

Usage

To achieve the small code size, two patterns are used throughout the decode library.

  • Every decode call will first check the type and refuse to decode if the CBOR element is not of the required type.
  • Every decode call will, on successful decode, advance the decode context to the next CBOR element.

This allows using code to call decode functions and check the return code of the function without requiring an if value of type, decode value, advance to next item dance, and requiring only a single call to decode an expected type and advance to the next element.

Start the decoding of a buffer with:

nanocbor_value_t decoder;
nanocbor_decoder_init(&decoder, buffer, buffer_len);

Where buffer is a const uint8_t array containing a CBOR structure.

To decode an int32_t from a CBOR structure and bail out if the element is not of the integer type:

int32_t value = 0;
if (nanocbor_get_int32(&decoder, &value) < 0) {
  return ERR_INVALID_STRUCTURE;
}
return use_value(value);

Iterating over an CBOR array and calling a function passing every element is as simple as:

nanocbor_value_t arr; /* Array value instance */

if (nanocbor_enter_array(&decoder, &arr) < 0) {
    return ERR_INVALID_STRUCTURE;
}
while (!nanocbor_at_end(&arr)) {
    handle_array_element(&arr);
}

Decoding a map is similar to an array, except that every map entry consists of two CBOR elements requiring separate decoding. For example, a map using integers as keys and strings as values can be decoded with:

while (!nanocbor_at_end(&map)) {
    int32_t key;
    const char *value;
    size_t value_len;
    if (nanocbor_get_int32(&map, &integer_key) < 0) {
        return ERR_INVALID_STRUCTURE;
    }
    if (nanocbor_get_tstr(&map, &value, &value_len) < 0) {
        return ERR_INVALID_STRUCTURE;
    }
    handle_map_element(key, value, value_len);
}

Dependencies:

Only dependency are two functions to provide endian conversion. These are not provided by the library and have to be configured in the header file. On a bare metal ARM platform, __builtin_bswap64 and __builtin_bswap32 can be used for this conversion.

Contributing

Open an issue, PR, the usual.

nanocbor's People

Contributors

azymohliad avatar benpicco avatar bergzand avatar fjmolinas avatar jue89 avatar maribu avatar mcr avatar mguetschow avatar nagrawal63 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

nanocbor's Issues

automated and encode tests overlap?

It seems that there is a lot of duplicate code between:

% cd tests; diff automated/test_decoder.c encode/main.c

Is there a reason not to just merge it all into automated?

Error with the waspmote-pro board

When using NanoCBOR with the waspmote-pro board (in RIOT), I get the following error:

$ BOARD=waspmote-pro make -C tests/pkg_nanocbor
make: Entering directory '/home/jdavid/sandboxes/UiO/RIOT/tests/pkg_nanocbor'
Building application "tests_pkg_nanocbor" for "waspmote-pro" with MCU "atmega1281".

"make" -C /home/jdavid/sandboxes/UiO/RIOT/pkg/nanocbor/ 
"make" -C /home/jdavid/sandboxes/UiO/RIOT/build/pkg/nanocbor/src -f /home/jdavid/sandboxes/UiO/RIOT/Makefile.base MODULE=nanocbor
/home/jdavid/sandboxes/UiO/RIOT/build/pkg/nanocbor/src/encoder.c: In function ‘nanocbor_fmt_double’:
/home/jdavid/sandboxes/UiO/RIOT/build/pkg/nanocbor/src/encoder.c:324:21: error: array subscript ‘uint64_t {aka long long unsigned int}[0]’ is partly outside array bounds of ‘double[1]’ [-Werror=array-bounds]
  324 |     uint16_t exp = (*unum >> DOUBLE_EXP_POS) & DOUBLE_EXP_MASK;
      |                     ^~~~~
/home/jdavid/sandboxes/UiO/RIOT/build/pkg/nanocbor/src/encoder.c:321:57: note: while referencing ‘num’
  321 | int nanocbor_fmt_double(nanocbor_encoder_t *enc, double num)
      |                                                  ~~~~~~~^~~
cc1: all warnings being treated as errors
make[2]: *** [/home/jdavid/sandboxes/UiO/RIOT/Makefile.base:146: /home/jdavid/sandboxes/UiO/RIOT/tests/pkg_nanocbor/bin/waspmote-pro/nanocbor/encoder.o] Error 1
make[1]: *** [Makefile:9: all] Error 2
make: *** [/home/jdavid/sandboxes/UiO/RIOT/tests/pkg_nanocbor/../../Makefile.include:802: pkg-build] Error 2
make: Leaving directory '/home/jdavid/sandboxes/UiO/RIOT/tests/pkg_nanocbor'

This may be related to issue #18

No support for indefinite array

The decoder given in the test/fuzz isn't able to decode an indefinite array encoded as: nanocbor_fmt_array_indefinite(enc); nanocbor_fmt_int(enc, 8); nanocbor_fmt_end_indefinite(enc);

Wrong docstring for nanocbor_put_bstr

The docstring on nanocbor_put_bstr states that the return value is the number of bytes written.

int nanocbor_put_bstr(nanocbor_encoder_t *enc, const uint8_t *str, size_t len);

If you look at the source code the return value depends on the call to _fits which either returns 0 or -1. The docstring should probably be updated to something like:

nanocbor_skip() doesn't work on nested structures

According to its documentation it should:

"skip over nested structures in the CBOR stream such as (nested) arrays and maps"

However it doesn't seem to work. A simple test case such as the following fails:

nanocbor_value_t decoder;
uint8_t byteval = 0xA0; /* empty map, size 0 */

nanocbor_decoder_init(&decoder, &byteval, sizeof(byteval));
nanocbor_skip(&decoder);

CU_ASSERT_EQUAL(&byteval + 1, decoder.cur);

Investigating this I found something suspicious in _skip_limited. The part that recursively skips the content of the map/array is inside a conditional branch which is never entered. It checks if the return code from nanocbor_enter_map/array is > 0, which it never is, since those function either returns something < 0 or 0. Changing that expression to == 0, the test case works.

diff --git a/src/decoder.c b/src/decoder.c
index 6b69405..51a07cc 100644
--- a/src/decoder.c
+++ b/src/decoder.c
@@ -365,7 +365,7 @@ static int _skip_limited(nanocbor_value_t *it, uint8_t limit)
         res = (type == NANOCBOR_TYPE_MAP
                ? nanocbor_enter_map(it, &recurse)
                : nanocbor_enter_array(it, &recurse));
-        if (res > 0) {
+        if (res == 0) {
             while (!nanocbor_at_end(&recurse)) {
                 res = _skip_limited(&recurse, limit - 1);
                 if (res < 0) {

Is this actually an issue or have I misunderstood something about skip on nested structures?

nanocbor_get_key_tstr returns NANOCBOR_OK even when key is not present

Code for reproducing

#include<stdio.h>
#include<nanocbor/nanocbor.h>

int main(void) {
    char encode_buf[1024];
    nanocbor_encoder_t cbor_encoder;
    nanocbor_encoder_init(&cbor_encoder, encode_buf, 1024);

    nanocbor_fmt_map(&cbor_encoder, 2);
    nanocbor_put_tstr(&cbor_encoder, "key1");
    nanocbor_put_tstr(&cbor_encoder, "value1");
    nanocbor_put_tstr(&cbor_encoder, "key2");
    nanocbor_put_tstr(&cbor_encoder, "value2");

    nanocbor_value_t cbor_decoder;
    nanocbor_value_t cbor_encoded_val;
    nanocbor_value_t parsed_map;
    nanocbor_decoder_init(&cbor_decoder, encode_buf, nanocbor_encoded_len(&cbor_encoder));

    if(nanocbor_enter_map(&cbor_decoder, &parsed_map) == NANOCBOR_OK 
            && nanocbor_get_key_tstr(&parsed_map, "key3", &cbor_encoded_val) == NANOCBOR_OK) {
        printf("key3 found\n");
    } else {
        printf("key3 not found\n");
    }
}

This prints "key3 found". I haven't been able to figure out why this is happening.

nanocbor_enter_array() does not return NANOCBOR_OK on success

The documentation states

* @return NANOCBOR_OK on success

NANOCBOR_OK is defined to be 0.

However, nanocbor_enter_array() returns the result of _enter_container() which return 1 on success.

Checking the return value of nanocbor_enter_array() to match NANOCBOR_OK will therefore always fail.

Some operations are not 8/16 bit platform proof

Should be fixed by properly declaring the constants

"make" -C /tmp/dwq.0.7453450615585222/ad9984d6052f52acdf6a81056f4fa44d/build/pkg/nanocbor/src \
	-f /tmp/dwq.0.7453450615585222/ad9984d6052f52acdf6a81056f4fa44d/pkg/nanocbor/Makefile.nanocbor
encoder.c: In function '_single_is_zero':
encoder.c:210:34: error: left shift count >= width of type [-Werror=shift-count-overflow]
 #define FLOAT_SIGN_MASK      (1U << FLOAT_SIGN_POS)
                                  ^
encoder.c:212:35: note: in expansion of macro 'FLOAT_SIGN_MASK'
 #define FLOAT_IS_ZERO          (~(FLOAT_SIGN_MASK))
                                   ^
encoder.c:234:19: note: in expansion of macro 'FLOAT_IS_ZERO'
     return (num & FLOAT_IS_ZERO) == 0;
                   ^
encoder.c: In function 'nanocbor_fmt_double':
encoder.c:210:34: error: left shift count >= width of type [-Werror=shift-count-overflow]
 #define FLOAT_SIGN_MASK      (1U << FLOAT_SIGN_POS)
                                  ^
encoder.c:326:68: note: in expansion of macro 'FLOAT_SIGN_MASK'
         uint32_t single = (*unum >> (DOUBLE_SIZE - FLOAT_SIZE)) & (FLOAT_SIGN_MASK);
                                                                    ^
encoder.c:331:43: error: left shift count >= width of type [-Werror=shift-count-overflow]
         single |= ((exp & FLOAT_EXP_MASK) << FLOAT_EXP_POS) |
                                           ^
cc1: all warnings being treated as errors

Pre-buffer offset?

The separation of buffer (with its end) and length already allows having nanoCBOR encoded into the first block of a CoAP block-wise response. Would it be reasonable to extend the encoder by a field that counts how many initially encoded bytes to discard? (Similar to how the produced length tells how many bytes were discarded after the buffer).

I don't have a full use case yet, but looked at @silkeh's RIOT-OS/RIOT#17571, and that would appear to work great with CoAP block-wise. So this is not a full feature request yet, more like a "would it make sense to add this feature?"

(And it may happen that the next request is then for a checksum field that all the input is fed into, which would then generate ETags to detect whether the blocks can even be assembled).

Support double

Only floats seem to be supported currently. A function for doubles (nanocbor_fmt_double?) would be greatly appreciated!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.