pjk / libcbor Goto Github PK
View Code? Open in Web Editor NEWCBOR protocol implementation for C
License: MIT License
CBOR protocol implementation for C
License: MIT License
We need to set the baseline
There are build functions for bools, but no reading functions. I expected something like cbor_get_bool
but failed to find any.
In file included from /home/drasko/cbor/libcbor/test/assertions.c:8:0:
/home/drasko/cbor/libcbor/test/assertions.h:4:20: fatal error: cmocka.h: No such file or directory
Hello,
I create the negative integer as such:
cbor_item_t *hkdf_alg = cbor_new_int8();
cbor_mark_negint(hkdf_alg);
int abs_i = abs(ECDH_SS_HKDF_256) - 1;
cbor_set_uint8(hkdf_alg, abs_i);
Where ECDH_SS_HKDF_256 = -27
.
Then when calling int8_t item = cbor_get_uint8(hkdf_alg)
I get a positive value. The strange thing is that when I copy paste the output of cbor for the hkdf_alg
(which I have written to a file) to cbor.me I can see that the value is -27.
Am I missing something?
"CMakeFiles/cbor_shared.dir/cbor.c.o: file not recognized: File format not recognized"
I get this error when linking C library libcbor.so
Can someone help me ? thank you
Im running Debian i386
Can I build this library on a 32bit machine ?
Thank you for your time
Maybe it's worth it to 'specialize' for cbor_load
?
Hello.
I found a NULL pointer dereference in libcbor.
Please confirm.
Thanks.
Summary: NULL pointer dereference
OS: CentOS 7 64bit
Version: commit f4ec5a4
PoC Download: crash.zip
Steps to reproduce:
1.Download the .POC files.
2.Compile the source code with ASan.
3.Execute the following command
: ./readfile $POC
ASAN:DEADLYSIGNAL
=================================================================
==31346==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000020 (pc 0x0000004f32d7 bp 0x7ffe7ef988f0 sp 0x7ffe7ef988d0 T0)
==31346==The signal is caused by a READ memory access.
==31346==Hint: address points to the zero page.
#0 0x4f32d6 in cbor_typeof /home/karas/libcbor/src/cbor/common.c:61:15
#1 0x4e9683 in _cbor_nested_describe /home/karas/libcbor/src/cbor.c:260:10
#2 0x4e9df8 in _cbor_nested_describe /home/karas/libcbor/src/cbor.c:353:4
#3 0x4e963e in cbor_describe /home/karas/libcbor/src/cbor.c:388:2
#4 0x4e7d5d in main /home/karas/libcbor/examples/readfile.c:77:2
#5 0x7fdca2c113f0 in __libc_start_main /build/glibc-mXZSwJ/glibc-2.24/csu/../csu/libc-start.c:291
#6 0x41ba09 in _start (/home/karas/libcbor/examples/readfile+0x41ba09)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/karas/libcbor/src/cbor/common.c:61:15 in cbor_typeof
==31346==ABORTING
================
[Acknowledgement]
This work was supported by ICT R&D program of MSIP/IITP. [R7518-16-1001, Innovation hub for high Performance Computing]
otool says:
/usr/local/opt/libcbor/lib/libcbor.0.dylib (compatibility version 0.0.0, current version 0.0.0)
I found that there is used by default GCC option -std=c99 and this option causes automatic definition of __STRICT_ANSI__
macro.
Because in MinGW some typedefs are protected by using this macro then they are not visible and library cannot be compiled i.e.:
#ifndef __STRICT_ANSI__
typedef __int64 off64_t;
#endif /* __STRICT_ANSI__ */
I try to define C_FLAGS in CMake configuration but user C_FLAGS is put before predefined one and then I cannot override them.
After I changed CMAKE_C_FLAGS in the CMakeLists.txt to:
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -std=gnu99")
then everything build without problems.
cmake 5.5.2
mingw32-gcc 3.9.3.-1
$ mingw32-make cbor
[ 7%] Building C object src/CMakeFiles/cbor.dir/cbor.c.obj
In file included from c:\mingw\include\unistd.h:36:0,
from C:\tmp\libcbor-0.4.0\src\cbor.c:245:
c:\mingw\include\io.h:301:14: error: unknown type name 'off64_t'
__CRT_INLINE off64_t lseek64 (int, off64_t, int);
^
c:\mingw\include\io.h:301:36: error: unknown type name 'off64_t'
__CRT_INLINE off64_t lseek64 (int, off64_t, int);
^
c:\mingw\include\io.h:302:14: error: unknown type name 'off64_t'
__CRT_INLINE off64_t lseek64 (int fd, off64_t offset, int whence) {
^
c:\mingw\include\io.h:302:39: error: unknown type name 'off64_t'
__CRT_INLINE off64_t lseek64 (int fd, off64_t offset, int whence) {
^
In file included from C:\tmp\libcbor-0.4.0\src\cbor.c:245:0:
c:\mingw\include\unistd.h:65:20: error: unknown type name 'off_t'
int ftruncate(int, off_t);
^
src\CMakeFiles\cbor.dir\build.make:62: recipe for target 'src/CMakeFiles/cbor.dir/cbor.c.obj' failed
mingw32-make[3]: *** [src/CMakeFiles/cbor.dir/cbor.c.obj] Error 1
CMakeFiles\Makefile2:1037: recipe for target 'src/CMakeFiles/cbor.dir/all' failed
mingw32-make[2]: *** [src/CMakeFiles/cbor.dir/all] Error 2
CMakeFiles\Makefile2:1049: recipe for target 'src/CMakeFiles/cbor.dir/rule' failed
mingw32-make[1]: *** [src/CMakeFiles/cbor.dir/rule] Error 2
Makefile:571: recipe for target 'cbor' failed
mingw32-make: *** [cbor] Error 2
Doxyfile contains
PROJECT_NUMBER = 0.3.1
would please be so kind and correct it with next relase?
Thanks
Currently annoying to do due to old binutils, which results in linker issues (see e.g. Project-OSRM/osrm-backend#3216)
/usr/bin/gcc-6 -fPIC -std=c99 -g -O0 -Wall -pedantic -g -ggdb -DDEBUG=true -fsanitize=address -fsanitize=bounds -fsanitize=alignment -shared -Wl,-soname,libcbor.so.0.5.0 -o libcbor.so.0.5.0 CMakeFiles/cbor_shared.dir/cbor.c.o CMakeFiles/cbor_shared.dir/cbor/streaming.c.o CMakeFiles/cbor_shared.dir/cbor/internal/encoders.c.o CMakeFiles/cbor_shared.dir/cbor/internal/builder_callbacks.c.o CMakeFiles/cbor_shared.dir/cbor/internal/loaders.c.o CMakeFiles/cbor_shared.dir/cbor/internal/memory_utils.c.o CMakeFiles/cbor_shared.dir/cbor/internal/stack.c.o CMakeFiles/cbor_shared.dir/cbor/internal/unicode.c.o CMakeFiles/cbor_shared.dir/cbor/encoding.c.o CMakeFiles/cbor_shared.dir/cbor/serialization.c.o CMakeFiles/cbor_shared.dir/cbor/arrays.c.o CMakeFiles/cbor_shared.dir/cbor/common.c.o CMakeFiles/cbor_shared.dir/cbor/floats_ctrls.c.o CMakeFiles/cbor_shared.dir/cbor/bytestrings.c.o CMakeFiles/cbor_shared.dir/cbor/callbacks.c.o CMakeFiles/cbor_shared.dir/cbor/strings.c.o CMakeFiles/cbor_shared.dir/cbor/maps.c.o CMakeFiles/cbor_shared.dir/cbor/tags.c.o CMakeFiles/cbor_shared.dir/cbor/ints.c.o CMakeFiles/cbor_shared.dir/allocators.c.o
/usr/bin/x86_64-linux-gnu-ld: unrecognized option '--push-state'
More constrained clients benefit from knowing the sizes upfront.
Jacob Teplitsky has contributed the following snippet. It should be quite easy to integrate it.
#include <string.h>
#include "platform.h"
#include "cbor.h"
#include "cbor/internal/builder_callbacks.h"
#include "cbor/internal/loaders.h"
#include "cJSON.h"
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#ifndef CBOR_LOAD_CALLBACK_T_DEFINED
typedef struct cbor_decoder_result(*cbor_load_callback_t)(void *, size_t, const struct cbor_callbacks *, void *);
#endif
cbor_item_t *cjson_cbor_load(void *source,
size_t source_size,
struct cbor_load_result *result,
cbor_load_callback_t *cbor_load_callback)
{
/* Context stack */
static struct cbor_callbacks callbacks = {
.uint8 = &cbor_builder_uint8_callback,
.uint16 = &cbor_builder_uint16_callback,
.uint32 = &cbor_builder_uint32_callback,
.uint64 = &cbor_builder_uint64_callback,
.negint8 = &cbor_builder_negint8_callback,
.negint16 = &cbor_builder_negint16_callback,
.negint32 = &cbor_builder_negint32_callback,
.negint64 = &cbor_builder_negint64_callback,
.byte_string = &cbor_builder_byte_string_callback,
.byte_string_start = &cbor_builder_byte_string_start_callback,
.string = &cbor_builder_string_callback,
.string_start = &cbor_builder_string_start_callback,
.array_start = &cbor_builder_array_start_callback,
.indef_array_start = &cbor_builder_indef_array_start_callback,
.map_start = &cbor_builder_map_start_callback,
.indef_map_start = &cbor_builder_indef_map_start_callback,
.tag = &cbor_builder_tag_callback,
.null = &cbor_builder_null_callback,
.undefined = &cbor_builder_undefined_callback,
.boolean = &cbor_builder_boolean_callback,
.float2 = &cbor_builder_float2_callback,
.float4 = &cbor_builder_float4_callback,
.float8 = &cbor_builder_float8_callback,
.indef_break = &cbor_builder_indef_break_callback
};
if (source_size == 0) {
result->error.code = CBOR_ERR_NODATA;
return NULL;
}
struct _cbor_stack stack = _cbor_stack_init();
/* Target for callbacks */
struct _cbor_decoder_context context = (struct _cbor_decoder_context) {
.stack = &stack,
.creation_failed = false,
.syntax_error = false
};
struct cbor_decoder_result decode_result;
*result = (struct cbor_load_result) {.read = 0, .error = {.code = CBOR_ERR_NONE}};
do {
if (source_size > result->read) { /* Check for overflows */
decode_result = (*cbor_load_callback)(
source, 24,
&callbacks,
&context);
} else {
result->error = (struct cbor_error) {
.code = CBOR_ERR_NOTENOUGHDATA,
.position = result->read
};
goto error;
}
switch (decode_result.status) {
case CBOR_DECODER_FINISHED:
/* Everything OK */
{
result->read += decode_result.read;
break;
}
case CBOR_DECODER_NEDATA:
/* Data length doesn't match MTB expectation */
{
result->error.code = CBOR_ERR_NOTENOUGHDATA;
goto error;
}
case CBOR_DECODER_EBUFFER:
/* Fallthrough */
case CBOR_DECODER_ERROR:
/* Reserved/malformated item */
{
result->error.code = CBOR_ERR_MALFORMATED;
goto error;
}
}
if (context.creation_failed) {
/* Most likely unsuccessful allocation - our callback has failed */
result->error.code = CBOR_ERR_MEMERROR;
goto error;
} else if (context.syntax_error) {
result->error.code = CBOR_ERR_SYNTAXERROR;
goto error;
}
} while (stack.size > 0);
/* Move the result before free */
cbor_item_t *result_item = context.root;
return result_item;
error:
result->error.position = result->read;
/*debug_print("Failed with decoder error %d at %d\n",
* result->error.code, result->error.position); */
/* cbor_describe(stack.top->item, stdout); */
/* Free the stack */
while (stack.size > 0) {
cbor_decref(&stack.top->item);
_cbor_stack_pop(&stack);
}
return NULL;
}
struct cbor_decoder_result cjson_cbor_stream_decode(void *source_void, size_t source_size,
const struct cbor_callbacks *callbacks,
void *context)
{
cJSON *source = source_void;
/* If we have no data, we cannot read even the MTB */
if (source_size < 1) {
return (struct cbor_decoder_result) {0, CBOR_DECODER_EBUFFER};
}
/* If we have a byte, assume it's the MTB */
struct cbor_decoder_result result = {1, CBOR_DECODER_FINISHED};
const char *name = source->string;
if (name) {
callbacks->string(context, (unsigned char *)name, strlen(name));
}
switch (source->type) {
case cJSON_False:
{
callbacks->boolean(context, false);
return result;
}
case cJSON_True:
{
callbacks->boolean(context, false);
return result;
}
/* case cJSON_NULL: */
case cJSON_Number:
{
callbacks->uint32(context, source->valueint);
return result;
}
case cJSON_String:
{
int res;
struct sockaddr_in sw_addr;
size_t len = 0;
const char *str;
str = source->valuestring;
res = inet_pton(AF_INET, str, &sw_addr.sin_addr);
if (res > 0) {
len = 4;
} else {
res = inet_pton(AF_INET6, str, &sw_addr.sin_addr);
if (res > 0) {
len = 16;
}
}
if (len == 0) {
callbacks->string(context, (unsigned char *)source->valuestring, strlen(source->valuestring));
} else {
callbacks->byte_string(context, (unsigned char *)&sw_addr.sin_addr, len);
}
return result;
}
case cJSON_Array:
{
callbacks->array_start(context, cJSON_GetArraySize(source));
cJSON *item;
cJSON_Foreach_Array_Item_Begin(source, item)
{
cjson_cbor_stream_decode(item , 2, callbacks, context);
}
cJSON_Foreach_Array_Item_End(source, item)
return result;
}
case cJSON_Object:
{
callbacks->map_start(context, cJSON_GetArraySize(source));
cJSON *item;
cJSON_Foreach_Object_Item_Begin(source, item)
{
cjson_cbor_stream_decode(item , 2, callbacks, context);
}
cJSON_Foreach_Object_Item_End(source, item)
return result;
}
default: /* Never happens - this shuts up the compiler */
{
return result;
}
}
}
cbor_item_t *
cjson_to_cbor(cJSON *json)
{
struct cbor_load_result result;
cbor_load_callback_t call = &cjson_cbor_stream_decode;
return (cjson_cbor_load((void *)json, 10, &result, &call));
}
static cbor_item_t * _cbor_copy_int(cbor_item_t * item, bool negative)
{
cbor_item_t * res;
switch (cbor_int_get_width(item)) {
case CBOR_INT_8: res = cbor_build_uint8(cbor_get_uint8(item)); break;
case CBOR_INT_16: res = cbor_build_uint16(cbor_get_uint16(item)); break;
case CBOR_INT_32: res = cbor_build_uint32(cbor_get_uint32(item)); break;
case CBOR_INT_64: res = cbor_build_uint64(cbor_get_uint64(item)); break;
default: return NULL;
}
if (negative)
cbor_mark_negint(res);
return res;
}
static cbor_item_t * _cbor_copy_float_ctrl(cbor_item_t * item)
{
switch (cbor_float_get_width(item)) {
case CBOR_FLOAT_0:
return cbor_build_ctrl(cbor_ctrl_value(item));
case CBOR_FLOAT_16:
return cbor_build_float2(cbor_float_get_float2(item));
case CBOR_FLOAT_32:
return cbor_build_float4(cbor_float_get_float4(item));
case CBOR_FLOAT_64:
return cbor_build_float8(cbor_float_get_float8(item));
}
return NULL;
}
cbor_item_t *cbor_copy_definite(cbor_item_t * item)
{
if (item == NULL) {
return (NULL);
}
switch (cbor_typeof(item)) {
case CBOR_TYPE_UINT:
return _cbor_copy_int(item, false);
case CBOR_TYPE_NEGINT:
return _cbor_copy_int(item, true);
case CBOR_TYPE_BYTESTRING:
if (cbor_bytestring_is_definite(item)) {
return cbor_build_bytestring(cbor_bytestring_handle(item), cbor_bytestring_length(item));
} else {
size_t i;
cbor_item_t * res = cbor_new_indefinite_bytestring();
for (i = 0; i < cbor_bytestring_chunk_count(item); i++)
cbor_bytestring_add_chunk(
res,
cbor_move(
cbor_copy_definite(cbor_bytestring_chunks_handle(item)[i])
)
);
return res;
}
case CBOR_TYPE_STRING:
if (cbor_string_is_definite(item)) {
return cbor_build_stringn((const char *) cbor_string_handle(item), cbor_string_length(item));
} else {
size_t i;
cbor_item_t * res = cbor_new_indefinite_string();
for (i = 0; i < cbor_string_chunk_count(item); i++)
cbor_string_add_chunk(
res,
cbor_move(
cbor_copy_definite(cbor_string_chunks_handle(item)[i])
)
);
return res;
}
case CBOR_TYPE_ARRAY: {
cbor_item_t * res;
size_t i;
if (cbor_array_size(item) == 0) {
res = cbor_new_null();
} else {
res = cbor_new_definite_array(cbor_array_size(item));
for (i = 0; i < cbor_array_size(item); i++)
cbor_array_push(
res,
cbor_move(cbor_copy_definite(cbor_move(cbor_array_get(item, i))))
);
}
return res;
}
case CBOR_TYPE_MAP: {
cbor_item_t * res;
size_t i;
res = cbor_new_definite_map(cbor_map_size(item));
struct cbor_pair * it = cbor_map_handle(item);
for (i = 0; i < cbor_map_size(item); i++)
cbor_map_add(res, (struct cbor_pair) {
.key = cbor_move(cbor_copy_definite(it[i].key)),
.value = cbor_move(cbor_copy_definite(it[i].value))
});
return res;
}
case CBOR_TYPE_TAG:
return cbor_build_tag(
cbor_tag_value(item),
cbor_move(cbor_copy(cbor_tag_item(item)))
);
case CBOR_TYPE_FLOAT_CTRL:
return _cbor_copy_float_ctrl(item);
}
return NULL;
}
All cbor_new_* and cbor_build_* functions can segfault in case memory allocation fails
(reported by an external user)
Hi,
I'm new to CBOR and I'm having some trouble. I'm trying to encode something like this, and I'm not able to successfully implement it.
{
var_1 : 0
var_2 : 1
var_3 : "some_string"
var_4 :
{
var_5 : true
var_6 : true
}
}
I'm not able to encode var_1, what type should I use to encode value 0?
I'm able to encode vars 2 to 3 but I can not encode vars 4 to 6. Is it possible to do that with this library?
Thank you in advance
Lots of undefined references in building some of the examples:
.../Library/Homebrew/shims/mac/super/clang -std=c99 -O3 -flto -Wall -pedantic -DNDEBUG -Wl,-search_paths_first -Wl,-headerpad_max_install_names -flto CMakeFiles/readfile.dir/readfile.c.o -o readfile ../src/libcbor.a
undef: _cbor_build_bool
undef: _cbor_build_uint8
undef: _cbor_build_string
undef: _cbor_move
undef: _cbor_new_definite_map
undef: _cbor_map_add
undef: _cbor_serialize_alloc
undef: _cbor_decref
Undefined symbols for architecture x86_64:
"_cbor_build_bool", referenced from:
_main in lto.o
"_cbor_build_uint8", referenced from:
_main in lto.o
"_cbor_build_string", referenced from:
_main in lto.o
"_cbor_move", referenced from:
_main in lto.o
"_cbor_new_definite_map", referenced from:
_main in lto.o
"_cbor_map_add", referenced from:
_main in lto.o
"_cbor_serialize_alloc", referenced from:
_main in lto.o
"_cbor_decref", referenced from:
_main in lto.o
ld: symbol(s) not found for architecture x86_64
undef: _cbor_new_definite_array
undef: _cbor_get_uint8
undef: _cbor_build_uint8
undef: _cbor_array_push
undef: _cbor_move
undef: _cbor_array_handle
undef: _cbor_array_size
undef: _cbor_describe
Undefined symbols for architecture x86_64:
"_cbor_new_definite_array", referenced from:
_main in lto.o
"_cbor_get_uint8", referenced from:
_comparUint in lto.o
"_cbor_build_uint8", referenced from:
_main in lto.o
"_cbor_array_push", referenced from:
_main in lto.o
"_cbor_move", referenced from:
_main in lto.o
"_cbor_array_handle", referenced from:
_main in lto.o
"_cbor_array_size", referenced from:
_main in lto.o
"_cbor_describe", referenced from:
_main in lto.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [examples/create_items] Error 1
make[1]: *** [examples/CMakeFiles/create_items.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
undef: _cbor_empty_callbacks
undef: _cbor_stream_decode
Undefined symbols for architecture x86_64:
"_cbor_empty_callbacks", referenced from:
_main in lto.o
"_cbor_stream_decode", referenced from:
_main in lto.o
ld: symbol(s) not found for architecture x86_64
make[2]: *** [examples/sort] Error 1
make[1]: *** [examples/CMakeFiles/sort.dir/all] Error 2
undef: _cbor_load
undef: _cbor_describe
undef: _cbor_decref
Undefined symbols for architecture x86_64:
"_cbor_load", referenced from:
_main in lto.o
"_cbor_describe", referenced from:
_main in lto.o
"_cbor_decref", referenced from:
_main in lto.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [examples/streaming_parser] Error 1
make[1]: *** [examples/CMakeFiles/streaming_parser.dir/all] Error 2
make[2]: *** [examples/readfile] Error 1
make[1]: *** [examples/CMakeFiles/readfile.dir/all] Error 2
[100%] Built target hello
cd .../tmp/libcbor-20190119-19049-1m752yw/libcbor-0.5.0/build/src && .../Cellar/cmake/3.13.3/bin/cmake -E cmake_symlink_library libcbor.0.0.0.dylib libcbor.0.dylib libcbor.dylib
[100%] Built target cbor_shared
make: *** [all] Error 2
b263837 disables ARM CI because of changes in the Rapsbian image we used to use for QEMU. Either fix that setup, or use Docker to fully virtualize the build (travis-ci/travis-ci#3376)
Cloning into 'test/cmocka'...
warning: templates not found /usr/local/git/share/git-core/templates
fatal: unable to connect to git.cryptomilk.org:
git.cryptomilk.org[0: 78.46.80.163]: errno=Operation timed out
In current master/src/cbor/encoding.c
line 140, there appears to be a constant encoded value for 16-bit float NaN value as 0x00e700
. I think this is a typo as RFC 7049 in Section 3.9 states that
If NaN is an allowed value, it must always be represented as 0xf97e00
Notice the first data octet difference e7
vs 7e
.
The same 7e00
encoding comes up in the RFC 7049 NaN example in Appendix A.
Also add docs on how to use it
Way too painful + causes problems such as PJK/libcbor-ruby#2
Hello.
I found a heap-buffer-overflow bug in libcbor.
Please confirm.
Thanks.
OS: CentOS 7 64bit
PoC Download: crash_overflow.zip
Steps to reproduce:
1.Download the .POC files.
2.Compile the source code with ASan.
3.Execute the following command
: ./streaming_parser $POC
=================================================================
==28627==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000002a at pc 0x0000004a7e60 bp 0x7ffeeb255cd0 sp 0x7ffeeb255480
READ of size 5 at 0x60300000002a thread T0
#0 0x4a7e5f in printf_common(void*, char const*, __va_list_tag*) /home/karas/src/llvm/projects/compiler-rt/lib/ASan/../sanitizer_common/sanitizer_common_interceptors_format.inc:544
#1 0x4a8bda in __interceptor_vprintf /home/karas/src/llvm/projects/compiler-rt/lib/ASan/../sanitizer_common/sanitizer_common_interceptors.inc:1388
#2 0x4a8c97 in printf /home/karas/src/llvm/projects/compiler-rt/lib/ASan/../sanitizer_common/sanitizer_common_interceptors.inc:1434
#3 0x5145f7 in find_string /home/karas/ASan/libcbor/examples/streaming_parser.c:31:3
#4 0x514d11 in cbor_stream_decode /home/karas/ASan/libcbor/src/cbor/streaming.c
#5 0x5148c9 in main /home/karas/ASan/libcbor/examples/streaming_parser.c:56:19
#6 0x7f589bf27c04 in __libc_start_main (/lib64/libc.so.6+0x21c04)
#7 0x41b71b in _start (/home/karas/ASan/libcbor/examples/streaming_parser+0x41b71b)
0x60300000002a is located 0 bytes to the right of 26-byte region [0x603000000010,0x60300000002a)
allocated by thread T0 here:
#0 0x4df0b6 in __interceptor_malloc /home/karas/src/llvm/projects/compiler-rt/lib/ASan/ASan_malloc_linux.cc:66
#1 0x514805 in main /home/karas/ASan/libcbor/examples/streaming_parser.c:48:27
#2 0x7f589bf27c04 in __libc_start_main (/lib64/libc.so.6+0x21c04)
SUMMARY: AddressSanitizer: heap-buffer-overflow /home/karas/src/llvm/projects/compiler-rt/lib/ASan/../sanitizer_common/sanitizer_common_interceptors_format.inc:544 in printf_common(void*, char const*, __va_list_tag*)
Shadow bytes around the buggy address:
0x0c067fff7fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c067fff7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c067fff7fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c067fff7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c067fff7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c067fff8000: fa fa 00 00 00[02]fa fa fa fa fa fa fa fa fa fa
0x0c067fff8010: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c067fff8020: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c067fff8030: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c067fff8040: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c067fff8050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==28627==ABORTING
=================
[Acknowledgement]
This work was supported by ICT R&D program of MSIP/IITP. [R7518-16-1001, Innovation hub for high Performance Computing]
Hi,
I've tried to integrate the licbor into a QT lib. I've been running in several issues, all related in C99 C++11 related features. The lib is statically build and could be integrated easily, except that the headers generate errors while building:
Using -std=c++11 nor -std=c99 (which fails for c++ / qt anyhow) doesn't solve the issue.
What can be done to solve it?
Is it fine for libcbor to be buildable with MSVC (at least with VS 2013)?
It is possible, but the cost is losing the C99 compliance:
restrict
keyword with MSVC-specific __restrict
.inline
- replace it with __inline
.These two replacements can be made via CMakeLists:
if(MSVC)
set(CBOR_RESTRICT "__restrict")
set(CBOR_INLINE "__inline")
else()
set(CBOR_RESTRICT "restrict")
set(CBOR_INLINE "inline")
endif()
#include <stdlib.h>
instead of #include <unistd.h>
.const
parameter qualifier to cbor_new_definite_map/array
implementations (typo?).When running ctest on i686 arch, there is failed test_float8 with cbor_float_get_float8(float_ctrl) == 1.0e+300, on x64_86 arch works as expected.
build_test log attached
Hi, first of all, thanks for this great library.
It looks that both cbor_decref
and cbor_move
decrement number of references, however, when refcount
reaches zero, cbor_decref
will destroy an object. It seems that it is possible to make refcount
equal to zero by using cbor_move
, so I was wondering how a cbor_item_t
with zero reference count could be used apart from moving it's content into cbor map or cbor array?
Thanks!
I see you have created .deb files, but I don't see the debian/ directory in git. Is it possible to have that committed as well, so I can get this setup with my internal repository? I need to compile this for amd64 and arm64, and I'd prefer to not have to recreate the configs you already have so far if possible.
Thanks.
To allow easy translation from other formats.
To prevent e.g. #35
I tried to test the example cjson2cbor.c. I used cjson.h from https://github.com/DaveGamble/cJSON
and I built the library into the project. If I run this example with the input json_example.cjson:
{
"a": [1, 2.2, -1, "žluťoučký kůň!!!"],
"b": {
"c": [true, false, null]
}
}
the corresponding output is:
?aa?@
??;vžluťoučký kůň!!!ab?ac????
but verifying with the tool cbor.me I expected something like this:
A2 # map(2)
61 # text(1)
61 # "a"
84 # array(4)
01 # unsigned(1)
FB 400199999999999A # primitive(4612136378390124954)
20 # negative(0)
76 # text(22)
C5BE6C75C5A56F75C48D6BC3BD206BC5AFC588212121 # "\xC5\xBElu\xC5\xA5ou\xC4\x8Dk\xC3\xBD k\xC5\xAF\xC5\x88!!!"
61 # text(1)
62 # "b"
A1 # map(1)
61 # text(1)
63 # "c"
83 # array(3)
F5 # primitive(21)
F4 # primitive(20)
F6 # primitive(22)
I note that the hexadecimal numbers are converted in ASCII. For example 61 = a
but A2 should be =ì instead of =?. Same error for 84 and many others numbers.
Hi,
I'm trying to parse CBOR data and get the values of each field. I receive a payload from a client and already have the entire CBOR. I tried following the examples in https://libcbor.readthedocs.io/en/v0.5.0/using.html and https://libcbor.readthedocs.io/en/v0.5.0/streaming/decoding.html, but does not work. Is there an example of data parsing, getting the data from each field of the cbor?
Thank you
Recently tried wrap your lib in objective - c (for iOS/MAC OS client - server exchange purposes) and bump into couple issues conformance with RFC 7049 standard.
First: wrong byte order after encoding
RFC 7049, Appendix A
//...
int value = 1000000;
cbor_item_t *cborItem = cbor_build_uint32(value);
unsigned char *buffer;
size_t buffer_size,
length = cbor_serialize_alloc(cborItem, &buffer, &buffer_size);
NSData *bufferData = [NSData dataWithBytes:(const void *)buffer length:sizeof(unsigned char)*length];
//...
Result binary data (in ascii hex): 0x1a40420f00
Must be (according to RFC 7049): 0x1a000f4240
_All bytes after first byte in reverse order._
Second: negative values of all wides of integers encoding incorrect. For example, if I even reorder bytes in correct way - result not confirms to RFC 7049:
//...
short value = -1000;
cbor_item_t *cborItem = cbor_build_uint16(value);
cbor_mark_negint(cborItem);
unsigned char *buffer;
size_t buffer_size,
length = cbor_serialize_alloc(cborItem, &buffer, &buffer_size);
NSData *bufferData = [NSData dataWithBytes:(const void *)buffer length:sizeof(unsigned char)*length];
//...
Result binary data (in ascii hex): 0x39fc18
Must be (according to RFC 7049): 0x3903e7
Third: floats encoding into something absolutely crazy. But any float encoded before with cbor_build_float4 can't be decoded with cbor_float_get_float4... It always returns 0.0
//...
case CBOR_TYPE_FLOAT_CTRL:
switch (cbor_float_get_width(itemFromData)) {
case CBOR_FLOAT_0:{
if (cbor_ctrl_is_bool(itemFromData)) {
resultNumber = [NSNumber numberWithBool:cbor_ctrl_value(itemFromData)];
}
}
break;
case CBOR_FLOAT_16:
resultNumber = [NSNumber numberWithFloat:cbor_float_get_float2(itemFromData)];
break;
case CBOR_FLOAT_32:
resultNumber = [NSNumber numberWithFloat:cbor_float_get_float4(itemFromData)];
break;
case CBOR_FLOAT_64:
resultNumber = [NSNumber numberWithDouble:cbor_float_get_float8(itemFromData)];
break;
}
break;
//...
There is chance, that I am using your lib in some wrong way, but I quit trying understand this... Maybe you can help me?
E.g. http://lcamtuf.coredump.cx/afl/ via #15
cbor_serialize
returns 0 if data size was insufficient.
It would be useful to also get some report of how many more bytes should be supplied to finish serialization.
Or maybe additional function (something like size_t cbor_calculate(const cbor_item *)
) should be added that calculates required data size.
cbor_serialize_alloc
provides something similar, but it does it in another way. What I want is to alloc only once, and exactly as many bytes as it is required to store serialized data.
E.g. http://libcbor.readthedocs.io/en/latest/api/item_types.html
Something must have changed in RTD setup
File findings_000100 causes a heap-buffer-overflow in cbor_new_definite_array
in arrays.c line 111
cbor_item_t ** data = _CBOR_MALLOC(sizeof(cbor_item_t *) * size);
if (data == NULL) {
_CBOR_FREE(item);
return NULL;
}
for (size_t i = 0; i < size; i++)
data[i] = NULL; // <==
size
is 0x8000000000000000
and i
is 0x4198
on crash
a2 9b 80 00 00 00 00 00 00 00 00 00 00 00 00 00
#0 0x0000000000406578 in cbor_new_definite_array (size=0x8000000000000000) at libcbor/src/cbor/arrays.c:111
#1 0x0000000000405925 in cbor_builder_array_start_callback (context=0x7fffffffe480, size=0x8000000000000000)
at libcbor/src/cbor/internal/builder_callbacks.c:254
#2 0x00000000004048ba in cbor_stream_decode (source=0x60e251 "\233\200", source_size=0xf, callbacks=0x60d0e0 <callbacks>, context=0x7fffffffe480)
at libcbor/src/cbor/streaming.c:413
#3 0x00000000004030b6 in cbor_load (source=0x60e250 "\242\233\200", source_size=0x10, result=0x7fffffffe4e0) at libcbor/src/cbor.c:68
#4 0x0000000000402fb0 in main (argc=0x2, argv=0x7fffffffe5e8) at libcbor/examples/readfile.c:37
#5 0x00007ffff7a36f45 in __libc_start_main (main=0x402ee5 <main>, argc=0x2, argv=0x7fffffffe5e8, init=<optimized out>, fini=<optimized out>,
rtld_fini=<optimized out>, stack_end=0x7fffffffe5d8) at libc-start.c:287
#6 0x0000000000402e09 in _start ()
==9484== ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60040000dfd0 at pc 0x40a33b bp 0x7fffa1348c20 sp 0x7fffa1348c18
WRITE of size 8 at 0x60040000dfd0 thread T0
#0 0x40a33a (readfile_asan+0x40a33a)
#1 0x4087bd (readfile_asan+0x4087bd)
#2 0x405e6c (readfile_asan+0x405e6c)
#3 0x4036be (readfile_asan+0x4036be)
#4 0x40344e (readfile_asan+0x40344e)
#5 0x7f70ab268f44 (/lib/x86_64-linux-gnu/libc-2.19.so+0x21f44)
#6 0x403208 (readfile_asan+0x403208)
0x60040000dfd1 is located 0 bytes to the right of 1-byte region [0x60040000dfd0,0x60040000dfd1)
allocated by thread T0 here:
#0 0x7f70ab62141a (/usr/lib/x86_64-linux-gnu/libasan.so.0.0.0+0x1541a)
#1 0x40a2dc (readfile_asan+0x40a2dc)
#2 0x4087bd (readfile_asan+0x4087bd)
#3 0x405e6c (readfile_asan+0x405e6c)
#4 0x4036be (readfile_asan+0x4036be)
#5 0x40344e (readfile_asan+0x40344e)
#6 0x7f70ab268f44 (/lib/x86_64-linux-gnu/libc-2.19.so+0x21f44)
Shadow bytes around the buggy address:
0x0c00ffff9ba0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c00ffff9bb0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c00ffff9bc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c00ffff9bd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c00ffff9be0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c00ffff9bf0: fa fa fa fa fa fa fa fa fa fa[01]fa fa fa 00 00
0x0c00ffff9c00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c00ffff9c10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c00ffff9c20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c00ffff9c30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c00ffff9c40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Found with American Fuzzy Lop
After a fresh install:
ls -h /usr/local/Cellar/libcbor/0.5.0/lib
libcbor.0.0.0.dylib libcbor.0.dylib libcbor.a libcbor.dylib pkgconfig
I'm now integrating libcbor library into my iOS project and I had all the source files included in the project. When I try to compile the library I got many errors:
many thanks for any suggestion.
The file findings_000000 (hex dump below) causes the readline
example to crash with a segfault due to a NULL pointer in common.c cbor_typeof
carried over from main.c
00000000 e3 64 68 61 6c 66 f9 00 00 66 73 69 6e 67 6c 65 |.dhalf...fsingle|
00000010 fa 7f 7f ff ff 6d 73 69 6d 70 6c 65 20 76 61 6c |.....msimple val|
00000020 75 65 73 83 f5 f4 f6 |ues....|
00000027
#0 0x000000000040670b in cbor_typeof (item=0x0) at libcbor/src/cbor/common.c:61
#1 0x000000000040370d in _cbor_nested_describe (item=0x0, out=0x7ffff7dd4400 <_IO_2_1_stdout_>, indent=0x0) at libcbor/src/cbor.c:260
#2 0x0000000000403e06 in cbor_describe (item=0x0, out=0x7ffff7dd4400 <_IO_2_1_stdout_>) at libcbor/src/cbor.c:388
#3 0x0000000000402fca in main (argc=0x2, argv=0x7fffffffe5c8) at libcbor/examples/readfile.c:39
#4 0x00007ffff7a36f45 in __libc_start_main (main=0x402ee5 <main>, argc=0x2, argv=0x7fffffffe5c8, init=<optimized out>, fini=<optimized out>,
rtld_fini=<optimized out>, stack_end=0x7fffffffe5b8) at libc-start.c:287
#5 0x0000000000402e09 in _start ()
The docs do not mention the cmocka dependency in the Building & Installing Prerequisites.
Also for cmake-newbies like me it would be nice to clarify the '# Assuming you are in the directory where you want to build' comment. I did not know where I wanted to build.. Turns out cmake expects you to create a build directory, and then run:
cmake -DCMAKE_BUILD_TYPE=Release ../
Just trying to smoothen the onboarding here ;-)
PS: I also added -DCMAKE_INSTALL_PREFIX:PATH=$HOME to the cmake commandline, since I don't want to actually install each library I try out. You may want to clarify that root permissions are not really required for installation.
The RFC indicates how to do canonical CBOR serialization. Is this something that libcbor can support? It shouldn't be hard to have it sort the keys by byte order in maps to get this supported
Cmake conf is done in a way that installs .so (or .so.X) files in /usr/lib regardless of target architecture (i686/x86_64). It should be placed in /usr/lib OR /usr/lib64 respectively.
Thanks
The subject says it. It does not look like there's anything special in my environment, it's a pretty stock FreeBSD 11.1 amd64 installation.
After git clone, creating a temp dir and running cmake from it, make cbor is successful, but make cbor_shared fails with:
[ 95%] Building C object src/CMakeFiles/cbor_shared.dir/cbor/ints.c.o
[100%] Linking C shared library libcbor.so
/usr/bin/ld: unrecognized option '-plugin'
/usr/bin/ld: use the --help option for usage information
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error code 1
Is there a way to access the underlying cbor information including type information? I know that with (cbor_item_t)->data I can access the data of the item. But I am also interested in the type information, especially for arrays.
Is there a way?
From here: http://tools.ietf.org/html/rfc7049#appendix-A
There is this example:
0.00006103515625 -> 0xf90400
It turns out the left hand side is 2^-14.
Using libcbor 0.3.1 on XCode simulator (little-endian), cbor_serialize_alloc fails after cbor_build_float2 for the provided number.
Actually it appears to get into an endless loop until aborted by an illegal malloc request.
Here's a function that demonstrates the issue:
Attached is an image showing my test program and the results.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.