GithubHelp home page GithubHelp logo

siemens / embb Goto Github PK

View Code? Open in Web Editor NEW
178.0 26.0 41.0 19.37 MB

Embedded Multicore Building Blocks (EMB²): Library for parallel programming of embedded systems. Star us on GitHub? +1

License: Other

CMake 2.67% C 35.43% C++ 58.76% Assembly 0.36% Lua 0.86% Shell 1.31% Cuda 0.02% Python 0.58%
embedded-systems multicore parallel-computing mtapi algorithms task-scheduler data-structures dataflow c-plus-plus c

embb's Introduction

Coverity Scan Status CII Best Practices

Embedded Multicore Building Blocks (EMB²)

Introduction

Overview

The Embedded Multicore Building Blocks (EMB²) are an easy to use yet powerful and efficient C/C++ library for the development of parallel applications. EMB² has been specifically designed for embedded systems and the typical requirements that accompany them, such as real-time capability and constraints on memory consumption. As a major advantage, low-level operations are hidden in the library which relieves software developers from the burden of thread management and synchronization. This not only improves productivity of parallel software development, but also results in increased reliability and performance of the applications.

EMB² is independent of the hardware architecture (x86, ARM, ...) and runs on various platforms, from small devices to large systems containing numerous processor cores. It builds on MTAPI, a standardized programming interface for leveraging task parallelism in embedded systems containing symmetric or asymmetric (heterogeneous) multicore processors. A core feature of MTAPI is low-overhead scheduling of fine-grained tasks among the available cores during runtime. Unlike existing libraries, EMB² supports task priorities and affinities, which allows the creation of soft real-time systems. Additionally, the scheduling strategy can be optimized for non-functional requirements such as minimal latency and fairness.

Besides the task scheduler, EMB² provides basic parallel algorithms, concurrent data structures, and skeletons for implementing stream processing applications (see figure below). These building blocks are largely implemented in a non-blocking fashion, thus preventing frequently encountered pitfalls like lock contention, deadlocks, and priority inversion. As another advantage in real-time systems, the algorithms and data structures give certain progress guarantees. For example, wait-free data structures guarantee system-wide progress which means that every operation completes within a finite number of steps independently of any other concurrent operations on the same data structure.

Building blocks of EMB²

Important Links

GitHub:

Repository:

Mailing list:

Community (help, bug reports, etc.):

Contact:

License

See the file COPYING.md in the project's root directory.

Contributions

See the file CONTRIBUTING.md in the project's root directory.

Build and Installation

General

It is strongly recommended to build from a release file and not from a repository snapshot in order to get the documentation and the examples out-of-the box. The release files can be found at https://github.com/siemens/embb/releases.

Platforms

EMB² is regularly built and tested on a variety of OS/compiler/architecture combinations including Linux (x86 and ARM using GCC/Clang) and Windows (x86 using MSVC). Moreover, it has been successfully built and tested on RTEMS and FreeBSD. Other platforms may be supported without any changes to the source code. The included unit tests can be used to find out whether a system not officially supported is suitable to run EMB². If the build process or the unit tests fail on your system, please contact us.

Prerequisites

The project is based on the standards C99 (for C code) and C++03 (for C++ code) to be usable on a wide range of target systems. Besides a C/C++ compiler supporting these standards, CMake 2.8.9 or higher is required to build EMB² (CMake is a build file generator which abstracts from the concrete build tools). It is possible to select the standards C11 (for C code) and C++11 (for C++ code) to enable the use of standard provided atomic operations.

Quick Installation on Linux

To generate and invoke the platform-specific build files, open a shell and change to the project's root directory. Create a subdirectory, where you want to build the library, e.g., "build", and change to that subdirectory. In the following, it is assumed that the project's root directory is the parent directory. Now you can generate the build files using CMake:

cmake ..

As the next step, compile EMB²:

cmake --build .

After compilation has finished, execute the tests:

binaries/run_tests.sh

Finally, install EMB² (the default path is /usr/local):

sudo cmake --build . --target install

Quick Installation on Windows

To generate and invoke the platform-specific build files, open a Developer Command Prompt for Visual Studio and change to the project's root directory. Create a subdirectory, where you want to build the library, e.g., "build", and change to that subdirectory. In the following, it is assumed that the project's root directory is the parent directory. Now you can generate the build files using CMake (a list of supported CMake generators can be displayed by typing cmake --help). For example:

cmake -G "Visual Studio 14 2015" ..

As the next step, compile EMB²:

cmake --build . --config Release

After compilation has finished, execute the tests:

binaries\run_tests.bat

Finally, install EMB² with administrator privileges:

cmake --build . --target install --config Release

Detailed Installation Instructions

EMB² provides several options which allow you to configure it to your needs. This section explains these options and describes the build process in more detail.

1. Generation of Native Build Files

As mentioned above, it is recommended to build EMB² in a subdirectory such as "build". The actual build files are generated by the following command (a list of available generators can be displayed by typing cmake --help):

cmake -G <generator> .. [OPTIONS]

Note that on Linux, the architecture (32/64 bit) cannot be selected by the generator. The default is "Unix Makefiles" for which reason -G <generator> may be omitted.

To select the C11 and C++11 standards use:

 cmake .. -DUSE_C11_AND_CXX11=ON

EMB² can be built in Release or Debug mode. The latter contains additional checks during runtime and is only recommended for development purposes. On Linux, the build mode can be specified using the option -DCMAKE_BUILD_TYPE=[Release|Debug], for example:

cmake .. -DCMAKE_BUILD_TYPE=Debug

If no build mode is given, the default (Release) is used. The Visual Studio generators create build files for both modes (the selection is done at build time as described below).

You may choose a custom compiler instead the default one by defining CMAKE_CXX_COMPILER and/or CMAKE_C_COMPILER. For example, to use Clang on Linux, type:

cmake .. -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang

In the same way, you may cross compile to another platform. For example, to cross compile to ARM v7 using GCC, you need to specify the cross compiler itself and the target architecture as an argument to the compiler:

cmake .. -DCMAKE_CXX_COMPILER=arm-linux-gnueabi-gcc++ \
         -DCMAKE_CXX_FLAGS=-march=armv7-a \
         -DCMAKE_C_COMPILER=arm-linux-gnueabi-gcc \
         -DCMAKE_C_FLAGS=-march=armv7-a

EMB² can be built with C++ exception handling (default) or without exceptions. When exceptions are turned off, a message is emitted in case of an error and the program aborts. To disable exceptions, add the option -DUSE_EXCEPTIONS=OFF.

Similarly, automatic initialization of the task scheduler by the MTAPI C++ interface can be disabled with -DUSE_AUTOMATIC_INITIALIZATION=OFF. This way, unexpected delays after startup can be avoided, e.g. for timing measurements.

EMB² comes with OpenCL and CUDA plugins to support execution on GPUs that may be built by setting -DBUILD_OPENCL_PLUGIN=ON and -DBUILD_CUDA_PLUGIN=ON, respectively. The CUDA build process requires an installed CUDA SDK.

If multiple applications use EMB², it might be desireable to build shared libraries by specifying -DBUILD_SHARED_LIBS=ON.

Furthermore, EMB² can be built to work with threading analysis tools such as Helgrind or ThreadSanitizer with -DTHREADING_ANALYSIS_MODE=ON. This uses mutexes around atomics to avoid false positives and degrades performance significantly.

Warnings can be treated as errors by the option -DWARNINGS_ARE_ERRORS=ON.

The tutorial of EMB² comes with a number of examples in doc/examples/. These can be built using the option -DBUILD_EXAMPLES=ON. Note, however, that the examples use C++11 features and require an appropriate compiler.

The documentation may be built by setting -DBUILD_DOCUMENTATION=ON if Doxygen is installed.

By default, the included unit tests are built as part of the installation process. To override the default behavior, add the option -DBUILD_TESTS=OFF.

2. Compiling and Linking

As the next step, you can compile the library using the generated build files. On Linux, the build mode (Release|Debug) is already given in the build files, whereas on Windows, it has to be specified now.

For a Linux build, type

cmake --build .

For a Windows Release build, type

cmake --build . --config Release

If you are a developer working on a repository snapshot of EMB², you can build the documentation as follows (provided that you have Doxygen installed):

cmake --build . --target doxygen

Note that this is not necessary if you build from a release file.

3. Running the Tests

To check whether EMB² was compiled correctly, run the tests. The test executables are contained in the subfolder "binaries".

On Linux, type

binaries/run_tests.sh

On Windows, type

binaries\run_tests.bat

If no error message occurs, EMB² works fine.

4. Installation

The default installation path on Linux is

/usr/local/

and on Windows

C:\Program Files\embb-X.Y.Z\

or

C:\Program Files (x86)\embb-X.Y.Z

depending on the target architecture.

If you want a different installation path, you can change it now by typing

cmake -DINSTALL_PREFIX=YourCustomPath ..

To install the files, use the command

cmake --build . --target install

which copies the contents of the "install" folder to the "bin", "lib", and "include" folders in the installation path. For the default paths, the installation has to be run with administrator / root privileges.

Using the Library

Components

For some of the components, there exist C and C++ versions, wheras others are only implemented in C++. The directory names are postfixed with either "_cpp" or "_c" for the C++ and C versions, respectively. Currently, EMB² is composed of the following components:

  • Base library: base_c, base_cpp
  • MTAPI: mtapi_c, mtapi_cpp, and mtapi_plugins_c (mtapi_network_c, mtapi_opencl_c, mtapi_cuda_c)
  • Algorithms: algorithms_cpp
  • Dataflow: dataflow_cpp
  • Containers: containers_cpp

Directory "base_c" contains abstractions for threading, synchronization, atomic operations, and other functionalities. As the name indicates, the code is implemented in C. Directory "base_cpp" contains C++ wrappers around the "base_c" functions. Similarly, the MTAPI task scheduler is available for programs written in C ("mtapi_c") or C++ ("mtapi_cpp"). Heterogeneous and distributed systems are supported via the plugins contained in "mtapi_plugins_c". Directory "algorithms_cpp" provides high-level constructs for typical parallelization tasks in C++, and "dataflow_cpp" generic skeletons for the development of parallel stream-based applications. Finally, "containers_cpp" provides data structures for storing objects in a thread-safe way.

Using C++

If you want to use the C++ functionalities of EMB², you have to link the following libraries (names will be slightly different on Windows and on Linux) in the given order:

embb_dataflow_cpp, embb_algorithms_cpp, embb_containers_cpp, embb_mtapi_cpp, embb_mtapi_c, embb_base_cpp, embb_base_c

The C++ header files can be included as follows:

#include <embb/base/base.h>
#include <embb/mtapi/mtapi.h>
#include <embb/containers/containers.h>
#include <embb/dataflow/algorithms.h>
#include <embb/dataflow/dataflow.h>

Using C

If you only want to use the C versions of MTAPI and the base library, link them in the following order:

embb_mtapi_c, embb_base_c

The C header files can be included as follows:

#include <embb/base/c/base.h>
#include <embb/mtapi/c/mtapi.h>

Alternatively, you can include MTAPI by #include <mtapi.h>.

Integration using CMake

If you are using CMake for your application, integration of EMB² is easy. After installing EMB², the installation folder contains a CMake folder with a simple finder which is used as follows:

find_package(EMBB REQUIRED NO_MODULE)
include(${EMBB_USE_FILE})

After that, you can link the libraries by name (embb_base_c, embb_mtapi_c, etc.) and the include directories are set up as described above.

If EMB² was not installed to the default directory, you have to specify the directory of the CMake script before using find_package:

set(EMBB_DIR ${your_install_directory}/CMake)

Documentation

The release files of EMB² come with a tutorial, example programs, and a reference manual describing the APIs. All documentation is contained in the "doc" folder. The root document of the reference manual (HTML) is doc/reference/index.html. Note that the generated documentation files are not under version control and hence not contained in the repository. As mentioned above, it is therefore recommended to download one of the packaged release files in order to have ready-to-use documentation.

Limitations and Important Notes

  • For memory management reasons, the number of threads EMB² can deal with is bounded by a predefined but modifiable constant (see functions embb_thread_get_max_count() / embb_thread_set_max_count() and class embb::base::Thread).
  • The MTAPI C++ interface supports automatic initialization, which allows for easy usage of the MTAPI C++, Algorithms, and Dataflow components. For performance measurements, explicit initialization is strongly recommended since the measurements will otherwise include the initialization time of MTAPI.
  • When using ThreadSanitizer, a bug causes the built-in CMake type size determination to fail which in turn leads to a broken configuration. Therefore, you have to do a normal build first and then run CMake again with flags and libs configured for ThreadSanitizer.
  • Compilation with option -DUSE_C11_AND_CXX11=ON to enable C11/C++11-based atomics has no effect with MSVC.

embb's People

Contributors

bernhard-gatzhammer avatar bufferoverflow avatar christian-kern avatar danklmn avatar ephemeralriggs avatar fuchsto avatar marcus-winter avatar sebhub avatar tobias-schuele avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

embb's Issues

Run AppVerifier as part of CI

Integrate AppVerifier into Jenkins:

  • Weekly or on-demand execution (e.g., before releases)
  • Multiple iterations of complete test suite, at least 12 hours runtime to achieve high coverage (nightly builds may be disabled during that time)
  • Enabled tests: Basic (all) and Cuzz

Nightly static analysis

Extend nightly Jenkins jobs so that in case of successful builds, the development branch is not only merged into 'master' but also into 'coverity_scan'. This automatically triggers Coverity Scan via Travis CI.

GPU device context

Provide API extensions for OpenCL / CUDA plugins that enable users to retrieve the device context.

Task / job attributes for problem size

For scheduling in heterogeneous systems, the user must be able to specify the problem size. Introduce the following attributes (prioritized):

  • Tasks can have optional attribute (value) => MTAPI_TASK_PROBLEM_SIZE
  • Jobs can have optional attribute for the computation of the problem size (function) => MTAPI_JOB_PROBLEM_SIZE_FUNCTION
  • Jobs can have optional attribute (value) =>MTAPI_JOB_DEFAULT_PROBLEM_SIZE (default: 1)

Optimization of spinlocks

To avoid frequent cache invalidations, a CAS should only be performed if the spinlock is released.

When using ForEach in dataflow serial process, new slice is processed before completion

When using ForEach inside a Serial Process in Dataflow, it seems like the Process is respawning the task before the ForEach has finished computation. This can lead to wrong results, as the code behind the ForEach may depend on an internal state.

Pseudocode:
SerialProcess
DebugOutput BEFORE
ForEach
DebugOutput AFTER

Without ForEach, the debug outputs are printed in serial. With the ForEach, they are grouped to blocks. E.g. Before, Before, Before, Before, After, After, After, After. This may indicate that a new slice is processed when the task waits for the ForEach to finish.

CMake options

The CMake files define the following options:

CMakeCommon/CreateDoxygenDocumentationTarget.cmake: option(BUILD_DOCUMENTATION "Use Doxygen to create the HTML based API documentation" ON)
CMakeLists.txt:option(BUILD_TESTS "Specify whether tests should be built" ON)
CMakeLists.txt:option(BUILD_EXAMPLES "Specify whether examples should be built" OFF)
CMakeLists.txt:option(USE_EXCEPTIONS "Specify whether exceptions should be activated in C++" ON)
CMakeLists.txt:option(INSTALL_DOCS "Specify whether Doxygen docs should be installed" ON)
CMakeLists.txt:option(WARNINGS_ARE_ERRORS "Specify whether warnings should be treated as errors" OFF)
CMakeLists.txt:option(USE_AUTOMATIC_INITIALIZATION "Specify whether the MTAPI C++ interface, algorithms and dataflow should automatically intialize the MTAPI node if no explicit initialization is present" ON)
CMakeLists.txt:option(BUILD_OPENCL_PLUGIN "Specify whether the MTAPI OpenCL plugin should be built" OFF)

However, when running CMake, only the following options are actually listed in the output:

-- Warnings are not treated as errors (default)
(set with command line option -DWARNINGS_ARE_ERRORS=ON/OFF)
-- MTAPI/Tasks automatic initialization enabled (default)
(set with command line option -DUSE_AUTOMATIC_INITIALIZATION=ON/OFF)
-- Exceptions enabled (default)
(set with command line option -DUSE_EXCEPTIONS=ON/OFF)
-- Building tests enabled (default)
(set with command line option -DBUILD_TESTS=ON/OFF)
-- Building examples disabled (default)
(set with command line option -DBUILD_EXAMPLES=ON/OFF)

What about the following?

BUILD_DOCUMENTATION
INSTALL_DOCS
BUILD_OPENCL_PLUGIN

Please clarify and make consistent.

MTAPI callbacks

Document extensions of MTAPI that enable specification of callback functions.

Warning in embb_mtapi_job_t.c

With GCC 7.1 I get the following warning:

mtapi_c/src/embb_mtapi_job_t.c:239:31: warning: argument to 'sizeof' in 'memcpy' call is the same pointer type 'mtapi_uint_t (**)(const mtapi_task_hndl_t) {aka unsigned int (**)(const struct mtapi_task_hndl_struct)}' as the destination; expected 'mtapi_ext_problem_size_function_t {aka unsigned int (*)(const struct mtapi_task_hndl_struct)}' or an explicit length [-Wsizeof-pointer-memaccess] &attribute, sizeof(mtapi_ext_problem_size_function_t*));

JIRA issues

With the introduction of the 'Projects' feature in GitHub, we decided to retire JIRA, which we have used so far for issue tracking and planning. All open issues will soon be moved to GitHub. Apologies in advance for the number of mails you will receive in case you are watching the repository.

This issue serves as a placeholder for already closed JIRA issues as of Nov 4, 2016. A summary of these issues is attached (rename as CSV files are not supported by GitHub).

JIRA-issues.txt

Blocking data structures

Many users need concurrent data structures but not necessarily lock-free or wait-free ones (which are already available in EMB²). For example, if an application is not timing critical, blocking data structures might be beneficial w.r.t memory consumption and context switching. Implement thread-safe wrappers for the following data structures provided by the standard library:

  • set
  • map
  • stack
  • queue
  • priority_queue

Revise tutorial

The tutorial needs to be revised and extended. As a first step, outline what has to be changed in which sections. This includes a description of those parts that have been added since the last version of the tutorial such as support for heterogeneous systems (algorithms, dataflow, CUDA plugin, etc.)

Enable dynamic linking in Jenkins

Introduce a row in the build matrix that builds dynamic libraries. Otherwise, missing symbols may not be detected as part of the CI process. To reduce build times, remove the option "examples=off".

Performance analysis on ARM/Linux

For some applications, better scalability was observed on x86 (Windows) than on ARM (Linux). Investigate the differences using the following sample applications:

  • Quicksort
  • ForEach on a vector with complex floating point operations per element in order to circumvent memory bandwidth limitations
  • Very fine-granular tasks (FFT / feature extraction)

Low-level optimizations (spinlocks, atomics)

Proposals for reducing overhead when processing extremely small tasks (to be discussed):

  • Align embb_atomic_int atomic_spin_variable_ using EMBB_PLATFORM_ALIGN (central definition of cache line size woud be useful as well as an additional macro for padding, e.g. EMBB_PLATFORM_ALIGN_PADDING; background: the aligned variable may be followed by an unaligned one for which reason char padding[CACHE_LINE_SIZE-SIZEOF(var)] should follow)
  • Inline atomics on Windows (problem: inline assemblies not supported on x86_64, solution: compiler intrinsics?)
  • Release spinlock without mfence (requires explicit barrier specification for atomics)
  • Inline spinlock functions (lock/unlock) to avoid function calls

Support for heterogeneous systems in dataflow components

Currently, the dataflow component only supports homogeneous systems. As a user, however, I want to be able to schedule processes of a dataflow network on different MTAPI nodes, either explicitly or implicitly. For example, some stages of a pipeline might be scheduled on the CPU and others on a GPU. Relates to #10.

Compilation problem with VS 2013 when using default constructor of dataflow network

On Visual Studio 2013, the compiler emits an error message when omitting the number of slices in the constructor of a network:

Error 2 error C2664: 'embb::dataflow::Network::Source<ArrayWrapper<float,1,6>,embb::base::internal::Nil,embb::base::internal::Nil,embb::base::internal::Nil,embb::base::internal::Nil>::Source(const embb::dataflow::Network::Source<ArrayWrapper<float,1,6>,embb::base::internal::Nil,embb::base::internal::Nil,embb::base::internal::Nil,embb::base::internal::Nil> &)' : cannot convert argument 1 from 'MyNetwork (__cdecl *)(void)' to 'embb::dataflow::Network &'
The same program compiles when the number of slices is specified.

Add support for heterogeneous systems in parallel algorithms

Currently, the algorithms provided by EMB² focus on homogeneous systems. Extend the API to support heterogeneous system. For example, it should be possible to not only pass ForEach a lambda function, but also an MTAPI Job that can be executed on a GPU/DSP etc.

Add mutex implementation for atomic operations to support threading analysis tools

As a user of EMB², I want to be able to run dynamic threading analysis tools such as Helgrind or ThreadSanitier on my application. However, most tools are not able to deal with atomic operations, especially from external libraries. To avoid false positive results, atomic operations should be "simulated" using mutexes for the analysis.

Add an implementation of atomic operations at C level using EMB² mutexes. For example, when calling embb_base_atomic_compare_and_swap_int, the CAS should be protected by a mutex (embb_mutex_t) without any assembly code. The implementation is enabled by a CMake option which should only be used for analysis.

Use mutex in spinlock for threading analysis

Threading analysis mode allows to run tools such as Helgrind for finding data races etc. However, in case of spinlocks, the actual semantics is hidden from Helgrind as atomics are replaced by mutexes. To solve this problem, use simple mutexes inside spinlocks instead of a loop (if and only if analysis mode is enabled).

Resource leak in unit test for pools

4
memcheck

Memcheck, a memory error detector Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al. Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info Command: ./embb_containers_cpp_test

2855
2082
memcheck

/usr/bin/valgrind.bin --tool=memcheck --xml=yes --xml-file=embb_containers_cpp_test.xml ./embb_containers_cpp_test RUNNING 00:00:00:00.061 FINISHED 00:00:18:13.602 0x0 1 Leak_DefinitelyLost 400 bytes in 1 blocks are definitely lost in loss record 1 of 3 400 1 0x4C2E80F /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so operator new[](unsigned long) 0x41E059 /home/tobias/embb-master/build/binaries/embb_containers_cpp_test embb::containers::test::PoolTest<embb::containers::WaitFreeArrayValuePool<int, -1, embb::base::Allocator<embb::base::Atomic<int> > > >::PoolTestStatic() 0x42B958 /home/tobias/embb-master/build/binaries/embb_containers_cpp_test partest::TestMethod::Run(partest::TestInstanceInfo*) 0x429E59 /home/tobias/embb-master/build/binaries/embb_containers_cpp_test partest::internal::partestthreads::ThreadClosureArg1<int (*)(partest::TestInstanceInfo*), partest::TestInstanceInfo*>::ThreadStart(void*) 0x429F59 /home/tobias/embb-master/build/binaries/embb_containers_cpp_test partest::internal::partestthreads::InternalThreadStart(void*) 0x4E416B9 /lib/x86_64-linux-gnu/libpthread-2.23.so start_thread /build/glibc-t3gR2i/glibc-2.23/nptl pthread_create.c 333 0x56F582C /lib/x86_64-linux-gnu/libc-2.23.so clone /build/glibc-t3gR2i/glibc-2.23/misc/../sysdeps/unix/sysv/linux/x86_64 clone.S 109 0x1 1 Leak_DefinitelyLost 400 bytes in 1 blocks are definitely lost in loss record 2 of 3 400 1 0x4C2E80F /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so operator new[](unsigned long) 0x41E68C /home/tobias/embb-master/build/binaries/embb_containers_cpp_test embb::containers::test::PoolTest<embb::containers::LockFreeTreeValuePool<int, -1, embb::base::Allocator<embb::base::Atomic<int> >, embb::base::Allocator<embb::base::Atomic<int> > > >::PoolTestStatic() 0x42B958 /home/tobias/embb-master/build/binaries/embb_containers_cpp_test partest::TestMethod::Run(partest::TestInstanceInfo*) 0x429E59 /home/tobias/embb-master/build/binaries/embb_containers_cpp_test partest::internal::partestthreads::ThreadClosureArg1<int (*)(partest::TestInstanceInfo*), partest::TestInstanceInfo*>::ThreadStart(void*) 0x429F59 /home/tobias/embb-master/build/binaries/embb_containers_cpp_test partest::internal::partestthreads::InternalThreadStart(void*) 0x4E416B9 /lib/x86_64-linux-gnu/libpthread-2.23.so start_thread /build/glibc-t3gR2i/glibc-2.23/nptl pthread_create.c 333 0x56F582C /lib/x86_64-linux-gnu/libc-2.23.so clone /build/glibc-t3gR2i/glibc-2.23/misc/../sysdeps/unix/sysv/linux/x86_64 clone.S 109

Reduce number of iterations in tests for threading analysis

Many of the tests are actually stress tests which is not optimal if tools are employed for threading analysis: Firstly, this significantly increases the runtimes of tools like Helgrind without increasing the likelihood of revealing additional problems. Secondly, stress tests reduce the number of schedules that can be explored in a given amount of time using concurrency testing tools such as Cuzz (https://www.microsoft.com/en-us/research/project/cuzz-concurrency-fuzzing/), which is part of AppVerif contained in the Windows SDK. For these reasons, the number of iterations for certain actions in the tests should be reduced to reasonable small value if and only if THREADING_ANALYSIS_MODE=ON (the whole test suite should run through very quickly without any tools attached).

Tutorial application

Review and revise video processing tutorial if necessary. Describe the application in the tutorial and include the source code in the examples directory.

The tasklimit is reached in some configurations of a dataflow network

Check whether there is an error in the determination of the maximum number of slices when constructing the MTAPI scheduler. Also, add "\note" in Doxygen documentation that tell users how

  • autoconfiguration works
  • task limit of scheduler may have to be increased when increasing the number of slices (as otherwise, the latter may not have any effects)

mtapi_c Fibonacci crushes

Hello,
We have tested the Fibonacci example code on our server. And we found the code will cause "core dumped" once the parameter is larger than 15; the program runs normally with parameter of 15 and smaller.
The server has Intel(R) Xeon(R) CPU E5-2640, 64GB memory running with Fedora 18.
Do you have any suggestions?
Thanks!

Regards,
Peng

Destruction of object pool elements

embb_containers_cpp_test fails when run with AppVerifier due to objects with active critical sections being deleted. Looks like destructors of contained objects are not called in destructor of object pool (cf. #37).

Enable dynamic linking

Libraries are linked statically by default, users should be able to build dynamically too.

Add comments to CMake files

Add brief comments to the following files explaining their purpose:

  • EMBBConfig.cmake.in
  • UseEMBB.cmake.in

Is the (inconsistent) naming scheme intended?

Run Helgrind as part of CI

Resolve remaining Helgrind warnings and eliminate false positives. Create nightly Jenkins job and make sure that threading analysis mode is enabled during compilation.

MTAPI CPP status exceptions

Clarify whether mtapi_cpp/include/embb/mtapi/status_exception.h is still needed (currently not used by any test).

Zip release files

Currently, only a .tar.gz file is generated for releases. This is a bit cumbersome for Windows. Additionally, a .zip file would be useful.

Yocto recipe

Yocto allows to generate customized Linux images for embedded systems. Create a recipe for easy integration of EMB².

I got errors "CMAKE_CXX_COMPILER-NOTFOUND" while build the code

Hello,

I try to compile the code in Ubuntu 4.8.1 and GCC 4.8.1
I follow your instructions to build with generator and running "cmake" without options inside "build" subfolder I make. But errors occured, it said "CMAKE_CXX_COMPILER-NOTFOUND" repeatedly.

Please advise and guide me, I'm curious about example and tutorial that available inside the source. But, it needs to compile first, this is first time to me.

Thanks

AppVerifier warnings

Microsoft's App Verifier V 10.0, which is part of the Windows SDK, reports some problems for the tests. It is not clear whether these are real or just false positives.

  • If Basics -> Locks is enabled, embb_base_c_test.exe and embb_base_cpp_test.exe embb_containers_cpp_text.exe fail
  • If Basics -> Handles is enabled, embb_mtapi_network_c_test.exe fails

Modify scripts that build tutorial

Changes to bake_tutorial.py:

  • Remove indentation according to first line, e.g. if first line has 4 spaces, remove 4 spaces from the beginning of each line.
  • Embed code blocks as follows to enable syntax highlighting (with pandoc):
Some text

```cpp
Some code
```

Some more text

Parallel for loops

Users sometimes want a more flexible way to implement for loops than possible with ForEach. For example, ForEach is designed to be used with iterators and does not support strides not equal to one. Implement a function ForLoop which works with integers (size_t, ...).

DoD:
Source code incl. Doxygen documentation merged into development branch
Unit tests created and successfully executed
Tutorial extended (example demonstrating for loops)
Moved iterators from tutorial app to algorithms (not public)
Adapted tutorial application

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.