pnxs / dots-cpp Goto Github PK

There are currently several inconsistencies that make the handling of type descriptors more complicated than it has to be. These issues include code bloat due to unclear usages of shared pointers, flawed application of const correctness, as wells as unintuitive naming.

For situations in which some kind of general information about channel endpoints is required (e.g. for verifying authentication credentials), channels currently provide some basic info via the Medium class.

However, this information is very limited and cannot easily be extended to other use cases where endpoint information is currently handled implicitly (e.g. specifying connection information).

Instead, universal endpoint information could be expressed via uniform resource identifiers (URIs), where the scheme designates the channel type (e.g. TPC, WebSocket or UNIX domain socket). Since URIs can contain both host information and paths, they are flexible enough to express endpoints for both IP- and file-based protocols and could also later be extended to specify user information.

The C++ CMake layer for the DOTS code generator currently uses
add_custom_command to create generation targets for every passed DOTS
model file. These targets are then implicitly added as dependencies to
all targets created in the same directory (see [1]).

However, this only applies to targets that are actually being built.
Some CMake targets, particularly interface libraries (i.e. header-only
libraries), are not being built and are therefore not generating any
header files. This makes it impossible to create header-only model
libraries without using workarounds.

References:

• [1] https://cmake.org/cmake/help/v3.12/command/add_custom_command.html

When an active subscription is assigned, no unsubscribe is currently
performed, which leaves the previous subscription in a dangling state.

Unbeknownst to the owner, this leaves the previous subscription active,
which can result in undefined behavior, particularly when the handler
relies on resources that are bound to the subscriptions lifetime.

Publishing information and reacting to publish events from others is an
essential aspect of most DOTS applications. Creating corresponding
(unit) tests is currently quite complex, requires a lot of additional
code and is also tedious and error prone to do.

Instead, creating unit tests should be properly supported by the
library; with the following design goals in mind:

• Unit testing DOTS components or applications should be easy, to
  encourage the creation of tests in the first place. The user must
  therefore be able to focus on the tests itself, rather than the
  technicalities behind them.

• Test results must be meaningful. The user must be able to tell which
  expectation was not met and why it failed. This also includes proper
  printing of values in the test output.

• The tests have to be representative of interactions with actual DOTS
  hosts. In particular, the user cannot be expected to mock
  implementation details of a transceiver to get representative results.

Direct or indirect logging of transmissions is currently scattered
around several components; often with inconsistent logging levels and
formats.

Instead, transmission logging can be performed exclusively in the
connection, which allows consistent logging in both host and guest
connections for debugging purposes.

For historical reasons, the event and serialization components are currently part of the IO layer and namespace.

However, both are not intrinsically connected to what is called IO in the library and should rather be moved into dedicated layers to have a clearer and more decoupled structure.

There are various use cases which require basic introspection of DOTS
types. Even though introspection is precisely the purpose of
descriptors, it is often cumbersome to do, because there is no simple
way to check the type of a descriptor or perform a necessary conversion
afterwards.

This results in a lot of manual type checks, as well as static casts to
compatible types. Using dynamic casts would include implicit type
checking, but they are generally not desirable to use, due to the
non-trivial costs.

Instead, descriptors should support basic introspection via easy to use
convenience functions. These have to support both the use case for
specific types, as well as type categories, such as struct types.

Since creation of DOTS, the CBOR standard made some progress with specifications like
https://datatracker.ietf.org/doc/rfc8742/ which can be used for dumps and records.
Using https://datatracker.ietf.org/doc/draft-ietf-cbor-file-magic/ is is possible to let "file" print out more information about
the dump or record.

On most UNIX systems, the library currently uses the 'select' system
call for IO events (e.g. for TCP sockets). This is done implicitly by
disabling 'epoll' with the BOOST_ASIO_DISABLE_EPOLL option, which
results in ASIO using 'select' as a fallback.

Even though 'select' is generally more portable than 'epoll', it
struggles with more complex applications, which use a larger number of
file descriptors, particularly in conjunction with using blocking and
non-blocking sockets simultaneously. The way ASIO is using 'select' can
result in the event loop to seemingly "stall", because calls on an open
socket suddenly return "-1 EBADF (Bad file descriptor)".

Even though ASIO supports threadsafe posts to IO contexts from
arbitrary threads, there does not seem to be an easy way to wait for
the result with the currently used version of ASIO.

Serialization for various formats is currently implemented in multiple locations, resulting in quite a lot of redundant code. There is no general abstraction for serialization or type traversal. Instead all kinds of serialization are implemented independent of each other.

Since type traversal and serialization are a core part of the library, they should be overhauled to be easier to maintain, extend and also to allow for various additional optimizations that are not possible with the way serialization is currently implemented.

String values in all string based serializers are currently always
delimited by double quotes without any way of escaping, which makes it
impossible for string values to contain double quotes themselves.

In particular, this results in incomplete support for valid JSON string
values, whose escaping rules are defined in a similar way to C , which
also requires escaping of various control characters.

The legacy authentication manager does currently not correctly evaluate
the list of matching authentication rules when determining whether a
network requires authentication or not.

The first issue is, that rules that do not include a secret are not
considered during evaluation. This can result in a guest being required
to authenticate themselves, even if they connect from a network that is
always accepted without a secret.

Additionally, the evaluation currently always checks all matching
rules, which defeats the purpose of having a prioritized rules list.

Finally, the default accept policy is applied in the inverted way,
resulting in an authentication requirement, even if a guest would end
up being accepted without it and vice versa.

The application currently automatically sets the endpoint port when it
is not set by the user.

However, the default port is only defined for TCP connections and
should not be used for other endpoint types. Additionally, for
endpoints that do not have hostnames (e.g. UNIX domain sockets), this
results in an exception, making it currently impossible to use UDS
sockets via an application argument.

Instead, the default port must only be set for TCP endpoints.

The library currently provides Google Test support via several classes and macros, along with the concept of "call expectations" to produce sequences.

However, particularly the current call expectations are flawed in multiple ways. Among other things, they are lacking as follows:

• They do not allow non-publish expectations as arguments.

• They are unable to include negative publish expectations (i.e. expecting that a mock function is never called).

• They produce hard to dissect output, because the publish expectations are technically all created from the same source location.

• They can be unintuitive to read, because default publish expectations are created implicitly from arguments (e.g. DOTS struct types).

• They are technically complex, demanding to compile and take a lot of convoluted macro work to extend and maintain.

Since these issues are also intertwined with the other components, the entire testing support should be overhauled to rectify these issues.

Note that the current testing support should remain available, albeit as deprecated, to retain backwards compatibility for some time.

The recently introduced visitor based serializers are still missing some features that would improve usability:

• Serialize and deserialize operations should return processed bytes
• Serializers should properly support consecutive operations
• Serializers should be able to tell whether there is input data available
• Serializers should only serializer valid struct properties by default
• RapidJSON should be supported for more complex use cases
• Various smaller bugfixes and adjustments including global convenience functions for many serializers

Building the project currently still yields some compiler warnings;
particularly in regards to the MSVC compiler. These warnings include
usage of deprecated functions or conversion of incompatible arithmetic
types.

Additionally, some warnings come from third-party dependencies and
should be properly suppressed so that the build output is not cluttered
with warnings that are not directly connected to the project.