alibaba / hiactor

Hiactor is a distributed C++ actor framework.

Hiactor

What is Hiactor

Hiactor is an open-source hierarchical actor framework for building high-performance，concurrent and scalable event-driven systems using C++. Hiactor features a light-weight and ease-of-use distributed implementation of the actor model with concepts like object, reference and future/promise. With Hiactor, developers familiar with single-server programming can experience a seamless transition to the distributed environments.

Building Hiactor

Requirements:

• A compiler with good C++17 support (e.g. gcc >= 9.0). This was tested successfully on g++ 9.4.0.
• A higher version linux system is recommended. This was tested successfully on Ubuntu 20.04.
• CMake >= 3.13.1

Hiactor has a dependency of Seastar. Before building Hiactor, update the project submodules and install dependencies for Seastar:

$ git submodule update --init --recursive
$ sudo ./seastar/seastar/install-dependencies.sh

Next, build Hiactor as follows:

$ mkdir build
$ cd build
$ cmake [OPTIONS] ..
$ make
$ make install

The following cmake options can be specified:

• Hiactor_DEBUG: Build Hiactor with debug mode. Default is OFF.
• Hiactor_INSTALL: Enable installing Hiactor targets. Default is ON.
• Hiactor_DEMOS: Enable demos of Hiactor. Default is ON.
• Hiactor_TESTING: Enable tests of Hiactor. Default is ON.
• Hiactor_GPU_ENABLE: Enable gpu devices for Hiactor (Cuda environments required!) Default is OFF.
• Hiactor_DPDK: Enable DPDK support for Hiactor. Default is OFF.
• Hiactor_CXX_DIALECT: Specify the C++ dialect for Hiactor. Default is gnu++17.
• Hiactor_CXX_FLAGS: Specify other compilation flags for Hiactor.
• Hiactor_COOK_DEPENDENCIES: Whether to download and build Seastar's dependencies for use if you don't want to use system packages (RPMs or DEBs). Default is OFF.
• Hiactor_COOK_EXCLUDES: A semicolon-separated list of dependency names to exclude from building if Hiactor_COOK_DEPENDENCIES is enabled.
• Hiactor_CPU_STALL_REPORT: Enable warning reports at cpu stalls. Default is OFF.
• Hiactor_UNUSED_RESULT_ERROR: Make [[nodiscard]] violations an error (instead of a warning). Default is OFF.

Seastar is embedded with add_subdirectory in hiactor, you can add cmake options of Seastar directly when configuring. Note that the apps, demos, docs and tests of Seastar are disabled in its embedding mode, besides, Seastar_CXX_DIALECT will be overridden by Hiactor_CXX_DIALECT and Seastar_DPDK will be overridden by Hiactor_DPDK.

An example cmake options:

$ cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DHiactor_CXX_DIALECT=gnu++17 \
    -DHiactor_TESTING=OFF -DSeastar_CXX_FLAGS="-DSEASTAR_DEFAULT_ALLOCATOR" ..

Getting started

Follow the tutorial to understand how to use Hiactor.

hiactor's Issues

There's not even a compilation document. It can't be compiled at all.

There are some limitations to the distributed network connection

Currently, when a hiactor cluster needs to be started, the order in which the nodes are started must be strictly in accordance with their machine id order. The reason is: The current connection rule is that a node with a larger machine id will act as a client to connect to a node with a smaller machine id. If the node with a smaller machine id has not been started at this time, a connection error will occur.

Fail to compile on `arm64`

When building multi-platform docker image with Hiactor v0.1.1 installed, error occurred on arm64 arch.

docker buildx build --platform linux/amd64,linux/arm64 -f interactive-base.Dockerfile -t registry.cn-hongkong.aliyuncs.com/graphscope/interactive-base --push .

# install hiactor
RUN cd /tmp && git clone https://github.com/alibaba/hiactor.git -b v0.1.1 --single-branch && cd hiactor && \
    git submodule update --init --recursive && ./seastar/seastar/install-dependencies.sh && mkdir build && cd build && \
    cmake -DCMAKE_INSTALL_PREFIX=/opt/flex -DHiactor_DEMOS=OFF -DHiactor_TESTING=OFF -DHiactor_DPDK=OFF -DHiactor_CXX_DIALECT=gnu++17 -DSeastar_CXX_FLAGS="-DSEASTAR_DEFAULT_ALLOCATOR -mno-avx512" .. && \
    make -j && make install && rm -rf /tmp/hiactor

[BUG] Calling the same actor method across machines cannot be guaranteed to be executed in the FIFO order

At present, when it is necessary to call an actor method across machines, hiactor will look for a local shard connected to the target machine on the current machine to forward it on its behalf.
However, the calculation of this proxy shard adopts a round-robin method, so multiple calls to an actor's method may be forwarded by different shards, which leads to an inconsistency at the dataflow level and cannot guarantee that the actor method called first will be certain executed first.

building error in gcc7

building with gcc7 will result in the following error:

[  4%] Building CXX object build/seastar/CMakeFiles/seastar.dir/src/core/future.cc.o
/opt/hiactor/build/seastar/src/core/future.cc:105:50: error: explicit qualification in declaration of 'seastar::future<> seastar::internal::current_exception_as_future()'
 future<> internal::current_exception_as_future() noexcept;
                                                  ^~~~~~~~
build/seastar/CMakeFiles/seastar.dir/build.make:245: recipe for target 'build/seastar/CMakeFiles/seastar.dir/src/core/future.cc.o' failed
make[2]: *** [build/seastar/CMakeFiles/seastar.dir/src/core/future.cc.o] Error 1
CMakeFiles/Makefile2:670: recipe for target 'build/seastar/CMakeFiles/seastar.dir/all' failed
make[1]: *** [build/seastar/CMakeFiles/seastar.dir/all] Error 2
Makefile:162: recipe for target 'all' failed
make: *** [all] Error 2

[BUG] Create two `hqps_actor_autogen` targets cause cmake error

As I create two hqps_actor_autogen targets in my CMakeLists.txt, cmake error occurs

CMake Error at /usr/local/bin/hiactor_codegen/ActorAutoGen.cmake:42 (add_custom_target):
  add_custom_target cannot create target "installed_libclang" because another
  target with the same name already exists.  The existing target is a custom
  target created in source directory
  "/root/GraphScope/balabala/graph_db".  See documentation for policy
  CMP0002 for more details.
Call Stack (most recent call first):
  engines/balabala/CMakeLists.txt:39 (include)

The content length of seastar http request to be read is undefined

in seastar/core/httpd.cc

195   static future<std::unique_ptr<httpd::request>>
196   read_request_body(input_stream<char>& buf, std::unique_ptr<httpd::request> req) {
197     if (!req->content_length) {
198          return make_ready_future<std::unique_ptr<httpd::request>>(std::move(req));
199     }
200      return buf.read_exactly(req->content_length).then([req = std::move(req)] (temporary_buffer<char> body) mutable {
201         req->content = seastar::to_sstring(std::move(body));
202          return make_ready_future<std::unique_ptr<httpd::request>>(std::move(req));
203     });
204   }

the req->content_length is undefined at line 200, as req has been moved.

[BUG] Submitted task never got executed when actor reaching `max_concurrency`

When max_concurrency is set to 1, and two tasks are send to a same actor, the actor will only execute the first submitted taks, while the second one got stuck and will never be executed.