omegastick / pytorch-cpp-rl

PyTorch C++ Reinforcement Learning

CppRl - PyTorch C++ Reinforcement Learning

Above: results on LunarLander-v2 after 60 seconds of training on my laptop

CppRl is a reinforcement learning framework, written using the PyTorch C++ frontend.

It is very heavily based on Ikostrikov's wonderful pytorch-a2c-ppo-acktr-gail. You could even consider this a port. The API and underlying algorithms are almost identical (with the necessary changes involved in the move to C++).

It also contains an implementation of a simple OpenAI Gym server that communicates via ZeroMQ to test the framework on Gym environments.

CppRl aims to be an extensible, reasonably optimized, production-ready framework for using reinforcement learning in projects where Python isn't viable. It should be ready to use in desktop applications on user's computers with minimal setup required on the user's side.

Motivation

At the time of writing, there are no general-use reinforcement learning frameworks for C++. I needed one for a personal project, and the PyTorch C++ frontend had recently been released, so I figured I should make one.

Features

• Implemented algorithms:
  • A2C
  • PPO
• Recurrent policies (GRU based)
• Continuous control
• Discrete control
• Cross-platform compatibility (tested on Windows 10, Ubuntu 16.04, and Ubuntu 18.04)
• Solid test coverage
• Decently optimized (always open to pull requests improving optimization though)

Example

An example that uses the included OpenAI Gym server is provided in example. It can be run as follows: Terminal 1:

./launch_gym_server.py

Terminal 2:

build/example/gym_server

It takes about 60 seconds to train an agent to 200 average reward on my laptop (i7-8550U processor).

The environment and hyperparameters can be set in example/gym_client.cpp.

Note: The Gym server and client aren't very well optimized, especially when it comes to environments with image observations. There are a few extra copies necessitated by using an inter-process communication system, and then gym_client.cpp has an extra copy or two to turn the observations into PyTorch tensors. This is why the performance isn't that good when compared with Python libraries running Gym environments.

Building

CMake is used for the build system. Most dependencies are included as submodules (run git submodule update --init --recursive to get them). Libtorch has to be installed seperately.

Linux

cd pytorch-cpp-rl
mkdir build && cd build
cmake ..
make -j4

Windows

cd pytorch-cpp-rl
mkdir build && cd build
cmake -G "Visual Studio 15 2017 Win64" -DCMAKE_PREFIX_PATH=C:/path/to/libtorch ..
cmake --build . --config Release

Before running, make sure to add libtorch/lib to your PATH environment variable.

Windows performance is about 75% that of Linux's at the moment. I'm looking into how to speed things up.

Testing

You can run the tests with build/cpprl_tests (build/Release/cpprl_tests.exe on Windows).

pytorch-cpp-rl's Issues

bernoulli.cpp(46): error C2653: 'Reduction': is not a class o r namespace name...

Using windows with VS 15 2017, using the commands in the readme. When I build with cmake I get errors on the bernoulli.cpp.

C:\Users\jespe\Documents\pytorch-cpp-rl\src\distributions\bernoulli.cpp(46): error C2653: 'Reduction': is not a class o
r namespace name [C:\Users\jespe\Documents\pytorch-cpp-rl\build\cpprl.vcxproj]
C:\Users\jespe\Documents\pytorch-cpp-rl\src\distributions\bernoulli.cpp(46): error C2065: 'None': undeclared identifier
 [C:\Users\jespe\Documents\pytorch-cpp-rl\build\cpprl.vcxproj]
C:\Users\jespe\Documents\pytorch-cpp-rl\src\distributions\bernoulli.cpp(52): error C2653: 'Reduction': is not a class o
r namespace name [C:\Users\jespe\Documents\pytorch-cpp-rl\build\cpprl.vcxproj]
C:\Users\jespe\Documents\pytorch-cpp-rl\src\distributions\bernoulli.cpp(52): error C2065: 'None': undeclared identifier
 [C:\Users\jespe\Documents\pytorch-cpp-rl\build\cpprl.vcxproj]

Error in python server "module 'gym.envs.box2d' has no attribute 'LunarLander'"

ubuntu 16.04, python 3.6, no gpu

I am getting this on the server side:
(venv) ➜ pytorch-cpp-rl git:(master) ✗ ./launch_gym_server.py
20190418 11:25:27 main [INFO]: Initializing gym server
20190418 11:25:27 main [INFO]: Connecting to client
20190418 11:25:51 main [INFO]: Connected
20190418 11:25:51 init [INFO]: Gym server initialized
20190418 11:25:51 serve [INFO]: Serving
20190418 11:25:51 make [INFO]: Making 8 LunarLander-v2s
Process SpawnProcess-5:
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/home/reduct/git/baselines/baselines/common/vec_env/subproc_vec_env.py", line 9, in worker
env = env_fn_wrapper.x()
File "/home/reduct/git/pytorch-cpp-rl/gym_server/envs.py", line 112, in _thunk
env = gym.make(env_id)
File "/home/reduct/git/pytorch-cpp-rl/venv/lib/python3.6/site-packages/gym/envs/registration.py", line 183, in make
return registry.make(id, **kwargs)
File "/home/reduct/git/pytorch-cpp-rl/venv/lib/python3.6/site-packages/gym/envs/registration.py", line 125, in make
env = spec.make(**kwargs)
File "/home/reduct/git/pytorch-cpp-rl/venv/lib/python3.6/site-packages/gym/envs/registration.py", line 88, in make
cls = load(self._entry_point)
File "/home/reduct/git/pytorch-cpp-rl/venv/lib/python3.6/site-packages/gym/envs/registration.py", line 18, in load
fn = getattr(mod, attr_name)
AttributeError: module 'gym.envs.box2d' has no attribute 'LunarLander'

edit: answering myself , I was missing pip install box2d-py

where is the formula in c++ file

https://github.com/Mikoto10032/DeepLearning/blob/master/books/%5B%E6%B7%B1%E5%BA%A6%E5%BC%BA%E5%8C%96%E5%AD%A6%E4%B9%A0%5D%5BHung-yi%20Lee%5D/PPO%20(v3).pdf

in this pdf page 9. formula as this
𝑝𝜃 𝜏 = 𝑝 𝑠1 𝑝𝜃 𝑎1|𝑠1 𝑝 𝑠2|𝑠1, 𝑎1 𝑝𝜃 𝑎2|𝑠2 𝑝 𝑠3|𝑠2, 𝑎2 ⋯

where is the formula in c++ file? which function implement it? or where define it?
help me find out

Any thoughts on license?

Greetings,

Right now I'm trying to implement RL algorithms with the physics engine (PhysX) as the environment, while avoiding bindings between C++ and Python. As there aren't that many projects using the libtorch C++ frontend, this repo has been really helpful to people like me. However, I noticed that there wasn't any mentioning of a license, so it made me anxious about using parts of the code (Note that without an explicit license, formally Github users only have the right to fork or read the code, and other than that typical copyright laws hold.) Any thoughts about license? (Nowadays most people choose MIT as their license because it doesn't have any strings attached, but other licenses are also fine; it's your choice.)

Best regards,
Phil

Example code doesn't converge

Some time around ae03039 we regressed and the example code in gym_client.cpp doesn't converge any more. Presumably because of some difference in our observation normalization compared to the OpenAI Baselines one.

I'll look in more detail this weekend and confirm if it's that exact commit causing the problem.

Compiler error in observation_normalizer.cpp

compiling with g++ 9.2.1 on Ubuntu 16.04 I get the following compiler error; any ideas what might

output of make command:

Scanning dependencies of target cpprl
[ 1%] Building CXX object CMakeFiles/cpprl.dir/src/storage.cpp.o
[ 1%] Building CXX object CMakeFiles/cpprl.dir/src/observation_normalizer.cpp.o
/home/dimid/Documents/misc/pytorch-cpp-rl/src/observation_normalizer.cpp: In member function ‘std::vector<float, std::allocator > cpprl::ObservationNormalizerImpl::get_mean() const’:
/home/dimid/Documents/misc/pytorch-cpp-rl/src/observation_normalizer.cpp:79:30: error: expected primary-expression before ‘(’ token
79 | return std::vector(mean.data_ptr(), mean.data_ptr() + mean.numel());
| ^
/home/dimid/Documents/misc/pytorch-cpp-rl/src/observation_normalizer.cpp:79:45: error: expected primary-expression before ‘float’
79 | return std::vector(mean.data_ptr(), mean.data_ptr() + mean.numel());
| ^~~~~
/home/dimid/Documents/misc/pytorch-cpp-rl/src/observation_normalizer.cpp:79:69: error: expected primary-expression before ‘float’
79 | return std::vector(mean.data_ptr(), mean.data_ptr() + mean.numel());
| ^~~~~
/home/dimid/Documents/misc/pytorch-cpp-rl/src/observation_normalizer.cpp: In member function ‘std::vector<float, std::allocator > cpprl::ObservationNormalizerImpl::get_variance() const’:
/home/dimid/Documents/misc/pytorch-cpp-rl/src/observation_normalizer.cpp:85:30: error: expected primary-expression before ‘(’ token
85 | return std::vector(variance.data_ptr(), variance.data_ptr() + variance.numel());
| ^
/home/dimid/Documents/misc/pytorch-cpp-rl/src/observation_normalizer.cpp:85:49: error: expected primary-expression before ‘float’
85 | return std::vector(variance.data_ptr(), variance.data_ptr() + variance.numel());
| ^~~~~
/home/dimid/Documents/misc/pytorch-cpp-rl/src/observation_normalizer.cpp:85:77: error: expected primary-expression before ‘float’
85 | return std::vector(variance.data_ptr(), variance.data_ptr() + variance.numel());
| ^~~~~
CMakeFiles/cpprl.dir/build.make:75: recipe for target 'CMakeFiles/cpprl.dir/src/observation_normalizer.cpp.o' failed
make[2]: *** [CMakeFiles/cpprl.dir/src/observation_normalizer.cpp.o] Error 1
CMakeFiles/Makefile2:285: recipe for target 'CMakeFiles/cpprl.dir/all' failed
make[1]: *** [CMakeFiles/cpprl.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

Make error

consider using CMake's built in cppcheck support

pytorch-cpp-rl/CMakeLists.txt

Line 12 in b4a244e

list(APPEND CPPCHECK_ARGS

You might want to consider using the built in cmake support for cppcheck:
https://blog.kitware.com/static-checks-with-cmake-cdash-iwyu-clang-tidy-lwyu-cpplint-and-cppcheck/

When build under Ubuntu 18 have error "error: ‘double torch::optim::RMSpropOptions::learning_rate_’ is private within this context..""

In member function ‘virtual std::vectorcpprl::UpdateDatum cpprl::A2C::update(cpprl::RolloutStorage&, float)’:
/home/geotyper/CPP_torch/pytorch-cpp-rl-master/src/algorithms/a2c.cpp:39:24: error: ‘double torch::optim::RMSpropOptions::learning_rate_’ is private within this context
optimizer->options.learning_rate_ = original_learning_rate * decay_level;

Reinforcement Learning works, but not the visualization

Hi, thank you for the program. I get from the server side following Render not defined.

Render not defined for <baselines.common.vec_env.subproc_vec_env.SubprocVecEnv object at 0x7f99daea02b0> Render not defined for <baselines.common.vec_env.subproc_vec_env.SubprocVecEnv object at 0x7f99daea02b0>

I also tried a simple example that worked:

import gym import time env = gym.make('CartPole-v0') env.reset() for _ in range(1000): env.render() env.step(env.action_space.sample()) #take a random action time.sleep(0.03) env.close()

I tried several environments, not only CartPole-v0

I also tried to set the env_id on the server side
def make_env(env_id, seed, rank): def _thunk(): env = gym.make(env_id)

I hope someone can help with that problem.

best, Michael

Typos in some ppo equations

I am wondering about lines 111 to 116 in ppo.cpp. Does it make sense to have mean() for surr_1 and surr_2 here? I was under the impression that surrogate losses should still be per sample at this point, and the mean() is done only after the min at line 124?

auto surr_1 = ratio * mini_batch.advantages.mean();
auto surr_2 = (torch::clamp(ratio,
1.0 - clip_param,
1.0 + clip_param) *
mini_batch.advantages)
.mean();
auto action_loss = -torch::min(surr_1, surr_2).mean();

Also, the way you compute advantages works but might be suboptimal:
advantages = (advantages - advantages.mean() / (advantages.std() + 1e-5));
should probably be
advantages = (advantages - advantages.mean()) / (advantages.std() + 1e-5);

./launch_gym_server.py not working

I did managed to build the code.. the issue is that I couldn't run the gym server. I keep getting this error.

Latitude-E50:~/eba/pytorch-cpp-rl$ ./launch_gym_server.py
Traceback (most recent call last):
File "./launch_gym_server.py", line 7, in
from gym_server.server import Server
File "/home/meron/eba/sniper/test/pytorch-cpp-rl/gym_server/server.py", line 24
def init(self, zmq_client: ZmqClient):
^
SyntaxError: invalid syntax

So does with python3

Latitude-E50:~/eba/pytorch-cpp-rl$ python3 launch_gym_server.py
Traceback (most recent call last):
File "launch_gym_server.py", line 7, in
from gym_server.server import Server
File "/home/meron/eba/sniper/test/pytorch-cpp-rl/gym_server/server.py", line 10, in
from gym_server.messages import (InfoMessage, MakeMessage, ResetMessage,
File "/home/meron/eba/sniper/test/pytorch-cpp-rl/gym_server/messages.py", line 6, in
import msgpack
ModuleNotFoundError: No module named 'msgpack'

How to visualize train and test?

With instruction in this repo, I can only see terminal output word, is there any method to show game play on srceen? And, wish it can show on Windows.

cmake fails

Hello,

I'm getting the following error when trying to use cmake:

CMake Error in CMakeLists.txt:
Imported target "torch" includes non-existent path

"C:/Program Files/NVIDIA Corporation/NvToolsExt/include"

in its INTERFACE_INCLUDE_DIRECTORIES. Possible reasons include:

• The path was deleted, renamed, or moved to another location.

• An install or uninstall procedure did not complete successfully.

• The installation package was faulty and references files it does not
  provide.

Error building project: "The source directory <repo>/lib/spdlog does not contain a CMakeLists.txt file."

Hi there, I am trying to build the project but I'm currently running into the following problem when running cmake -DCMAKE_PREFIX_PATH=~/Documents/libtorch/ ..:

-- The CXX compiler identification is AppleClang 10.0.1.10010046
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE
-- Found torch: /Users/angelolovatto/Documents/libtorch/lib/libtorch.dylib
CMake Error at CMakeLists.txt:45 (add_subdirectory):
  The source directory

    /Users/angelolovatto/Documents/pytorch-cpp-rl/lib/spdlog

  does not contain a CMakeLists.txt file.


-- Configuring incomplete, errors occurred!
See also "/Users/angelolovatto/Documents/pytorch-cpp-rl/build/CMakeFiles/CMakeOutput.log".

I managed to successfully run the libtorch example before building this project. I'm a newbie to cmake projects and the error above might be something trivial, so I'm asking for help.

will not compile in Visual Studio 2022

perhaps things have changed in the latest versions of torch since you wrote this?

C10_LIBRARY

HI, authors. Thanks for your great job.

When I am building this repo on Windows 10 platform with the libtorch 1.4 library. I am faced with the C10_LIBRARY NOT FOUND problem. I am using Cmake 3.22 and Visual Studio 2017. Does anyone know how to solve this problem?

Thanks!

Make -j4 returns an error.

hello, when every I try to 'make -j4' it returns this error, I don't know how to get around it, can you please help me with it. Thank you.

[ 0%] Building C object example/lib/libzmq/CMakeFiles/objects.dir/src/tweetnacl.c.o
[ 1%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/precompiled.cpp.o
[ 1%] Building CXX object CMakeFiles/cpprl_tests.dir/src/storage.cpp.o
[ 1%] Building CXX object CMakeFiles/cpprl.dir/src/storage.cpp.o
[ 2%] Building CXX object CMakeFiles/cpprl.dir/src/observation_normalizer.cpp.o
[ 3%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/address.cpp.o
[ 3%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/client.cpp.o
[ 4%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/clock.cpp.o
[ 4%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ctx.cpp.o
[ 5%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/curve_mechanism_base.cpp.o
[ 6%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/curve_client.cpp.o
[ 6%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/curve_server.cpp.o
[ 7%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/dealer.cpp.o
[ 7%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/devpoll.cpp.o
[ 8%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/dgram.cpp.o
[ 9%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/dist.cpp.o
[ 9%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/endpoint.cpp.o
[ 10%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/epoll.cpp.o
[ 11%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/err.cpp.o
[ 11%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/fq.cpp.o
[ 12%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/io_object.cpp.o
[ 12%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/io_thread.cpp.o
[ 13%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ip.cpp.o
[ 14%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ipc_address.cpp.o
[ 14%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ipc_connecter.cpp.o
[ 15%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ipc_listener.cpp.o
[ 16%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/kqueue.cpp.o
[ 16%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/lb.cpp.o
[ 17%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mailbox.cpp.o
[ 17%] Building CXX object CMakeFiles/cpprl.dir/src/running_mean_std.cpp.o
[ 18%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mailbox_safe.cpp.o
[ 18%] Building CXX object CMakeFiles/cpprl.dir/src/algorithms/a2c.cpp.o
[ 19%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mechanism.cpp.o
[ 20%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mechanism_base.cpp.o
[ 20%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/metadata.cpp.o
[ 21%] Building CXX object CMakeFiles/cpprl_tests.dir/src/observation_normalizer.cpp.o
[ 22%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/msg.cpp.o
[ 23%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mtrie.cpp.o
[ 23%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/norm_engine.cpp.o
[ 24%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/object.cpp.o
[ 24%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/options.cpp.o
[ 25%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/own.cpp.o
[ 26%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/null_mechanism.cpp.o
[ 26%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pair.cpp.o
[ 27%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pgm_receiver.cpp.o
[ 28%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pgm_sender.cpp.o
[ 28%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pgm_socket.cpp.o
[ 29%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pipe.cpp.o
[ 29%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/plain_client.cpp.o
[ 30%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/plain_server.cpp.o
[ 31%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/poll.cpp.o
[ 31%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/poller_base.cpp.o
[ 32%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/polling_util.cpp.o
[ 32%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pollset.cpp.o
[ 33%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/proxy.cpp.o
[ 34%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pub.cpp.o
[ 34%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pull.cpp.o
[ 35%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/push.cpp.o
[ 36%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/random.cpp.o
[ 36%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/raw_encoder.cpp.o
[ 37%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/raw_decoder.cpp.o
[ 37%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/reaper.cpp.o
[ 38%] Building CXX object CMakeFiles/cpprl.dir/src/algorithms/ppo.cpp.o
[ 39%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/rep.cpp.o
[ 39%] Building CXX object CMakeFiles/cpprl.dir/src/distributions/bernoulli.cpp.o
[ 40%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/req.cpp.o
[ 40%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/router.cpp.o
[ 41%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/select.cpp.o
[ 42%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/server.cpp.o
[ 42%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/session_base.cpp.o
[ 43%] Building CXX object CMakeFiles/cpprl_tests.dir/src/running_mean_std.cpp.o
[ 44%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/signaler.cpp.o
[ 44%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/socket_base.cpp.o
/home/kbots/eba/pytorch-cpp-rl/src/distributions/bernoulli.cpp: In member function ‘virtual at::Tensor cpprl::Bernoulli::entropy()’:
/home/kbots/eba/pytorch-cpp-rl/src/distributions/bernoulli.cpp:46:101: error: ‘Reduction’ has not been declared
ntropy_with_logits(logits, probs, torch::Tensor(), torch::Tensor(), Reduction::None);
^~~~~~~~~

/home/kbots/eba/pytorch-cpp-rl/src/distributions/bernoulli.cpp: In member function ‘virtual at::Tensor cpprl::Bernoulli::log_prob(at::Tensor)’:
/home/kbots/eba/pytorch-cpp-rl/src/distributions/bernoulli.cpp:52:135: error: ‘Reduction’ has not been declared
nsors[0], broadcasted_tensors[1], torch::Tensor(), torch::Tensor(), Reduction::None);
^~~~~~~~~

[ 45%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/socks.cpp.o
[ 46%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/socks_connecter.cpp.o
[ 46%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/stream.cpp.o
[ 47%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/stream_engine.cpp.o
[ 48%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/sub.cpp.o
make[2]: *** [CMakeFiles/cpprl.dir/build.make:128: CMakeFiles/cpprl.dir/src/distributions/bernoulli.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 48%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tcp.cpp.o
[ 48%] Building CXX object CMakeFiles/cpprl_tests.dir/src/algorithms/a2c.cpp.o
[ 50%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tcp_address.cpp.o
[ 50%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tcp_connecter.cpp.o
[ 51%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tcp_listener.cpp.o
[ 52%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/thread.cpp.o
[ 52%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/trie.cpp.o
[ 53%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/radix_tree.cpp.o
[ 53%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/v1_decoder.cpp.o
[ 54%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/v1_encoder.cpp.o
make[1]: *** [CMakeFiles/Makefile2:73: CMakeFiles/cpprl.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 55%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/v2_decoder.cpp.o
[ 56%] Building CXX object CMakeFiles/cpprl_tests.dir/src/algorithms/ppo.cpp.o
[ 56%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/v2_encoder.cpp.o
[ 57%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/xpub.cpp.o
[ 58%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/xsub.cpp.o
[ 58%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/zmq.cpp.o
[ 59%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/zmq_utils.cpp.o
[ 59%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/decoder_allocators.cpp.o
[ 60%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/socket_poller.cpp.o
[ 61%] Building CXX object CMakeFiles/cpprl_tests.dir/src/distributions/bernoulli.cpp.o
[ 62%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/timers.cpp.o
[ 62%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/radio.cpp.o
[ 63%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/dish.cpp.o
[ 64%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/udp_engine.cpp.o
[ 64%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/udp_address.cpp.o
[ 65%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/scatter.cpp.o
[ 65%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/gather.cpp.o
[ 66%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ip_resolver.cpp.o
/home/kbots/eba/pytorch-cpp-rl/src/distributions/bernoulli.cpp: In member function ‘virtual at::Tensor cpprl::Bernoulli::entropy()’:
/home/kbots/eba/pytorch-cpp-rl/src/distributions/bernoulli.cpp:46:101: error: ‘Reduction’ has not been declared
return torch::binary_cross_entropy_with_logits(logits, probs, torch::Tensor(), torch::Tensor(), Reduction::None);
^~~~~~~~~
/home/kbots/eba/pytorch-cpp-rl/src/distributions/bernoulli.cpp: In member function ‘virtual at::Tensor cpprl::Bernoulli::log_prob(at::Tensor)’:
/home/kbots/eba/pytorch-cpp-rl/src/distributions/bernoulli.cpp:52:135: error: ‘Reduction’ has not been declared
return -torch::binary_cross_entropy_with_logits(broadcasted_tensors[0], broadcasted_tensors[1], torch::Tensor(), torch::Tensor(), Reduction::None);
^~~~~~~~~
[ 67%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/zap_client.cpp.o
[ 67%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/stream_connecter_base.cpp.o
[ 68%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/stream_listener_base.cpp.o
[ 69%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tipc_address.cpp.o
[ 69%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tipc_connecter.cpp.o
[ 70%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tipc_listener.cpp.o
[ 70%] Building CXX object CMakeFiles/cpprl_tests.dir/src/distributions/categorical.cpp.o
[ 70%] Built target objects
[ 71%] Building CXX object CMakeFiles/cpprl_tests.dir/src/distributions/distribution.cpp.o
make[2]: *** [CMakeFiles/cpprl_tests.dir/build.make:128: CMakeFiles/cpprl_tests.dir/src/distributions/bernoulli.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:142: CMakeFiles/cpprl_tests.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

Error when compiling: 'class torch::optim::RMSprop/Adam' has no member named 'options'

Hi author,

Thanks for your great work first!
I'm having following error when I tried to compile it on Linux:

lesley@cafe1:~/pytorch-cpp-rl/build$ cmake -DCMAKE_PREFIX_PATH=/home/lesley/pytorch-cpp-rl/libtorch ..
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found Torch: /home/lesley/pytorch-cpp-rl/libtorch/lib/libtorch.so  
-- Build type: Release
-- The C compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Detected ZMQ Version - 4.3.2
-- Performing Test COMPILER_SUPPORTS_CXX11
-- Performing Test COMPILER_SUPPORTS_CXX11 - Success
-- Performing Test COMPILER_SUPPORTS_C11
-- Performing Test COMPILER_SUPPORTS_C11 - Success
-- Performing Test CXX_HAS_TAUT_WARNING
-- Performing Test CXX_HAS_TAUT_WARNING - Success
-- Performing Test CC_HAS_TAUT_WARNING
-- Performing Test CC_HAS_TAUT_WARNING - Success
-- Using tweetnacl for CURVE security
-- Using radix tree implementation to manage subscriptions
-- Looking for kqueue
-- Looking for kqueue - not found
-- Looking for epoll_create
-- Looking for epoll_create - found
-- Looking for epoll_create1
-- Looking for epoll_create1 - found
-- Using polling method in I/O threads: epoll
-- Using polling method in zmq_poll(er)_* API: poll
-- Using 64 bytes alignment for lock-free data structures
-- Looking for include file windows.h
-- Looking for include file windows.h - not found
-- Looking for include file condition_variable
-- Looking for include file condition_variable - found
-- Using condition_variable_t implementation: stl11
-- Looking for include file ifaddrs.h
-- Looking for include file ifaddrs.h - found
-- Looking for include file sys/uio.h
-- Looking for include file sys/uio.h - found
-- Looking for include file sys/eventfd.h
-- Looking for include file sys/eventfd.h - found
-- Checking whether EFD_CLOEXEC is supported
-- Performing Test ZMQ_HAVE_EVENTFD_CLOEXEC
-- Performing Test ZMQ_HAVE_EVENTFD_CLOEXEC - Success
-- Looking for SO_PEERCRED
-- Looking for SO_PEERCRED - found
-- Looking for LOCAL_PEERCRED
-- Looking for LOCAL_PEERCRED - not found
-- Looking for clock_gettime
-- Looking for clock_gettime - found
-- Looking for fork
-- Looking for fork - found
-- Looking for gethrtimei
-- Looking for gethrtimei - not found
-- Looking for mkdtemp
-- Looking for mkdtemp - found
-- Looking for accept4
-- Looking for accept4 - found
-- Looking for strnlen
-- Looking for strnlen - found
-- Performing Test HAVE_FLAG_-Wall
-- Performing Test HAVE_FLAG_-Wall - Failed
-- Performing Test HAVE_FLAG_-Wextra
-- Performing Test HAVE_FLAG_-Wextra - Failed
-- Performing Test HAVE_FLAG_-Wno-long-long
-- Performing Test HAVE_FLAG_-Wno-long-long - Failed
-- Performing Test HAVE_FLAG_-Wno-uninitialized
-- Performing Test HAVE_FLAG_-Wno-uninitialized - Failed
-- Performing Test HAVE_FLAG_-pedantic
-- Performing Test HAVE_FLAG_-pedantic - Failed
-- Checking whether noexcept is supported
-- Performing Test ZMQ_HAVE_NOEXCEPT
-- Performing Test ZMQ_HAVE_NOEXCEPT - Success
-- Checking whether SOCK_CLOEXEC is supported
-- Performing Test ZMQ_HAVE_SOCK_CLOEXEC
-- Performing Test ZMQ_HAVE_SOCK_CLOEXEC - Success
-- Checking whether O_CLOEXEC is supported
-- Performing Test ZMQ_HAVE_O_CLOEXEC
-- Performing Test ZMQ_HAVE_O_CLOEXEC - Success
-- Checking whether SO_BINDTODEVICE is supported
-- Performing Test ZMQ_HAVE_SO_BINDTODEVICE
-- Performing Test ZMQ_HAVE_SO_BINDTODEVICE - Success
-- Checking whether SO_KEEPALIVE is supported
-- Performing Test ZMQ_HAVE_SO_KEEPALIVE
-- Performing Test ZMQ_HAVE_SO_KEEPALIVE - Success
-- Checking whether TCP_KEEPCNT is supported
-- Performing Test ZMQ_HAVE_TCP_KEEPCNT
-- Performing Test ZMQ_HAVE_TCP_KEEPCNT - Success
-- Checking whether TCP_KEEPIDLE is supported
-- Performing Test ZMQ_HAVE_TCP_KEEPIDLE
-- Performing Test ZMQ_HAVE_TCP_KEEPIDLE - Success
-- Checking whether TCP_KEEPINTVL is supported
-- Performing Test ZMQ_HAVE_TCP_KEEPINTVL
-- Performing Test ZMQ_HAVE_TCP_KEEPINTVL - Success
-- Checking whether TCP_KEEPALIVE is supported
-- Performing Test ZMQ_HAVE_TCP_KEEPALIVE
-- Performing Test ZMQ_HAVE_TCP_KEEPALIVE - Failed
-- Checking whether TIPC is supported
-- Performing Test ZMQ_HAVE_TIPC
-- Performing Test ZMQ_HAVE_TIPC - Success
-- Checking pthread_setname signature
-- Performing Test ZMQ_HAVE_PTHREAD_SETNAME_1
-- Performing Test ZMQ_HAVE_PTHREAD_SETNAME_1 - Failed
-- Performing Test ZMQ_HAVE_PTHREAD_SETNAME_2
-- Performing Test ZMQ_HAVE_PTHREAD_SETNAME_2 - Success
-- Performing Test ZMQ_HAVE_PTHREAD_SETNAME_3
-- Performing Test ZMQ_HAVE_PTHREAD_SETNAME_3 - Failed
-- Performing Test ZMQ_HAVE_PTHREAD_SET_NAME
-- Performing Test ZMQ_HAVE_PTHREAD_SET_NAME - Failed
-- Checking pthread_setaffinity signature
-- Performing Test ZMQ_HAVE_PTHREAD_SETAFFINITY
-- Performing Test ZMQ_HAVE_PTHREAD_SETAFFINITY - Success
-- Checking whether getrandom is supported
-- Performing Test ZMQ_HAVE_GETRANDOM
-- Performing Test ZMQ_HAVE_GETRANDOM - Failed
-- Could NOT find AsciiDoc (missing: ASCIIDOC_EXECUTABLE) 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/lesley/pytorch-cpp-rl/build
lesley@cafe1:~/pytorch-cpp-rl/build$ make -j4
Scanning dependencies of target cpprl
Scanning dependencies of target cpprl_tests
Scanning dependencies of target objects
[  0%] Building CXX object CMakeFiles/cpprl.dir/src/storage.cpp.o
[  1%] Building CXX object CMakeFiles/cpprl.dir/src/observation_normalizer.cpp.o
[  1%] Building CXX object CMakeFiles/cpprl_tests.dir/src/storage.cpp.o
[  1%] Building C object example/lib/libzmq/CMakeFiles/objects.dir/src/tweetnacl.c.o
[  2%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/precompiled.cpp.o
[  3%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/address.cpp.o
[  3%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/client.cpp.o
[  4%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/clock.cpp.o
[  4%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ctx.cpp.o
[  5%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/curve_mechanism_base.cpp.o
[  6%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/curve_client.cpp.o
[  6%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/curve_server.cpp.o
[  7%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/dealer.cpp.o
[  7%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/devpoll.cpp.o
[  8%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/dgram.cpp.o
[  9%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/dist.cpp.o
[  9%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/endpoint.cpp.o
[ 10%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/epoll.cpp.o
[ 11%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/err.cpp.o
[ 11%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/fq.cpp.o
[ 12%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/io_object.cpp.o
[ 12%] Building CXX object CMakeFiles/cpprl.dir/src/running_mean_std.cpp.o
[ 12%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/io_thread.cpp.o
[ 13%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ip.cpp.o
[ 14%] Building CXX object CMakeFiles/cpprl.dir/src/algorithms/a2c.cpp.o
[ 15%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ipc_address.cpp.o
[ 15%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ipc_connecter.cpp.o
[ 16%] Building CXX object CMakeFiles/cpprl_tests.dir/src/observation_normalizer.cpp.o
[ 17%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ipc_listener.cpp.o
[ 18%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/kqueue.cpp.o
[ 18%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/lb.cpp.o
[ 19%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mailbox.cpp.o
[ 19%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mailbox_safe.cpp.o
[ 20%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mechanism.cpp.o
/home/lesley/pytorch-cpp-rl/src/algorithms/a2c.cpp: In member function 'virtual std::vector<cpprl::UpdateDatum> cpprl::A2C::update(cpprl::RolloutStorage&, float)':
/home/lesley/pytorch-cpp-rl/src/algorithms/a2c.cpp:39:16: error: 'class torch::optim::RMSprop' has no member named 'options'
     optimizer->options.learning_rate(original_learning_rate * decay_level);
                ^
[ 21%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mechanism_base.cpp.o
[ 21%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/metadata.cpp.o
[ 22%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/msg.cpp.o
[ 23%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/mtrie.cpp.o
[ 23%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/norm_engine.cpp.o
[ 24%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/object.cpp.o
[ 24%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/options.cpp.o
[ 25%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/own.cpp.o
[ 26%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/null_mechanism.cpp.o
CMakeFiles/cpprl.dir/build.make:101: recipe for target 'CMakeFiles/cpprl.dir/src/algorithms/a2c.cpp.o' failed
make[2]: *** [CMakeFiles/cpprl.dir/src/algorithms/a2c.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 26%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pair.cpp.o
[ 27%] Building CXX object CMakeFiles/cpprl_tests.dir/src/running_mean_std.cpp.o
[ 28%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pgm_receiver.cpp.o
[ 29%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pgm_sender.cpp.o
[ 29%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pgm_socket.cpp.o
[ 30%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pipe.cpp.o
[ 30%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/plain_client.cpp.o
CMakeFiles/Makefile2:285: recipe for target 'CMakeFiles/cpprl.dir/all' failed
make[1]: *** [CMakeFiles/cpprl.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 31%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/plain_server.cpp.o
[ 31%] Building CXX object CMakeFiles/cpprl_tests.dir/src/algorithms/a2c.cpp.o
[ 32%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/poll.cpp.o
[ 32%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/poller_base.cpp.o
[ 33%] Building CXX object CMakeFiles/cpprl_tests.dir/src/algorithms/ppo.cpp.o
[ 34%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/polling_util.cpp.o
[ 34%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pollset.cpp.o
[ 35%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/proxy.cpp.o
[ 36%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pub.cpp.o
[ 36%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/pull.cpp.o
/home/lesley/pytorch-cpp-rl/src/algorithms/a2c.cpp: In member function 'virtual std::vector<cpprl::UpdateDatum> cpprl::A2C::update(cpprl::RolloutStorage&, float)':
/home/lesley/pytorch-cpp-rl/src/algorithms/a2c.cpp:39:16: error: 'class torch::optim::RMSprop' has no member named 'options'
     optimizer->options.learning_rate(original_learning_rate * decay_level);
                ^
[ 37%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/push.cpp.o
[ 38%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/random.cpp.o
[ 38%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/raw_encoder.cpp.o
[ 39%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/raw_decoder.cpp.o
/home/lesley/pytorch-cpp-rl/src/algorithms/ppo.cpp: In member function 'virtual std::vector<cpprl::UpdateDatum> cpprl::PPO::update(cpprl::RolloutStorage&, float)':
/home/lesley/pytorch-cpp-rl/src/algorithms/ppo.cpp:48:16: error: 'class torch::optim::Adam' has no member named 'options'
     optimizer->options.learning_rate(original_learning_rate * decay_level);
                ^
[ 39%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/reaper.cpp.o
[ 40%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/rep.cpp.o
[ 41%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/req.cpp.o
[ 41%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/router.cpp.o
CMakeFiles/cpprl_tests.dir/build.make:101: recipe for target 'CMakeFiles/cpprl_tests.dir/src/algorithms/a2c.cpp.o' failed
make[2]: *** [CMakeFiles/cpprl_tests.dir/src/algorithms/a2c.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
[ 42%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/select.cpp.o
[ 43%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/server.cpp.o
[ 43%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/session_base.cpp.o
[ 44%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/signaler.cpp.o
[ 44%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/socket_base.cpp.o
[ 45%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/socks.cpp.o
[ 46%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/socks_connecter.cpp.o
[ 46%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/stream.cpp.o
CMakeFiles/cpprl_tests.dir/build.make:114: recipe for target 'CMakeFiles/cpprl_tests.dir/src/algorithms/ppo.cpp.o' failed
make[2]: *** [CMakeFiles/cpprl_tests.dir/src/algorithms/ppo.cpp.o] Error 1
CMakeFiles/Makefile2:339: recipe for target 'CMakeFiles/cpprl_tests.dir/all' failed
make[1]: *** [CMakeFiles/cpprl_tests.dir/all] Error 2
[ 47%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/stream_engine.cpp.o
[ 48%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/sub.cpp.o
[ 48%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tcp.cpp.o
[ 50%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tcp_address.cpp.o
[ 50%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tcp_connecter.cpp.o
[ 51%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tcp_listener.cpp.o
[ 52%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/thread.cpp.o
[ 52%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/trie.cpp.o
[ 53%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/radix_tree.cpp.o
[ 53%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/v1_decoder.cpp.o
[ 54%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/v1_encoder.cpp.o
[ 55%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/v2_decoder.cpp.o
[ 55%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/v2_encoder.cpp.o
[ 56%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/xpub.cpp.o
[ 57%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/xsub.cpp.o
[ 57%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/zmq.cpp.o
[ 58%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/zmq_utils.cpp.o
[ 58%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/decoder_allocators.cpp.o
[ 59%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/socket_poller.cpp.o
[ 60%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/timers.cpp.o
[ 60%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/radio.cpp.o
[ 61%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/dish.cpp.o
[ 62%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/udp_engine.cpp.o
[ 62%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/udp_address.cpp.o
[ 63%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/scatter.cpp.o
[ 63%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/gather.cpp.o
[ 64%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/ip_resolver.cpp.o
[ 65%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/zap_client.cpp.o
[ 65%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/stream_connecter_base.cpp.o
[ 66%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/stream_listener_base.cpp.o
[ 67%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tipc_address.cpp.o
[ 67%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tipc_connecter.cpp.o
[ 68%] Building CXX object example/lib/libzmq/CMakeFiles/objects.dir/src/tipc_listener.cpp.o
[ 68%] Built target objects
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

I already added the libtorch path to environment.
Do you have any idea how to solve this?