Hi, I am currently doing a project about trying to use DDPG to train the same agent in this paper. It seems that I need to change the network architecture and the way to update the network a little bit. I have tried diving into the source code for two days, but there are just too many classes and I still have not found what I should modify. Could you give me some advice or explain a little bit which class is doing what in training? (e.g. cScenarioTrain, cScenarioExp, cNeuralNetLearner, etc.) Thanks a lot!

Hi, sorry to bother you. May I ask how do you get the specific data of the different state of the dog, I mean
those data such as in data/characters/dog.txt and data/controllers/dog/bound.txt. I want to redesign the posture of the dog in four states of FSM. Is there any tools or operation interface? Or where of the source code should I dive into?

The Windows solution references a dynamic_lib.zip, but I couldn't find it in the releases.

Just builded TerrainRL in Debug x64 mode (win10, vs2013) and launched it with sim_dog_args.txt arguments. It gives only 1.9 fps on the average on 4-core i7-6700 CPU (3.4 GHz). Weirdly enough, the process takes only 18% of the CPU load. It this performance expected? Or maybe something is wrong with the libraries. However, only CUDA version (8.0) is different from the original dependency list.
Also wanted to try new DeepMimic approach, but didn't find the code. How much time it takes to simulate an animation frame of that awesome Ziva Dynamics lion?
Thanks!

After successfully running the demo, I found that TerrainRL is only using one CPU core on my PC (Ubuntu 14.04) and the framerate is 5fps. My PC is 4 CPU cores and 8G memory.

Did I miss any detail? Or this is normal? Thanks for all.

Shihui

In the paper ,there are 21 joints,but when I print the out_tau in ImpPDcontroller ,there are 23 numbers. I want to ask what the other two numbers are about? thanks a lot!

I think I followed all the steps and there is no error message (except for warning about the unused variables and comparison between signed/unsigned expression).

But after the compilation, I only found one executable: TerrainRL, but didn't find another one: TerrainRL_Optimize. But when I run the TerrainRL, the window pops up, the framerate number is flashing, but I saw nothing except a blank window.

Did I miss anything?

Where download library.zip file for Windows build?

where is it？ could I get one copy of required library? I am configuring on Windows.

1>learning\NeuralNet.cpp(484): error C2039: 'AddData' : is not a member of 'caffe::MemoryDataLayercNeuralNet::tNNData'
1>learning\NeuralNet.cpp(1126): error C2039: 'AddData' : is not a member of 'caffe::MemoryDataLayercNeuralNet::tNNData'

build on vs2013 . win10 can you help me?

When I run ./TerrainRL_Optimizer -arg_file= args/opt_args_train_mace.txt command, it shows :
NNsolver.cpp:Unknown SolverType:257
Could you please tell me what the problem it may be ? I did not change any code.
caffe::SolverParameter_SolverType type = param.solver_type();
The value of the type is 257, how can I get the right value of the type?
Thanks !
Best wishes!!!

I'm not sure if it's intentional or not, but /Makefile is in .gitignore and I don't see any other means to build it except a visual studio project, which is strange since there's only Linux instructions in the readme.

Is a Makefile intended to be distributed with this?

Forgive me, I wonder if you have something like a simple booklet for us to understand the architecture of this code.

Hi, I have tried training both the raptor and the dog using the original code, and both wasn't successful due to large memory usage. Ubuntu keeps killing the program after running out of memory and swap.
I tried reducing memory usage by changing trainer_replay_mem_size and trainer_num_init_samples, and I get the error:

Actor Iter 0
Update Net 0:
F0302 12:29:19.209728 6773 memory_data_layer.cpp:93] Check failed: labels.size() / label_size_ == num (210847752626046076 vs. 32) Number of labels must be the same as data.
*** Check failure stack trace: ***
Aborted (core dumped)

I can run the simulation but can't train it. I didn't change any of the code. Can you tell me what went wrong? Is this caused by the version of caffe?

Thanks a lot!

When I run ./TerrainRL_Optimizer -arg_file= args/opt_args_train_mace.txt command, it shows :
[libprotobuf ERROR google/protobuf/text_format.cc:274] Error parsing text-format caffe.NetParameter: 14:19: Message type "caffe.MemoryDataParameter" has no field named "label_size".
F0522 8:48:02.638092 14423 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: data/policies/dog/nets/dog_mace3_train.prototxt
what the problem it may be ?

Hi,
I would like to create a bipedal humanoid character and simulate the new humanoid character. Is it possible to modify the raptor controller to control a humanoid character? Which files do I need to modify, and how do I modify it?
Thanks

would ask one more request ?
there is no shader file on Git rep.
can you upload please?

Hi! After seeing the impressive power of iLQR (Iterative Linear Quadratic Regulator), I'd like to apply this method to this problem and see if it works well. I'd like to first try to use this to control the same actions of the RL method, i.e., the FSM parameters. I may also consider directly controlling torques as a future work.

However, I'm really new to this and don't know what to start first. Could you give me some advice on the files that I may look at, such as the FSM parameters and controllers, and the files in the project which may be related to my implementation? I guess the actions are acting as a reference state or forces to the FSM phases, but don't know if I'm correct. Could you please explain more about their relation?

Thank you in advance for the help!

Can I regenerate the proto files in caffe?

I tried the commands

selman@selman-Inspiron-7559:~/DeepTerrainRL/external/caffe$ protoc src/caffe/proto/caffe.proto --cpp_out=.

selman@selman-Inspiron-7559:/DeepTerrainRL/external/caffe$ mkdir include/caffe/proto
selman@selman-Inspiron-7559:/DeepTerrainRL/external/caffe$ mv src/caffe/proto/caffe.pb.h include/caffe/proto

but I received same error
~/DeepTerrainRL/external/caffe$ make
find: ‘tools’: No such file or directory
find: ‘examples’: No such file or directory
find: ‘matlab/+caffe/private’: No such file or directory
find: ‘examples’: No such file or directory
find: ‘tools’: No such file or directory
find: ‘matlab/’: No such file or directory
find: ‘examples’: No such file or directory
find: ‘tools’: No such file or directory
CXX .build_release/src/caffe/proto/caffe.pb.cc
In file included from .build_release/src/caffe/proto/caffe.pb.cc:5:0:
.build_release/src/caffe/proto/caffe.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^
.build_release/src/caffe/proto/caffe.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update

1. I created a folder('library') acording to premake4.lua and prepared the deps
2. I used premake4 generate sln

I tried vs2013 and after fixing some other erros related to caffe, there are lots of errors...
like:
'isnan' : is not a member of 'std'
Cannot open include file: 'mutex'

Could you please answer? what is problem?
~/DeepTerrainRL/external/caffe$ make
find: ‘tools’: No such file or directory
find: ‘examples’: No such file or directory
find: ‘matlab/+caffe/private’: No such file or directory
find: ‘examples’: No such file or directory
find: ‘tools’: No such file or directory
find: ‘matlab/’: No such file or directory
find: ‘examples’: No such file or directory
find: ‘tools’: No such file or directory
CXX .build_release/src/caffe/proto/caffe.pb.cc
In file included from .build_release/src/caffe/proto/caffe.pb.cc:5:0:
.build_release/src/caffe/proto/caffe.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^
.build_release/src/caffe/proto/caffe.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update
^
.build_release/src/caffe/proto/caffe.pb.h:14:2: error: #error your headers.
#error your headers.
^
In file included from .build_release/src/caffe/proto/caffe.pb.cc:5:0:

What license is this provided under, if any? I don't see anything specified.

How can I install DeepTerrainRL ubuntu 16.04?
I have two questions
1-.....
Could you please give me detailed instruction steps?

I downloaded external into DeepTerrainRL.

There is two caffes in external folder. caffe and caffe-master

Which one should I build?
I tried both.

The first one is resulted.
selman@selman-Inspiron-7559:~/DeepTerrainRL-master/external/caffe/build$ make
make: *** No targets specified and no makefile found. Stop.

The other is resulted
selman@selman-Inspiron-7559:~/DeepTerrainRL-master/external/caffe-master$ make
CXX .build_release/src/caffe/proto/caffe.pb.cc
In file included from .build_release/src/caffe/proto/caffe.pb.cc:5:0:
.build_release/src/caffe/proto/caffe.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^
.build_release/src/caffe/proto/caffe.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update

2......
In ddition ı have caffe in home in my computer. Does distrub my caffe in home building a cafee in DeepTerrainRL-master?

Hello. I'm testing this project on my Mac OSX. However, when I run the command "./TerrainRL -arg_file= args/sim_dog_args.txt", it got a segmentation fault. The following is the log:

"
*** Aborted at 1505824491 (unix time) try "date -d @1505824491" if you are using GNU date ***
PC: @ 0x0 (unknown)
*** SIGSEGV (@0x0) received by PID 666 (TID 0x7fff7de7e000) stack trace: ***
@ 0x7fff9788152a _sigtramp
@ 0x10bc33d91 gBKGColor
@ 0x10b9246c7 cTextureDesc::cTextureDesc()
@ 0x10b92472c cTextureDesc::cTextureDesc()
@ 0x10bb8b32b InitOpenGl()
@ 0x10bb8b5d3 main
@ 0x7fff988895ad start
Segmentation fault: 11
"

My computer has 16G memory, and I didn't change the code, except premake and makefile to make it work on my machine. I also tested that TerrainRL_Optimizer works fine. Could you help me with this issue? Thanks in advance!

An extra null pointer check is not needed in the function "cWorld::tConstraintHandle::Clear".

Is there any specific reason for adopting the usage of unique_ptr and shared_ptr? Is it related to the data synchronisation during the parallel computing?

Thanks.

Shihui

• System: Windows10
• So I got the TerrainRL.exe file made in x64/Debug/
• I run

./TerrainRL -arg_file= ../../args/sim_dog_args.txt

• This causes Error1
• Then I click Ignore and I get Error2

Errors:

• Error1:

Expression: Vector Subscript out of Range

• Error2:

Expression: Standard C++ Libraries out of range && 0

So what do you suppose could be the problem. YOu can find files attached for the above errors.

do you have clear correct instruction steps?
I applied below steps:

1

git clone https://github.com/xbpeng/DeepTerrainRL.git
cd DeepTerrainRL/

2

wget https://github.com/xbpeng/DeepTerrainRL/releases/download/v1.0/TerrainRL-external-Linux.tar.gz
tar zxvf TerrainRL-external-Linux.tar.gz

3

wget https://github.com/xbpeng/DeepTerrainRL/files/517743/terrainnRL_premake4.zip
unzip terrainnRL_premake4.zip

4

cp ~/DeepTerrainRL/caffe_mods/caffe.proto ~/DeepTerrainRL/external/caffe/src/caffe/proto/caffe.proto
cp ~/DeepTerrainRL/caffe_mods/memory_data_layer.cpp ~/DeepTerrainRL/external/caffe/src/caffe/layers/memory_data_layer.cpp
cp ~/DeepTerrainRL/caffe_mods/memory_data_layer.hpp ~/DeepTerrainRL/external/caffe/include/caffe/layers/memory_data_layer.hpp

5

premake4 gmake

6

make

I received error after make
==== Building TerrainRL (debug32) ====
CaclaTrainer.cpp
In file included from /usr/include/c++/5/memory:62:0,
from ./learning/NeuralNetTrainer.h:2,
from ./learning/ACTrainer.h:3,
from learning/CaclaTrainer.h:3,
from learning/CaclaTrainer.cpp:1:
/usr/include/c++/5/bits/stl_algobase.h:59:28: fatal error: bits/c++config.h: No such file or directory
compilation terminated.
TerrainRL.make:275: recipe for target 'obj/x32/Debug/CaclaTrainer.o' failed
make[1]: *** [obj/x32/Debug/CaclaTrainer.o] Error 1
Makefile:16: recipe for target 'TerrainRL' failed
make: *** [TerrainRL] Error 2

Where do I get the library.zip file? There is no link, information, download, or trace of library.zip on your page.

hi !
I want to know how many kinds of actions does the MACE output? and what is the difference between run and leap in your paper.
thanks a lot ! best wishes!

Can I regenerate the proto files in caffe?

I tried the commands

selman@selman-Inspiron-7559:~/DeepTerrainRL/external/caffe$ protoc src/caffe/proto/caffe.proto --cpp_out=.

selman@selman-Inspiron-7559:/DeepTerrainRL/external/caffe$ mkdir include/caffe/proto
selman@selman-Inspiron-7559:/DeepTerrainRL/external/caffe$ mv src/caffe/proto/caffe.pb.h include/caffe/proto

but I received same error
~/DeepTerrainRL/external/caffe$ make
find: ‘tools’: No such file or directory
find: ‘examples’: No such file or directory
find: ‘matlab/+caffe/private’: No such file or directory
find: ‘examples’: No such file or directory
find: ‘tools’: No such file or directory
find: ‘matlab/’: No such file or directory
find: ‘examples’: No such file or directory
find: ‘tools’: No such file or directory
CXX .build_release/src/caffe/proto/caffe.pb.cc
In file included from .build_release/src/caffe/proto/caffe.pb.cc:5:0:
.build_release/src/caffe/proto/caffe.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^
.build_release/src/caffe/proto/caffe.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update

I expect that exception handling is usually supported by a C++ program. I wonder why your function "main" does not contain corresponding try and catch instructions so far.

How do you think about recommendations by Matthew Wilson in an article?

Would you like to adjust the implementation if you consider effects for uncaught/unhandled exceptions like they are described by Danny Kalev?

I first met some problems when unpressed the external file.i ignored this.
finally when i compiled the whole project,i just cant run the training process ,it can initialized and i saw the stage turned to training,but then it stops like this.

i find some problems in learning/NeuralNet.cpp file

I rewrite the adddata func ,just print something, it output nothing,i dont know what happened , it didnt run the function or there might be some other functions?but i find nothing, just one adddata func in the whole file .
so if anybody ever run the training file successfully ,i want to ask for some help.

Hello! I'm currently doing a project based on this work. I'm using Mac OS X, and when I run "make config=debug64", I found out that there are some problems of linking. It shows the following log:

"
Undefined symbols for architecture x86_64:
"cSpAlg::gSpVecSize", referenced from:
cRBDModel::InitJointSubspaceArr() in RBDModel.o
cRBDUtil::SolveInvDyna(cRBDModel const&, Eigen::Matrix<double, -1, 1, 0, -1, 1> const&, Eigen::Matrix<double, -1, 1, 0, -1, 1>&) in RBDUtil.o
cRBDUtil::BuildJacobianDot(cRBDModel const&, Eigen::Matrix<double, -1, -1, 0, -1, -1>&) in RBDUtil.o
cRBDUtil::CalcGravityForce(cRBDModel const&, Eigen::Matrix<double, -1, 1, 0, -1, 1>&) in RBDUtil.o
cSpAlg::CrossMs(Eigen::Matrix<double, 6, 1, 0, 6, 1> const&, Eigen::Matrix<double, -1, -1, 0, -1, -1> const&) in SpAlg.o
cSpAlg::CrossFs(Eigen::Matrix<double, 6, 1, 0, 6, 1> const&, Eigen::Matrix<double, -1, -1, 0, -1, -1> const&) in SpAlg.o
cSpAlg::ApplyTransM(Eigen::Matrix<double, 3, 4, 0, 3, 4> const&, Eigen::Matrix<double, -1, -1, 0, -1, -1> const&) in SpAlg.o
...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [TerrainRL] Error 1
make: *** [TerrainRL] Error 2
"

If I got it right, it seems that cSpAlg was not properly linked, but in the code it is written correctly... I have done some changes to the premake file, only to add the correct library locations.
In my new makefile, I'm sure that Eigen, caffe, cuda, opencv, openblas, openGL, glew and bullet are included and linked correctly.

Could you give me some suggestions on this problem? Thanks!

Hi, I found that there are two networks in the cNeuralNet class, one is mNet, the other is the network in the mSolver, and it seems that the two networks synchronize frequently while training. Could you please explain briefly what's the functionality of them respectively? Thanks!

Hi! After I copied files inside caffe_mods to the specific locations inside caffe folder, and run "make clean" and "make all", I found some errors. The following is the log:

"
src/caffe/layers/infogain_loss_layer.cpp:18:68: error: no member named 'axis' in 'caffe::InfogainLossParameter'
softmax_param->set_axis(this->layer_param_.infogain_loss_param().axis());
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
src/caffe/layers/infogain_loss_layer.cpp:221:19: note: in instantiation of member function 'caffe::InfogainLossLayer::LayerSetUp' requested here
INSTANTIATE_CLASS(InfogainLossLayer);
^
./include/caffe/common.hpp:43:18: note: expanded from macro 'INSTANTIATE_CLASS'
template class classname;
^
src/caffe/layers/infogain_loss_layer.cpp:57:48: error: no member named 'axis' in 'caffe::InfogainLossParameter'
this->layer_param_.infogain_loss_param().axis());
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
src/caffe/layers/infogain_loss_layer.cpp:221:19: note: in instantiation of member function 'caffe::InfogainLossLayer::Reshape' requested here
INSTANTIATE_CLASS(InfogainLossLayer);
^
./include/caffe/common.hpp:43:18: note: expanded from macro 'INSTANTIATE_CLASS'
template class classname;
^
src/caffe/layers/infogain_loss_layer.cpp:18:68: error: no member named 'axis' in 'caffe::InfogainLossParameter'
softmax_param->set_axis(this->layer_param_.infogain_loss_param().axis());
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
src/caffe/layers/infogain_loss_layer.cpp:221:19: note: in instantiation of member function 'caffe::InfogainLossLayer::LayerSetUp' requested here
INSTANTIATE_CLASS(InfogainLossLayer);
^
./include/caffe/common.hpp:44:18: note: expanded from macro 'INSTANTIATE_CLASS'
template class classname
^
src/caffe/layers/infogain_loss_layer.cpp:57:48: error: no member named 'axis' in 'caffe::InfogainLossParameter'
this->layer_param_.infogain_loss_param().axis());
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
src/caffe/layers/infogain_loss_layer.cpp:221:19: note: in instantiation of member function 'caffe::InfogainLossLayer::Reshape' requested here
INSTANTIATE_CLASS(InfogainLossLayer);
^
./include/caffe/common.hpp:44:18: note: expanded from macro 'INSTANTIATE_CLASS'
template class classname
^
4 errors generated.
make: *** [.build_release/src/caffe/layers/infogain_loss_layer.o] Error 1
"

I tried to add "label_size" to MemoryDataParameter message inside caffe.proto by hand, and copied the other files instead. However, errors still occurred when I trained the model.
I also tried to copy the caffe.proto file after make caffe, and generate "caffe.pb.h" and "caffe.pb.cc" by the following command:
protoc caffe/proto/caffe.proto --cpp_out=../include/
and then move the two files to build folder. But it also results in error when training.

Could you suggest the way to regenerate "caffe.pb.h" and "caffe.pb.cc" files? I would really appreciate your help.

Hi !
When I run ./TerrainRL_Optimizer -arg_file= args/dog_slopes_mixed_args.txt command, my computer is computing something. It's about Episodes、Cycles、Avg dist. I want to know what are these about?
Thanks a lot ! Best wishes !

sudo apt-get install gcc-4.9-multilib g++-4.9-multilib

I didt it
/DeepTerrainRL$ make config=debug64
==== Building TerrainRL (debug64) ====
CaclaTrainer.cpp
In file included from external/caffe/include/caffe/blob.hpp:9:0,
from external/caffe/include/caffe/net.hpp:10,
from ./learning/NeuralNet.h:3,
from ./learning/TrainerInterface.h:3,
from ./learning/NeuralNetTrainer.h:4,
from ./learning/ACTrainer.h:3,
from learning/CaclaTrainer.h:3,
from learning/CaclaTrainer.cpp:1:
external/caffe/build/src/caffe/proto/caffe.pb.h:12:2: error: #error This file was generated by a newer version of protoc which is
#error This file was generated by a newer version of protoc which is
^
external/caffe/build/src/caffe/proto/caffe.pb.h:13:2: error: #error incompatible with your Protocol Buffer headers. Please update
#error incompatible with your Protocol Buffer headers. Please update
^
external/caffe/build/src/caffe/proto/caffe.pb.h:14:2: error: #error your headers.

1. git clone https://github.com/xbpeng/DeepTerrainRL.git
2. downloaded external(https://github.com/xbpeng/DeepTerrainRL/releases/download/v1.0/TerrainRL-external-Linux.tar.gz) and extract it into DeepTerrainRL
3. premake4 gmake
4. make, with this error

Hi! I'm building this project on my Mac OS X system. I downloaded "library.zip" file and got the caffe built successfully. However, when I make the project, some linking issues occurred:

"
==== Building TerrainRL (debug64) ====
Linking TerrainRL
Undefined symbols for architecture x86_64:
"caffe::Net::Net(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, caffe::Phase, caffe::Net const*)", referenced from:
cNeuralNet::cCaffeNetWrapper::cCaffeNetWrapper(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, caffe::Phase) in NeuralNet.o
"caffe::Solver::Solver(caffe::SolverParameter const&, caffe::Solver const*)", referenced from:
caffe::SGDSolver::SGDSolver(caffe::SolverParameter const&) in NNSolver.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [TerrainRL] Error 1
make: *** [TerrainRL] Error 2
"

However, in the makefile, I have correctly set the path. To add: the "make test" of caffe also failed because of linking issues like this.
Could you help me with this problem?

Why does external folder include two caffe folders, caffe and caffe-master.

Which one should I use?
~/DeepTerrainRL/external/caffe-master$ make
or

~/DeepTerrainRL/external/caffe$ make