erwincoumans / tiny-differentiable-simulator

Tiny Differentiable Simulator is a header-only C++ and CUDA physics library for reinforcement learning and robotics with zero dependencies.

Tiny Differentiable Simulator

Tiny Differentiable Simulator is a header-only C++ (and CUDA) physics library with zero dependencies.

• Note that the main repository is transfered from google-research to Erwin Coumans

It currently implements various rigid-body dynamics algorithms, including forward and inverse dynamics, as well as contact models based on impulse-level LCP and force-based nonlinear spring-dampers. Actuator models for motors, servos, and Series-Elastic Actuator (SEA) dynamics are implemented.

The entire codebase is templatized so you can use forward- and reverse-mode automatic differentiation scalar types, such as CppAD, Stan Math fvar and ceres::Jet. The library can also be used with regular float or double precision values. Another option is to use the included fix-point integer math, that provide cross-platform deterministic computation.

TDS can run thousands of simulations in parallel on a single RTX 2080 CUDA GPU at 50 frames per second:

Laikago.running_trained_with_TDS_.using_ARS_on_CUDA_GPU.mp4

Multiple visualizers are available, see below.

Bibtex

Please use the following reference to cite this research:

@inproceedings{heiden2021neuralsim,
  author =	  {Heiden, Eric and Millard, David and Coumans, Erwin and Sheng, Yizhou and Sukhatme, Gaurav S},
  year =		  {2021},
  title =		  {Neural{S}im: Augmenting Differentiable Simulators with Neural Networks},
  booktitle = {Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)},
  url =		    {https://github.com/google-research/tiny-differentiable-simulator}
}

Related Papers

• "Inferring Articulated Rigid Body Dynamics from RGBD Video" (IROS 2022 submission) Eric Heiden, Ziang Liu, Vibhav Vineet, Erwin Coumans, Gaurav S. Sukhatme Project Page
• "NeuralSim: Augmenting Differentiable Simulators with Neural Networks" (ICRA 2021) Eric Heiden, David Millard, Erwin Coumans, Yizhou Sheng, Gaurav S. Sukhatme. PDF on Arxiv
• “Augmenting Differentiable Simulators with Neural Networks to Close the Sim2Real Gap” (RSS 2020 sim-to-real workshop), Eric Heiden, David Millard, Erwin Coumans, Gaurav Sukhatme. PDF on Arxiv and video
• "Interactive Differentiable Simulation", 2020, Eric Heiden, David Millard, Hejia Zhang, Gaurav S. Sukhatme. PDF on Arxiv

Related Research using TDS by Others

• "Efficient Differentiable Simulation of Articulated Bodies" (ICML 2021) Yi-Ling Qiao, Junbang Liang, Vladlen Koltun, Ming C. Lin PDF on Arxiv

Getting started

The open-source version builds using CMake and requires a compiler with C++17 support.

mkdir build
cd build
cmake ..
make -j

Examples

For visualization, two options are supported:

OpenGL 3+ Visualization

• tiny_opengl3_app, an OpenGL3 visualizer

This visualizer is native part of this library under src/visualizer/opengl

MeshCat Visualization

• MeshCat, a web-based visualizer that uses WebGL

A C++ ZMQ interface is provided.

Before running the example, install python, pip and meshcat, run the meshcat-server and open the web browser (Chrome is recommended for a good three.js experience.)

pip install meshcat
meshcat-server --open
This should open Chrome at http://localhost:7000/static/
Then compile and run tiny_urdf_parser_meshcat_example in optimized/release build.

URDF files can be loaded using a provided parser based on TinyXML2.

All dependencies for meshcat visualization are included in third_party.

---

Disclaimer: This is not an official Google product.

tiny-differentiable-simulator's Issues

Tiny geometry

I am working with a very simple aproach of a simulator. I am using tds::RigidBody and tds::RigidBody and tds::MultiBody. All the basis work well, but three are strange lacks in the geometries. I can see that some capabilities are not implemented, but I also found 'funny' things, like an apparent inconsistency in the definition of a box. The field that defines its dimensions (exent) sometimes is used as the lenght of the sides (when calculating max and min), and sometimes as the hal of it. Is anyone using this geometry? Could we make a branch to fix some basic things? I have no time to work a lot on it, but if someone is also interested maybe we can team up...

running in web (js, wasm, webgl)

Is there any prospect of this ever being ported to run on the web platform? ie state represented on the GPU, code compiled to WASM.

libasound2-dev is a requirement to compile

Compiling with make -j resulted in fatal error: alsa/asoundlib.h: No such file or directory without install of deb pkg libasound2-dev. Error can be resolved with a simple sudo apt install libasound2-dev on ubuntu focal fossa.

unable to use python bindings in linux

I am able to run CPP examples after following the build instructions mentioned in the readme. However, I am unable to run python examples present in python/examples directory.

Setup steps

• mkdir build && cd build
• cmake .. && make
• export PYTHONPATH=$(pwd)/python
• export LD_LIBRARY_PATH=$(pwd)/python

Usage

• python python/examples/billiard_optimization.py

Error

Traceback (most recent call last):
  File "python/examples/billiard_optimization.py", line 7, in <module>
    import pytinyopengl3 as p
ModuleNotFoundError: No module named 'pytinyopengl3'

I do see libpytinydiffsim_dual.so libpytinydiffsim.so libpytinyopengl3.so in the build/python folder. It seems to that python is unable to load the modules.

Kindly let me know if there are other steps in using python bindings.

Fail to make

By following the instructions in the "Getting Started" part, I got the following error when running make -j (after mkdir build,cd build, cmake ..):

make *** no targets specified and no makefile found. stop

So how can I solve this? Thanks!

Make Error

Thank you for always doing great research and making it public.

I tried to follow 'getting started'. But I faced some errors.

When I did the following procedures

mkdir build && cd build
cmake .. 
make -j

this error has occured

In file included from /home/onsoulkang/tiny-differentiable-simulator/examples/ars/ars_train_policy_cuda.cpp:171:
/home/onsoulkang/tiny-differentiable-simulator/examples/ars/ars_learner.h: In constructor ‘ARSLearner<ARSVecEnvironment>::ARSLearner(ARSVecEnvironment&, const ARSConfig&)’:
/home/onsoulkang/tiny-differentiable-simulator/examples/ars/ars_learner.h:42:83: error: class template argument deduction failed:
         worker_ = new Worker (tmpenv,  params_dim, policy_params, deltas_, config_);
                                                                                   ^
/home/onsoulkang/tiny-differentiable-simulator/examples/ars/ars_learner.h:42:83: error: no matching function for call to ‘Worker()’
In file included from /home/onsoulkang/tiny-differentiable-simulator/examples/ars/ars_train_policy_cuda.cpp:169:
/home/onsoulkang/tiny-differentiable-simulator/examples/ars/ars_vectorized_worker.h:23:5: note: candidate: ‘template<class ARSVecEnvironment> Worker(ARSVecEnvironment&, int, const PolicyParams&, const std::vector<double>&, const ARSConfig&)-> Worker<ARSVecEnvironment>’
     Worker(ARSVecEnvironment& env, int params_dim, const PolicyParams& policy_params, const std::vector<double>& deltas, const ARSConfig& config)
     ^~~~~~
/home/onsoulkang/tiny-differentiable-simulator/examples/ars/ars_vectorized_worker.h:23:5: note:   template argument deduction/substitution failed:
In file included from /home/onsoulkang/tiny-differentiable-simulator/examples/ars/ars_train_policy_cuda.cpp:171:
/home/onsoulkang/tiny-differentiable-simulator/examples/ars/ars_learner.h:42:83: note:   candidate expects 5 arguments, 0 provided
         worker_ = new Worker (tmpenv,  params_dim, policy_params, deltas_, config_);
                                                                                   ^
examples/ars/CMakeFiles/ars_train_policy_cuda.dir/build.make:75: recipe for target 'examples/ars/CMakeFiles/ars_train_policy_cuda.dir/ars_train_policy_cuda.cpp.o' failed
make[2]: *** [examples/ars/CMakeFiles/ars_train_policy_cuda.dir/ars_train_policy_cuda.cpp.o] Error 1
CMakeFiles/Makefile2:1390: recipe for target 'examples/ars/CMakeFiles/ars_train_policy_cuda.dir/all' failed
make[1]: *** [examples/ars/CMakeFiles/ars_train_policy_cuda.dir/all] Error 2
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2

I'm using Ubuntu 18.04, gcc 8.4.0, cmake 3.22.2.
Any help on how to fix this error would be appreciated.

Can't seem to compile tds

I've looked at this for a couple days and cannot seem to compile tds. The cmake looks good but it does say pthreads failed. The web seems to indicate this is not fatal.
I've attached the results of the make command... sorry, github won't allow me to attach so here it is

[ 0%] Building CXX object rbdl/CMakeFiles/rbdl-static.dir/src/rbdl_version.cc.o
[ 0%] Building CXX object rbdl/CMakeFiles/rbdl-static.dir/src/rbdl_mathutils.cc.o
[ 1%] Building CXX object rbdl/CMakeFiles/rbdl-static.dir/src/rbdl_utils.cc.o
[ 1%] Building CXX object rbdl/CMakeFiles/rbdl-static.dir/src/Constraints.cc.o
[ 2%] Building CXX object rbdl/CMakeFiles/rbdl-static.dir/src/Dynamics.cc.o
[ 2%] Building CXX object rbdl/CMakeFiles/rbdl-static.dir/src/Logging.cc.o
[ 3%] Building CXX object rbdl/CMakeFiles/rbdl-static.dir/src/Joint.cc.o
[ 3%] Building CXX object rbdl/CMakeFiles/rbdl-static.dir/src/Model.cc.o
[ 3%] Building CXX object rbdl/CMakeFiles/rbdl-static.dir/src/Kinematics.cc.o
[ 4%] Linking CXX static library librbdl-static.a
[ 4%] Built target rbdl-static
[ 5%] Building CXX object dear_imgui/CMakeFiles/dear_imgui.dir/imgui.cpp.o
[ 5%] Building CXX object dear_imgui/CMakeFiles/dear_imgui.dir/imgui_demo.cpp.o
[ 5%] Building CXX object dear_imgui/CMakeFiles/dear_imgui.dir/imgui_draw.cpp.o
[ 6%] Building CXX object dear_imgui/CMakeFiles/dear_imgui.dir/imgui_widgets.cpp.o
[ 6%] Building CXX object dear_imgui/CMakeFiles/dear_imgui.dir/imnodes.cpp.o
[ 7%] Linking CXX shared library libdear_imgui.so
[ 7%] Built target dear_imgui
[ 7%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/urdfreader.cc.o
[ 8%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/urdf/urdfdom/urdf_parser/src/check_urdf.cpp.o
[ 8%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/urdf/urdfdom/urdf_parser/src/pose.cpp.o
[ 8%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/urdf/urdfdom/urdf_parser/src/model.cpp.o
[ 9%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/urdf/urdfdom/urdf_parser/src/link.cpp.o
[ 9%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/urdf/urdfdom/urdf_parser/src/joint.cpp.o
[ 10%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/tinyxml/tinystr.cpp.o
[ 10%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/tinyxml/tinyxml.cpp.o
[ 10%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/tinyxml/tinyxmlerror.cpp.o
[ 11%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/tinyxml/tinyxmlparser.cpp.o
[ 11%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/urdf/boost_replacement/printf_console.cpp.o
[ 12%] Building CXX object rbdl/CMakeFiles/rbdl-urdfreader.dir/addons/urdfreader/urdf/boost_replacement/string_split.cpp.o
[ 12%] Linking CXX static library librbdl-urdfreader.a
[ 12%] Built target rbdl-urdfreader
[ 12%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_x11_opengl_window.cpp.o
[ 12%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_camera.cpp.o
[ 13%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_egl_opengl_window.cpp.o
[ 13%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_font_stash.cpp.o
[ 14%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_fonts.cpp.o
[ 14%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_gl_instancing_renderer.cpp.o
[ 15%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_gl_primitive_renderer.cpp.o
[ 15%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_gl_render_to_texture.cpp.o
[ 15%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_glfw_opengl_window.cpp.o
[ 16%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_load_shader.cpp.o
[ 16%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_open_sans.cpp.o
[ 17%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_opengl3_app.cpp.o
[ 17%] Building CXX object opengl/CMakeFiles/opengl_window.dir/tiny_opengl_fontstashcallbacks.cpp.o
[ 17%] Building C object opengl/CMakeFiles/opengl_window.dir////third_party/glad/gl.c.o
[ 18%] Building C object opengl/CMakeFiles/opengl_window.dir////third_party/glad/glx.c.o
[ 18%] Building CXX object opengl/CMakeFiles/opengl_window.dir////third_party/tinyobjloader/tiny_obj_loader.cc.o
[ 19%] Building CXX object opengl/CMakeFiles/opengl_window.dir/utils/tiny_clock.cpp.o
[ 19%] Building CXX object opengl/CMakeFiles/opengl_window.dir/utils/tiny_chrome_trace_util.cpp.o
/home/gary/tds/src/visualizer/opengl/utils/tiny_chrome_trace_util.cpp: In member function ‘void TinyTimings::flush()’:
/home/gary/tds/src/visualizer/opengl/utils/tiny_chrome_trace_util.cpp:106:71: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 4 has type ‘long long unsigned int’ [-Wformat=]
threadId, startTimeDiv1000, startTimeRem1000Str, newname);
^
/home/gary/tds/src/visualizer/opengl/utils/tiny_chrome_trace_util.cpp:110:67: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 4 has type ‘long long unsigned int’ [-Wformat=]
threadId, endTimeDiv1000, endTimeRem1000Str, newname);
^
[ 19%] Building CXX object opengl/CMakeFiles/opengl_window.dir/utils/tiny_logging.cpp.o
[ 20%] Building CXX object opengl/CMakeFiles/opengl_window.dir////third_party/stb_image/stb_image_write.cpp.o
[ 20%] Building CXX object opengl/CMakeFiles/opengl_window.dir///__/third_party/stb_image/stb_image.cpp.o
[ 21%] Linking CXX shared library libopengl_window.so
[ 21%] Built target opengl_window
[ 22%] Building CXX object rbdl/CMakeFiles/rbdl_gui_example.dir/examples/rbdl_gui_example.cpp.o
In file included from /home/gary/tds/third_party/rbdl/../../src/visualizer/opengl/tiny_camera.h:18:0,
from /home/gary/tds/third_party/rbdl/../../src/visualizer/opengl/tiny_common_graphics_app_interface.h:18,
from /home/gary/tds/third_party/rbdl/../../src/visualizer/opengl/tiny_opengl3_app.h:18,
from /home/gary/tds/third_party/rbdl/examples/rbdl_gui_example.cpp:14:
/home/gary/tds/third_party/rbdl/../../src/math/tiny/tiny_float_utils.h:20:0: warning: "_USE_MATH_DEFINES" redefined
#define _USE_MATH_DEFINES 1
^
/home/gary/tds/third_party/rbdl/examples/rbdl_gui_example.cpp:11:0: note: this is the location of the previous definition
#define _USE_MATH_DEFINES
^
In file included from /home/gary/tds/third_party/rbdl/examples/rbdl_gui_example.cpp:22:0:
/home/gary/tds/third_party/rbdl/../../src/utils/file_utils.hpp:40:36: error: ‘root’ declared as an ‘inline’ field
static inline std::string root{""};
^
/home/gary/tds/third_party/rbdl/../../src/utils/file_utils.hpp:40:36: error: in-class initialization of static data member ‘std::__cxx11::string tds::FileUtils::root’ of non-literal type
/home/gary/tds/third_party/rbdl/../../src/utils/file_utils.hpp:40:36: error: non-constant in-class initialization invalid for static member ‘tds::FileUtils::root’
/home/gary/tds/third_party/rbdl/../../src/utils/file_utils.hpp:40:36: note: (an out of class initialization is required)
/home/gary/tds/third_party/rbdl/../../src/utils/file_utils.hpp:45:26: error: ‘prefixes’ declared as an ‘inline’ field
"../../tds/data/"}};
^
/home/gary/tds/third_party/rbdl/../../src/utils/file_utils.hpp:45:26: error: in-class initialization of static data member ‘std::vector<std::__cxx11::basic_string > tds::FileUtils::prefixes’ of non-literal type
/home/gary/tds/third_party/rbdl/../../src/utils/file_utils.hpp:45:26: error: non-constant in-class initialization invalid for static member ‘tds::FileUtils::prefixes’
In file included from /home/gary/tds/third_party/rbdl/../../src/math/pose.hpp:19:0,
from /home/gary/tds/third_party/rbdl/../../src/geometry.hpp:23,
from /home/gary/tds/third_party/rbdl/../../src/contact_point.hpp:3,
from /home/gary/tds/third_party/rbdl/examples/rbdl_gui_example.cpp:23:
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp: In static member function ‘static tds::Conversion<AlgebraFrom, AlgebraTo>::QuaternionTo tds::Conversion<AlgebraFrom, AlgebraTo>::convert(const QuaternionFrom&)’:
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:36:8: error: expected ‘(’ before ‘constexpr’
if constexpr (std::is_same_v<AlgebraFrom, AlgebraTo>) {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:38:7: error: ‘else’ without a previous ‘if’
} else {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp: In static member function ‘static tds::Conversion<AlgebraFrom, AlgebraTo>::Vector3To tds::Conversion<AlgebraFrom, AlgebraTo>::convert(const Vector3From&)’:
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:50:8: error: expected ‘(’ before ‘constexpr’
if constexpr (std::is_same_v<AlgebraFrom, AlgebraTo>) {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:52:7: error: ‘else’ without a previous ‘if’
} else {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp: In static member function ‘static tds::Conversion<AlgebraFrom, AlgebraTo>::VectorXTo tds::Conversion<AlgebraFrom, AlgebraTo>::convert(const VectorXFrom&)’:
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:62:8: error: expected ‘(’ before ‘constexpr’
if constexpr (std::is_same_v<AlgebraFrom, AlgebraTo>) {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:64:7: error: ‘else’ without a previous ‘if’
} else {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp: In static member function ‘static tds::Conversion<AlgebraFrom, AlgebraTo>::Matrix3To tds::Conversion<AlgebraFrom, AlgebraTo>::convert(const Matrix3From&)’:
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:74:8: error: expected ‘(’ before ‘constexpr’
if constexpr (std::is_same_v<AlgebraFrom, AlgebraTo>) {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:76:7: error: ‘else’ without a previous ‘if’
} else {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp: In static member function ‘static tds::Conversion<AlgebraFrom, AlgebraTo>::Matrix3XTo tds::Conversion<AlgebraFrom, AlgebraTo>::convert(const Matrix3XFrom&)’:
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:88:8: error: expected ‘(’ before ‘constexpr’
if constexpr (std::is_same_v<AlgebraFrom, AlgebraTo>) {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:90:7: error: ‘else’ without a previous ‘if’
} else {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp: In static member function ‘static tds::Conversion<AlgebraFrom, AlgebraTo>::Matrix6To tds::Conversion<AlgebraFrom, AlgebraTo>::convert(const Matrix6From&)’:
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:103:8: error: expected ‘(’ before ‘constexpr’
if constexpr (std::is_same_v<AlgebraFrom, AlgebraTo>) {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:105:7: error: ‘else’ without a previous ‘if’
} else {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp: In static member function ‘static tds::Conversion<AlgebraFrom, AlgebraTo>::MatrixXTo tds::Conversion<AlgebraFrom, AlgebraTo>::convert(const MatrixXFrom&)’:
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:117:8: error: expected ‘(’ before ‘constexpr’
if constexpr (std::is_same_v<AlgebraFrom, AlgebraTo>) {
^
/home/gary/tds/third_party/rbdl/../../src/utils/conversion.hpp:119:7: error: ‘else’ without a previous ‘if’
} else {
^
In file included from /home/gary/tds/third_party/rbdl/../../src/contact_point.hpp:3:0,
from /home/gary/tds/third_party/rbdl/examples/rbdl_gui_example.cpp:23:
/home/gary/tds/third_party/rbdl/../../src/geometry.hpp: In member function ‘tds::Capsule::Scalar tds::Capsule::distance(const Vector3&) const’:
/home/gary/tds/third_party/rbdl/../../src/geometry.hpp:140:24: error: ‘clamp’ is not a member of ‘std’
p.z() - std::clamp(p.z(), -this->length / Scalar(2),
^
In file included from /home/gary/tds/third_party/rbdl/examples/rbdl_gui_example.cpp:23:0:
/home/gary/tds/third_party/rbdl/../../src/contact_point.hpp: In function ‘int tds::contact_sphere_sphere(const tds::Geometry, const tds::Pose&, const tds::Geometry, const tds::Pose&, std::vector<tds::ContactPoint >&)’:
/home/gary/tds/third_party/rbdl/../../src/contact_point.hpp:62:6: error: expected ‘(’ before ‘constexpr’
if constexpr (is_cppad_scalar::value) {
^
/home/gary/tds/third_party/rbdl/../../src/contact_point.hpp:75:5: error: ‘else’ without a previous ‘if’
} else {
^
rbdl/CMakeFiles/rbdl_gui_example.dir/build.make:75: recipe for target 'rbdl/CMakeFiles/rbdl_gui_example.dir/examples/rbdl_gui_example.cpp.o' failed
make[2]: *** [rbdl/CMakeFiles/rbdl_gui_example.dir/examples/rbdl_gui_example.cpp.o] Error 1
CMakeFiles/Makefile2:397: recipe for target 'rbdl/CMakeFiles/rbdl_gui_example.dir/all' failed
make[1]: *** [rbdl/CMakeFiles/rbdl_gui_example.dir/all] Error 2
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2

'ceres' does not name a type

When running the install instructions as described in README.md:

mkdir build
cd build
cmake ..
make -j

The build fails with the following error:

tiny-differentiable-simulator/examples/billiard_opt_example_gui.cpp: 
  In member function ‘bool CeresFunctional::operator()(const T*, T*) const’:
tiny-differentiable-simulator/examples/billiard_opt_example_gui.cpp:249:13: error: ‘ceres’ does not name a type
  249 |     typedef ceres::Jet<double, 2> Jet;

Since the error message appears to be referencing some missing ceres definitions, I tried something similar to what was suggested in #20 and added:

find_package(Ceres)
add_definitions(-DUSE_CERES)

To examples/CMakeLists.txt. I then get the following build error:

/tiny-differentiable-simulator/examples/billiard_opt_example_gui.cpp:43:10:
  fatal error: ceres_utils.h: No such file or directory
   43 | #include "ceres_utils.h"

Going off this, I also tried as a last effort adding:

include_directories("../src/math/tiny")

But still no luck, this gives error:

tiny-differentiable-simulator/examples/billiard_opt_example_gui.cpp:349:63:
 error: no matching function for call to ‘rollout<double, TINY::DoubleUtils>(double&, double&, int&, TinyOpenGL3App*)’
  349 |     rollout<double, DoubleUtils>(force_x, force_y, steps, &app);

I am running on :

• Ubuntu 20.04.1 LTS
• cmake version 3.18.4
• g++ version 9.3

Code Generation Tool Question

Hello Erwin,

This is an amazing package! I am just curious to know the code generation tool you used to generate dynamics models such as this one:
https://github.com/erwincoumans/tiny-differentiable-simulator/blob/master/examples/environments/omp_model_ant_forward_zero.h

Thanks a lot for your help!
Cheers,
Simon

Example run issue

After make -j step, the package is successfully compiling. The issue is with running scripts given in example. Every time I try to compile scripts in the example folder, the compiler is not able to find the existing header files. I tried providing path to the compiler for including header. Nothing works. Please help.

Adding friction property

Combining the friction requires the friction_coefs of the two bodies in contact.
Since contacts are computed between geometries, I suggest a friction property should be added to the Geometry class.

To keep it compatible, we can add a friction property on RigidBody and MultiBody also.

The idea is to go one level above if friction is not defined, i.e. Geometry -> Rigid-/MultiBody -> World.
A negative coefficient can indicate that the property is not defined.

I can go ahead and implement this if I get approval.

(edit 1)
Just as a side note, my motivation is to perform system identification on the friction coefficients.

Related to :

tiny-differentiable-simulator/src/world.hpp

Lines 186 to 190 in c1a9e06

	// TODO(erwincoumans): combine friction and restitution based on
	// material properties of the two touching bodies
	rb_pt.restitution = restitution;
	rb_pt.friction = friction;
	contactsOut.push_back(rb_pt);

and

tiny-differentiable-simulator/src/world.hpp

Lines 266 to 270 in c1a9e06

	// TODO(erwincoumans): combine friction and restitution
	// based on material properties of the two touching bodies
	mb_pt.restitution = restitution;
	mb_pt.friction = friction;
	contacts_ab.push_back(mb_pt);

Example run issue

I followed following commands :
$ git clone https://github.com/google-research/tiny-differentiable-simulator.git
$ cd tiny-differentiable-simulator
$ mkdir build
$ cd build
$ cmake ..
$ make -j

cmake and make-j was successful.

After executing these commands I went ahead to test the example scripts
$ cd tiny-differentiable-simulator/examples
$ g++ tiny_urdf_parser_meshcat_example.cpp

errors are popping up.
fatal error: cpp_base64/base64.h: No such file or directory
20 | #include <cpp_base64/base64.h>
| ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
cpp_base64/base64.h is present in build directory. So I tried providing compiler path to build and later src but again guid.hpp is not found. guid.hpp is not present inside crossguid
fatal error: crossguid/guid.hpp: No such file or directory
19 | #include <crossguid/guid.hpp>
| ^~~~~~~~~~~~~~~~~~~~
compilation terminated.

I tried $g++ -I/path/to/tiny-directory/build tiny_urdf_parser_meshcat_example.cpp

I tried $g++ -I/path/to/tiny-directory/src tiny_urdf_parser_meshcat_example.cpp

Please help to get the examples running.

Docker build error

I cannot create a docker image using the Dockerfile.
There is no python/requirements.txt file.
When I comment out this part(Line 111. RUN pip3 install -r python/requirements.txt) and run it again, I get the following error.
I would like to know the solution.

Environment
Ubuntu 18.04

Run command

$ docker build -t tiny_diff_sim . 

make[2]: *** [CMakeFiles/pagmo.dir/src/algorithms/nlopt.cpp.o] Error 1
CMakeFiles/pagmo.dir/build.make:81: recipe for target 'CMakeFiles/pagmo.dir/src/algorithms/nlopt.cpp.o' failed
make[1]: *** [CMakeFiles/pagmo.dir/all] Error 2
CMakeFiles/Makefile2:122: recipe for target 'CMakeFiles/pagmo.dir/all' failed
Makefile:159: recipe for target 'all' failed
make: *** [all] Error 2
The command '/bin/sh -c cmake -DCMAKE_BUILD_TYPE=Release . -DPAGMO_WITH_NLOPT=ON && cmake -DCMAKE_BUILD_TYPE=Release --build . . && cmake --build . . --target install' returned a non-zero code: 2

Use of inverse dynamics and collision detection

In the demo "laikago_footstep_ik", I find the process of :

• "forward kinematics"(laikago_footstep_ik.cpp, Line412);
• "inverse kinematics"(laikago_footstep_ik.cpp, Line406~459);
• "forward dynamic"(laikago_footstep_ik.cpp, Line191);

But do not find "inverse dynamic" of the MultiBody. However, there exist corresponding function defined in "src/daynamcs/inverse_dynamics.hpp Line23", there does not appear to be a call to this function in the provided example.

Is that mean "inverse dynamic" has not been considered in this current tds framework?

A similar question for "collision detection" : "dispatcher.compute_contactsXXX" defined in "world.hpp" and "contact_point.hpp" only used in the demo "rbdl_gui_example". Is collision detection not required in other demos?

Fmin is not a member of ceres

make

Scanning dependencies of target neural_contact
[ 75%] Building CXX object examples/CMakeFiles/neural_contact.dir/neural_contact.o
In file included from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/tiny_ceres_estimator.h:27:0,
                 from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/neural_contact.cpp:6:
/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/ceres_utils.h: In static member function ‘static T CeresUtils<N>::min1(const T&, const T&)’:
/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/ceres_utils.h:105:19: error: ‘fmin’ is not a member of ‘ceres’
     return ceres::fmin(a, b);
                   ^~~~
/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/ceres_utils.h:105:19: note: suggested alternative: ‘asin’
     return ceres::fmin(a, b);
                   ^~~~
                   asin
/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/ceres_utils.h: In static member function ‘static T CeresUtils<N>::max1(const T&, const T&)’:
/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/ceres_utils.h:110:19: error: ‘fmax’ is not a member of ‘ceres’
     return ceres::fmax(a, b);
                   ^~~~
/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/ceres_utils.h:110:19: note: suggested alternatives:
In file included from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/../tiny_neural_network.h:21:0,
                 from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/neural_scalar.h:8,
                 from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/neural_contact.cpp:3:
/usr/include/c++/7/cmath:1444:5: note:   ‘std::fmax’
     fmax(_Tp __x, _Up __y)
     ^~~~
In file included from /usr/include/features.h:424:0,
                 from /usr/include/x86_64-linux-gnu/c++/7/bits/os_defines.h:39,
                 from /usr/include/x86_64-linux-gnu/c++/7/bits/c++config.h:533,
                 from /usr/include/c++/7/iosfwd:38,
                 from /usr/include/c++/7/ios:38,
                 from /usr/include/c++/7/istream:38,
                 from /usr/include/c++/7/fstream:38,
                 from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/neural_contact.cpp:1:
/usr/include/x86_64-linux-gnu/bits/mathcalls.h:329:1: note:   ‘fmax’
 __MATHCALLX (fmax,, (_Mdouble_ __x, _Mdouble_ __y), (__const__));
 ^
examples/CMakeFiles/neural_contact.dir/build.make:62: recipe for target 'examples/CMakeFiles/neural_contact.dir/neural_contact.o' failed
make[2]: *** [examples/CMakeFiles/neural_contact.dir/neural_contact.o] Error 1
CMakeFiles/Makefile2:353: recipe for target 'examples/CMakeFiles/neural_contact.dir/all' failed
make[1]: *** [examples/CMakeFiles/neural_contact.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

fixed by adding -DCERES_USE_CXX11 to define flags

vi tiny-differentiable-simulator/build/examples/CMakeFiles/neural_contact.dir/flags.make

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.10

# compile CXX with /usr/bin/c++
CXX_FLAGS =  -fPIC   -std=gnu++1z

CXX_DEFINES = -DBT_USE_DOUBLE_PRECISION -DCPPAD -DCPPADCG -DDYNAMIC_LOAD_X11_FUNCTIONS -DGFLAGS_IS_A_DLL=0 -DHPIPM -DSTATIC_LINK_SPD_PLUGIN -DUSE_EIGEN -DZMQ_STATIC -DCERES_USE_CXX11

CXX_INCLUDES = -I/home/duane/PycharmProjects/tiny-differentiable-simulator/third_party/eigen3 -I/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/. -I/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/.. -I/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/../third_party/zeromq/include -I/usr/local/include/bullet -I/usr/include/python3.6m -I/usr/lib/llvm-7/include -isystem /usr/local/include -I/usr/include/eigen3 -I/home/duane/Downloads/libtorch/include -I/home/duane/Downloads/libtorch/include/torch/csrc/api/include -I/home/duane/PycharmProjects/tiny-differentiable-simulator/third_party/cpp_base64/include -I/home/duane/PycharmProjects/tiny-differentiable-simulator/third_party/tinyxml2/include

Python API segmentation fault

Using world.get_mb_constraint_solver() and world.set_mb_constraint_solver() will lead to segmentation fault due to double free from both world's destructor and PyBind11's destructor.

One solution is to remove them from the Python API until they are made safe. I can make a PR if this is a desired fix.

Spherical Joints at Contact, Mass Matrix

Hi all,

I created a simple humanoid model with 1 spherical joint and all other joints fixed. When I throw this humanoid to the floor, I got array out of bound assertion failures and simulation stopped without error messages.

After some digging, it appears that when constructing the mass matrix for contact resolution, the spherical joint case might indeed have some issues. I ended up changing this line to Algebra::assign_block(*M, Algebra::transpose(Fi), 0, qd_i, 6, 3);, as the way assign_block is implemented for 6*3 matrices seemed to cause out-of-bound errors due to the "fake transpose".

With this mod, the sim successfully runs, but the energy does not seem to decrease over time with zero restitution. Please see the video attached. In fact, if we wait long enough, the sim eventually explodes and gives NaNs.

I checked related issues #76 , #91 and made sure my model does not have inertial offset at root. The urdf model and python code is also attached.

Attachments

Thanks for helping with this issue!
Yifeng

Get Coriolis matrix

Is there a way to get the coriolis matrix using this approach? I realize that we can get the "C(q, qdot) * qdot" term using the inverse dynamics function, but is there a way to isolate just the C(q, qdot) matrix?

apple clang 12 issues

In order to compile under apple clang 12, I had to make a couple of tweaks to the code. Note that I didn't have time to get to the bottom of why the NeuralNetwork py::class won't compile. It complains that the various functions, initialize, and so on can't be matched. Accordingly I commented it out for the purposes of getting everything else compiling and running.

The remainder of the diffs allow the library to built, and it runs properly, without the NeuralNetwork pybinding, of course.

diff --git a/python/pytinydiffsim.inl b/python/pytinydiffsim.inl
index 6ca0b04..accd6d9 100644
--- a/python/pytinydiffsim.inl
+++ b/python/pytinydiffsim.inl
@@ -327,6 +327,7 @@
   m.def("point_jacobian", &MyPointJacobian);
   m.def("inverse_kinematics", &MyInverseKinematics);
   
+  #if 0
   py::class_<NeuralNetwork<MyAlgebra>>(      m, "NeuralNetwork")
       .def(py::init<int, bool>())
       .def(py::init<int, const std::vector<int>&,NeuralNetworkActivation, bool >())
@@ -337,7 +338,7 @@
       .def("save_graphviz", &NeuralNetwork<MyAlgebra>::save_graphviz)
       ;
       //.def("compute_contacts", &CollisionDispatcher<MyAlgebra>::compute_contacts2);
-
+#endif
   py::class_<NeuralNetworkSpecification>(
       m, "NeuralNetworkSpecification")
       .def(py::init<int, bool>())
diff --git a/src/math/neural_network.hpp b/src/math/neural_network.hpp
index a05829e..b9c5513 100644
--- a/src/math/neural_network.hpp
+++ b/src/math/neural_network.hpp
@@ -259,7 +259,7 @@ class NeuralNetworkSpecification {
       file << "n_" << i - 1 << "_0--n_" << i << "_0;\n\t";
     }
     file << "edge[style=solid, tailport=e, headport=w];\n\t";
-    Algebra::Scalar max_weight = Algebra::zero();
+    typename Algebra::Scalar max_weight = Algebra::zero();
     if (!weights.empty()) {
       max_weight = (Algebra::abs(weights[0]));
       for (const auto& w : weights) {
diff --git a/src/tiny_inverse_kinematics.h b/src/tiny_inverse_kinematics.h
index 6b439d9..3328ddf 100644
--- a/src/tiny_inverse_kinematics.h
+++ b/src/tiny_inverse_kinematics.h
@@ -136,7 +136,7 @@ namespace TINY {
             VectorX& q) const {
 
             TinyIKResult<Scalar, Utils> result;
-            result.residual = -1;
+            result.residual = Scalar(-1);
 
             assert(q_init.size() == mb.dof());
             assert(q_reference.empty() || q_reference.size() == q_init.size());
@@ -208,10 +208,10 @@ namespace TINY {
                         JJTe_lambda2_I(i, i) += lambda * lambda;
                     }
                     Eigen::Matrix<double, Eigen::Dynamic, 1> eigen_mat =
-                        helper::to_eigen(JJTe_lambda2_I)
+                        to_eigen(JJTe_lambda2_I)
                         .colPivHouseholderQr()
-                        .solve(helper::to_eigen(e));
-                    VectorX z = helper::from_eigen_v<double, Utils>(eigen_mat);
+                        .solve(to_eigen(e));
+                    VectorX z = from_eigen_v<double, Utils>(eigen_mat);
                     delta_theta = J.mul_transpose(z);
                 }

How to enforce joint limits (if they are supported)?

Hello,

I've recently started playing around with Tiny and I've reached the point where I'd like to limit the joint angles of the laikago model. I've tried to change the joint type in the urd file from continuous to revolute and specifying the lower and upper parameters but that didn't seem to affect the simulation at all. Going through the code it seems like joint limits are not implemented (but it might just be me looking in the wrong places).

Is there a better way of doing this than what I've described above?

Also if interested I have a working example of an OpenGL TDS viewer used via Python that could be added to the main repo :)

Thanks in advance for your time.

Best,
-Vittorio

make -j failing at 70% on Ubuntu 20.04 (on Windows Subsystem for Linux)

Hi! This is really cool project, and I can't wait to get past this issue and get started.

I don't think the fact that it's on WSL makes a difference, but I thought I would mention that for completeness.

I ran the installation commands as suggested.

mkdir build
cd build
cmake ..
make -j

For cmake .., I got the following, which I take to mean the command succeeded:

-- Build files have been written to: /mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/build

But make -j got stuck at 70%.

[ 70%] Built target tiny_urdf_parser_example
In file included from /mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/../src/dynamics/../multi_body.hpp:8,
                 from /mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/../src/dynamics/kinematics.hpp:3,
                 from /mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/../src/dynamics/mass_matrix.hpp:3,
                 from /mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/pytinydiffsim_dual.cc:19:
/mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/../src/dynamics/../geometry.hpp: In function ‘tds::Geometry<AlgebraTo>* tds::clone(const tds::Geometry<Algebra>*) [with AlgebraFrom = TinyAlgebra<TINY::TinyDual<double>, TINY::TinyDualDoubleUtils>; AlgebraTo = TinyAlgebra<TINY::TinyDual<double>, TINY::TinyDualDoubleUtils>]’:
/mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/../src/dynamics/../geometry.hpp:324:1: warning: control reaches end of non-void function [-Wreturn-type]
  324 | }
      | ^
In file included from /mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/pytinydiffsim_includes.h:9,
                 from /mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/pytinydiffsim.cc:27:
/mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/../src/geometry.hpp: In function ‘tds::Geometry<AlgebraTo>* tds::clone(const tds::Geometry<Algebra>*) [with AlgebraFrom = TinyAlgebra<double, TINY::DoubleUtils>; AlgebraTo = TinyAlgebra<double, TINY::DoubleUtils>]’:
/mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/python/../src/geometry.hpp:324:1: warning: control reaches end of non-void function [-Wreturn-type]
  324 | }
      | ^
[ 70%] Linking CXX shared library libpytinydiffsim_dual.so
[ 70%] Built target pytinydiffsim_dual
[ 70%] Linking CXX shared library libpytinydiffsim.so
[ 70%] Built target pytinydiffsim
make: *** [Makefile:141: all] Error 2

compilation error- getting stuck

I git cloned the package Tiny Differentiable Simulator. Inside the package created a build directory. Inside build as I execute 'make -j', my ubuntu freezes. I am using Ubuntu 20.04, gcc 11(supports cpp 17). Am I doing something wrong?

closing and re-opening visualizer window doesn't work

todo: repro case
see python/render_gl.py
deleting the app and re-opening doesn't work.
Add a close to p.TinyOpenGL3App("title")

Problem with running ARS CUDA training demos

Hi,

I used the following commands:

git clone --recurse-submodules -j12 [email protected]:google-research/tiny-differentiable-simulator.git
cd tiny-differentiable-simulator
mkdir build
cd build
cmake ..
make -j

And when I run ars_train_policy_cuda it complains about not finding the correct file(s):

X11 functions dynamically loaded using dlopen/dlsym OK!
X11 functions dynamically loaded using dlopen/dlsym OK!
Creating context
Created GL 3.3 context
Direct GLX rendering context obtained
Making context current
GL_VENDOR=NVIDIA Corporation
GL_RENDERER=GeForce GTX 1050 Ti/PCIe/SSE2
GL_VERSION=3.3.0 NVIDIA 460.80
GL_SHADING_LANGUAGE_VERSION=3.30 NVIDIA via Cg compiler
pthread_getconcurrency()=0
Version = 3.3.0 NVIDIA 460.80
Vendor = NVIDIA Corporation
Renderer = GeForce GTX 1050 Ti/PCIe/SSE2
Loading URDF "/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/data/plane_implicit.urdf".
Could not find file "laikago/laikago_toes_zup_xyz_xyzrot.urdf".
Loading URDF "../../tds/data/laikago/laikago_toes_zup_xyz_xyzrot.urdf".
Error, cannot open file_name: ../../tds/data/laikago/laikago_toes_zup_xyz_xyzrot.urdf

I added USE_ANT flag in ars_train_policy_cuda.cpp and I seem to miss another file:

X11 functions dynamically loaded using dlopen/dlsym OK!
X11 functions dynamically loaded using dlopen/dlsym OK!
Creating context
Created GL 3.3 context
Direct GLX rendering context obtained
Making context current
GL_VENDOR=NVIDIA Corporation
GL_RENDERER=GeForce GTX 1050 Ti/PCIe/SSE2
GL_VERSION=3.3.0 NVIDIA 460.80
GL_SHADING_LANGUAGE_VERSION=3.30 NVIDIA via Cg compiler
pthread_getconcurrency()=0
Version = 3.3.0 NVIDIA 460.80
Vendor = NVIDIA Corporation
Renderer = GeForce GTX 1050 Ti/PCIe/SSE2
Loading URDF "/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/data/plane_implicit.urdf".
Loading URDF "/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/data/gym/ant_org_xyz_xyzrot.urdf".
Could not find file "cuda_model_ant.so".
terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to dynamically load library 'cuda_model_ant': ../../tds/data/cuda_model_ant.so: cannot open shared object file: No such file or directory
Aborted (core dumped)

I tried adding the -D USE_CPPAD=ON with cmake command and I am getting compilation errors:

In file included from /home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/cuda_laikago_codegen.cpp:21:
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/laikago_environment.h: In instantiation of ‘std::vector<typename Algebra::Scalar> ContactSimulation<Algebra>::operator()(const std::vector<typename Algebra::Scalar>&) [with Algebra = tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > >; typename Algebra::Scalar = CppAD::AD<CppAD::cg::CG<double> >]’:
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/cuda_laikago_codegen.cpp:58:21:   required from here
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/laikago_environment.h:111:29: error: conversion from ‘double’ to non-scalar type ‘ContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  111 |                 Scalar kp = 100.;
      |                             ^~~~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/laikago_environment.h:112:29: error: conversion from ‘double’ to non-scalar type ‘ContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  112 |                 Scalar kd = 2.;
      |                             ^~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/laikago_environment.h:113:36: error: conversion from ‘double’ to non-scalar type ‘ContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  113 |                 Scalar max_force = 50.;
      |                                    ^~~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/laikago_environment.h:130:51: error: conversion from ‘double’ to non-scalar type ‘ContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  130 |                             Scalar ACTION_LIMIT = 0.4;
      |                                                   ^~~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/laikago_environment.h:140:55: error: conversion from ‘int’ to non-scalar type ‘ContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  140 |                             Scalar desired_velocity = 0;
      |                                                       ^
In file included from /home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/cuda_ant_codegen.cpp:26:
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h: In instantiation of ‘AntContactSimulation<Algebra>::AntContactSimulation() [with Algebra = tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > >]’:
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/cuda_ant_codegen.cpp:51:37:   required from here
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h:79:43: error: conversion from ‘double’ to non-scalar type ‘AntContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
   79 |         Scalar combinedContactStiffness = 1.;
      |                                           ^~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h:80:41: error: conversion from ‘double’ to non-scalar type ‘AntContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
   80 |         Scalar combinedContactDamping = .1;
      |                                         ^~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h: In instantiation of ‘std::vector<typename Algebra::Scalar> AntContactSimulation<Algebra>::operator()(const std::vector<typename Algebra::Scalar>&) [with Algebra = tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > >; typename Algebra::Scalar = CppAD::AD<CppAD::cg::CG<double> >]’:
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/cuda_ant_codegen.cpp:65:21:   required from here
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h:149:33: error: conversion from ‘int’ to non-scalar type ‘AntContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  149 |                     Scalar kp = 15;
      |                                 ^~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h:150:33: error: conversion from ‘double’ to non-scalar type ‘AntContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  150 |                     Scalar kd = 0.3;
      |                                 ^~~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h:151:40: error: conversion from ‘double’ to non-scalar type ‘AntContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  151 |                     Scalar max_force = 3.;
      |                                        ^~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h:166:40: error: conversion from ‘double’ to non-scalar type ‘AntContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  166 |                                 Scalar ACTION_LIMIT_LOW = (i%2==0)? -0.5 : -1.1+0.5;
      |                                        ^~~~~~~~~~~~~~~~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h:167:40: error: conversion from ‘double’ to non-scalar type ‘AntContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  167 |                                 Scalar ACTION_LIMIT_HIGH = (i%2==0)? 0.5 : 0;
      |                                        ^~~~~~~~~~~~~~~~~
/home/shahryar/hdd/test-diff/tiny-differentiable-simulator/examples/environments/ant_environment.h:177:59: error: conversion from ‘int’ to non-scalar type ‘AntContactSimulation<tds::EigenAlgebraT<CppAD::AD<CppAD::cg::CG<double> > > >::Scalar’ {aka ‘CppAD::AD<CppAD::cg::CG<double> >’} requested
  177 |                                 Scalar desired_velocity = 0;
      | 

I was wondering if the way I tried to build the source for these demos are the right way.

Issue during initial install

When trying to install the make -j step returns

Rerunning does the same.

[ 22%] Built target pytinydiffsim
[ 26%] Built target opengl_imgui_test
[ 28%] Built target pendulum_example_gui
[ 31%] Built target tiny_urdf_parser_opengl_example
[ 35%] Built target opengl_test
[ 36%] Built target pytinyopengl3
[ 98%] Built target zmq
make: *** [all] Error 2

I am using GCC and G++ 10, same bug comes up with GCC and G++ 7.
Ran on MacOS and Ubuntu 16

tds joint issue

simulating floating base, or fixed base (with x,y,z translation + spherical joint, to mimic floating base), has some issue.
Attached is a repro case, and pybullet comparison.
tds_joint_issues.zip

Parts of franka_panda not showing

Hi
I tried to run the "tiny_urdf_parser_meshcat_example" with the franka_panda from https://github.com/bulletphysics/bullet3/tree/master/examples/pybullet/gym/pybullet_data/franka_panda

and "link5" , "link6" and "hand" are not shown (see picture)

tiny_urdf_parser_meshcat_example.cpp

// Copyright 2020 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//      http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#include <chrono>
#include <fstream>
#include <iostream>
#include <streambuf>
#include <string>
#include <thread>

#include "visualizer/meshcat/meshcat_urdf_visualizer.h"
#include "math/tiny/tiny_double_utils.h"
#include "utils/file_utils.hpp"
#include "urdf/urdf_parser.hpp"
#include "urdf/urdf_to_multi_body.hpp"
#include "dynamics/forward_dynamics.hpp"
#include "dynamics/integrator.hpp"

typedef double TinyDualScalar;
typedef double MyScalar;
typedef ::TINY::DoubleUtils MyTinyConstants;
#include "math/tiny/tiny_algebra.hpp"
typedef TinyAlgebra<double, MyTinyConstants> MyAlgebra;

using namespace tds;

using namespace TINY;
double knee_angle = 0.0;
double abduction_angle = 0.0;
int frameskip_gfx_sync = 16; // only sync every 10 frames (sim at 1000 Hz, gfx at ~60hz)

double initial_poses[] = {
    abduction_angle,
    0.,
    knee_angle,
    abduction_angle,
    0.,
    knee_angle,
    abduction_angle,
    0.,
    knee_angle,
    abduction_angle,
    0.,
    knee_angle,
};

int main(int argc, char *argv[])
{
  World<MyAlgebra> world;
  UrdfParser<MyAlgebra> parser;

  // create graphics
  MeshcatUrdfVisualizer<MyAlgebra> meshcat_viz;
  meshcat_viz.delete_all();

  std::string plane_file_name;
  FileUtils::find_file("plane_implicit.urdf", plane_file_name);
  char plane_search_path[TINY_MAX_EXE_PATH_LEN];
  FileUtils::extract_path(plane_file_name.c_str(), plane_search_path, TINY_MAX_EXE_PATH_LEN);


  MultiBody<MyAlgebra> &plane_mb = *world.create_multi_body();
  plane_mb.set_floating_base(false);
  {
    UrdfStructures<MyAlgebra> plane_urdf_structures =
        parser.load_urdf(plane_file_name);
    UrdfToMultiBody<MyAlgebra>::convert_to_multi_body(
        plane_urdf_structures, world, plane_mb);
    std::string texture_path = "checker_purple.png";
    meshcat_viz.m_path_prefix = plane_search_path;
    meshcat_viz.convert_visuals(plane_urdf_structures, texture_path);
  }

  char search_path[TINY_MAX_EXE_PATH_LEN];
  std::string file_name;
  FileUtils::find_file("franka_panda/panda.urdf", file_name);
  FileUtils::extract_path(file_name.c_str(), search_path, TINY_MAX_EXE_PATH_LEN);
  std::ifstream ifs(file_name);
  std::string urdf_string;
  if (!ifs.is_open())
  {
    std::cout << "Error, cannot open file_name: " << file_name << std::endl;
    exit(-1);
  }

  urdf_string = std::string((std::istreambuf_iterator<char>(ifs)), std::istreambuf_iterator<char>());
  StdLogger logger;
  UrdfStructures<MyAlgebra> urdf_structures;
  int flags = 0;
  parser.load_urdf_from_string(urdf_string, flags, logger, urdf_structures);


  // create graphics structures
  std::string texture_path = "./meshes/visual/colors.png";
  meshcat_viz.m_path_prefix = search_path;
  meshcat_viz.convert_visuals(urdf_structures, texture_path);

  
  MultiBody<MyAlgebra> &mb = *world.create_multi_body();
  UrdfToMultiBody<MyAlgebra>::convert_to_multi_body(urdf_structures, world, mb);
  mb.initialize();


  TinyVector3<double, DoubleUtils> grav (DoubleUtils::zero(), DoubleUtils::zero(), DoubleUtils::fraction(-981, 100));
  double dt = 1. / 1000.;
  int sync_counter = 0;

  while (1)
  {
    std::this_thread::sleep_for(std::chrono::duration<double>(dt));

    forward_kinematics(mb);

    // pd control
    if (1)
    {
      // use PD controller to compute tau
      int qd_offset = mb.is_floating() ? 6 : 0;
      int q_offset = mb.is_floating() ? 7 : 0;
      int num_targets = mb.tau_.size() - qd_offset;
      std::vector<double> q_targets;
      q_targets.resize(mb.tau_.size());

      double kp = 150;
      double kd = 3;
      double max_force = 550;
      int param_index = 0;

      for (int i = 0; i < mb.tau_.size(); i++)
      {
        mb.tau_[i] = 0;
      }
      int tau_index = 0;
      int pose_index = 0;
      for (int i = 0; i < mb.links_.size(); i++)
      {
        if (mb.links_[i].joint_type != JOINT_FIXED)
        {
          double q_desired = initial_poses[pose_index++];
          double q_actual = mb.q_[q_offset];
          double qd_actual = mb.qd_[qd_offset];
          double position_error = (q_desired - q_actual);
          double desired_velocity = 0;
          double velocity_error = (desired_velocity - qd_actual);
          double force = kp * position_error + kd * velocity_error;

          if (force < -max_force)
            force = -max_force;
          if (force > max_force)
            force = max_force;
          mb.tau_[tau_index] = force;
          q_offset++;
          qd_offset++;
          param_index++;
          tau_index++;
        }
      }
    }

    forward_dynamics(mb, grav);

    world.step(dt);

    integrate_euler(mb, dt);
    sync_counter++;
    if (sync_counter > frameskip_gfx_sync)
    {
      sync_counter = 0;
      meshcat_viz.sync_visual_transforms(&mb);
    }
  }

  printf("finished\n");
  return EXIT_SUCCESS;
}

panda.urdf

<?xml version="1.0" ?>
<!-- =================================================================================== -->
<!-- |    This document was autogenerated by xacro from panda_arm_hand.urdf.xacro      | -->
<!-- |    EDITING THIS FILE BY HAND IS NOT RECOMMENDED                                 | -->
<!-- =================================================================================== -->
<robot name="panda" xmlns:xacro="http://www.ros.org/wiki/xacro">
  <link name="panda_link0">
  	<inertial>
      <origin rpy="0 0 0" xyz="0 0 0.05"/>
       <mass value="2.9"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/collision/link0.obj"/>
      </geometry>
      <material name="panda_white">
    		<color rgba="1. 1. 1. 1."/>
  		</material>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/link0.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <link name="panda_link1">
  	<inertial>
      <origin rpy="0 0 0" xyz="0 -0.04 -0.05"/>
       <mass value="2.7"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/visual/link1.obj"/>
      </geometry>
      <material name="panda_white"/>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/link1.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <joint name="panda_joint1" type="revolute">
    <safety_controller k_position="100.0" k_velocity="40.0" soft_lower_limit="-2.8973" soft_upper_limit="2.8973"/>
    <origin rpy="0 0 0" xyz="0 0 0.333"/>
    <parent link="panda_link0"/>
    <child link="panda_link1"/>
    <axis xyz="0 0 1"/>
    <limit effort="87" lower="-2.9671" upper="2.9671" velocity="2.1750"/>
  </joint>
  <link name="panda_link2">
  	<inertial>
      <origin rpy="0 0 0" xyz="0 -0.04 0.06"/>
       <mass value="2.73"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/visual/link2.obj"/>
      </geometry>
      <material name="panda_white"/>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/link2.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <joint name="panda_joint2" type="revolute">
    <safety_controller k_position="100.0" k_velocity="40.0" soft_lower_limit="-1.7628" soft_upper_limit="1.7628"/>
    <origin rpy="-1.57079632679 0 0" xyz="0 0 0"/>
    <parent link="panda_link1"/>
    <child link="panda_link2"/>
    <axis xyz="0 0 1"/>
    <limit effort="87" lower="-1.8326" upper="1.8326" velocity="2.1750"/>
  </joint>
  <link name="panda_link3">
	  <inertial>
      <origin rpy="0 0 0" xyz="0.01 0.01 -0.05"/>
       <mass value="2.04"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/visual/link3.obj"/>
      </geometry>
      <material name="panda_red">
    		<color rgba="1. 1. 1. 1."/>
  		</material>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/link3.obj"/>
      </geometry>
    </collision>
  </link>
  <joint name="panda_joint3" type="revolute">
    <safety_controller k_position="100.0" k_velocity="40.0" soft_lower_limit="-2.8973" soft_upper_limit="2.8973"/>
    <origin rpy="1.57079632679 0 0" xyz="0 -0.316 0"/>
    <parent link="panda_link2"/>
    <child link="panda_link3"/>
    <axis xyz="0 0 1"/>
    <limit effort="87" lower="-2.9671" upper="2.9671" velocity="2.1750"/>
  </joint>
  <link name="panda_link4">
  	<inertial>
      <origin rpy="0 0 0" xyz="-0.03 0.03 0.02"/>
       <mass value="2.08"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/visual/link4.obj"/>
      </geometry>
      <material name="panda_white"/>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/link4.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <joint name="panda_joint4" type="revolute">
    <safety_controller k_position="100.0" k_velocity="40.0" soft_lower_limit="-3.0718" soft_upper_limit="-0.0698"/>
    <origin rpy="1.57079632679 0 0" xyz="0.0825 0 0"/>
    <parent link="panda_link3"/>
    <child link="panda_link4"/>
    <axis xyz="0 0 1"/>
    <limit effort="87" lower="-3.1416" upper="0.0" velocity="2.1750"/>
  </joint>
  <link name="panda_link5">
  	<inertial>
      <origin rpy="0 0 0" xyz="0 0.04 -0.12"/>
       <mass value="3"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/visual/link5.obj"/>
      </geometry>
      <material name="panda_white"/>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/link5.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <joint name="panda_joint5" type="revolute">
    <safety_controller k_position="100.0" k_velocity="40.0" soft_lower_limit="-2.8973" soft_upper_limit="2.8973"/>
    <origin rpy="-1.57079632679 0 0" xyz="-0.0825 0.384 0"/>
    <parent link="panda_link4"/>
    <child link="panda_link5"/>
    <axis xyz="0 0 1"/>
    <limit effort="12" lower="-2.9671" upper="2.9671" velocity="2.6100"/>
  </joint>
  <link name="panda_link6">
  	<inertial>
      <origin rpy="0 0 0" xyz="0.04 0 0"/>
       <mass value="1.3"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/visual/link6.obj"/>
      </geometry>
      <material name="panda_white"/>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/link6.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <joint name="panda_joint6" type="revolute">
    <safety_controller k_position="100.0" k_velocity="40.0" soft_lower_limit="-0.0175" soft_upper_limit="3.7525"/>
    <origin rpy="1.57079632679 0 0" xyz="0 0 0"/>
    <parent link="panda_link5"/>
    <child link="panda_link6"/>
    <axis xyz="0 0 1"/>
    <limit effort="12" lower="-0.0873" upper="3.8223" velocity="2.6100"/>
  </joint>
  <link name="panda_link7">
  	<inertial>
      <origin rpy="0 0 0" xyz="0 0 0.08"/>
       <mass value=".2"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/collision/link7.obj"/>
      </geometry>
      <material name="panda_white"/>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/link7.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <joint name="panda_joint7" type="revolute">
    <safety_controller k_position="100.0" k_velocity="40.0" soft_lower_limit="-2.8973" soft_upper_limit="2.8973"/>
    <origin rpy="1.57079632679 0 0" xyz="0.088 0 0"/>
    <parent link="panda_link6"/>
    <child link="panda_link7"/>
    <axis xyz="0 0 1"/>
    <limit effort="12" lower="-2.9671" upper="2.9671" velocity="2.6100"/>
  </joint>
  <link name="panda_link8">
  	 <inertial>
      <origin rpy="0 0 0" xyz="0 0 0"/>
       <mass value="0.0"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
  </link>
  <joint name="panda_joint8" type="fixed">
    <origin rpy="0 0 0" xyz="0 0 0.107"/>
    <parent link="panda_link7"/>
    <child link="panda_link8"/>
    <axis xyz="0 0 0"/>
  </joint>
  <joint name="panda_hand_joint" type="fixed">
    <parent link="panda_link8"/>
    <child link="panda_hand"/>
    <origin rpy="0 0 -0.785398163397" xyz="0 0 0"/>
  </joint>
  <link name="panda_hand">
  	<inertial>
      <origin rpy="0 0 0" xyz="0 0 0.04"/>
       <mass value=".81"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/visual/hand.obj"/>
      </geometry>
      <material name="panda_white"/>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/hand.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <link name="panda_leftfinger">
       <contact>
      <friction_anchor/>
      <stiffness value="30000.0"/>
      <damping value="1000.0"/>
      <spinning_friction value="0.1"/>
      <lateral_friction value="1.0"/>
    </contact>
  	<inertial>
      <origin rpy="0 0 0" xyz="0 0.01 0.02"/>
       <mass value="0.1"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <geometry>
        <mesh filename="./meshes/visual/finger.obj"/>
      </geometry>
      <material name="panda_white"/>
    </visual>
    <collision>
      <geometry>
        <mesh filename="./meshes/collision/finger.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <link name="panda_rightfinger">
        <contact>
      <friction_anchor/>
      <stiffness value="30000.0"/>
      <damping value="1000.0"/>
      <spinning_friction value="0.1"/>
      <lateral_friction value="1.0"/>
    </contact>

  	<inertial>
      <origin rpy="0 0 0" xyz="0 -0.01 0.02"/>
       <mass value="0.1"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
    <visual>
      <origin rpy="0 0 3.14159265359" xyz="0 0 0"/>
      <geometry>
        <mesh filename="./meshes/visual/finger.obj"/>
      </geometry>
      <material name="panda_white"/>
    </visual>
    <collision>
      <origin rpy="0 0 3.14159265359" xyz="0 0 0"/>
      <geometry>
        <mesh filename="./meshes/collision/finger.obj"/>
      </geometry>
      <material name="panda_white"/>
    </collision>
  </link>
  <joint name="panda_finger_joint1" type="prismatic">
    <parent link="panda_hand"/>
    <child link="panda_leftfinger"/>
    <origin rpy="0 0 0" xyz="0 0 0.0584"/>
    <axis xyz="0 1 0"/>
    <limit effort="20" lower="0.0" upper="0.04" velocity="0.2"/>
  </joint>
  <joint name="panda_finger_joint2" type="prismatic">
    <parent link="panda_hand"/>
    <child link="panda_rightfinger"/>
    <origin rpy="0 0 0" xyz="0 0 0.0584"/>
    <axis xyz="0 -1 0"/>
    <limit effort="20" lower="0.0" upper="0.04" velocity="0.2"/>
    <mimic joint="panda_finger_joint1"/>
  </joint>
   <link name="panda_grasptarget">
 <inertial>
      <origin rpy="0 0 0" xyz="0 0 0"/>
       <mass value="0.0"/>
       <inertia ixx="0.1" ixy="0" ixz="0" iyy="0.1" iyz="0" izz="0.1"/>
    </inertial>
   </link>
   <joint name="panda_grasptarget_hand" type="fixed">
    <parent link="panda_hand"/>
    <child link="panda_grasptarget"/>
    <origin rpy="0 0 0" xyz="0 0 0.105"/>
  </joint>
  
</robot>

issue (regression?) with floating base

See attached simple case, a rolling sphere.

tiny_urdf_parser_opengl_example.zip

Any support for 6d point Jacobian?

point_jacobian returns 3 x nDof jacobian matrix, how to get 6D jacobian(i.e., similar to rbdl CalcPointJacobian6D)?

The register_graphics_instance call needs one more input argument

I downloaded and installed TDS by doing pip install -e . in the root directory. I think setup.py ran successfully.

Then I tried to run render_obj_gl.py in the python/examples directory. It gave me the following error:

X11 functions dynamically loaded using dlopen/dlsym OK!
X11 functions dynamically loaded using dlopen/dlsym OK!
Creating context
Created GL 3.3 context
Direct GLX rendering context obtained
Making context current
GL_VENDOR=Intel Open Source Technology Center
GL_RENDERER=Mesa DRI Intel(R) UHD Graphics (CML GT2)
GL_VERSION=4.6 (Core Profile) Mesa 20.0.8
GL_SHADING_LANGUAGE_VERSION=4.60
pthread_getconcurrency()=0
Version = 4.6 (Core Profile) Mesa 20.0.8
Vendor = Intel Open Source Technology Center
Renderer = Mesa DRI Intel(R) UHD Graphics (CML GT2)
Traceback (most recent call last):
File "render_obj_gl.py", line 36, in
mesh = app.renderer.register_graphics_instance(shape, pos, orn, color, scaling, opacity)
TypeError: register_graphics_instance(): incompatible function arguments. The following argument types are supported:
1. (self: pytinyopengl3.TinyGLInstancingRenderer, arg0: int, arg1: TINY::TinyVector3<float, TINY::FloatUtils>, arg2: TINY::TinyQuaternion<float, TINY::FloatUtils>, arg3: TINY::TinyVector3<float, TINY::FloatUtils>, arg4: TINY::TinyVector3<float, TINY::FloatUtils>, arg5: float, arg6: bool) -> int
Invoked with: <pytinyopengl3.TinyGLInstancingRenderer object at 0x7f43611bb0f0>, 1, [0.000000 0.000000 0.000000], [0.707107 0.000000 0.000000 0.707107], [1.000000 1.000000 1.000000], [0.100000 0.100000 0.100000], 0.4

So it looks like register_graphics_instance expects 7 input arguments instead of 6. The last input argument should be a bool. I searched online for what the last argument should be, True or False, but I couldn't find it. I entered either one of them. The example ran successfully, so I thought I would give you guys a head up. I'd love to know the correct value though, True or False.

BTW, register_graphics_instance is used in probably all of the python examples.

Thanks!

Additional Collision Models

Hi,
Looking at the collision models available with TDS only planes, spheres and capsules are implemented. Cubes and meshes are currently unavailable. Is there a plan for more collision models to be supported and if so is there a timeline for this?

Missing lBulletRoboticsGUI and lBulletDynamics

OS: MacOS Catalina

I did a homebrew install of Bullet:

I did a python3 install of pybullet:

Now I am trying to build tiny-differntiable-simulator but getting the following clang error:

However, when I look into the brew install of bullet I see those libraries there.

I thought that this issue: bulletphysics/bullet3#2758 might address my problem but I dont see where to add the
TARGET_LINK_LIBRARIES(<YOUR_APP> BulletRoboticsGUI BulletExampleBrowserLib BulletRobotics BulletFileLoader BulletWorldImporter BulletSoftBody BulletDynamics BulletCollision BulletInverseDynamicsUtils BulletInverseDynamics LinearMath OpenGLWindow gwen BussIK Bullet3Common)

Where can I get "cuda_model_laikago.dll" ?

Error

Version = 4.6.0 NVIDIA 531.14
Vendor = NVIDIA Corporation
Renderer = NVIDIA GeForce RTX 3090/PCIe/SSE2
Loading URDF "C:\Users\GLAB\Downloads\dev\tiny-differentiable-simulator\data\plane_implicit.urdf".
Loading URDF "C:\Users\GLAB\Downloads\dev\tiny-differentiable-simulator\data\laikago\laikago_toes_zup_xyz_xyzrot.urdf".
v.geom_type=3v.geom_type=3v.geom_type=3v.geom_type=3v.geom_type=0v.geom_type=3v.geom_type=3v.geom_type=3v.geom_type=0v.geom_type=3v.geom_type=3v.geom_type=3v.geom_type=0v.geom_type=3v.geom_type=3v.geom_type=3v.geom_type=0Creating VectorizedEnvironment for laikago
Creating a normal distribution of size 25000000.
Finished creating a normal distribution.
Could not find file "cuda_model_laikago.dll".
Assertion failed: found, file C:\Users\GLAB\Downloads\dev\tiny-differentiable-simulator\examples\ars\ars_train_policy_cuda.cpp, line 333

After running ars_train_policy_cuda project, I got this error.
Where can I get "cuda_model_laikago.dll" file?

undefined reference to `xg::newGuid()'

I tried to run "make -j", but it has a error.

CMakeFiles/meshcat_zmq_example.dir/meshcat_zmq_example.o: In function generate_uuidabi:cxx11':
meshcat_zmq_example.cpp:(.text+0x13b): undefined reference to xg::newGuid()' collect2: error: ld returned 1 exit status examples/CMakeFiles/meshcat_zmq_example.dir/build.make:105: recipe for target 'examples/meshcat_zmq_example' failed make[2]: *** [examples/meshcat_zmq_example] Error 1

any example about the gradient calculation?

is there any demo about calculating the gradient of system output q, q_{dot} with respect to physical properties parameters(link length/ mass/ inertia...) in python?

Missing link attribute reference in get_world_com()

Hi!

In multibody.get_world_com() there is a reference to link_[link].I.com.
However link.I.com does not exist. Should this perhaps be link.rbi.com?

https://github.com/google-research/tiny-differentiable-simulator/blob/7dfba7c8075622355a557a0c8982d09a7681b8d3/src/multi_body.hpp#L447-L454

Thanks!
Erik

inclined_plane_gui.cpp missing #include <memory>

Thanks for making this resource available. In running make -j from 'getting started' it failed with:

tiny-differentiable-simulator-master/examples/inclined_plane_gui.cpp:35:20: error: ‘shared_ptr’ is not a member of ‘std’
   35 |   std::vector<std::shared_ptr<tds::Geometry<Algebra>>> geoms;
      |                    ^~~~~~~~~~
tiny-differentiable-simulator-master/examples/inclined_plane_gui.cpp:27:1: note: ‘std::shared_ptr’ is defined in header ‘<memory>’; did you forget to ‘#include <memory>’?
   26 | #include "world.hpp"
  +++ |+#include <memory>

The message is correct and adding #include <memory> fixes the issue.

Ubuntu 22.04, gcc 11.2.0.

Differentiable reward functions?

Is it possible to make an environment using TDS such that the reward function is differentiable? It seems like if the reward in question is just a direct output from the simulator (maybe just the velocity along a certain axis), or if I construct the reward function using some of the functionality provided in src/math etc that this should be possible, but perhaps I am misunderstanding something.

Compilation error: "‘dof_’ was not declared in this scope"

The following error was thrown during compilation. (Ubuntu 20.04)

/home/kvasios/tiny-differentiable-simulator/examples/../src/urdf/../multi_body.hpp:97:17: error: ‘dof_’ was not declared in this scope; did you mean ‘dof’?
97 | conv.dof_ = dof_;
| ^~~~
| dof
/home/kvasios/tiny-differentiable-simulator/examples/../src/urdf/../multi_body.hpp: In constructor ‘tds::MultiBody::MultiBody(const tds::MultiBody&)’:
/home/kvasios/tiny-differentiable-simulator/examples/../src/urdf/../multi_body.hpp:371:9: error: class ‘tds::MultiBody’ does not have any field named ‘dof_’
371 | dof_(mb.dof_),
| ^~~~
make[2]: *** [examples/CMakeFiles/tiny_urdf_parser_example.dir/build.make:63: examples/CMakeFiles/tiny_urdf_parser_example.dir/tiny_urdf_parser_example.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:901: examples/CMakeFiles/tiny_urdf_parser_example.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:141: all] Error 2

issue with spherical joint humanoid, mass matrix assert

Unzip/copy attached into the examples directory and compile and run. They are modified to load the humanoid urdf with spherical joints. Below are also callstacks pointing to the assert (in Visual Studio).

Eigen version:
laikago_opengl_example.zip

callstack:

>	laikago_opengl_eigen_example.exe!tds::EigenAlgebraT<double>::assign_block(Eigen::Matrix<double,-1,-1,0,-1,-1> & output, const Eigen::Matrix<double,-1,-1,0,-1,-1> & input, int i, int j, int m, int n, int input_i, int input_j) Line 452	C++
 	laikago_opengl_eigen_example.exe!tds::mass_matrix<tds::EigenAlgebraT<double>>(tds::MultiBody<tds::EigenAlgebraT<double>> & mb, const Eigen::Matrix<double,-1,1,0,-1,1> & q, Eigen::Matrix<double,-1,-1,0,-1,-1> * M) Line 84	C++
 	laikago_opengl_eigen_example.exe!tds::mass_matrix<tds::EigenAlgebraT<double>>(tds::MultiBody<tds::EigenAlgebraT<double>> & mb, Eigen::Matrix<double,-1,-1,0,-1,-1> * M) Line 132	C++
 	laikago_opengl_eigen_example.exe!tds::MultiBodyConstraintSolver<tds::EigenAlgebraT<double>>::resolve_collision(const std::vector<tds::MultiBodyContactPoint<tds::EigenAlgebraT<double>>,std::allocator<tds::MultiBodyContactPoint<tds::EigenAlgebraT<double>>>> & cps, const double & dt) Line 201	C++
 	laikago_opengl_eigen_example.exe!tds::World<tds::EigenAlgebraT<double>>::step(const double & dt) Line 357	C++
 	laikago_opengl_eigen_example.exe!LaikagoSimulation<tds::EigenAlgebraT<double>>::operator()(const std::vector<double,std::allocator<double>> & v) Line 180	C++
 	laikago_opengl_eigen_example.exe!main(int argc, char * * argv) Line 364	C++

Tiny (has known issues with spherical, so best to get Eigen version fixed first)
tiny_urdf_parser_opengl_example.zip

callstack:

 	tiny_urdf_parser_opengl_example.exe!TINY::DoubleUtils::FullAssert(bool a) Line 91	C++
 	tiny_urdf_parser_opengl_example.exe!TINY::TinyMatrixXxX_<double,TINY::DoubleUtils,TINY::TinyVectorX>::operator()(int row, int col) Line 84	C++
 	tiny_urdf_parser_opengl_example.exe!TinyAlgebra<double,TINY::DoubleUtils>::assign_block<TINY::TinyVectorX>(TINY::TinyMatrixXxX_<double,TINY::DoubleUtils,TINY::TinyVectorX> & output, const TINY::TinyMatrix6x3<double,TINY::DoubleUtils> & input, int i, int j, int m, int n, int input_i, int input_j) Line 365	C++
>	tiny_urdf_parser_opengl_example.exe!tds::mass_matrix<TinyAlgebra<double,TINY::DoubleUtils>>(tds::MultiBody<TinyAlgebra<double,TINY::DoubleUtils>> & mb, const TINY::TinyVectorX<double,TINY::DoubleUtils> & q, TINY::TinyMatrixXxX_<double,TINY::DoubleUtils,TINY::TinyVectorX> * M) Line 86	C++
 	tiny_urdf_parser_opengl_example.exe!tds::mass_matrix<TinyAlgebra<double,TINY::DoubleUtils>>(tds::MultiBody<TinyAlgebra<double,TINY::DoubleUtils>> & mb, TINY::TinyMatrixXxX_<double,TINY::DoubleUtils,TINY::TinyVectorX> * M) Line 132	C++
 	
[tiny_urdf_parser_opengl_example.zip](https://github.com/google-research/tiny-differentiable-simulator/files/5936022/tiny_urdf_parser_opengl_example.zip)
[tiny_urdf_parser_opengl_example.zip](https://github.com/google-research/tiny-differentiable-simulator/files/5936023/tiny_urdf_parser_opengl_example.zip)
[tiny_urdf_parser_opengl_example.zip](https://github.com/google-research/tiny-differentiable-simulator/files/5936024/tiny_urdf_parser_opengl_example.zip)


tiny_urdf_parser_opengl_example.exe!tds::MultiBodyConstraintSolver<TinyAlgebra<double,TINY::DoubleUtils>>::resolve_collision(const std::vector<tds::MultiBodyContactPoint<TinyAlgebra<double,TINY::DoubleUtils>>,std::allocator<tds::MultiBodyContactPoint<TinyAlgebra<double,TINY::DoubleUtils>>>> & cps, const double & dt) Line 201	C++
 	tiny_urdf_parser_opengl_example.exe!tds::World<TinyAlgebra<double,TINY::DoubleUtils>>::step(const double & dt) Line 357	C++
 	tiny_urdf_parser_opengl_example.exe!ContactSimulation<TinyAlgebra<double,TINY::DoubleUtils>>::operator()(const std::vector<double,std::allocator<double>> & v) Line 156	C++
 	tiny_urdf_parser_opengl_example.exe!main(int argc, char * * argv) Line 335	C++

The effect of accelerations on the velocities is added twice

Considering the example in ant_environment.h or laikago_environment.h, the Semi-Implicit-Euler integration takes place in the following snippet

tiny-differentiable-simulator/examples/environments/ant_environment.h

Lines 196 to 200 in c1a9e06

	integrate_euler_qdd(*mb_, dt);
	world.step(dt);
	tds::integrate_euler(*mb_, dt);

First integrate_euler_qdd(*mb_, dt); is called to add the effect of accelerations into change of velocities,
then tds::integrate_euler(*mb_, dt); is called to integrate once again.

But if we look into integrator.hpp, we find the integration of acceleration on both functions, i.e in integrate_euler(..):

tiny-differentiable-simulator/src/dynamics/integrator.hpp

Lines 95 to 98 in c1a9e06

	if (link.joint_type == JOINT_SPHERICAL) {
	qd[qdindex] += qdd[qdindex] * dt;
	qd[qdindex + 1] += qdd[qdindex + 1] * dt;
	qd[qdindex + 2] += qdd[qdindex + 2] * dt;

and in integrate_euler_qdd(..)

tiny-differentiable-simulator/src/dynamics/integrator.hpp

Lines 173 to 175 in c1a9e06

	qd[qdindex] += qdd[qdindex] * dt;
	qd[qdindex + 1] += qdd[qdindex + 1] * dt;
	qd[qdindex + 2] += qdd[qdindex + 2] * dt;

Is this meant to be like this or is this an error and it should be taken care of?

Support for prismatic series elastic actuator?

Hi, is it possible to model a prismatic joint which has a spring and can be supplied force via a linear elastic actuator?

Thank you!

ModuleNotFoundError for pytinydiffsim_ad and AttributeError for pytinydiffsim when running Python code

I thought I'd just gotten Python working after finding #63. But I was mistaken. I've found other errors as I've played with the example files for Python. Apologies if you would prefer if I had created multiple issues, one for each error.

It's especially odd to me that I can import pytinydiffsim_dual just fine, but not pytinydiffsim_ad.

$ python cppad_example.py
Traceback (most recent call last):
  File "cppad_example.py", line 1, in <module>
    import pytinydiffsim_ad as pd
ModuleNotFoundError: No module named 'pytinydiffsim_ad'

$ python b_urdf_structs_example.py
Traceback (most recent call last):
  File "b_urdf_structs_example.py", line 47, in <module>
    sphere_mb.set_base_position(pd.Vector3(0.,0.,25.))
AttributeError: 'pytinydiffsim.TinyMultiBody' object has no attribute 'set_base_position'

But other examples do work (each imports pytinydiffsim and/or pytinydiffsim_dual):

$ python e_load_urdf.py
robot_name= plane

$ python billiard_optimization.py
sphere2red_path= /mnt/c/users/mretc/OneDrive/Documents/Grad School/thesis/motion/tiny-differentiable-simulator/data/sphere2red.urdf

I don't know enough about automatic differentiation to know if I need pytinydiffsim_ad if I have pytinydiffsim_dual.

torch_example.cpp no known converstion from at:Tensor to const btScalar&

/usr/local/include/bullet/LinearMath/btVector3.h:126:20: note: no known conversion for argument 1 from ‘at::Tensor’ to ‘const btScalar& {aka const double&}’

In this example, the sync_graphics_transforms function appears to try to create a btVector3 directly using at:Tensor

  static void sync_graphics_transforms(
      const TinyMultiBody<TinyScalar, TinyConstants>* body,
      class b3RobotSimulatorClientAPI_NoDirect& viz_api) {
    for (int v = 0; v < body->m_visual_uids1.size(); v++) {
      int visual_id = body->m_visual_uids1[v];
      TinyQuaternion<TinyScalar, TinyConstants> rot;
      TinySpatialTransform<TinyScalar, TinyConstants> geom_X_world =
          body->m_base_X_world * body->m_X_visuals[v];
      btVector3 base_pos(geom_X_world.m_translation.getX(),
                         geom_X_world.m_translation.getY(),
                         geom_X_world.m_translation.getZ());
      geom_X_world.m_rotation.getRotation(rot);
      btQuaternion base_orn(rot.getX(), rot.getY(), rot.getZ(), rot.getW());
      viz_api.resetBasePositionAndOrientation(visual_id, base_pos, base_orn);
    }

I noticed that in other functions there is a wrapper function to to a conversion. Perhaps it's missing one here?

      joint.joint_origin_xyz.setValue(TinyConstants::scalar_from_double(pos.getOrigin()[0]),
                                      TinyConstants::scalar_from_double(pos.getOrigin()[1]),
                                      TinyConstants::scalar_from_double(pos.getOrigin()[2]));

here is the compilation line... I'm using c++ torch 1.6

~/PycharmProjects/tiny-differentiable-simulator/examples$ /usr/bin/c++ -DBT_USE_DOUBLE_PRECISION -DCERES_USE_CXX11 -DCPPAD -DCPPADCG -DHPIPM -DSTATIC_LINK_SPD_PLUGIN -DUSE_EIGEN -DZMQ_STATIC -I/home/duane/PycharmProjects/tiny-differentiable-simulator/third_party/eigen3 -I/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/. -I/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/.. -I/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/../third_party/zeromq/include -I/usr/local/include/bullet -I/usr/include/python3.6m -I/usr/lib/llvm-7/include -I/usr/local/include -I/usr/include/eigen3 -isystem /home/duane/Downloads/libtorch/include -I/usr/local/include/bullet/LinearMath -isystem /home/duane/Downloads/libtorch/include/torch/csrc/api/include -fPIC -D_GLIBCXX_USE_CXX11_ABI=0 -std=gnu++1z -o CMakeFiles/torch_example.dir/torch_example.o -c /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/torch_example.cpp
In file included from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/tiny_system_constructor.h:20:0,
from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/torch_example.cpp:15:
/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/pybullet_urdf_import.h: In instantiation of ‘static void PyBulletUrdfImport<TinyScalar, TinyConstants>::sync_graphics_transforms(const TinyMultiBody<TinyScalar, TinyConstants>*, b3RobotSimulatorClientAPI_NoDirect&) [with TinyScalar = at::Tensor; TinyConstants = TorchUtils]’:
/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/torch_example.cpp:173:40: required from here
/home/duane/PycharmProjects/tiny-differentiable-simulator/examples/pybullet_urdf_import.h:153:17: error: no matching function for call to ‘btVector3::btVector3(at::Tensor, at::Tensor, at::Tensor)’
btVector3 base_pos(geom_X_world.m_translation.getX(),
^~~~~~~~
In file included from /usr/local/include/bullet/SharedMemory/b3RobotSimulatorClientAPI_NoDirect.h:9:0,
from /usr/local/include/bullet/SharedMemory/b3RobotSimulatorClientAPI_NoGUI.h:4,
from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/pybullet_visualizer_api.h:22,
from /home/duane/PycharmProjects/tiny-differentiable-simulator/examples/torch_example.cpp:10:
/usr/local/include/bullet/LinearMath/btVector3.h:126:20: note: candidate: btVector3::btVector3(const btScalar&, const btScalar&, const btScalar&)
SIMD_FORCE_INLINE btVector3(const btScalar& _x, const btScalar& _y, const btScalar& _z)
^~~~~~~~~
/usr/local/include/bullet/LinearMath/btVector3.h:126:20: note: no known conversion for argument 1 from ‘at::Tensor’ to ‘const btScalar& {aka const double&}’
/usr/local/include/bullet/LinearMath/btVector3.h:117:20: note: candidate: btVector3::btVector3()
SIMD_FORCE_INLINE btVector3()
^~~~~~~~~
/usr/local/include/bullet/LinearMath/btVector3.h:117:20: note: candidate expects 0 arguments, 3 provided
/usr/local/include/bullet/LinearMath/btVector3.h:81:1: note: candidate: constexpr btVector3::btVector3(const btVector3&)
btVector3
^~~~~~~~~
/usr/local/include/bullet/LinearMath/btVector3.h:81:1: note: candidate expects 1 argument, 3 provided
/usr/local/include/bullet/LinearMath/btVector3.h:81:1: note: candidate: constexpr btVector3::btVector3(btVector3&&)
/usr/local/include/bullet/LinearMath/btVector3.h:81:1: note: candidate expects 1 argument, 3 provided

unable to use python bindings

I am able to run CPP examples after following the build instructions mentioned in the readme. However, I am unable to run python examples present in python/examples directory.

Setup steps

• mkdir build && cd build
• cmake .. && make
• export PYTHONPATH=$(pwd)/python
• export LD_LIBRARY_PATH=$(pwd)/python

Usage

• python python/examples/billiard_optimization.py

Error

Traceback (most recent call last):
  File "python/examples/billiard_optimization.py", line 7, in <module>
    import pytinyopengl3 as p
ModuleNotFoundError: No module named 'pytinyopengl3'

I do see libpytinydiffsim_dual.so libpytinydiffsim.so libpytinyopengl3.so in the build/python folder. It seems to that python is unable to load the modules.

Kindly let me know if there are other steps in using python bindings.