tud-amr / fabrics Goto Github PK

I am going to implement that radius_body_panda_link{i} can be provided as a list of inputs.

When trying to concretize() the unpulled geometry leave of a goal geometry, it gives the error that weight_goal_0 is not provided.
# mapping = leave.map()
# mapping.concretize()
# unpulled_geometry=leave._geo #weight_goal_0=goal.sub_goals()[0].weight() must be inputted somewhere.
# unpulled_geometry.concretize()

Hi @maxspahn ,

I have forked Fabrics and created a new local environment for it. When manually running files in 'examples' and 'tutorials', I encountered some issues, as described below:

1. The color of the obstacles has changed to black instead of red.
2. When I run point_robot_esdf directly, I receive the following errors. This didn't happen before and I suspect it's due to def test_edf_evaluation(proj_rgb):. Initially, I installed pytest and attempted to run it, but this only led to more error messages. Commenting out the code seems to solve the problem temporarily.
   File "", line 1007, in _find_and_load
   File "", line 984, in _find_and_load_unlocked
   ModuleNotFoundError: No module named 'pytest'

Additionally, it seems like the package 'plt' isn't imported even though some codes are using it.
if logging.root.level <= 10: plt.subplot(1, 2, 1) plt.imshow(proj_rgb) plt.subplot(1, 2, 2) plt.imshow(proj_depth) plt.show() test_edf_evaluation(proj_rgb)

3. For the file point robot urdf passage, In the file point_robot_urdf_passage, the following code doesn't work, producing error messages regardless of whether the value is 0 or 1. Modifying the line to res = run_point_robot_urdf(n_steps=6000, render=True) seems to make it work fine.
   import sys res = run_point_robot_urdf(n_steps=6000, render=bool(int(sys.argv[1])))

@xueminchi

Current version of Mpscenes in the devel branch is too low to use the multiple color blocks that were added to mpscenes (should be version=0.4.2)

In urdfenvs 0.7.0, the support for specific robots was stopped, thus relying on generic urdf models with specified urdf files. For some reason, this is breaking the non-holonomic robos. I suspect it is related to the forward kinematics.

Collision avoidance is inherently difficult with obstacles that are not spherical.
However, plane-robot collision avoidance is feasible and should be accessible. See https://arxiv.org/abs/2109.10443.

As suggested in the talk by Nathan, https://www.youtube.com/watch?v=VsM-kdk74d8&list=PL_oEZ6dld4ignAdbFvcP_LAJgNbdrNBKC&index=10, it seems better to clearly split between geomtry and forcing policy.

This should also be reflected in the code structure. Potentially, the pure geometry should be serialized so that the forcing can be changed at runtime easily.

Mentioned in #96

For the file point robot urdf passage, In the file point_robot_urdf_passage, the following code doesn't work, producing error messages regardless of whether the value is 0 or 1. Modifying the line to res = run_point_robot_urdf(n_steps=6000, render=True) seems to make it work fine.
import sys res = run_point_robot_urdf(n_steps=6000, render=bool(int(sys.argv[1])))

If no command line argument is passed, rendering should be activated.

Collision avoidance (and self-collision avoidance) for that matter should be specified by the user.
In fabrics, such things are realized using spheres on individuals parts of the robot, such that the union of such sphere englobe the robot.
Ideally, the paramaterized_planner takes the name of the links you want to place spheres on.
collision_avoidance_links: list = [<names>]

For self-collision avoidance you would want to specify pairs as dictonaries
self_collision_avoidance_links: dict = {'panda_ee': [<link_names>], <name_2>: [<link_names_2>]}

Do you agree with that @tckarenchiang

For if others want to install the 'fabrics' package using Windows, installing it using Poetry on Windows with Python v3.8.10 gave me an error while trying to install Pybullet v3.2.2. Apparently, Pybullet requires Build Tools for Visual Studio 2022. Installing this solved the issue for me.

Thanks @casparvv .
Will be added to the readme guide for windows.

I made a new fork of the fabrics develop branch.
To make the poetry environment work, I still had to install urdfenvs manually via: poetry run pip install urdfenvs.

if I select one of the last links for obstacle avoidance (panda_link_8), the planar constraint seems to be overconservative. Therefore the block on the ground cannot be reached. The panda_hand link (for goal reaching) stops at around 25 cm above the ground.
If for example panda_link_3 is only considered for obstacle avoidance, this is not an issue.

Fabrics output acceleration which is almost impossible to track for any robot as acceleration measurements are almost sure to be inaccurate. Therefore, the output almost always have to be integrated to obtain velocity actions. Similar to the RMP implementation in omniverse, we should add the integration to fabrics directly to allow better comparison to other local motion generators.

https://arxiv.org/abs/2309.07368

That paper shows how inequaliy and equality constraints can be realized with fabrics paving the way for contact integration.

I learned about WARP today and it seems that it allows to create CUDA kernels from python.
That might be the way to go to compute fabrics on GPU.

https://github.com/NVIDIA/warp

Speed control is done using several different energization which seems opaque to most users.
This should be improved by allowing for very simple damping/speed control schemes.

https://github.com/maxspahn/fabrics/blob/24ab36e84fd5101591a5c376dbdbd5eaec71ffd9/fabrics/planner/parameterized_planner.py#L335-L336

With the changes in 849d532, the rotation matrices are broken.

As mentioned in #96, the example point_robot_esdf.py requires some packages that are not installed by default.

When I run point_robot_esdf directly, I receive the following errors. This didn't happen before and I suspect it's due to def test_edf_evaluation(proj_rgb):. Initially, I installed pytest and attempted to run it, but this only led to more error messages. Commenting out the code seems to solve the problem temporarily.
File "", line 1007, in _find_and_load
File "", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'pytest'

Suggested fix, remove all these parts that are not needed and if needed add comment.

Hi Max, the newest version of ForwardKinematics=1.2.0 causes an error in (as far as I know) all examples on the develop branch.

ModuleNotFoundError: No module named 'forwardkinematics.fksCommon.fk_creator'

I didn't examine the problem that well for all cases, but for me it worked to limit to maximal forwardkinematics==1.1.5

When trying to poetry update the current develop branch, the following error occurs:

SolverProblemError

 Because planarenvs (rev master) depends on forwardkinematics (^0.5.0)
  and fabrics depends on forwardkinematics (rev main), planarenvs is forbidden.
 So, because fabrics depends on planarenvs (rev master), version solving failed.

 at ~/.poetry/lib/poetry/puzzle/solver.py:241 in _solve
     237│             packages = result.packages
     238│         except OverrideNeeded as e:
     239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
     240│         except SolveFailure as e:
   → 241│             raise SolverProblemError(e)
     242│ 
     243│         results = dict(
     244│             depth_first_search(
     245│                 PackageNode(self._package, packages), aggregate_package_nodes

Looks like there is a mismatched python version for forwardkinematics

Geomtry and finsler structure can be symbolic.
That means that for every limit there will be a new variable. Currently, the same variable will be used for the upper and lower limit as the name of the variable does not take into account which limit is currently used.

https://github.com/maxspahn/fabrics/blob/4b8b4b9ab8f0200f0cd239b379a962abbe568175/fabrics/components/leaves/geometry.py#L59-L68

Does fabrics support cylinder, capsule or urdf obstacles? It seems now only support static sphere or moving sphere and static cuboid obstacles. Also, if num of obstacles increases to 5 or more, the planning time is quite slow.

Hello, thank you for sharing the code. However, I encountered some errors while trying to run the examples in this repository which caused it to stop running. Specifically, below is the error information that occurred when attempting to run "panda_dynamic_avoidance.py".

Unknown tag "inertial" in /robot[@name='panda']
Unknown tag "inertial" in /robot[@name='panda']
WARNING:root:Casadi pseudo inverse is used in Lagrangian
WARNING:root:Casadi pseudo inverse is used in spec

Traceback (most recent call last):
File "/home/cxm/PycharmProjects/fabrics/examples/panda_dynamic_avoidance.py", line 185, in
res = run_panda_example(n_steps=5000)
File "/home/cxm/PycharmProjects/fabrics/examples/panda_dynamic_avoidance.py", line 172, in run_panda_example
ob, *_ = env.step(action)
File "/home/cxm/anaconda3/envs/fabric_test/lib/python3.9/site-packages/gym/wrappers/order_enforcing.py", line 13, in step
observation, reward, done, info = self.env.step(action)
File "/home/cxm/PycharmProjects/fabrics/urdfenvs/urdf_common/urdf_env.py", line 184, in step
ob = self._get_ob()
File "/home/cxm/PycharmProjects/fabrics/urdfenvs/urdf_common/urdf_env.py", line 205, in _get_ob
check_observation(self.observation_space, observation)
File "/home/cxm/PycharmProjects/fabrics/urdfenvs/urdf_common/urdf_env.py", line 28, in check_observation
check_observation(obs[key], value)
File "/home/cxm/PycharmProjects/fabrics/urdfenvs/urdf_common/urdf_env.py", line 28, in check_observation
check_observation(obs[key], value)
File "/home/cxm/anaconda3/envs/fabric_test/lib/python3.9/site-packages/gym/spaces/dict.py", line 122, in getitem
return self.spaces[key]
KeyError: 'FullSensor'

I have no idea about this error. When I try to run other examples, I still encounter this error.

Potentially using a jupyter notebook so that people can actually click through it.
It should explain the things that Ratliff is usually talking about in his talks.

When the number of geomeries to be combined increases, the planner initiialization scales very badly (probably exponentially). This is mainly due to multiple calls of the function applyEulerLagrange in the energy. However, many of these calls are not needed as th corresponding spec and hamiltonian can be found by simple summation. In this snippet, the constructor call could be atered to allow the spec and hamlitonion to be passed.

Hi @maxspahn ,

I've finished the simple tests on point robot and panda robot considering any shapes. I'm working on non-holonomic robots now. I try to add some visualizations but I don't know the size of the 'boxer', the size of 'chassis_link'. Where I can get the information?

best,
@xueminchi

https://github.com/maxspahn/fabrics/blob/20f0b25ce683caec22e051a5e77171f2b9f0c5fd/fabrics/components/leaves/geometry.py#L90-L100

The radius for the self-collision links is fixed and cannot be modified like the collision link sizes.

Some text is missing in the README file:

Hi @maxspahn ,

The error message is under below:

Traceback (most recent call last):
  File "/home/cxm/PycharmProjects/fabrics/examples/panda_capsules.py", line 181, in <module>
    res = run_panda_capsule_example(n_steps=5000)
  File "/home/cxm/PycharmProjects/fabrics/examples/panda_capsules.py", line 148, in run_panda_capsule_example
    planner = set_planner(goal)
  File "/home/cxm/PycharmProjects/fabrics/examples/panda_capsules.py", line 129, in set_planner
    tf_capsule_origin =  forward_kinematics.casadi(
  File "/home/cxm/anaconda3/envs/fabrics/lib/python3.9/site-packages/forwardkinematics/**urdfFks/generic_urdf_fk.py**", line 59, in casadi
    if positionOnly:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

if positionOnly:
    fk = fk[0:3, 3]
return fk

@xueminchi

For multiple static obstacles, the planner.compute_action() can use x_obsts and radius_obsts as an input.
If I try to add multiple dynamic obstacles, it is not possible to get an action with x_dynamic_obsts (or something similar), and I have to give the input for each robot separately. This would be nice to add :)

Hi @maxspahn,

I want to contribute to Fabrics, but I'm not entirely sure if the steps I'm planning to follow are correct. As I'm not very familiar with GitHub, could you please review them? I'm sharing these steps here in case there are other people in the same situation as me.

First, I plan to fork the repo and then clone it to my local server. This will allow me to develop locally. Once I've completed and checked my work, I intend to initiate a pull request to you.

Could you please confirm if this process is correct? Or is there another preferred method?

Best,
@xueminchi

When I would like to reset the environment (to for example run as second experiment), this is not possible, because the goal is already defined. I will get the error:
AttributeError: 'GoalComposition' object has no attribute 'position'
This is quite impractical for running multiple experiments in one script with 1 environment (also for RL).

Type annotations have been proven very useful for understanding of the functions and the entire package actually. However, most functions and attributes are not annotated, yet.
Addressing this issue should ease the entrance into geometric fabrics.

The example agents in gym_agents/running/panda and gym_agents/running/planar using the older fabrics/optFabrics_DEPRACATED package don't work due to missing dependencies.

All the example agents in gym_agents/running/planar/diffGeometry do work.

I installed the package with poetry install -E agents using Poetry v1.1.12 and Python v3.8.10.

Example of an agent not working:
gym_agents/running/planar/pointMass.py

It doesn't run since the following imports, on line 2 and lines 9-12 can't be found:

import pointRobot
from optFabrics.controllers.staticController import StaticController
from obstacle import Obstacle
from robotPlot import RobotPlot

Suggestion: clean them up/state that these agents are also deprecated, or include the dependencies into the Poetry installation.

The videos are outdated and should better reflect the examples/tutorials.
Also, the list of references should be complete:

• Videos by Ratliff
• Papers by nvidia
• Paper by me

As the size of the planning problem increases (10+ obstacles, 10+ DoF), the generation of the solver is quite costly.
This could be improved by 'storing' some of the planners. Serialization seems to be a good solution to that.

All orientation task in examples are defined in childlink. How to set orientation in world base frame?