rr-learning / causalworld Goto Github PK

pybullet IK is causing a memory leak.

The 'get_achieved_goal' function, in e.g. reaching, returns the end_effector_positions from the '_latest_full_state' dict. This is not updated in 'joint_torque' mode, giving constant values for the achieved goal.

I suggest to split the purpose of the last_full_state.
There should be a structure to store the endeffector position: current_state that is updated just once every step (I think in
do_simulation() after the physics update)

then the storage of the last_full_state and its update is a different issue that I report in a moment.

CausalWorld/causal_world/wrappers/env_wrappers.py", line 6, in <module>
    class HERGoalEnvWrapper(gym.GoalEnv):
AttributeError: module 'gym' has no attribute 'GoalEnv'

gym.GoalEnv was removed with version 0.22: openai/gym@616b071

In 'get_reward' in base_task you add the goal_distance to the reward. Given that you typically try to maximize the reward in the RL setting, shouldn't be the negative distance added to the reward?

Hi, when performing an intervention on the friction of the floor or stage, I noticed that the 'ground truth' info dictionary did not return the new, correct friction values. However, the environment physics showed that the effect corresponding to the changed frictions. To me, it seems that this line is a bug: https://github.com/rr-learning/CausalWorld/blob/master/causal_world/envs/scene/stage.py#L516. The client ID is there set to a bool, which would be constant 'True', i.e. interpreted as 1, while the actual client ID is 0. Replacing this line with client = self._pybullet_client_w_o_goal_id fixes the issue for me.

Is this indeed a bug, or a misinterpretation on my side? If a bug, I can set up a pull request if wanted.

Hey,

I would like to use trifinger_pro (trifinger) but there are some stl files missing (SIM_*.stl). Is it possible to add them again?
Also I get the following error, if I try to run the pro version (with the stl files taken from the challenge code)

Traceback (most recent call last):
  File "~/.local/share/virtualenvs/mbrl-COlWRWWx/src/causalworld/python/src/causal_world/task_generators/base_task.py", line 677, in init_task
    self.save_state()
  File "~/.local/share/virtualenvs/mbrl-COlWRWWx/src/causalworld/python/src/causal_world/task_generators/base_task.py", line 157, in save_state
    self._robot.get_full_env_state()
  File "~/.local/share/virtualenvs/mbrl-COlWRWWx/src/causalworld/python/src/causal_world/envs/robot/trifinger.py", line 118, in get_full_env_state
    return self.get_current_scm_values()
  File "~/.local/share/virtualenvs/mbrl-COlWRWWx/src/causalworld/python/src/causal_world/envs/robot/trifinger.py", line 539, in get_current_scm_values
    [WorldConstants.VISUAL_SHAPE_IDS[robot_finger_link]][7][:3]
IndexError: tuple index out of range

Best,
Sebastian

Hi, when trying to create an environment that runs in the observation mode "pixel", the environment crashes due to the attribute self.observation_space not being defined:

from causal_world.envs import CausalWorld
from causal_world.task_generators.task import generate_task

task = generate_task(task_generator_id='stacked_blocks')
env = CausalWorld(task=task,
                  observation_mode='pixel')

Error:

File causalworld.py:238, in CausalWorld._reset_observations_space(self)
    237 def _reset_observations_space(self):
--> 238     if self._observation_mode == "pixel" and self.observation_space is None:
    239         self._stage.select_observations(["goal_image"])
    240         self.observation_space = combine_spaces(
    241             self._robot.get_observation_spaces(),
    242             self._stage.get_observation_spaces())

AttributeError: 'CausalWorld' object has no attribute 'observation_space'

I am not sure if this might be caused by a new gym version. A quick fix is to simply define self.observation_space = None before calling self._reset_observations_space() in the init (line 171).

We noticed that set_full_env_state of the stage is recreating the objects in the scene which will create a different physical state, so running the simulation from a reloaded state yields different results than reloading.

In case interventions should be applied on the fly it might be good to only change properties and not recreate objects all the time.

For our use case we only want to store and set states without changing the structure of the envs (no new objects, etc.). For that reason we tried to use stage.set_full_state/get_full_state pair of function instead of the full_env_state functions. These didn't worked out of the box but we fixed some errors, see the diffs below.

With this changes we getting already smaller deviations between the original environment and a reseted copy. But we still get some deviations for the object and we don't know where they come from.

(In general, the list version of the state should be removed for clarity/bugs reasons)

Index: python/src/causal_world/envs/scene/silhouette.py
===================================================================
--- python/src/causal_world/envs/scene/silhouette.py	(date 1597787231831)
+++ python/src/causal_world/envs/scene/silhouette.py	(date 1597787231831)
@@ -211,11 +211,14 @@
                 self._pybullet_client_ids[0])
             position = np.array(position)
             position[-1] -= WorldConstants.FLOOR_HEIGHT
+            cylindrical_position = cart2cyl(np.array(position))
             for name in self._state_variable_names:
                 if name == 'type':
                     state.append(self._type_id)
                 elif name == 'cartesian_position':
                     state.extend(position)
+                elif name == 'cylindrical_position':
+                    state.extend(cylindrical_position)
                 elif name == 'orientation':
                     state.extend(orientation)
                 elif name == 'size':
@@ -445,7 +448,7 @@
         position[-1] += WorldConstants.FLOOR_HEIGHT
         shape_id = pybullet.createVisualShape(
             shapeType=pybullet.GEOM_BOX,
-            halfExtents=self._size / 2,
+            halfExtents=np.asarray(self._size) / 2,
             rgbaColor=np.append(self._color, self._alpha),
             physicsClientId=pybullet_client_id)
         block_id = pybullet.createMultiBody(baseVisualShapeIndex=shape_id,

Index: python/src/causal_world/envs/scene/objects.py
===================================================================
--- python/src/causal_world/envs/scene/objects.py	(date 1597787843999)
+++ python/src/causal_world/envs/scene/objects.py	(date 1597787843999)
@@ -384,6 +384,7 @@
                 self._pybullet_client_ids[0])
             position = np.array(position)
             position[-1] -= WorldConstants.FLOOR_HEIGHT
+            cylindrical_position = cart2cyl(np.array(position))
             if self.is_not_fixed():
                 linear_velocity, angular_velocity = pybullet.\
                     getBaseVelocity(
@@ -395,6 +396,8 @@
                     state.append(self._type_id)
                 elif name == 'cartesian_position':
                     state.extend(position)
+                elif name == 'cylindrical_position':
+                    state.extend(cylindrical_position)
                 elif name == 'orientation':
                     state.extend(orientation)
                 elif name == 'linear_velocity':
@@ -457,15 +460,12 @@
             position = interventions_dict['cartesian_position']
         if 'orientation' in interventions_dict:
             orientation = interventions_dict['orientation']
-        if 'mass' in interventions_dict:
-            self._mass = interventions_dict['mass']
-        if 'friction' in interventions_dict:
-            self._lateral_friction = interventions_dict['friction']
         if 'size' in interventions_dict:
-            self._size = interventions_dict['size']
-            self._set_volume()
-            self.reinit_object()
-        elif 'cartesian_position' in interventions_dict or 'orientation' in \
+            if not np.isclose(self._size, interventions_dict['size']):
+                self._size = interventions_dict['size']
+                self._set_volume()
+                self.reinit_object()
+        if 'cartesian_position' in interventions_dict or 'orientation' in \
                 interventions_dict:
             for i in range(0, len(self._pybullet_client_ids)):
                 position[-1] += WorldConstants.FLOOR_HEIGHT
@@ -474,14 +474,16 @@
                     position,
                     orientation,
                     physicsClientId=self._pybullet_client_ids[i])
-        elif 'mass' in interventions_dict:
-            for i in range(0, len(self._pybullet_client_ids)):
-                pybullet.changeDynamics(
-                    self._block_ids[i],
-                    -1,
-                    mass=self._mass,
-                    physicsClientId=self._pybullet_client_ids[i])
-        elif 'friction' in interventions_dict:
+        if 'mass' in interventions_dict:
+            if not np.isclose(self._mass, interventions_dict['mass']):
+                self._mass = interventions_dict['mass']
+                for i in range(0, len(self._pybullet_client_ids)):
+                    pybullet.changeDynamics(
+                        self._block_ids[i],
+                        -1,
+                        mass=self._mass,
+                        physicsClientId=self._pybullet_client_ids[i])
+        if 'friction' in interventions_dict:
             self._set_lateral_friction(self._lateral_friction)
 
         if 'color' in interventions_dict:

base_task.py, l. 502:
if self._stage.get_rigid_objects()[rigid_object].is_not_fixed:

'is_not_fixed' in the if statement is a function but doesn't get called. I guess is_not_fixed is supposed to be a property method.

'_latest_full_state' is used for computing the pd feedbacks but is updated at different places in the code which might cause trouble.

For a more consistent data flow, maybe it would be good to update the state only once also higher up the calling stack.

I suggest to update it once at the beginning of do_simulation (as a copy of the current state). See also the other issue about the endeffector position.

Also, the _latest_full_state should be stored in 'save_state' and 'restore_state' from 'task'. Otherwise, the PD controller is in a weird state at a restoration.

I'm trying out the Model Predictive Control script provided in the docs. I notice that when setting enable_visualization=True when instantiating a CausalWorld environment, the real-time visualization of the CEM is very high resolution, which is great. However, the recording of the CEM obtained via gym.wrappers.monitoring.video_recorder.VideoRecorder has much poorer quality.

Is there a way to record the videos such that their resolution is as good as the real-time visualization, or at least better than what we currently obtain from the VideoRecorder?

I suspect that one would have to modify the render() function of CausalWorld to achieve this, but I don't know what exactly needs to be done.

Hello, I have trained the baseline models for all of the tasks and the results are only good for the pushing and picking tasks, and not even that good for the picking one. As for pick and place and stacking, the trained baseline model fails consistently. Is this expected behavior?

I ran the reproduce_experiments.py script and then evaluated each trained model with the evaluation pipeline. I can post the videos and the evaluation script if needed.

Issue:

• _safety_torque_check is slow and possibly incorrect

Proposed solution:

• implement a fix speeds up the function (fix 1 or fix 2)
• optionally also change logic (fix 2) of function (would break backward compatability)

I profiled causalworlld env.step() function.
It seems roughly 66% is speed in the robot apply action, whereas most in self.step_simulation()

Of that, 6% of the step time (40 mu s per call) is spent on this safety check implemented in numpy. I did not understand the current implementation of the torque check from a logical point and also believe its rather slow

First of all, this function performs exactly the same function in roughly 5-10 times faster. (5 mu s per call)
Reason is that self._max_motor_torque is broadcasted and np.clip is slow. step() Simulation time would speed up by ~4%
Fix 1

    def _safety_torque_check(self, desired_torques):
        """

        :param desired_torques: (nd.array) the desired torque commands to be applied
                                            to the robot.

        :return: (nd.array) the modified torque commands after applying a safety check.
        """       
        applied_torques = np.maximum(
            np.minimum(
                np.asarray(desired_torques),
                +self._max_motor_torque,
            ),
            -self._max_motor_torque,
        )
        
        applied_torques -=  self._safety_kd * self._latest_full_state[
            'velocities']

        applied_torques = np.maximum(
            np.minimum(
                applied_torques,
                +self._max_motor_torque,
            ),
            -self._max_motor_torque,
        )
        
        return applied_torques.tolist()

In my opinion, this function in general does not implement a reasonable logic.
Currently, the safety factor _safety_kd is only applied where there are positive torques and positive velocities.
The torque limits are asymmetric, I don't understand how this makes sense on the robot. Maybe, the upper (or respectively; lower ) bounds should be tightened when moving with positive (or respectively; negative) velocity.
Fix 2

    def _safety_torque_check(self, desired_torques):
        applied_torques = np.asarray(desired_torques)
        
        # alternative 
        # torque_bound = self._max_motor_torque - np.abs(self._safety_kd * self._latest_full_state['velocities'])
        
        torque_bound_upper = self._max_motor_torque - self._safety_kd * np.maximum(self._latest_full_state['velocities'],0.)
        torque_bound_lower = - self._max_motor_torque - self._safety_kd * np.minimum(self._latest_full_state['velocities'],0.)
        
        # clip upper torque
        applied_torques = np.minimum(
            applied_torques, 
            torque_bound_upper, 
        )
        # clip lower torque
        applied_torques = np.maximum(
            applied_torques, 
            torque_bound_lower, 
        )
        
        return applied_torques.tolist()
    ```

-- reward
-- goal sampling
-- time index
-- initial joints

suggestion
rename 'structured' -> 'state'
rename 'cameras' -> 'pixel'

when I test the CausalWorld/tutorials/requesting_task/tutorial_one.py ,I met the issue like this:
pybullet build time: May 20 2022 19:43:01
startThreads creating 1 threads.
starting thread 0
started thread 0
argc=2
argv[0] = --unused
argv[1] = --start_demo_name=Physics Server
ExampleBrowserThreadFunc started
X11 functions dynamically loaded using dlopen/dlsym OK!
X11 functions dynamically loaded using dlopen/dlsym OK!
Creating context
Failed to create GL 3.3 context ... using old-style GLX context
Failed to create an OpenGL context
And I failed to solve it,it seems like X11 cannot match with opengl3,can you help me.