rail-berkeley / serl Goto Github PK

There is an error when running python serl_robot_infra/robot_servers/franka_server.py on the real world franka robot. When we try to run, it gives this error:

[ERROR] [1709469050.695963]: bad callback: <bound method FrankaServer._set_currpos of <main.FrankaServer object at 0x7f0c4b5b5330>>
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/hyperarm/zichen/serl/serl_robot_infra/robot_servers/franka_server.py", line 161, in _set_currpos
self.vel = self.jacobian @ self.dq
AttributeError: 'FrankaServer' object has no attribute 'jacobian'

[ERROR] [1709469050.730088]: bad callback: <bound method FrankaServer._set_currpos of <main.FrankaServer object at 0x7f0c4b5b5330>>
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/hyperarm/zichen/serl/serl_robot_infra/robot_servers/franka_server.py", line 161, in _set_currpos
self.vel = self.jacobian @ self.dq
AttributeError: 'FrankaServer' object has no attribute 'jacobian'

[ERROR] [1709469050.764108]: bad callback: <bound method FrankaServer._set_currpos of <main.FrankaServer object at 0x7f0c4b5b5330>>
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/hyperarm/zichen/serl/serl_robot_infra/robot_servers/franka_server.py", line 161, in _set_currpos
self.vel = self.jacobian @ self.dq
AttributeError: 'FrankaServer' object has no attribute 'jacobian'

Any help on how to solve this issue, or fixes to the codebase is appreciated!

Could you please update the requirements.txt (edit: in serl_launcher)?
gym is listed, but serl uses gymansium. Are there any other updates that need to be done to the file?

Hi,

Thanks for open-sourcing your software.

I had a doubt with regards to camera equipment to run the experiments - is there a requirement to run with realsense cameras only. Am I right to assume that as long as the camera is exposed with ROS drivers, and the image dimensions are correct, the agent should be able to train with images coming from any camera.

Wanted to check if there is a hard requirement for realsense.

Thanks for your support

Has anyone encountered this when using SERL? I've submitted an issue with Intel to see if it's a bug on their part because when using the old FW everything is ok and I can set the auto exposure.

Hi!
I meet a problems on loading checkpoints.

    ckpt = checkpoints.restore_checkpoint(
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/flax/training/checkpoints.py", line 1129, in restore_checkpoint
    restored = orbax_checkpointer.restore(
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/checkpointer.py", line 165, in restore
    restored = self._restore_with_args(directory, *args, **kwargs)
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/checkpointer.py", line 103, in _restore_with_args
    restored = self._handler.restore(directory, args=ckpt_args)
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/pytree_checkpoint_handler.py", line 1019, in restore
    byte_limiter = get_byte_limiter(self._concurrent_gb)
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/pytree_checkpoint_handler.py", line 169, in get_byte_limiter
    return asyncio.run(_create_byte_limiter())
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/pytree_checkpoint_handler.py", line 167, in _create_byte_limiter
    return LimitInFlightBytes(concurrent_bytes)  # pylint: disable=protected-access
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/jax/experimental/array_serialization/serialization.py", line 144, in __init__
    self._cv = asyncio.Condition(lock=asyncio.Lock())
  File "/home/wk/anaconda3/envs/copy2/lib/python3.10/asyncio/locks.py", line 234, in __init__
    raise ValueError("loop argument must agree with lock")
ValueError: loop argument must agree with lock

I have found similar issue in the Orbax project,It seems like conflicts between different package versions , but the solution to this issue is to switch the Python version to 3.11, not the project's 3.10. Could you provide the specific version numbers of the packages that need to be installed in this project?

I sincerely look forward to your advice and guidance.

@charlesxu0124 Hello, dear author. I am a postgraduate and I am a novice in robot and machine learning fields. I have read your paper and code,but I haven't understand them all.Now, I can run the examples in sim environment and save the trajectories.But In the examples, there is not any method about achieve the data Extraction and data visualization offered. Therefore, could you give me some advice or offer some method?

Thanks for open-sourcing your software.

I've worked with the repo for a while now and implemented a controller and environment for the robotiq UR5 arm. Let me know if the contribution to the main repo would be welcome.

During the implementation of my own DRQ agent I've collected some questions about the code, which were not mentioned in the SERL paper:

1. For the DRQ examples, the data sampled is 50/50, half from demo and half from online experience. Is there a reason for not using a shared replay buffer?
2. When using the pretrained ResNet model, the pooling method spatial_learned_embeddings is used instead of the standard average. I've seen it in Googles Rainbow DQN, but it was not used there. Did you notice performance increases with this pooling method or what was the reason behind the decision?
3. In the soft actor critic implementation, the step labeled backup_entropy was implemented without the discount factor (compared to the original jaxrl_m implementation). Since it was not used in SERL (question 4) it does not matter, but I'm still curious about it.
4. Following the last question, the parameter backup_entropy was set to False for all the experiments. What led to this decision?

(supervisor: @vhartman)

Thank you in advance for your time and assistance. I look forward to your response.

Hi,
Do you plan to support Mujoco XLA for simulation training in the near future?

I recently got back to working on serl after a 2 week break and now I'm getting the following message when trying to run bc training bash run_bc.sh. The script worked previously.
Has anyone encountered this issue?

  File "/home/user/serl/serl_launcher/serl_launcher/utils/launcher.py", line 10, in <module>
    from agentlace.data.tfds import populate_datastore
  File "/home/user/miniconda3/envs/serl/lib/python3.10/site-packages/agentlace/data/tfds.py", line 8, in <module>
    from agentlace.data.jaxrl_data_store import ReplayBufferDataStore
  File "/home/user/miniconda3/envs/serl/lib/python3.10/site-packages/agentlace/data/jaxrl_data_store.py", line 17, in <module>
    from jaxrl_m.data.replay_buffer import ReplayBuffer
ModuleNotFoundError: No module named 'jaxrl_m.data'

I wanted to ask if you had issues with running bc in the bin example, specifically with operating the gripper; I'm running an experiment similar to yours and the model doesn't close/open the gripper.
What happens is that when the model runs, the arm moves to the object that we want to pick up and stays there. If I intervene with the spacemouse by clicking the close gripper button, the robot closes the gripper and continues on the trajectory. The same happens when the gripper has to release an object: the robot stays in place until I press the gripper open button and then it continues with the trajectory.
I suspected the issue may be that there weren't enough data points for gripper close, so during data collection I pressed the buttons longer, but it still didn't work.
Have you encountered similar problems when using bc?

I'm using a Franka Panda with the panda hand

It would be good to have the ability to pause, stop, and resume training. As part of the stop, it would be good to have the option to save training state (including replay buffers, optimizer state, etc.). This might also involve changes in agentlace by @youliangtan .

Edit: added separate options for pausing v/s stopping training

when saving replay buffer with
` if FLAGS.rlds_logger_path is not None:
rlds_logger = RLDSLogger(
observation_space=env.observation_space,
action_space=env.action_space,
dataset_name="serl_rlds_dataset",
directory=FLAGS.rlds_logger_path,
max_episodes_per_file=5,
)
else:
rlds_logger = None

def create_replay_buffer_and_wandb_logger():
    replay_buffer = MemoryEfficientReplayBufferDataStore(
        env.observation_space,
        env.action_space,
        capacity=FLAGS.replay_buffer_capacity,
        image_keys=image_keys,
        rlds_logger=rlds_logger,
    )`

i find that datastore update always fail with a message: Failed to send message: Resource temporarily unavailable.Could you help me with this? Thank you.

Hi,

I don't have demo data for real-world experiments. So, I did not load any demo file. What changes should I make so the algorithm will not sample data from demos? Will commenting the lines that incorporate prior data [1, 2] suffice?

Is there a simple way to add timesteps as another key in the RLDS record directly in the data collection pipeline