rail-berkeley / serl Goto Github PK
View Code? Open in Web Editor NEWSERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
Home Page: https://serl-robot.github.io/
License: MIT License
SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning
Home Page: https://serl-robot.github.io/
License: MIT License
There is an error when running python serl_robot_infra/robot_servers/franka_server.py on the real world franka robot. When we try to run, it gives this error:
[ERROR] [1709469050.695963]: bad callback: <bound method FrankaServer._set_currpos of <main.FrankaServer object at 0x7f0c4b5b5330>>
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/hyperarm/zichen/serl/serl_robot_infra/robot_servers/franka_server.py", line 161, in _set_currpos
self.vel = self.jacobian @ self.dq
AttributeError: 'FrankaServer' object has no attribute 'jacobian'
[ERROR] [1709469050.730088]: bad callback: <bound method FrankaServer._set_currpos of <main.FrankaServer object at 0x7f0c4b5b5330>>
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/hyperarm/zichen/serl/serl_robot_infra/robot_servers/franka_server.py", line 161, in _set_currpos
self.vel = self.jacobian @ self.dq
AttributeError: 'FrankaServer' object has no attribute 'jacobian'
[ERROR] [1709469050.764108]: bad callback: <bound method FrankaServer._set_currpos of <main.FrankaServer object at 0x7f0c4b5b5330>>
Traceback (most recent call last):
File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback
cb(msg)
File "/home/hyperarm/zichen/serl/serl_robot_infra/robot_servers/franka_server.py", line 161, in _set_currpos
self.vel = self.jacobian @ self.dq
AttributeError: 'FrankaServer' object has no attribute 'jacobian'
Any help on how to solve this issue, or fixes to the codebase is appreciated!
Could you please update the requirements.txt (edit: in serl_launcher)?
gym is listed, but serl uses gymansium. Are there any other updates that need to be done to the file?
Hi,
Thanks for open-sourcing your software.
I had a doubt with regards to camera equipment to run the experiments - is there a requirement to run with realsense cameras only. Am I right to assume that as long as the camera is exposed with ROS drivers, and the image dimensions are correct, the agent should be able to train with images coming from any camera.
Wanted to check if there is a hard requirement for realsense.
Thanks for your support
Has anyone encountered this when using SERL? I've submitted an issue with Intel to see if it's a bug on their part because when using the old FW everything is ok and I can set the auto exposure.
Hi!
I meet a problems on loading checkpoints.
ckpt = checkpoints.restore_checkpoint(
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/flax/training/checkpoints.py", line 1129, in restore_checkpoint
restored = orbax_checkpointer.restore(
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/checkpointer.py", line 165, in restore
restored = self._restore_with_args(directory, *args, **kwargs)
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/checkpointer.py", line 103, in _restore_with_args
restored = self._handler.restore(directory, args=ckpt_args)
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/pytree_checkpoint_handler.py", line 1019, in restore
byte_limiter = get_byte_limiter(self._concurrent_gb)
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/pytree_checkpoint_handler.py", line 169, in get_byte_limiter
return asyncio.run(_create_byte_limiter())
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
return future.result()
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/orbax/checkpoint/pytree_checkpoint_handler.py", line 167, in _create_byte_limiter
return LimitInFlightBytes(concurrent_bytes) # pylint: disable=protected-access
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/site-packages/jax/experimental/array_serialization/serialization.py", line 144, in __init__
self._cv = asyncio.Condition(lock=asyncio.Lock())
File "/home/wk/anaconda3/envs/copy2/lib/python3.10/asyncio/locks.py", line 234, in __init__
raise ValueError("loop argument must agree with lock")
ValueError: loop argument must agree with lock
I have found similar issue in the Orbax project,It seems like conflicts between different package versions , but the solution to this issue is to switch the Python version to 3.11, not the project's 3.10. Could you provide the specific version numbers of the packages that need to be installed in this project?
I sincerely look forward to your advice and guidance.
@charlesxu0124 Hello, dear author. I am a postgraduate and I am a novice in robot and machine learning fields. I have read your paper and code,but I haven't understand them all.Now, I can run the examples in sim environment and save the trajectories.But In the examples, there is not any method about achieve the data Extraction and data visualization offered. Therefore, could you give me some advice or offer some method?
Thanks for open-sourcing your software.
I've worked with the repo for a while now and implemented a controller and environment for the robotiq UR5 arm. Let me know if the contribution to the main repo would be welcome.
During the implementation of my own DRQ agent I've collected some questions about the code, which were not mentioned in the SERL paper:
spatial_learned_embeddings
is used instead of the standard average
. I've seen it in Googles Rainbow DQN, but it was not used there. Did you notice performance increases with this pooling method or what was the reason behind the decision?backup_entropy
was implemented without the discount factor (compared to the original jaxrl_m
implementation). Since it was not used in SERL (question 4) it does not matter, but I'm still curious about it.backup_entropy
was set to False
for all the experiments. What led to this decision?(supervisor: @vhartman)
Thank you in advance for your time and assistance. I look forward to your response.
Hi,
Do you plan to support Mujoco XLA for simulation training in the near future?
I recently got back to working on serl after a 2 week break and now I'm getting the following message when trying to run bc training bash run_bc.sh
. The script worked previously.
Has anyone encountered this issue?
File "/home/user/serl/serl_launcher/serl_launcher/utils/launcher.py", line 10, in <module>
from agentlace.data.tfds import populate_datastore
File "/home/user/miniconda3/envs/serl/lib/python3.10/site-packages/agentlace/data/tfds.py", line 8, in <module>
from agentlace.data.jaxrl_data_store import ReplayBufferDataStore
File "/home/user/miniconda3/envs/serl/lib/python3.10/site-packages/agentlace/data/jaxrl_data_store.py", line 17, in <module>
from jaxrl_m.data.replay_buffer import ReplayBuffer
ModuleNotFoundError: No module named 'jaxrl_m.data'
I wanted to ask if you had issues with running bc in the bin example, specifically with operating the gripper; I'm running an experiment similar to yours and the model doesn't close/open the gripper.
What happens is that when the model runs, the arm moves to the object that we want to pick up and stays there. If I intervene with the spacemouse by clicking the close gripper button, the robot closes the gripper and continues on the trajectory. The same happens when the gripper has to release an object: the robot stays in place until I press the gripper open button and then it continues with the trajectory.
I suspected the issue may be that there weren't enough data points for gripper close, so during data collection I pressed the buttons longer, but it still didn't work.
Have you encountered similar problems when using bc?
I'm using a Franka Panda with the panda hand
It would be good to have the ability to pause, stop, and resume training. As part of the stop, it would be good to have the option to save training state (including replay buffers, optimizer state, etc.). This might also involve changes in agentlace
by @youliangtan .
Edit: added separate options for pausing v/s stopping training
Traceback (most recent call last):
File "/ros_ws/src/serl/examples/pick_place_real/train_reward_classifier.py", line 160, in <module>
app.run(main)
File "/usr/local/lib/python3.10/dist-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/usr/local/lib/python3.10/dist-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/ros_ws/src/serl/examples/pick_place_real/train_reward_classifier.py", line 97, in main
classifier = create_classifier(key, sample["next_observations"], image_keys)
File "/ros_ws/src/serl/serl_launcher/serl_launcher/networks/reward_classifier.py", line 74, in create_classifier
new_params = classifier.params.unfreeze()
AttributeError: 'dict' object has no attribute 'unfreeze'
I0515 10:52:56.182127 134645912520512 core.py:560] signal_shutdown [atexit]
when saving replay buffer with
` if FLAGS.rlds_logger_path is not None:
rlds_logger = RLDSLogger(
observation_space=env.observation_space,
action_space=env.action_space,
dataset_name="serl_rlds_dataset",
directory=FLAGS.rlds_logger_path,
max_episodes_per_file=5,
)
else:
rlds_logger = None
def create_replay_buffer_and_wandb_logger():
replay_buffer = MemoryEfficientReplayBufferDataStore(
env.observation_space,
env.action_space,
capacity=FLAGS.replay_buffer_capacity,
image_keys=image_keys,
rlds_logger=rlds_logger,
)`
i find that datastore update always fail with a message: Failed to send message: Resource temporarily unavailable.Could you help me with this? Thank you.
Is there a simple way to add timesteps as another key in the RLDS record directly in the data collection pipeline
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.