improbable-ai / walk-these-ways Goto Github PK

unitree@unitree-desktop:~/go1_gym/go1_gym_deploy/autostart$ cd /go1_gym/go1_gym_deploy/docker
unitree@unitree-desktop:/go1_gym/go1_gym_deploy/docker$ sudo make autostart
make: Warning: File ‘Makefile’ has modification time 51435344 s in the future
docker stop foxy_controller || true
Error response from daemon: No such container: foxy_controller
docker rm foxy_controller || true
Error: No such container: foxy_controller
docker run -d
--env=“DISPLAY”
--env=“QT_X11_NO_MITSHM=1"
--volume=“/tmp/.X11-unix:/tmp/.X11-unix:rw”
--env=“XAUTHORITY=”
--volume=“:”
--volume=“/home/unitree/go1_gym:/home/isaac/go1_gym”
--privileged
--runtime=nvidia
--net=host
--workdir=“/home/isaac/go1_gym”
--name=“foxy_controller”
jetson-model-deployment tail -f /dev/null
0578640c8df46a393abd625e7900b766944f04fabb9f825bafa90dd77c14a2c9
docker start foxy_controller
foxy_controller
docker exec foxy_controller bash -c ‘cd /home/isaac/go1_gym/ && python3 setup.py install && cd go1_gym_deploy/scripts && ls && python3 deploy_policy.py’
running install
running bdist_egg
running egg_info
writing go1_gym.egg-info/PKG-INFO
writing dependency_links to go1_gym.egg-info/dependency_links.txt
writing top-level names to go1_gym.egg-info/top_level.txt
reading manifest file ‘go1_gym.egg-info/SOURCES.txt’
writing manifest file ‘go1_gym.egg-info/SOURCES.txt’
installing library code to build/bdist.linux-aarch64/egg
running install_lib
running build_py
creating build/bdist.linux-aarch64/egg
creating build/bdist.linux-aarch64/egg/go1_gym_deploy
creating build/bdist.linux-aarch64/egg/go1_gym_deploy/utils
copying build/lib/go1_gym_deploy/utils/network_config_unitree.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/utils
copying build/lib/go1_gym_deploy/utils/deployment_runner.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/utils
copying build/lib/go1_gym_deploy/utils/init.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/utils
copying build/lib/go1_gym_deploy/utils/logger.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/utils
copying build/lib/go1_gym_deploy/utils/command_profile.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/utils
copying build/lib/go1_gym_deploy/utils/cheetah_state_estimator.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/utils
creating build/bdist.linux-aarch64/egg/go1_gym_deploy/envs
copying build/lib/go1_gym_deploy/envs/init.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/envs
copying build/lib/go1_gym_deploy/envs/lcm_agent.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/envs
copying build/lib/go1_gym_deploy/envs/history_wrapper.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/envs
copying build/lib/go1_gym_deploy/init.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy
creating build/bdist.linux-aarch64/egg/go1_gym_deploy/tests
copying build/lib/go1_gym_deploy/tests/init.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/tests
copying build/lib/go1_gym_deploy/tests/check_camera_msgs.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/tests
copying build/lib/go1_gym_deploy/setup.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy
creating build/bdist.linux-aarch64/egg/go1_gym_deploy/scripts
copying build/lib/go1_gym_deploy/scripts/deploy_policy.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/scripts
copying build/lib/go1_gym_deploy/scripts/init.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/scripts
creating build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types
copying build/lib/go1_gym_deploy/lcm_types/camera_message_rect_wide.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types
copying build/lib/go1_gym_deploy/lcm_types/rc_command_lcmt.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types
copying build/lib/go1_gym_deploy/lcm_types/camera_message_lcmt.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types
copying build/lib/go1_gym_deploy/lcm_types/leg_control_data_lcmt.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types
copying build/lib/go1_gym_deploy/lcm_types/init.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types
copying build/lib/go1_gym_deploy/lcm_types/pd_tau_targets_lcmt.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types
copying build/lib/go1_gym_deploy/lcm_types/state_estimator_lcmt.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types
creating build/bdist.linux-aarch64/egg/go1_gym_deploy/unitree_legged_sdk_bin
copying build/lib/go1_gym_deploy/unitree_legged_sdk_bin/init.py -> build/bdist.linux-aarch64/egg/go1_gym_deploy/unitree_legged_sdk_bin
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/utils/network_config_unitree.py to network_config_unitree.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/utils/deployment_runner.py to deployment_runner.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/utils/init.py to init.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/utils/logger.py to logger.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/utils/command_profile.py to command_profile.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/utils/cheetah_state_estimator.py to cheetah_state_estimator.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/envs/init.py to init.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/envs/lcm_agent.py to lcm_agent.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/envs/history_wrapper.py to history_wrapper.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/init.py to init.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/tests/init.py to init.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/tests/check_camera_msgs.py to check_camera_msgs.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/setup.py to setup.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/scripts/deploy_policy.py to deploy_policy.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/scripts/init.py to init.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types/camera_message_rect_wide.py to camera_message_rect_wide.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types/rc_command_lcmt.py to rc_command_lcmt.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types/camera_message_lcmt.py to camera_message_lcmt.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types/leg_control_data_lcmt.py to leg_control_data_lcmt.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types/init.py to init.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types/pd_tau_targets_lcmt.py to pd_tau_targets_lcmt.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/lcm_types/state_estimator_lcmt.py to state_estimator_lcmt.cpython-36.pyc
byte-compiling build/bdist.linux-aarch64/egg/go1_gym_deploy/unitree_legged_sdk_bin/init.py to init.cpython-36.pyc
creating build/bdist.linux-aarch64/egg/EGG-INFO
copying go1_gym.egg-info/PKG-INFO -> build/bdist.linux-aarch64/egg/EGG-INFO
copying go1_gym.egg-info/SOURCES.txt -> build/bdist.linux-aarch64/egg/EGG-INFO
copying go1_gym.egg-info/dependency_links.txt -> build/bdist.linux-aarch64/egg/EGG-INFO
copying go1_gym.egg-info/top_level.txt -> build/bdist.linux-aarch64/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
creating ‘dist/go1_gym-1.0.0-py3.6.egg’ and adding ‘build/bdist.linux-aarch64/egg’ to it
removing ‘build/bdist.linux-aarch64/egg’ (and everything under it)
Processing go1_gym-1.0.0-py3.6.egg
Copying go1_gym-1.0.0-py3.6.egg to /usr/local/lib/python3.6/dist-packages
Adding go1-gym 1.0.0 to easy-install.pth file
Installed /usr/local/lib/python3.6/dist-packages/go1_gym-1.0.0-py3.6.egg
Processing dependencies for go1-gym==1.0.0
Finished processing dependencies for go1-gym==1.0.0
init.py
deploy_policy.py
deployment_image.tar
send_to_unitree.sh
Traceback (most recent call last):
File “deploy_policy.py”, line 6, in
from go1_gym_deploy.utils.deployment_runner import DeploymentRunner
ImportError: cannot import name ‘DeploymentRunner’
Makefile:23: recipe for target ‘autostart’ failed
make: *** [autostart] Error 1

hi guys, i ran into this problem when i ran: python train.py.
i already set the num_envs to 1 but it doesn't work, any idea how to handle this?

PxgCudaDeviceMemoryAllocator fail to allocate memory 339738624 bytes!! Result = 2
PxgCudaDeviceMemoryAllocator fail to allocate memory 339738624 bytes!! Result = 2
PxgCudaDeviceMemoryAllocator fail to allocate memory 339738624 bytes!! Result = 2
PxgCudaDeviceMemoryAllocator fail to allocate memory 339738624 bytes!! Result = 2
/buildAgent/work/99bede84aa0a52c2/source/gpunarrowphase/src/PxgNarrowphaseCore.cpp (2155) : internal error : memcpy failed fail! 2 700

/buildAgent/work/99bede84aa0a52c2/source/gpunarrowphase/src/PxgNarrowphaseCore.cpp (2165) : internal error : GPU cudaMainGjkEpa or prepareLostFoundPairs kernel fail!
700

/buildAgent/work/99bede84aa0a52c2/source/gpucommon/include/PxgCudaUtils.h (54) : internal error : SynchronizeStreams cuEventRecord failed

/buildAgent/work/99bede84aa0a52c2/source/gpucommon/include/PxgCudaUtils.h (60) : internal error : SynchronizeStreams cuStreamWaitEvent failed

/buildAgent/work/99bede84aa0a52c2/source/gpunarrowphase/src/PxgNarrowphaseCore.cpp (2408) : internal error : memcpy failed fail!
700

/buildAgent/work/99bede84aa0a52c2/source/gpucommon/src/PxgCudaMemoryAllocator.cpp (59) : warning : Failed to allocate pinned memory.

Thank you for this awesome work!

I want to resume training, but now I have some problems.
According to my understanding, I modified the following code in go1_gym_learn/ppo_cse/__init__.py and tried to resume training.

class RunnerArgs(PrefixProto, cli=False):
    
    ...

    # load and resume
    resume = True
    load_run = -1  # -1 = last run
    checkpoint = -1  # -1 = last saved model

    label = "gait-conditioned-agility/2023-05-18/train"
    dirs = glob.glob(f"../runs/{label}/*")
    logdir = sorted(dirs)[0]

    resume_path = logdir[3:-1]  # updated from load_run and chkpt
    resume_curriculum = True

loader = ML_Logger(root="http://127.0.0.1:8081",
                               prefix=RunnerArgs.resume_path)

After starting the traning I can see the corresponding output in ml_dash.server

[2023-05-18 21:25:56 +0800] - (sanic.access)[INFO][127.0.0.1:53682]: GET http://127.0.0.1:8081/files/runs/gait-conditioned-agility/base-policy/train/053912.36310/checkpoints/ac_weights_last.pt  404 651

and

Traceback (most recent call last):
  File "train.py", line 263, in <module>
    train_go1(headless=True)
  File "train.py", line 222, in train_go1
    runner = Runner(env, device=f"cuda:{gpu_id}")
  File "/home/mao/Desktop/FunPro/walk-these-ways/go1_gym_learn/ppo_cse/__init__.py", line 102, in __init__
    weights = loader.load_torch("checkpoints/ac_weights_last.pt")
  File "/home/mao/.conda/envs/legged/lib/python3.8/site-packages/ml_logger/ml_logger.py", line 2098, in load_torch
    return torch.load(fn_or_buff, map_location=map_location, **kwargs)
  File "/home/mao/.conda/envs/legged/lib/python3.8/site-packages/torch/serialization.py", line 608, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/home/mao/.conda/envs/legged/lib/python3.8/site-packages/torch/serialization.py", line 777, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '<'.

Could you give me some advice to solve this problem?

Hi Gabriel,

I'm trying to install go1_gym and I encounter an issue with conflicting dependencies for gym, ml-dash, ml-logger. May I know if you you used a different set of requirements?

#0 9.230 The conflict is caused by:
#0 9.230     gym 0.14.0 depends on cloudpickle~=1.2.0
#0 9.230     ml-dash 0.3.20 depends on cloudpickle
#0 9.230     ml-logger 0.8.117 depends on cloudpickle==1.3.0

I get the following error when I run train.py script.

  File "scripts/train.py", line 256, in <module>
    train_go1(headless=False)
  File "scripts/train.py", line 216, in train_go1
    runner.learn(num_learning_iterations=100000, init_at_random_ep_len=True, eval_freq=100)
  File "/data/william/walk-these-ways/go1_gym_learn/ppo_cse/__init__.py", line 204, in learn
    mean_value_loss, mean_surrogate_loss, mean_adaptation_module_loss, mean_decoder_loss, mean_decoder_loss_student, mean_adaptation_module_test_loss, mean_decoder_test_loss, mean_decoder_test_loss_student = self.alg.update()
  File "/data/william/walk-these-ways/go1_gym_learn/ppo_cse/ppo.py", line 110, in update
    self.actor_critic.act(obs_history_batch, masks=masks_batch)
  File "/data/william/walk-these-ways/go1_gym_learn/ppo_cse/actor_critic.py", line 119, in act
    self.update_distribution(observation_history)
  File "/data/william/walk-these-ways/go1_gym_learn/ppo_cse/actor_critic.py", line 116, in update_distribution
    self.distribution = Normal(mean, mean * 0. + self.std)
  File "/home/william/anaconda3/envs/rob/lib/python3.8/site-packages/torch/distributions/normal.py", line 50, in __init__
    super(Normal, self).__init__(batch_shape, validate_args=validate_args)
  File "/home/william/anaconda3/envs/rob/lib/python3.8/site-packages/torch/distributions/distribution.py", line 55, in __init__
    raise ValueError(
ValueError: Expected parameter loc (Tensor of shape (24000, 12)) of distribution Normal(loc: torch.Size([24000, 12]), scale: torch.Size([24000, 12])) to satisfy the constraint Real(), but found invalid values:
tensor([[nan, nan, nan,  ..., nan, nan, nan],
        [nan, nan, nan,  ..., nan, nan, nan],
        [nan, nan, nan,  ..., nan, nan, nan],
        ...,
        [nan, nan, nan,  ..., nan, nan, nan],
        [nan, nan, nan,  ..., nan, nan, nan],
        [nan, nan, nan,  ..., nan, nan, nan]], device='cuda:0',
       grad_fn=<AddmmBackward0>)

@gmargo11 thanks for the amazing repo, it was very intuitive to follow. Looking forward to updates on Deploying a Custom Model. Here is a video of Go1 running around with the policy.

Here are some minor fixes that could make deployment easier:

Installing the Deployment Utility

1. The cd go1_gym_deploy/scripts && ./send_to_unitree.sh downloads the deployment_image.tar to the scripts folder. While in the next step cd ~/go1_gym/go1_gym_deploy/installer && ./install_deployment_code.sh the script searches for the deployment_image.tar in /go1_gym/go1_gym_deploy/installer.

Running the Controller

1. There seems to be no safe way to exit the policy, the policy continued to run even after the script was terminated.

Some information that might be helpful for others

1. If you run out of disk space during cd ~/go1_gym/go1_gym_deploy/installer && ./install_deployment_code.sh consider changing the script to use 192.168.123.13 instead (at least in my Go1 Edu with 3 Jetson nano, I only had the required disk space to copy the tar and extract the image in only 192.168.123.13). Alternatively, consider deploying on an external PC.

2. When deploying on an external PC/NUC, if you get the following error

➜ sudo ./start_unitree_sdk.sh
./lcm_position: 1: Syntax error: word unexpected (expecting ")")

It is likely because the ./lcm_position has been compiled for ARM aarch64 (to run on the jetson), please recompile it for your architecture(external PC/ NUC) using https://github.com/Improbable-AI/unitree_legged_sdk.

Hi there and thanks for providing the repo in addition to a Pretrained model.

I am wondering where the Pretrained model comes from? What iteration, more or less? What is the status of the curriculum?

When I inspect the status of the provided curriculum:

runs/gait-conditioned-agility/pretrain-v0/train/025417.456545/curriculum/distribution.pkl

I see that distribution["iteration"] returns 0, and looking at the distribution["distribution"], it seems the distribution is limited to values that are the initial values. [-1, 1] for the linear/angular velocities.

I am curious because I would like to do some fine-tuning on your set of weights that seem to be working well enough. However, if I want to pick up training with those weights, I would assume I also want to pick up with the curriculum no?

the docker .tar file is in the scripts directory and requires root user to install. So the current instructions fail. However if you change it to this it works:

chmod +x installer/install_deployment_code.sh
cd ~/go1_gym/go1_gym_deploy/scripts
sudo ../installer/install_deployment_code.sh

I have created a NN with a getup behavior but when it falls it just shuts down. How do I keep it from shutting off when it falls over?

Thank you for your work. I have been trying to deploy your policy on a unitree A1 robot but it doesnt work. I have followed all your instruction as specified on your page. I editted the Lcm_position.cpp to suit UDP requirement of A1 and i have compiled and updated the lcm_position file. I have verified the deployemt.tar file to ensure that the check md5sum deployment_image.tar has
ca70e4a10063fa39f026d32b1fe5d88b. While troubleshooting this problem, I came across a similar issue on your webpage https://github.com/Improbable-AI/walk-these-ways/issues/1] and it still doesnt provide any solution.

Each time try to run this code, i keep getting stuck at the image below

Please help with a solution to this!!

As shown in the screenshot, the provided urdf commented all hip collison except RR-hip

Hi, I ran your script play.py in the fold "scripts". I found that there is a slow but fatal GPU memory leak problem. Could you provide any suggestions about it? Thanks!

Hi,

I would like to build the docker container on my own for a slightly different JetPack version (our Go1 has L4T R32.4.4). Did you install ROS (melodic) on it (as those commands are commented out and there's no install_scripts/install_ros.sh in the repo)?

I am getting some incompatibility issues when I try mixing pytorch with ros melodic, so I was wondering if/how you managed to get it to work?

Thank you!

Hi! bro, i want do try deploy this work in my NUC but nano in go1, so could you please compile lcm_position.cpp in amd64 or send to me state_estimator_lcmt.hpp....(I don't want this code, just don't want to trouble you), you can send compiled lcm_position to [email protected]
thank you!

In the training script it seems the joint data were collected at 50Hz, while compute_torque() in isaac_sim actually calculates torque at 200Hz (dt=0.005, though the target stays same for 4 iteration but joint state error is changing). Will this collect frequency influence the performance of actuator net?

When I tried to start the controller with the command: ./start_unitree_sdk.sh, the problem happened. It seemed that the docker haven't installed the lcm correctly?

I want to know how to set the config file that can let the go1 performing Gait-free type.
maybe turn Cfg.commands.gaitwise_curricula = False is no enough.

What's more, I noticed that go1_env_learn/ppo might use RMA which had not been use in this project. Why?ter/go1_gym_learn/ppo/ppo.py)` might use RMA which had not been use in this project. Why?

I've trained my own NN and want to have it automatically run when the Go1 is turned on. How would you go about doing this?

Ideally it would be great to be able to toggle between both control systems (mine, and original).

Hi @gmargo11,

Thanks for including the actuator network training and evaluation scripts! I was wondering how you go about generating the training data for the network. From the scripts it seems that you only need joint position and velocity measurements, torques measurements (tau_est?), desired joint positions and torques - is the last one the computed torques based on the PD law?

Do you run random commands or directly run a pretrained policy on the real robot and save the data? I was thinking of running my policy with the robot hung up and then recording the expected vs real torques and joint angles / velocities, but I'd be curious to know how you did it, too!

Thanks!

Hi.

I have some problem.

when we use simulation in legged_gym(original learning in minuites code), on downstairs or upstairs, robots initial position is Almost stuck. i mean like below

But in this walk-these-ways code express like below.
I mean robots start in the high position ( same height as plane. so it looks like in the air )

Which part should i change?
I really want to know it thank you.

Dear Gabriel,

I just wanted to express my gratitude for your wonderful library. Kudos to your great work!

I encountered a couple of issues while running python scripts/test.py after following the instructions for installation. Specifically, I received the following errors:

1. ImportError: cannot import name 'PrefixProto' from 'params_proto'.
2. AttributeError: module 'numpy' has no attribute 'float'.

I was able to resolve these issues by upgrading params_proto and downgrading numpy with the following commands:

1. pip install --upgrade params_proto.
2. pip install "numpy<1.24".

It might be helpful to include the specific versions of these packages, as well as the Python version, in the setup.py file to prevent similar issues for other users.

Sincerely,
Weizhe

Hello,
When I try to install docker image for the deployment code, it gives me the following error
[Step 1] Extracting docker image...
open /var/lib/docker/tmp/docker-import-137802504/repositories: no such file or directory
How do I resolve it?

Hello,

Thank you very much for sharing this codebase. I'm currently trying to reproduce the results from the paper.

I was wondering how to generate the figure in Table 3 that shows the energy consumption for each behavior. Additionally, since I've run train.py, I'm not sure which line in outputs.log displays the total reward for each iteration（I guess it's "train/episode/rew total/mean" but not sure）. Could you please provide some information on these two issues? Thank you.

Best regards,
Elgce

Hello,

I have been studying your project and reading the associated paper, where it mentioned the utilization of a state estimator network for velocity estimation. However, while examining the code, I was unable to locate the implementation of this network.

Could you please provide some guidance on where I can find the code related to the state estimator network? Additionally, if there are any specific functions or files I should look into, please let me know.

Thank you for your assistance!

Hi,

First I would like to express my gratitude again on the great work of open sourcing the project and providing the deployment procedure!

I just started trying the deployment of the checkpoint provided in this repo. In the paper Kp = 20 is used as the stiffness for the joint controller and this can be confirmed from both the training code and deployment code.

Based on my experience with unitree Go1 I think Kp = 20 might be too soft. In the deployment script deployment_runner.py the calibrate() function will create an interpolated action sequence from the current qpos to the target qpos, which is

[0.1, 0.8, -1.5, -0.1, 0.8, -1.5, 0.1, 1, -1.5, -0.1, 1, -1.5]

I tried the same thing on our Unitree Go1 and it ends up "kneeling" on the ground with qpos below

[-0.07242385 0.66453123 -1.6734996 0.06848777 0.670284 -1.6667578, -0.13219169 0.8420787 -2.137513 0.12692341 0.8130728 -2.0752623 ]

Note that the rear leg joints have very big terminal errors.

And pictures:

Would like to confirm with the author before proceeding - is such behavior expected?

Thanks!

Hi! I am trying to figure out how you made this and a little bit puzzled by the need to run locomotion on one of the nanos. Why is that?

When I was trying to install the docker to unitree-desktop by the command: sudo ../installer/install_deployment_code.sh, the error as the title shown has happened. Have some convenient ways to solve this problem? Thanks a lot!

Hello,
Thank you for sharing this great work.

I have some question about foot clearance.

def _reward_feet_clearance_cmd_linear(self):
    phases = 1 - torch.abs(1.0 - torch.clip((self.env.foot_indices * 2.0) - 1.0, 0.0, 1.0) * 2.0)
    foot_height = (self.env.foot_positions[:, :, 2]).view(self.env.num_envs, -1) # - reference_heights
    target_height = self.env.commands[:, 9].unsqueeze(1) * phases + 0.02 # offset for foot radius 2cm
    rew_foot_clearance = torch.square(target_height - foot_height) * (1 - self.env.desired_contact_states)
    return torch.sum(rew_foot_clearance, dim=1)

This code is extracted from corl_reward.py.
If I understand correctly, this function runs according to the swing phase of the legged system.
However, the 'target_height' variable for this function appears to increase linearly within the specified period.

I think at the end of this cycle we need to zero the target_height again to prepare for the next stance phase, is there a reason why it's in the same form as the code above?

Hey,

Thanks for sharing this code, we find it usefull for actually deployment on the Go1.
However, we are struggling to lower the stepping frequency down to lower than 2Hz. Could you elaborate where this limit is programmed?

Things we've tried so far:

• Changing default trotting gait parameters in cheetah_state_estimator.py: self.cmd_freq, cmd_freq, min_freq, max_freq in joystick commands
• Training a new policy with train.py and Cfg.commands.limit_gait_frequency and Cfg.commands.gait_frequency_cmd_range adjusted.

Hi @gmargo11,
What kind of behavior should we observe when running test.py ?
The robots don't walk in IsaacGym when running the script. They seem to fall on their hind legs.

Thank you very much for your feedback and congratulations on your work.

I would like to retrain the actuator net to be using a p_gain=30 and a slightly higher d_gain. The defaults have an amazing spring to them but not great for some of my motion captured gaits. I have LCMAgent all set up and working. I am assuming it can capture the data live from the agent. How do I set this up and what kinds of movements should it make to get the best data for training?

Thanks for taking the time to answer these questions. I am so close to getting it all my NN up and running on the Go1.

Hi!

Thanks for making your amazing work open source! I'm trying to adapt the go1_gym_deploy code and use the images received from go1's on-board camera, but I'm having difficulty finding the LCM publisher for the camera images on Go1; to my knowledge Go1's camera SDK uses ROS for communication instead of LCM, do you mind pointing out what is the lcm camera subscriber subscribing to in the code? Specifically the _camera_cb in cheetah_state_estimator.py:

def _camera_cb(self, channel, data):
        msg = camera_message_lcmt.decode(data)

        img = np.fromstring(msg.data, dtype=np.uint8)
        img = img.reshape((3, 200, 464)).transpose(1, 2, 0)

        cam_id = int(channel[-1])
        if cam_id == 1:
            self.camera_image_front = img
        elif cam_id == 2:
            self.camera_image_bottom = img
        elif cam_id == 3:
            self.camera_image_left = img
        elif cam_id == 4:
            self.camera_image_right = img
        elif cam_id == 5:
            self.camera_image_rear = img
        else:
            print("Image received from camera with unknown ID#!")

The image decoded is empty, and this happens after I start running the default policy provided.

Thanks!

Hi,

I've been attempting to utilize Ml-Dash tools to display the progress of my training, but the only available information on "localhost:3001" is output.log and video data. Whenever I choose a different file, the 'charts' section always appears empty. Is there a way to visualize the training progress using charts? Thank you.

May I ask if this code can be deployed on Unitree A1

In StateEstimator you get back 3 sets of leg values: joint_pos, joint_vel, and tau_est. Is tau_est the force feedback from the joints? If not do you know if its possible to get this information from the Go1?

I am getting ready to put my own neural network that takes all it's movements from motion capture. Your repo has been super helpful. Thanks for posting it.

Thank you for sharing your awesome code!

After installing the isaac gym and go1_gym, the ml-dash worked fine. However, when I ran python scripts/test.py, I have error messages like this:

File "/home/walk-these-ways/go1_gym/envs/base/legged_robot_config.py", line 3, in <module>
    from params_proto import PrefixProto, ParamsProto
ImportError: cannot import name 'PrefixProto' from 'params_proto' (/home/venv/walk-these-ways/lib/python3.8/site-packages/params_proto/__init__.py)

After that, I updated params_proto via pip install params-proto==2.10.0, and then I confirmed that python scripts/test.py works well,, but I have other error messages as below when I ran `python -m ml_dash.app'

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/venv/walk-these-ways/lib/python3.8/site-packages/ml_dash/app.py", line 5, in <module>
    from params_proto import cli_parse, Proto
ImportError: cannot import name 'cli_parse' from 'params_proto' (/home/venv/walk-these-ways/lib/python3.8/site-packages/params_proto/__init__.py)

Have you ever experienced such these issues?

I ran your code on Ubuntu 22.04, python3.8.

In the README, it requires python3.10, but my IssacGym does not support Python >= 3.9. I am using IsaacGym_Preview_4_Package. How did you manage to use python3.10? Thanks!

Hi, I'm trying to test this repository on a Go1 robot and I encountered a problem seemingly related to the power protection.

In the README you warn about the protection system cutting off power during fast running because of PowerProtect level 9. However, in my case, this happens very frequently even when the robot is moving relatively slowly.
Interestingly, when that happens I don't get an "Error: Power Protection" message (which instead happens when I set a lower PowerProtect level, as expected), also I lose ssh access and all the boards seem to reboot. This seems to indicate that a hardware power protection/failure is happening instead of the software power protection.
This was happening every time I ran the default lcm_position program, I had to reduce the initial P gain in order to be able to even start the controller.

I contacted Unitree, but they have not been really helpful, so I wanted to ask if you also encountered this problem.

I have trained a new network and want to deploy it onto my Go1. It looks to be using rsync to move all the runs to the go1. How do I get it to choose my run rather than pretrain-v0? (which I am assuming it automatically uses).

Thank you again for this repo.

I am wondering if you could share the data collection/training code for obtaining the actuator network.
Since the original paper uses the Anymal robot, which has actuators at its joint, the joint torque and the actuator torque should be the same. However, in Go1, the knee joint is driven by an actuator connected by a rod, thus the joint torque and the actuator torque are not equal in the knee, and I think they need some extra care.
I would appreciate it if you can share with me how you trained your actuator network for Go1.

Hi Gabe,

I was able to deploy the provided checkpoint on the Go1 EDU. Under neutral commands, i.e. when the 15 individual commands are

cmd_x = 0
cmd_y = 0
cmd_yaw = 0 
cmd_height = 0
cmd_freq = 3.0
# Mode = 1
cmd_phase = 0.5
cmd_offset = 0.0
cmd_bound = 0.0
cmd_duration = 0.5
cmd_footswing = 0.08
cmd_ori_pitch = 0.0
cmd_ori_roll = 0.0
cmd_stance_width = 0.33
cmd_stance_length = 0.4
cmd_aux_reward = 0.0

I expect the robot to stay where it is (with feet lifting and dropping). However there was a positive v_x and nonzero v_y observed so that the robot is actually walking towards front-right. Is this because I misunderstood some of the command above?

Thanks a lot!

Hi! I just noticed that there is a difference in the reward function between the paper and the code in this repo. In the paper, it is written as follows:

However, the code implementation is
`

def _reward_tracking_contacts_shaped_force(self):
    foot_forces = torch.norm(self.env.contact_forces[:, self.env.feet_indices, :], dim=-1)
    desired_contact = self.env.desired_contact_states

    reward = 0
    for i in range(4):
        reward += - (1 - desired_contact[:, i]) * (
                    1 - torch.exp(-1 * foot_forces[:, i] ** 2 / self.env.cfg.rewards.gait_force_sigma))
    return reward / 4

def _reward_tracking_contacts_shaped_vel(self):
    foot_velocities = torch.norm(self.env.foot_velocities, dim=2).view(self.env.num_envs, -1)
    desired_contact = self.env.desired_contact_states
    reward = 0
    for i in range(4):
        reward += - (desired_contact[:, i] * (
                    1 - torch.exp(-1 * foot_velocities[:, i] ** 2 / self.env.cfg.rewards.gait_vel_sigma)))
    return reward / 4

`

The 1-torch.exp() part makes it different with the ones in the paper. Could you tell me which one is the correct one?

Thanks!

Hi! I just noticed that the URDF that you used in this repo is different with the original URDF (https://github.com/unitreerobotics/unitree_ros/blob/master/robots/go1_description/urdf/go1.urdf). For example, the max. effort in the original repo is 23.7, while yours is 33.5. Is there any reason for that?

Thanks in advance! Have a nice day

Thank you for sharing the code for this awesome work.

I followed the setup in the README to set up your code and I would like to have some advice on the Running the Controller section. After running the second command (cd ~/go1_gym/go1_gym_deploy/docker; sudo make autostart), the code seems to stop after a while (please refer to the attached image).

I pressed [R2] several times at this moment but it did not do anything.

Do you have any suggestions about this situation?

Sincerely,

My networks seem to get a bit shaky (lots of random vibration) at a decimation of 4 (I am using a different NN based trained on motion capture). Have you tried lower ones like 2? Wondering if there are any problems with the go1 keeping up with such a small dt.

I also noticed you set armature=0. If I increase it to 0.01 as found in the mujco go1.xml file supplied by unitree the simulation seems to get more stable.

Hey, I had a basic question regarding the deployment. In the low level mode of the robot, is the robot set to a walking joint configuration when hung in the air, and then placed on the ground for the remote commands to work?

Not sure if this is the right place for this query, if not, I'm sorry and I will drop an email with this question.

I trained and deployed my own policy. Then used L2 button to start and then stop recording data. Then tried to train an actuator net with the data but it doesn't contain any "tau_est" values so it fails on line 166:

if "tau_est" not in datas[0].keys():
return

How do I get it to record the "tau_est"?

fyi: I managed to get my motion captured gaits working on the physical robot and they look great just a bit shaky. It would be great to get this actuator net working to see how good they get. Thank you for all your help.

Hi, thank you for the great work.

I encounter some problems while using this repo. I am wondering whether I got the right way to compute the reward. I found that in the training process, the reward is calculated by these 18 functions: 'tracking_lin_vel', 'tracking_ang_vel', 'lin_vel_z', 'ang_vel_xy', 'orientation', 'torques', 'dof_vel', 'dof_acc', 'collision', 'action_rate', 'tracking_contacts_shaped_force', 'tracking_contacts_shaped_vel', 'jump', 'dof_pos_limits', 'feet_slip', 'dof_pos', 'action_smoothness_1', 'action_smoothness_2'. However, some functions described in the paper such as 'raibert_heuristic_footswing_tracking' is not used in the reward calculation (although it exists in the CoRLReward file).

Could you please help clarify this? Did I miss some points?