Comments (10)
Hi @Bargez908 , I would like to how you are you sending actions to the endeffector of the robot in unity. Is it through ZMQ sockets? I wanted to know if you are just using unity for rendering or your simulation is also unity itself?
from mujoco.
Hi, currently we are controlling the position and rotation of the hand with a Mj Mocap (vive tracker) with the "tracked pose diver" as parent for the hand. Currently we are using Unity with the MuJoCo plugin where all the objects are decleared as MuJoCo objects (the hand too).
from mujoco.
@Bargez908 This looks like you need to configure the MjGeoms in your scene to use higher condim
(if its at 1, only normal forces are generated). Similarly by tweaking solimp
and solref
you can make geoms more resistant to penetration.
Relevant documentation:
https://mujoco.readthedocs.io/en/stable/XMLreference.html#body-geom-condim
If that's not what causing it let me know.
Also, to confirm, you control a mocap body, then use a weld constraint to softly move the arm's base there, right?
from mujoco.
Changing condim solved most of the interpenetration problems, the shadowand sometimes still slightly interpenatrate the objects if moved too fast tho.
Regarding the tangential friction, we have tried with both condim = 3 and condim = 6 but it doesn't seem to affect the simulation.
Desktop.2024.05.09.-.10.15.13.01_edit.mp4
The setting for the cylinder are the following:
(these are the same for both the cube underneath the cylinder and the shadow hand)
We are not using any soft costraints for the hand movement but instead is a direct control with the position of the HTC VIVE Pro Eye controller.
from mujoco.
So is the forearm's global position set simply using a mocap body? Then are there no joints in the hand? (As mocap bodies can't have child bodies)
Or do you set qpos directly manually?
If that is the case, the approach using a weld constraint detailed here is more appropriate for this purpose. The approach with a weld constraint constrains the maximum forces applied on the hand to make it track the mocap, otherwise massive contact forces can arise during interaction, which could prevent sliding/rolling.
from mujoco.
What we did was firstly import the shadow hand xml file using MuJoCo import, than we did "create empty" -> "Add component" -> "Mj Mocap Body", then we also added as component to the same game object "Tracked Pose Driver" (as shown in the image below)
After creating the Mocap we put underneath it in hierarchy the shadow hand (named right_hand870 in the image below) so that it's attached to the Vive controller. ART and ARR are respectively the translational and rotational joints of the base of the hand (the foreharm) while geom 1 to 4 are the Mj Geom that compose it. the hierarchy is Tracker (mocap) -> right hand (the beginning of the body) -> forearm -> wrist -> palm -> fingers.
(this is the right hand component, a simple change of coordinates)
from mujoco.
Okay in this case I can recommend trying the weld approach, that will give you more reliable interactions with the environment.
- Unparent the right hand from the mocap body
- Add a freejoint to the hand base
- Create a separate gameobject and add a MjWeld to it
- Set the tracker as Body1 and the hand base as Body2
- For the weld offset, assign any one transform that has 0 local position and no local rotation.
from mujoco.
It is working now thank you so much!
We can now correctly apply tangential forces and grasp objects.
There is one last problem tho, sometimes the simulation resets with the following error:
(we always had this issue but it was secondary to the tangential forces)
Desktop.2024.05.09.-.14.59.53.02_edit.mp4
from mujoco.
I'm glad it worked! Here's a discussion on simulation stability:
https://mujoco.readthedocs.io/en/stable/overview.html#divergence
If you haven't already, I recommend reducing the fixed timestep of your unity scene in your project settings.
The default physics timestep in Unity is much larger than usually expected by mujoco models.
If a high timestep is a necessity, you can explore switching to a more stable integrator (e.g. implicit) in your mujoco options component. Lastly you can add a tiny amount to the armature field to all of your joints.
from mujoco.
Alright, I will try those things, thank you again for your help!
from mujoco.
Related Issues (20)
- Camera rendering in MuJoCo - How to take a picture from a simulated camera and still keeping the GUI active
- MJX Documentation API Update HOT 1
- how to get flexible body position and velocity
- Possible to use different fluid force configuration among different links? HOT 2
- Problem using xpos to obtain the location of the xml file loaded model
- Manipulating Individual Joint Stiffness of a Elastic Cable HOT 1
- Issue with Camera Visibility of Dynamically Merged XML Models in MuJoCo HOT 1
- Issue with STL placements in new Model
- Support for Rendering in MJX for Simulated Camera Reinforcement Learning HOT 5
- `.skn` file not working with replicated bodies (dmcontrol rodent model) HOT 1
- Domain randomization for MJX HOT 7
- Resource not found (ValueError), but only on consecutive runs. HOT 12
- Equation for the Scalar Length l_i in the Actuator Transmission for Hinge Joints HOT 2
- Estimate qvel from sequence of rotations HOT 5
- How to simulate glass window/door in mujoco? HOT 8
- Assistance Required with Renderer Updates for Multiple XML Models in Reinforcement Learning HOT 2
- Modelling Shallow Water Correctly
- GLFW crashes on aarch64 wayland HOT 3
- [Unity plugin] Add kv property for position actuators HOT 2
- MJX False Collision in 3.1.4 and 3.1.5 only (no collision in MuJoCo) HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mujoco.