Hi, after following the steps as specified in ROS–Unity Initial Setup, I get the following errors/warnings when importing the ROS TCP Connector package:

error CS1501: No overload for method 'ToString' takes 2 arguments

In the specified files, I believe it concerns these lines of code:

public string ToString(string format) => internalVector.ToString(format);
public override string ToString() => internalVector.ToString();
public string ToString(string format, System.IFormatProvider formatProvider) => internalVector.ToString(format, formatProvider);

Do you have an idea what causes this issue and how it could be solved?`

Possible relevant info: using Unity 2019.3.6f1

Hello,
I am sending a videostream (CompressedImage msgs) from ROS (Melodic) to Unity (2020.2.1f1). But when when applying the image as texture I can only see half the image (as if the data buffer was not loaded fully). The same happens if I send only one image - sometimes the whole image is applied as texture, sometime only half. Sometimes I also get the following errors:

Exception raised!! System.ArgumentOutOfRangeException: Index and count must refer to a location within the buffer.
Parameter name: bytes
at System.Text.UTF8Encoding.GetString (System.Byte[] bytes, System.Int32 index, System.Int32 count) [0x00057] in <9577ac7a62ef43179789031239ba8798>:0
at RosMessageGeneration.Message.DeserializeString (System.Byte[] data, System.Int32 offset, System.Int32 stringLength)

Exception raised!! System.ArgumentOutOfRangeException: Non-negative number required.
Parameter name: count
at System.Text.UTF8Encoding.GetString (System.Byte[] bytes, System.Int32 index, System.Int32 count) [0x0003a] in <9577ac7a62ef43179789031239ba8798>:0
at RosMessageGeneration.Message.DeserializeString (System.Byte[] data, System.Int32 offset, System.Int32 stringLength)

Maybe someone else had the same problem or any ideas on how I could improve this?
Thanks!

When trying to load the PickandPlaceProject in Unity versions. 2019.4.24f1. 2020.3.2f1, 2021.1.1f1 and 2021.2.0a9, Unity outputs the following error.
[Package Manager] Done resolving packages in 2.67s seconds An error occurred while resolving packages: Project has invalid dependencies: com.unity.robotics.ros-tcp-connector: Error when executing git command. fatal: update_ref failed for ref 'HEAD': cannot update ref 'refs/heads/master': trying to write non-commit object b75c1fa508989fb7502bfcc0a1a89a318758b61c to branch 'refs/heads/master' com.unity.robotics.urdf-importer: Error when executing git command. fatal: update_ref failed for ref 'HEAD': cannot update ref 'refs/heads/master': trying to write non-commit object f663d6f959dac5abde446bd4ab19e799552273d7 to branch 'refs/heads/master' A re-import of the project may be required to fix the issue or a manual modification of /home/unity-linux/Documents/Dev/Unity-Robotics-Hub/tutorials/pick_and_place/PickAndPlaceProject/Packages/manifest.json file.

This is a known issue and is being tracked here.
https://issuetracker.unity3d.com/issues/package-resolution-error-when-using-a-git-dependency-referencing-an-annotated-tag-in-its-git-url

To solve this problem:

Choosing Ignore to Safemode allows you to manually add the packages via Package Manager via
https://github.com/Unity-Technologies/URDF-Importer.git?path=/com.unity.robotics.urdf-importer#main
https://github.com/Unity-Technologies/ROS-TCP-Connector.git?path=/com.unity.robotics.ros-tcp-connector#main

Just as stated in the Troubleshooting Section, "ROS-Unity server listening..." printed on the Unity side but no "ROS-Unity Handshake received" on the ROS side.
I am using Docker and I have set the ROS Settings in Unity as stated in the tutorial, and I also have set the "ROS_IP" into the correct one. I wonder what is going wrong?
Besides, I am using Unity 2020.2.3f1c1 Personal

The end effector breaks (as shown in the video below) for the pick-and-place tutorial.

Hi, I have been testing the new Object Pose Estimation Demo, and encountered that the gripper "broke" when using the new and better Temporal Gauss Seidel (TGS) physics solver with it. Here is a repro of the problem, where I open the pre-made project and change the solver from Projected GS to TGS:

Hi guys,

According to the pick-and-place tutorial, it seems the unity side just send the pose to the moveit, so the moveit cannot perform the collision detection, right?

Correct me if I miss something, thanks!

Hi,

When looking at the niryo_one.urdf file, it seems that for the grippers the wrong clamps are attached. Is this correct?

My OS is: Windows 10
My Unity Version is: Unity 2020.2.2f1
My ROS Distribution is: ROS Melodic Morenia
My Build Target Platform is: Android

Hi, I'm trying to build a project using Unity Robotics Hub in Android but it's not working.

I have no problems while connecting the Unity Editor with my ROS machine, but once I build it on Android it doesn´t work. The log says something related with serialization of the messages.

Looking for this issue I found something related in the ROS# package. They say it's the way the newtonsoft.Json interacts with the IL2CPP scripting backend, which is mandatory for Android. They solve this issue by changing the package which handles the serialization and the protocol. This is the topic where they solve it: siemens/ros-sharp#287.

Is there a way to do some workaround or something in order to achieve this with Unity Robotics Hub? Or I should migrate my project to their package if I want to build an Android apk?

Thanks in advance!

Hi,

I am following through the pick-and-place tutorial. I'm having an issue with ROS-Unity communication step.

roslaunch niryo_moveit part_3.launch

Output:

ERROR: cannot launch node of type [moveit_ros_move_group/move_group]: moveit_ros_move_group
ROS path [0]=/opt/ros/melodic/share/ros
ROS path [1]=/home/nullbyte/Desktop/Unity-Robotics-Hub/tutorials/pick_and_place/ROS/src
ROS path [2]=/opt/ros/melodic/share
[ INFO] [1607061951.680464153]: Starting niryo_one driver thread (frequency : 100.000000)
[ INFO] [1607061951.681187457]: Starting Fake Communication... It will just echo cmd into current position
[ INFO] [1607061951.682142155]: NiryoOne communication has been successfully started
Traceback (most recent call last):
  File "/home/nullbyte/Desktop/Unity-Robotics-Hub/tutorials/pick_and_place/ROS/src/niryo_moveit/scripts/mover.py", line 10, in <module>
    import moveit_commander
ImportError: No module named moveit_commander
[ INFO] [1607061951.782290067]: Start hardware control loop

Regards,
Jegathesan S

I am wondering whether it is possible to move a specific articulation body (e.g. gripper) with respect to the base frame in x, y, and z coordinates. I would like to build such a controller to mimic a teleoperation mode provided by a robot but in Unity simulation (https://www.kinovarobotics.com/site...Ultra_lightweight_robot-User_guide_EN_R01.pdf, page 27) but the only variables the default controller allows me to manipulate are the joint angles of each individual joint. The default controller referenced here is this one: https://github.com/Unity-Technologi...pendix.md##Guide-to-write-your-own-controller). Can someone provide a little guidance on how I may get started? I assume that applying velocity to an individual articulation body along the chain should cause all bodies attached to it to also move, but I am unable to make this happen. I did get this idea working on a simple cube articulation body without a chain.

Hi,

I am following the pick-and-place tutorial. I have followed every step and have encountered no problems up until the very last step. When clicking the 'publish' button in Unity to get the robot to perform the pick and place action, I get the following output in my terminal:

You can start planning now!

ROS-Unity Handshake received, will connect to 192.168.2.7:5005
[ INFO] [1606469570.519531449]: Loading robot model 'niryo_one'...
[ INFO] [1606469570.519943415]: No root/virtual joint specified in SRDF. Assuming fixed joint
[ WARN] [1606469570.541337238]: Kinematics solver doesn't support #attempts anymore, but only a timeout.
Please remove the parameter '/robot_description_kinematics/arm/kinematics_solver_attempts' from your configuration.
[ INFO] [1606469571.595206419]: Ready to take commands for planning group arm.
[ INFO] [1606469571.825751695]: Combined planning and execution request received for MoveGroup action. Forwarding to planning and execution pipeline.
[ WARN] [1606469571.825966961]: Execution of motions should always start at the robot's current state. Ignoring the state supplied as start state in the motion planning request
[ INFO] [1606469571.826019492]: Planning attempt 1 of at most 1
[ INFO] [1606469571.829529746]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1606469571.830429183]: RRTConnect: Starting planning with 1 states already in datastructure
[ERROR] [1606469576.839300394]: RRTConnect: Unable to sample any valid states for goal tree
[ INFO] [1606469576.839339522]: RRTConnect: Created 1 states (1 start + 0 goal)
[ INFO] [1606469576.839352531]: No solution found after 5.009120 seconds
[ INFO] [1606469576.840151143]: Unable to solve the planning problem
[ INFO] [1606469576.840341969]: ABORTED: No motion plan found. No execution attempted.
[ERROR] [1606469576.840967]: Error processing request: 
            Trajectory could not be planned for a destination of position: 
  x: 0.153000488877
  y: 0.264993906021
  z: 0.7437190786
orientation: 
  x: 0.5
  y: 0.5
  z: -0.5
  w: 0.5 with starting joint angles position: 
  x: 0.153000488877
  y: 0.264993906021
  z: 0.7437190786
orientation: 
  x: 0.5
  y: 0.5
  z: -0.5
  w: 0.5.
            Please make sure target and destination are reachable by the robot.
        
['Traceback (most recent call last):\n', '  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/impl/tcpros_service.py", line 632, in _handle_request\n    response = convert_return_to_response(self.handler(request), self.response_class)\n', '  File "/home/stefan/unity_ws/Unity-Robotics-Hub/tutorials/pick_and_place/ROS/src/niryo_moveit/scripts/mover.py", line 89, in plan_pick_and_place\n    pre_grasp_pose = plan_trajectory(move_group, req.pick_pose, current_robot_joint_configuration)\n', '  File "/home/stefan/unity_ws/Unity-Robotics-Hub/tutorials/pick_and_place/ROS/src/niryo_moveit/scripts/mover.py", line 53, in plan_trajectory\n    raise Exception(exception_str)\n', 'Exception: \n            Trajectory could not be planned for a destination of position: \n  x: 0.153000488877\n  y: 0.264993906021\n  z: 0.7437190786\norientation: \n  x: 0.5\n  y: 0.5\n  z: -0.5\n  w: 0.5 with starting joint angles position: \n  x: 0.153000488877\n  y: 0.264993906021\n  z: 0.7437190786\norientation: \n  x: 0.5\n  y: 0.5\n  z: -0.5\n  w: 0.5.\n            Please make sure target and destination are reachable by the robot.\n        \n']
Service Exception raised. Attempt: 1

My environment in Unity looks like this:

It looks like the robot should be easily able to reach the target. I have tried to move the objects but the issue persists.

The positions of the entities in Unity are:

target: -0.265, 0.644, 0.153
targetplacement: 0.281, 0.640, 0.124
nitryo_one (base_link): 0, 0.63, 0
Table: 0, 0, 0

It seems like the target is converted correctly from Unity to ROS (X,Y,Z) to (Z, -X, Y)

I am working on Ubuntu 18.04 and Unity 2020.2.0b9.

Any clue why this is happening? Thanks!

Environment:

• Unity 2021.1b
• MacOS Catalina (10.15.7)

As per the installation instructions, I added the following git URL via the Package Manager:
https://github.com/Unity-Technologies/ROS-TCP-Connector.git

However, the following errors show up:

[Package Manager Window] Cannot perform upm operation: Unable to add package [https://github.com/Unity-Technologies/ROS-TCP-Connector.git]:
  [https://github.com/Unity-Technologies/ROS-TCP-Connector.git] does not point to a valid package repository. No package manifest was found. Verify the repository URL and make sure the package is located in the root folder of the repository. [NotFound].
UnityEditor.EditorApplication:Internal_CallUpdateFunctions () (at /Users/bokken/buildslave/unity/build/Editor/Mono/EditorApplication.cs:327)

[Package Manager Window] Error adding package: https://github.com/Unity-Technologies/ROS-TCP-Connector.git.
UnityEditor.EditorApplication:Internal_CallUpdateFunctions () (at /Users/bokken/buildslave/unity/build/Editor/Mono/EditorApplication.cs:327)

Unity Version: 2020.3.0f1

If I make the imported robot arm into a prefab, it loses its mesh colliders.

Steps to reproduce:

1. Open up the pick-n-place tutorial project
2. Import the Niryo One arm using the URDF importer
3. Drag the new niryo_one instance into the prefabs directory
4. Open the newly made prefab and check the mesh_collider of any link (the mesh reference will be None)

Example:

Hi,

There are two ros msg package[ros-melodic-octomap-msgs, ros-melodic-object-recognition-msgs] need to install to compile src code by using catkin_make.

We can update this in README.

Regards,
Jegathesan S

There are nice tutorials about how to set publiser, subscriber in ros-unity integration in ros_unity_integration/README.md. As for the service, I can only find an example which sets the ROS as the server and Unity as the client.

Is there any example about how to set the ROS as the client and Unity as the server? What is the code to achieve this in Unity scripts?

Thank you very much!

I am getting a KeyError for /ROS_IP on my demo server that was generated in catkin. No changes were done to ros environment or catkin environment.

Traceback (most recent call last):
  File "/mnt/c/Users/rosen/OneDrive/Desktop/Imperial/MRes_Research/Projects/catkin_space/src/robotics_demo/scripts/server_endpoint.py", line 26, in <module>
    main()
  File "/mnt/c/Users/rosen/OneDrive/Desktop/Imperial/MRes_Research/Projects/catkin_space/src/robotics_demo/scripts/server_endpoint.py", line 13, in main
    tcp_server = TcpServer(ros_node_name, buffer_size, connections)
  File "/mnt/c/Users/rosen/OneDrive/Desktop/Imperial/MRes_Research/Projects/catkin_space/src/ROS-TCP-Endpoint/src/ros_tcp_endpoint/server.py", line 45, in __init__
    self.tcp_ip = rospy.get_param("/ROS_IP")
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/client.py", line 467, in get_param
    return _param_server[param_name] #MasterProxy does all the magic for us
  File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/msproxy.py", line 123, in __getitem__
    raise KeyError(key)
KeyError: '/ROS_IP'

Any tips on how to repair?

Expected Behaviour:

Following the tuorial, the Pick and Place should be imported correctly.

Actual Behaviour :
On importing the PickandPlaceProject folder, an error prevents the code from being executed and thus the project from being correctly imported.

Steps to reproduce:

Set up acompletelynew Unity installation on Windows 10.
Try to follow the PickandPlace tutorial.
On the step to import the project, it will fail.

Workaround Part 1:
Import the Editor Coroutines in the UnityPackageManager.
I would have expected that to be mentioned in the tutorial.

-> Still, Compilation fails. Now at line 188 at the Demo.cs file.

Workaround Part 2: Replace 2 Lines in Demo.cs with one
old lines:
188 var robotImporter = UrdfRobotExtensions.Create(<THREE arguments, cant be bothered to go back to that state right now, sry>);
189 while (robotImporter.MoveNext()) {}

replace with

188 UrdfRobotExtensions.Create(urdfFilepath, urdfImportSettings);

Now it compiles without errors and actually works.

Hi, thanks for helping in advance.

I encountered this error while I was adding the ROS-TCP-Connector through Unity's Package Manager:

Must I have ROS installed first before import this package? Also, may it be a platform-related question? I am currently trying this on my mac.

For a project, we are trying to stream 4 CompressedImage topics which are publishing at 30Hz to Unity. In the search for a ROS-Unity connection that can handle this, we came across your Unity Robotics Hub which promises to provide faster speeds and better performance than for example the ROS# package.

After following the guides, we managed to successfully setup a connection. However, as soon as we start publishing the image data (even from only just one camera), the Unity screen basically freezes and the FPS shown in the statistics widget of Unity drastically drops from ~190 FPS to 0.1 FPS. At the moment we are using the settings as shown in the screenshot below.

Is there anything specific that we need to change/set correctly before the connection can handle images?

Hi guys,

According to the pick-and-place tutorial, it seems the unity side just send the pose to the moveit, so the moveit cannot perform the collision detection, right?

Correct me if I miss something, thanks!

Hi,

I have a question regarding grasping a rigidbody. I've found that for my specific robot, the grasping is quite inconsistent. Most of the time, the robot attempts to pick up the object but it will slip out of its reach. Other times, it is able to hold on to the object. I was wondering if anybody has suggestions about how I might go about ensuring a successful grasp all the time, particularly the case where the object slips after it has already been held. Which parameters could be tuned to ensure its success? Thanks!

Hello.

I have managed to replicate the steps all the way from part 0 till part 3. The last step would be to press the publish button. Clicking on this button triggers multiple motion planning/trajectory calculation requests on the ROS side. The robot in unity just stays perfectly still. I noticed after some time the gripper opens.

I'm running ROS noetic on WSL for windows 10 with Ubuntu 20.04. I have two versions of Unity installed: 2020.2.4f1 as well as 2020.2.0b9, with the tutorial running on the latter.

ROS-side console output :

[ INFO] [1613574625.397392500]: Loading robot model 'niryo_one'...
[ INFO] [1613574625.398079400]: No root/virtual joint specified in SRDF. Assuming fixed joint
[ WARN] [1613574625.705789400]: Kinematics solver doesn't support #attempts anymore, but only a timeout.
Please remove the parameter '/robot_description_kinematics/arm/kinematics_solver_attempts' from your configuration.
[ INFO] [1613574626.809087400]: Ready to take commands for planning group arm.
[ INFO] [1613574627.035894900]: Combined planning and execution request received for MoveGroup action. Forwarding to planning and execution pipeline.
[ WARN] [1613574627.036133900]: Execution of motions should always start at the robot's current state. Ignoring the state supplied as start state in the motion planning request
[ INFO] [1613574627.036223600]: Planning attempt 1 of at most 1
[ INFO] [1613574627.038109100]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1613574627.038721700]: arm/arm: Starting planning with 1 states already in datastructure
[ INFO] [1613574627.049564200]: arm/arm: Created 4 states (2 start + 2 goal)
[ INFO] [1613574627.049646100]: Solution found in 0.011161 seconds
[ INFO] [1613574627.052300400]: SimpleSetup: Path simplification took 0.001223 seconds and changed from 3 to 2 states
[ INFO] [1613574627.054228800]: Disabling trajectory recording
[ INFO] [1613574635.455153100]: Received trajectory RESULT
[ INFO] [1613574635.455391600]: Controller niryo_one_follow_joint_trajectory_controller successfully finished
[ INFO] [1613574635.455560300]: Completed trajectory execution with status SUCCEEDED ...
[ INFO] [1613574635.475760800]: Planning request received for MoveGroup action. Forwarding to planning pipeline.
[ INFO] [1613574635.476896200]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1613574635.477080700]: arm/arm: Starting planning with 1 states already in datastructure
[ INFO] [1613574635.487536000]: arm/arm: Created 4 states (2 start + 2 goal)
[ INFO] [1613574635.487615800]: Solution found in 0.010578 seconds
[ INFO] [1613574635.490486900]: SimpleSetup: Path simplification took 0.002458 seconds and changed from 3 to 2 states
[ INFO] [1613574635.515298900]: Combined planning and execution request received for MoveGroup action. Forwarding to planning and execution pipeline.
[ WARN] [1613574635.515413000]: Execution of motions should always start at the robot's current state. Ignoring the state supplied as start state in the motion planning request
[ INFO] [1613574635.515471800]: Planning attempt 1 of at most 1
[ INFO] [1613574635.516218700]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1613574635.516379900]: arm/arm: Starting planning with 1 states already in datastructure
[ INFO] [1613574635.526968700]: arm/arm: Created 4 states (2 start + 2 goal)
[ INFO] [1613574635.527057100]: Solution found in 0.010711 seconds
[ INFO] [1613574635.528996500]: SimpleSetup: Path simplification took 0.001051 seconds and changed from 3 to 2 states
[ INFO] [1613574637.929882100]: Received trajectory RESULT
[ INFO] [1613574637.930037700]: Controller niryo_one_follow_joint_trajectory_controller successfully finished
[ INFO] [1613574637.930146600]: Completed trajectory execution with status SUCCEEDED ...
[ INFO] [1613574637.955422600]: Planning request received for MoveGroup action. Forwarding to planning pipeline.
[ INFO] [1613574637.956379000]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1613574637.956764300]: arm/arm: Starting planning with 1 states already in datastructure
[ INFO] [1613574637.977600100]: arm/arm: Created 5 states (2 start + 3 goal)
[ INFO] [1613574637.977691400]: Solution found in 0.021039 seconds
[ INFO] [1613574637.978974000]: SimpleSetup: Path simplification took 0.001227 seconds and changed from 4 to 2 states
[ INFO] [1613574638.015486400]: Combined planning and execution request received for MoveGroup action. Forwarding to planning and execution pipeline.
[ WARN] [1613574638.015629600]: Execution of motions should always start at the robot's current state. Ignoring the state supplied as start state in the motion planning request
[ INFO] [1613574638.015697800]: Planning attempt 1 of at most 1
[ INFO] [1613574638.016609500]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1613574638.016827600]: arm/arm: Starting planning with 1 states already in datastructure
[ INFO] [1613574638.027434000]: arm/arm: Created 4 states (2 start + 2 goal)
[ INFO] [1613574638.027548600]: Solution found in 0.010770 seconds
[ INFO] [1613574638.029289100]: SimpleSetup: Path simplification took 0.001298 seconds and changed from 3 to 2 states
[ INFO] [1613574640.430118800]: Received trajectory RESULT
[ INFO] [1613574640.430311900]: Controller niryo_one_follow_joint_trajectory_controller successfully finished
[ INFO] [1613574640.430411800]: Completed trajectory execution with status SUCCEEDED ...
[ INFO] [1613574640.455818800]: Planning request received for MoveGroup action. Forwarding to planning pipeline.
[ INFO] [1613574640.456663700]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1613574640.456821800]: arm/arm: Starting planning with 1 states already in datastructure
[ INFO] [1613574640.467361300]: arm/arm: Created 4 states (2 start + 2 goal)
[ INFO] [1613574640.467434000]: Solution found in 0.010641 seconds
[ INFO] [1613574640.469247100]: SimpleSetup: Path simplification took 0.001671 seconds and changed from 3 to 2 states
[ INFO] [1613574640.495590200]: Combined planning and execution request received for MoveGroup action. Forwarding to planning and execution pipeline.
[ WARN] [1613574640.495711400]: Execution of motions should always start at the robot's current state. Ignoring the state supplied as start state in the motion planning request
[ INFO] [1613574640.495771400]: Planning attempt 1 of at most 1
[ INFO] [1613574640.496638800]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1613574640.496840900]: arm/arm: Starting planning with 1 states already in datastructure
[ INFO] [1613574640.507447200]: arm/arm: Created 5 states (2 start + 3 goal)
[ INFO] [1613574640.507536800]: Solution found in 0.010743 seconds
[ INFO] [1613574640.510536600]: SimpleSetup: Path simplification took 0.002870 seconds and changed from 4 to 2 states
[ INFO] [1613574650.662529200]: Received trajectory RESULT
[ INFO] [1613574650.662774300]: Controller niryo_one_follow_joint_trajectory_controller successfully finished
[ INFO] [1613574650.662884900]: Completed trajectory execution with status SUCCEEDED ...
[ INFO] [1613574650.675734400]: Planning request received for MoveGroup action. Forwarding to planning pipeline.
[ INFO] [1613574650.676551500]: Planner configuration 'arm' will use planner 'geometric::RRTConnect'. Additional configuration parameters will be set when the planner is constructed.
[ INFO] [1613574650.676733500]: arm/arm: Starting planning with 1 states already in datastructure
[ INFO] [1613574650.687357000]: arm/arm: Created 5 states (2 start + 3 goal)
[ INFO] [1613574650.687433100]: Solution found in 0.010743 seconds
[ INFO] [1613574650.690867300]: SimpleSetup: Path simplification took 0.003085 seconds and changed from 4 to 2 states

I would appreciate any help with tracking down the source of the issue on this. Thanks.

Hi

I would like to combine the functionalities of the URDF importer and TCP Connector with the ml-agents framework, to mimic a real-world setup with ROS while still making use of the features provided by ml-agents for interfacing with the environment in python such as the gym-interface and the side-channels.

However at first sight I cannot find any guidelines on how to do this. Is there any information available on how to combine the functionalities of these two packages?

Hi,

I seem to be missing the "Robotics" menu in the Unity editor. Instead my menu looks like this:

Following the tutorial for pick and place, I need the menu for the ROS settings. Is the tutorial outdated or is there something wrong with my installation?

Thanks

Hi, When I Press the UI Button to send the joint configurations to ROS in Pick-and-Place Tutorial: Part 3,there is an error in unity console:

I got a error when I add the package from the package manager(quick setup part).

[Package Manager Window] Unable to add package [https://github.com/Unity-Technologies/ROS-TCP-Connector.git]:
[https://github.com/Unity-Technologies/ROS-TCP-Connector.git] does not point to a valid package repository. No package manifest was found. Verify the repository URL and make sure the package is located in the root folder of the repository.
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

how should I solve this?

Thanks in advance!

Hi guys,

According to the pick-and-place tutorial, it seems the unity side just send the pose to the moveit, so the moveit cannot perform the collision detection, right?

Correct me if I miss something, thanks!

Dear all,
thanks for the great examples and apologies if I ask a silly question.

If I move the robot arm in unity, how can I make the solver aware that its position changed? It happily continues to plan a trajectory even if the targets are now out of reach

Thanks and regards,

Mario

Hi!

I tried to apply the tutorial on an Android smartphone and a HoloLens 2.. both builds show the same strange behavior, that the robot arm is randomly jumping. Any solution for that? HoloLens 2 seems to connect to the ROS node (I get the request in the ROS terminal), but with Android this does not work.

Thank you!

Hi!

I get the following error opening the Unity project..

An error occurred while resolving packages:
Project has invalid dependencies:
com.unity.robotics.urdf-importer: Cannot checkout repository [https://github.com/Unity-Technologies/URDF-Importer.git]:
Error when executing git command. Submodule 'VHACD' (https://github.com/Unity-Technologies/VHACD) registered for path 'Runtime/VHACD'
Cloning into 'D:/Unity/Unity-Robotics-Hub/tutorials/pick_and_place/PickAndPlaceProject/Library/PackageCache/.tmp7792ZYEX83svpiql/clone/Runtime/VHACD'...
remote: Repository not found.
fatal: repository 'https://github.com/Unity-Technologies/VHACD/' not found
fatal: clone of 'https://github.com/Unity-Technologies/VHACD' into submodule path 'D:/Unity/Unity-Robotics-Hub/tutorials/pick_and_place/PickAndPlaceProject/Library/PackageCache/.tmp7792ZYEX83svpiql/clone/Runtime/VHACD' failed
Failed to clone 'Runtime/VHACD'. Retry scheduled
Cloning into 'D:/Unity/Unity-Robotics-Hub/tutorials/pick_and_place/PickAndPlaceProject/Library/PackageCache/.tmp7792ZYEX83svpiql/clone/Runtime/VHACD'...
remote: Repository not found.
fatal: repository 'https://github.com/Unity-Technologies/VHACD/' not found
fatal: clone of 'https://github.com/Unity-Technologies/VHACD' into submodule path 'D:/Unity/Unity-Robotics-Hub/tutorials/pick_and_place/PickAndPlaceProject/Library/PackageCache/.tmp7792ZYEX83svpiql/clone/Runtime/VHACD' failed
Failed to clone 'Runtime/VHACD' a second time, aborting

A re-import of the project may be required to fix the issue or a manual modification of D:/Unity/Unity-Robotics-Hub/tutorials/pick_and_place/PickAndPlaceProject/Packages/manifest.json file.

Where can I find the VHACD repro? Thank you!

I checked everything !!
but Unity fails to find a network connection

OS: MAC

successfully started, and that a message
"[INFO] [1614939422.340810]: Starting server on 172.17.0.2:10000" is printed.

src/niryo_moveit/config/params.yaml <- Check!! "ROS_IP: 172.17.0.2"

Unity/ROS Settings/ROS IP Address and ROS Port <- Check!! IP: 172.17.0.2 Port: 10000

Terminal Port Check
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
com.docke 511 saehyeonkim 35u IPv6 0x8359fceeaaff148d 0t0 TCP *:ndmp (LISTEN)
Unity 939 saehyeonkim 69u IPv4 0x8359fceeab8c63b5 0t0 TCP 192.168.35.53:50095->172.17.0.2:ndmp (SYN_SENT)
Unity 939 saehyeonkim 83u IPv4 0x8359fceeab8c1d0d 0t0 TCP 192.168.35.53:50096->172.17.0.2:ndmp (SYN_SENT)
Unity 939 saehyeonkim 112u IPv4 0x8359fceeacde8ec5 0t0 TCP 192.168.35.53:50094->172.17.0.2:ndmp (SYN_SENT)

****Error message 1 ****
SocketException: Connection timed out
System.Net.Sockets.SocketAsyncResult.CheckIfThrowDelayedException () (at :0)
System.Net.Sockets.Socket.EndConnect (System.IAsyncResult asyncResult) (at :0)
System.Net.Sockets.TcpClient.EndConnect (System.IAsyncResult asyncResult) (at :0)
System.Threading.Tasks.TaskFactory1[TResult].FromAsyncCoreLogic (System.IAsyncResult iar, System.Func2[T,TResult] endFunction, System.Action1[T] endAction, System.Threading.Tasks.Task1[TResult] promise, System.Boolean requiresSynchronization) (at <9577ac7a62ef43179789031239ba8798>:0)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <9577ac7a62ef43179789031239ba8798>:0)
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) (at <9577ac7a62ef43179789031239ba8798>:0)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) (at <9577ac7a62ef43179789031239ba8798>:0)
System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) (at <9577ac7a62ef43179789031239ba8798>:0)
System.Runtime.CompilerServices.TaskAwaiter.GetResult () (at <9577ac7a62ef43179789031239ba8798>:0)
Unity.Robotics.ROSTCPConnector.ROSConnection+d__23`1[RESPONSE].MoveNext () (at Library/PackageCache/com.unity.robotics.ros-tcp-connector@b75c1fa508/Runtime/TcpConnector/ROSConnection.cs:109)
--- End of stack trace from previous location where exception was thrown ---
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <9577ac7a62ef43179789031239ba8798>:0)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.b__6_0 (System.Object state) (at <9577ac7a62ef43179789031239ba8798>:0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at /Users/bokken/buildslave/unity/build/Runtime/Export/Scripting/UnitySynchronizationContext.cs:153)
UnityEngine.UnitySynchronizationContext:ExecuteTasks() (at /Users/bokken/buildslave/unity/build/Runtime/Export/Scripting/UnitySynchronizationContext.cs:107)

Error message 2
TCPConnector Exception: System.Net.Sockets.SocketException (0x80004005): Connection timed out
at System.Net.Sockets.SocketAsyncResult.CheckIfThrowDelayedException () [0x00014] in :0
at System.Net.Sockets.Socket.EndConnect (System.IAsyncResult asyncResult) [0x0002c] in :0
at System.Net.Sockets.TcpClient.EndConnect (System.IAsyncResult asyncResult) [0x0000c] in :0
at System.Threading.Tasks.TaskFactory1[TResult].FromAsyncCoreLogic (System.IAsyncResult iar, System.Func2[T,TResult] endFunction, System.Action1[T] endAction, System.Threading.Tasks.Task1[TResult] promise, System.Boolean requiresSynchronization) [0x00019] in <9577ac7a62ef43179789031239ba8798>:0
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <9577ac7a62ef43179789031239ba8798>:0
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <9577ac7a62ef43179789031239ba8798>:0
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <9577ac7a62ef43179789031239ba8798>:0
at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <9577ac7a62ef43179789031239ba8798>:0
at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <9577ac7a62ef43179789031239ba8798>:0
at Unity.Robotics.ROSTCPConnector.ROSConnection+d__47.MoveNext () [0x00051] in /Users/saehyeonkim/Unity3D/StudyProject/Unity-Robotics-Hub/Unity-Robotics-Hub/tutorials/pick_and_place/PickAndPlaceProject/Library/PackageCache/com.unity.robotics.ros-tcp-connector@b75c1fa508/Runtime/TcpConnector/ROSConnection.cs:518
UnityEngine.Debug:LogError (object)
Unity.Robotics.ROSTCPConnector.ROSConnection/d__47:MoveNext () (at Library/PackageCache/com.unity.robotics.ros-tcp-connector@b75c1fa508/Runtime/TcpConnector/ROSConnection.cs:531)
UnityEngine.UnitySynchronizationContext:ExecuteTasks () (at /Users/bokken/buildslave/unity/build/Runtime/Export/Scripting/UnitySynchronizationContext.cs:107)

Hello,

It looks like the latest DockerFile provided in the ROS Setup is not building correctly on my machine.

I am receiving the current error in the 12th step when I run docker build -t unity-robotics:pick-and-place -f docker/Dockerfile .

Am I missing something?

 => CACHED [11/12] RUN /bin/bash -c "find /catkin_ws -type f -prin  0.0s
 => ERROR [12/12] RUN dos2unix /tutorial && dos2unix /setup.sh &&   8.4s
------
 > [12/12] RUN dos2unix /tutorial && dos2unix /setup.sh && chmod +x /setup.sh && /setup.sh && rm /setup.sh:
#17 0.700 dos2unix: converting file /tutorial to Unix format...
#17 0.702 dos2unix: converting file /setup.sh to Unix format...
#17 1.119 -- The C compiler identification is GNU 7.5.0
#17 1.221 -- The CXX compiler identification is GNU 7.5.0
.
.
.
#17 7.335 -- Generating .msg files for action moveit_msgs/Pickup /catkin_ws/src/moveit_msgs/action/Pickup.action
#17 7.354 Generating for action Pickup
#17 7.357 -- Generating .msg files for action moveit_msgs/Place /catkin_ws/src/moveit_msgs/action/Place.action
#17 7.384 Generating for action Place
#17 7.938 -- moveit_msgs: 81 messages, 24 services
#17 8.271 -- +++ processing catkin package: 'niryo_moveit'
#17 8.271 -- ==> add_subdirectory(niryo_moveit)
#17 8.301 -- Could NOT find ros_tcp_endpoint (missing: ros_tcp_endpoint_DIR)
#17 8.301 -- Could not find the required component 'ros_tcp_endpoint'. The following CMake error indicates that you either need to install the package with the same name or change your environment so that it can be found.
#17 8.301 CMake Error at /opt/ros/melodic/share/catkin/cmake/catkinConfig.cmake:83 (find_package):
#17 8.301   Could not find a package configuration file provided by "ros_tcp_endpoint"
#17 8.301   with any of the following names:
#17 8.301 
#17 8.301     ros_tcp_endpointConfig.cmake
#17 8.301     ros_tcp_endpoint-config.cmake
#17 8.301 
#17 8.301   Add the installation prefix of "ros_tcp_endpoint" to CMAKE_PREFIX_PATH or
#17 8.301   set "ros_tcp_endpoint_DIR" to a directory containing one of the above
#17 8.301   files.  If "ros_tcp_endpoint" provides a separate development package or
#17 8.301   SDK, be sure it has been installed.
#17 8.301 Call Stack (most recent call first):
#17 8.301   niryo_moveit/CMakeLists.txt:4 (find_package)
#17 8.301 
#17 8.301 
#17 8.307 -- Configuring incomplete, errors occurred!
#17 8.307 See also "/catkin_ws/build/CMakeFiles/CMakeOutput.log".
#17 8.307 See also "/catkin_ws/build/CMakeFiles/CMakeError.log".
#17 8.314 Invoking "cmake" failed
#17 8.314 Base path: /catkin_ws
#17 8.314 Source space: /catkin_ws/src
#17 8.314 Build space: /catkin_ws/build
#17 8.314 Devel space: /catkin_ws/devel
#17 8.314 Install space: /catkin_ws/install
#17 8.314 Creating symlink "/catkin_ws/src/CMakeLists.txt" pointing to "/opt/ros/melodic/share/catkin/cmake/toplevel.cmake"
#17 8.314 ####
#17 8.314 #### Running command: "cmake /catkin_ws/src -DCATKIN_DEVEL_PREFIX=/catkin_ws/devel -DCMAKE_INSTALL_PREFIX=/catkin_ws/install -G Unix Makefiles" in "/catkin_ws/build"
#17 8.314 ####
------
executor failed running [/bin/sh -c dos2unix /tutorial && dos2unix /setup.sh && chmod +x /setup.sh && /setup.sh && rm /setup.sh]: exit code: 1

Hello,

I am running into an issue where I have configured the robotics_demo server_endpoint and am running the RosPublisher from Unity but receive a TCPConnecectorException stating: No connection could be made because the target machine actively refused it. The ROS Settings have been correctly configured.

Is this an issue with connecting to ROS running in WSL?

In the tutorials theres a docker image that is used as an alternative for the pickandplace example.
Can the DockerFile used for the image: unity-robotics:pick-and-place be shared. Attempting to get this to work on WSL has become quite a hassle.

When I load the Part1Done scene I see that the niryo_one works and is fully functional as seen here:

Yet when I delete the niryo_one and import the same niryo_one.urdf file into the project I see this behavior when I play the scene (No errors in the console):

Additionally, I'm noticing this error when I attempt to import the urdf on my other windows machine. It was previously running successful but now fails due to an issue finding the vahcdl.dll

What could be the issues?

DllNotFoundException: libvhacd
MeshProcess.VHACD.GenerateConvexMeshes (UnityEngine.Mesh mesh) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Runtime/VHACD/VHACD.cs:138)
RosSharp.Urdf.Editor.UrdfGeometryCollision.ConvertMeshToColliders (UnityEngine.GameObject gameObject, System.Boolean setConvex) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/NeedsRuntimeConversion/Extensions/UrdfGeometryCollision.cs:131)
RosSharp.Urdf.Editor.UrdfGeometryCollision.CreateMeshCollider (RosSharp.Urdf.Link+Geometry+Mesh mesh) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/NeedsRuntimeConversion/Extensions/UrdfGeometryCollision.cs:71)
RosSharp.Urdf.Editor.UrdfGeometryCollision.Create (UnityEngine.Transform parent, RosSharp.Urdf.GeometryTypes geometryType, RosSharp.Urdf.Link+Geometry geometry) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/NeedsRuntimeConversion/Extensions/UrdfGeometryCollision.cs:44)
RosSharp.Urdf.Editor.UrdfCollisionExtensions.Create (UnityEngine.Transform parent, RosSharp.Urdf.Link+Collision collision) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/NeedsRuntimeConversion/Extensions/UrdfCollisionExtensions.cs:54)
RosSharp.Urdf.Editor.UrdfCollisionsExtensions.Create (UnityEngine.Transform parent, System.Collections.Generic.List`1[T] collisions) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/NeedsRuntimeConversion/Extensions/UrdfCollisionsExtensions.cs:35)
RosSharp.Urdf.Editor.UrdfLinkExtensions.Create (UnityEngine.Transform parent, RosSharp.Urdf.Link link, RosSharp.Urdf.Joint joint) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/NeedsRuntimeConversion/Extensions/UrdfLinkExtensions.cs:27)
RosSharp.Urdf.Editor.UrdfLinkExtensions.ImportLinkData (RosSharp.Urdf.UrdfLink urdfLink, RosSharp.Urdf.Link link, RosSharp.Urdf.Joint joint) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/NeedsRuntimeConversion/Extensions/UrdfLinkExtensions.cs:65)
RosSharp.Urdf.Editor.UrdfLinkExtensions.Create (UnityEngine.Transform parent, RosSharp.Urdf.Link link, RosSharp.Urdf.Joint joint) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/NeedsRuntimeConversion/Extensions/UrdfLinkExtensions.cs:31)
RosSharp.Urdf.Editor.UrdfRobotExtensions.Create (System.String filename, RosSharp.ImportSettings settings) (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/NeedsRuntimeConversion/Extensions/UrdfRobotExtensions.cs:74)
RosSharp.Urdf.Editor.FileImportMenu.OnGUI () (at Library/PackageCache/com.unity.robotics.urdf-importer@5ba94b3aac/Editor/MenuItems/FileImportMenu.cs:58)
UnityEditor.HostView.InvokeOnGUI (UnityEngine.Rect onGUIPosition, UnityEngine.Rect viewRect) (at <27ac941b3da34983800e9a41b73eceef>:0)
UnityEditor.DockArea.DrawView (UnityEngine.Rect viewRect, UnityEngine.Rect dockAreaRect) (at <27ac941b3da34983800e9a41b73eceef>:0)
UnityEditor.DockArea.OldOnGUI () (at <27ac941b3da34983800e9a41b73eceef>:0)
UnityEngine.UIElements.IMGUIContainer.DoOnGUI (UnityEngine.Event evt, UnityEngine.Matrix4x4 parentTransform, UnityEngine.Rect clippingRect, System.Boolean isComputingLayout, UnityEngine.Rect layoutSize, System.Action onGUIHandler, System.Boolean canAffectFocus) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, UnityEngine.Matrix4x4 worldTransform, UnityEngine.Rect clippingRect, System.Action onGUIHandler, System.Boolean canAffectFocus) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, System.Action onGUIHandler, System.Boolean canAffectFocus) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, System.Boolean canAffectFocus) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.IMGUIContainer.SendEventToIMGUIRaw (UnityEngine.UIElements.EventBase evt, System.Boolean canAffectFocus, System.Boolean verifyBounds) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.IMGUIContainer.SendEventToIMGUI (UnityEngine.UIElements.EventBase evt, System.Boolean canAffectFocus, System.Boolean verifyBounds) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.IMGUIContainer.HandleEvent (UnityEngine.UIElements.EventBase evt) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.CallbackEventHandler.HandleEventAtTargetPhase (UnityEngine.UIElements.EventBase evt) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.MouseCaptureDispatchingStrategy.DispatchEvent (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.EventDispatcher.ApplyDispatchingStrategies (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel, System.Boolean imguiEventIsInitiallyUsed) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.EventDispatcher.ProcessEvent (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.EventDispatcher.ProcessEventQueue () (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.EventDispatcher.OpenGate () (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.EventDispatcherGate.Dispose () (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.EventDispatcher.ProcessEvent (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.EventDispatcher.Dispatch (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel, UnityEngine.UIElements.DispatchMode dispatchMode) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.BaseVisualElementPanel.SendEvent (UnityEngine.UIElements.EventBase e, UnityEngine.UIElements.DispatchMode dispatchMode) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.UIElementsUtility.DoDispatch (UnityEngine.UIElements.BaseVisualElementPanel panel) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.UIElementsUtility.UnityEngine.UIElements.IUIElementsUtility.ProcessEvent (System.Int32 instanceID, System.IntPtr nativeEventPtr, System.Boolean& eventHandled) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.UIEventRegistration.ProcessEvent (System.Int32 instanceID, System.IntPtr nativeEventPtr) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.UIElements.UIEventRegistration+<>c.<.cctor>b__1_2 (System.Int32 i, System.IntPtr ptr) (at <519636828bf44b44aa4c317f8b947afe>:0)
UnityEngine.GUIUtility.ProcessEvent (System.Int32 instanceID, System.IntPtr nativeEventPtr, System.Boolean& result) (at <931b26b61f444e3485195721b0227b20>:0)

I want to rotate my robote's last three fingers, but give values in range to three fingers' XDrive's target cannot move fingers. Other parents' body can be moved(stiffnes and other param is same), I think may be these fingers don't have child articulation body so I add child body to three finges then it works! it's may be a bug. Hope you can resolve this!

Hi,

I'm trying to launch my robot and subscribe to a joint states topic from ROS. I find that the robot is extremely unstable at times and does not reach the start state of the ROS robot. Other times, this works perfectly. I've uploaded a short gif of a few runs where the first few are fine (https://media.giphy.com/media/ePnzMhBtlsfA5d1pRy/giphy.gif). Sometimes the robot keeps moving to different positions in a rapid manner (https://media.giphy.com/media/Y7bQxNHlhBgVEBozZ0/giphy.gif). Could someone help me troubleshoot the issue?

Thanks!

Hi,

I got Unity-Robotics-Hub updated this morning (got docker folder in \Unity-Robotics-Hub\tutorials\pick_and_place) on my windows10 laptop. then I follow the instructions in tutorial Part 2 -> The ROS Side -> Use Docker Container. I can finish step “2. Build the ROS docker image” without problem. But there is error when I do 3.

Here are the steps I did:

1. cd C:\Users\Public\Documents\UnityProjects2020\RobotSim\Unity-Robotics-Hub\tutorials\pick_and_place
2. docker build -t unity-robotics:pick-and-place -f docker/Dockerfile .
3. docker run -it --rm -p 10000:10000 -p 5005:5005 unity-robotics:pick-and-place part_2 /bin/bash
   then it keeps restarting like this [server_endpoint-2] restarting process
   and the error is
   _/usr/bin/env: ‘python\r’: No such file or directory

The output in the shell window is
C:\Users\Public\Documents\UnityProjects2020\RobotSim\Unity-Robotics-Hub\tutorials\pick_and_place>docker run -it --rm -p 10000:10000 -p 5005:5005 unity-robotics:pick-and-place part_2 /bin/bash
... logging to /root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/roslaunch-2813e20d274d-32.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://2813e20d274d:38871/

SUMMARY

PARAMETERS

• /ROS_IP: 172.17.0.2
• /rosdistro: melodic
• /rosversion: 1.14.10

NODES
/
server_endpoint (niryo_moveit/server_endpoint.py)
trajectory_subscriber (niryo_moveit/trajectory_subscriber.py)

auto-starting new master
process[master]: started with pid [42]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 6baeae64-3fbf-11eb-b080-0242ac110002
process[rosout-1]: started with pid [53]
started core service [/rosout]
process[server_endpoint-2]: started with pid [56]
/usr/bin/env: ‘python\r’: No such file or directory
[server_endpoint-2] process has died [pid 56, exit code 127, cmd /catkin_ws/src/niryo_moveit/scripts/server_endpoint.py --wait __name:=server_endpoint __log:=/root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/server_endpoint-2.log].
log file: /root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/server_endpoint-2*.log
[server_endpoint-2] restarting process
/usr/bin/env: ‘python\r’: No such file or directory
process[trajectory_subscriber-3]: started with pid [61]
process[server_endpoint-2]: started with pid [62]
/usr/bin/env: ‘python\r’: No such file or directory
[server_endpoint-2] process has died [pid 62, exit code 127, cmd /catkin_ws/src/niryo_moveit/scripts/server_endpoint.py --wait __name:=server_endpoint __log:=/root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/server_endpoint-2.log].
log file: /root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/server_endpoint-2*.log
[trajectory_subscriber-3] process has died [pid 61, exit code 127, cmd /catkin_ws/src/niryo_moveit/scripts/trajectory_subscriber.py --wait __name:=trajectory_subscriber __log:=/root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/trajectory_subscriber-3.log].
log file: /root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/trajectory_subscriber-3*.log
[server_endpoint-2] restarting process
process[server_endpoint-2]: started with pid [63]
/usr/bin/env: ‘python\r’: No such file or directory
[server_endpoint-2] process has died [pid 63, exit code 127, cmd /catkin_ws/src/niryo_moveit/scripts/server_endpoint.py --wait __name:=server_endpoint __log:=/root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/server_endpoint-2.log].
log file: /root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/server_endpoint-2*.log
[server_endpoint-2] restarting process
process[server_endpoint-2]: started with pid [64]
/usr/bin/env: ‘python\r’: No such file or directory
[server_endpoint-2] process has died [pid 64, exit code 127, cmd /catkin_ws/src/niryo_moveit/scripts/server_endpoint.py --wait __name:=server_endpoint __log:=/root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/server_endpoint-2.log].
log file: /root/.ros/log/6baeae64-3fbf-11eb-b080-0242ac110002/server_endpoint-2*.log

I stumbled across Iviz recently after moving off of ROS#. Are there plans to incorporate a similar protocol? The nicest part of Iviz is that you don't need to run anything extra on the ROS side of things. I haven't benchmarked any latency differences yet but will likely in the near future.

Hello,
when I'm trying to add the TCP connector package from window/packagemanager and add package from git url , I get the following error:
[Package Manager Window] Error adding package: https://github.com/Unity-Technologies/ROS-TCP-Connector.git.
UnityEditor.EditorApplication:Internal_CallUpdateFunctions () (at /Users/bokken/buildslave/unity/build/Editor/Mono/EditorApplication.cs:327)

I'm using Unity 2020.2.0f1

I've recreated the "Pick-and-Place" demo available on the Unity-Robotics-Hub.
I'm having trouble with the "ROS-Unity Integration".

Execution environment
OS: Ubuntu18.04
Unity: 2020.2.0b9
I have built the ROS environment with docker following the tutorial.

We will explain it in detail below.

This is the part of the ROS TCP connection to be configured in Unity.

In section 9, "Next, the ROS TCP connection needs to be created. Select Robotics -> ROS Settings from the top menu bar.
I recommended the following. I am using Docker, so I entered "127.0.0.1" in ROS IP Address and Override Unity IP Address.

Find the IP address of your ROS machine. In Ubuntu, open a terminal window, and enter hostname -I.

If you are not running ROS services in a Docker container, replace the ROS IP Address value with the IP address of your ROS machine. is set to 10000.

If you are running ROS services in a Docker container, fill ROS IP Address and Override Unity IP Address with the loopback IP address 127.0.0.1, Otherwise, leave the Override Unity IP Address field empty.

However, when I start Unity with the Play button, I get an error message in the console saying "Address already in use" and it stops.

The ROS side confirms that there is no problem.
You will see the following in the terminal.
//Terminal on the ROS side///

[INFO] [1615885069.373331]: Starting server on ///ROS side IP///:10000
ROS-Unity Handshake received, will connect to 127.0.0.1:5005

Then I read Troubleshooting.
The value in src/niryo_moveit/config/params.yaml is correctly the IP address of the ROS side.

I would be very grateful if you could give me some advice.

I am not sure if this is the right place to put this question. I am working on a 3D lidar sensor implementation ...

private void FixedUpdate()
{
    for (int i = 0; i < samples; i++)
    {
        for (int y = 0; y < numberOfLayers; y++)
        {
            float angle = minVerticalAngle + (float)y * vertIncrement;
            directions = transform.rotation * Quaternion.Euler(-angle, azimutIncrAngle * i, 0) * transform.forward;

            raycastHits = new RaycastHit();
            if (Physics.Raycast(transform.position, directions, out raycastHits, rangeMax))
            {
                if (raycastHits.distance >= rangeMin && raycastHits.distance <= rangeMax)
                {
                    ranges = raycastHits.distance;
                    Debug.DrawRay(transform.position, directions * ranges, Color.red);
                }
            }
            else
            {
                Debug.DrawRay(transform.position, directions * rangeMax, Color.green);
            }
        }
    }
}

After saving the points, I want to publish it to a ros node as a pointcloud2 message. I looked at Velodyne Implementation by vwaurich and
GazeboRosVelodyneLaser.cpp but couldn't wrap my head around it! Can someone kindly guide me on this issue ?
Thank you in advance.

hi,
can i use this to combine on MQTT, the mechanism same like MQTT (publisher and subscriber)
and how to do that in here?

hi,

im working on digital twin arm robot
i found this link
https://new.siemens.com/global/en/company/stories/research-technologies/digitaltwin/robotics-simulation.html

thats what i need
https://assets.new.siemens.com/siemens/assets/api/uuid:1f8e967d-ddb6-4705-8ad6-3f9654e8a21d/width:1266/quality:high/martin.gif

are you use base pick and place demo?

how to make prediction like .gif,that awesome
can you give me clue about this
so real robot move, and in unity thats will prediction motion and very acurat movement

thanks
fahrul

Hello,

I am following the pick-and-place tutorial which comprises of 3 parts. I have followed every step and have encountered no problems up until the very last step of the Part 2. When I click on the 'publish' button in the game mode of Unity, I am getting the following exception:

On the ROS side, "roslaunch" command is running but it doesn't print any data :

I have disabled firewall and tried too but no success. Can you please let me know if I am missing something? I apologize if this is a simple issue but I am new to this. Hence I am not sure what else can I do next to figure it out? Thank you.

Additional Information:
Unity - on Windows
ROS - on VM (Ubuntu 18.04)

Hi,

I was trying to publish my robot states using ros connector. It worked properly with revolute joints, however, when I tried to access the states of primatic finger joints, I got this error:

IndexOutOfRangeException: Index was outside the bounds of the array.
UnityEngine.ArticulationReducedSpace.get_Item (System.Int32 i) (at /home/bokken/buildslave/unity/build/Modules/Physics/ScriptBindings/Dynamics.bindings.cs:964)
RosSharp.Urdf.UrdfJointPrismatic.GetPosition () (at Library/PackageCache/com.unity.robotics.urdf-importer@ca9f90f7d3/Runtime/UrdfComponents/UrdfJoints/UrdfJointPrismatic.cs:51)
JointStatePublisher.FixedUpdate () (at Assets/Scripts/JointStatePublisher.cs:102)

Here is how I accessed the states:

message.position[i+numJoints] = UrdfPrismaticJoints[i].GetPosition();  
message.velocity[i+numJoints] = UrdfPrismaticJoints[i].GetVelocity();  
message.effort[i+numJoints] = UrdfPrismaticJoints[i].GetEffort();  

After checking the code on urdf prismatic joints in UrdfJointPrismatic.cs, I found the source of this error:

public override float GetPosition()
{
    #if UNITY_2020_1_OR_NEWER
        return unityJoint.jointPosition[3];
    #else
    return Vector3.Dot(unityJoint.transform.localPosition - unityJoint.connectedAnchor, unityJoint.axis);
    #endif
}

Apperantly the GetPosition is returning the 4th dimension of the joint position, which led to the aforementioned error. Could this by any chance be a bug? Or if it is not, could anybody kindly share some ideas on how this might be resolved? Any thoughts are appreciated. Thanks in advance!

Are there any plans to add UWP support so that we can connect HL2 and ROS using this package?

Thanks!

I am interested in publishing another type of sensor in ROS, however, I am not sure of how to it. For example, I want to add a sensor_msgs/CompressedImage, how can I do it?. I understand the basics of message generation, but I don't get how to convert the camera to the requesting input type.

Hello,I want to add my own robot (ABB IRB1200) using pick-and-place demo,how can I replace the corresponding file?