Comments (10)
Hi @pianita ,
What version of the TCP Connector are you using? You can find this in the Package Manager window or in the package
file in the ROS-TCP-Connector directory.
Aside from the Exceptions, I ran into a similar issue a while ago where only the top half of an image was being applied to the texture. I believe I resolved the issue by setting the width and height of the image game object to match the resolution of the image coming from ROS. If you don't know the resolution of the images coming from ROS you should be able to find it by doing something like math.sqrt(len(CompressedImage.data) / 4)
in Python. The 4 is for the RGBA values of the image.
Would you be able to share a simple project that reproduces this issue so that I can take a look?
from unity-robotics-hub.
Hi @mpinol,
The ROS TCP Connector version is 0.0.3.
I have to display the videostream on a non-image game object and do so by applying a 2Dtexture.
I attached a ROS workspace (rosrun server_endpoint.py, load params.yaml, rosrun image_publisher.py) and for the Unity Project I used the PickandPlace Project with a new scene and the attached Subscriber Script.
catkin_ws (copy).zip
CameraSubscriber.zip
Thanks for your reply! :)
from unity-robotics-hub.
from unity-robotics-hub.
Hey @pianita
Thank you for the files!
I tested everything using ROS TCP Connector version 0.1.2-preview and was able to get the full image displayed on a cube. Version 1.2 includes a variety of improvements and bug fixes that might resolve your issue.
Would you be able to update the TCP Connector and try again?
from unity-robotics-hub.
Hey @mpinol
I tried today with the TCP Connector in version 0.1.2.
If I publish the single compressed image in a loop it gets displayed correctly. But if I publish it once every time I run my image_publisher.py node I still get only half images sometimes. If instead of a single image I publish a stream of different compressed images (the video stream) I also still get half images. If I look at the content of the received message I can see that the lower half of the byte array are 0.
Do you get the full image even in a stream of e.g. two different images?
from unity-robotics-hub.
Hi @pianita, I am able to get full images when toggling between two different images in a stream with the TCP Connector version mentioned, where the two images around the size of the provided jpeg. Would you mind sharing an updated version of your Unity project and ROS workspace so we can take another look?
from unity-robotics-hub.
Hi @at669!
Sorry my mistake - toggling images works fine. I am figuring it might really just be the resolution. If I print math.sqrt(len(CompressedImage.data) / 4) of the .jpeg image I get values around 63. For the images of the video stream I get more than double that value.
Do you know until which resolution this should work? I want of course highest quality but also the full image :D.
Thank you!
from unity-robotics-hub.
Hi @pianita--sorry for the delay! After some investigation, it seems like more of a specific bottleneck issue, and we're in the process of making some fixes for it that will be added to the repositories soon. In the meantime, feel free to implement them on your end:
- In
ROS-TCP-Endpoint/src/ros_tcp_endpoint/tcp_sender.py
'ssend_unity_message()
, we increaseds.settimeout
to 10. The original value would cause larger images to time out on send. - Also in
ROS-TCP-Endpoint/src/ros_tcp_endpoint/tcp_sender.py
'ssend_unity_message()
, we changed the call froms.send(serialized_message)
tos.sendall(serialized_message)
. Usingsendall
ensures that the full message will try to be sent. - In the Unity
ROS-TCP-Connector
package'sROSConnection.cs
(located in Runtime/TcpConnector), inReadMessageContents
, we changedwhile (networkStream.DataAvailable && bytesRemaining > 0)
to justwhile (bytesRemaining > 0)
. It appears thatnetworkStream.DataAvailable
can be an unreliable verification for the TCP connection. Note of caution on this point: if the full message is not being sent from ROS, thiswhile
loop will hang indefinitely and will need to be force quit. You can implement a timeout on this end as well, or ensure the message is always being fully sent from ROS. We are working on a fix for this. - Also in the Unity
ROS-TCP-Connector
package'sROSConnection.cs
ReadMessageContents
, we attempt to read the incoming data in chunks instead of as a whole, changing thebytesRead
line toint bytesRead = networkStream.Read(readBuffer, totalBytesRead, Math.Min(2048, bytesRemaining));
.
You may need to test the exact timeouts and resolutions for your use case, depending on the rate of your stream and resolution of your images. Hopefully these points can help with your project! Let us know if these steps resolve this issue.
from unity-robotics-hub.
Hey @at669,
yes I get the full image now! Thank you so much :)
The videostream is still very jerky but I will do some testing by adjusting resolution, publish rate of my video and timeouts.
Of course if you have a setup you know runs smoothly I would be interested to know the values.
But for me this issue is solved and can be closed.
from unity-robotics-hub.
Glad these points were able to help! Using jpegs that translated to a serialized message of below ~250k length byte array seemed to be working relatively well for my simple test case with the referenced PR changes to my Connector and Endpoint, but definitely feel free to test higher or lower values if they work for you.
I will close this issue, but feel free to re-open if you have any follow-ups. Thanks!
from unity-robotics-hub.
Related Issues (20)
- Run roslaunch niryo_moveit part_2.launch but process has died [pid 10010, exit code 127 HOT 4
- Reporting a vulnerability
- More than one message was registered as "sensor_msgs/Image"
- WebGL build HOT 1
- ROS Connection script doesn't work
- Extraction of MessageQueue full via variable Bug in msg getting duplicated.
- Unity doesn't show point cloud
- Project lost when updating Unity Hub
- ERROR: failed to solve: process "/bin/sh -c sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys HOT 2
- I want to use unity to connect to multiple ROS systems, but it doesn't seem to work HOT 3
- Lidar Scan Visualization Inconsistent with Gazebo HOT 1
- unity_robotics_demo_msgs failing for colcon build HOT 3
- "unknown message class 'unity_robotics_demo_msgs/PosRot'"
- failed to deserialize ROS message: Fast CDR exception deserializing message of type std_srvs::srv::dds_::Empty_Response_.
- Continuously connecting and disconnecting while throwing exceptions on server side HOT 1
- RosConnection.Update() spiking to >180ms execution. This thread-blocking causes bad frame spikes when Playing. HOT 2
- error creating docker file:
- ROSConnection.cs:854
- Communicate with the URDF robot model without using ROS
- import robot's urdf without color
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unity-robotics-hub.