Comments (14)
can you post some screenshots of the actor and tags? I can't tell what the issue is unless I see your setup
from dataset_synthesizer.
thanks@marckernest, solved it after putting a tags。
i have another problem, box annotation includes invisible point position in 2d image, how to get invisible key point and set it to 0?
from dataset_synthesizer.
invisible point position
=> which point position is this? How did you set up keypoints for the object?
from dataset_synthesizer.
one box have 8 points,when looking at the box, from any view you can only see 7 points at most, one point will be blocked by itself. invisible points means blocked points in 2d.
from dataset_synthesizer.
I understand your problem now. NDDS doesn't check whether a point is blocked/occluded or not.
1 of our main user is DOPE: https://github.com/NVlabs/Deep_Object_Pose. That network actually quite good at predicting occluded keypoints, whether they are occluded by the object itself or by other objects in the scene.
Does the hidden keypoint causing problem for your network?
from dataset_synthesizer.
Yes,now we are using key points for measuring, network couldn’t predict keypoint very well, or the predicted hidden keypoint(or the point is out of view) confidence coefficient is not high.
from dataset_synthesizer.
Do you have solutions for blocking the hidden points?
from dataset_synthesizer.
Sorry, I don't, the current NDDS doesn't have any solution for that. It's a feature we need to implement and add to the later version.
from dataset_synthesizer.
When are you going to release the later version?
from dataset_synthesizer.
Right now we don't have the exact plan but I will let you know when we do.
For now, I think you can limit the random rotation (e.g: yaw in range (-30, 30) instead of the full 360) of your object so the same corner is behind and in your training code you can always ignore the same one. It's a bad hack but it should work for now.
from dataset_synthesizer.
Now I think more about it, I have a solution for you.
A 3d point is consider occluded if its depth value is larger than the depth value captured in the depth map. This is the basic concept of the occlusion check.
With this idea, you can export the depth of the scene and use the keypoint's projected 2d points as index [x, y] into the depth image to get the depth value at the keypoint location and compare it to the keypoint's Z value (from its location, since we are in OpenCV coordinate system, Z value is the depth value). If the keypoint's Z value is larger than the depth value then it mean it's occluded.
For the depth sensor, you can use:
- RawDepth - it captured the raw depth value and encode the values into RGBA - 32 bits value. This sensor give you the best accuracy but the exported image is really big.
- Depth 16 - it capture the depth value and quantify it: depth_value / MAX_DEPTH and store them in a grayscale 16 bits, to get the real depth value, you can do: depth_image[x, y] * MAX_DEPTH / 65535. This sensor give you better accuracy than the default 8 bits sensor but it's not as good as the RawDepth.
You need to navigate to the Feature Extractors
section of the camera and choose to add feature extractors to see those sensors.
You should try this solution with some simple setup where you can see exactly which corner is occluded. Please let me know if it work or not. In the future version we may add this as a feature of the tool so you don't need to handle it yourself.
from dataset_synthesizer.
got you,I think it could work,thanks
from dataset_synthesizer.
Did this solution work for you?
from dataset_synthesizer.
Please reopen as needed
from dataset_synthesizer.
Related Issues (20)
- _object_settings.json has no data
- How To add NDDS to own project in which the scenes are already created so that it can be used to generate ground truth for persons,objects etc of the created scenes?
- Random background images loading in very low resolution HOT 1
- cannot get right instance segmentation results and instance seg id while "Time Between Scene Capture" > 0 HOT 1
- NDDS cannot capture data while i eject from simulation HOT 1
- Issue with selecting the Focal Target Actor for OrbitalMovement of Camera
- Error when trying to build the plugin HOT 1
- Engine modules are out of date, and cannot be compiled while the engine is ruining. Please build through your IDE.
- Ubuntu Version
- Incorrect units in _object_settings.json
- Random position and random background
- .
- BOP Format HOT 1
- object mask which is not visible
- Unreal engine 5
- Do you have a more detailed field interpretation of the json file?
- Limit spawn randomization HOT 1
- bounding box issue
- Ubuntu Version
- How to achieve background is a random VOC dataset picture?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dataset_synthesizer.