Comments (5)
Hi @petercmh01. Here's an updated version of mixed_reality.py
which supports this. You can call it with the same arguments as usual, but now you can also add --save_correspondences
to your command to save dense_correspondences.pt
in the output folder. You can open it with torch.load("dense_correspondences.pt")
. The tensor has shape (num_frames, num_points, 2)
with continuous (sub-pixel) values usually* in range [0, frame_resolution - 1] that represent the location of each point. The points are arranged in (x, y) format.
I haven't had time to test this updated script particularly thoroughly (but it at least runs :), so let me know if you encounter any issues. It should work on both single-GPU and distributed.
*Actually, the range of values is technically unbounded since some correspondences may lie beyond the boundary of the image. So, e.g., you may sometimes find negative numbers in that tensor.
UPDATE June 29, 2022: This feature has been merged into the main codebase, so I have removed the code snippet in this comment to make this thread more readable.
from gangealing.
Hi @petercmh01. Hmm, I'm pretty sure that the format should be in (x,y). Could it be that the points you're propagating are in (y,x) format, and so converting dense_correspondences.pt
to (y,x) makes it consistent? Also, could you let me know if you ran the script with one versus multiple GPUs?
Regarding the splatting, it should usually be fine to round the predicted points to int format if you don't need sub-pixel accurate correspondences. But, our splatting implementation takes advantage of subpixel information so you'll get higher quality results in general if you use our splat_points
function instead of quantizing to int. You can find documentation for splat_points
in utils/vis_tools/helpers.py
, but feel free to reach out if you have any trouble. Note that it only supports GPU currently (we didn't write a CPU implementation). The way it works under-the-hood is by placing a Gaussian at each (x,y) location, and so the final color assigned to a given quantized pixel is proportional to the distance to each Gaussian.
from gangealing.
Hello: @wpeebles , I tested the code and I was able to get some coordinates, but it seems like the coordinates for each frame is in a wrong order. I was able to get full dense tracking mask but it seems like the mask should be in other frames when I map it.
Can you check if there's a order issue when collect the propagated points?
By the way, the coordinates were all in floats, can I take its int or is there anything else (i.e. interpolation) that I need to aware of? can you roughly explain how the splat was done?
Thank you so much again for consistent help.
from gangealing.
Edit: sorry you're right I was iterating the x and y in a wrong way. The coordination was correct
from gangealing.
The save_correspondences
feature for mixed_reality.py
has now been merged into the main codebase. Thanks for testing it!
from gangealing.
Related Issues (20)
- single color channel HOT 2
- No module named 'fused' HOT 4
- loss going up HOT 3
- RuntimeError: Error building extension '_splat' during visualization HOT 2
- add Gradio Web Demo to cvpr 2022 organization HOT 2
- about StyleGan2 ADA training process HOT 4
- model synthesizes features to fool the loss HOT 10
- RuntimeError in colab HOT 2
- how to train LPIPS(Perceptual Losses) myself HOT 1
- Adding masks to the upper forehead/lower chin HOT 4
- Missing State Dict Keys for CelebA HOT 2
- can't load CUB dataset HOT 2
- Is there anyway that I get the predicted transformation of an image? HOT 7
- failed to creat process
- How can I use dense_correspondences in downstream tasks?
- Permutations of key point indices of CUB and Spair-71k
- ImportError: Cannot compile splatting CUDA library.
- Running Multi-modal spatial transformer models
- PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x00000169BDDD2E00>
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gangealing.