Comments (4)
Hi, thank you for your question. Yes, SAM-PT can track an object that is defined in the initial frame. It outputs segmentation masks for all subsequent frames in the video. For the first frame, the target object is defined using "query" points. Have I understood your question well, I'm not sure what you might have referred to with "automatically"?
from sam-pt.
Hi, thank you so much for sharing SAM-PT and your explanation.
I've experiment with SAM-PT and as @m43 kindly explained, SAM-PT uses the points from the first frame to track the object automatically in all subsequent frames of the video.
Maybe related to @jimmylihui initial question, I was wondering if there is a way to include new query points for subsequent frames in the video while keeping the initial query points. In my case, I had some problems because new objects that were not in the first frame appeared later in the video. Therefore, while the objects from the first frame were being tracked automatically , the new objects from subsequent frames were not.
Again, congratulations on the work and thank you for sharing.
Edited: I shortened the comment to remain within the limits of the initial question
from sam-pt.
Thanks for the question!
Yes, the model supports being passed query points with arbitrary and varying timesteps for the same mask (see here). The inputted query points are defined as a tensor of shape (num_masks, n_points_per_mask, 3)
, with each element denoting the (t, x, y)
(timestep, x location, y location) of the query point. For example, if you track one mask with (1920, 360) at timestep 0 and (960, 720) as timestep 2, then you would have a query points tensor like torch.tensor([ [[0, 1920,360], [2, 960,720]] ])
.
However, this functionality hasn't been utilized in the simple demo where I fixed the query timestep of all points to 0 here for simplicity. Maybe you want to adapt (or contribute) the code in the way necessary for your use case, or perhaps I could update the demo sometime.
from sam-pt.
Thank you very much @m43 for your clear and detailed answer!
I feel it gives relevant information for this thread, particularly regarding the timestep element of the query points and its possibilities.
from sam-pt.
Related Issues (20)
- want to get image mask, for use in bundlesdf project, just run demo line, HOT 3
- (sam) D:\computer_vision\sam-pt>python -m demo.demo \frames_path='sam-pt/data/demo_data/bees/' \query_points_path=null \ longest_side_length=1024 frame_stride=1 max_frames=-1 LexerNoViableAltException: \frames_path='sam-pt/data/demo_data/bees/' ^ See https://hydra.cc/docs/1.2/advanced/override_grammar/basic for details Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace. HOT 3
- An error occurred while running the interactive demo, how to solve it HOT 2
- RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 2.00 GiB total capacity; 1.69 GiB already allocated; 0 bytes free; 1.74 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. HOT 1
- How to generate a mask file with a demo program HOT 1
- problem when installing requirements.txt HOT 1
- How to use mask sampling for tracing? HOT 3
- Disappearing tracked objects HOT 2
- Confused about the key to enter when running demo or test HOT 1
- AssertionError: No frames found in /content/sam-pt/data/demo_data/bees/
- HQ-SAM predictor issue HOT 2
- Model creation outside repo directory HOT 1
- Add feature points in real time HOT 2
- Tracking ability with a small number of frames HOT 1
- How to get evaluation results for DAVIS2016, YouTube2018 and MOSE? HOT 2
- Error in call to target 'sam_pt.point_tracker.pips.tracker.PipsPointTracker' HOT 2
- Error to load model when running the demo HOT 1
- Error running the Non-interactivte Demo HOT 1
- detectron2 fails installation HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sam-pt.