Comments (8)
Correct, to fine-tune SAM-PT, you'd finetune PIPS or SAM or both
from sam-pt.
Hej, micro-sam uses the same SAM architecture as the original SAM as it uses SAM directly (see here and here). Thus, it must be possible to load the state directly into an instance of SAM with sam_model.load_state_dict(your_model_state)
.
However, the training checkpoints in micro-sam might be saved in a peculiar way so that they cannot be directly loaded into SAM. What code have you used to save the checkpoint? You might want to try this short snippet out that could convert the micro-sam training checkpoint to a normal SAM checkpoint. I didn't have a training checkpoint to test it with, perhaps you could test if it works well on your side. The cleaned model_state
should look something like odict_keys(['image_encoder.pos_embed', 'image_encoder.patch_embed.proj.weight', ... ])
.
from sam-pt.
On a side note, I'd also suggest you try out different point trackers from the repository for your data. For example, you can try using CoTracker instead of PIPs by adding model/point_tracker=cotracker model.positive_points_per_mask=16
to the command line arguments.
I'd also suggest you try varying the number of iterative refinement iterations to better tune the parameter to your fine-tuned SAM checkpoint. You can do so by adding and varying the model.iterative_refinement_iterations=3
command line argument. Cheers!
from sam-pt.
Thank you for your interest in training on a custom dataset.
We utilize pre-trained checkpoints provided by the respective authors for all point trackers and SAM variants. If you're keen on training or fine-tuning the models on a custom point tracking or segmentation dataset, I'd advise delving into PIPS and SAM/HQ-SAM.
To give you a brief overview:
- PIPS is exclusively trained on the synthetic FlyingThings++, derived from the FlyingThings optical flow dataset.
- SAM was developed using the expansive SA-1B dataset.
- HQ-SAM training relied on HQSeg-44K.
I hope this helps guide your efforts!
from sam-pt.
Thanks for the quick response.
Regarding your suggestion, in order to fine-tune the sam-pt model, I have to fine-tune both PIPS and SAM(or HQ-SAM) and at last load the obtained checkpoints from both models to run sam-pt, did I get your point?
from sam-pt.
Thanks for your answer.
Can you also explain which repository did you use to fine-tune the SAM?
from sam-pt.
Hej @MahdiKoochali, we rely on the original checkpoints provided by the authors and do not perform additional fine-tuning. For SAM, you can download these checkpoints using the instructions provided here. Let us know if you need further assistance!
from sam-pt.
Thank you so much!
I am trying to use SAM-PT on a custom dataset (in the microscopic image domain). Unfortunately, the results were not satisfying in this domain. Regarding your answer earlier, about fine-tuning, I decided to fine-tune SAM.
I used micro-sam and fine-tuned it, but I got the attached error when I wanted to run SAM-PT with my custom checkpoint.
My suspicion is that this error might stem from an incompatibility issue with the SAM models.
It would be perfect if you help me to solve this error or help me to fine-tune SAM.
from sam-pt.
Related Issues (20)
- How to generate a mask file with a demo program HOT 1
- problem when installing requirements.txt HOT 1
- How to use mask sampling for tracing? HOT 3
- Disappearing tracked objects HOT 2
- Confused about the key to enter when running demo or test HOT 1
- AssertionError: No frames found in /content/sam-pt/data/demo_data/bees/
- HQ-SAM predictor issue HOT 2
- Model creation outside repo directory HOT 1
- Add feature points in real time HOT 2
- Tracking ability with a small number of frames HOT 1
- How to get evaluation results for DAVIS2016, YouTube2018 and MOSE? HOT 2
- Error in call to target 'sam_pt.point_tracker.pips.tracker.PipsPointTracker' HOT 2
- Error to load model when running the demo HOT 1
- Cab sam-pt automatically track? HOT 4
- Error running the Non-interactivte Demo HOT 1
- detectron2 fails installation HOT 1
- how much gpu and how much memory are required for training and testing? HOT 1
- Is there any Demo Notebook to test on One Image with Positive, negative and occluded points being passed HOT 1
- Fail to download UVO dense dataset HOT 1
- Different number of positive points
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sam-pt.