Comments (2)
Thanks for your interest! We have not planned a PyTorch release for CLIPPO.
If you're interested in using the pretrained checkpoints with PyTorch, it shouldn't be too hard to convert them, as they are stored as npz files and we follow the standard ViT design. If you're interested in training in pytorch, you could just port the preprocessing function to your favorite CLIP library and adapt the code to do two forward passes through the vision encoder (one for the natural image and one for the text image).
from big_vision.
Thank you for the detailed explanation!
OK, I will do the conversion by myself. 😄
from big_vision.
Related Issues (20)
- Error with putting arrays on CPU in cloud TPUs HOT 1
- How to save fine tuned PaliGemma model? HOT 3
- [QUESTION] How to perform inference on trained model? HOT 8
- Can I convert paligemma npz model to pytorch to safetensors? HOT 3
- Question about SigLIP HOT 7
- Memory Efficient Attention integration HOT 2
- Question about ViT-augreg ("How to train?") fine-tuning transfer HOT 2
- Contrastive Input Pipeline HOT 2
- [BUG] in big_vision.models.proj.flexi.vit HOT 1
- Reproduced result for flexivit HOT 2
- SigLIP and canonicalize
- Text lowering issue
- Negative rho values in GSAM training HOT 1
- FlexiVit is also flexible with image resolution?
- Load ViT with CLIPPO Weights HOT 2
- Mixup Per Example?
- Clarification: SigLIP Image Transform
- Errors in notebooks
- Confusion on FlexiViT
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from big_vision.