kevinzakka / clip_playground

An ever-growing playground of notebooks showcasing CLIP's impressive zero-shot capabilities

Playground for CLIP-like models

Demo	Colab Link
GradCAM Visualization
Naive Zero-shot Detection
Smarter Zero-shot Detection
Captcha Solver

If you find this playground useful, consider citing it:

@software{zakka2021clipplayground,
    author = {Zakka, Kevin},
    month = {7},
    title = {{A Playground for CLIP-like Models}},
    url = {https://github.com/kevinzakka/clip_playground},
    version = {0.0.1},
    year = {2021}
}

Changelog

2021-07-28

• Better plotting for reCAPTCHA.

2021-07-27

• Allow multiple captions in detection query, colon separated.
• Allow the user to resize an image during selective search.
• Tuned the rejection parameters of selective search.
• Minor bugfix in naive patch detector.

clip_playground's Issues

A related paper : region clip

Hi,
Cool repo and notebooks !

I think you could be interested by https://arxiv.org/abs/2112.09106v1 (region clip), they use basically the same method as you to build a region-concept dataset then use that to train a better model

Could you provide the Grad CAM in CLIP ViT model?

I hope you will provide soon.

Could you provide the Grad CAM in CLIP ViT model?