GithubHelp home page GithubHelp logo

zhangyuanhan-ai / visual_prompt_retrieval Goto Github PK

View Code? Open in Web Editor NEW
155.0 155.0 5.0 16.26 MB

[NeurIPS2023] Official implementation and model release of the paper "What Makes Good Examples for Visual In-Context Learning?"

License: Creative Commons Zero v1.0 Universal

Python 99.69% Shell 0.31%
in-context-learning prompt visual-prompting

visual_prompt_retrieval's People

Contributors

zhangyuanhan-ai avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

visual_prompt_retrieval's Issues

Failed to download pre-trained model

The SupPR models for each pascal-5i split is uploaded in link.But the last.pth file corresponding to folder2 and folder3 in this link is invalid and gives an error after downloading, is there a problem here?

The generation of support set in detection

Hello!

I am quite fascinated by your work and have gained significant knowledge from it. However, I am encountering some difficulty with the detection task. It appears that the support pairs have been preloaded. After reviewing the code you've published, I was unable to locate them.

Could you kindly assist me in generating the support set for detection task, similar to the one in the segmentation part?

srun_train_pretrain.sh

SupPR.md mentions the command sh srun_train_pretrain.sh, but I can't find the srun_train_pretrain.sh file, is there a specific training file for that?

The code of detection part

Hello, I am very interested in this job. But I am a little confused about the implementation of the detection part. When will the detection code be released?

Looking forward to your reply.

[Bug] KeyError in get_top50_images_val

Hi, I met a bug in the running supervised prompt retrieval.
sh evaluate/srun_seg_evaluate.sh features_vit-laion2b_trn output_vit-laion2b-clip_trn
image
The step is followed with readme instructions. This bug is raised from loading the Rank50 prompt images calculated by python tools/calculate_similariity.py features_vit-laion2b trn trn. Interestingly, I smoothly run the unsupervised version and generated the segmentation results using the same code.

After comparing these two JSON file of top50-similarity in supervised and unsupervised versions, I notice their keys are different. Did I make mistakes?

Input error

does the input in line 10 - line 12 of tools/calculate_similariity.py mean sys.argv[1], sys.argv[2], sys.argv[3]?

the current code is:

eatures_name = sys.argv[1]
source_split = sys.argv[1]
target_split = sys.argv[2]

Empty iou_dict in get_positive_negative.py

Thanks for your work, I had the same problem:
In the course of debugging get_positive_negative.py, I found that running the:
for img_name in iou_dict: if img_name in iou_dict[img_name]: del iou_dict[img_name][img_name]
Then iou_dict will become empty,like this:
image
So the sorted_iou and the sorted_iou_names is empty.
Are there any steps here that I'm missing?

last.ckpt

Thank you for this inspiring work! I want to know which last.ckpt weight for VQGAN should be used in the Step3 of supervised prompt retrieval?

huggingface_hub.utils._errors.LocalEntryNotFoundError:

When I enter the python tools/featextrater_folderwise_UnsupPR.py vit_large_patch14_clip_224.laion2b features_vit-laion2b trn command, an error is reported:
huggingface_ hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the Local cache. Please check your connection and try again or make sure your Internet connection is on.
The server can't be networked to huggingface to download the pre-training weights, and there is no corresponding url link for the model.
How to solve this problem?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.