Comments (5)
Hi Jason,
Can you share the command you used?
I tried the following and it gave me no errors
python -m autoprompt.create_trigger --train data/sentiment_analysis/train.tsv --dev data/sentiment_analysis/dev.tsv --template '<s> {sentence} [T] [T] [T] [P] . </s>' --num-cand 10 --accumulation-steps 1 --model-name roberta-base --bsz 8 --eval-size 8 --iters 1 --label-map '{"0": ["Ġworse", "Ġincompetence", "ĠWorse", "Ġblamed", "Ġsucked"], "1": ["ĠCris", "Ġmarvelous", "Ġphilanthrop", "Ġvisionary", "Ġwonderful"]}'
from autoprompt.
Thanks a lot for you replying me about my questions, I'll try the command that you provided later! Thank you again!
from autoprompt.
Dear Taylorshin,
After using the command you provided, the "create_trigger. py" file can run normally. Thank you again for your reply to my question! By the way, after running "create_trigger. py", how can I evaluate the trigger words obtained from "create_trigger. py" for sentiment analysis task? Wishing for your reply!
from autoprompt.
To evaluate the trigger, you would do the following
- Append the trigger tokens to each sentence from the test set
- Pass in each prompt into the model of your choice
- Get the probabilities for each class label by marginalizing over the set of associated label tokens (i.e. label map argument from the command I posted above)
- Compute the average accuracy across the test set
Figure 1 of the paper has a good explanation of the process.
from autoprompt.
OK, thank you a lot to reply for me, I'll try to evluate the trigger by following your instructions, thanks a lot!
from autoprompt.
Related Issues (20)
- Question on using this in conjunction with CLIP / Open_CLIP / VQGAN HOT 3
- How to run the code --queries HOT 1
- Two arguments missed in label_search.py HOT 1
- where is scripts/run_experiments.py HOT 2
- About accuracy HOT 2
- "Gradient-Based Prompt Search"&" Automating Label Token Selection".which one is the first step? HOT 1
- Label Search of NLI HOT 2
- I got different triggers with the one in fact_retrieval_bert_prompts.jsonl HOT 1
- How to run the code HOT 3
- Query about the function "encode_label" in autoprompt/utils.py HOT 4
- Is there the code for downstream task in this project? HOT 2
- AttributeError: 'str' object has no attribute 'masked_select' HOT 2
- A question for the meaning of letter 'Ġ' HOT 1
- where are the found prompts? HOT 1
- What is the T in your label search algorithm? HOT 2
- Question about code running on SNLI dataset
- Zero value for evaluation_fn in create_trigger.py
- 请问自己生成的的提示在哪里,我运行完prompt的程序也没见到自己生成的提示啊?
- Question about proper settings of prompt extraction attack.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from autoprompt.