alinlab / masker Goto Github PK

Hello. Thank you for your good thesis and code.

I have a question while comparing paper and code.

I think this code is carried out in a multi-class classification method, is that right? If I misunderstood, I wonder what part of "1-vs-rest" is implemented.

Thanks for your excellent idea and implementation first.
I mentioned that 1 vs rest structure was used for OoD Detection in the paper, but currently it's not included in this repo. Could you please public the part for the community to reprouduce the results? Thank you very much!

I run the code on multiple datasets that used in the paper. But I could not achieve the precision described in the paper. Can you publish the optimal hyperparameters of each experiment?

Hi , I really like and appreciate what you guys are doing.
Is it possible to implement the method on other languages ? etc :Chinese
General-usage of language model was being used in the method , I was curious about the difference between domain-specific finetuned language model and models with MASKER mentioned in the paper and how they affect OOD and performance on downstream tasks. Have you given a try ,yet?
Really looking forward to your response. Thanks.

In readme, there is a command :
python train.py --dataset review --split_ratio 0.25 --seed 0
--train_type base
--backbone bert --classifier_type softmax --optimizer adam_ood \

But in the code, train.py, there is no option adam_ood.

I run the following code：

python train.py --dataset foods --split_ratio 0.25 --seed 0
--train_type base
--backbone bert --classifier_type softmax --optimizer adam_vanilla \

Encountered the following problem：

Loading pre-trained backbone network...
Initializing dataset and model...
Initializing base dataset... (name: foods)
Training model...
Traceback (most recent call last):
File "train.py", line 109, in
main()
File "train.py", line 75, in main
train_base(args, train_loader, model, optimizer, epoch)
File "/home/bigdata11/pzw/MASKER/training/base.py", line 27, in train_base
out_cls = model(tokens) # (B, C)
File "/home/bigdata11/anaconda3/envs/pzw/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/bigdata11/pzw/MASKER/models.py", line 48, in forward
out_p = self.dropout(out_p)
File "/home/bigdata11/anaconda3/envs/pzw/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/bigdata11/anaconda3/envs/pzw/lib/python3.6/site-packages/torch/nn/modules/dropout.py", line 58, in forward
return F.dropout(input, self.p, self.training, self.inplace)
File "/home/bigdata11/anaconda3/envs/pzw/lib/python3.6/site-packages/torch/nn/functional.py", line 983, in dropout
else _VF.dropout(input, p, training))
TypeError: dropout(): argument 'input' (position 1) must be Tensor, not str

can you give a requirements.txt file?
Thanks