Comments (12)
Hi ,I was testing RE2 (Pytorch 1.3.1) on quora dataset, and I got results that were aligned with the paper :
Notes :
- I had to use a batch_size=256 as I was facing memory issue with a 1080ti : here is the log of the experiment : https://gist.github.com/jeremypoulain/8ca932a0fc5701578fc4adb127da7adf#file-re2_pytorch_quora_logs-txt
- for the latest versions of Pytorch, you can also modify : mask = torch.matmul(mask_a.float(), mask_b.transpose(1, 2).float()).byte() to mask = torch.matmul(mask_a.float(), mask_b.transpose(1, 2).float()).bool() to get rid of the warning messages.
from simple-effective-text-matching-pytorch.
You should set "eval_file" to "test" (like what "main.json5" does), which evaluates the test set directly in each evaluation during training. This is not what we should do in production, but aligns with previous works and makes the results comparable.
from simple-effective-text-matching-pytorch.
I use "main.json5", and the acc of quora test set can't get 89.2%, only about 84%.
from simple-effective-text-matching-pytorch.
This is the log of training model by using quora.
from simple-effective-text-matching-pytorch.
Did you download the Quora dataset from the link in ReadMe and prepare the data using "prepare_quora.py"?
from simple-effective-text-matching-pytorch.
yes~
from simple-effective-text-matching-pytorch.
Download and unzip Quora dataset (pre-processed by Wang et al.) to data/orig.
cd data && python prepare_quora.py
I did.
from simple-effective-text-matching-pytorch.
Then that's strange... What's your pytorch version? I'll rerun the experiment later
from simple-effective-text-matching-pytorch.
my pytorch version is 1.1.0 .
Other dataset can achieve the score in your paper except the quora dataest.
from simple-effective-text-matching-pytorch.
Hi ,I was testing RE2 (Pytorch 1.3.1) on quora dataset, and I got results that were aligned with the paper :
I'm trying to reproduce the result on Pytorch 1.5.1 and can confirm that the masking issue is gone with the above fix. I, however, ran into another issue, as below:
07/31/2020 01:00:55 train (384348) | dev (10000)
07/31/2020 01:01:00 setup complete: 0:01:36s.
07/31/2020 01:01:00 Epoch: 1
07/31/2020 01:05:56 > epoch 1 updates 3000 loss: 0.2415 lr: 0.0011 gnorm: 0.3525
07/31/2020 01:05:58
Traceback (most recent call last):
File "train.py", line 48, in <module>
main()
File "train.py", line 31, in main
states = trainer.train()
File "code\simple-effective-text-matching-pytorch\src\trainer.py", line 63, in train
self.log.log_eval(dev_stats)
File "code\simple-effective-text-matching-pytorch\src\utils\logger.py", line 90, in log_eval
train_stats_str = ' '.join(f'{key}: ' + self._format_number(val) for key, val in self.train_meters.items())
File "code\simple-effective-text-matching-pytorch\src\utils\logger.py", line 90, in <genexpr>
train_stats_str = ' '.join(f'{key}: ' + self._format_number(val) for key, val in self.train_meters.items())
File "code\simple-effective-text-matching-pytorch\src\utils\logger.py", line 59, in _format_number
return f'{x:.4f}' if float(x) > 1e-3 else f'{x:.4e}'
TypeError: AverageMeter.__float__ returned non-float (type Tensor)
I'm fairly new to Pytorch and still learning the basics but it does seem like 'model.eval()' was changed somehow, in the newer version. Has anyone encountered this issue before and is there a workaround? Thanks in advance!
from simple-effective-text-matching-pytorch.
@TheMnBN
Hi, sorry for the late reply.
In pytorch 1.5, the return type of torch.nn.utils.clip_grad_norm_
has change from "float" to "torch.Tensor". Change line 82 of model.py to
'gnorm': grad_norm.item(),
should solve the problem.
from simple-effective-text-matching-pytorch.
@TheMnBN
Hi, sorry for the late reply.
In pytorch 1.5, the return type oftorch.nn.utils.clip_grad_norm_
has change from "float" to "torch.Tensor". Change line 82 of model.py to'gnorm': grad_norm.item(),should solve the problem.
Hey, Very Nice Work.
I started training this model and it got failed after few hours giving out of memory error.
result = self.forward(*input, **kwargs)
File "/content/simple-effective-text-matching-pytorch/src/modules/fusion.py", line 50, in forward
x2 = self.fusion2(torch.cat([x, x - align], dim=-1))
RuntimeError: CUDA out of memory. Tried to allocate 98.00 MiB (GPU 0; 14.76 GiB total capacity; 12.74 GiB already allocated; 85.75 MiB free; 13.75 GiB reserved in total by PyTorch)
If possible can anyone please provide me the pre-trained model for inference directly?
Thanks in advance!!
from simple-effective-text-matching-pytorch.
Related Issues (15)
- Failure to evaluate HOT 2
- 关于中文 HOT 6
- Abalation error HOT 5
- 关于xlnet中的test.py文件里的spiece.model HOT 1
- pytorch versioin?
- 亲测可用,snli数据集上可复现实验结果 HOT 3
- 这个代码执行顺序是什么?执行入口在哪里?不报错但是没有结果 HOT 6
- Error HOT 4
- About training HOT 1
- How can I solve the problem "RuntimeError: Mask tensor can take 0 and 1 values only at..." with Pytorch 1.2.0 HOT 5
- Problems when training on wikiQA HOT 7
- 数据下载不了 HOT 3
- 这个源码可以在windows环境下运行吗? HOT 2
- 运行训练模型报错 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from simple-effective-text-matching-pytorch.