yuhongqian / ance-prf Goto Github PK

Hi

When I run the

bash get_ance_embs.sh

This doesn't seem to be working, showing “ 404 The specified resource does not exist. ”

The download link in the script seems to be dead, please update the download link for ANCE-FirstP.
Many thanks.

Have you really checked your code thoroughly? I found a lot of basic mistakes. For example in "train_encoder.sh"，you have not defined "dataset", you used it in "--train_data_dir ${data_dir}/${dataset}_train_prf"

sorry，I cannot find "msmarco_data" file，It is seems that your project has some problems.

File "run_ann_data_gen.py", line 15, in
from msmarco_data import GetProcessingFn

I want to reproduce your works, but the link to checkpoint seems to be empty?

Now, I came to the last step. However, when I run "bash train_encoder.sh", I encounter a problem and I try to solve it but failed. So what should I do?

Traceback (most recent call last):
File "/home/zhangxy/QA/ANCE-PRF-main/main.py", line 115, in
main()
File "/home/zhangxy/QA/ANCE-PRF-main/main.py", line 94, in main
trainer.train()
File "/home/zhangxy/QA/ANCE-PRF-main/runner.py", line 144, in train
batch = next(train_dataloader_iter)
File "/home/zhangxy/anaconda3/envs/ANCE-PRF-ori/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 345, in next
data = self._next_data()
File "/home/zhangxy/anaconda3/envs/ANCE-PRF-ori/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 385, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/home/zhangxy/anaconda3/envs/ANCE-PRF-ori/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 28, in fetch
data.append(next(self.dataset_iter))
File "/home/zhangxy/QA/ANCE-PRF-main/utils/util.py", line 723, in iter
for i, element in enumerate(self.elements):
TypeError: 'Namespace' object is not iterable

Process finished with exit code 1

Hi, thanks again for your excellent work! I'm trying to reproduce your experiment, however, I'm having a lot of problems. When I try to get the prf data, I get the following error. How should I fix it? I ran the code strictly as you instructed in README.md. Looking forward to your reply！

(ANCE-PRF-ori) zhangxy@s2:/QA/ANCE-PRF-main/data_prep$ bash get_prf_data.sh
Generating PRF training data from ANCE top ranking on MARCO training set...
Generated PRF data for all datasets.
(ANCE-PRF-ori) zhangxy@s2:/QA/ANCE-PRF-main/data_prep$ Loaded 2 chunks of embeddings.
Loaded 2 chunks of embeddings.
Traceback (most recent call last):
File "get_prf_data.py", line 165, in
psg_embeds = get_psg_embeds(args, passage_embedding, passage_embedding2id)
File "get_prf_data.py", line 123, in get_psg_embeds
with open(tsv_path, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/zhangxy/QA/ANCE-PRF-main/data/marco_output/prf_train.tsv'

Hi, I guess you missed class 'EmbeddingCache' and 'StreamingDataset' in utils/util.py, which are used in 'preprocess_data.py'
and 'get_ance_embs.py', respectively.