shayneobrien / coreference-resolution Goto Github PK

Hey,
Is there a problem In utils.py, "def remove_overlapping(sorted_spans):" function in line 98 ?
I think we want to accept "span i" when "si.i1 < sj.i1 <= si.i2 < sj.i2 OR sj.i1 < si.i1 <= sj.i2 < si.i2".
But "if len(set(taken)) == 1 or (taken[0] == taken[-1] == False):" seems do the opposite thing.
For example, if seen = [2, 3, 4, 5, 6], when sj.i1 = 3, sj.2 = 5, then taken = [True, True, True], so len(set(taken)) == 1, it will be appended to the nonoverlapping[] list.

When the training epoches starts to evaluate, it will raise this Runtime Error. Hope anyone can help me to solve it.
The detail of the error message is below:
line 567, in evaluate
predicted_docs = [self.predict(doc) for doc in tqdm(val_corpus) if len(doc) != 0]
line 567, in
predicted_docs = [self.predict(doc) for doc in tqdm(val_corpus) if len(doc) != 0]
line 596, in predict
spans, probs = self.model(doc)
line 1102, in _call_impl
return forward_call(*input, **kwargs)
line 423, in forward
states, embeds = self.encoder(doc)
line 1102, in _call_impl
return forward_call(*input, **kwargs)
line 205, in forward
packed, reorder = pack(embeds)
line 73, in pack
packed = pack_sequence(sorted_tensors)
line 398, in pack_sequence
return pack_padded_sequence(pad_sequence(sequences), lengths, enforce_sorted=enforce_sorted)
line 363, in pad_sequence
return torch._C._nn.pad_sequence(sequences, batch_first, padding_value)
RuntimeError: received an empty list of sequences

During evaluation stage on development dataset, I am facing below error intermittently. Have you ever faced this issue and how did you resolve it?

Traceback (most recent call last):
  File "coref.py", line 693, in <module>
    trainer.train(150)
  File "coref.py", line 459, in train
    self.train_epoch(epoch, *args, **kwargs)
  File "coref.py", line 490, in train_epoch
    corefs_found, total_corefs, corefs_chosen = self.train_doc(doc)
  File "coref.py", line 523, in train_doc
    spans, probs = self.model(document)
  File "/home/rupimanoj/anaconda3/envs/project/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "coref.py", line 424, in forward
    states, embeds = self.encoder(doc)
  File "/home/rupimanoj/anaconda3/envs/project/lib/python3.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "coref.py", line 206, in forward
    packed, reorder = pack(embeds)
  File "/home/rupimanoj/coref/coreference-resolution/src/utils.py", line 74, in pack
    packed = pack_sequence(sorted_tensors)
  File "/home/rupimanoj/anaconda3/envs/project/lib/python3.7/site-packages/torch/nn/utils/rnn.py", line 353, in pack_sequence
    return pack_padded_sequence(pad_sequence(sequences), [v.size(0) for v in sequences])
  File "/home/rupimanoj/anaconda3/envs/project/lib/python3.7/site-packages/torch/nn/utils/rnn.py", line 311, in pad_sequence
    max_size = sequences[0].size()
IndexError: list index out of range

The code "probs = [F.softmax(tensor) for tensor in with_epsilon] " in class Trainer in coref.py.When i train,i get the prob(size:span_size*(antecedent_size+1)*1 ) with all cell has a fixed value 1.Maybe the right code is "probs = [F.softmax(tensor,dim=0) for tensor in with_epsilon]".
My torch version is 1.4.0.
Ps:My english is pool,i hope you can understand what i say.

Evaluating on validation corpus...
217it [12:27, 5.54s/it]Traceback (most recent call last):
File "./src/coref.py", line 690, in
trainer.train(150)
File "./src/coref.py", line 467, in train
results = self.evaluate(self.val_corpus)
File "./src/coref.py", line 566, in evaluate
predicted_docs = [self.predict(doc) for doc in tqdm(val_corpus)]
File "./src/coref.py", line 566, in
predicted_docs = [self.predict(doc) for doc in tqdm(val_corpus)]
File "./src/coref.py", line 595, in predict
spans, probs = self.model(doc)
File "/home/xtan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
tracing_state._traced_module_stack.append(self)
File "./src/coref.py", line 429, in forward
spans, coref_scores = self.score_pairs(spans, g_i, mention_scores)
File "/home/xtan/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
tracing_state._traced_module_stack.append(self)
File "./src/coref.py", line 347, in forward
pairs = torch.cat((i_g, j_g, i_g*j_g, phi), dim=1)
RuntimeError: CUDA error: out of memory

I got this error when evaluating on validation corpus.

Gamma should be 0.999 and step_size=1, so that the learning rate is decayed by 0.1% as recommended in the paper. Otherwise the learning rate is just cut abruptly after 10k steps.

have you test your model on ConLL2012?
and the avg F1 on test dataset?

Coreference indices are not being remapped after document truncation: while tokens are updated, gold coreference spans are not:

Hi, shayneobrien:
I'am Xiangyu from China, I have some quetions about your code. I used your code, but when I am training, the loss has not been descent. I have checked the code several times. The recall ande the precison is almost zero.
Best wish!

First of all, thank you for this pytorch translation.
I (like many others) don't have access to ontonotes dataset. I know you're not authorized to give away the data but can you please share the model weights trained on the dataset?
Thank you

Hi, when I ran coref.py file, I encountered a RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation. I've tried pytorch 0.4.1 in the requirements.txt and pytorch 1.0 but they got same error. Could you please look into this? Thanks!

File "coref.py", line 692, in
trainer.train(150)
File "coref.py", line 458, in train
self.train_epoch(epoch, *args, **kwargs)
File "coref.py", line 488, in train_epoch
corefs_found, total_corefs, corefs_chosen = self.train_doc(doc)
File "coref.py", line 555, in train_doc
loss.backward()
File "/opt/conda/envs/mlkit36/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/opt/conda/envs/mlkit36/lib/python3.6/site-packages/torch/autograd/init.py", line 90, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

Hi,

I have implemented your code with the ontonotes dataset and have found that the model doesn't predict clusters (it predicts that each span is a cluster with itself only). I tried training for 10 and 150 epochs with the same outcome.

After training, I load the trained model and predict (with a simple adaption to the predict function from the Trainer class to also return the clusters variable). I get no outputted clusters for any document in the dataset.

I have attached the terminal output from the training and was wondering if you could tell me why this is.

During implementation, we found the code had a specific bug only when dealing with some documents (a small subset of the overall dataset). To offset this we added a try/except in the train_epoch function such that these documents are not trained on. (and omitted the evaluation step every ten epochs). No other changes were made to the code.

PS. The error we encountered, RuntimeError: split_with_sizes expects split_sizes to sum exactly to 1 (input tensor's size at dimension 0), but got split_sizes=[], is demonstrated at the beginning of the attached file.

It would be great to get some insight into what might be going wrong during training/prediction.

Thanks!

Terminal Saved Output.txt

I think there's a bug around the above code.
The LSTM encoding does not go along each sentence, as

pack_sequence

does not work as you thought.

How to preprocess the data of OntoNotes Release 5.0 ? I can't open the link you gave in README, the website is gone. So, can you give another link or some other way to show how to preprocess the data ?

I downloaded OntoNotes Release 5.0.

and I did e2e-coref's getting started.

I created directories (data/train,data/development,data/test)
and data(output of getting started) are located in directories like data/train/train.english.v4_gold_conll

Did I miss anything or do something wrong?

Thanks.

I got this error while running
python coref.py

loss = torch.sum(torch.log(torch.sum(torch.mul(probs, gold_indexes), dim=1).clamp_(eps, 1-eps), dim=0) * -1)
TypeError: log() got an unexpected keyword argument 'dim'

could you please upload dataset?

Epoch: 150 | Loss: 2649.815816 | Mention recall: 0.054297 | Coref recall: 0.003106 | Coref precision: 0.000000