26hzhang / neural_sequence_labeling Goto Github PK

Hi is there pretrained models available for punctuation restoration?

tensorflow.python.framework.errors_impl.InvalidArgumentError: All dimensions except 2 must match. Input 1 has shape [20 40 200] and doesn't match input 0 with shape [20 0 300].
how to solve this error

Hi @IsaacChanghau ,

Thanks for your great implementation.
I'm interested in testing the following structure as baseline system >> Words Embeddings + BiLSTM + CRF for POS tagging. I'm not sure what should be updated in the current version of the implementation to have this structure work.
My apology if my question looks silly.
Thanks in advance for your help.

I found you used the dot attention with self-attention, i dont know how to write the concat attention when socre is v * tanh(W[ht; hs]) rather than ht * hs
because i was a beginner in tensorflow . thanks!

I have used the train_conll_ner_blstm_cnn_crf.py for training works and it also gets a good results. But when I tried to do predict works for new files, it runs a bad results. Is there any Eval code in the public codes?

Hi @IsaacChanghau,
Thanks for your great work.
I have an issue when running inference. I restore the checkpoint and predict tag for the input sentence. But the error occurs. Can you help me?

2019-06-04 10:58:58.520794: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
2019-06-04 10:58:58.700894: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at conv_ops.cc:437 : Invalid argument: Computed output size would be negative: -1 [input_size: 3, effective_filter_size: 5, stride: 1]
2019-06-04 10:58:58.700985: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at conv_ops.cc:437 : Invalid argument: Computed output size would be negative: -1 [input_size: 3, effective_filter_size: 5, stride: 1]
Computed output size would be negative: -1 [input_size: 3, effective_filter_size: 5, stride: 1]
	 [[node embeddings/multi_conv1d/conv1d_1/Conv2D (defined at /home/erv-trinn/workspace/keyphrase_extraction/neural_sequence_labeling/models/nns.py:56) ]]
	 [[node project/dense/BiasAdd (defined at /home/erv-trinn/workspace/keyphrase_extraction/neural_sequence_labeling/models/blstm_cnn_crf_model.py:117) ]]

Caused by op 'embeddings/multi_conv1d/conv1d_1/Conv2D', defined at:
  File "predict_text.py", line 106, in <module>
    model = SequenceLabelModel(config)
  File "/home/erv-trinn/workspace/keyphrase_extraction/neural_sequence_labeling/models/blstm_cnn_crf_model.py", line 13, in __init__
    super(SequenceLabelModel, self).__init__(config)
  File "/home/erv-trinn/workspace/keyphrase_extraction/neural_sequence_labeling/models/base_model.py", line 16, in __init__
    self._build_embedding_op()
  File "/home/erv-trinn/workspace/keyphrase_extraction/neural_sequence_labeling/models/blstm_cnn_crf_model.py", line 68, in _build_embedding_op
    drop_rate=self.drop_rate, is_train=self.is_train)
  File "/home/erv-trinn/workspace/keyphrase_extraction/neural_sequence_labeling/models/nns.py", line 69, in multi_conv1d
    scope="conv1d_{}".format(i))
  File "/home/erv-trinn/workspace/keyphrase_extraction/neural_sequence_labeling/models/nns.py", line 56, in conv1d
    xxc = tf.nn.conv2d(in_, filter_, strides, padding) + bias
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1026, in conv2d
    data_format=data_format, dilations=dilations, name=name)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3300, in create_op
    op_def=op_def)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
    self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Computed output size would be negative: -1 [input_size: 3, effective_filter_size: 5, stride: 1]
	 [[node embeddings/multi_conv1d/conv1d_1/Conv2D (defined at /home/erv-trinn/workspace/keyphrase_extraction/neural_sequence_labeling/models/nns.py:56) ]]
	 [[node project/dense/BiasAdd (defined at /home/erv-trinn/workspace/keyphrase_extraction/neural_sequence_labeling/models/blstm_cnn_crf_model.py:117) ]]```

Traceback (most recent call last):
File "train_punct_attentive_model.py", line 86, in
model.train(train_set, valid_data, valid_text, test_texts)
File "/home/dell/桌面/punc/neural_sequence_labeling-master/models/punct_attentive_model.py", line 149, in train
self._add_summary()
File "/home/dell/桌面/punc/neural_sequence_labeling-master/models/base_model.py", line 68, in _add_summary
summary_str = self.sess.run(self.summary)
File "/home/dell/.local/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 967, in run
result = self._run(None, fetches, feed_dict, options_ptr,
File "/home/dell/.local/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 1175, in _run
fetch_handler = _FetchHandler(
File "/home/dell/.local/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 487, in init
self._fetch_mapper = _FetchMapper.for_fetch(fetches)
File "/home/dell/.local/lib/python3.8/site-packages/tensorflow/python/client/session.py", line 264, in for_fetch
raise TypeError('Fetch argument %r has invalid type %r' %
TypeError: Fetch argument None has invalid type <class 'NoneType'>

你好，错误日志如上，貌似是self.summary变量没有定义，请问该如何解决呢？谢谢

hi, I am quite interested in your code on the NER task, especially, for CONLL 2003 dataset, you have stated that your best test F1 is 91.8%, which is awesome. However, it looks weird to me that the valid and test F1 scores are so close by referring to the paper "End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF" and my experiments. Could you check this issue? Thanks a lot!

Epoch 47/100:
703/703 [==============================] - 95s - Global Step: 33041 - Train Loss: 0.0246
Valid dataset -- accuracy: 98.53, precision: 91.54, recall: 92.12, FB1: 91.83
-- new BEST score on valid dataset: 91.83
Test dataset -- accuracy: 98.52, precision: 91.53, recall: 92.11, FB1: 91.82

what is the version of tensorflow you have used?

I want to understand how you used attention in NEr task, any paper or article which explains this? Thanks

直接就跳出来了，也没什么错误提示

Hi, I wonder to know how we can save the model with .pb extensions for serving. Thanks

Thanks for the code, but i am unable to achieve F1 91.82 in ner by using the same config given in repo. Any help?

tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[7,11,0] = 5981 is not in [0, 5980)
[[Node: char_rep_method/char_embeddings = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@char_rep_method/_char_embeddings"], validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](char_rep_method/_char_embeddings/read, _arg_char_ids_0_0)]]
Caused by op 'char_rep_method/char_embeddings', defined at:

First of all than you for your effort. I want to ask if you can help me to apply the attention for NER Task using the idea of paper "Attention-based Recurrent Neural Network for Sequence Labeling" where the author used a window for attention to be helpful for sequence labeling. I want to use this with Normal attention mechanism.

Any help will be appreciated,
Thanks