lopezgg / nn_ner_tensorflow Goto Github PK
View Code? Open in Web Editor NEWImplementing , learning and re implementing "End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF" in Tensorflow
Implementing , learning and re implementing "End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF" in Tensorflow
Thanks for your contribution. I am running the code successfully. I wonder whether the reimplementation of the program is approximate to the original paper result F1=91.21% or not.
[https://github.com/XuezheMax/LasagneNLP]. The Notes.pdf shows four named entity (P,R,F1), not the whole P,R,F1. Look forward to your reply. Many thanks.
In the test_NER.py file, you forget to add the last ''' to the string.
When I run the model on my dataset. The Dev Dataset is a little larger than the conll2003. So in the Evaluation, this error
2018-12-28 12:14:05.446349: W tensorflow/core/framework/allocator.cc:122] Allocation of 8827056000 exceeds 10% of system memory.
came up. Maybe the Evaluation should also be fed batch by batch.
2018-03-15T11:38:40.251583: step 1, loss 68.8194
Traceback (most recent call last):
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1039, in _do_call
return fn(*args)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1021, in _run_fn
status, run_metadata)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/contextlib.py", line 66, in exit
next(self.gen)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: transitions_0/grad/hist
[[Node: transitions_0/grad/hist = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](transitions_0/grad/hist/tag, gradients/AddN_4)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "BasicTextPreprocessing_CNN_CRF.py", line 298, in
embedd_table,char_batch,char_embedd_table)
File "BasicTextPreprocessing_CNN_CRF.py", line 284, in train_step
feed_dict)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 778, in run
run_metadata_ptr)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 982, in _run
feed_dict_string, options, run_metadata)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1032, in _do_run
target_list, options, run_metadata)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1052, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Nan in summary histogram for: transitions_0/grad/hist
[[Node: transitions_0/grad/hist = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](transitions_0/grad/hist/tag, gradients/AddN_4)]]
Caused by op 'transitions_0/grad/hist', defined at:
File "BasicTextPreprocessing_CNN_CRF.py", line 226, in
grad_hist_summary = tf.summary.histogram("{}/grad/hist".format(v.name), g)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/summary/summary.py", line 209, in histogram
tag=scope.rstrip('/'), values=values, name=scope)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/ops/gen_logging_ops.py", line 139, in _histogram_summary
name=name)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
op_def=op_def)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/framework/ops.py", line 1228, in init
self._traceback = _extract_stack()
InvalidArgumentError (see above for traceback): Nan in summary histogram for: transitions_0/grad/hist
[[Node: transitions_0/grad/hist = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](transitions_0/grad/hist/tag, gradients/AddN_4)]]
Exception ignored in: <bound method Session.del of <tensorflow.python.client.session.Session object at 0x7ff256b115f8>>
Traceback (most recent call last):
File "/dhome/jayhsu/miniconda2/envs/ENV/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 587, in del
AttributeError: 'NoneType' object has no attribute 'TF_NewStatus'
Hi, @LopezGG , I run your original code by using the sample data, however, it always has an error "InvalidArgumentError (see above for traceback): Nan in summary histogram for: transitions_0/grad/hist
[[Node: transitions_0/grad/hist = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](transitions_0/grad/hist/tag, gradients/AddN_4)]]"
Can you help me ? I am looking forward to your reply. Thank you very much!
# CalculateMean cross-entropy loss
with tf.name_scope("loss"):
log_likelihood, self.transition_params = tf.contrib.crf.crf_log_likelihood(
self.logits, self.input_y, self.sequence_lengths)
self.loss = tf.reduce_mean(-log_likelihood,name="loss")
==================================================
the parametere "self.transition_params" is nan returned by function tf.contrib.crf.crf_log_likelihood
error details ::
InvalidArgumentError (see above for traceback): Nan in summary histogram for: transitions_0/grad/hist
[[Node: transitions_0/grad/hist = HistogramSummary[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](transitions_0/grad/hist/tag, gradients/AddN_4)]]
......
你好,感谢你的分享,让我有机会下载研究你的代码~
我在运行过程中遇到了问题,梯度值总是nan,只有第一次迭代可以正常显示loss,调整参数也没有解决问题,不知道你是否遇到,或能给我一些建议
另外我没有在项目中找到用于测试的数据,希望能分享下,感谢!
Hi,
I have a problem with the tagging scheme. In your sample data(https://github.com/LopezGG/NN_NER_tensorFlow/tree/master/Sample_Data), the scheme is the BIOES, but the data I find(https://github.com/glample/tagger/tree/master/dataset) is BIO-1 scheme. Where can I find the BIOES version of CoNLL-2003 dataset? Thanks~
pooled = tf.nn.max_pool(
h_expand,
ksize=[1,sequence_length * max_char_per_word,1, 1],
strides=[1, max_char_per_word, 1, 1],
padding='SAME',
name="pooled")
I'm confused about the second dimension of ksize, which covers all characters in one sentence . According to the paper, as what I have understand, the filter of max pooling should cover characters in one word once rather than in the whole sentence. So I think it should be changed to "ksize=[1, max_char_per_word, 1, 1]" and I think the original code would generate same character representation for each word in a sentence (I have tested this part with small examples). I'm not sure if I misunderstand something here, what do you think?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.