saiwaiyanyu / bi-lstm-crf-ner-tf2.0 Goto Github PK
View Code? Open in Web Editor NEWNamed Entity Recognition (NER) task using Bi-LSTM-CRF model implemented in Tensorflow 2.0(tensorflow2.0 +)
Named Entity Recognition (NER) task using Bi-LSTM-CRF model implemented in Tensorflow 2.0(tensorflow2.0 +)
hidden_num从512修改为1024后,
报错:
[-INFO-] 2021-01-21 16:53:14,307 20534 train. line:32 hidden_num:1024, vocab_size:4733, label_size:14
Traceback (most recent call last):
File "train.py", line 77, in
loss, logits, text_lens = train_one_step(text_batch, labels_batch)
File "train.py", line 47, in train_one_step
logits, text_lens, log_likelihood = model(text_batch, labels_batch,training=True)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 968, in call
outputs = self.call(cast_inputs, *args, **kwargs)
File "/mnt/python/tensorflow2/bi-lstm-crf-ner-tf2.0/model.py", line 35, in call
logits = self.dense(self.biLSTM(inputs))
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/layers/wrappers.py", line 531, in call
return super(Bidirectional, self).call(inputs, **kwargs)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 964, in call
self._maybe_build(inputs)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 2416, in _maybe_build
self.build(input_shapes) # pylint:disable=not-callable
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/layers/wrappers.py", line 685, in build
self.forward_layer.build(input_shape)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/layers/recurrent.py", line 573, in build
self.cell.build(step_input_shape)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 316, in wrapper
output_shape = fn(instance, input_shape)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/layers/recurrent.py", line 2324, in build
caching_device=default_caching_device)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 577, in add_weight
caching_device=caching_device)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 724, in _add_variable_with_custom_getter
name=name, shape=shape)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 791, in _preload_simple_restoration
checkpoint_position=checkpoint_position, shape=shape)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 75, in init
self.wrapped_value.set_shape(shape)
File "/usr/local/lib64/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1107, in set_shape
(self.shape, shape))
ValueError: Tensor's shape (300, 2048) is not compatible with supplied shape (300, 4096)
这个输出维度,应该是可以自己定义的吧?
这种分类问题,使用precision 和 recall 作为 metrics 会比较好;假如训练数据中绝大多数都是O,那么acc一开始就会很高,不能准确反应模型的能力啊
I noticed that you implemented the training process by your customer functions train_one_step
and evaluate_one_step
. Is there a particular reason that you did not use model.fit
?
ValueError: tf.function-decorated function tried to create variables on non-first call.
I use my own data
[-INFO-] 2020-08-18 17:03:03,229 31422 train.<module> line:80 epoch 22, step 880, loss -102.3357 , accuracy 0.8097
[-INFO-] 2020-08-18 17:03:08,052 31422 train.<module> line:80 epoch 23, step 900, loss -55.5771 , accuracy 0.8884
[-INFO-] 2020-08-18 17:03:12,845 31422 train.<module> line:80 epoch 23, step 920, loss -109.6482 , accuracy 0.8704
[-INFO-] 2020-08-18 17:03:17,668 31422 train.<module> line:80 epoch 24, step 940, loss 6.0771 , accuracy 0.9116
[-INFO-] 2020-08-18 17:03:22,638 31422 train.<module> line:80 epoch 24, step 960, loss -25.0925 , accuracy 0.9362
[-INFO-] 2020-08-18 17:03:27,504 31422 train.<module> line:80 epoch 25, step 980, loss 3.5970 , accuracy 0.9642
[-INFO-] 2020-08-18 17:03:32,433 31422 train.<module> line:80 epoch 25, step 1000, loss -180.6551 , accuracy 0.8611
[-INFO-] 2020-08-18 17:03:37,352 31422 train.<module> line:80 epoch 26, step 1020, loss -185.5950 , accuracy 0.9019
[-INFO-] 2020-08-18 17:03:42,195 31422 train.<module> line:80 epoch 26, step 1040, loss -26.1356 , accuracy 0.9035
[-INFO-] 2020-08-18 17:03:47,002 31422 train.<module> line:80 epoch 27, step 1060, loss -60.3808 , accuracy 0.9388
[-INFO-] 2020-08-18 17:03:52,013 31422 train.<module> line:80 epoch 27, step 1080, loss -368.6034 , accuracy 0.7799
[-INFO-] 2020-08-18 17:03:56,875 31422 train.<module> line:80 epoch 28, step 1100, loss -136.8763 , accuracy 0.8971
[-INFO-] 2020-08-18 17:04:01,815 31422 train.<module> line:80 epoch 28, step 1120, loss 4.8511 , accuracy 0.9637
[-INFO-] 2020-08-18 17:04:06,698 31422 train.<module> line:80 epoch 29, step 1140, loss -145.3242 , accuracy 0.8888
[-INFO-] 2020-08-18 17:04:11,561 31422 train.<module> line:80 epoch 29, step 1160, loss -140.1508 , accuracy 0.9304
[-INFO-] 2020-08-18 17:04:16,441 31422 train.<module> line:80 epoch 30, step 1180, loss -191.4668 , accuracy 0.9232
[-INFO-] 2020-08-18 17:04:21,178 31422 train.<module> line:80 epoch 30, step 1200, loss -193.8787 , accuracy 0.8962
[-INFO-] 2020-08-18 17:04:26,084 31422 train.<module> line:80 epoch 31, step 1220, loss -158.3943 , accuracy 0.9505
[-INFO-] 2020-08-18 17:04:30,918 31422 train.<module> line:80 epoch 31, step 1240, loss -118.8575 , accuracy 0.8672
[-INFO-] 2020-08-18 17:04:35,754 31422 train.<module> line:80 epoch 32, step 1260, loss -367.0381 , accuracy 0.7739
[-INFO-] 2020-08-18 17:04:40,663 31422 train.<module> line:80 epoch 32, step 1280, loss 4.8814 , accuracy 0.9808
[-INFO-] 2020-08-18 17:04:40,716 31422 train.<module> line:84 model saved
[-INFO-] 2020-08-18 17:04:45,581 31422 train.<module> line:80 epoch 33, step 1300, loss -83.2353 , accuracy 0.9721
[-INFO-] 2020-08-18 17:04:50,272 31422 train.<module> line:80 epoch 33, step 1320, loss -386.9019 , accuracy 0.7725
[-INFO-] 2020-08-18 17:04:55,229 31422 train.<module> line:80 epoch 34, step 1340, loss -42.3887 , accuracy 0.9648
[-INFO-] 2020-08-18 17:05:00,131 31422 train.<module> line:80 epoch 34, step 1360, loss 4.1282 , accuracy 0.9734
[-INFO-] 2020-08-18 17:05:05,001 31422 train.<module> line:80 epoch 35, step 1380, loss -139.4714 , accuracy 0.9339
[-INFO-] 2020-08-18 17:05:09,776 31422 train.<module> line:80 epoch 35, step 1400, loss 2.0760 , accuracy 0.9880
[-INFO-] 2020-08-18 17:05:09,833 31422 train.<module> line:84 model saved
[-INFO-] 2020-08-18 17:05:14,682 31422 train.<module> line:80 epoch 36, step 1420, loss -151.9778 , accuracy 0.9138
[-INFO-] 2020-08-18 17:05:19,641 31422 train.<module> line:80 epoch 36, step 1440, loss -94.4556 , accuracy 0.8666
[-INFO-] 2020-08-18 17:05:24,617 31422 train.<module> line:80 epoch 37, step 1460, loss -201.3645 , accuracy 0.9187
[-INFO-] 2020-08-18 17:05:29,423 31422 train.<module> line:80 epoch 37, step 1480, loss -142.1119 , accuracy 0.9106
[-INFO-] 2020-08-18 17:05:34,226 31422 train.<module> line:80 epoch 38, step 1500, loss -349.6097 , accuracy 0.8306
[-INFO-] 2020-08-18 17:05:39,113 31422 train.<module> line:80 epoch 38, step 1520, loss 2.9786 , accuracy 0.9374
总是报这个错,很奇怪
Traceback (most recent call last):
File "/Applications/anaconda3/envs/ner_tf2/lib/python3.6/logging/config.py", line 565, in configure
handler = self.configure_handler(handlers[name])
File "/Applications/anaconda3/envs/ner_tf2/lib/python3.6/logging/config.py", line 738, in configure_handler
result = factory(**kwargs)
File "/Applications/anaconda3/envs/ner_tf2/lib/python3.6/logging/handlers.py", line 202, in init
BaseRotatingHandler.init(self, filename, 'a', encoding, delay)
File "/Applications/anaconda3/envs/ner_tf2/lib/python3.6/logging/handlers.py", line 57, in init
logging.FileHandler.init(self, filename, mode, encoding, delay)
File "/Applications/anaconda3/envs/ner_tf2/lib/python3.6/logging/init.py", line 1032, in init
StreamHandler.init(self, self._open())
File "/Applications/anaconda3/envs/ner_tf2/lib/python3.6/logging/init.py", line 1061, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/shf/PycharmProjects/bi-lstm-crf-ner-tf2.0/log/error.log'
tensorflow:7 out of the last 7 calls to <function _make_execution_function..distributed_function at 0x0000019E6F4A59D8> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings is likely due to passing python objects instead of tensors. Also, tf.function has experimental_relax_shapes=True option that relaxes argument shapes that can avoid unnecessary retracing. Please refer to https://www.tensorflow.org/tutorials/customization/performance#python_or_tensor_args and https://www.tensorflow.org/api_docs/python/tf/function for more details.
但是我使用@tf.function的时候,提示 tnesor 没有numpy
i tried on my environment with tensorflow 2.3 and tensorflow-addons 11.2, encountered exceptions. do you have plan to upgrade the code?
screenshot pasted below:
: Invalid argument: slice index 0 of dimension 0 out of bounds.
Traceback (most recent call last):
File "train.py", line 79, in
accuracy = get_acc_one_step(logits, text_lens, labels_batch)
File "train.py", line 59, in get_acc_one_step
viterbi_path, _ = tf_ad.text.viterbi_decode(logit[:text_len], model.transition_params)
File "C:\Python37\lib\site-packages\tensorflow_addons\text\crf.py", line 362, in viterbi_decode
trellis[0] = score[0]
File "C:\Python37\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
return target(*args, **kwargs)
File "C:\Python37\lib\site-packages\tensorflow\python\ops\array_ops.py", line 1024, in _slice_helper
name=name)
File "C:\Python37\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
return target(*args, **kwargs)
File "C:\Python37\lib\site-packages\tensorflow\python\ops\array_ops.py", line 1196, in strided_slice
shrink_axis_mask=shrink_axis_mask)
File "C:\Python37\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 10320, in strided_slice
_ops.raise_from_not_ok_status(e, name)
File "C:\Python37\lib\site-packages\tensorflow\python\framework\ops.py", line 6843, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: slice index 0 of dimension 0 out of bounds. [Op:StridedSlice] name: strided_slice/
thanks
sanyorke
train.py里面在线评估没用到
predict.py里面离线预测也没用到
得用用吧~
train.py 训练model的时候,text.txt应该用来在线评估的,
但是好像并未使用这个测试集,这是为了加快模型的训练速度吗?
否则转移状态矩阵怎么更新呢
is problem at calculate loss
debugged accuracy ,feeling it's ok
这是输出的日志记录,代码没有动过
input:我是范冰冰 dataset: [[ 131 2067 3239 2234 2234]] logits [[[-0.72535753 4.161972 -2.9507422 -3.1868236 -3.2397275 -3.269389 -3.0774546 -3.2384624 -3.2812405 -2.9897215 -3.1217413 -3.6321018 -3.1315987 -3.0792222 ] [-2.8050935 4.9421215 -2.3537152 -2.844575 -2.8227744 -2.926732 -2.4319122 -2.8924425 -2.7267509 -2.4868493 -2.7142 -3.325798 -2.5980093 -2.7135816 ] [-0.40876862 1.085415 -1.0298109 -1.1841301 -1.3309526 -1.5018826 -1.0938866 -1.1450801 -1.3572048 -1.1883298 -1.3080059 -1.4665012 -1.1049844 -1.3270234 ] [ 0.03211104 0.3558294 -0.6737744 -0.6955142 -0.8401693 -0.9766958 -0.69395465 -0.74898857 -0.8396182 -0.77023965 -0.86173874 -0.88923264 -0.68174887 -0.8035339 ] [ 0.13178386 -0.05146006 -0.43199152 -0.3135421 -0.44383863 -0.5618567 -0.37870246 -0.36361524 -0.4879145 -0.49257016 -0.50506824 -0.50199145 -0.37561214 -0.4440806 ]]] text_lens [5] viterbi_path,va: [1, 1, 1, 1, 1] 15.173229 paths: [[1, 1, 1, 1, 1]] [id2tag[id] for id in paths[0]]: ['O', 'O', 'O', 'O', 'O'] entities_result: [] json_result: []
Traceback (most recent call last):
File "C:/Users/87124/PycharmProjects/bilstm-crf-ner-tf2.0/model.py", line 8, in
import tensorflow_addons as tf_ad
File "C:\Users\87124.conda\envs\tensorflow2.0.0\lib\site-packages\tensorflow_addons_init_.py", line 21, in
from tensorflow_addons import activations
File "C:\Users\87124.conda\envs\tensorflow2.0.0\lib\site-packages\tensorflow_addons\activations_init_.py", line 17, in
from tensorflow_addons.activations.gelu import gelu
File "C:\Users\87124.conda\envs\tensorflow2.0.0\lib\site-packages\tensorflow_addons\activations\gelu.py", line 26, in
@tf.keras.utils.register_keras_serializable(package="Addons")
AttributeError: module 'tensorflow_core.keras.utils' has no attribute 'register_keras_serializable'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.