Comments (6)
Hi, thanks for asking. Due to a bug found in the old implementation, the results reported in the paper can not be directly compared with other papers' result (but the comparison among methods 12/13/14/15 is still valid, since they are evaluated with the same script). The comparable results are updated in the readme file, which is 91.24+/-0.12.
from lm-lstm-crf.
So you mean the reported scores in the paper are indeed not valid caused by a bug in the code?
from lm-lstm-crf.
The reported performance of the POS-tagging is valid, but the reported F1 scores are not precise (pls refer to the reported number in the ReadMe file as more accurate results).
from lm-lstm-crf.
I will modify the ReadMe file and highlight this information.
from lm-lstm-crf.
I see, could you kindly explain the bug causing this issue? I thought using the official scorer to compute the F1 score for NER should be 100% safe, isn't it? I want to make sure that my computed score using the official scorer is accurate. Many thanks!
from lm-lstm-crf.
Yes, the official scorer is accurate.
As to the bug, we re-implemented that script with python for better efficiency (the decode speed of the best model in our recent work is 156K words/sec, https://github.com/LiyuanLucasLiu/LD-Net). However, an elif
is accidentally modified as if
during the implementation refactoring.
from lm-lstm-crf.
Related Issues (20)
- train_w.py Error HOT 2
- How can I include handcrafted features in NER ?
- evaluator.calc_score HOT 3
- where is the dictory './data'
- AttributeError: module 'torch._C' has no attribute '_cuda_setDevice' HOT 3
- KeyError in predictor.py class predict HOT 1
- Question about POS performance HOT 2
- How to deal with sentences with different lengths?
- train_w.py Error, TypeError: can't convert np.ndarray of type numpy.object_ HOT 2
- Is word level bi-lstm reflected in the code ? HOT 1
- About the score given a sequence and a target
- anyone can update to pytorch1.0
- Missing "eval_batch" in train_w.py line 163 HOT 1
- RuntimeError: HOT 1
- dropout
- RuntimeError: expand(torch.LongTensor{[50, 1]}, size=[50]): the number of sizes provided (1) must be greater or equal to the number of dimensions in the tensor (2)
- How do you tune the model to get a large # of keywords outputted by the CRF layer? HOT 1
- Incorrect Precision Output for test_rec HOT 1
- Line 530 in utils.py is too slow with huge datasets HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lm-lstm-crf.