Comments (6)
I will suggest you use some tools for Chinese word segmentation and then need to do dependency parsing. I think it shouldn’t be difficult to convert those information into conll format. You can simply write them to a file.
from ner_with_dependency.
In fact , I asked same question weeks ago and I found that it is not easy to transform a Chinese sentence to conll-x format. Could you provide some examples that how to transform Chinese sentences to conll-x format? Appreciate.
from ner_with_dependency.
I'm not sure what difficulties you are facing now. Would you like to tell me that?
Basically, you are going to do the following steps.
- Chinese word segmentation. (Some tools can help you do that, such as Jieba segmentation or Stanza by Stanford)
- Dependency parsing (You can also use Stanza by Stanford)
Would you like to let me know which step you are in trouble?
from ner_with_dependency.
Dependency parse need to tokenize first and tokenizer result may be wrong.
for example
请:O 帮:O 我:O 打开:O 推特:S-App(means open the twitter for me)and slot is
the correct result is
请\帮\我\打开\推特
Maybe the tokenizer result is
请\帮\我\打\开推特
and the Dependency parse will be wrong.
If you use the correct word segmentation(you can get it by slot), Dependency parse can not use custom tokenizer.
Although stanza claims that you can use pretagged document but upos, xpos, feats, lemma need tokenize first.
from ner_with_dependency.
Yeah. I guess you are right and this is the painful part we need to deal with.
Given a Chinese sentence, I think the model might not be able to get the "correct" segmentation unless we annotate them. Thus, using an external tokenizer/segmenter should be the only choice. I would suggest you use the Jieba segmentation tool. Then pass the sentence to Stanza for dependency parsing.
from ner_with_dependency.
Thx.
from ner_with_dependency.
Related Issues (10)
- Convert the constituency trees into the Stanford dependency trees HOT 7
- help ,i don't know how to use your code HOT 4
- Potential bugs when running "dggcn" mode HOT 4
- Is the F1 value in your paper the best result? HOT 6
- conllU
- How to use Chinese data HOT 6
- label not found error HOT 2
- A question about reproduction HOT 15
- Do you have conll2003 datasets with denpendency? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ner_with_dependency.