aatkinson / deep-named-entity-recognition Goto Github PK

I am writing my diploma thesis, and while on the search for dataset (which I found in several other projects too) I found this project, which helped me tremendously. (So thank you for your work)
But could you tell me the origin of the dataset? (as I am using it too) I cannot seem to find any mention on the internet about it, so I cannot cite it properly in my work.

Hi!

1.) I changed your wordvec.txt to a new one (trained with google wordvec, and 2.5 M tokens, 100 features).
2.) Changed the hardcoded 300 values to 100.
3.) Changed the annotated corpus to a new one with 9576 words in it.
4.) Started ner_train.py
I get massive memory overflow during this row in ner_train.py:
reader = DataUtil(WORDVEC_FILEPATH, TAGGED_NEWS_FILEPATH)
In data_util.py this row generates a lot of memory (after 4000 rows in the raw_data it fills 120 GB of RAM (with swap)):
self.wordvecs = np.vstack((self.wordvecs, new_wv))
Also getting this error when limiting the wordvec's number of rows to 100000 or 1000.
Can you help me?

ValueError: Improper config format: {u'l2': 0.0, u'name': u'WeightRegularizer', u'l1': 0.0}

Any idea what causes this?

Dear author,

The results are fascinating! I am looking forward to see your paper!
plz let me know what's name of the paper if u did release a paper of your work.

BEST wishes,
Young

how did you create the word2vec file?

Using TensorFlow backend.
Traceback (most recent call last):
File "ner_train.py", line 16, in
layer_arg = int(sys.argv[2])
ValueError: invalid literal for int() with base 10: 'news_tagged_data.txt'

When i run python3 ner_train.py wordvecs.txt news_tagged_data.txt