ottokart / punctuator Goto Github PK
View Code? Open in Web Editor NEWAn LSTM RNN for restoring missing punctuation in unsegmented text.
License: MIT License
An LSTM RNN for restoring missing punctuation in unsegmented text.
License: MIT License
hi,
i tried you system, i have trained successfully but when i try to test it i'm unable to get any result.
what is the command/script to test it.
currently i'm using
python punctuate_text_with_nn_model.py ..\out\model 1 text temp
is this correct?
Hi, i am using punctuator library in my REST API. It works fine for one request.
But, if multiple requests are sent from calling application, then it it giving error:
ERROR (theano.gof.opt): Optimization failure due to: constant_folding
Please help to resolve it
It would be great if a fully working (small) example of training data was provided along with the project.
Do you have any models trained on English data? I'm interested in using this project for my transcription project, but I don't have any English speech data to work with at the moment.
After a couple of epochs, the 2nd phase crashes with the following error:
File "main.py", line 52, in <module>
trainer.train(model_name, PHASE1_TRAIN_PATH, PHASE1_DEV_PATH, PHASE2_TRAIN_PATH, PHASE2_DEV_PATH)
File "punctuator/src/trainer.py", line 157, in train
_train(net, training_data, validation_data, model_name, conf.PHASE2["LEARNING_RATE"], conf.PHASE2["MAX_EPOCHS"], conf.PHASE2["MIN_IMPROVEMENT"])
File "punctuator/src/trainer.py", line 76, in _train
net = utils.load_model("../out/" + model_name)
File "punctuator/src/utils.py", line 31, in load_model
net.load(model)
File "punctuator/src/models.py", line 534, in load
super(TA_LSTM, self).load(model)
File "punctuator/src/models.py", line 44, in load
self.reset_state()
File "punctuator/src/models.py", line 347, in reset_state
self.t_lstm.reset_state()
AttributeError: 'NoneType' object has no attribute 'reset_state'
Hi,
i really appreciate your work, I want to test the script on non punctuated English or German text. So, i guess i have to make a new model with that language.
Can you briefly describe the tasks to implement a new model in a specific language?
Regards,
Josef
I run punctuator with a fresh clone from GitHub. The command is python main.py model
, but I got an error below:
Converting data...
Traceback (most recent call last):
File "main.py", line 40, in
converter.convert_files(conf.PHASE1["TRAIN_DATA"], vocabulary, conf.PUNCTUATIONS, conf.BATCH_SIZE, False, PHASE1_TRAIN_PATH)
File "/home/hudan/Git/punctuator/src/converter.py", line 44, in convert_files
inputs = np.array(inputs, dtype=dtype)[:batch_size*num_batches].reshape((batch_size, num_batches)).T
TypeError: slice indices must be integers or None or have an index method
I tested punctuator using Python 2.7.12 and Numpy 1.14.3.
Please see the following diff that I had to do to make the punctuate code work correctly.
21c21
word_index = utils.input_word_index(net.in_vocabulary, word)
hi,
i tried you system, i have trained successfully but when i try to test it i'm unable to get any result.
what is the command/script to test it.
currently i'm using
python punctuate_text_with_nn_model.py ..\out\model 1 text temp
is this correct?
I think there is a need to allow iteration over previous training, and also to allow multiple threads during training. Overall the process is very slow. What are your thoughts about adapting http://www.rnnlm.org/ to this task. I think the LSTM cell is missing there.
Do we need 1MB, 10MB 100MB of training data ?
what is the exact purpose of the dev set ?
cheers,
In _backpropagate() function,I saw that there 's only dE_dz_y[range(len(output_word_index)), output_word_index] -= 1.
. I have no idea that how you get the CrossEntropy, and is this a trick for backpropagating?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.