bshao001 / chatlearner Goto Github PK

so training is done

# Finished epoch 60 @ step 37980 @ 2017-11-15 20:22:16. In the epoch, learning rate = 0.000096, mean loss = 0.3683, perplexity = 1.4453, and 552.35 seconds elapsed.

the evaluate part creating some reports, thanks for any help

flyn@tron:~/git/ChatLearner/chatbot$ python3.5 botui.py 
# Prepare dataset placeholder and hyper parameters ...
# Loading hparams from /home/flyn/git/ChatLearner/Data/Corpus/hparams.json ...
# Creating inference model ...
# Building graph for the model ...
# Restoring model weights ...
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
    status, run_metadata)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /home/flyn/git/ChatLearner/Data/Result/basic
	 [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
	 [[Node: save/RestoreV2_4/_17 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_64_save/RestoreV2_4", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "botui.py", line 51, in <module>
    bot_ui()
  File "botui.py", line 32, in bot_ui
    result_dir=res_dir, result_file='basic')
  File "/home/flyn/git/ChatLearner/chatbot/botpredictor.py", line 48, in __init__
    self.model.saver.restore(session, os.path.join(result_dir, result_file))
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1666, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 889, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
    options, run_metadata)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /home/flyn/git/ChatLearner/Data/Result/basic
	 [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
	 [[Node: save/RestoreV2_4/_17 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_64_save/RestoreV2_4", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Caused by op 'save/RestoreV2', defined at:
  File "botui.py", line 51, in <module>
    bot_ui()
  File "botui.py", line 32, in bot_ui
    result_dir=res_dir, result_file='basic')
  File "/home/flyn/git/ChatLearner/chatbot/botpredictor.py", line 45, in __init__
    batch_input=self.infer_batch)
  File "/home/flyn/git/ChatLearner/chatbot/modelcreator.py", line 103, in __init__
    self.saver = tf.train.Saver(tf.global_variables())
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1218, in __init__
    self.build()
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1227, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1263, in _build
    build_save=build_save, build_restore=build_restore)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 751, in _build_internal
    restore_sequentially, reshape)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 427, in _AddRestoreOps
    tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 267, in restore_op
    [spec.tensor.dtype])[0])
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1021, in restore_v2
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
    op_def=op_def)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1470, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

NotFoundError (see above for traceback): Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /home/flyn/git/ChatLearner/Data/Result/basic
	 [[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
	 [[Node: save/RestoreV2_4/_17 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_64_save/RestoreV2_4", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

that line makes me a little nervous

...Failed to find any matching files for /home/flyn/git/ChatLearner/Data/Result/basic

Hi!
I've been trying to run this code but I keep getting an error as follows when i run bottrainer.py:

......
/job:localhost/replica:0/task:0/device:CPU:0
dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/cell_0/gru_cell/concat_1/axis: (Const): /job:localhost/replica:0/task:0/device:CPU:0
dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/cell_0/gru_cell/split/split_dim: (Const): /job:localhost/replica:0/task:0/device:CPU:0
dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/cell_0/gru_cell/Const: (Const): /job:localhost/replica:0/task:0/device:CPU:0
dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/cell_0/gru_cell/concat/axis: (Const): /job:localhost/replica:0/task:0/device:CPU:0
dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/dropout/random_uniform/max: (Const): /job:localhost/replica:0/task:0/device:CPU:0
dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/dropout/random_uniform/min: (Const): /job:localhost/replica:0/task:0/device:CPU:0
dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/dropout/keep_prob: (Const): /job:localhost/replica:0/task:0/device:CPU:0
dynamic_seq2seq/decoder/LuongAttention/concat/axis: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Adam/value: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_16: (Const): /job:localhost/replica:0/task:0/device:CPU:0
num_parallel_calls_1: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_15: (Const): /job:localhost/replica:0/task:0/device:CPU:0
num_parallel_calls: (Const): /job:localhost/replica:0/task:0/device:CPU:0
seed2: (Const): /job:localhost/replica:0/task:0/device:CPU:0
seed: (Const): /job:localhost/replica:0/task:0/device:CPU:0
buffer_size_3: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_14: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_13: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_12: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_11: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_10: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_9: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_8: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_7: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_6: (Const): /job:localhost/replica:0/task:0/device:CPU:0
buffer_size_2: (Const): /job:localhost/replica:0/task:0/device:CPU:0
compression_type_2: (Const): /job:localhost/replica:0/task:0/device:CPU:0
filenames_2: (Const): /job:localhost/replica:0/task:0/device:CPU:0
count_1: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_5: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_4: (Const): /job:localhost/replica:0/task:0/device:CPU:0
buffer_size_1: (Const): /job:localhost/replica:0/task:0/device:CPU:0
compression_type_1: (Const): /job:localhost/replica:0/task:0/device:CPU:0
filenames_1: (Const): /job:localhost/replica:0/task:0/device:CPU:0
count: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_3: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const_2: (Const): /job:localhost/replica:0/task:0/device:CPU:0
buffer_size: (Const): /job:localhost/replica:0/task:0/device:CPU:0
compression_type: (Const): /job:localhost/replica:0/task:0/device:CPU:0
filenames: (Const): /job:localhost/replica:0/task:0/device:CPU:0
hash_table/Const: (Const): /job:localhost/replica:0/task:0/device:CPU:0
hash_table: (HashTableV2): /job:localhost/replica:0/task:0/device:CPU:0
Const_1: (Const): /job:localhost/replica:0/task:0/device:CPU:0
Const: (Const): /job:localhost/replica:0/task:0/device:CPU:0
string_to_index/hash_table/table_init/asset_filepath: (Const): /job:localhost/replica:0/task:0/device:CPU:0
string_to_index/hash_table/Const: (Const): /job:localhost/replica:0/task:0/device:CPU:0
string_to_index/hash_table: (HashTableV2): /job:localhost/replica:0/task:0/device:CPU:0
Segmentation fault

Any idea what might be causing this?

I'll be really grateful for any help.

Thanks

Excuse me，when I trianing use my custom chinese dialog data,it raised the following error,
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [All values in memory_sequence_length must greater than zero.] [Condition x > 0 did not hold element-wise:] [x (IteratorGetNext:3) = ] [8 9 8...],
which part may I make mistake?

In the joke corpus:
Q: Do you know the story of Snow White ?
A: Yes . Here is the story : _func_val_get_story_name_para1_snow_white

There are million of stories in the world, I guess we can not put all of those into corpus. Could we amend _func_val_get_story_name, so every time it is called, the function will lookup the user input and use NLP to extract the story name and search from database or else

I read the code in functiondata.py, but how do you map user input into corresponding function like get_date_time?

I am gald to see this fantastic repository.
I encounter a problem, this link (https://drive.google.com/file/d/1mVWFScBHFeA7oVxQzWb8QbKfTi3TToUr/view?usp=sharing) is not available now, so counld you send your trained model to [email protected], Thanks a lot!

no module named "chatbot"

Hi, @bshao001 your work has been best, and have encourage me to learn more.
Can you please tell me how to save and load the pervious chat session.
For example:
When I chat with bot and provide my name it will remember name through out the session but once I exit the session it will forget. Is there a way that I can save the chat session and load it back or resume chatting from the previous point .
Thanks before

I'm getting this error..
File "D:/Python Chatbot/ChatLearner-master/chatbot/botui.py", line 20, in <module> from chatbot.botpredictor import BotPredictor ModuleNotFoundError: No module named 'chatbot'

First, awesome program but just to see how it works I only trained for 16 epochs and my perplexity only got to 15 or so but in my "results" folder I didn't see a "basic" file just the training logs.

Also, have you experimented with adding more layers, if so, did you observe an improvement in the quality of the conversations or perhaps in increased ability for the program to abstract certain aspects of language?

No defense but your code are such mass. First when I run cd chatbot && python3 bottrainner.py it says:

Traceback (most recent call last):
  File "bottrainer.py", line 20, in <module>
    from chatbot.tokenizeddata import TokenizedData
ModuleNotFoundError: No module named 'chatbot'

So I run python3 chatbot/bottrainer.py then it says:

self.text_set = self.text_set.concatenate(src_tgt_dataset)
AttributeError: 'RepeatDataset' object has no attribute 'concatenate'

obviously, this is totally nonsense line of:

self.text_set = self.text_set.concatenate(src_tgt_dataset)

I uncomment the concatenate line, and it goes this error:

Traceback (most recent call last):
  File "chatbot/bottrainer.py", line 142, in <module>
    bt = BotTrainer(corpus_dir=corp_dir)
  File "chatbot/bottrainer.py", line 30, in __init__
    tokenized_data = TokenizedData(corpus_dir=corpus_dir)
  File "/Volumes/xs/CodeSpace/ng/ai/experiment/ChatLearner/chatbot/tokenizeddata.py", line 72, in __init__
    self._convert_to_tokens(buffer_size)
  File "/Volumes/xs/CodeSpace/ng/ai/experiment/ChatLearner/chatbot/tokenizeddata.py", line 249, in _convert_to_tokens
    output_buffer_size=buffer_size)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/contrib/data/python/ops/dataset_ops.py", line 813, in map
    return MapDataset(self, map_func, num_threads, output_buffer_size)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/contrib/data/python/ops/dataset_ops.py", line 1436, in __init__
    self._map_func.add_to_graph(ops.get_default_graph())
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/function.py", line 619, in add_to_graph
    self._create_definition_if_needed()
  File "/usr/local/lib/python3.6/site-packages/tensorflow/contrib/data/python/framework/function.py", line 167, in _create_definition_if_needed
    outputs = self._func(*inputs)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/contrib/data/python/ops/dataset_ops.py", line 1422, in tf_map_func
    ret = map_func(*nested_args)
  File "/Volumes/xs/CodeSpace/ng/ai/experiment/ChatLearner/chatbot/tokenizeddata.py", line 248, in <lambda>
    (self.case_table.lookup(src), self.case_table.lookup(tgt)),
  File "/usr/local/lib/python3.6/site-packages/tensorflow/contrib/lookup/lookup_ops.py", line 186, in lookup
    self._table_ref, key_tensor, self._default_value, name=scope)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/ops/gen_lookup_ops.py", line 288, in _lookup_table_find
    name=name)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/contrib/data/python/framework/function.py", line 81, in create_op
    data_types, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/function.py", line 361, in create_op
    **kwargs)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/usr/local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1250, in __init__
    input_types))
TypeError: In op 'hash_table_Lookup', input types ([tf.string, tf.string, tf.string]) are not compatible with expected types ([tf.string_ref, tf.string, tf.string])

I have no idea how you write your code, but this is really not worth to waste time on this buggy project.

I'm using GeForce GTX 660, and I get the error even on batch size 1.

Hey,
can you please tell me where the embeddings happens in code, or are you using trained embeddings?
thx

If I changed any line of a_papaya_new.txt Should, i also should changed in vocab.txt.
I want to slighlty modified the conversion in my way where the bot can interact.
Please mention how to make the new rule and make possible to change contents of that file before training.
Let's I change; Dialogue of a_papaya_new.txt as,

Q: What is your name? ----> Q: What is your name?
A: My name is Papaya. ----->A: My name is AI-Chatbot

and several others.
> I am Strange by knowing how every word are linked to vocab.txt and Tensorflow finds out the pattern between the input and trained datas and it worked.
Also, How can I papaya, Corpus, Reddit data-set whole used to train my bot.
I deep look on it and not found any ideas on it.
Please ! clarify me.
Thanks in advance.

I’ve been trying to run bottrainer.py, but whenever i try i get this error:

Traceback (most recent call last):
File "chatbot/bottrainer.py", line 20, in
from chatbot.tokenizeddata import TokenizedData
ImportError: No module named chatbot.tokenizeddata

Any ideas?

@bshao001
Hey,
i am currently trying to extract names from input, here my progress.
two different ways, first with nltk then with the stanfordPOSTTagger

text = "My name is Ari the great One together with James Uppercase and john lower case."


import nltk
for sent in nltk.sent_tokenize(text):
    for chunk in nltk.ne_chunk(nltk.pos_tag(nltk.word_tokenize(sent))):
        print chunk
         

from nltk.tag.stanford import CoreNLPPOSTagger, CoreNLPNERTagger
from nltk.tokenize.stanford import CoreNLPTokenizer
stpos, stner = CoreNLPPOSTagger('http://localhost:9000'), CoreNLPNERTagger('http://localhost:9000')
sttok = CoreNLPTokenizer('http://localhost:9000')

print "nltk stanford"
for i in stner.tag(sttok.tokenize(text)):
    print i

result

('My', 'PRP$')
('name', 'NN')
('is', 'VBZ')
('Ari', 'NNP')
('the', 'DT')
('great', 'JJ')
('One', 'CD')
('together', 'RB')
('with', 'IN')
(PERSON James/NNP Uppercase/NNP)
('and', 'CC')
('john', 'VB')
('lower', 'JJR')
('case', 'NN')
('.', '.')
nltk stanford
(u'My', u'O')
(u'name', u'O')
(u'is', u'O')
(u'Ari', u'PERSON')
(u'the', u'PERSON')
(u'great', u'PERSON')
(u'One', u'PERSON')
(u'together', u'O')
(u'with', u'O')
(u'James', u'PERSON')
(u'Uppercase', u'PERSON')
(u'and', u'O')
(u'john', u'O')
(u'lower', u'O')
(u'case', u'O')
(u'.', u'O')

the title stuff like "the great One" is working great but the the lower case still not recognized and that is annoying, maybe Papaya should ask the name(as a work around), you told me that you already tried, can you tell me your workflow please...

hi,
I want to have a simple chat-bot.
I have a bunch of question and answers and I will train the bot on the data.
can I create a data set like this and use it?

Q: سلام !
A: سلام چطوری !
Q: چخبر ?
A: خوبم .

===
Q: چکار کنم !
A: برو به فلان ادرس !

@bshao001 great work! So I am using my own dataset to build vocab.txt and run training on it.

My dataset is 500 mb which produces a vocab.txt of 15 mb . As I train on this on AWS sagemaker, I am getting out of memory errors.

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[2048,1024] [[Node: dynamic_seq2seq/decoder/attention/attention_layer/kernel/Initializer/random_uniform/RandomUniform = RandomUniform[T =DT_INT32, _class=["loc:@dynamic_seq2seq/decoder/attention/attention_layer/kernel"], dtype=DT_FLOAT, seed=87654321, seed2=0, _device ="/job:localhost/replica:0/task:0/device:GPU:0"](dynamic_seq2seq/decoder/attention/attention_layer/kernel/Initializer/random_uniform /shape)]]

I am using :

You do mention that vocab.txt will affect the performance , what is the optimal dataset size/vocab size for this configuration in your opinion ?

Hi Bsha001,

Thanks for the excellent work and repo to begin with AI Chatbots. Really helped a lot in understanding the AI chatbot concepts and implementations.

I m able to train my custom corpus data with small sets after some good amount of training time. But now I want to expand my training data. So for that I have to update new data files to Corpus folder.
Then model training will start from beginning with all old trained data too which is time taking again.

Is to possible to pass latest or specific model to training script and start training from that checkpoint.
I have seen such implementation here in this Repo @ https://github.com/daniel-kukiela/nmt-chatbot
Can we also have something like this .. ???

And also is the current implementation is bidirectional RNN ?? And I have seen code for nmt implementation. I didnt find any such module imported, are we using it ? As far I understand its a concept but not a module, but just to make sure and to understand how NMT is applied in this repo.

Once again thanks for an amazing work and sharing them with us. Keeping going broo... !! Cheers ..

Awaiting a quick reply for my queries ...

Thanks
Sam

Hi I want to train chatbot in Arabic
should I change the source code for it will work fine?

tnx

Hi Shao,
So I finished running the bottrain.py, but when I interact with the chat bot, it gives me really messy and random reply.

The training perplexity converges to ~1.3, so it seems the problem is not in the model. Have you ever faced this problem? Is it possible tokenizeddata.py didn't read the lines well?

Any insight is helpful before I give my code a full scan... Thanks!

Sample outputs:

'>' Hello!
America members to.,...,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,.,..,...,..
'>' Hi
Korea,,.,.,......,.,.,......!.!.!.!.!.!.!.!.!.!.!.!.!.

hello.
I have an error here and it seems chatbot package cause it. how can I run bottrainer.py?

marn@mint18 ~/c/C/chatbot> python3 bottrainer.py 
Traceback (most recent call last):
  File "bottrainer.py", line 20, in <module>
    from chatbot.tokenizeddata import TokenizedData
  File "/usr/local/lib/python3.5/dist-packages/chatbot.py", line 166
    print 'Couldn\'t log in: Quitting.'
                                      ^
SyntaxError: Missing parentheses in call to 'print'
marn@mint18 ~/c/C/chatbot> pip3 install chatbot --upgrade
Requirement already up-to-date: chatbot in /usr/local/lib/python3.5/dist-packages

Hi, I've changed my batch size to 128, 256, 512, 1024, 2048 (I get an OOM above 1000) none of them seem to utilize any but one of the GPU's. Tensorflow see's the other GPU's and so does Nvidia-SMI. I'm using 4x Nvidia Tesla P100's.

Nvidia-SMI:

Is this an issue on my part or something else?

This line doesn't work for me in basicmodel.py:
from chatbot.tfcopy.seq2seq import embedding_attention_seq2seq

I'm having No module named 'chatbot' with Python 3.5.2 Because basicmodel.py is already in ./chatbot directory. Does not it have to be like this:
from tfcopy.seq2seq import embedding_attention_seq2seq

I've realized you do that everywhere. So I'm sure there is something that I don't know but Google didn't work this time. Can you please explain this or give me a link about that?

@bshao001
Hi we have increased the dataset size by including open-subtitles dataset.We have preprocessed the dataset.We have problem in generating vocabulary.
How can i make changes such the bot initiates a conversation by saying hello...
Kindly help me in solving this issue.

Hey @bshao001,
can you please, tell me a little about the Attention Mechanism, i saw the implementation on the nmt site.
Thanks

Hi! Love the project! Was just wondering what the input and output node names of your pretrained model were? Been trying to figure that out for the past couple of days and thought I just ask! Thanks again!

Hello!
Great Repository!

I am trying to train model with custom data, i read one help reagrding this in which i came to know that we need to regenrate vocab.txt by running vocabgenerator.py in Corpus folder.
So my questions are:

1. When we run bottrainer.py, does this program just lookup for vocab.txt and not text files?
2. I looked a little into code it is looking for corpus directory then how data from "KnowledgeBase" folder is taken?

I am new to tensorflow nmt model, as i understand

1. We have collection of text files with series of questions/answers,
2. Then we converted those text files in the format so that our model can understand as vocab.txt
3. Then when we train, program looks for vocab.txt and generate files in result folder that are used by chat program.

Please ! clarify me.
Thanks in advance.

Hi, you have done a great project, but if this project could save the model use saved_model api?

i am running into this issue with gtx 1070, 8gb, looks like i am running out of memory!? if yes where would be the right place to change the batch_size?

during training, nvidia-smi:

| 0 5341 C python3.5 7497MiB |

string_to_index/hash_table/Const: (Const): /job:localhost/replica:0/task:0/device:CPU:0
# Training loop started @ 2017-11-14 19:50:02
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
    status, run_metadata)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[1024,50785]
     [[Node: gradients/dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul_grad/MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=true, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dynamic_seq2seq/decoder/output_projection/Tensordot/Reshape, gradients/dynamic_seq2seq/decoder/output_projection/Tensordot_grad/Reshape)]]
     [[Node: gradients/dynamic_seq2seq/decoder/decoder/while/BasicDecoderStep/TrainingHelperNextInputs/cond/Merge_grad/cond_grad/_205 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1599_gradients/dynamic_seq2seq/decoder/decoder/while/BasicDecoderStep/TrainingHelperNextInputs/cond/Merge_grad/cond_grad", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^_cloopgradients/dynamic_seq2seq/decoder/decoder/while/BasicDecoderStep/TrainingHelperNextInputs/cond/TensorArrayReadV3_grad/TensorArrayWrite/TensorArrayWriteV3/Switch/_45)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bottrainer.py", line 141, in <module>
    bt.train(res_dir)
  File "bottrainer.py", line 71, in train
    step_result = self.model.train_step(sess, learning_rate=learning_rate)
  File "/home/flyn/git/ChatLearner/chatbot/modelcreator.py", line 122, in train_step
    feed_dict={self.learning_rate: learning_rate})
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 889, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
    options, run_metadata)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[1024,50785]
     [[Node: gradients/dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul_grad/MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=true, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dynamic_seq2seq/decoder/output_projection/Tensordot/Reshape, gradients/dynamic_seq2seq/decoder/output_projection/Tensordot_grad/Reshape)]]
     [[Node: gradients/dynamic_seq2seq/decoder/decoder/while/BasicDecoderStep/TrainingHelperNextInputs/cond/Merge_grad/cond_grad/_205 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1599_gradients/dynamic_seq2seq/decoder/decoder/while/BasicDecoderStep/TrainingHelperNextInputs/cond/Merge_grad/cond_grad", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^_cloopgradients/dynamic_seq2seq/decoder/decoder/while/BasicDecoderStep/TrainingHelperNextInputs/cond/TensorArrayReadV3_grad/TensorArrayWrite/TensorArrayWriteV3/Switch/_45)]]

Caused by op 'gradients/dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul_grad/MatMul_1', defined at:
  File "bottrainer.py", line 140, in <module>
    bt = BotTrainer(corpus_dir=corp_dir)
  File "bottrainer.py", line 35, in __init__
    batch_input=self.train_batch)
  File "/home/flyn/git/ChatLearner/chatbot/modelcreator.py", line 86, in __init__
    gradients = tf.gradients(self.train_loss, params)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/gradients_impl.py", line 581, in gradients
    grad_scope, op, func_call, lambda: grad_fn(op, *out_grads))
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/gradients_impl.py", line 353, in _MaybeCompile
    return grad_fn()  # Exit early
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/gradients_impl.py", line 581, in <lambda>
    grad_scope, op, func_call, lambda: grad_fn(op, *out_grads))
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/math_grad.py", line 922, in _MatMulGrad
    grad_b = math_ops.matmul(a, grad, transpose_a=True)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py", line 1891, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/gen_math_ops.py", line 2437, in _mat_mul
    name=name)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
    op_def=op_def)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1470, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

...which was originally created as op 'dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul', defined at:
  File "bottrainer.py", line 140, in <module>
    bt = BotTrainer(corpus_dir=corp_dir)
[elided 0 identical lines from previous traceback]
  File "bottrainer.py", line 35, in __init__
    batch_input=self.train_batch)
  File "/home/flyn/git/ChatLearner/chatbot/modelcreator.py", line 65, in __init__
    res = self.build_graph(hparams, scope=scope)
  File "/home/flyn/git/ChatLearner/chatbot/modelcreator.py", line 134, in build_graph
    encoder_outputs, encoder_state, hparams)
  File "/home/flyn/git/ChatLearner/chatbot/modelcreator.py", line 221, in _build_decoder
    logits = self.output_layer(outputs.rnn_output)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/layers/base.py", line 575, in __call__
    outputs = self.call(inputs, *args, **kwargs)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/layers/core.py", line 156, in call
    [0]])
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py", line 2520, in tensordot
    ab_matmul = matmul(a_reshape, b_reshape)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/math_ops.py", line 1891, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/ops/gen_math_ops.py", line 2437, in _mat_mul
    name=name)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
    op_def=op_def)

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[1024,50785]
     [[Node: gradients/dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul_grad/MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=true, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dynamic_seq2seq/decoder/output_projection/Tensordot/Reshape, gradients/dynamic_seq2seq/decoder/output_projection/Tensordot_grad/Reshape)]]
     [[Node: gradients/dynamic_seq2seq/decoder/decoder/while/BasicDecoderStep/TrainingHelperNextInputs/cond/Merge_grad/cond_grad/_205 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1599_gradients/dynamic_seq2seq/decoder/decoder/while/BasicDecoderStep/TrainingHelperNextInputs/cond/Merge_grad/cond_grad", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^_cloopgradients/dynamic_seq2seq/decoder/decoder/while/BasicDecoderStep/TrainingHelperNextInputs/cond/TensorArrayReadV3_grad/TensorArrayWrite/TensorArrayWriteV3/Switch/_45)]]

chatbot/tokenizeddata.py", line 55, in init
self.src_max_len = self.hparams.src_max_len

AttributeError: 'NoneType' object has no attribute 'src_max_len'

I'm trying to get chatbot to run with ec2 instance( by g2.2xlarge) but get this error

Training loop started @ 2018-01-27 11:28:24
Traceback (most recent call last):
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[1024,50799]
[[Node: gradients/dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul_grad/MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=true, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dynamic_seq2seq/decoder/output_projection/Tensordot/Reshape, gradients/dynamic_seq2seq/decoder/output_projection/Tensordot_grad/Reshape)]]
[[Node: gradients/dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/dropout/div_grad/Reshape/_331 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_2653_gradients/dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/dropout/div_grad/Reshape", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "bottrainer.py", line 142, in
bt.train(res_dir)
File "bottrainer.py", line 71, in train
step_result = self.model.train_step(sess, learning_rate=learning_rate)
File "/home/ubuntu/ChatLearner/chatbot/modelcreator.py", line 122, in train_step
feed_dict={self.learning_rate: learning_rate})
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[1024,50799]
[[Node: gradients/dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul_grad/MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=true, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dynamic_seq2seq/decoder/output_projection/Tensordot/Reshape, gradients/dynamic_seq2seq/decoder/output_projection/Tensordot_grad/Reshape)]]
[[Node: gradients/dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/dropout/div_grad/Reshape/_331 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_2653_gradients/dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/dropout/div_grad/Reshape", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Caused by op 'gradients/dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul_grad/MatMul_1', defined at:
File "bottrainer.py", line 141, in
bt = BotTrainer(corpus_dir=corp_dir)
File "bottrainer.py", line 35, in init
batch_input=self.train_batch)
File "/home/ubuntu/ChatLearner/chatbot/modelcreator.py", line 86, in init
gradients = tf.gradients(self.train_loss, params)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 581, in gradients
grad_scope, op, func_call, lambda: grad_fn(op, *out_grads))
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 353, in _MaybeCompile
return grad_fn() # Exit early
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py", line 581, in
grad_scope, op, func_call, lambda: grad_fn(op, *out_grads))
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/math_grad.py", line 922, in _MatMulGrad
grad_b = math_ops.matmul(a, grad, transpose_a=True)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1891, in matmul
a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 2437, in _mat_mul
name=name)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

...which was originally created as op 'dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul', defined at:
File "bottrainer.py", line 141, in
bt = BotTrainer(corpus_dir=corp_dir)
[elided 0 identical lines from previous traceback]
File "bottrainer.py", line 35, in init
batch_input=self.train_batch)
File "/home/ubuntu/ChatLearner/chatbot/modelcreator.py", line 65, in init
res = self.build_graph(hparams, scope=scope)
File "/home/ubuntu/ChatLearner/chatbot/modelcreator.py", line 134, in build_graph
encoder_outputs, encoder_state, hparams)
File "/home/ubuntu/ChatLearner/chatbot/modelcreator.py", line 221, in _build_decoder
logits = self.output_layer(outputs.rnn_output)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/layers/base.py", line 575, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/layers/core.py", line 156, in call
[0]])
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 2520, in tensordot
ab_matmul = matmul(a_reshape, b_reshape)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1891, in matmul
a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 2437, in _mat_mul
name=name)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[1024,50799]
[[Node: gradients/dynamic_seq2seq/decoder/output_projection/Tensordot/MatMul_grad/MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=true, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:GPU:0"](dynamic_seq2seq/decoder/output_projection/Tensordot/Reshape, gradients/dynamic_seq2seq/decoder/output_projection/Tensordot_grad/Reshape)]]
[[Node: gradients/dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/dropout/div_grad/Reshape/_331 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_2653_gradients/dynamic_seq2seq/encoder/rnn/while/rnn/multi_rnn_cell/cell_0/dropout/div_grad/Reshape", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Hello!
Great repository!

Have you tried to save-restore encoder state from sequence to sequence to be able to have a context of dialogue?

I train my own model based on nmt, which will try to predict very long dialogues, and I split long dialogs by <author, sentence> sequences, don't know if it'll work yet.

Nevermind, I'm an idiot and figured out my own problem. Missed the pythonpath part of the readme

.

Hello,
Thank you for creating this wonderful repo. It is really helping me understand how a generative model is built.
Just a quick question, could you please tell me how to add new rules to your existing model ?
Thanks.

First thanks a lot for this fantastic repository. This is not an issue, it is just a question.

1. How should I create custom dataset, what format is needed ?
2. In which files should I write my custom questions and answers ?
3. How should I train the bot with custom dataset ?

It would be really great, if you can help me on above questions.

Thanks in advance.

@bshao001
We have followed the same procedure for Ubuntu.I have installed tornado from Pycharm Edition and Tomcat from ubuntu terminal.We are not getting response for it.I didnt understand how the Tomcat-client sending request to Tornado Server.i didn't understand how tornado server contacting client internally.

Please solve my issue as soon as possible

Thanks in advance

I started the running the algorithme 4 day ago , and it still till now. My Pc is 12G RAM and the CPU is i5
How many epoch takes to finish!?

I tried to run the botui.py file and I got this error. I tried to transfer the result files from my remote host back to my local machine first and I'm afraid I've corrupted the data somehow. Have you ever had this happen and is there any fix besides retraining the whole model?

NotFoundError (see above for traceback): Key dynamic_seq2seq/decoder/attention/multi_rnn_cell/cell_2/gru_cell/gates/kernel not found in checkpoint
[[Node: save/RestoreV2_13 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_13/tensor_names, save/RestoreV2_13/shape_and_slices)]]
[[Node: save/RestoreV2_13/_11 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_74_save/RestoreV2_13", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

Hi,Thanks for your great project.
I have deployed trainer and finished fine ,but when start to run botui.py I get the folloing error.
Result folder just have the file /Data/Result/train_log/events.out.tfevents.1516479132.ali ,no file with name basic.

/home/ali/PycharmProjects/ChatLearner/venv/bin/python /home/ali/PycharmProjects/ChatLearner/chatbot/botui.py
/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)# Prepare dataset placeholder and hyper parameters ...# Loading hparams from /home/ali/PycharmProjects/ChatLearner/Data/Corpus/hparams.json ...
Creating inference model ...
Building graph for the model ...
Restoring model weights ...
Traceback (most recent call last):
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /home/ali/PycharmProjects/ChatLearner/Data/Result/basic
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/ali/PycharmProjects/ChatLearner/chatbot/botui.py", line 53, in
bot_ui()
File "/home/ali/PycharmProjects/ChatLearner/chatbot/botui.py", line 32, in bot_ui
result_dir=res_dir, result_file='basic')
File "/home/ali/PycharmProjects/ChatLearner/chatbot/botpredictor.py", line 62, in init
self.model.saver.restore(session, os.path.join(result_dir, result_file))
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1666, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /home/ali/PycharmProjects/ChatLearner/Data/Result/basic
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

Caused by op 'save/RestoreV2', defined at:
File "/home/ali/PycharmProjects/ChatLearner/chatbot/botui.py", line 53, in
bot_ui()
File "/home/ali/PycharmProjects/ChatLearner/chatbot/botui.py", line 32, in bot_ui
result_dir=res_dir, result_file='basic')
File "/home/ali/PycharmProjects/ChatLearner/chatbot/botpredictor.py", line 59, in init
batch_input=self.infer_batch)
File "/home/ali/PycharmProjects/ChatLearner/chatbot/modelcreator.py", line 103, in init
self.saver = tf.train.Saver(tf.global_variables())
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1218, in init
self.build()
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1227, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1263, in _build
build_save=build_save, build_restore=build_restore)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 751, in _build_internal
restore_sequentially, reshape)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 427, in _AddRestoreOps
tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 267, in restore_op
[spec.tensor.dtype])[0])
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1021, in restore_v2
shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/home/ali/PycharmProjects/ChatLearner/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

NotFoundError (see above for traceback): Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /home/ali/PycharmProjects/ChatLearner/Data/Result/basic
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

Process finished with exit code 1

Hi @bshao001 ,
My stack comprises of:
Tensorflow-1.4
Python-3.5
CPU-8GB RAM
Pycharm IDE-2017.
NLTK-3.2.5

I'm trying to use your pre-trained model while running it I'm facing error like
from settings import PROJECT_ROOT
ImportError: No module named 'settings'

Can you please resolve my issue as early as possible.
Thanks in advance.

So I have now both tornado server and tomcat running and I face two errors. I was hoping you could help out in those:

1. The tornado webserver starts fine (I dont add papaya.net to my hosts file) but when I try to run
   http://localhost:8080/ChatService?wsdl I get internal server error (500). The weird thing is this goes away as soon as I switch off my VPN. The error I get with VPN on is :

# Restoring model weights ...
Web service started.
ERROR:tornado.application:Uncaught exception GET /ChatService?wsdl (127.0.0.1)
HTTPServerRequest(protocol='http', host='0.0.0.0:8080', method='GET', uri='/ChatService?wsdl', version='HTTP/1.1', remote_ip='127.0.0.1')
Traceback (most recent call last):
  File "/usr/local/lib64/python3.6/site-packages/tornado/web.py", line 1541, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "/home/ec2-user/elian/webui/server/tornadows/soaphandler.py", line 117, in get
    tornado.httpserver.socket.gethostbyname(tornado.httpserver.socket.gethostname()))
socket.gaierror: [Errno -2] Name or service not known
ERROR:tornado.access:500 GET /ChatService?wsdl (127.0.0.1) 5.01ms

2. As I run tomcat it spins up the webclient. I enter Hi and the webclient fails with

org.apache.jasper.JasperException: Unable to compile class for JSP: 

An error occurred at line: 7 in the jsp file: /ajax/getChatReply.jsp
ChatClient cannot be resolved to a type
4:     response.setContentType("text/html; charset=utf-8");
5:     response.setHeader("Cache-Control", "no-cache");
6: 	
7: 	ChatClient cc = (ChatClient)session.getAttribute("ChatClient");
8:     if (cc == null) {
9: 		cc = new ChatClient("http://papayachat.net:8080/ChatService", 5);
10: 		session.setAttribute("ChatClient", cc);

Any idea for both the errors?

Hi
@bshao001
@ArashHosseini
My stack comprises of:
Centos 6.9
Anaconda3
Tensorflow 1.3.
Pycharm IDE
CPU RAM-16
As you mentioned (It works only with Tensor-flow 1.4, not any earlier releases because the tf.data API used here was newly updated in TF 1.4.)I have gone through this.But please suggest me necessary changes for TENSOR-FLOW 1.3.
Even for ubuntu how to change modifications to Tensorflow 1.3.

I am waiting for your reply.Please respond Quickly.
Thanks in advance

I have installed cud using https://yangcha.github.io/GTX-1080/ ,then trying to run but still work with cpu not gpu,How to run with gpu?

error as follows:
tensorflow.python.framework.errors_impl.UnimplementedError: TensorArray has size zero, but element shape [?,1024] is not fully defined. Currently only static shapes are supported when packing zero-size TensorArrays.
[[Node: dynamic_seq2seq/encoder/rnn/TensorArrayStack/TensorArrayGatherV3 = TensorArrayGatherV3[_class=["loc:@dynamic_seq2seq/encoder/rnn/TensorArray"], dtype=DT_FLOAT, element_shape=[?,1024], _device="/job:localhost/replica:0/task:0/device:GPU:0"](dynamic_seq2seq/encoder/rnn/TensorArray, dynamic_seq2seq/encoder/rnn/TensorArrayStack/range, dynamic_seq2seq/encoder/rnn/while/Exit_1)]]
what's wrong? help me, please

NotFoundError (see above for traceback): Unsuccessful TensorSliceReader constructor: Failed to find any matching files for /var/www/html/python_ai/ChatLearner/Data/Result/basic
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_INT32], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

Please suggest something.

Hello,
my question about Dataset, which data set you to use to train chatlearner and where i get the dataset. and any tips to process data
thanks

AttributeError: 'NoneType' object has no attribute 'src_max_len'
这个错误怎么解决，请指教。
vocab.txt数据能否传给我一份

In Corpus/reddit_cleaned_part, It is error if I add chinese dialogue。
report error as follows:
InvalidArgumentError (see above for traceback): assertion failed: [All values in memory_sequence_length must greater than zero.]