This repository contains code examples for the Stanford's course: TensorFlow for Deep Learning Research.
Home Page: http://cs20.stanford.edu
License: MIT License
This repository contains code examples for the course CS 20: TensorFlow for Deep Learning Research.
It will be updated as the class progresses.
Detailed syllabus and lecture notes can be found here.
For this course, I use python3.6 and TensorFlow 1.4.1.
For the code and notes of the previous year's course, please see the folder 2017 and the website https://web.stanford.edu/class/cs20si/2017
For setup instruction and the list of dependencies, please see the setup folder of this repository.
Hi I am going thorough the stanforf-tutorials and trying to run the fourth Task which is 04_word2vec_no_frills but it is asking for Process_data package and I am trying to load it through pip but its showing no such package,later I found there is a python script for process_data.py file and I had ran it, its still showing an error! Any help would appreciated!
Thanks In advance!!!
I left the chatbot.py running in training mode for 2 hours and it's still going, when does it end (if it even does)?
in 03_logistic_regression_mnist_starter.py, the test part:
for i in range(n_batches):
X_batch, Y_batch = mnist.test.next_batch(batch_size)
_, loss_batch, logits_batch = sess.run([optimizer, loss, logits], feed_dict={X: X_batch, Y:Y_batch})
preds = tf.nn.softmax(logits_batch)
correct_preds = tf.equal(tf.argmax(preds, 1), tf.argmax(Y_batch, 1))
accuracy = tf.reduce_sum(tf.cast(correct_preds, tf.float32)) # need numpy.count_nonzero(boolarr) :(
total_correct_preds += sess.run(accuracy)
for every iteration, it will add ops for testing to the graph, I think the test part should be written only once, and the in every iteration, we can pass data through placeholder.
It seems that the order of inputs and labels parameter is changed in the new tf.nn.sampled_softmax_loss API.
But after changing the order, it still reports error: TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type int32 of argument 'x'.
CODE: modle.py
`def _inference(self):
print('Create inference')
# If we use sampled softmax, we need an output projection.
# Sampled softmax only makes sense if we sample less than vocabulary size.
if config.NUM_SAMPLES > 0 and config.NUM_SAMPLES < config.DEC_VOCAB:
w = tf.get_variable('proj_w', [config.HIDDEN_SIZE, config.DEC_VOCAB])
b = tf.get_variable('proj_b', [config.DEC_VOCAB])
self.output_projection = (w, b)
def sampled_loss(inputs, labels):
labels = tf.reshape(labels, [-1, 1])
#return tf.nn.sampled_softmax_loss(tf.transpose(w), b, inputs, labels,
return tf.nn.sampled_softmax_loss(tf.transpose(w), b, labels, inputs,
config.NUM_SAMPLES, config.DEC_VOCAB)
self.softmax_loss_function = sampled_loss
#single_cell = tf.nn.rnn_cell.GRUCell(config.HIDDEN_SIZE)
#self.cell = tf.nn.rnn_cell.MultiRNNCell([single_cell] * config.NUM_LAYERS)
single_cell = tf.contrib.rnn.GRUCell(config.HIDDEN_SIZE)
self.cell = tf.contrib.rnn.MultiRNNCell([single_cell] * config.NUM_LAYERS)`
ERROR REPORT:
Traceback (most recent call last):
File "chatbot.py", line 261, in
main()
File "chatbot.py", line 258, in main
chat()
File "chatbot.py", line 203, in chat
model.build_graph()
File "D:\chatbot\tf-stanford-tutorials-master\tf-stanford-tutorials-master\assignments\chatbot\model.py", line 142, in build_graph
self._create_loss()
File "D:\chatbot\tf-stanford-tutorials-master\tf-stanford-tutorials-master\assignments\chatbot\model.py", line 94, in _create_loss
softmax_loss_function=self.softmax_loss_function)
File "D:\Python3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1195, in model_with_buckets
softmax_loss_function=softmax_loss_function))
File "D:\Python3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1110, in sequence_loss
softmax_loss_function=softmax_loss_function))
File "D:\Python3\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1067, in sequence_loss_by_example
crossent = softmax_loss_function(target, logit)
File "D:\chatbot\tf-stanford-tutorials-master\tf-stanford-tutorials-master\assignments\chatbot\model.py", line 64, in sampled_loss
config.NUM_SAMPLES, config.DEC_VOCAB)
File "D:\Python3\lib\site-packages\tensorflow\python\ops\nn_impl.py", line 1189, in sampled_softmax_loss
name=name)
File "D:\Python3\lib\site-packages\tensorflow\python\ops\nn_impl.py", line 972, in _compute_sampled_logits
array_ops.reshape(true_w, new_true_w_shape))
File "D:\Python3\lib\site-packages\tensorflow\python\ops\math_ops.py", line 267, in multiply
return gen_math_ops._mul(x, y, name)
File "D:\Python3\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 1625, in _mul
result = _op_def_lib.apply_op("Mul", x=x, y=y, name=name)
File "D:\Python3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 522, in apply_op
inferred_from[input_arg.type_attr]))
TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type int32 of argument 'x'.
Hello,
I'm trying to run the chatbot, but I'm receiving many errors that I'd appreciate if you could help me solve. The last error that has appeared is one in the middle of the section "Loading parameters for the Chatbot".
Before getting to this point, I received an error that fixed by editing lines around 848 in the seq2seq.py file:
#encoder_cell = copy.deepcopy(cell)
cell = copy.deepcopy(cell)
encoder_cell = core_rnn_cell.EmbeddingWrapper(
#encoder_cell,
cell,
This is the result of the execution:
# python3.6 chatbot.py
Data ready!
Bucketing conversation number 9999
Bucketing conversation number 19999
Bucketing conversation number 9999
Bucketing conversation number 19999
Bucketing conversation number 29999
Bucketing conversation number 39999
Bucketing conversation number 49999
Bucketing conversation number 59999
Bucketing conversation number 69999
Bucketing conversation number 79999
Bucketing conversation number 89999
Bucketing conversation number 99999
Bucketing conversation number 109999
Bucketing conversation number 119999
Bucketing conversation number 129999
Bucketing conversation number 139999
Bucketing conversation number 149999
Bucketing conversation number 159999
Bucketing conversation number 169999
Bucketing conversation number 179999
Bucketing conversation number 189999
Number of samples in each bucket:
[38095, 34447, 30931]
Bucket scale:
[0.36816367554821067, 0.7010717771785876, 1.0]
Initialize new model
Create placeholders
Create inference
Creating loss...
It might take a couple of minutes depending on how many buckets you have.
Time: 14.253957271575928
Create optimizer...
It might take a couple of minutes depending on how many buckets you have.
Creating opt for bucket 0 took 5.574482440948486 seconds
Creating opt for bucket 1 took 8.757641077041626 seconds
Creating opt for bucket 2 took 11.90887451171875 seconds
2017-10-15 17:22:01.217183: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-10-15 17:22:01.217281: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-10-15 17:22:01.217307: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
Running session
Loading parameters for the Chatbot
Iter 400: loss 4.7150276041030885, time 5.993304967880249
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1327, in _do_call
return fn(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1306, in _run_fn
status, run_metadata)
File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
next(self.gen)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[2] = 24675 is not in [0, 24671)
[[Node: model_with_buckets/sequence_loss_2/sequence_loss_by_example/sampled_softmax_loss_14/embedding_lookup_1 = Gather[Tindices=DT_INT64, Tparams=DT_FLOAT, _class=["loc:@proj_b"], validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](proj_b/read, model_with_buckets/sequence_loss_2/sequence_loss_by_example/sampled_softmax_loss_14/concat)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "chatbot.py", line 261, in <module>
main()
File "chatbot.py", line 256, in main
train()
File "chatbot.py", line 155, in train
_, step_loss, _ = run_step(sess, model, encoder_inputs, decoder_inputs, decoder_masks, bucket_id, False)
File "chatbot.py", line 81, in run_step
outputs = sess.run(output_feed, input_feed)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1124, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1321, in _do_run
options, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[2] = 24675 is not in [0, 24671)
[[Node: model_with_buckets/sequence_loss_2/sequence_loss_by_example/sampled_softmax_loss_14/embedding_lookup_1 = Gather[Tindices=DT_INT64, Tparams=DT_FLOAT, _class=["loc:@proj_b"], validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](proj_b/read, model_with_buckets/sequence_loss_2/sequence_loss_by_example/sampled_softmax_loss_14/concat)]]
Caused by op 'model_with_buckets/sequence_loss_2/sequence_loss_by_example/sampled_softmax_loss_14/embedding_lookup_1', defined at:
File "chatbot.py", line 261, in <module>
main()
File "chatbot.py", line 256, in main
train()
File "chatbot.py", line 137, in train
model.build_graph()
File "/root/Desktop/chatbot/model.py", line 132, in build_graph
self._create_loss()
File "/root/Desktop/chatbot/model.py", line 100, in _create_loss
softmax_loss_function=self.softmax_loss_function)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1223, in model_with_buckets
softmax_loss_function=softmax_loss_function))
File "/usr/local/lib/python3.6/dist-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1136, in sequence_loss
softmax_loss_function=softmax_loss_function))
File "/usr/local/lib/python3.6/dist-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1091, in sequence_loss_by_example
crossent = softmax_loss_function(labels=target, logits=logit)
File "/root/Desktop/chatbot/model.py", line 59, in sampled_loss
config.NUM_SAMPLES, config.DEC_VOCAB)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/nn_impl.py", line 1247, in sampled_softmax_loss
name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/nn_impl.py", line 986, in _compute_sampled_logits
biases, all_ids, partition_strategy=partition_strategy)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/embedding_ops.py", line 294, in embedding_lookup
transform_fn=None)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/embedding_ops.py", line 123, in _embedding_lookup_and_transform
result = _gather_and_clip(params[0], ids, max_norm, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/embedding_ops.py", line 57, in _gather_and_clip
embs = array_ops.gather(params, ids, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py", line 2409, in gather
validate_indices=validate_indices, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 1219, in gather
validate_indices=validate_indices, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 1204, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): indices[2] = 24675 is not in [0, 24671)
[[Node: model_with_buckets/sequence_loss_2/sequence_loss_by_example/sampled_softmax_loss_14/embedding_lookup_1 = Gather[Tindices=DT_INT64, Tparams=DT_FLOAT, _class=["loc:@proj_b"], validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](proj_b/read, model_with_buckets/sequence_loss_2/sequence_loss_by_example/sampled_softmax_loss_14/concat)]]
TensorFlow 1.3 appears to have moved the seq2seq module you are using to: tf.contrib.legacy_seq2seq. Then they have new ones with completely different arguments. With TensorFlow 1.3 you still have the problem someone reported with TensorFlow 1.2.1 where you get the error:
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 1089, in sequence_loss_by_example
crossent = softmax_loss_function(labels=target, logits=logit)
TypeError: sampled_loss() got an unexpected keyword argument 'logits'
from the sampled loss function
Hi,
I'm new to TF and came across your excellent course.
I was trying to perform the quadratic function in lecture 3, but I get NaN for my Epochs when I run.
Is the quadratic function solution / code avail anywhere?
Attached are the changes I made in line w/ the lecture notes but I think more changes are necessary than what is in the lecture notes. From the quadratic function plot, I see that the result is not a line but a series of X, Y plots (presumably Y_predicted) but I don't know how to get that value from TF (as of yet, that's why I'm going through this excellent course! :)
Thanks!
HI, Tks for the post, very helpful.
As title, I found several variable undefined,
for prev_layer_name, I think, it should be prev_name.name, however ':' is not accepted as scope name. So I changed ':' to '_', and it works
For CONTENT_WEIGHT and STYLE_WEIGHT, how to define it ?
(of course, omitted the weight could let the program keep running)
Tks
Larry
Line 76 should be
loss_batch, logits_batch = sess.run([loss, logits], feed_dict={X: X_batch, Y:Y_batch})
We need to test it based on test data, not train it based on test data.
The current code is throwing error in the line
w, b = sess.run([w, b])
Hence,below code should be changed
w = tf.Variable(0.0, name='weights') to tf.Variable(0.0, name='w')
b = tf.Variable(0.0, name='bias') to tf.Variable(0.0, name='b')
running python data.py is causing followin error, must be because of opening it in binary mode.
Tried it with python 3.5.3 on macOS Sierra
Traceback (most recent call last):
File "data.py", line 255, in <module>
prepare_raw_data()
File "data.py", line 178, in prepare_raw_data
id2line = get_lines()
File "data.py", line 34, in get_lines
parts = line.split(' +++$+++ ')
TypeError: a bytes-like object is required, not 'str'
at 03_logistic_regression_mnist_sol.py line 84-87
for i in range(n_batches):
X_batch, Y_batch = mnist.test.next_batch(batch_size)
accuracy_batch = sess.run([accuracy], feed_dict={X: X_batch, Y:Y_batch})
total_correct_preds += accuracy_batch
this yielded a TypeError on my running of python 3.5 tf 1.2 on windows
maybe add brackets as [accuracy_batch] would fix
Can you give Autoencoder/autoencoder.py example code? Thanks!
Should you be using
model.embed
instead of
model.embed_matrix
on line 129 of file 04_word2vec_visualize.py ?
I hv trouble in running the chatbot.py, is it because of the tensorflow version? how to fix it?
python chatbot.py --mode train
Data ready!
Bucketing conversation number 9999
Bucketing conversation number 19999
Bucketing conversation number 9999
Bucketing conversation number 19999
Bucketing conversation number 29999
Bucketing conversation number 39999
Bucketing conversation number 49999
Bucketing conversation number 59999
Bucketing conversation number 69999
Bucketing conversation number 79999
Bucketing conversation number 89999
Bucketing conversation number 99999
Bucketing conversation number 109999
Bucketing conversation number 119999
Bucketing conversation number 129999
Bucketing conversation number 139999
Bucketing conversation number 149999
Bucketing conversation number 159999
Bucketing conversation number 169999
Bucketing conversation number 179999
Bucketing conversation number 189999
Number of samples in each bucket:
[103459]
Bucket scale:
[1.0]
Initialize new model
Create placeholders
Create inference
Creating loss...
It might take a couple of minutes depending on how many buckets you have.
Traceback (most recent call last):
File "chatbot.py", line 262, in
main()
File "chatbot.py", line 257, in main
train()
File "chatbot.py", line 138, in train
model.build_graph()
File "C:\Users\Alan\Documents\Udacity Deep learning\chatbot\stanford-tensorflow-tutorials-master\stanford-tensorflow-tutorials-master\assignments\chatbot\model.py", line 132, in build_graph
self._create_loss()
File "C:\Users\Alan\Documents\Udacity Deep learning\chatbot\stanford-tensorflow-tutorials-master\stanford-tensorflow-tutorials-master\assignments\chatbot\model.py", line 100, in _create_loss
softmax_loss_function=self.softmax_loss_function)
File "C:\Program Files\Anaconda3\envs\rnn2\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1221, in model_with_buckets
softmax_loss_function=softmax_loss_function))
File "C:\Program Files\Anaconda3\envs\rnn2\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1134, in sequence_loss
softmax_loss_function=softmax_loss_function))
File "C:\Program Files\Anaconda3\envs\rnn2\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py", line 1089, in sequence_loss_by_example
crossent = softmax_loss_function(labels=target, logits=logit)
TypeError: sampled_loss() got an unexpected keyword argument 'logits'
could someone tell me how to deal with it.
Traceback (most recent call last):
File "C:/Users/Jerin/PycharmProjects/pyflow/main.py", line 81, in
total_correct_preds += accuracy_batch
TypeError: unsupported operand type(s) for +=: 'int' and 'list'
After running python data.py I get the following error:
Preparing raw data into train set and test set ...
Traceback (most recent call last):
File "data.py", line 250, in
prepare_raw_data()
File "data.py", line 173, in prepare_raw_data
id2line = get_lines()
File "data.py", line 29, in get_lines
parts = line.split(' +++$+++ ')
TypeError: a bytes-like object is required, not 'str'
Hi Hadi,
I use Python 2.7.13 and Tensorflow 1.3.0 on CPU.
I want to use DensNet( https://github.com/pudae/tensorflow-densenet ) for regression problem. My data contains 60000 jpeg images with 37 float labels for each image.
I saved my data into tfrecords files as you mentioned on your page by:
`
def Read_Labels(label_path):
labels_csv = pd.read_csv(label_path)
labels = np.array(labels_csv)
return labels[:,1:]
def load_image(addr):
# read an image and resize to (224, 224)
img = cv2.imread(addr)
img = cv2.resize(img, (224, 224), interpolation=cv2.INTER_CUBIC)
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
img = img.astype(np.float32)
return img
def Shuffle_images_with_labels(shuffle_data, photo_filenames, labels):
if shuffle_data:
c = list(zip(photo_filenames, labels))
shuffle(c)
addrs, labels = zip(*c)
return addrs, labels
def image_to_tfexample_mine(image_data, image_format, height, width, label):
return tf.train.Example(features=tf.train.Features(feature={
'image/encoded': bytes_feature(image_data),
'image/format': bytes_feature(image_format),
'image/class/label': _float_feature(label),
'image/height': int64_feature(height),
'image/width': int64_feature(width),
}))
def _convert_dataset(split_name, filenames, labels, dataset_dir):
assert split_name in ['train', 'validation']
num_per_shard = int(math.ceil(len(filenames) / float(_NUM_SHARDS)))
with tf.Graph().as_default():
for shard_id in range(_NUM_SHARDS):
output_filename = _get_dataset_filename(dataset_path, split_name, shard_id)
with tf.python_io.TFRecordWriter(output_filename) as tfrecord_writer:
start_ndx = shard_id * num_per_shard
end_ndx = min((shard_id+1) * num_per_shard, len(filenames))
for i in range(start_ndx, end_ndx):
sys.stdout.write('\r>> Converting image %d/%d shard %d' % (
i+1, len(filenames), shard_id))
sys.stdout.flush()
img = load_image(filenames[i])
image_data = tf.compat.as_bytes(img.tostring())
label = labels[i]
example = image_to_tfexample_mine(image_data, image_format, height, width, label)
# Serialize to string and write on the file
tfrecord_writer.write(example.SerializeToString())
sys.stdout.write('\n')
sys.stdout.flush()
def run(dataset_dir):
labels = Read_Labels(dataset_dir + '/training_labels.csv')
photo_filenames = _get_filenames_and_classes(dataset_dir + '/images_training')
shuffle_data = True
photo_filenames, labels = Shuffle_images_with_labels(
shuffle_data,photo_filenames, labels)
training_filenames = photo_filenames[_NUM_VALIDATION:]
training_labels = labels[_NUM_VALIDATION:]
validation_filenames = photo_filenames[:_NUM_VALIDATION]
validation_labels = labels[:_NUM_VALIDATION]
_convert_dataset('train',
training_filenames, training_labels, dataset_path)
_convert_dataset('validation',
validation_filenames, validation_labels, dataset_path)
print('\nFinished converting the Flowers dataset!')`
And I decode it by:
`
with tf.Session() as sess:
feature = {
'image/encoded': tf.FixedLenFeature((), tf.string, default_value=''),
'image/format': tf.FixedLenFeature((), tf.string, default_value='jpeg'),
'image/class/label': tf.FixedLenFeature(
[37,], tf.float32, default_value=tf.zeros([37,], dtype=tf.float32)),
}
filename_queue = tf.train.string_input_producer([data_path], num_epochs=1)
reader = tf.TFRecordReader()
_, serialized_example = reader.read(filename_queue)
features = tf.parse_single_example(serialized_example, features=feature)
image = tf.decode_raw(features['image/encoded'], tf.float32)
print(image.get_shape())
label = tf.cast(features['image/class/label'], tf.float32)
image = tf.reshape(image, [224, 224, 3])
images, labels = tf.train.shuffle_batch([image, label], batch_size=10, capacity=30, num_threads=1, min_after_dequeue=10)
init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
sess.run(init_op)
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
for batch_index in range(6):
img, lbl = sess.run([images, labels])
img = img.astype(np.uint8)
print(img.shape)
for j in range(6):
plt.subplot(2, 3, j+1)
plt.imshow(img[j, ...])
plt.show()
coord.request_stop()
coord.join(threads)`
It's all fine up to this point. But when I use the bellow commands for decoding TFRecord files:
`
reader = tf.TFRecordReader
keys_to_features = {
'image/encoded': tf.FixedLenFeature((), tf.string, default_value=''),
'image/format': tf.FixedLenFeature((), tf.string, default_value='raw'),
'image/class/label': tf.FixedLenFeature(
[37,], tf.float32, default_value=tf.zeros([37,], dtype=tf.float32)),
}
items_to_handlers = {
'image': slim.tfexample_decoder.Image('image/encoded'),
'label': slim.tfexample_decoder.Tensor('image/class/label'),
}
decoder = slim.tfexample_decoder.TFExampleDecoder(
keys_to_features, items_to_handlers)`
I get the following error.
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.InvalidArgumentError'>, assertion failed: [Unable to decode bytes as JPEG, PNG, GIF, or BMP]
[[Node: case/If_0/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert = Assert[T=[DT_STRING], summarize=3, _device="/job:localhost/replica:0/task:0/cpu:0"](case/If_0/decode_image/cond_jpeg/cond_png/cond_gif/is_bmp, case/If_0/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert/data_0)]]
INFO:tensorflow:Caught OutOfRangeError. Stopping Training.
INFO:sensorflow:Finished training! Saving model to disk.
To use Densenet for my problem, I should fix this error first.
Could you please help me out of this problem. This code works perfectly for the datasets like flowers, MNIST and CIFAR10 available at https://github.com/pudae/tensorflow-densenet/tree/master/datasets but does not work for my data.
Hello, like the title i need some help with "data.py" file
When i run the original "data.py" file that clone from here it go error like below
Traceback (most recent call last):
File "/Users/NGUYENQUANGHUY/PycharmProjects/stanford-tensorflow-tutorials/assignments/chatbot/data.py", line 255, in <module>
prepare_raw_data()
File "/Users/NGUYENQUANGHUY/PycharmProjects/stanford-tensorflow-tutorials/assignments/chatbot/data.py", line 178, in prepare_raw_data
id2line = get_lines()
File "/Users/NGUYENQUANGHUY/PycharmProjects/stanford-tensorflow-tutorials/assignments/chatbot/data.py", line 34, in get_lines
parts = line.split(' +++$+++ ')
TypeError: a bytes-like object is required, not 'str'
So i try do not open "movie_lines.txt" file in binary mode, but this time i got the below error
Traceback (most recent call last):
File "/Users/NGUYENQUANGHUY/PycharmProjects/stanford-tensorflow-tutorials/assignments/chatbot/data.py", line 255, in <module>
prepare_raw_data()
File "/Users/NGUYENQUANGHUY/PycharmProjects/stanford-tensorflow-tutorials/assignments/chatbot/data.py", line 178, in prepare_raw_data
id2line = get_lines()
File "/Users/NGUYENQUANGHUY/PycharmProjects/stanford-tensorflow-tutorials/assignments/chatbot/data.py", line 32, in get_lines
lines = f.readlines()
File "/Users/NGUYENQUANGHUY/.pyenv/versions/anaconda3-4.0.0/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xad in position 3767: invalid start byte
Then i decided to change some string variable to bytes. So my "data.py" file become like this
from __future__ import print_function
import os
import random
import re
import numpy as np
import config
def get_lines():
id2line = {}
file_path = os.path.join(config.DATA_PATH, config.LINE_FILE)
with open(file_path, 'rb') as f:
lines = f.readlines()
for line in lines:
parts = line.split(b' +++$+++ ')
if len(parts) == 5:
if parts[4][-1] == '\n':
parts[4] = parts[4][:-1]
id2line[parts[0]] = parts[4]
return id2line
def get_convos():
""" Get conversations from the raw data """
file_path = os.path.join(config.DATA_PATH, config.CONVO_FILE)
convos = []
with open(file_path, 'rb') as f:
for line in f.readlines():
parts = line.split(b' +++$+++ ')
if len(parts) == 4:
convo = []
for line in parts[3][1:-2].split(b', '):
convo.append(line[1:-1])
convos.append(convo)
return convos
def question_answers(id2line, convos):
""" Divide the dataset into two sets: questions and answers. """
questions, answers = [], []
for convo in convos:
for index, line in enumerate(convo[:-1]):
questions.append(id2line[convo[index]])
answers.append(id2line[convo[index + 1]])
assert len(questions) == len(answers)
return questions, answers
def prepare_dataset(questions, answers):
# create path to store all the train & test encoder & decoder
make_dir(config.PROCESSED_PATH)
# random convos to create the test set
test_ids = random.sample([i for i in range(len(questions))],config.TESTSET_SIZE)
filenames = ['train.enc', 'train.dec', 'test.enc', 'test.dec']
files = []
for filename in filenames:
files.append(open(os.path.join(config.PROCESSED_PATH, filename),'wb'))
for i in range(len(questions)):
if i in test_ids:
files[2].write(questions[i] + b'\n')
files[3].write(answers[i] + b'\n')
else:
files[0].write(questions[i] + b'\n')
files[1].write(answers[i] + b'\n')
for file in files:
file.close()
def make_dir(path):
""" Create a directory if there isn't one already. """
try:
os.mkdir(path)
except OSError:
pass
def basic_tokenizer(line, normalize_digits=True):
""" A basic tokenizer to tokenize text into tokens.
Feel free to change this to suit your need. """
line = re.sub(b'<u>', b'', line)
line = re.sub(b'</u>', b'', line)
line = re.sub(b'\[', b'', line)
line = re.sub(b'\]', b'', line)
words = []
_WORD_SPLIT = re.compile(b"([.,!?\"'-<>:;)(])")
_DIGIT_RE = re.compile(b"\d")
for fragment in line.strip().lower().split():
for token in re.split(_WORD_SPLIT, fragment):
if not token:
continue
if normalize_digits:
token = re.sub(_DIGIT_RE, b'#', token)
words.append(token)
return words
def build_vocab(filename, normalize_digits=True):
in_path = os.path.join(config.PROCESSED_PATH, filename)
out_path = os.path.join(config.PROCESSED_PATH, 'vocab.{}'.format(filename[-3:]))
vocab = {}
with open(in_path, 'rb') as f:
for line in f.readlines():
for token in basic_tokenizer(line):
if not token in vocab:
vocab[token] = 0
vocab[token] += 1
sorted_vocab = sorted(vocab, key=vocab.get, reverse=True)
with open(out_path, 'wb') as f:
f.write(b'<pad>' + b'\n')
f.write(b'<unk>' + b'\n')
f.write(b'<s>' + b'\n')
f.write(b'<\s>' + b'\n')
index = 4
for word in sorted_vocab:
if vocab[word] < config.THRESHOLD:
with open('config.py', 'a') as cf:
if filename[-3:] == 'enc':
cf.write('ENC_VOCAB = ' + str(index) + '\n')
else:
cf.write('DEC_VOCAB = ' + str(index) + '\n')
break
f.write(bytes(word) + b'\n')
index += 1
def load_vocab(vocab_path):
with open(vocab_path, 'rb') as f:
words = f.read().splitlines()
return words, {words[i]: i for i in range(len(words))}
def sentence2id(vocab, line):
return [vocab.get(token, vocab[b'<unk>']) for token in basic_tokenizer(line)]
def token2id(data, mode):
""" Convert all the tokens in the data into their corresponding
index in the vocabulary. """
vocab_path = 'vocab.' + mode
in_path = data + '.' + mode
out_path = data + '_ids.' + mode
_, vocab = load_vocab(os.path.join(config.PROCESSED_PATH, vocab_path))
in_file = open(os.path.join(config.PROCESSED_PATH, in_path), 'rb')
out_file = open(os.path.join(config.PROCESSED_PATH, out_path), 'wb')
lines = in_file.read().splitlines()
for line in lines:
if mode == 'dec': # we only care about '<s>' and </s> in encoder
ids = [vocab[b'<s>']]
else:
ids = []
ids.extend(sentence2id(vocab, line))
# ids.extend([vocab.get(token, vocab['<unk>']) for token in basic_tokenizer(line)])
if mode == 'dec':
ids.append(vocab[b'<\s>'])
out_file.write(b' '.join(bytes(id_) for id_ in ids) + b'\n')
def prepare_raw_data():
print('Preparing raw data into train set and test set ...')
id2line = get_lines()
convos = get_convos()
questions, answers = question_answers(id2line, convos)
prepare_dataset(questions, answers)
def process_data():
print('Preparing data to be model-ready ...')
build_vocab('train.enc')
build_vocab('train.dec')
token2id('train', 'enc')
token2id('train', 'dec')
token2id('test', 'enc')
token2id('test', 'dec')
def load_data(enc_filename, dec_filename, max_training_size=None):
encode_file = open(os.path.join(config.PROCESSED_PATH, enc_filename), 'rb')
decode_file = open(os.path.join(config.PROCESSED_PATH, dec_filename), 'rb')
encode, decode = encode_file.readline(), decode_file.readline()
data_buckets = [[] for _ in config.BUCKETS]
i = 0
while encode and decode:
if (i + 1) % 10000 == 0:
print("Bucketing conversation number", i)
encode_ids = [str(id_) for id_ in encode.split()]
decode_ids = [str(id_) for id_ in decode.split()]
for bucket_id, (encode_max_size, decode_max_size) in enumerate(config.BUCKETS):
if len(encode_ids) <= encode_max_size and len(decode_ids) <= decode_max_size:
data_buckets[bucket_id].append([encode_ids, decode_ids])
break
encode, decode = encode_file.readline(), decode_file.readline()
i += 1
return data_buckets
def _pad_input(input_, size):
return input_ + [config.PAD_ID] * (size - len(input_))
def _reshape_batch(inputs, size, batch_size):
""" Create batch-major inputs. Batch inputs are just re-indexed inputs
"""
batch_inputs = []
for length_id in range(size):
batch_inputs.append(np.array([inputs[batch_id][length_id]
for batch_id in range(batch_size)], dtype=str))
return batch_inputs
def get_batch(data_bucket, bucket_id, batch_size=1):
""" Return one batch to feed into the model """
# only pad to the max length of the bucket
encoder_size, decoder_size = config.BUCKETS[bucket_id]
encoder_inputs, decoder_inputs = [], []
for _ in range(batch_size):
encoder_input, decoder_input = random.choice(data_bucket)
# pad both encoder and decoder, reverse the encoder
encoder_inputs.append(list(reversed(_pad_input(encoder_input, encoder_size))))
decoder_inputs.append(_pad_input(decoder_input, decoder_size))
# now we create batch-major vectors from the data selected above.
batch_encoder_inputs = _reshape_batch(encoder_inputs, encoder_size, batch_size)
batch_decoder_inputs = _reshape_batch(decoder_inputs, decoder_size, batch_size)
# create decoder_masks to be 0 for decoders that are padding.
batch_masks = []
for length_id in range(decoder_size):
batch_mask = np.ones(batch_size, dtype=np.float32)
for batch_id in range(batch_size):
# we set mask to 0 if the corresponding target is a PAD symbol.
# the corresponding decoder is decoder_input shifted by 1 forward.
if length_id < decoder_size - 1:
target = decoder_inputs[batch_id][length_id + 1]
if length_id == decoder_size - 1 or target == config.PAD_ID:
batch_mask[batch_id] = 0.0
batch_masks.append(batch_mask)
return batch_encoder_inputs, batch_decoder_inputs, batch_masks
if __name__ == '__main__':
prepare_raw_data()
process_data()
With the changing this "data.py" file can run smoothly, but i found that i have some problems with this dataset afterward. Actually i got many errors in training step but i think the reason for that errors come from dataset so i appreciate it if you guy can take a look for my post. Thank you.
I couldn't find the text data here. Could you please provide the url for the data set? Thank you.
I notice this file is missing?
thanks
I have trouble run chatbot code when I update tensorflow.
no module seq2seq.model_with_buckets
python style_transfer.py
Dataset ready
2017-06-12 09:57:17.190398: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-12 09:57:17.374553: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-12 09:57:17.374598: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-06-12 09:57:17.374646: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Traceback (most recent call last):
File "style_transfer.py", line 195, in
main()
File "style_transfer.py", line 184, in main
input_image, content_image, style_image)
File "style_transfer.py", line 102, in _create_losses
p = sess.run(model[CONTENT_LAYER])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 778, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 969, in _run
fetch_handler = _FetchHandler(self._graph, fetches, feed_dict_string)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 408, in init
self._fetch_mapper = _FetchMapper.for_fetch(fetches)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 227, in for_fetch
(fetch, type(fetch)))
TypeError: Fetch argument None has invalid type <type 'NoneType'>
H,
I type "python 03_logistic_regression_mnist_sol.py"
the error is
File "03_logistic_regression_mnist_sol.py", line 87, in
total_correct_preds += accuracy_batch
TypeError: unsupported operand type(s) for +=: 'int' and 'list'
what is problem?
Thanks
Hi,
We have successfully trained stanford chatbot using cornell movie dialog corpus.But it is giving random answers.We are trying to use Ubuntu Dialog Corpus dataset but we are unable to pre-process it .How can we change the format similar to cornell movie dialog corpus.
Otherwise,Can you please suggest any other dataset which is similar to cornell movie dialog corpus.
Thanks in advance.
Traceback (most recent call last):
File "D:/stanford_tensorflow_tutorials/tf_oreilly/01_linear_regression_sol.py", line 19, in
book = xlrd.open_workbook(DATA_FILE, encoding_override="utf-8")
File "C:\Anaconda3\lib\site-packages\xlrd_init_.py", line 441, in open_workbook
ragged_rows=ragged_rows,
File "C:\Anaconda3\lib\site-packages\xlrd\book.py", line 107, in open_workbook_xls
bk.fake_globals_get_sheet()
File "C:\Anaconda3\lib\site-packages\xlrd\book.py", line 687, in fake_globals_get_sheet
self.get_sheets()
File "C:\Anaconda3\lib\site-packages\xlrd\book.py", line 678, in get_sheets
self.get_sheet(sheetno)
File "C:\Anaconda3\lib\site-packages\xlrd\book.py", line 669, in get_sheet
sh.read(self)
File "C:\Anaconda3\lib\site-packages\xlrd\sheet.py", line 1475, in read
self.update_cooked_mag_factors()
File "C:\Anaconda3\lib\site-packages\xlrd\sheet.py", line 1543, in update_cooked_mag_factors
elif not (10 <= zoom <= 400):
TypeError: unorderable types: int() <= NoneType()
Hi everyone, I am new to tensorflow, can anyone guide me in solving this error? Is it because of old version of tensorflow?
Data ready!
Bucketing conversation number 9999
Bucketing conversation number 19999
Bucketing conversation number 29999
Bucketing conversation number 39999
Bucketing conversation number 49999
Bucketing conversation number 59999
Bucketing conversation number 69999
Bucketing conversation number 79999
Bucketing conversation number 89999
Bucketing conversation number 99999
Bucketing conversation number 109999
Bucketing conversation number 119999
Bucketing conversation number 129999
Bucketing conversation number 139999
Bucketing conversation number 149999
Bucketing conversation number 159999
Bucketing conversation number 169999
Bucketing conversation number 179999
Bucketing conversation number 189999
Number of samples in each bucket:
[103411]
Bucket scale:
[1.0]
Initialize new model
Create placeholders
Create inference
Creating loss...
It might take a couple of minutes depending on how many buckets you have.
Traceback (most recent call last):
File "chatbot.py", line 262, in <module>
main()
File "chatbot.py", line 257, in main
train()
File "chatbot.py", line 138, in train
model.build_graph()
File "/home/bot/tensorflow/model.py", line 132, in build_graph
self._create_loss()
File "/home/bot/tensorflow/model.py", line 93, in _create_loss
self.outputs, self.losses = tf.nn.seq2seq.model_with_buckets(
AttributeError: 'module' object has no attribute 'seq2seq'
Thank you so much for spending your time. I really appreciate your help
hi, here is a suggestion, how about add a "Jupyter Notebook" in example PATH?
I got a lot of errors where Python doesn't like doing things interchangably with strings and bytearrays. If fixed them and got running. If you want the fixed file its here: https://drive.google.com/open?id=0B3d1md_6mdBRcVlraVRsWHUyR0U
after changing all 'rb' and 'bw' to 'r' and 'b' it gives the following error
File "data.py", line 76, in prepare_dataset
files.append(open(os.path.join(config.PROCESSED_PATH, filename),'b'))
ValueError: Must have exactly one of create/read/write/append mode and at most one plus
does anybody knows? what to do
Hello, please help
then i run 07_convnet_mnist.py graph looks good
but then i run very similar code with my data (code below), graph looks awfully
and may be it absolutely another problem but accuracy not increase
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from data3 import DataSet
import os
import time
import tensorflow as tf
width = 28
height = 28
ds = DataSet(batch=128, w=width, h=height)
ds.train_directory = '/home/alexander/PycharmProjects/Diplom/DB/train'
ds.test_directory = '/home/alexander/PycharmProjects/Diplom/DB/test'
N_CLASSES = 4
LEARNING_RATE = 0.001
BATCH_SIZE = 128
SKIP_STEP = 10
DROPOUT = 0.75
N_EPOCHS = 1
with tf.name_scope('data'):
X = tf.placeholder(tf.float32, [None, 784], name="X_placeholder")
Y = tf.placeholder(tf.float32, [None, N_CLASSES], name="Y_placeholder")
global_step = tf.Variable(0, dtype=tf.int32, trainable=False, name='global_step')
with tf.variable_scope('conv1') as scope:
images = tf.reshape(X, shape=[-1, width, height, 1])
kernel = tf.get_variable('kernel', [5, 5, 1, 32],
initializer=tf.truncated_normal_initializer())
biases = tf.get_variable('biases', [32],
initializer=tf.random_normal_initializer())
conv = tf.nn.conv2d(images, kernel, strides=[1, 1, 1, 1], padding='SAME')
conv1 = tf.nn.relu(conv + biases, name=scope.name)
with tf.variable_scope('pool1') as scope:
pool1 = tf.nn.max_pool(conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
padding='SAME')
with tf.variable_scope('conv2') as scope:
kernel = tf.get_variable('kernels', [5, 5, 32, 64],
initializer=tf.truncated_normal_initializer())
biases = tf.get_variable('biases', [64],
initializer=tf.random_normal_initializer())
conv = tf.nn.conv2d(pool1, kernel, strides=[1, 1, 1, 1], padding='SAME')
conv2 = tf.nn.relu(conv + biases, name=scope.name)
with tf.variable_scope('pool2') as scope:
pool2 = tf.nn.max_pool(conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
padding='SAME')
with tf.variable_scope('fc') as scope:
input_features = 7 * 7 * 64
w = tf.get_variable('weights', [input_features, 1024],
initializer=tf.truncated_normal_initializer())
b = tf.get_variable('biases', [1024],
initializer=tf.random_normal_initializer())
pool2 = tf.reshape(pool2, [-1, input_features])
fc = tf.nn.relu(tf.matmul(pool2, w) + b, name='relu')
fc = tf.nn.dropout(fc, 0.75, name='relu_dropout')
with tf.variable_scope('softmax_linear') as scope:
w = tf.get_variable('weights', [1024, N_CLASSES],
initializer=tf.truncated_normal_initializer())
b = tf.get_variable('biases', [N_CLASSES],
initializer=tf.random_normal_initializer())
logits = tf.matmul(fc, w) + b
with tf.name_scope('loss'):
entropy = tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y)
loss = tf.reduce_mean(entropy, name='loss')
optimizer = tf.train.AdamOptimizer(LEARNING_RATE).minimize(loss,
global_step=global_step)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
saver = tf.train.Saver()
writer = tf.summary.FileWriter('tb', sess.graph)
ckpt = tf.train.get_checkpoint_state(os.path.dirname('tb/checkpoint'))
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
initial_step = global_step.eval()
start_time = time.time()
n_batches = int(ds.num_train_files / BATCH_SIZE)
total_loss = 0.0
total_correct_preds = 0
for index in range(initial_step, n_batches * N_EPOCHS):
X_batch, Y_batch = ds.next_train_batch()
_, loss_batch, logits_batch = sess.run([optimizer, loss, logits],
feed_dict={X: X_batch, Y: Y_batch})
preds = tf.nn.softmax(logits_batch)
correct_preds = tf.equal(tf.argmax(preds, 1), tf.argmax(Y_batch, 1))
accuracy = tf.reduce_sum(tf.cast(correct_preds, tf.float32))
total_correct_preds += sess.run(accuracy)
total_loss += loss_batch
if (index + 1) % SKIP_STEP == 0:
print('Average loss at step {}: {:5.1f}'.format(index + 1, total_loss / SKIP_STEP))
print('Accuracy: ', total_correct_preds / (BATCH_SIZE * 10))
total_loss = 0.0
total_correct_preds = 0
saver.save(sess, 'tb', index)
print("Optimization Finished!")
print("Total time: {0} seconds".format(time.time() - start_time))
n_batches = int(ds.num_test_files / BATCH_SIZE)
total_correct_preds = 0
for i in range(n_batches):
X_batch, Y_batch = ds.next_test_batch()
_, loss_batch, logits_batch = sess.run([optimizer, loss, logits],
feed_dict={X: X_batch, Y: Y_batch})
preds = tf.nn.softmax(logits_batch)
correct_preds = tf.equal(tf.argmax(preds, 1), tf.argmax(Y_batch, 1))
accuracy = tf.reduce_sum(tf.cast(correct_preds, tf.float32))
total_correct_preds += sess.run(accuracy)
print("Accuracy {0}".format(total_correct_preds / ds.num_test_files))
self.outputs, self.losses = tf.nn.seq2seq.model_with_buckets(
self.encoder_inputs,
self.decoder_inputs,
self.targets,
self.decoder_masks,
config.BUCKETS,
lambda x, y: _seq2seq_f(x, y, False),
softmax_loss_function=self.softmax_loss_function)
what function we can replace with tf.nn.seq2seq.model_with_buckets? thanks
Hello Chip,
I am a Stanford student following your class. Thanks a lot for the amazing sample code gists which are very useful. I had a small suggestion in 11_char_rnn_gist.py at Line 32. I believe, for start in range(0, len(text) - window, overlap): the overlap should be (window-overlap)?
Hi, could you please post week 4 notes/slides & assignment 2 on the web site?
Thanks!
In the file /examples/04_word2vec_starter.py the "with" keyword is missing on 3 lines, when defining scopes. Lines are 35, 44, 52.
Each line is repeated 72 times. I guess this will cripple the expected performance on the network as there is a really small number of distinct training samples. Is this by design for the course?
When I ran the 04_word2vec_starter.py and 04_word2vec_no_frills.py and 04_word2vec_visualize.py, functions in the process_data.py are called, and occured the following error messages:
Dataset ready
Traceback (most recent call last):
File "/home/luffy/Desktop/tf-stanford-tutorials/examples/04_word2vec_visualize.py", line 151, in
main()
File "/home/luffy/Desktop/tf-stanford-tutorials/examples/04_word2vec_visualize.py", line 147, in main
batch_gen = process_data(VOCAB_SIZE, BATCH_SIZE, SKIP_WINDOW)
File "/home/luffy/Desktop/tf-stanford-tutorials/examples/process_data.py", line 87, in process_data
dictionary, _ = build_vocab(words, vocab_size)
File "/home/luffy/Desktop/tf-stanford-tutorials/examples/process_data.py", line 50, in build_vocab
with open('processed/vocab_1000.tsv', "w") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'processed/vocab_1000.tsv'
It seems that the text8.zip is successfully downloaded, but where is the processed/vocab_1000.tsv ?
File "/Users/apple/tensorflow/stanford-tensorflow-tutorials/assignments/chatbot/model.py", line 51, in _inference
if config.NUM_SAMPLES > 0 and config.NUM_SAMPLES < config.DEC_VOCAB:
AttributeError: module 'config' has no attribute 'AttributeError: module 'config' has no attribute 'DEC_VOCAB''
I cannot file DEC_VOCAB and NUM_SAMPLES in config file, which number should i use for both value?
I change it to below to make it pass:
mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
maybe this question is already answered.....How long is average training time? should i interrupt the training?, its now training since 6 days and on iter 4374600......thanks
update:
training data:
THRESHOLD = 2
PAD_ID = 0
UNK_ID = 1
START_ID = 2
EOS_ID = 3
TESTSET_SIZE = 25000
BUCKETS = [(16, 19)]
NUM_LAYERS = 3
HIDDEN_SIZE = 256
BATCH_SIZE = 64
LR = 0.5
MAX_GRAD_NORM = 5.0
NUM_SAMPLES = 512
i interrupted the training, the conversation content looks not really good....
Data ready!
Initialize new model
Create placeholders
Create inference
Creating loss...
It might take a couple of minutes depending on how many buckets you have.
Time: 2.98553586006
Create optimizer...
It might take a couple of minutes depending on how many buckets you have.
Loading parameters for the Chatbot
Welcome to TensorBro. Say something. Enter to exit. Max length is 16
> hey
[[-0.9518705 9.31446075 -0.70666641 ..., -0.87015301 -0.81906992
-0.90472138]]
> hello
[[-0.82278359 9.14360237 -0.60981351 ..., -0.74742228 -0.81174237
-0.78287232]]
i ' m .
> hey you
[[-0.88960081 9.31346798 -0.64204782 ..., -0.73884577 -0.83127314
-0.81598616]]
i ' .
> Hey
[[-0.95187062 9.31446075 -0.70666641 ..., -0.87015295 -0.81906992
-0.90472144]]
> Hey how are You?
[[-0.96689004 9.22152805 -0.65112358 ..., -0.72538173 -0.89104617
-0.75631899]]
' .
> hi.
[[-0.9177267 9.12739277 -0.68629146 ..., -0.84158963 -0.79811877
-0.86087358]]
' re .
Hi, I'm reading the code of 11_char_rnn_gist.py, and I found the following problem:
In line 57, we encode the sequence seq with one-hot code with depth=len(vocab).
However, seq is generated with [vocab.index(x) + 1 for x in text if x in vocab], so the code of characters is between 1 to len(vocab), then we pad them with 0. So with tf.one_hot, the last character in vocab is neglected, and the PAD symbol is encoded to [1 0 0 0 ...].
When we run the demo, it seems ok because the last character } hardly appears in our dataset.
If we change vocab from (let a be the last character in 'vocab')
vocab = (
" $%'()+,-./0123456789:;=?ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"\\^_abcdefghijklmnopqrstuvwxyz{|}")
to
vocab = (
" $%'()+,-./0123456789:;=?ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"\\^_bcdefghijklmnopqrstuvwxyz{|}a")
Then the outputs are completely non sense (it outputs something like T8WrtP sVM -reca5 r, ...), while it should work as before.
Hi,
in note_03.pdf, p13, in the testing model part, the code states:
_, loss_batch, logits_batch = sess.run([optimizer, loss, logits], feed_dict={X: X_batch, Y:Y_batch}).
Does this call to optimizer trains the model on the test set and has an impact on the next epoch?
Thanks for your great course material.
When trying to run this code, I get the error below. The zip file looks okay to me, though.
(tensorflow) adamg:examples adamg$ python 04_word2vec_visualize.py
Dataset ready
Traceback (most recent call last):
File "04_word2vec_visualize.py", line 157, in <module>
main()
File "04_word2vec_visualize.py", line 153, in main
batch_gen = process_data(VOCAB_SIZE, BATCH_SIZE, SKIP_WINDOW)
File "/Users/adamg/Dropbox/Programming Tutorials/MachineLearning/TensorFlow/Stanford-tf-repo/examples/process_data.py", line 90, in process_data
words = read_data(file_path)
File "/Users/adamg/Dropbox/Programming Tutorials/MachineLearning/TensorFlow/Stanford-tf-repo/examples/process_data.py", line 43, in read_data
with zipfile.ZipFile(file_path) as f:
File "/Users/adamg/anaconda/envs/tensorflow/lib/python3.5/zipfile.py", line 1026, in __init__
self._RealGetContents()
File "/Users/adamg/anaconda/envs/tensorflow/lib/python3.5/zipfile.py", line 1094, in _RealGetContents
raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file
I tried to add an additional variable 'u' as the notes_03.pdf suggested, however the loss and parameters became very large so that the final output is 'nan'. I tried to use another learning rate, e.g 1e^10 which can help to get specific values. However, I don't know why the loss will be very large (inf) with the square loss function. I mean even if the learning rate is very big, the loss will fluctuate between a range instead of getting out of limit.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.