Hi, there are some errors in the CBOW example but this one is something I can't solve.

Traceback (most recent call last):
File "05_Working_With_CBOW.py", line 125, in
window_size, method='cbow')
File "/home/test/cbow/text_helpers.py", line 97, in generate_batch_data
batch, labels = [list(x) for x in zip(*batch_and_labels)]
ValueError: not enough values to unpack (expected 2, got 0)

Thanks,

In line 136:
self.lstm_cell = tf.contrib.rnn.BasicLSTMCell(rnn_size)
change to
self.lstm_cell = tf.contrib.rnn.BasicLSTMCell(rnn_size, reuse=tf.get_variable_scope().reuse)

In line 137:
self.lstm_cell = tf.contrib.rnn.MultiRNNCell([self.lstm_cell] * self.num_layers)
change to
self.lstm_cell = tf.contrib.rnn.MultiRNNCell([self.lstm_cell for self.lstm_cell in range(num_layers)])

In line 138:
self.initial_state = self.lstm_cell.zero_state(self.batch_size, tf.float32)

Please help, as I am getting the error:
/core_rnn_cell_impl.py", line 929, in
return tuple(cell.zero_state(batch_size, dtype) for cell in self._cells)

AttributeError: 'int' object has no attribute 'zero_state'

Or else:

In line 137:
change to
self.lstm_cell = tf.contrib.rnn.MultiRNNCell([self.lstm_cell for _ in range(num_layers)])

Then an error occurs:

self._value = int(value)

TypeError: int() argument must be a string, a bytes-like object or a number, not 'tuple'

First, I change the type of x_val with np.float32 into y_val. When I run:
mov_avg_layer= tf.nn.conv2d(y_val, my_filter, my_strides,
padding='SAME', name='Moving_Avg_Window')
print(sess.run(mov_avg_layer))
I get the result. But when I try the feed_dict, my kernel runs out.

When I put the codes in my terminal python envirment, my operation system runs wrong.
I use the Gpu size of tensorflow, did this double? Or other things make my feed_dict goes out?
mov_avg_layer= tf.nn.conv2d(x_data, my_filter, my_strides,
padding='SAME', name='Moving_Avg_Window')
print(sess.run(mov_avg_layer,feed_dict={x_data:y_val}))
The above command make my kernel die without any information.

Hi, sorry, i'm absoultely new in Tensorflow and may be telling nonesence.

This example
https://github.com/nfmcclure/tensorflow_cookbook/blob/master/09_Recurrent_Neural_Networks/03_Implementing_LSTM/03_implementing_lstm.py

raise with this error:

AttributeError                            Traceback (most recent call last)
<ipython-input-1-2bd8df52813f> in <module>()
    211     # Define LSTM Model
    212     lstm_model = LSTM_Model(rnn_size, batch_size, learning_rate,
--> 213                             training_seq_len, vocab_size)
    214     scope.reuse_variables()
    215     test_lstm_model = LSTM_Model(rnn_size, batch_size, learning_rate,

<ipython-input-1-2bd8df52813f> in __init__(self, rnn_size, batch_size, learning_rate, training_seq_len, vocab_size, infer_sample)
    134             self.training_seq_len = training_seq_len
    135 
--> 136         self.lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(rnn_size)
    137         self.initial_state = self.lstm_cell.zero_state(self.batch_size, tf.float32)
    138 

AttributeError: 'module' object has no attribute 'rnn_cell'

>>> import pkg_resources
>>> pkg_resources.get_distribution("tensorflow").version
'1.0.0'

import platform
print(platform.python_version())
2.7.6

i'm using this Docker image

tensorflow/tensorflow   latest              ea40dcc45724        2 weeks ago         1.029 GB

thanks

When running:

print('Creating Translation Model')
input_vocab_size = vocab_size
output_vocab_size = vocab_size

translate_model = translation_model(sess=sess,
                                    input_vocab_size=vocab_size,
                                    output_vocab_size=vocab_size,
                                    buckets=buckets,
                                    rnn_size=rnn_size,
                                    num_layers=num_layers,
                                    max_gradient=max_gradient,
                                    learning_rate=learning_rate,
                                    lr_decay_rate=lr_decay_rate,
                                    forward_only=False)

# Tell TensorFlow to reuse the variables for the test model
with tf.variable_scope(tf.get_variable_scope(), reuse=True):
    #Reuse the variables for the test model
    test_model = translation_model(sess=sess,
                                    input_vocab_size=vocab_size,
                                    output_vocab_size=vocab_size,
                                    buckets=buckets,
                                    rnn_size=rnn_size,
                                    num_layers=num_layers,
                                    max_gradient=max_gradient,
                                    learning_rate=learning_rate,
                                    lr_decay_rate=lr_decay_rate,
                                    forward_only=True)
    test_model.batch_size = 1
    

Got output:

Creating Translation Model

---

TypeError Traceback (most recent call last)
in ()
12 learning_rate=learning_rate,
13 lr_decay_rate=lr_decay_rate,
---> 14 forward_only=FCreating Translation Model

TypeError Traceback (most recent call last)
in ()
12 learning_rate=learning_rate,
13 lr_decay_rate=lr_decay_rate,
---> 14 forward_only=False)
15
16 # Tell TensorFlow to reuse the variables for the test model

in translation_model(sess, input_vocab_size, output_vocab_size, buckets, rnn_size, num_layers, max_gradient, learning_rate, lr_decay_rate, forward_only)
14 lr_decay_rate,
15 forward_only=forward_only,
---> 16 dtype=tf.float32)
17 return(model)

~/cookbook_py3/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in init(self, source_vocab_size, target_vocab_size, buckets, size, num_layers, max_gradient_norm, batch_size, learning_rate, learning_rate_decay_factor, use_lstm, num_samples, forward_only, dtype)
177 self.target_weights, buckets,
178 lambda x, y: seq2seq_f(x, y, False),
--> 179 softmax_loss_function=softmax_loss_function)
180
181 # Gradients and SGD update operation for training the model.

~/anaconda2/envs/cookbook_py3/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py in model_with_buckets(encoder_inputs, decoder_inputs, targets, weights, buckets, seq2seq, softmax_loss_function, per_example_loss, name)
1204 variable_scope.get_variable_scope(), reuse=True if j > 0 else None):
1205 bucket_outputs, _ = seq2seq(encoder_inputs[:bucket[0]],
-> 1206 decoder_inputs[:bucket[1]])
1207 outputs.append(bucket_outputs)
1208 if per_example_loss:

~/cookbook_py3/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in (x, y)
176 self.encoder_inputs, self.decoder_inputs, targets,
177 self.target_weights, buckets,
--> 178 lambda x, y: seq2seq_f(x, y, False),
179 softmax_loss_function=softmax_loss_function)
180

~/cookbook_py3/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in seq2seq_f(encoder_inputs, decoder_inputs, do_decode)
140 output_projection=output_projection,
141 feed_previous=do_decode,
--> 142 dtype=dtype)
143
144 # Feeds for inputs.

~/anaconda2/envs/cookbook_py3/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py in embedding_attention_seq2seq(encoder_inputs, decoder_inputs, cell, num_encoder_symbols, num_decoder_symbols, embedding_size, num_heads, output_projection, feed_previous, dtype, scope, initial_state_attention)
846 dtype = scope.dtype
847 # Encoder.
--> 848 encoder_cell = copy.deepcopy(cell)
849 encoder_cell = core_rnn_cell.EmbeddingWrapper(
850 encoder_cell,

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
159 copier = getattr(x, "deepcopy", None)
160 if copier:
--> 161 y = copier(memo)
162 else:
163 reductor = dispatch_table.get(cls)

~/anaconda2/envs/cookbook_py3/lib/python3.6/site-packages/tensorflow/python/layers/base.py in deepcopy(self, memo)
474 setattr(result, k, copy.copy(v))
475 else:
--> 476 setattr(result, k, copy.deepcopy(v, memo))
477 return result
478

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_list(x, memo, deepcopy)
213 append = y.append
214 for a in x:
--> 215 append(deepcopy(a, memo))
216 return y
217 d[list] = _deepcopy_list

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
167 reductor = getattr(x, "reduce_ex", None)
168 if reductor:
--> 169 rv = reductor(4)
170 else:
171 reductor = getattr(x, "reduce", None)

TypeError: can't pickle _thread.lock objectsalse)
15
16 # Tell TensorFlow to reuse the variables for the test model

in translation_model(sess, input_vocab_size, output_vocab_size, buckets, rnn_size, num_layers, max_gradient, learning_rate, lr_decay_rate, forward_only)
14 lr_decay_rate,
15 forward_only=forward_only,
---> 16 dtype=tf.float32)
17 return(model)

~/cookbook_py3/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in init(self, source_vocab_size, target_vocab_size, buckets, size, num_layers, max_gradient_norm, batch_size, learning_rate, learning_rate_decay_factor, use_lstm, num_samples, forward_only, dtype)
177 self.target_weights, buckets,
178 lambda x, y: seq2seq_f(x, y, False),
--> 179 softmax_loss_function=softmax_loss_function)
180
181 # Gradients and SGD update operation for training the model.

~/anaconda2/envs/cookbook_py3/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py in model_with_buckets(encoder_inputs, decoder_inputs, targets, weights, buckets, seq2seq, softmax_loss_function, per_example_loss, name)
1204 variable_scope.get_variable_scope(), reuse=True if j > 0 else None):
1205 bucket_outputs, _ = seq2seq(encoder_inputs[:bucket[0]],
-> 1206 decoder_inputs[:bucket[1]])
1207 outputs.append(bucket_outputs)
1208 if per_example_loss:

~/cookbook_py3/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in (x, y)
176 self.encoder_inputs, self.decoder_inputs, targets,
177 self.target_weights, buckets,
--> 178 lambda x, y: seq2seq_f(x, y, False),
179 softmax_loss_function=softmax_loss_function)
180

~/cookbook_py3/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in seq2seq_f(encoder_inputs, decoder_inputs, do_decode)
140 output_projection=output_projection,
141 feed_previous=do_decode,
--> 142 dtype=dtype)
143
144 # Feeds for inputs.

~/anaconda2/envs/cookbook_py3/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py in embedding_attention_seq2seq(encoder_inputs, decoder_inputs, cell, num_encoder_symbols, num_decoder_symbols, embedding_size, num_heads, output_projection, feed_previous, dtype, scope, initial_state_attention)
846 dtype = scope.dtype
847 # Encoder.
--> 848 encoder_cell = copy.deepcopy(cell)
849 encoder_cell = core_rnn_cell.EmbeddingWrapper(
850 encoder_cell,

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
159 copier = getattr(x, "deepcopy", None)
160 if copier:
--> 161 y = copier(memo)
162 else:
163 reductor = dispatch_table.get(cls)

~/anaconda2/envs/cookbook_py3/lib/python3.6/site-packages/tensorflow/python/layers/base.py in deepcopy(self, memo)
474 setattr(result, k, copy.copy(v))
475 else:
--> 476 setattr(result, k, copy.deepcopy(v, memo))
477 return result
478

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_list(x, memo, deepcopy)
213 append = y.append
214 for a in x:
--> 215 append(deepcopy(a, memo))
216 return y
217 d[list] = _deepcopy_list

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
167 reductor = getattr(x, "reduce_ex", None)
168 if reductor:
--> 169 rv = reductor(4)
170 else:
171 reductor = getattr(x, "reduce", None)

TypeError: can't pickle _thread.lock objects

Creating Translation Model

TypeError Traceback (most recent call last)
in ()
12 learning_rate=learning_rate,
13 lr_decay_rate=lr_decay_rate,
---> 14 forward_only=False)
15
16 # Tell TensorFlow to reuse the variables for the test model

in translation_model(sess, input_vocab_size, output_vocab_size, buckets, rnn_size, num_layers, max_gradient, learning_rate, lr_decay_rate, forward_only)
14 lr_decay_rate,
15 forward_only=forward_only,
---> 16 dtype=tf.float32)
17 return(model)

~/cookbook_py3/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in init(self, source_vocab_size, target_vocab_size, buckets, size, num_layers, max_gradient_norm, batch_size, learning_rate, learning_rate_decay_factor, use_lstm, num_samples, forward_only, dtype)
177 self.target_weights, buckets,
178 lambda x, y: seq2seq_f(x, y, False),
--> 179 softmax_loss_function=softmax_loss_function)
180
181 # Gradients and SGD update operation for training the model.

~/anaconda2/envs/cookbook_py3/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py in model_with_buckets(encoder_inputs, decoder_inputs, targets, weights, buckets, seq2seq, softmax_loss_function, per_example_loss, name)
1204 variable_scope.get_variable_scope(), reuse=True if j > 0 else None):
1205 bucket_outputs, _ = seq2seq(encoder_inputs[:bucket[0]],
-> 1206 decoder_inputs[:bucket[1]])
1207 outputs.append(bucket_outputs)
1208 if per_example_loss:

~/cookbook_py3/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in (x, y)
176 self.encoder_inputs, self.decoder_inputs, targets,
177 self.target_weights, buckets,
--> 178 lambda x, y: seq2seq_f(x, y, False),
179 softmax_loss_function=softmax_loss_function)
180

~/cookbook_py3/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in seq2seq_f(encoder_inputs, decoder_inputs, do_decode)
140 output_projection=output_projection,
141 feed_previous=do_decode,
--> 142 dtype=dtype)
143
144 # Feeds for inputs.

~/anaconda2/envs/cookbook_py3/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py in embedding_attention_seq2seq(encoder_inputs, decoder_inputs, cell, num_encoder_symbols, num_decoder_symbols, embedding_size, num_heads, output_projection, feed_previous, dtype, scope, initial_state_attention)
846 dtype = scope.dtype
847 # Encoder.
--> 848 encoder_cell = copy.deepcopy(cell)
849 encoder_cell = core_rnn_cell.EmbeddingWrapper(
850 encoder_cell,

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
159 copier = getattr(x, "deepcopy", None)
160 if copier:
--> 161 y = copier(memo)
162 else:
163 reductor = dispatch_table.get(cls)

~/anaconda2/envs/cookbook_py3/lib/python3.6/site-packages/tensorflow/python/layers/base.py in deepcopy(self, memo)
474 setattr(result, k, copy.copy(v))
475 else:
--> 476 setattr(result, k, copy.deepcopy(v, memo))
477 return result
478

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_list(x, memo, deepcopy)
213 append = y.append
214 for a in x:
--> 215 append(deepcopy(a, memo))
216 return y
217 d[list] = _deepcopy_list

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
178 y = x
179 else:
--> 180 y = _reconstruct(x, memo, *rv)
181
182 # If is its own copy, don't memoize.

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _reconstruct(x, memo, func, args, state, listiter, dictiter, deepcopy)
278 if state is not None:
279 if deep:
--> 280 state = deepcopy(state, memo)
281 if hasattr(y, 'setstate'):
282 y.setstate(state)

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
148 copier = _deepcopy_dispatch.get(cls)
149 if copier:
--> 150 y = copier(x, memo)
151 else:
152 try:

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in _deepcopy_dict(x, memo, deepcopy)
238 memo[id(x)] = y
239 for key, value in x.items():
--> 240 y[deepcopy(key, memo)] = deepcopy(value, memo)
241 return y
242 d[dict] = _deepcopy_dict

~/anaconda2/envs/cookbook_py3/lib/python3.6/copy.py in deepcopy(x, memo, _nil)
167 reductor = getattr(x, "reduce_ex", None)
168 if reductor:
--> 169 rv = reductor(4)
170 else:
171 reductor = getattr(x, "reduce", None)

TypeError: can't pickle _thread.lock objects

I use ubuntu 16.04, tensorflow1.2.1, python3.6,

Hello! I have a question about the mlp network which has around 200 parameters. But I check the sample size is less than 200. Depending on the seeds of random number, it's able to get converged results sometimes. my question is that how is it possible and is it the optimal model to run regression analysis to get the birth weight? Thanks!

Hi ,
you are using formulas given below to calculate your accuracy.

prediction_output = tf.matmul(tf.mul(tf.transpose(y_target),b), pred_kernel)
prediction = tf.sign(prediction_output-tf.reduce_mean(prediction_output))
accuracy = tf.reduce_mean(tf.cast(tf.equal(tf.squeeze(prediction), tf.squeeze(y_target)), tf.float32))

you are using directly y_target (ground truth) to make a prediction and then comparing the prediction with y_target.
Firstly I dont understand what is going on there. I mean "predicting by using target value" sounds meaningless a little bit. Probably I am missing something, but I couldnt find what it is.

At the end of the day, if I want to predict output when test input data are applied without knowing output how can I predict my outputs ?

By the way, there is a line in the code

model_output = tf.matmul(b, my_kernel)

But you dont use model_output anywhere in your code.

Thanks.

The examle with default 50 batch size can't converge, so I replace it with 10 or 15 receiving better condition.
I think the batch size matters, but don't understand.
And I have another question, the y_target has the shape [None, 1], and the final_ouput has the shape [1].
Why can they subtract with each other?

Clicking on the link "Low Birthweight Data" in the 07_Working_with_Data_Sources directory gets:

You don't have permission to access /statdata/statdata/data/lowbwt.txt on this server.

rnn_size = 128 # RNN Model size, has to equal embedding size

I don't know why there has to be equal between these two hyper-params? could you be kind to explain it for us, thanks.

Dear Nick,
I started reading your book entitled “TensorFlow Machine Learning Cookbook” about a month ago, and I should say it's a well-written book.

I really liked some of the ideas that you wrote and I would like to share them with students in my country. I would like to request your permission as author so that I can translate some parts of your book into the Persian language with some modifications. The modifications are required to provide context to ensure the translation is understood correctly. I am not doing this for financial gain; I would just like students in my country become more familiar with the topic and your book provides an excellent resource on this subject. Needless to say, I will keep all links and give proper attribution to you as the author. The translated sections will be published on my blog, and I can provide you with the link once complete.

I really appreciate you considering my request. If you have any questions regarding the intent of my translation or the exact sections I wish to translate, please do not hesitate to ask. I look forward to receiving your response.

Make sure that all significant models save the model space (graph, variables, and definitions) properly and have a script that allows for (1) loading of most recent model, and (2) evaluation of new input.

Hi, I am wondering Is it possible to use the new dynamic_rnn_decoder to replace the legacy rnn decoder in rnn chapter for text generation, and update the code?

Looks like the ipynb files are using some deprecated functions:

For section: Create and Format Tensorboard outputs for viewing

Error:

AttributeError Traceback (most recent call last)
in ()
----> 1 merged = tf.merge_all_summaries()
2
3 if not os.path.exists('tensorboard_logs/'):
4 os.makedirs('tensorboard_logs/')
5

AttributeError: module 'tensorflow' has no attribute 'merge_all_summaries'

the following change fixes it:

# merged = tf.merge_all_summaries()
merged = tf.summary.merge_all(key='summaries')

Error:

AttributeError Traceback (most recent call last)
in ()
6
7
----> 8 my_writer = tf.train.SummaryWriter('tensorboard_logs/', sess.graph)
9 # my_writer = tf.summary.FileWriter('tensorboard_logs/', sess.graph)

AttributeError: module 'tensorflow.python.training.training' has no attribute 'SummaryWriter'

the following change fixes it:

# my_writer = tf.train.SummaryWriter('tensorboard_logs/', sess.graph)
my_writer = tf.summary.FileWriter('tensorboard_logs/', sess.graph)

@nfmcclure ... while working on the LASSO regression I noticed one issue. As with the other notebooks, I added code for reproducibility. Depending on the random seed, the LASSO regression returns nan results.

Here are some examples.

LASSO code fragment:

# For Lasso, uncomment the following four lines (and comment out the Ridge Regression loss below)
lasso_param = tf.constant(0.9)
heavyside_step = tf.truediv(1., tf.add(1., tf.exp(tf.multiply(-100., tf.subtract(A, lasso_param)))))
regularization_param = tf.multiply(heavyside_step, 99.)
loss = tf.add(tf.reduce_mean(tf.square(y_target - model_output)), regularization_param)

With this seed setting:

seed = 13
np.random.seed(seed)
tf.set_random_seed(seed)

The training code

# Initialize variables
init = tf.global_variables_initializer()
sess.run(init)

# Training loop
loss_vec = []
for i in range(1500):
    rand_index = np.random.choice(len(x_vals), size=batch_size)
    rand_x = np.transpose([x_vals[rand_index]])
    rand_y = np.transpose([y_vals[rand_index]])
    sess.run(train_step, feed_dict={x_data: rand_x, y_target: rand_y})
    temp_loss = sess.run(loss, feed_dict={x_data: rand_x, y_target: rand_y})
    loss_vec.append(temp_loss[0])
    if (i+1)%300==0:
        print('Step #' + str(i+1) + ' A = ' + str(sess.run(A)) + ' b = ' + str(sess.run(b)))
        print('Loss = ' + str(temp_loss))
        #print('regularization_param='+ str(regularization_param))
        #print('heavyside step=' + str(sess.run(heavyside_step)))
        #print('\n')

Results in this output:

Step #300 A = [[ nan]] b = [[ nan]]
Loss = [[ nan]]
Step #600 A = [[ nan]] b = [[ nan]]
Loss = [[ nan]]
Step #900 A = [[ nan]] b = [[ nan]]
Loss = [[ nan]]
Step #1200 A = [[ nan]] b = [[ nan]]
Loss = [[ nan]]
Step #1500 A = [[ nan]] b = [[ nan]]
Loss = [[ nan]]

With this random seed setting:

seed = 31
np.random.seed(seed)
tf.set_random_seed(seed)

Results in this output:

Step #300 A = [[ 0.82903057]] b = [[ 1.62271953]]
Loss = [[ 10.61604691]]
Step #600 A = [[ 0.82230437]] b = [[ 3.08032537]]
Loss = [[ 3.35721326]]
Step #900 A = [[ 0.81690294]] b = [[ 3.88035631]]
Loss = [[ 1.26691902]]
Step #1200 A = [[ 0.81232369]] b = [[ 4.32818317]]
Loss = [[ 0.69326025]]
Step #1500 A = [[ 0.80592763]] b = [[ 4.57499456]]
Loss = [[ 0.33951065]]

Additional note, there are no issues with the Ridge Regression.

Hello, thanks for the excellent work.

I am confused in 04_svm_kernels.ipynb of Chapter 4 Support Vector Machine, where we always require y_target in prediction because of dual calculation. However, in real world practice, if we only have X_test to predict Y_test, how do we get prediction without the labels?

Do you have any insight about this?

Thanks

Hi Nick,

First, thanks for the great book. It's well-written and I like the way you present things.

I have followed "Creating Sequence-to-Sequence Models" and I managed to make it work. It worked for English German, but when I used the same with English Arabic and basically got the same data from "http://www.manythings.org/anki/ara-eng.zip", the Arabic text for the examples is not displayed correctly.

Here is what I am getting:

English: the quick brown fox jumped over the lazy dog arabic: ['ظ‡ظ„', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†', 'ظ…ظ†']

What should I do to get this displayed correctly?

My other question:

once the model is trained and ready to use, how can I pass a sentence for translation? I would like to use some other sentences.

Thanks in advance for your support.

Kind regards

This error will show up when I run the example 02_tensors.ipynb

Hi,
I have adapted the code of multiclass SVM to my dataset, it is working well, however I don't know how to use k-fold cross validation in the training loop. Any help or guidance would be very appreciated. Thanks

the example can work well, but I wonder the mathematical theory of the code, could you show me how the code solve the svm problem.
To be specific, I tried the SVM in sklearn, but it is too slow. But in sklearn there is a parameter "C" which can control the softmargin, I want to know if the code support the softmargin and how it works.
thanks!

I'm not sure how many people have the same problem with me, when to finish the 02_layering_nested_operations. In my version , the default type of numpy and the constant of tensorflow is not the same. So when you meet the problem, you can try to use np.float32 or dtype=tf.float32, to make sure all variables have the same types to help sess.run() .

Love this cookbook!
and a suggestion. The last part in 02_Tensorflow_... can be replaced as:

merged = tf.summary.merge_all()
if not os.path.exists('tensorboard_logs/'):
os.makedirs('tensorboard_logs/')

my_writer = tf.summary.FileWriter('tensorboard_logs/', sess.graph)

when defining translate_model, running this cell:

print('Creating Translation Model')
input_vocab_size = vocab_size
output_vocab_size = vocab_size

translate_model = translation_model(sess=sess,
                                    input_vocab_size=vocab_size,
                                    output_vocab_size=vocab_size,
                                    buckets=buckets,
                                    rnn_size=rnn_size,
                                    num_layers=num_layers,
                                    max_gradient=max_gradient,
                                    learning_rate=learning_rate,
                                    lr_decay_rate=lr_decay_rate,
                                    forward_only=False)

# Tell TensorFlow to reuse the variables for the test model
with tf.variable_scope(tf.get_variable_scope(), reuse=True):
    #Reuse the variables for the test model
    test_model = translation_model(sess=sess,
                                    input_vocab_size=vocab_size,
                                    output_vocab_size=vocab_size,
                                    buckets=buckets,
                                    rnn_size=rnn_size,
                                    num_layers=num_layers,
                                    max_gradient=max_gradient,
                                    learning_rate=learning_rate,
                                    lr_decay_rate=lr_decay_rate,
                                    forward_only=True)
    test_model.batch_size = 1

I got error as:

AttributeErrorTraceback (most recent call last)
in ()
12 learning_rate=learning_rate,
13 lr_decay_rate=lr_decay_rate,
---> 14 forward_only=False)
15
16 # Tell TensorFlow to reuse the variables for the test model

in translation_model(sess, input_vocab_size, output_vocab_size, buckets, rnn_size, num_layers, max_gradient, learning_rate, lr_decay_rate, forward_only)
14 lr_decay_rate,
15 forward_only=forward_only,
---> 16 dtype=tf.float32)
17 return(model)

/home/wenyanli/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in init(self, source_vocab_size, target_vocab_size, buckets, size, num_layers, max_gradient_norm, batch_size, learning_rate, learning_rate_decay_factor, use_lstm, num_samples, forward_only, dtype)
177 self.target_weights, buckets,
178 lambda x, y: seq2seq_f(x, y, False),
--> 179 softmax_loss_function=softmax_loss_function)
180
181 # Gradients and SGD update operation for training the model.

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.pyc in model_with_buckets(encoder_inputs, decoder_inputs, targets, weights, buckets, seq2seq, softmax_loss_function, per_example_loss, name)
1204 variable_scope.get_variable_scope(), reuse=True if j > 0 else None):
1205 bucket_outputs, _ = seq2seq(encoder_inputs[:bucket[0]],
-> 1206 decoder_inputs[:bucket[1]])
1207 outputs.append(bucket_outputs)
1208 if per_example_loss:

/home/wenyanli/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in (x, y)
176 self.encoder_inputs, self.decoder_inputs, targets,
177 self.target_weights, buckets,
--> 178 lambda x, y: seq2seq_f(x, y, False),
179 softmax_loss_function=softmax_loss_function)
180

/home/wenyanli/tensorflow_cookbook/09_Recurrent_Neural_Networks/05_Creating_A_Sequence_To_Sequence_Model/temp/tutorials/rnn/translate/seq2seq_model.py in seq2seq_f(encoder_inputs, decoder_inputs, do_decode)
140 output_projection=output_projection,
141 feed_previous=do_decode,
--> 142 dtype=dtype)
143
144 # Feeds for inputs.

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.pyc in embedding_attention_seq2seq(encoder_inputs, decoder_inputs, cell, num_encoder_symbols, num_decoder_symbols, embedding_size, num_heads, output_projection, feed_previous, dtype, scope, initial_state_attention)
846 dtype = scope.dtype
847 # Encoder.
--> 848 encoder_cell = copy.deepcopy(cell)
849 encoder_cell = core_rnn_cell.EmbeddingWrapper(
850 encoder_cell,

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
172 copier = getattr(x, "deepcopy", None)
173 if copier:
--> 174 y = copier(memo)
175 else:
176 reductor = dispatch_table.get(cls)

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/site-packages/tensorflow/python/layers/base.pyc in deepcopy(self, memo)
474 setattr(result, k, copy.copy(v))
475 else:
--> 476 setattr(result, k, copy.deepcopy(v, memo))
477 return result
478

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_list(x, memo)
228 memo[id(x)] = y
229 for a in x:
--> 230 y.append(deepcopy(a, memo))
231 return y
232 d[list] = _deepcopy_list

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
188 raise Error(
189 "un(deep)copyable object of type %s" % cls)
--> 190 y = _reconstruct(x, rv, 1, memo)
191
192 memo[d] = y

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _reconstruct(x, info, deep, memo)
332 if state is not None:
333 if deep:
--> 334 state = deepcopy(state, memo)
335 if hasattr(y, 'setstate'):
336 y.setstate(state)

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_dict(x, memo)
255 memo[id(x)] = y
256 for key, value in x.iteritems():
--> 257 y[deepcopy(key, memo)] = deepcopy(value, memo)
258 return y
259 d[dict] = _deepcopy_dict

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
188 raise Error(
189 "un(deep)copyable object of type %s" % cls)
--> 190 y = _reconstruct(x, rv, 1, memo)
191
192 memo[d] = y

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _reconstruct(x, info, deep, memo)
332 if state is not None:
333 if deep:
--> 334 state = deepcopy(state, memo)
335 if hasattr(y, 'setstate'):
336 y.setstate(state)

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_dict(x, memo)
255 memo[id(x)] = y
256 for key, value in x.iteritems():
--> 257 y[deepcopy(key, memo)] = deepcopy(value, memo)
258 return y
259 d[dict] = _deepcopy_dict

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
188 raise Error(
189 "un(deep)copyable object of type %s" % cls)
--> 190 y = _reconstruct(x, rv, 1, memo)
191
192 memo[d] = y

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _reconstruct(x, info, deep, memo)
332 if state is not None:
333 if deep:
--> 334 state = deepcopy(state, memo)
335 if hasattr(y, 'setstate'):
336 y.setstate(state)

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_dict(x, memo)
255 memo[id(x)] = y
256 for key, value in x.iteritems():
--> 257 y[deepcopy(key, memo)] = deepcopy(value, memo)
258 return y
259 d[dict] = _deepcopy_dict

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
188 raise Error(
189 "un(deep)copyable object of type %s" % cls)
--> 190 y = _reconstruct(x, rv, 1, memo)
191
192 memo[d] = y

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _reconstruct(x, info, deep, memo)
332 if state is not None:
333 if deep:
--> 334 state = deepcopy(state, memo)
335 if hasattr(y, 'setstate'):
336 y.setstate(state)

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_dict(x, memo)
255 memo[id(x)] = y
256 for key, value in x.iteritems():
--> 257 y[deepcopy(key, memo)] = deepcopy(value, memo)
258 return y
259 d[dict] = _deepcopy_dict

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_dict(x, memo)
255 memo[id(x)] = y
256 for key, value in x.iteritems():
--> 257 y[deepcopy(key, memo)] = deepcopy(value, memo)
258 return y
259 d[dict] = _deepcopy_dict

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_list(x, memo)
228 memo[id(x)] = y
229 for a in x:
--> 230 y.append(deepcopy(a, memo))
231 return y
232 d[list] = _deepcopy_list

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
188 raise Error(
189 "un(deep)copyable object of type %s" % cls)
--> 190 y = _reconstruct(x, rv, 1, memo)
191
192 memo[d] = y

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _reconstruct(x, info, deep, memo)
332 if state is not None:
333 if deep:
--> 334 state = deepcopy(state, memo)
335 if hasattr(y, 'setstate'):
336 y.setstate(state)

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_dict(x, memo)
255 memo[id(x)] = y
256 for key, value in x.iteritems():
--> 257 y[deepcopy(key, memo)] = deepcopy(value, memo)
258 return y
259 d[dict] = _deepcopy_dict

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
188 raise Error(
189 "un(deep)copyable object of type %s" % cls)
--> 190 y = _reconstruct(x, rv, 1, memo)
191
192 memo[d] = y

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _reconstruct(x, info, deep, memo)
332 if state is not None:
333 if deep:
--> 334 state = deepcopy(state, memo)
335 if hasattr(y, 'setstate'):
336 y.setstate(state)

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_dict(x, memo)
255 memo[id(x)] = y
256 for key, value in x.iteritems():
--> 257 y[deepcopy(key, memo)] = deepcopy(value, memo)
258 return y
259 d[dict] = _deepcopy_dict

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
188 raise Error(
189 "un(deep)copyable object of type %s" % cls)
--> 190 y = _reconstruct(x, rv, 1, memo)
191
192 memo[d] = y

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _reconstruct(x, info, deep, memo)
332 if state is not None:
333 if deep:
--> 334 state = deepcopy(state, memo)
335 if hasattr(y, 'setstate'):
336 y.setstate(state)

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_dict(x, memo)
255 memo[id(x)] = y
256 for key, value in x.iteritems():
--> 257 y[deepcopy(key, memo)] = deepcopy(value, memo)
258 return y
259 d[dict] = _deepcopy_dict

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_list(x, memo)
228 memo[id(x)] = y
229 for a in x:
--> 230 y.append(deepcopy(a, memo))
231 return y
232 d[list] = _deepcopy_list

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_tuple(x, memo)
235 y = []
236 for a in x:
--> 237 y.append(deepcopy(a, memo))
238 d = id(x)
239 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
161 copier = _deepcopy_dispatch.get(cls)
162 if copier:
--> 163 y = copier(x, memo)
164 else:
165 try:

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _deepcopy_dict(x, memo)
255 memo[id(x)] = y
256 for key, value in x.iteritems():
--> 257 y[deepcopy(key, memo)] = deepcopy(value, memo)
258 return y
259 d[dict] = _deepcopy_dict

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in deepcopy(x, memo, _nil)
188 raise Error(
189 "un(deep)copyable object of type %s" % cls)
--> 190 y = _reconstruct(x, rv, 1, memo)
191
192 memo[d] = y

/home/wenyanli/anaconda2/envs/cookbook/lib/python2.7/copy.pyc in _reconstruct(x, info, deep, memo)
341 slotstate = None
342 if state is not None:
--> 343 y.dict.update(state)
344 if slotstate is not None:
345 for key, value in slotstate.iteritems():

AttributeError: 'NoneType' object has no attribute 'update'

previous codes all work fine. How can I get this code work?

I'm not quite understand how this function work, but in my computer. Do the following changes can make code works.Otherwise, errors are given.
Tensorflow 1.0.0

sparse_xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=unscaled_logits,labels= sparse_target_dist)
print(sess.run(sparse_xentropy))

https://github.com/nfmcclure/tensorflow_cookbook/blob/master/07_Natural_Language_Processing/02_Working_with_Bag_of_Words/02_bag_of_words.py#L138

https://github.com/nfmcclure/tensorflow_cookbook/blob/master/09_Recurrent_Neural_Networks/02_Implementing_RNN_for_Spam_Prediction/02_implementing_rnn.py#L120

shouldn't use softmax activation in output layer, when use sparse_softmax_cross_entropy_with_logits

refer:https://www.tensorflow.org/api_docs/python/tf/nn/sparse_softmax_cross_entropy_with_logits

After running the Seq2Seq example I got:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-8e0fa45c722e> in <module>()
    209                                     learning_rate=learning_rate,
    210                                     lr_decay_rate=lr_decay_rate,
--> 211                                     forward_only=False)
    212 
    213 # Tell TensorFlow to reuse the variables for the test model

<ipython-input-1-8e0fa45c722e> in translation_model(sess, input_vocab_size, output_vocab_size, buckets, rnn_size, num_layers, max_gradient, learning_rate, lr_decay_rate, forward_only)
    193           lr_decay_rate,
    194           forward_only=forward_only,
--> 195           dtype=tf.float32)
    196     return(model)
    197 

/Users/user/Jupyter/quora_challenge/temp/tutorials/rnn/translate/seq2seq_model.py in __init__(self, source_vocab_size, target_vocab_size, buckets, size, num_layers, max_gradient_norm, batch_size, learning_rate, learning_rate_decay_factor, use_lstm, num_samples, forward_only, dtype)
    177           self.target_weights, buckets,
    178           lambda x, y: seq2seq_f(x, y, False),
--> 179           softmax_loss_function=softmax_loss_function)
    180 
    181     # Gradients and SGD update operation for training the model.

/usr/local/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py in model_with_buckets(encoder_inputs, decoder_inputs, targets, weights, buckets, seq2seq, softmax_loss_function, per_example_loss, name)
   1199           variable_scope.get_variable_scope(), reuse=True if j > 0 else None):
   1200         bucket_outputs, _ = seq2seq(encoder_inputs[:bucket[0]],
-> 1201                                     decoder_inputs[:bucket[1]])
   1202         outputs.append(bucket_outputs)
   1203         if per_example_loss:

/Users/user/Jupyter/quora_challenge/temp/tutorials/rnn/translate/seq2seq_model.py in <lambda>(x, y)
    176           self.encoder_inputs, self.decoder_inputs, targets,
    177           self.target_weights, buckets,
--> 178           lambda x, y: seq2seq_f(x, y, False),
    179           softmax_loss_function=softmax_loss_function)
    180 

/Users/user/Jupyter/quora_challenge/temp/tutorials/rnn/translate/seq2seq_model.py in seq2seq_f(encoder_inputs, decoder_inputs, do_decode)
    140           output_projection=output_projection,
    141           feed_previous=do_decode,
--> 142           dtype=dtype)
    143 
    144     # Feeds for inputs.

/usr/local/lib/python3.6/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py in embedding_attention_seq2seq(encoder_inputs, decoder_inputs, cell, num_encoder_symbols, num_decoder_symbols, embedding_size, num_heads, output_projection, feed_previous, dtype, scope, initial_state_attention)
    853         embedding_size=embedding_size)
    854     encoder_outputs, encoder_state = core_rnn.static_rnn(
--> 855         encoder_cell, encoder_inputs, dtype=dtype)
    856 
    857     # First calculate a concatenation of encoder outputs to put attention on.

/usr/local/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn.py in static_rnn(cell, inputs, initial_state, dtype, sequence_length, scope)
    195             state_size=cell.state_size)
    196       else:
--> 197         (output, state) = call_cell()
    198 
    199       outputs.append(output)

/usr/local/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn.py in <lambda>()
    182       if time > 0: varscope.reuse_variables()
    183       # pylint: disable=cell-var-from-loop
--> 184       call_cell = lambda: cell(input_, state)
    185       # pylint: enable=cell-var-from-loop
    186       if sequence_length is not None:

/usr/local/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py in __call__(self, inputs, state, scope)
    879         embedded = embedding_ops.embedding_lookup(
    880             embedding, array_ops.reshape(inputs, [-1]))
--> 881     return self._cell(embedded, state)
    882 
    883 

/usr/local/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py in __call__(self, inputs, state, scope)
    951                 state, [0, cur_state_pos], [-1, cell.state_size])
    952             cur_state_pos += cell.state_size
--> 953           cur_inp, new_state = cell(cur_inp, cur_state)
    954           new_states.append(new_state)
    955     new_states = (tuple(new_states) if self._state_is_tuple else

/usr/local/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py in __call__(self, inputs, state, scope)
    144   def __call__(self, inputs, state, scope=None):
    145     """Gated recurrent unit (GRU) with nunits cells."""
--> 146     with _checked_scope(self, scope or "gru_cell", reuse=self._reuse):
    147       with vs.variable_scope("gates"):  # Reset gate and update gate.
    148         # We start with bias of 1.0 to not reset and not update.

/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib/python3.6/contextlib.py in __enter__(self)
     80     def __enter__(self):
     81         try:
---> 82             return next(self.gen)
     83         except StopIteration:
     84             raise RuntimeError("generator didn't yield") from None

/usr/local/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py in _checked_scope(cell, scope, reuse, **kwargs)
     75             "this error will remain until then.)"
     76             % (cell, cell_scope.name, scope_name, type(cell).__name__,
---> 77                type(cell).__name__))
     78     else:
     79       weights_found = False

ValueError: Attempt to reuse RNNCell <tensorflow.contrib.rnn.python.ops.core_rnn_cell_impl.GRUCell object at 0x129464ac8> with a different variable scope than its first use.  First use of cell was with scope 'embedding_attention_seq2seq/embedding_attention_decoder/attention_decoder/multi_rnn_cell/cell_0/gru_cell', this attempt is with scope 'embedding_attention_seq2seq/rnn/multi_rnn_cell/cell_0/gru_cell'.  Please create a new instance of the cell if you would like it to use a different set of weights.  If before you were using: MultiRNNCell([GRUCell(...)] * num_layers), change to: MultiRNNCell([GRUCell(...) for _ in range(num_layers)]).  If before you were using the same cell instance as both the forward and reverse cell of a bidirectional RNN, simply create two instances (one for forward, one for reverse).  In May 2017, we will start transitioning this cell's behavior to use existing stored weights, if any, when it is called with scope=None (which can lead to silent model degradation, so this error will remain until then.)

Any idea of how to fix it?.

/01_Introduction/07_Working_with_Data_Sources/ is missing a link to the CIFAR-10 dataset

when I run the chapter8：08_Convolutional_Neural_Networks，02_Intro_to_CNN_MNIST ,the example give me a error ,I don't know how to sovle.

the error as follow:

ValueError Traceback (most recent call last)
in ()
109
110 # Declare Loss Function (softmax cross entropy)
--> 111 loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(model_output, y_target))
112
113 # Create a prediction function

E:\Anaconda3\lib\site-packages\tensorflow\python\ops\nn_ops.py in sparse_softmax_cross_entropy_with_logits(_sentinel, labels, logits, name)
1682 """
1683 _ensure_xent_args("sparse_softmax_cross_entropy_with_logits", _sentinel,
-> 1684 labels, logits)
1685
1686 # TODO(pcmurray) Raise an error when the label is not an index in

E:\Anaconda3\lib\site-packages\tensorflow\python\ops\nn_ops.py in _ensure_xent_args(name, sentinel, labels, logits)
1531 if sentinel is not None:
1532 raise ValueError("Only call %s with "
-> 1533 "named arguments (labels=..., logits=..., ...)" % name)
1534 if labels is None or logits is None:
1535 raise ValueError("Both labels and logits must be provided.")

ValueError: Only call sparse_softmax_cross_entropy_with_logits with named arguments (labels=..., logits=..., ...)

I just tried 08_logistic_regression code. However, I got the error in follows.
Give me some advice?

IndexError Traceback (most recent call last)
in ()
26
27 # Pull out target variable
---> 28 y_vals = np.array([x[1] for x in birth_data])
29 # Pull out predictor variables (not id, not target, and not birthweight)
30 x_vals = np.array([x[2:9] for x in birth_data])

in (.0)
26
27 # Pull out target variable
---> 28 y_vals = np.array([x[1] for x in birth_data])
29 # Pull out predictor variables (not id, not target, and not birthweight)
30 x_vals = np.array([x[2:9] for x in birth_data])

IndexError: list index out of range

In 09_Recurrent_Neural_Networks\03_Implementing_LSTM\03_implementing_lstm.py ,
there could not find seq2seq. I think maybe update to version 1.0 of tf caused this problem.
"
--> 166 decoder = tf.nn.seq2seq.rnn_decoder
167 outputs, last_state = decoder(rnn_inputs_trimmed,
168 self.initial_state,

AttributeError: module 'tensorflow.python.ops.nn' has no attribute 'seq2seq'
"

Python 3.5
Tensorflow GPU 1.0.1
Commit: 69396a0

File: 06_multiclass_svm.py
Line: 74
second_term = tf.reduce_sum(tf.multiply(my_kernel, tf.multiply(b_vec_cross, y_target_cross)),[1,2])

Error:
ValueError: Dimensions must be equal, but are 2 and 3 for 'Mul_3' (op: 'Mul') with input shapes: [2,?,1], [3,50,50].

tf.nn.sigmoid_cross_entropy_with_logits(_sentinel=None, labels=None, logits=None, name=None) don't suggest using _sentinel , because it refer to positional parameters.
But in my computer and python environment, if just put in
tf.nn.sigmoid_cross_entropy_with_logits(x_vals,targets)
It gives warnings, so I think maybe this function receive x_vals as _sentinel. When add the labels , I get the resule:
xentropy_sigmoid_y_vals = tf.nn.sigmoid_cross_entropy_with_logits(labels=x_vals,logits= targets)
This can work on python3.5 and tensorflow 1.0.0

From Line 29 to Line 34 these codes block show as follows:
"
if os.path.isfile(save_file_name):
text_data = []
with open(save_file_name, 'r') as temp_output_file:
reader = csv.reader(temp_output_file)
for row in reader:
text_data.append(row)
"
There lack a little small code when the 'temp_spam_data.csv' have been exists in local.
There needs to split the "row" data with ',' to give to text_data. Otherwise, there will be some error message like:
"---> 53 texts = [x[1] for x in text_data]
54 target = [x[0] for x in text_data]
IndexError: list index out of range
"

Hi Nick,
get_bach function in 06_siamese_similarity_driver.ipynb generates data with targets -1 or 1, however the loss function in siamese_similarity_model.py seems to be configured to work with targets 0 or 1:

#If y-target is -1 to 1, then do the following
#pos_mult = tf.add(tf.multiply(0.5, y_target), 0.5)
#Else if y-target is 0 to 1, then do the following
pos_mult = tf.cast(y_target, tf.float32)

This part of code also remains commented:

#Change targets from (0,1) --> (-1, 1)
#via (2 * x - 1)
#y_target_int = tf.sub(tf.mul(y_target_int, 2), 1)

I wonder if I should change the loss function if I want to run this model on my data with target -1/1. Best regards!

In 09_Recurrent_Neural_Networks\05_Creating_A_Sequence_To_Sequence_Model
05_seq2seq_translation.py
There was some import error as follows:
ImportError Traceback (most recent call last)
in ()
21 from zipfile import ZipFile
22 from collections import Counter
---> 23 from tensorflow.models.rnn.translate import data_utils
24 from tensorflow.models.rnn.translate import seq2seq_model
25 from tensorflow.python.framework import ops

ImportError: No module named 'tensorflow.models.rnn'

And could you please give me some suggestion about how to find the right module in ver 1.0 after the update changed

When I run "tensorflow_cookbook-master\08_Convolutional_Neural_Networks\05_Stylenet_NeuralStyle\05_stylenet.py" , it comes out the below message:
D:\Anaconda3\lib\site-packages\scipy\misc\pilutil.py:98: RuntimeWarning: invalid value encountered in greater
bytedata[bytedata > high] = high
D:\Anaconda3\lib\site-packages\scipy\misc\pilutil.py:99: RuntimeWarning: invalid value encountered in less
bytedata[bytedata < 0] = 0
Generation 1000 out of 10000
Generation 1500 out of 10000
Generation 2000 out of 10000
Generation 2500 out of 10000
...
And finally, I got all the black picture. Are the two warning caused the black picture?

Hi, in 04_svm_kernels.py shows below, I have some questions to need your help, please.

1. In Line 54, the variable "model_output" seems not to be used in the whole py file, is there missed some codes?
2. Not very understand on how to calculate the loss function when use the my_kernel. From the code I guess the calculation is that: my_kernel *(b^T *b) * (y * y^T) ,and then reduce_sum of it.
   And the loss equals "reduce_sum(b)- the above result"
   So I don't understand why use this formula to get the loss function calculation result. Could you give me some help to understanding the process, please?
   "
   # Compute SVM Model
   54 model_output = tf.matmul(b, my_kernel)
   55 first_term = tf.reduce_sum(b)
   56 b_vec_cross = tf.matmul(tf.transpose(b), b)
   57 y_target_cross = tf.matmul(y_target, tf.transpose(y_target))
   58 second_term = tf.reduce_sum(tf.multiply(my_kernel, tf.multiply(b_vec_cross, y_target_cross)))
   59 loss = tf.negative(tf.subtract(first_term, second_term))
   "

When I finish one example, then switch to another. My jupyter notebook shows the problem
The kernel appears to die
Restart can't help, and I have to close all of them. After restarting jupyter notebook and run into the same example, it can work. But do this is not comfortable. Why my condition happens?

Hello, I am trying to understand this operiaton in Shakespeare example

# Create targets from shuffled batches
targets = [np.roll(x, -1, axis=1) for x in batches]

From my test below

import numpy as np
arr = np.array([
    [1, 2, 3, 4, 5],
    [6, 7, 8, 9, 10],
    [11, 12, 13, 14, 15],
])
np.roll(arr, -1, axis=1)

which results in

array([[ 2,  3,  4,  5,  1],
       [ 7,  8,  9, 10,  6],
       [12, 13, 14, 15, 11]])

I don't really understand what does this mean, because I think logically we should move the first world of next sentence to the end of current sentence (and in that case we are predicting the next word).

I even think put axis=0 may be better, because we are trying to predict the next sentence in the batch in that case.

Thank you for the time.

when reading codes for this algorithm, I found one of the most important step: calculate similarity( cosine) (in both skip gram and cbow) is wrong, as it is just generated randomly, and never updated when training, you can find the results printed are not different at any time. i.e., the results is wrong. please kindly correct this crucial error, thanks. @nfmcclure @jimthompson5802 @slcott @havk64 @cxxgtxy

I've been getting inconsistent results with the Deming regression example given in your text. I'm running Tensorflow locally on an iMac using MacOS Sierra 10.12.4. I then explored using different combinations of initial values for A (rather than the random normal example in the text), and then different learning rates. I found that starting values for A that were negative often led to poor fits (lines with negative slopes and large intercepts, suggesting gradients that were moving in the wrong direction -- obviously diverging from what the raw data would otherwise suggest on inspection), even though the loss function (or Deming distance) improved throughout the optimization given sufficient iterations.

I understand the impact that learning rate has on convergence as well, but wondered whether you could suggest a good way to determine the initial value for the variables and the learn rate. Also, what the best way to determine if the optimization is achieving reasonable results? I've looked at examining the actual gradient calculations during iteration, but thought the must be a better way...

Just wanted to point out that you have some examples where you use
logits_out = tf.nn.softmax(tf.matmul(last, weight) + bias)

and then define the loss function as:
losses = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits_out, labels=y_output)

You are calling softmax twice. From TensorFlow documentation:

WARNING: This op expects unscaled logits, since it performs a softmax on logits internally for efficiency. Do not call this op with the output of softmax, as it will produce incorrect results.

Have a nice day :)

Can you explain more detail in code SVM kernel, in loss function and predict, i try to read more about SVM to understand but can't, or you can send me some blog with math( in your code) it will better for me. Thank you so much.

They seem to be making changes with TensorFlow

It looks like the following initialization should be used..

init = tf.global_variables_initializer()

Below is the older code, which gives the following warning...

init = tf.initialize_all_variables()

WARNING:tensorflow:From <ipython-input-15-961de099ec6b>:61 in <module>.: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be 
removed after 2017-03-02.

https://github.com/nfmcclure/tensorflow_cookbook/blob/master/09_Recurrent_Neural_Networks/03_Implementing_LSTM/03_implementing_lstm.py

tf.nn.conv2d(input,filter,strides,padding) is the function of convolutional network in tensorflow.
Here, the author says the input of 4-D data should have the size [batch#, width, height, channels],
but the official web gives the description:
input tensor of shape [batch, in_height, in_width, in_channels]
Will this make difference , if we replace the place of width and height in our experience?

During compiling following piece of code:

birth_weight_file = 'birth_weight.csv'

download data and create data file if file does not exist in current directory

if not os.path.exists(birth_weight_file):
birthdata_url = 'https://github.com/nfmcclure/tensorflow_cookbook/raw/master/01_Introduction/07_Working_with_Data_Sources/birthweight_data/birthweight.dat'
birth_file = requests.get(birthdata_url)
birth_data = birth_file.text.split('\r\n')
birth_header = birth_data[0].split('\t')
birth_data = [[float(x) for x in y.split('\t') if len(x)>=1] for y in birth_data[1:] if len(y)>=1]
with open(birth_weight_file, "w") as f:
writer = csv.writer(f)
writer.writerows(birth_data)
f.close()

read birth weight data into memory

birth_data = []
with open(birth_weight_file, newline='') as csvfile:
csv_reader = csv.reader(csvfile)
birth_header = next(csv_reader)
for row in csv_reader:
birth_data.append(row)

birth_data = [[float(x) for x in row] for row in birth_data]

y_vals = np.array([x[10] for x in birth_data])

I faced a problem:

IndexError Traceback (most recent call last)
in ()
28
29 # Extract y-target (birth weight)
---> 30 y_vals = np.array([x[10] for x in birth_data])

in (.0)
28
29 # Extract y-target (birth weight)
---> 30 y_vals = np.array([x[10] for x in birth_data])

IndexError: list index out of range

It is obviously because

print(birth_data[10]) is given as empty row. However, I do not know fix it, could please someone help me?

I am new in python before I was working with Matlab and a little bit with C++, therefore a piece of code which is responsible for '# download data and creates data file if file does not exist in current directory'
is quite unclear. I will be extremely thankful if someone explains briefly what is happing there.