ARCHIVED: Contains historical course materials/Homework materials for the FREE MOOC course on "Creative Applications of Deep Learning w/ Tensorflow" #CADL
Home Page: https://www.kadenze.com/courses/creative-applications-of-deep-learning-with-tensorflow/info
License: Apache License 2.0
There is this code snippet:
# Grab the tensor defining the input to the network
x = g.get_tensor_by_name(names[0] + ":0")
# And grab the tensor defining the softmax layer of the network
softmax = g.get_tensor_by_name(names[-2] + ":0")
for img in [content_img, style_img]:
with tf.Session(graph=g) as sess, g.device('/cpu:0'):
# Remember from the lecture that we have to set the dropout
# "keep probability" to 1.0.
res = softmax.eval(feed_dict={x: img,
'net/dropout_1/random_uniform:0': np.ones(
g.get_tensor_by_name(
'net/dropout_1/random_uniform:0'
).get_shape().as_list()),
'net/dropout/random_uniform:0': np.ones(
g.get_tensor_by_name(
'net/dropout/random_uniform:0'
).get_shape().as_list())})[0]
print([(res[idx], net['labels'][idx])
for idx in res.argsort()[-5:][::-1]])The problem is that when you get the size as:
np.ones( g.get_tensor_by_name( 'net/dropout_1/random_uniform:0' ).get_shape().as_list()The result is [None, 4096]
When this list is fed into np.ones, there is an error.
Instead, I suggest the following code (or similar):
# Grab the tensor defining the input to the network
x = g.get_tensor_by_name(names[0] + ":0")
# And grab the tensor defining the softmax layer of the network
softmax = g.get_tensor_by_name(names[-2] + ":0")
for img in [content_img, style_img]:
with tf.Session(graph=g) as sess, g.device('/cpu:0'):
# Remember from the lecture that we have to set the dropout
# "keep probability" to 1.0.
d_0 = g.get_tensor_by_name('net/dropout/random_uniform:0')
d_0_list = d_0.get_shape().as_list()
d_0_ones = np.ones([1 if x == None else x for x in d_0_list])
print(d_0_ones.shape)
d_1 = g.get_tensor_by_name('net/dropout_1/random_uniform:0')
d_1_list = d_1.get_shape().as_list()
d_1_ones = np.ones([1 if x == None else x for x in d_1_list])
print(d_1_ones.shape)
res_obj = softmax.eval(feed_dict={x: img,
'net/dropout_1/random_uniform:0': d_1_ones,
'net/dropout/random_uniform:0': d_0_ones})
res = res_obj[0]
print([(res[idx], net['labels'][idx])
for idx in res.argsort()[-5:][::-1]])It's worth mentioning that in the lecture notes, you don't use np.ones, but instead, you explicitly just use [[1]*4096], which creates the same outcome.
Looking forward to hearing your thoughts - Thank you.
On Session 5, part 2, the check
if os.path.exists(ckpt_name):
saver.restore(sess, ckpt_name)
print("Model restored.")... does not match the provided checkpoint file name exactly trump.ckpt.data-00000-of-00001 so the load is not attempted.
Therefore, the model is untrained for the example and the predictions are all non-deterministic random strings,
!!--sssjjj44www???ggvvvwwwx??ggggvaaa577777t777t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t77t777t
??ffd88l:ttttt?efiiiii8880cc99v666444sszssxrpp/mmmxxxxr!//33gggggkgggkmmmeeiiiiiiii888ccc8994444sszszsírpp/mmmxxxxr!//33gggggkgggkmmmeeiiiiiiii888ccc8994444sszszsírpp/mmmxxxxr!//33gggggkgggkmmmeeiiiiiiii888ccc8994444sszszsírpp/mmmxxxxr!//33gggggkgggkmmmeeiiiiiiii888ccc8994444sszszsírpp/mmmxxxxr!//33gggggkgggkmmmeeiiiiiiii888ccc8994444sszszsírpp/mmmxxxxr!//33gggggkgggkmmmeeiiiiiiii888ccc8994444sszszsírpp/mmmxxxxr!//33gggggkgggkmmmeeiiiiiiii888ccc8994444sszszsírpp/mmmxxxxr!//33gggggkgggkmmmeeiiiii
etc.
If the check is removed, then the saver does pick up the provided model, but then cannot load it.
It blows up with the error below. I've also noticed, while chasing this, that the encoder and decoder used for this part of the exercise are actually initialized with a different text -- the one from Part 4 - Character-Level Language Model; I guess that's fine as the sets are probably equivalent in this case, but wouldn't it be better to add a cell just to regenerate these for the latter section?
I'm running TF 1.1.0 wit GPU support on a Debian 4.9.18-1 laptop.
INFO:tensorflow:Restoring parameters from ./trump.ckpt
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
~/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1038 try:
-> 1039 return fn(*args)
1040 except errors.OpError as e:
~/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
1020 feed_dict, fetch_list, target_list,
-> 1021 status, run_metadata)
1022
~/anaconda3/envs/potrero/lib/python3.6/contextlib.py in __exit__(self, type, value, traceback)
88 try:
---> 89 next(self.gen)
90 except StopIteration:
~/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py in raise_exception_on_not_ok_status()
465 compat.as_text(pywrap_tensorflow.TF_Message(status)),
--> 466 pywrap_tensorflow.TF_GetCode(status))
467 finally:
InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [2048] rhs shape= [800]
[[Node: save/Assign_17 = Assign[T=DT_FLOAT, _class=["loc:@rnn/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](rnn/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/biases, save/RestoreV2_17/_49)]]
During handling of the above exception, another exception occurred:
InvalidArgumentError Traceback (most recent call last)
<ipython-input-8-999cfb8483be> in <module>()
11 saver = tf.train.Saver()
12 # if os.path.exists(ckpt_name):
---> 13 saver.restore(sess, ckpt_name)
14 # print("Model restored.")
15
~/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/training/saver.py in restore(self, sess, save_path)
1455 logging.info("Restoring parameters from %s", save_path)
1456 sess.run(self.saver_def.restore_op_name,
-> 1457 {self.saver_def.filename_tensor_name: save_path})
1458
1459 @staticmethod
~/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
776 try:
777 result = self._run(None, fetches, feed_dict, options_ptr,
--> 778 run_metadata_ptr)
779 if run_metadata:
780 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)
~/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
980 if final_fetches or final_targets:
981 results = self._do_run(handle, final_targets, final_fetches,
--> 982 feed_dict_string, options, run_metadata)
983 else:
984 results = []
~/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1030 if handle is None:
1031 return self._do_call(_run_fn, self._session, feed_dict, fetch_list,
-> 1032 target_list, options, run_metadata)
1033 else:
1034 return self._do_call(_prun_fn, self._session, handle, feed_dict,
~/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1050 except KeyError:
1051 pass
-> 1052 raise type(e)(node_def, op, message)
1053
1054 def _extend_graph(self):
InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [2048] rhs shape= [800]
[[Node: save/Assign_17 = Assign[T=DT_FLOAT, _class=["loc:@rnn/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](rnn/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/biases, save/RestoreV2_17/_49)]]
Caused by op 'save/Assign_17', defined at:
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/ipykernel_launcher.py", line 16, in <module>
app.launch_new_instance()
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 477, in start
ioloop.IOLoop.instance().start()
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 177, in start
super(ZMQIOLoop, self).start()
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tornado/ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
self._handle_recv()
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
self._run_callback(callback, msg)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 235, in dispatch_shell
handler(stream, idents, msg)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 196, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 533, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2698, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2802, in run_ast_nodes
if self.run_code(code, result):
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2862, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-8-999cfb8483be>", line 11, in <module>
saver = tf.train.Saver()
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1056, in __init__
self.build()
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1086, in build
restore_sequentially=self._restore_sequentially)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 691, in build
restore_sequentially, reshape)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 419, in _AddRestoreOps
assign_ops.append(saveable.restore(tensors, shapes))
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 155, in restore
self.op.get_shape().is_fully_defined())
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/ops/state_ops.py", line 270, in assign
validate_shape=validate_shape)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/ops/gen_state_ops.py", line 47, in assign
use_locking=use_locking, name=name)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
op_def=op_def)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/gabriel/anaconda3/envs/potrero/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1228, in __init__
self._traceback = _extract_stack()
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [2048] rhs shape= [800]
[[Node: save/Assign_17 = Assign[T=DT_FLOAT, _class=["loc:@rnn/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](rnn/rnn/multi_rnn_cell/cell_1/basic_lstm_cell/biases, save/RestoreV2_17/_49)]]
Running docker build -t cadl . fails during the tensorflow install with
"
Could not find a version that satisfies the requirement tensorflow==1.5.0 (from -r /requirements.txt (line 16)) (from versions: )
No matching distribution found for tensorflow==1.5.0 (from -r /requirements.txt (line 16))
"
There is an unwanted "*" at line #40 in gif.py.
Hello,
I see in your ipython notebook about VAEGANs, this in your variational step:
z_mu = tf.nn.tanh(utils.linear(h, n_code, name='mu')[0])
z_log_sigma = 0.5 * tf.nn.tanh(utils.linear(h, n_code, name='log_sigma')[0])
Why are you applying the tanh activation here?
Hi, I was trying out the tutorials on windows and I found that the gaussian formula wasn't working. Something to do with the dtypes not converting correctly. None the less, there is a solution in the form of the Normal Distribution contribution in the update. I dunno if you want to re-record that bit though. I'll see if I can figure out how it works later and mention it in the Kadenze forum.
..a the 'Image manipulation > Croping images' paragraph, in the def of the imcrop_tosquare function : the comment says
The first branch says, if the rows of
imgare greater than the columns, then set the variableextrato their difference and divide by 2.
...whereas the variable extra is in fact not divided by 2 right away in the code. It's minnor but for a beginner (such as myself) it might be very brain freezing for a moment.
That's all :)
TensorFlow is moving towards numpy syntax where reduction_indices is now axis, and instead of it being the 1st argument, tensors are now the first argument. This breaks a lot of code.
Hi,
I'm trialing the first course in audit mode before signing up for the full paid course.
I'm familiar with Docker, or at least I thought I was, but running into an issue getting the notebook started.
echo $(pwd) returns
E:\Source Control\AI\cadl
so then if I run the recommended command:
E:\Source Control\AI\cadl> docker run -it -p 8888:8888 -p 6006:6006 -v /$(pwd)/session-1:/notebooks --name tf cadl /bin/bash
I get an error:
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: Mount denied:
The source path "/E:\Source Control\AI\cadl/session-0"
is not a valid Windows path.
Note:
I can get jupyter running aok by removing the $(pwd) [ie: docker run -it -p 8888:8888 -p 6006:6006 -v /session-1:/notebooks --name tf cadl /bin/bash ]
but
root@39c4441bcde8:/notebooks# ls
shows nothing as it's not in the correct subdirectory.
It does not show "README.md lecture-1.ipynb libs session-1.ipynb tests" but just an empty directory.
However, I can load the notebooks manually, but not sure how the subfolder content paths will be handled.
Any clues?
I get a type error when trying to generate image from random noise in the ipython notebook. Numpy seems to have a problem with upcasting the datatypes.
TypeError Traceback (most recent call last)
<ipython-input-48-b06014ec4265> in <module>()
1 # Create some noise, centered at gray
2 img_noise = inception.preprocess(
----> 3 (np.random.randint(100, 150, size=(224, 224, 3))))[np.newaxis]
4 print(img_noise.min(), img_noise.max())
/home/clu/Python/CADL-master/session-4/libs/inception.py in preprocess(img, crop, resize, dsize)
81 def preprocess(img, crop=True, resize=True, dsize=(299, 299)):
82 if img.dtype != np.uint8:
---> 83 img *= 255
84
85 if crop:
```
`
Hi mital,
I am trying to work on session-1. And started getting errors as below :
ImportError: Could not find 'cudart64_80.dll'. TensorFlow requires that this DLL be installed in a directory that is named in your %PATH% environment variable. Download and install CUDA 8.0 from this URL: https://developer.nvidia.com/cuda-toolkit
I am using windows 10 machine and also installed CUDA8 from nvidia site. Still dont see this issue going away. Also tried latest CUDA9, but with no luck. Later in web i saw tf1.4 does not support cuda9 for windows 10.
Can you please help with above error.
Regards
Vinay
In lecture 5
loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits, Y_true_flat)
raises ValueError while
loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits = logits, labels = Y_true_flat)
seems to work
see https://www.kadenze.com/forums/creative-applications-of-deep-learning-with-tensorflow-sessions-session-5-generative-models/threads/valueerror-only-call-sparse_softmax_cross_entropy_with_logits-with-named-arguments-labels-logits
ERROR: Train an autoencoder on Celeb Net.
----------------------------------------------------------------------
Traceback (most recent call last):
File "D:\Anaconda3\lib\site-packages\nose\case.py", line 198, in runTest
self.test(*self.arg)
File "D:\IDM\CADL-master\session-3\libs\vae.py", line 499, in test_celeb
ckpt_name='./celeb.ckpt')
File "D:\IDM\CADL-master\session-3\libs\vae.py", line 303, in train_vae
shape=input_shape)
File "D:\IDM\CADL-master\session-3\libs\dataset_utils.py", line 49, in create_input_pipeline
files, capacity=len(files))
TypeError: object of type 'NoneType' has no len()
-------------------- >> begin captured stdout << ---------------------
Could not find celeb dataset under ./img_align_celeba/.
Try downloading the dataset from the "Aligned and Cropped" link located here (imgs/img_align_celeba.zip [1.34 GB]): http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html
--------------------- >> end captured stdout << ----------------------
...
crop = img[max(0, extra // 2 + 1):min(-1, -(extra // 2)), :]
...The result will not be a square when extra is 1.
For example, the crop is img[1:-1, :] when extra is 1.
It should be:
...
crop = img[extra // 2:-(extra // 2) - 1, :]
...Cell 31 of the lecture notebook shows :
def total_variation_loss(x):
h, w = x.get_shape().as_list()[1], x.get_shape().as_list()[1]
Shouldn't it be :
def total_variation_loss(x):
h, w = x.get_shape().as_list()[1], x.get_shape().as_list()[2]
?
Does this course contains nlp with tensorflow?
When calling test_vgg() from vgg16.py output is:
ValueError: Cannot feed value of shape (1, 1) for Tensor 'vgg/dropout_1/random_uniform:0', which has shape '(?, 4096)'
When trying to change shape to
res = np.squeeze(softmax.eval(feed_dict={ x: img, 'vgg/dropout_1/random_uniform:0': [[1.0]*4096], 'vgg/dropout/random_uniform:0': [[1.0]*4096]}))
Output is:
InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 115200 values, but the requested shape requires a multiple of 25088 [[Node: vgg/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](vgg/pool5, vgg/Reshape/shape)]]
This output was received on OSX, running Python3 and Tensorflow 1.7
If the path contains spaces, then
docker run -it -p 8888:8888 -p 6006:6006 -v /$(pwd)/session-1:/notebooks --name tf cadl /bin/bash
might throw the error
docker: invalid reference format: repository name must be lowercase.
or behave unexpectedly.
Wrapping in quotation marks solves this:
docker run -it -p 8888:8888 -p 6006:6006 -v "/$(pwd)/session-1":/notebooks --name tf cadl /bin/bash
Some people have found using the hand written cross entropy calculation is not as numerically stable and leads to NaNs: https://www.kadenze.com/forums/creative-applications-of-deep-learning-with-tensorflow-sessions-session-3-unsupervised-and-supervised-learning/threads/convolutional-binary-classification-network-training-issues
When constructing the first bias, the comment says to use a constant value of 0 but the code initializes the value to 1
# For bias variables, we usually start with a constant value of 0.
B = tf.Variable(tf.constant([1], dtype=tf.float32), name='bias')
In the section where you reconstruct the astronaut image, the markdown says that you're going to redraw the image every 10 iterations, but the code actually uses 20 iterations:
"Every 10 iterations, we're going to draw the predicted image by evaluating the predicted image tensor, Y_pred, and giving it every location in the image to predict, the xs array."
if (it_i + 1) % 20 == 0:
I think the cost function minimization example in lecture 2 (input 4) has a bug. Following the initialization values given in the example, one should get to the local minimum just in a few steps. Below is the notebook code:
%matplotlib inline
import os
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.colors as colors
import matplotlib.cm as cmx
plt.style.use('ggplot')
def myMin(values):
import operator
min_index, min_value = min(enumerate(values),
key=operator.itemgetter(1))
return min_index, min_value
hz = tf.placeholder(tf.float32, name='hz')
ksize = tf.placeholder(tf.int32, name='ksize')
lr = tf.placeholder(tf.float32, name='learning_rate')
init_p = tf.placeholder(tf.int32, name = 'init_p')
x = tf.linspace(-1.0, 1.0, ksize)
x_cropped = x[init_p : init_p+2]
cost = tf.multiply(tf.sin(hzx_cropped),tf.exp(-x_cropped))
costFull = tf.multiply(tf.sin(hzx),tf.exp(-x))
grad = cost[1:] - cost[:-1]
x_out = tf.multiply(lr,grad)
init_op = tf.global_variables_initializer()
init_ps = 120 #int(120/2)
nr_iterations = 15
with tf.Session() as sess:
sess.run(init_op)
xs, cost_s = sess.run([x,costFull],{hz:10,ksize:200})
x_in = xs[init_ps]
x_ser = []
cost_ser = []
for i in range(nr_iterations):
x_ser.append(xs[init_ps])
x2, cost2 = sess.run([x_out,cost],
{hz:10,init_p:init_ps,x:xs,lr:1})
x2 = xs[init_ps] - x2
dx = np.abs(xs - x2)
init_ps, _ = myMin(dx)
cost_ser.append(cost2[0].flatten())
x_ser = np.array(x_ser).flatten()
cost_ser = np.array(cost_ser).flatten()
cmap = plt.get_cmap('coolwarm')
c_norm = colors.Normalize(vmin=0, vmax=nr_iterations)
scalar_map = cmx.ScalarMappable(norm=c_norm, cmap=cmap)
fig, axF = plt.subplots(2, figsize=(10, 8))
ax = axF[0]
ax.plot(xs, cost_s)
for i in range(nr_iterations):
ax.plot(x_ser[i], cost_ser[i],'ro',
color=scalar_map.to_rgba(i))
ax.set_ylabel('Cost')
ax.set_xlabel('x')
ax = axF[1]
for i in range(nr_iterations-1):
ax.plot(i, x_ser[i+1] - x_ser[i],'o',color=scalar_map.to_rgba(i))
ax.set_xlabel('Iteration')
In the code for the VAE function (lines 97-102 as per bcdb019) appears to be a bug that prevents the usage of corrupted inputs:
if denoising:
current_input = utils.corrupt(x) * corrupt_prob + x * (1 - corrupt_prob)
# 2d -> 4d if convolution
x_tensor = utils.to_tensor(x) if convolutional else x
current_input = x_tensorHere, the current_input tensor is directly overwritten by x_tensor. It appears to be the case for all sessions.
I know you started the course with py3.0
Some people like using 2.7 (like me). Not complaining, just if I may, I'd like to include the changes I'd make in code accordingly for 2.7.
Like the one in session - 0 notebook, urllib's request module doesn't exist for 2.7, so
basically two places you have change
from urllib import request
becomes
import urllib
And
urllib.request.urlretrieve(....
becomes
urllib.urlretrieve(....
Also
print (url, end=..)
will not work. So may be we can just write
print (url)
Should retrain a new model on the newer batch norm implementation in tf.contrib
In the file "session-3.ipynb", the line
recon = utils.montage(clipped)
should be replaced with
recon = utils.montage(clipped).astype(np.uint8)
Otherwise the generated pictures will not display correctly.
Hey @pkmital,
Awesome work!! I am really excited to dig into what you have going on. I really appreciate how well you document tf and offer docker builds with and without gpu.
I am having a problem building the gpu docker file. Of course it has to do with cuDNN. I am not sure why they make it so difficult to get a hold of that. Anyways, here is my last few lines of my docker build output:
sha256sum: WARNING: 1 computed checksum did NOT match
The command '/bin/sh -c CUDNN_DOWNLOAD_SUM=a87cb2df2e5e7cc0a05e266734e679ee1a2fadad6f06af82a76ed81a23b102c8 && curl -fsSL http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-x64-v5.1.tgz -O && echo "$CUDNN_DOWNLOAD_SUM cudnn-8.0-linux-x64-v5.1.tgz" | sha256sum -c --strict - && tar -xzf cudnn-8.0-linux-x64-v5.1.tgz -C /usr/local --wildcards 'cuda/lib64/libcudnn.so.*' && rm cudnn-8.0-linux-x64-v5.1.tgz && ldconfig && ln -s /usr/local/cuda/lib64/libcudnn.so.5 /usr/local/cuda/lib64/libcudnn.so' returned a non-zero code: 1
Maybe you want to use the caffe:0.15 base image (#71) to avoid the problem? I think they are different between ubuntu 14 and 16.
Hei!
I'm working on a Mac and (think) I don't really need to use docker, since I have all the libraries (including tensorflow) installed and also jupyter notebooks running.
I was thinking that you could add instructions about working with such a setup - or is there a specific reason to discourage this?
Thanks : )
The n_epochs variable goes unused in dataset_utils.create_input_pipeline(), so the line
producer = tf.train.string_input_producer(
files, capacity=len(files))
should probably be
producer = tf.train.string_input_producer(
files, num_epochs=n_epochs, capacity=len(files))
However, I also had to add a call to sess.run(tf.local_variables_initializer()) after making that change.
The text reads:
"By default, this will just return the first 1000 images because loading the entire dataset is a bit cumbersome"
The utils function only loads 100 images, not 1000.
I wanted to follow along with the video but I am having issues with copy-and-pasting each command. What to do so I have the iPython notebook?
For session 2 the line of instruction that says: (the tf.random_normal_intializer you should create)
the world 'intializer' is misspelled, it should be initializer
Thanks!
Alonso
The VAEGAN training code saves checkpoints using the value of the global training step, which results in checkpoints with names like 'vaegan.ckpt-800.index', for example. Any code that looks for an existing checkpoint also needs to account for this naming scheme, but the existence check used doesn't quite work with this scheme:
if os.path.exists(ckpt_name + '.index') or os.path.exists(ckpt_name):
I would suggest changing the check to something like this:
latest_checkpoint = tf.train.latest_checkpoint(os.path.dirname(ckpt_name))
if latest_checkpoint:
saver.restore(sess, latest_checkpoint)
print("Model restored from checkpoint {}.".format(latest_checkpoint))
else:
print("Model checkpoint not found.")
(This won't quite work if checkpoints from multiple models are created in the same directory, since it relies on the presence of a file named 'checkpoint'.)
this link in the readme to installing tensorflow via "pip using a 64-bit Python 3.5 environment" is broken: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/get_started/os_setup.md#pip-installation-on-windows
Session3 vae.VAE code for decoding layers is
shapes.reverse()
n_filters.reverse()
Ws.reverse()
n_filters += [input_shape[-1]]
# %%
# Decoding layers
for layer_i, n_output in enumerate(n_filters[1:]):
with tf.variable_scope('decoder/{}'.format(layer_i)):
shape = shapes[layer_i + 1]
if convolutional:
h, W = utils.deconv2d(x=current_input,
n_output_h=shape[1],
n_output_w=shape[2],
n_output_ch=shape[3],
n_input_ch=shapes[layer_i][3],
k_h=filter_sizes[layer_i],
k_w=filter_sizes[layer_i])
else:
h, W = utils.linear(x=current_input,
n_output=n_output)
h = activation(batch_norm(h, phase_train, 'dec/bn' + str(layer_i)))
if dropout:
h = tf.nn.dropout(h, keep_prob)
current_input = h
y = current_input
x_flat = utils.flatten(x)
y_flat = utils.flatten(y)
# l2 loss
loss_x = tf.reduce_sum(tf.squared_difference(x_flat, y_flat), 1)
This code seems to create new variables in utils.deconv2d and utils.linear
def linear(x, n_output, name=None, activation=None, reuse=None):
"""Fully connected layer.
Parameters
----------
x : tf.Tensor
Input tensor to connect
n_output : int
Number of output neurons
name : None, optional
Scope to apply
Returns
-------
h, W : tf.Tensor, tf.Tensor
Output of fully connected layer and the weight matrix
"""
if len(x.get_shape()) != 2:
x = flatten(x, reuse=reuse)
n_input = x.get_shape().as_list()[1]
with tf.variable_scope(name or "fc", reuse=reuse):
W = tf.get_variable(
name='W',
shape=[n_input, n_output],
dtype=tf.float32,
initializer=tf.contrib.layers.xavier_initializer())
b = tf.get_variable(
name='b',
shape=[n_output],
dtype=tf.float32,
initializer=tf.constant_initializer(0.0))
h = tf.nn.bias_add(
name='h',
value=tf.matmul(x, W),
bias=b)
if activation:
h = activation(h)
return h, W
rather than using transposed Ws from encoding layers such as in session 3 lecture
for layer_i, n_output in enumerate(dimensions):
# we'll use a variable scope again to help encapsulate our variables
# This will simply prefix all the variables made in this scope
# with the name we give it.
with tf.variable_scope("decoder/layer/{}".format(layer_i)):
# Now we'll grab the weight matrix we created before and transpose it
# So a 3072 x 784 matrix would become 784 x 3072
# or a 256 x 64 matrix, would become 64 x 256
W = tf.transpose(Ws[layer_i])
b = tf.get_variable(
name='b',
shape=[n_output],
dtype=tf.float32,
initializer=tf.constant_initializer(0.0))
# Now we'll multiply our input by our transposed W matrix
# and add the bias
h = tf.nn.bias_add(
name='h',
value=tf.matmul(current_input, W),
bias=b)
# And then use a relu activation function on its output
current_input = tf.nn.relu(h)
# We'll also replace n_input with the current n_output, so that on the
# next iteration, our new number inputs will be correct.
n_input = n_output
Why? I am a rookie and don't understand. Please explain.
Code:
examples_sorted = []
for i in indexes[1]:
examples_sorted.append(examples[i])
plt.figure(figsize=(15, 15))
img = utils.montage(np.array(examples_sorted)).astype(np.uint8)
plt.imshow(img,
interpolation='nearest')
plt.imsave(arr=img, fname='sorted.png')
Errror:
IndexError: index 3479 is out of bounds for axis 0 with size 100.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.