hvass-labs / tensorflow-tutorials

TensorFlow Tutorials with YouTube Videos

TensorFlow Tutorials

Original repository on GitHub

Original author is Magnus Erik Hvass Pedersen

Introduction

• These tutorials are intended for beginners in Deep Learning and TensorFlow.
• Each tutorial covers a single topic.
• The source-code is well-documented.
• There is a YouTube video for each tutorial.

Tutorials for TensorFlow 2

The following tutorials have been updated and work with TensorFlow 2 (some of them run in "v.1 compatibility mode").

1. Simple Linear Model (Notebook) (Google Colab)

2. Convolutional Neural Network (Notebook) (Google Colab)

3-C. Keras API (Notebook) (Google Colab)

10. Fine-Tuning (Notebook) (Google Colab)

13-B. Visual Analysis for MNIST (Notebook) (Google Colab)

16. Reinforcement Learning (Notebook) (Google Colab)

17. Hyper-Parameter Optimization (Notebook) (Google Colab)

18. Natural Language Processing (Notebook) (Google Colab)

19. Machine Translation (Notebook) (Google Colab)

20. Image Captioning (Notebook) (Google Colab)

21. Time-Series Prediction (Notebook) (Google Colab)

Tutorials for TensorFlow 1

The following tutorials only work with the older TensorFlow 1 API, so you would need to install an older version of TensorFlow to run these. It would take too much time and effort to convert these tutorials to TensorFlow 2.

3. Pretty Tensor (Notebook) (Google Colab)

3-B. Layers API (Notebook) (Google Colab)

4. Save & Restore (Notebook) (Google Colab)

5. Ensemble Learning (Notebook) (Google Colab)

6. CIFAR-10 (Notebook) (Google Colab)

7. Inception Model (Notebook) (Google Colab)

8. Transfer Learning (Notebook) (Google Colab)

9. Video Data (Notebook) (Google Colab)

10. Adversarial Examples (Notebook) (Google Colab)

11. Adversarial Noise for MNIST (Notebook) (Google Colab)

12. Visual Analysis (Notebook) (Google Colab)

13. DeepDream (Notebook) (Google Colab)

14. Style Transfer (Notebook) (Google Colab)

15. Estimator API (Notebook) (Google Colab)

16. TFRecords & Dataset API (Notebook) (Google Colab)

Videos

These tutorials are also available as YouTube videos.

Translations

These tutorials have been translated to the following languages:

• Chinese

New Translations

You can help by translating the remaining tutorials or reviewing the ones that have already been translated. You can also help by translating to other languages.

It is a very big job to translate all the tutorials, so you should just start with Tutorials #01, #02 and #03-C which are the most important for beginners.

New Videos

You are also very welcome to record your own YouTube videos in other languages. It is strongly recommended that you get a decent microphone because good sound quality is very important. I used vokoscreen for recording the videos and the free DaVinci Resolve for editing the videos.

Forks

See the selected list of forks for community modifications to these tutorials.

Installation

There are different ways of installing and running TensorFlow. This section describes how I did it for these tutorials. You may want to do it differently and you can search the internet for instructions.

If you are new to using Python and Linux then this may be challenging to get working and you may need to do internet searches for error-messages, etc. It will get easier with practice. You can also run the tutorials without installing anything by using Google Colab, see further below.

Some of the Python Notebooks use source-code located in different files to allow for easy re-use across multiple tutorials. It is therefore recommended that you download the whole repository from GitHub, instead of just downloading the individual Python Notebooks.

Git

The easiest way to download and install these tutorials is by using git from the command-line:

git clone https://github.com/Hvass-Labs/TensorFlow-Tutorials.git

This will create the directory TensorFlow-Tutorials and download all the files to it.

This also makes it easy to update the tutorials, simply by executing this command inside that directory:

git pull

Download Zip-File

You can also download the contents of the GitHub repository as a Zip-file and extract it manually.

Environment

I use Anaconda because it comes with many Python packages already installed and it is easy to work with. After installing Anaconda, you should create a conda environment so you do not destroy your main installation in case you make a mistake somewhere:

conda create --name tf python=3

When Python gets updated to a new version, it takes a while before TensorFlow also uses the new Python version. So if the TensorFlow installation fails, then you may have to specify an older Python version for your new environment, such as:

conda create --name tf python=3.6

Now you can switch to the new environment by running the following (on Linux):

source activate tf

Required Packages

The tutorials require several Python packages to be installed. The packages are listed in requirements.txt

To install the required Python packages and dependencies you first have to activate the conda-environment as described above, and then you run the following command in a terminal:

pip install -r requirements.txt

Starting with TensorFlow 2.1 it includes both the CPU and GPU versions and will automatically switch if you have a GPU. But this requires the installation of various NVIDIA drivers, which is a bit complicated and is not described here.

Python Version 3.5 or Later

These tutorials were developed on Linux using Python 3.5 / 3.6 (the Anaconda distribution) and PyCharm.

There are reports that Python 2.7 gives error messages with these tutorials. Please make sure you are using Python 3.5 or later!

How To Run

If you have followed the above installation instructions, you should now be able to run the tutorials in the Python Notebooks:

cd ~/development/TensorFlow-Tutorials/  # Your installation directory.
jupyter notebook

This should start a web-browser that shows the list of tutorials. Click on a tutorial to load it.

Run in Google Colab

If you do not want to install anything on your own computer, then the Notebooks can be viewed, edited and run entirely on the internet by using Google Colab. There is a YouTube video explaining how to do this. You click the "Google Colab"-link next to each tutorial listed above. You can view the Notebook on Colab but in order to run it you need to login using your Google account. Then you need to execute the following commands at the top of the Notebook, which clones the contents of this repository to your work-directory on Colab.

# Clone the repository from GitHub to Google Colab's temporary drive.
import os
work_dir = "/content/TensorFlow-Tutorials/"
if not os.path.exists(work_dir):
    !git clone https://github.com/Hvass-Labs/TensorFlow-Tutorials.git
os.chdir(work_dir)

All required packages should already be installed on Colab, otherwise you can run the following command:

!pip install -r requirements.txt

Older Versions

Sometimes the source-code has changed from that shown in the YouTube videos. This may be due to bug-fixes, improvements, or because code-sections are moved to separate files for easy re-use.

If you want to see the exact versions of the source-code that were used in the YouTube videos, then you can browse the history of commits to the GitHub repository.

License (MIT)

These tutorials and source-code are published under the MIT License which allows very broad use for both academic and commercial purposes.

A few of the images used for demonstration purposes may be under copyright. These images are included under the "fair usage" laws.

You are very welcome to modify these tutorials and use them in your own projects. Please keep a link to the original repository.

tensorflow-tutorials's Issues

PrittyTensor not working with TensorFlow r1.0

Here the source code for Tutorial #4 modified version, replacing PrettyTensor with TensorFlow primitives:

https://github.com/miga101/tf_mnist_cnn

I will try to modify the others too..

misspelling in 07_Inception_Model

In tutorial 07_Inception_Model

We can see that it has been squeezed so it is rectangular, and the resolution has been reduced so the image has become more pixelated and grainy.

The best way to input images to this Inception model, is to pad the image so it is rectangular and then resize the image to 299 x 299 pixels.

I think it should be square rather than rectangular here.

FailedPreconditionError with Pretty tensor tutorial 3

Hi,

When running your tutorial 3 on pretty tensor, I get FailedPreconditionError. I am using tensorflow_gpu-0.12.0rc0-cp27-none-linux_x86_64 and prettytensor-0.6.2 with python2.7. The error occurs at the first print_test_accuracy() call.

When I searched on internet, I see that this error can occur if the session variables are not initialized.
In your tutorial they are initialized. Moreover I haven't received this error with first two tutorials.

I am wondering if this has something to do with prettytensor variables. Below is the trace back error message.

Use `tf.global_variables_initializer` instead.
Traceback (most recent call last):
  File "train.py", line 139, in <module>
    print_test_accuracy(dataset.data)
  File "train.py", line 88, in print_test_accuracy
    cls_pred[i:j] = session.run(y_pred_cls, feed_dict=feed_dict)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 766, in run
    run_metadata_ptr)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 964, in _run
    feed_dict_string, options, run_metadata)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1014, in _do_run
    target_list, options, run_metadata)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1034, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value layer_conv1/weights
	 [[Node: layer_conv1/weights/read = Identity[T=DT_FLOAT, _class=["loc:@layer_conv1/weights"], _device="/job:localhost/replica:0/task:0/cpu:0"](layer_conv1/weights)]]

Caused by op u'layer_conv1/weights/read', defined at:
  File "train.py", line 4, in <module>
    from debug import *
  File "/home/badami/Codes/deeplearning/src/debug.py", line 4, in <module>
    from param import *
  File "/home/badami/Codes/deeplearning/src/param.py", line 14, in <module>
    y_pred, loss = model1(x_image, y_true)
  File "/home/badami/Codes/deeplearning/src/models.py", line 11, in model1
    conv2d(kernel=5, depth=16, name='layer_conv1').\
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/prettytensor/pretty_tensor_class.py", line 1981, in method
    result = func(non_seq_layer, *args, **kwargs)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/prettytensor/pretty_tensor_image_methods.py", line 159, in __call__
    params = self.variable('weights', size, init, dt=dtype)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/prettytensor/pretty_tensor_class.py", line 1695, in variable
    collections=variable_collections)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1024, in get_variable
    custom_getter=custom_getter)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 850, in get_variable
    custom_getter=custom_getter)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 346, in get_variable
    validate_shape=validate_shape)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 331, in _true_getter
    caching_device=caching_device, validate_shape=validate_shape)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 677, in _get_single_variable
    expected_shape=shape)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 224, in __init__
    expected_shape=expected_shape)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 370, in _init_from_args
    self._snapshot = array_ops.identity(self._variable, name="read")
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1424, in identity
    result = _op_def_lib.apply_op("Identity", input=input, name=name)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
    op_def=op_def)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/badami/Codes/deeplearning/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1128, in __init__
    self._traceback = _extract_stack()

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value layer_conv1/weights
	 [[Node: layer_conv1/weights/read = Identity[T=DT_FLOAT, _class=["loc:@layer_conv1/weights"], _device="/job:localhost/replica:0/task:0/cpu:0"](layer_conv1/weights)]]

softmax_classifier() unexpected keyword 'class_count'

Hi,

I'm running tf 0.12.0-rc0, CPU on Ubuntu 14.04 64-bit, Python 3.5
My prettytensor is '0.7.1'

I'm runing tutorial 9 and in wrapping the transfer-values as a Pretty Tensor object I get this error

softmax_classifier() got an unexpected keyword argument 'class_count'

I skipped tutorial 3 where prettytensor is introduced so I didn't run the code there but I would expect the same error with softmax.
I read issue #9 and it pointed out that version mismatch between pt and tf can cause errors but in my case I have the latest of both, to the best of my knowledge. my tf environment is initialized to python 3.5 by default.

Here is the full error

TypeError                                 Traceback (most recent call last)
<ipython-input-97-00471178f1d1> in <module>()
      3 
      4 with pt.defaults_scope(activation_fn=tf.nn.relu):
----> 5     y_pred, loss = x_pretty.        fully_connected(size=1024, name='layer_fc1').        softmax_classifier(class_count=num_classes, labels=y_true)

/home/ros/anaconda3/envs/tf/lib/python3.5/site-packages/prettytensor/pretty_tensor_class.py in method(input_layer, *args, **kwargs)
   2007     @functools.wraps(func)
   2008     def method(input_layer, *args, **kwargs):
-> 2009       return func(input_layer, *args, **self.fill_kwargs(input_layer, kwargs))
   2010 
   2011     return method

TypeError: softmax_classifier() got an unexpected keyword argument 'class_count'```

In implementing Tutorial 15, there is a URL error for downloading vgg16 model,

STOP!

Most of the problems people are having are already described in the installation instructions.

Python 3.5

These tutorials were developed in Python 3.5 and may give strange errors in Python 2.7

Missing Files

You need to download the whole repository, either using git clone or as a zip-file. See the installation instructions.

Questions about TensorFlow

General questions about TensorFlow should either be asked on StackOverflow or GitHub.

Suggestions for Changes

The tutorials cannot change too much because it would make the YouTube videos too different from the source-code.

Requests for New Tutorials

These tutorials were made by a single person on his own time. It took a very long time to
research and produce the tutorials. If a topic is not covered then the best thing is to make
a new tutorial by yourself. All you need is a decent microphone, a screen-grabbing tool, and a
video editor. I used the free version of DaVinci Resolve.

Other Issues?

Please carefully read the installation instructions and only open an issue if you are still having problems.

Error at protobuf decoding for VGG init.

Hi,

When I run the tutorial 15, I got the DecodeError: Truncated message. The error is due to extract the graph definition from vgg. The data length is not matched with the expected size. It seems fine on Ubuntu. What may cause this problem? Thanks for any reply.

--
Here is my environment:
OS X Sierra
ipython (5.1.0)
jupyter (1.0.0)
protobuf (3.1.0.post1)
tensorflow (0.12.1)
tensorflow-gpu (0.12.1)
python 3.5.2�

The following is the detail.

in style_transfer(content_image, style_image, content_layer_ids, style_layer_ids, weight_content, weight_style, weight_denoise, num_iterations, step_size)
27 # operations to the graph so it can grow very large
28 # and run out of RAM if we keep using the same instance.
---> 29 model = vgg16.VGG16()
30
31 # Create a TensorFlow-session.

/Users/hsucw/Documents/TensorFlow-Tutorials/vgg16.py in init(self)
111
112 # Then we load the proto-buf file into the graph-def.
--> 113 graph_def.ParseFromString(file.read())
114
115 # Finally we import the graph-def to the default TensorFlow graph.

/Users/hsucw/anaconda3/envs/tf/lib/python3.5/site-packages/google/protobuf/message.py in ParseFromString(self, serialized)
183 """
184 self.Clear()
--> 185 self.MergeFromString(serialized)
186
187 def SerializeToString(self):

/Users/hsucw/anaconda3/envs/tf/lib/python3.5/site-packages/google/protobuf/internal/python_message.py in MergeFromString(self, serialized)
1085 length = len(serialized)
1086 try:
-> 1087 if self._InternalParse(serialized, 0, length) != length:
1088 # The only reason _InternalParse would return early is if it
1089 # encountered an end-group tag.

/Users/hsucw/anaconda3/envs/tf/lib/python3.5/site-packages/google/protobuf/internal/python_message.py in InternalParse(self, buffer, pos, end)
1121 pos = new_pos
1122 else:
-> 1123 pos = field_decoder(buffer, new_pos, end, self, field_dict)
1124 if field_desc:
1125 self._UpdateOneofState(field_desc)

/Users/hsucw/anaconda3/envs/tf/lib/python3.5/site-packages/google/protobuf/internal/decoder.py in DecodeRepeatedField(buffer, pos, end, message, field_dict)
608 new_pos = pos + size
609 if new_pos > end:
--> 610 raise _DecodeError('Truncated message.')
611 # Read sub-message.
612 if value.add()._InternalParse(buffer, pos, new_pos) != new_pos:

DecodeError: Truncated message.

Can I run these in latest tensorlfow version?

STOP!

Most of the problems people are having are already described in the installation instructions.

Python 3.5

These tutorials were developed in Python 3.5 and may give strange errors in Python 2.7

Missing Files

You need to download the whole repository, either using git clone or as a zip-file. See the installation instructions.

Questions about TensorFlow

General questions about TensorFlow should either be asked on StackOverflow or GitHub.

Suggestions for Changes

The tutorials cannot change too much because it would make the YouTube videos too different from the source-code.

Requests for New Tutorials

These tutorials were made by a single person on his own time. It took a very long time to
research and produce the tutorials. If a topic is not covered then the best thing is to make
a new tutorial by yourself. All you need is a decent microphone, a screen-grabbing tool, and a
video editor. I used the free version of DaVinci Resolve.

Other Issues?

Please carefully read the installation instructions and only open an issue if you are still having problems.

np eye visualization.

result = np.eye(num_classes, dtype=float)
print(result)
ans = result[class_numbers]

i do not understand the array qualifier on the result for no.eye(). for lesson 9, you had over 4170 class_numbers as a list.

np.eye(3) returns
array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1]])
so how array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1]]) [4170] works?

i am seriously confused.

How can we use our own model with DeepDream example?

I have followed your tutorial so far and noticed deepdream and wanted to give it a try!
I noticed you are using the google's inception pretrained model.
I wanted to experiment a little bit, so trained a cifar10 model according to the tutorial #6, but the checkpoints seem not to be usable with the deepdream tutorial .
Is it possible for you to add a new tutorial or show in the deepdream tutorial how we can use our own trained models instead of inceptions?

Classification accuracy of the CIFAR10 model

What is the classification accuracy of the CIFAR10 model? When I train the network using tensorflow and GPU's it does not increase beyond 60%. Please confirm if this is the maximum accuracy that this model attained when you were training it.

Reinforcement Learning Experiments

This thread is for documenting your experiments with Reinforcement Learning in Tutorial 16.

It is also interesting for others to hear if your experiment failed, so they don't have to repeat the same mistakes.

Please write the following for each of your experiments:

• English description of the experiment.
• List the experimental details including the hyper-parameters you have used, how many states and episodes you have trained, etc.
• What are the test-results (and which epsilon value are you using during testing).
• Provide a link to your modified code as a Gist on GitHub.

Not an issue - conda env export fyi

Hi Magnus,

You tutorials have come a long way since I last looked.
You should put a bitcoin /ethereum donation link on your page - some one may feel like tipping you.

fyi - When you create a conda environment and activate it - and then do pip install - it transparently stores these dependencies into respective environment.

Did you know it's possible to spit out an associated environments.yml file with all dependencies for conda? eg after you've done pip install requirements
conda env export > environment.yml

you can then potentially discard using pip.
to restore environment - you can call
conda env create --file=/path/to/envfile.yaml

why would you want to do this? if users skip the conda / source activate part - then they can get in trouble with which version of pip conda is referencing (eg python2).

you may also like this conda plugin in conjunction with jupyter notebooks to simplify the environment settings.
conda install nb_conda

I have some screen shots in this gist (as well as my own ML conda dependencies here)
https://gist.github.com/johndpope/187b0dd996d16152ace2f842d43e3990
you can then easily configure your jupyter notebook to point to different environments
python2 - tensorflow
or
python 3 - tensorflow from within jupyter notebook. (again see gist link for better understanding)

I know I recommended pip to begin with - but now days am relying almost exclusively on conda to manage environments / dependencies.

also - there's this cheat sheet
https://conda.io/docs/_downloads/conda-cheatsheet.pdf

finally - having watched your videos - I've been throwing a lot more comments in my code these days and I think my colleagues appreciate that and they have been doing like wise - which I do appreciate. so thanks for taking time to teach people.

translate to chinese

Hi, I have translated this tutorial to chinese, tutorial #1 - #14 were finished so far.
You can add this link to your repo if you want =)
https://github.com/thrillerist/TensorFlow-Tutorials

Trying to use Conv nets with the knifey dataset

I'm trying to train the neural network in tutorial 02 from @Hvass-Labs using the knifey dataset. All is fine until I try to run a modified version of the print_test_accuracy() function.
In the while loop, images and labels are loaded correctly but when it tries to run the session to predict the result, everything crashes and the python kernel has to be restarted.
I modified placeholder x to have the following shape:
x = tf.placeholder(tf.float32, shape=[None, img_height, img_width, num_channels], name='x')

Could you please help me understand what am I doing wrong?
I pasted the modified print_test_accuracy() function below.
Thanks a lot for you help,

`# Split the test-set into smaller batches of this size.
test_batch_size = 64

def print_test_accuracy(show_example_errors=False,
show_confusion_matrix=False):

# Number of images in the test-set.
num_test = len(image_paths_test)

# Allocate an array for the predicted classes which
# will be calculated in batches and filled into this array.
cls_pred = np.zeros(shape=num_test, dtype=np.int)

# Now calculate the predicted classes for the batches.
# We will just iterate through all the batches.
# There might be a more clever and Pythonic way of doing this.

# The starting index for the next batch is denoted i.
i = 0

while i < num_test:
    # The ending index for the next batch is denoted j.
    j = min(i + test_batch_size, num_test)

    # Get the images from the test-set between index i and j.
    images = load_images(image_paths=image_paths_test[i:j])

    # Get the associated labels.
    labels = labels_test[i:j]

    # Create a feed-dict with these images and labels.
    feed_dict = {x: images, y_true: labels}

    # Calculate the predicted class using TensorFlow.
    cls_pred[i:j] = session.run(y_pred_cls, feed_dict=feed_dict)

    # Set the start-index for the next batch to the
    # end-index of the current batch.
    i = j

# Convenience variable for the true class-numbers of the test-set.
cls_true = test_classes

# Create a boolean array whether each image is correctly classified.
correct = (cls_true == cls_pred)

# Calculate the number of correctly classified images.
# When summing a boolean array, False means 0 and True means 1.
correct_sum = correct.sum()

# Classification accuracy is the number of correctly classified
# images divided by the total number of images in the test-set.
acc = float(correct_sum) / num_test

# Print the accuracy.
msg = "Accuracy on Test-Set: {0:.1%} ({1} / {2})"
print(msg.format(acc, correct_sum, num_test))

# Plot some examples of mis-classifications, if desired.
if show_example_errors:
    print("Example errors:")
    plot_example_errors(cls_pred=cls_pred, correct=correct)

# Plot the confusion matrix, if desired.
if show_confusion_matrix:
    print("Confusion Matrix:")
    plot_confusion_matrix(cls_pred=cls_pred) `

import cifar10 gives : ImportError: No module named 'cifar10'

When try to run your CIFAR10 demo code, I get
ImportError: No module named 'cifar10'
I searched and even tried
from tensorflow.image.cifar10 import cifar10
but this fails as well and says :
ImportError: No module named 'tensorflow.image'
Whats wrong here? is that a typo?
Should I be creating my own cifar10?
by the way I'm on version 0.10.0 as well

Can you also create a demo/example for Conv3D as well?

Hi,
I saw your video data example, and thought that you were going to talk about conv3d /N-D conv.
But apparently I was wrong. However, I searched for such a demo, but could not find any examples for tensorflow in this regard.
Would you please do a tutorial on this as well ?
Thank you very much

Find Graph definition for transfer learning

I found that if I want to use new pre-trained model, I need a file which is for the graph definition. However, I cannot find this file on the google research blog where the checkpoints and source code of new models can be downloaded. How to write such a graph definition file or how to download it? Moreover, based on the Transfer learning tutorial, how can I let the entire network fine tuning with a slow learning rate instead of only the last layer?

Hi, I use python 3.5 and tensorflow 1.0.1 . and when I download and load the inception model in tutorial no. 7 my python program gives the error " stop working" . can you please help me?

STOP!

Most of the problems people are having are already described in the installation instructions.

Python 3.5

These tutorials were developed in Python 3.5 and may give strange errors in Python 2.7

Missing Files

You need to download the whole repository, either using git clone or as a zip-file. See the installation instructions.

Questions about TensorFlow

General questions about TensorFlow should either be asked on StackOverflow or GitHub.

Suggestions for Changes

The tutorials cannot change too much because it would make the YouTube videos too different from the source-code.

Requests for New Tutorials

These tutorials were made by a single person on his own time. It took a very long time to
research and produce the tutorials. If a topic is not covered then the best thing is to make
a new tutorial by yourself. All you need is a decent microphone, a screen-grabbing tool, and a
video editor. I used the free version of DaVinci Resolve.

Other Issues?

Please carefully read the installation instructions and only open an issue if you are still having problems.

07_Inception_Model.ipynb give different results ?

I went through this notebook with given introduction. But when I use classify(image_path) function in the notebook it doesn't give correct predictions all the time.
If I test it on the given panda image it will first predict correct labels. But if I run in again in will give results like website, web .. etc

protobuf error

dataset.py mistake

at dataset.py line 48, should be "np.max(class_numbers) + 1" and not - 1 i believe

cifar10 training

i trained my cifar10 based on your notebook file and cifar10.py to read. but batch accuracy only 10% with any cnn architectures and data augmentation or not (with 100000 iterations). do you know what happened? thanks!

Pretty tensor with tensorboard

Sorry for opening an issue for this question.
Are you planning to add a tutorial for Tensorboard with prettytensor?
I figured how to use tensorboard with low level tensorflow network modelling but still do not understand how I can use it with prettytensor.

Tensor flow compile error

Hi ,

I am having the following error in running the CIFAR10 dataset training . Can you give me some where to fix this issue.
"
File "C:\Users\Prabir Sinha\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\ops\array_ops.py", line 1001, in concat
).assert_is_compatible_with(tensor_shape.scalar())
File "C:\Users\Prabir Sinha\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\tensor_shape.py", line 756, in assert_is_compatible_with
raise ValueError("Shapes %s and %s are incompatible" % (self, other))
ValueError: Shapes (2, 1) and () are incompatible "

Thanks
Prabir

Creating your own dataset

Hi , I am trying to train the model on my own dataset.
In what format should I prepare my dataset?

Prediction in new image :Tutorial 9

Dear Hvass
I am trying to test your tensorflow codes. I started by making a classification of my images and I applied all the steps in the tutorial 9 and all these good.

But the problem is that I do not know how to actually test it in other images are not in the folder trainnig test.
I want to evaluate the model by other images and that gives me percentage predictions like retraining in tensorflow examples. I give it an image and it tells me in which class it appertains and here is the code that I use it for the evaluation:

import tensorflow as tf
import sys

change this as you see fit

image_path = sys.argv[1]

Read in the image_data

image_data = tf.gfile.FastGFile(image_path, 'rb').read()

Loads label file, strips off carriage return

label_lines = [line.rstrip() for line
in tf.gfile.GFile("/home/ubuntu/labels.txt")]

Unpersists graph from file

with tf.gfile.FastGFile("/home/ubuntu/graph.pb", 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')

with tf.Session() as sess:
# Feed the image_data as input to the graph and get first prediction
softmax_tensor = sess.graph.get_tensor_by_name('final_result:0')

predictions = sess.run(softmax_tensor, \
         {'DecodeJpeg/contents:0': image_data})

# Sort to show labels of first prediction in order of confidence
top_k = predictions[0].argsort()[-len(predictions[0]):][::-1]

for node_id in top_k:
    human_string = label_lines[node_id]
    score = predictions[0][node_id]
    print('%s (score = %.5f)' % (human_string, score))

---

And here is the command that I use it: python3 test.py image.jpg (New image)
Result: 0.87 C
0. 13 O

But this script uses the graph of the inception, but for our case in the tutorial it is registered "pkl" either the classes or the training.
Or, How to export my training model in graph.pb t to use the same script above to do the prediction just I will replace the inception graph with my cnn training graph

Thank you.

AttributeError: 'DataSet' object has no attribute 'cls'

Getting an error while reading MNIST data in
"TensorFlow-Tutorials/02_Convolutional_Neural_Network.ipynb"

cls_true = data.test.cls[0:9]
AttributeError: 'DataSet' object has no attribute 'cls'

I'm using Tenserflow Version: '1.3.0'

Style Transfer Question

In this photo from youtube,Can you tell me what parameters you used for generating these result?
I hope I can get a detail answer,Thanks

A question about strides in conv and pooling layers

Hi,
What exactly are the first and last dimensions of strides for ?
In the second example you explained :

# Create the TensorFlow operation for convolution.
  # Note the strides are set to 1 in all dimensions.
  # The first and last stride must always be 1,
  # because the first is for the image-number and
  # the last is for the input-channel.
  # But e.g. strides=[1, 2, 2, 1] would mean that the filter
  # is moved 2 pixels across the x- and y-axis of the image.
  # The padding is set to 'SAME' which means the input image
  # is padded with zeroes so the size of the output is the same.
  layer = tf.nn.conv2d(input=input,
                       filter=weights,
                       strides=[1, 1, 1, 1],
                       padding='SAME')

you said, everything must be set to 1, the first dimension denotes the image-number (do you mean batch size?!) and the last one is for input-channel.
Is it 1 because mnist is black and white and has 1 input-channel? should we set the last dimension to 3 when dealing with color images?

if use_pooling:
      # This is 2x2 max-pooling, which means that we
      # consider 2x2 windows and select the largest value
      # in each window. Then we move 2 pixels to the next window.
      layer = tf.nn.max_pool(value=layer,
                             ksize=[1, 2, 2, 1],
                             strides=[1, 2, 2, 1],
                             padding='SAME')

what are the ones here as well?
this is really confusing

Connecting

Hi @Hvass-Labs,

I've forked your work and packaged it as a "Serverless" OpenFaaS function”. This effectively makes TensorFlow and ImageNet a web-service which takes a URL as input and gives categorisations back as a JSON response.

https://github.com/faas-and-furious/inception-function

I have several opportunities to demonstrate machine-learning with OpenFaaS at Dockercon EU (in two weeks) and KubeCon in December. I'd like to connect and talk more. If you're interested ping [email protected] or link back-to the repository.

The code has also been packed with Docker which can make the tutorial more accessible on any platform.

Store Trained Model Reference

Good day, how exactly can we store the trained model and use it in a custom API. The way it is currently it will always load a pickled model and train ?

Weird error when calling transfer_values_cache function

When I run the script and trying to call this function:

transfer_values_test = transfer_values_cache(file_path=file_path_cache_test, 
                                             images=images_scaled,
                                             model=model)

I face strange errror:

OSError                                   Traceback (most recent call last)
<ipython-input-13-d3f5056fe5a9> in <module>()
      9 transfer_values_test = transfer_values_cache(file_path=file_path_cache_test,
     10                                              images=images_scaled,
---> 11                                              model=model)

/Users/mylaptop/TensorFlow-Tutorials-master/inception.py in transfer_values_cache(file_path, images, model)
    553         # Load the transfer-values from the file.
    554         with open(file_path, mode='rb') as file:
--> 555             transfer_values = np.load(file)
    556 
    557         print("- Transfer-values loaded from cache-file.")

/Users/mylaptop/anaconda3/envs/tensorflow/lib/python3.5/site-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
    390         N = len(format.MAGIC_PREFIX)
    391         magic = fid.read(N)
--> 392         fid.seek(-N, 1)  # back-up
    393         if magic.startswith(_ZIP_PREFIX):
    394             # zip-file (assume .npz)

OSError: [Errno 22] Invalid argument

Does anyone have same issue or any idea how to solve it?
Maybe problem in my numpy library, but still have no idea what exactly is wrong.
My system: OSX El Capitan 10.11.6, Python 3.5 Anaconda

Problem with Exercise of 01_Simple_Linear_Model

This is my Codes that i am using:

`def begin(reg = 0):

x = tf.placeholder(tf.float32, [None, img_size_flat])
y_true = tf.placeholder(tf.float32, [None, num_classes]) # This will store the one-hot encoded labels for the image
y_true_cls = tf.placeholder(tf.int64, [None])
weights = tf.Variable(tf.zeros([img_size_flat, num_classes])) # This when multiply with x will give the probability of the image belonging to a particular class.
biases = tf.Variable(tf.zeros([num_classes]))
logits = tf.matmul(x, weights) + biases
y_pred = tf.nn.softmax(logits)
y_pred_cls = tf.argmax(y_pred, axis=1)
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(logits=logits,labels=y_true)
loss = tf.reduce_mean(cross_entropy)
regularizer = tf.nn.l2_loss(weights)
cost = tf.reduce_mean(loss + reg * regularizer)
`

`def optimize(num_iterations,batch_size = 100,lr = 0.5,print_loss = False,reg_lambda = 0,beta = 0):

begin(reg_lambda)
correct_prediction = tf.equal(y_pred_cls, y_true_cls)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
eff_loss = []
session = tf.Session()
session.run(tf.global_variables_initializer())
optimizer = tf.train.AdamOptimizer(learning_rate=lr).minimize(cost)

for i in range(num_iterations):
    # Get a batch of training examples.
    # x_batch now holds a batch of images and
    # y_true_batch are the true labels for those images.
    x_batch, y_true_batch = data.train.next_batch(batch_size) # 
    
    # Put the batch into a dict with the proper names
    # for placeholder variables in the TensorFlow graph.
    # Note that the placeholder for y_true_cls is not set
    # because it is not used during training.
    feed_dict_train = {x: x_batch,
                       y_true: y_true_batch}

    # Run the optimizer using this batch of training data.
    # TensorFlow assigns the variables in feed_dict_train
    # to the placeholder variables and then runs the optimizer.
    session.run(optimizer,feed_dict = feed_dict_train)
    eff_loss.append(loss)
    if print_loss and num_iterations % 10000 == 0:
        #loss = tf.reduce_mean(cross_entropy)
        print('The Loss is :',loss)
feed_dict_test = {x: data.test.images,
            y_true: data.test.labels,
            y_true_cls: data.test.cls}
# Use TensorFlow to compute the accuracy.
acc = session.run(accuracy, feed_dict=feed_dict_test)

# Print the accuracy.
print("Accuracy on test-set: {0:.1%}".format(acc))
plt.plot(np.arange(num_iterations),eff_loss)
plt.show()
session.close()

`

On running the below in jupyter notebook:
optimize(500,batch_size = 100,lr = 0.8,reg_lambda = 2,beta = 0.9)

I get the below error:

`
FailedPreconditionError Traceback (most recent call last)
~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1326 try:
-> 1327 return fn(*args)
1328 except errors.OpError as e:

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
1305 feed_dict, fetch_list, target_list,
-> 1306 status, run_metadata)
1307

~/anaconda3/envs/tensorflow/lib/python3.5/contextlib.py in exit(self, type, value, traceback)
65 try:
---> 66 next(self.gen)
67 except StopIteration:

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py in raise_exception_on_not_ok_status()
465 compat.as_text(pywrap_tensorflow.TF_Message(status)),
--> 466 pywrap_tensorflow.TF_GetCode(status))
467 finally:

FailedPreconditionError: Attempting to use uninitialized value beta1_power_7
[[Node: beta1_power_7/read = IdentityT=DT_FLOAT, _class=["loc:@Variable"], _device="/job:localhost/replica:0/task:0/cpu:0"]]

During handling of the above exception, another exception occurred:

FailedPreconditionError Traceback (most recent call last)
in ()
----> 1 optimize(500,batch_size = 100,lr = 0.8,reg_lambda = 2,beta = 0.9)

in optimize(num_iterations, batch_size, lr, print_loss, reg_lambda, beta)
24 # TensorFlow assigns the variables in feed_dict_train
25 # to the placeholder variables and then runs the optimizer.
---> 26 session.run(optimizer,feed_dict = feed_dict_train)
27 eff_loss.append(loss)
28 if print_loss and num_iterations % 10000 == 0:

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
893 try:
894 result = self._run(None, fetches, feed_dict, options_ptr,
--> 895 run_metadata_ptr)
896 if run_metadata:
897 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
1122 if final_fetches or final_targets or (handle and feed_dict_tensor):
1123 results = self._do_run(handle, final_targets, final_fetches,
-> 1124 feed_dict_tensor, options, run_metadata)
1125 else:
1126 results = []

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1319 if handle is None:
1320 return self._do_call(_run_fn, self._session, feeds, fetches, targets,
-> 1321 options, run_metadata)
1322 else:
1323 return self._do_call(_prun_fn, self._session, handle, feeds, fetches)

~/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1338 except KeyError:
1339 pass
-> 1340 raise type(e)(node_def, op, message)
1341
1342 def _extend_graph(self):

FailedPreconditionError: Attempting to use uninitialized value beta1_power_7
[[Node: beta1_power_7/read = IdentityT=DT_FLOAT, _class=["loc:@Variable"], _device="/job:localhost/replica:0/task:0/cpu:0"]]

Caused by op 'beta1_power_7/read', defined at:
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/ipykernel/kernelapp.py", line 478, in start
self.io_loop.start()
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/zmq/eventloop/ioloop.py", line 177, in start
super(ZMQIOLoop, self).start()
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tornado/ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
self._handle_recv()
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
self._run_callback(callback, msg)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 281, in dispatcher
return self.dispatch_shell(stream, msg)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 232, in dispatch_shell
handler(stream, idents, msg)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/ipykernel/kernelbase.py", line 397, in execute_request
user_expressions, allow_stdin)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/ipykernel/ipkernel.py", line 208, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/ipykernel/zmqshell.py", line 533, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2728, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2856, in run_ast_nodes
if self.run_code(code, result):
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/IPython/core/interactiveshell.py", line 2910, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
optimize(500,batch_size = 100,lr = 0.8,reg_lambda = 2,beta = 0.9)
File "", line 8, in optimize
optimizer = tf.train.AdamOptimizer(learning_rate=0.5).minimize(cost)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/optimizer.py", line 325, in minimize
name=name)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/optimizer.py", line 446, in apply_gradients
self._create_slots([_get_variable_for(v) for v in var_list])
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/adam.py", line 126, in _create_slots
trainable=False)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variable_scope.py", line 1679, in variable
caching_device=caching_device, name=name, dtype=dtype)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 199, in init
expected_shape=expected_shape)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 330, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1400, in identity
result = _op_def_lib.apply_op("Identity", input=input, name=name)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/saquib/anaconda3/envs/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1204, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value beta1_power_7
[[Node: beta1_power_7/read = IdentityT=DT_FLOAT, _class=["loc:@Variable"], _device="/job:localhost/replica:0/task:0/cpu:0"]]

`

Thank You for the Help

has the tensorflow module clstm((convolution Long short-term memory))

not an issue but an useful link

hi! thanks for the great tutorials.
but already know this (style transfer)?
https://github.com/log0/neural-style-painting

Wrong version of get_layer function getting called

I am working on retrieving Inception V3 model's top layer in Keras/Tensorflow (in Jupyter Notebook).

I could retrieve the Inception V3 model and its weights correctly. Now, I am trying to get Fully Connected layer (top layer) using following code snippet.

base_model = InceptionV3(weights=weights)
base_model.get_layer('flatten')

However, the function is failed saying "ValueError: No such layer: flatten"

When I looked at the stacktrace, get_layer() function from topology.py is getting called which is under 'keras/engine'.

Rather than this function, get_layer() function from models.py directly under keras should have been called.

What possibly can be the problem? How can I force Python to call the correct version? Or is there any other way to get the weights from InceptionV3 model?

Small confusion about convolution operator

Small confusion about how convolution is applied in
"TensorFlow-Tutorials/02_Convolutional_Neural_Network.ipynb"

Layer 1

28*28(1) -> Convolution (# Filters =16) -> 28*28(16) -> Max Pooling -> 14*14(16) -> Relu -> 14*14(16)

Layer 2

14*14(16) -> Convolution (# Filters =36) -> ? -> Max Pooling -> 7*7(36) -> Relu -> 7*7(36)

As in Layer 1 If we give an i/p of size 28*28 to the convolution (with 16 filters) then we have 16 Output of size 28*28 as each filter is applied on the i/p image. Now in Layer 2 If we give 16 i/p of size 14*14 to the convolution (with 36 filters) then we should have 16*36=576 Output of size 14*14 as each filter is applied on every i/p image, But according to this final o/p of Layer 2 will be different. Can anybody tell me how conv2d is applied in the second layer.

Import error, cannot import name one_hot_encoder

When I tried to import cifar10.py, this error occurs

 ImportError:   Traceback (most recent call last)
<ipython-input-2-4925188f42c0> in <module>()
----> 1 import cifar10

/Users/Neil/Documents/Developer/Machine Learning/Machine Learning with Python/CIFAR-10 Tensorflow/cifar10.py in <module>()
     37 import os
     38 import download
---> 39 from dataset import one_hot_encoded
     40 
     41 ########################################################################

ImportError: cannot import name one_hot_encoded

This can resolved by editing the cifar10.py file: replace from dataset import one_hot_encoded with from sklearn.preprocessing import OneHotEncoder

Notebook validation failed for TensorFlow Tutorial #07

I am reaching issue for 07_
TensorFlow Tutorial #7

Inception Model.

The error is following and I am using python 3.5. Other tutorial notebook works fine.

Notebook validation failed
An invalid notebook may not function properly. The validation error was:

Notebook Validation failed: {'data': {'image/jpeg': ['/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0a\nHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIy\nMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCABkAGQDA

5dR3Eiwwwyrzbs22/pSqKMDHBU5z1JbcmlQhWOoJWU7GnRmfOc0qVVJujyESOn3Eg3z3qU\nnvZxFjmpUqam9TEBeXDyh1kwysCGUjIIPUGq74t0HT7O0N9axGF2kCmND5N8747fht7UqVSpvcbi\nfQKNYzvSpVUiwzmtlO9KlWOBToFxLIiBmzyPsfwFEkUjAsc7nb6UqVT5PQom/O0nmY74HT4pUqVJ\nGH//2Q==\n'], 'text/plain': ['<IPython.core.display.Image object>']}, 'metadata': {}} is not valid under any of the given schemas:
{
"data": {
"image/jpeg": [
"/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0a\nHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIy\nMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCABkAGQDASIA\nAhEBAxEB/8QAHAAAAQQDAQAAAAAAAAAAAAAABgAEBQcBAgMI/8QANRAAAgEDAgQFAQYGAwEAAAAA\nAQIDAAQRBSEGEjFBEyJRYXGRBzKBobHBFBUjQlLwYtHh8f/EABkBAAMBAQEAAAAAAAAAAAAAAAID\nBAABBf/EAB8RAAICAwEBAAMAAAAAAAAAAAABAhEDEi

U5lXDDpjGaBB0qQ0u8Frc+bPI21LnG0\nZOmWzZ6gIIVYLzSYyBj1p218QlrHy+Zm8SRj27n8tqGLK9Ajj3PNzDv+VSi3DG88oGFjwM9vepZK\nhqZIza5dR3Eiwwwyrzbs22/pSqKMDHBU5z1JbcmlQhWOoJWU7GnRmfOc0qVVJujyESOn3Eg3z3qU\nnvZxFjmpUqam9TEBeXDyh1kwysCGUjIIPUGq74t0HT7O0N9axGF2kCmND5N8747fht7UqVSpvcbi\nfQKNYzvSpVUiwzmtlO9KlWOBToFxLIiBmzyPsfwFEkUjAsc7nb6UqVT5PQom/O0nmY74HT4pUqVJ\nGH//2Q==\n"
],
"text/plain": [
"<IPython.core.display.Image object>"
]
},
"metadata": {}
}

-------error from console----
[E 10:20:15.985 NotebookApp] Error while saving file: 07_Inception_Model.ipynb 'output_type'
Traceback (most recent call last):
File "/Users/george/venv3/lib/python3.6/site-packages/notebook/services/contents/filemanager.py", line 395, in save
self.check_and_sign(nb, path)
File "/Users/george/venv3/lib/python3.6/site-packages/notebook/services/contents/manager.py", line 430, in check_and_sign
if self.notary.check_cells(nb):
File "/Users/george/venv3/lib/python3.6/site-packages/nbformat/sign.py", line 519, in check_cells
if not self._check_cell(cell, nb.nbformat):
File "/Users/george/venv3/lib/python3.6/site-packages/nbformat/sign.py", line 496, in _check_cell
output_type = output['output_type']
KeyError: 'output_type'
[W 10:20:15.985 NotebookApp] Unexpected error while saving file: 07_Inception_Model.ipynb 'output_type'
[E 10:20:15.986 NotebookApp] {
"Host": "localhost:8888",
"Connection": "keep-alive",
"Content-Length": "1339391",
"Accept": "application/json, text/javascript, /; q=0.01",
"Origin": "http://localhost:8888",
"X-Requested-With": "XMLHttpRequest",
"X-Xsrftoken": "2|70a7ad28|30d00af3f4023e29207223ff62c8dec7|1490633765",
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.110 Safari/537.36",
"Content-Type": "application/json",
"Referer": "http://localhost:8888/notebooks/07_Inception_Model.ipynb",
"Accept-Encoding": "gzip, deflate, sdch, br",
"Accept-Language": "en-US,en;q=0.8",
"Cookie": "Pycharm-4c29cf31=f471dedd-8846-4455-897d-9c942b1d2dc2; Webstorm-1c198702=45d6dd50-61b4-4e0c-8b92-6f75374fb86a; _xsrf=2|70a7ad28|30d00af3f4023e29207223ff62c8dec7|1490633765; username-localhost-8888="2|1:0|10:1491321420|23:username-localhost-8888|44:ZDBkODVkYTk5NzFiNDU3OTk5MjU5MjI1MDRlNTJmOGI=|559c109a5e57333a232ffa931526b7f3169fc576a7bf738aa64ad5f1d61d00d9""
}
[E 10:20:15.986 NotebookApp] 500 PUT /api/contents/07_Inception_Model.ipynb (::1) 11.11ms referer=http://localhost:8888/notebooks/07_Inception_Model.ipynb

import inception ImportError: No module named request

When i try to import inception : i get this error.
WHen I tried to install requests , I find it here.
pip install requests
Requirement already satisfied (use --upgrade to upgrade): requests in ./anaconda/lib/python2.7/site-packages

ValueError: Cannot feed value of shape (64, 784) for Tensor 'x_data_2:0', which has shape '(?, 728)'

I tried to create the second tutorial, but I'm failing! this is the code for the whole tutorial

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
from tensorflow.examples.tutorials.mnist import input_data

#read data
data = input_data.read_data_sets('data/MNIST/', one_hot=True)

#useful info
image_size =28
img_size_flattened = 728
num_class = 10

#create place holders 
x_data = tf.placeholder(tf.float32,[None,img_size_flattened],name='x_data')

#since our convlayer accepts images in format of [#,imgheight,imgwidths,channels] we need to have a reshaped version as well
x_data_reshaped = tf.reshape(x_data,[-1,image_size,image_size,1],name='x_data_reshape')
y_label_vec = tf.placeholder(tf.float32,[None,num_class],name='y_label_vec')
y_label_true_indx = tf.arg_max(y_label_vec,dimension=1,name='y_label_true_indx')

#create helper methods for conv layer creations
#create methods for creating weights and biases 

def add_weight (shape):
    return tf.Variable(tf.truncated_normal(shape,stddev=0.05))
def add_bias(length):
    return tf.Variable(tf.ones(length))

#returns layer, and weights
def add_conv(input,num_channel,kernel_size,num_kernel,use_pool=True):
    
    shape = [kernel_size,kernel_size,num_channel,num_kernel]
    weights = add_weight(shape)
    biases = add_bias(num_kernel)
    layer = tf.nn.conv2d(input=input,filter=weights,strides=[1,1,1,1],padding='SAME')
    if use_pool:
        layer = tf.nn.max_pool(layer,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')
        
    layer = tf.nn.relu(layer)    
    return layer, weights

#create fc layer, but since we need flattened vectors  to connect to fc, lets create a flatten helper function

#returns layerflattended, and numberoffeatures(volumn length)
def flatten(layer):
    layer_shape = layer.get_shape()
    numberof_features = layer_shape[1:4].num_elements()
    layer_reshaped = tf.reshape(layer, [-1,numberof_features])
    return layer_reshaped,numberof_features
    
#now create helper function for fc 
def add_fc(flattenedlayer,num_input,num_output,Activation=None):
    weights = add_weight([num_input,num_output])
    biases = add_bias(num_output)
    output = tf.matmul(flattenedlayer,weights) + biases
    if Activation != None:
        output = Activation(output)
    return output    

#now lets create layers 
laye_conv1,w1 = add_conv(input=x_data_reshaped, num_channel=1, kernel_size=5, num_kernel=16)
layer_conv2,w2 = add_conv(input=laye_conv1, num_channel=16, kernel_size=5, num_kernel=36)

flat_Layer,num_features = flatten(layer_conv2)

layer_fc1 = add_fc(flattenedlayer=flat_Layer,num_input=num_features,num_output=128, Activation=tf.nn.relu)
layer_fc2 = add_fc(layer_fc1,num_input=128,num_output=10)

#calculate prediction 
pred = tf.nn.softmax(layer_fc2)
pred_labl_true_index = tf.argmax(pred, dimension=1)

#calculate accuracy
correctPredcition = tf.equal(pred_labl_true_index, y_label_true_indx)
accuracy = tf.reduce_mean(tf.cast(correctPredcition,tf.float32))

#crossendtropy
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(layer_fc2,y_label_vec)
loss = tf.reduce_mean(cross_entropy)
train = tf.train.GradientDescentOptimizer(0.2).minimize(loss)

sess = tf.Session()
sess.run(tf.initialize_all_variables())
data.test.true_lbl_indexes = [np.argmax(l) for l in data.test.labels]
for i in range(1000):
    x_dt, x_lbl = data.train.next_batch(100)
    dict = {x_data:x_dt, y_label_vec:x_lbl}
    sess.run(train, feed_dict=dict)
    if i%20 == 0:
        test_dic = {x_data:data.test.images,
                    y_label_vec:data.test.labels,
                    y_label_true_indx: data.test.true_lbl_indexes}
        acc = sess.run(accuracy,feed_dict=test_dic)
        print (acc) 

The whole code is based on the exact tutorial codes, but yet it fails! I checked it several times and I have no idea whats causing the error and why!

AttributeError: module 'download' has no attribute 'maybe_download_and_extract'

Hi Hvass-Labs, thank you for making these tutorials - they're a really fantastic online resource! I've run into an issue though - when I run the following code as instructed in the script cifar10.py:

maybe_download_and_extract()

I end up with the error:

AttributeError: module 'download' has no attribute 'maybe_download_and_extract'

Is there a way around this? I am currently using TensorFlow version 1.4.0 and have installed the module download with pip install.

CIFAR-10 data import problem

Hi
I was trying to import CIFAR-10 dataset in my machine but an error raised saying that no module named CIFAR-10. How am i suppose to import CIFAR-10 dataset in python. i am working in python 2.7 environment .

09_Video_Data Tutorials failed to calculate transfer-values

09_Video_Data Tutorials failed to calculate transfer-values

Tutorials failed to calculate transfer-values when I used more number of dataset. It was perfectly working when my dataset size was 3000+ images but after increasing my dataset size to 6000+ transfer_values_cache() throws an error at exactly when calculating transfer value for 5317th image in dataset.

Running on

Ubuntu 14.04 LTS
Jupyter 4.3.1
Python 2.7.13 |Anaconda custom (64-bit) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
Kernal Python 3.6
Tensorflow 1.0.1
Prettytensor 0.7.4

Error Message

Processing Inception transfer-values for training-images ...
- Processing image:   5317 / 6325

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1021     try:
-> 1022       return fn(*args)
   1023     except errors.OpError as e:

/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run_fn(session, feed_dict, fetch_list, target_list, options, run_metadata)
   1003                                  feed_dict, fetch_list, target_list,
-> 1004                                  status, run_metadata)
   1005 

/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/contextlib.py in __exit__(self, type, value, traceback)
     88             try:
---> 89                 next(self.gen)
     90             except StopIteration:

/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py in raise_exception_on_not_ok_status()
    465           compat.as_text(pywrap_tensorflow.TF_Message(status)),
--> 466           pywrap_tensorflow.TF_GetCode(status))
    467   finally:

InvalidArgumentError: Invalid JPEG data, size 1310720
	 [[Node: DecodeJpeg = DecodeJpeg[acceptable_fraction=1, channels=3, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_DecodeJpeg/contents_0)]]

During handling of the above exception, another exception occurred:

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-165-472ae91bad08> in <module>()
      5 transfer_values_train = transfer_values_cache(cache_path=file_path_cache_train,
      6                                               image_paths=image_paths_train,
----> 7                                               model=model)

/home/alwynpcs16/tutorial/inception.py in transfer_values_cache(cache_path, model, images, image_paths)
    592 
    593     # Read the transfer-values from a cache-file, or calculate them if the file does not exist.
--> 594     transfer_values = cache(cache_path=cache_path, fn=fn)
    595 
    596     return transfer_values

/home/alwynpcs16/tutorial/cache.py in cache(cache_path, fn, *args, **kwargs)
     63 
     64         # Call the function / class-init with the supplied arguments.
---> 65         obj = fn(*args, **kwargs)
     66 
     67         # Save the data to a cache-file.

/home/alwynpcs16/tutorial/inception.py in fn()
    589     # and fn=model.transfer_values to the cache()-function.
    590     def fn():
--> 591         return process_images(fn=model.transfer_values, images=images, image_paths=image_paths)
    592 
    593     # Read the transfer-values from a cache-file, or calculate them if the file does not exist.

/home/alwynpcs16/tutorial/inception.py in process_images(fn, images, image_paths)
    543             result[i] = fn(image=images[i])
    544         else:
--> 545             result[i] = fn(image_path=image_paths[i])
    546 
    547     # Print newline.

/home/alwynpcs16/tutorial/inception.py in transfer_values(self, image_path, image)
    487         # This calculates the values for the last layer of the Inception model
    488         # prior to the softmax-classification, which we call transfer-values.
--> 489         transfer_values = self.session.run(self.transfer_layer, feed_dict=feed_dict)
    490 
    491         # Reduce to a 1-dim array.

/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/client/session.py in run(self, fetches, feed_dict, options, run_metadata)
    765     try:
    766       result = self._run(None, fetches, feed_dict, options_ptr,
--> 767                          run_metadata_ptr)
    768       if run_metadata:
    769         proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
    963     if final_fetches or final_targets:
    964       results = self._do_run(handle, final_targets, final_fetches,
--> 965                              feed_dict_string, options, run_metadata)
    966     else:
    967       results = []

/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
   1013     if handle is None:
   1014       return self._do_call(_run_fn, self._session, feed_dict, fetch_list,
-> 1015                            target_list, options, run_metadata)
   1016     else:
   1017       return self._do_call(_prun_fn, self._session, handle, feed_dict,

/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
   1033         except KeyError:
   1034           pass
-> 1035       raise type(e)(node_def, op, message)
   1036 
   1037   def _extend_graph(self):

InvalidArgumentError: Invalid JPEG data, size 1310720
	 [[Node: DecodeJpeg = DecodeJpeg[acceptable_fraction=1, channels=3, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_DecodeJpeg/contents_0)]]

Caused by op 'DecodeJpeg', defined at:
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/ipykernel/__main__.py", line 3, in <module>
    app.launch_new_instance()
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/traitlets/config/application.py", line 658, in launch_instance
    app.start()
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/ipykernel/kernelapp.py", line 474, in start
    ioloop.IOLoop.instance().start()
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/zmq/eventloop/ioloop.py", line 177, in start
    super(ZMQIOLoop, self).start()
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tornado/ioloop.py", line 887, in start
    handler_func(fd_obj, events)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tornado/stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
    self._handle_recv()
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
    self._run_callback(callback, msg)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
    callback(*args, **kwargs)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tornado/stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 276, in dispatcher
    return self.dispatch_shell(stream, msg)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 228, in dispatch_shell
    handler(stream, idents, msg)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 390, in execute_request
    user_expressions, allow_stdin)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 196, in do_execute
    res = shell.run_cell(code, store_history=store_history, silent=silent)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/ipykernel/zmqshell.py", line 501, in run_cell
    return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2717, in run_cell
    interactivity=interactivity, compiler=compiler, result=result)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2821, in run_ast_nodes
    if self.run_code(code, result):
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2881, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-162-45804e0be4f2>", line 1, in <module>
    model = inception.Inception()
  File "/home/alwynpcs16/tutorial/inception.py", line 285, in __init__
    tf.import_graph_def(graph_def, name='')
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 288, in import_graph_def
    op_def=op_def)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2327, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/alwynpcs16/anaconda2/envs/py3k/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1226, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Invalid JPEG data, size 1310720
	 [[Node: DecodeJpeg = DecodeJpeg[acceptable_fraction=1, channels=3, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_DecodeJpeg/contents_0)]]

Add your youtube channel to the readme file as well

Hi and Thanks alot for the nice tutorial you created.
It would be a good idea to also include your youtube channel where you actually teach these tutorials.
Keep up the good job
Best Regards

Failed to load checkpoints for Breakout

When I try to load the checkpoints in Breakout, I get this error:
Assign requires shapes of both tensors to match. lhs shape= [1024,4] rhs shape= [1024,6]

and also alot of key errors:

how to make convolution neural network for binary features?

I have (ARFF) file or excel file with binary features (1,0) I want to make CNN for that file.
I read some resources which wrote about using CNN with image classification but I don't know how can I use it it my classifier?
Thanks

Can not run inception.py on Ubuntu 14.04 with Python 2.7

I get "ImportError: No module named download". See full trace below. I tried "pip install download" and pip does not seem to have a module "download". How do I install it?

$ python inception.py
Traceback (most recent call last):
File "inception.py", line 59, in
import download
ImportError: No module named download

I tried the rest of the code and also get "ImportError: cannot import name cache". See below.
[...]
Traceback (most recent call last):
File "inception.py", line 60, in
from cache import cache
ImportError: cannot import name cache

Can you please point out workaround? Here is my Python version:
$ python
Python 2.7.6 (default, Oct 26 2016, 20:30:19)
[GCC 4.8.4] on linux2

Thanks!

TFRecords with variable length data

Hey Hvass,
I don't know if this is the most proper channel to ask you a question,
but since I'm here I can try it anyway.

In the past 2 weeks, I'm working with tf-records and the tf-estimator & i can definitively feel your pain.
But I've a question which has been bothering me for a few days.

How do you use tf-records whereby you don't know which kind of shape they have?
Okey, creating a tf-record might be a mess, and you can do it in several ways. But as soon as you've a tf-record, it doesn't know which kind of shape it has. And that is a bit problematic when you create your own NN. because the first layer requires somehow a shape.

first_hidden_layer = tf.layers.dense(features['x'], 10, activation=tf.nn.relu)

Tensor("IteratorGetNext:0", shape=(?, ?), dtype=float64, device=/device:CPU:0)
-
ValueError: The last dimension of the inputs to `Dense` should be defined. Found `None`.

And that is a bit ... i think.
Because, i can imagine that you sometimes would like to use variable length arrays (like when you work with text) .
And I can also imagine that, if i create a tf-record now, then i would like to use that same record also in the future when I've forgotten the shape of that record.

Basically, I, as a person don't want to remember the shape of a tf-record. or write it down somewhere ... And this for FixedLenght as well as for the Variable Length tf-records.

Thus the question which I've is, do you've managed to use the tf-records and tf-estimator, together whereby you (act like) you don't know the shape of the tf-record? (with shape --> i mean, amount of features in a record (or pixels, or ...))

because in every single example these values are hard coded,
(even you, use img_size)

therefore, can you help me?

• I hope that my question is clear -
  -- if you want, i can provide you a notebook --

pretty tensor version?

I'm noting a prettytensor error with 0.7.1. (posted on their issue list), I'm wondering what version of prettytensor you are using? Thank you for the youtube tutorial series and github site.

version = '0.7.1'

import prettytensor as pt

File "lib\prettytensor_init_.py", line 25, in
from prettytensor import funcs
File "lib\prettytensor\funcs.py", line 25, in
from prettytensor.pretty_tensor_image_methods import *
File "lib\prettytensor\pretty_tensor_image_methods.py", line 135, in
class conv2d(prettytensor.VarStoreMethod):
File "lib\prettytensor\pretty_tensor_image_methods.py", line 145, in conv2d
bias=tf.zeros_initializer(),
TypeError: zeros_initializer() missing 1 required positional argument: 'shape'