morvanzhou / tensorflow-tutorial

Tensorflow tutorial from basic to hard, 莫烦Python 中文AI教学

If you'd like to use PyTorch, no worries, I made a new PyTorch Tutorial just like Tensorflow. Here is the link: https://github.com/MorvanZhou/PyTorch-Tutorial

Tensorflow 2017 Tutorials

Tensorflow 2+ has been released, here is my quick TF2+ tutorial codes

In these tutorials, we will build our first Neural Network and try to build some advanced Neural Network architectures developed recent years.

All methods mentioned below have their video and text tutorial in Chinese. Visit 莫烦 Python for more.

• Tensorflow basic
  • Session
  • Placeholder
  • Variable
  • Activation
• Build your first network
  • Regression
  • Classification
  • Save and reload
  • Optimizers
  • Tensorboard
  • Dataset
• Advanced neural network
  • CNN
  • RNN-Classification
  • RNN-Regression
  • AutoEncoder
  • DQN Reinforcement Learning
  • GAN (Generative Adversarial Nets) / Conditional GAN
  • Transfer Learning
• Others (WIP)
  • Dropout
  • Batch Normalization
  • Visualize Gradient Descent
  • Distributed training

Regression

Classification

CNN

RNN

Autoencoder

GAN (Generative Adversarial Nets)

Dropout

Batch Normalization

Visualize Gradient Descent

Donation

If this does help you, please consider donating to support me for better tutorials! Any contribution is greatly appreciated!

tensorflow-tutorial's Issues

frequency and shortcut to update tensorflow from source?

Hi Morvan,

Basically, to update tensorflow after installed from source, is to re-install from source, there is no shortcut, right?

Also, given it takes a while to install from source, how often it is recommended to re-install from source?

Thanks!

InvalidArgumentError about 305_tensorboard.py

InvalidArgumentError Traceback (most recent call last)
D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\client\session.py in _do_call(self, fn, *args)
1291 try:
-> 1292 return fn(*args)
1293 except errors.OpError as e:

D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\client\session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
1276 return self._call_tf_sessionrun(
-> 1277 options, feed_dict, fetch_list, target_list, run_metadata)
1278

D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\client\session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata)
1366 self._session, options, feed_dict, fetch_list, target_list,
-> 1367 run_metadata)
1368

InvalidArgumentError: You must feed a value for placeholder tensor 'input/y_input' with dtype float and shape [?,1]
[[{{node input/y_input}} = Placeholderdtype=DT_FLOAT, shape=[?,1], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

During handling of the above exception, another exception occurred:

InvalidArgumentError Traceback (most recent call last)
in ()
41 for step in range(100):
42 # train and net output
---> 43 _, result = sess.run([train_op, merge_op], {tf_x: x, tf_y: y})
44 writer.add_summary(result, step)
45

D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\client\session.py in run(self, fetches, feed_dict, options, run_metadata)
885 try:
886 result = self._run(None, fetches, feed_dict, options_ptr,
--> 887 run_metadata_ptr)
888 if run_metadata:
889 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\client\session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
1108 if final_fetches or final_targets or (handle and feed_dict_tensor):
1109 results = self._do_run(handle, final_targets, final_fetches,
-> 1110 feed_dict_tensor, options, run_metadata)
1111 else:
1112 results = []

D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\client\session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1284 if handle is None:
1285 return self._do_call(_run_fn, feeds, fetches, targets, options,
-> 1286 run_metadata)
1287 else:
1288 return self._do_call(_prun_fn, handle, feeds, fetches)

D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\client\session.py in _do_call(self, fn, *args)
1306 self._config.experimental.client_handles_error_formatting):
1307 message = error_interpolation.interpolate(message, self._graph)
-> 1308 raise type(e)(node_def, op, message)
1309
1310 def _extend_graph(self):

InvalidArgumentError: You must feed a value for placeholder tensor 'input/y_input' with dtype float and shape [?,1]
[[{{node input/y_input}} = Placeholderdtype=DT_FLOAT, shape=[?,1], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

Caused by op 'input/y_input', defined at:
File "D:\ProgramData\envs\tensorflow1\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "D:\ProgramData\envs\tensorflow1\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\ipykernel_main.py", line 3, in
app.launch_new_instance()
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\traitlets\config\application.py", line 658, in launch_instance
app.start()
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\ipykernel\kernelapp.py", line 477, in start
ioloop.IOLoop.instance().start()
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\zmq\eventloop\ioloop.py", line 177, in start
super(ZMQIOLoop, self).start()
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\tornado\ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\zmq\eventloop\zmqstream.py", line 440, in _handle_events
self._handle_recv()
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\zmq\eventloop\zmqstream.py", line 472, in _handle_recv
self._run_callback(callback, msg)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\zmq\eventloop\zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\tornado\stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\ipykernel\kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\ipykernel\kernelbase.py", line 235, in dispatch_shell
handler(stream, idents, msg)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\ipykernel\kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\ipykernel\ipkernel.py", line 196, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\ipykernel\zmqshell.py", line 533, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\IPython\core\interactiveshell.py", line 2698, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\IPython\core\interactiveshell.py", line 2802, in run_ast_nodes
if self.run_code(code, result):
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\IPython\core\interactiveshell.py", line 2862, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 30, in
ys=tf.placeholder(tf.float32,[None,1],name = 'y_input')
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\ops\array_ops.py", line 1745, in placeholder
return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 6038, in placeholder
"Placeholder", dtype=dtype, shape=shape, name=name)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\ops.py", line 3272, in create_op
op_def=op_def)
File "D:\ProgramData\envs\tensorflow1\lib\site-packages\tensorflow\python\framework\ops.py", line 1768, in init
self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input/y_input' with dtype float and shape [?,1]
[[{{node input/y_input}} = Placeholderdtype=DT_FLOAT, shape=[?,1], _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

initialize variable under variable scope?

The correct one:

    with tf.variable_scope(scope) as scope:
        w = tf.get_variable("weights", [x.shape[1], out_dim], initializer=tf.random_normal_initializer())
        b = tf.get_variable("biases", [out_dim], initializer=tf.constant_initializer(0.0))

The following code will raise the error: TypeError: Failed to convert object of type <class 'list'> to Tensor. Contents: [Dimension(100), 50]. Consider casting elements to a supported type.

    with tf.variable_scope(scope) as scope:
        w = tf.get_variable(initializer=tf.random_normal([x.shape[1], out_dim]), name='weights')
        b = tf.get_variable(initializer=tf.zeros([out_dim]), name='biases')

What's happened?

Add the version description at 'tutorial-contents/306_dataset.py'

Tensorflow-Tutorial/tutorial-contents/306_dataset.py line:9

In tensorFlow 1.3, get Dataset like 'from tensorflow.contrib.data import Dataset'.
But in 1.2 or other version, we can't get Dataset like that.
Please add the description, the newbie will be grateful.

load iamge

利用transfer learning的代码处理一批图片， 发现调用了一张图片，最后resize出现了2242244的格式，导致输入神经网络的数据格式报错(VGG 要2242243的size)
新手勿喷，想问这是什么原因？

403_RNN_regression ups bug!!

Dear,
When run the script,
Error ups,

WARNING:tensorflow:From D:\python\LogisticRegression-master\403_RNN_regression.py:32: BasicRNNCell.init (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is equivalent as tf.keras.layers.SimpleRNNCell, and will be replaced by that in Tensorflow 2.0.

How to solve the question?
Could U help me ,please ?
Thx

How to use CNN to get the parameters of the quadratic equation with one unknown?

Dear,
There is a question of the script,
Could it be a little hard,if the function is y=ax^2 +bx +c ?
Any idea to change the code?
Thx

transfer learning

您好，这个教程中有l两个不明白的地方。
1.如何控制卷积层的参数是固定不变，不在训练时候被改变的。。
CNN's filter is constant, NOT Variable that can be trained

2.如果需要微调卷积层的参数应该怎么改?

404_AutoEncoder.py mnist.train.labels[1] np.argmax

why the title is always 0?
for tf v 1.14,
404_AutoEncoder.py
line 33 plt.title('%i' % np.argmax(mnist.train.labels[0]))
np.argmax is redundant.

a problem with tensorboard

I can't open the link after running tensorboard --logdir logs but localhost:6006 is ok. Do you know why? Is there any difference between this two links?

how to assign values to parameters?

import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np

tf.set_random_seed(1)
np.random.seed(1)


n_data = np.ones((100, 2))

x0 = np.random.normal(2*n_data, 1)      # class0 x shape=(100, 2)

By referring to the documentation, we can know the defination:
def normal(loc=0.0, scale=1.0, size=None):

but how to understand"np.random.normal(2*n_data, 1)",please

About the training of autoencoder

Does the stacked autoencoder need pre-training or fine-tune?I see your code that your AE does not have pre-training or fine-tune.Did the process be executed in some function that TF has?

what is the tf.set_random_seed(1) ？

i want to know if its function is random initialization?

406Gan

D_loss = -tf.reduce_mean(tf.log(prob_artist0) + tf.log(1-prob_artist1))
G_loss = tf.reduce_mean(tf.log(1-prob_artist1))
这个损失函数能将下是什么含义吗

README里面这些个漂亮的gif是怎么产生的？tb？

有没有介绍

what is the difference between tf.contrib.layers an tf.layers ?

such as tf.contrib.layers.batch_norm / tf.layers.batch_normalization,...

402,403 Can Not Run

ok,还是用中文描述吧。应该是tensorflow版本更新的问题，在tensorflow1.1.0下这两个例子不能运行，当我降到tensorflow1.0.0时就可以了。
报错的内容是:ValueError: Attempt to have a second RNNCell use the weights of a variable scope that already has weights: 'Model/RNN/multi_rnn_cell/cell_0/basic_lstm_cell'; and the cell was not constructed as BasicLSTMCell(..., reuse=True). To share the weights of an RNNCell, simply reuse it in your second calculation, or create a new one with the argument reuse=True.

代码应该是正确的，只是版本的问题。但是按照网上查到的解决办法都不能正常运行。sad。
不知道莫凡老师知道在1.1.0版本下如何修改这个例子能够正常运行吗?谢谢了。

303_save_reload.py can not run.

saver.save(sess, 'params', write_meta_graph=False) # meta_graph is not recommended
'params' must be modified to 'my_net/params'.

another way to compute accuray on 402_RNN_classification.py

I get high accuracy score on this way :
isCorrect = tf.equal(tf.argmax(predict_Y, 1), tf.argmax(Y_holder, 1))
accuracy = tf.reduce_mean(tf.cast(isCorrect, tf.float32))
your code:
accuracy = tf.metrics.accuracy( # return (acc, update_op), and create 2 local variables
labels=tf.argmax(tf_y, axis=1), predictions=tf.argmax(output, axis=1),)[1]
I think 2 way is same, but get different result, can you tell me the reason?

img/255

transfer_learning中load image为什么需要img/255呢?

Batch Normalization combined with your DDPG implementation?

Hi Morvan,

I am trying to implement your Batch Normalization tutorial on your DDPG algorithme tutorial, but i have a hard time understanding the bits?

one of my problems is:

`        self.a_loss = - tf.reduce_mean(q)  # maximize the q
        self.atrain = tf.train.AdamOptimizer(LR_A).minimize(self.a_loss, var_list=a_params)

        update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)		
        with tf.control_dependencies(update_ops):		
			with tf.control_dependencies(target_update):    # soft replacement happened at here
				self.q_target = self.R + ((GAMMA * (1- self.Done)) * (q_ * (1 - self.Done)))
				self.td_error = tf.losses.mean_squared_error(labels=self.q_target, predictions=q)
				self.ctrain = tf.train.AdamOptimizer(LR_C).minimize(self.td_error, var_list=c_params) `

Since you said you need to have that update_ops i imagned that it should look something like this, but this then won't include the atrain, if not this being incorrect of course?

furthermore if you could give some signs of directions on how to implement it on your ddpg implementation that would be nice,

Jan

怎么回事

代码和视频根本不配套

想问下你readme中的动图怎么画的

请问一下您readme中动图是怎么画的呢