creatcodebuild / tensorflow-and-deeplearning-tutorial

A TensorFlow & Deep Learning online course I taught in 2016

TensorFlow and DeepLearning Tutorial

This is a collection of my deep learning tutorials.

这是我的深度学习教程集合。

Season 1 is introduction to TensorFlow, fully connected neural networks and convolutional neural networks.

Season 2 is about Natural Language Processing, sort of.

tensorflow-and-deeplearning-tutorial's Issues

请问关于tensorflow程序移植

刚刚接触tf，做了个网络训练了结果，接下来有个问题，如何把teensorflow程序移植到c，或者具体该如何应用起来？

error in run execution

If I use

net.run(train_samples, train_labels, test_samples, test_labels, train_data_iterator=train_data_iterator,
             iteration_steps=10000, test_data_iterator=test_data_iterator)

I will get error below but I can't understand reason. Can you explain?

Thanks

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'test/single_input' with dtype float and shape [1,32,32,1]
	 [[Node: test/single_input = Placeholder[dtype=DT_FLOAT, shape=[1,32,32,1], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

plt.imgshow()似乎只接受整数数组?

我是python3.6
matplotlib 2.2.3

只有这样才能显示

plt.imshow(dataset[i].astype(np.int32))

plt.show()

周末问答很好啊

有时间讲讲不均衡数据的学习方法吧 👍

全连接网络

全连接网络中的损失函数为什么是用tf.matmul(hidden, fc2_weight) + fc2_biases， 和 label比较呢？而不是用softmax， argmax后的比较呢？

thanks for your project

I learned your project from the bilibili，and study to use tensorflow by it. when i learned the lesson 9,there a problem I can not slove happened.
this is my code
`
import tensorflow as tf
from sklearn.metrics import confusion_matrix
import numpy as np
import load

def get_chunk(samples, labels, chunkSize):
if len(samples) != len(labels):
raise Exception('dataset is wrong!!!!!')

print(np.shape(labels))
stepStart = 0
i = 0
while stepStart < len(samples):
    stepEnd = stepStart + chunkSize
    if stepEnd < len(samples):
        print('==================this is the shape of placeholder======================')
        print(np.shape(labels[stepStart: stepEnd]))
        print('------------------------------------------------------------------------')
        yield i, samples[stepStart:stepEnd], labels[stepStart: stepStart]
        #print('========================================================================')
        i += 1
    stepStart = stepEnd

class Network():
def init(self, num_hidden, batch_size):
self.batch_size = batch_size
self.test_batch_size = 500
self.num_hidden = num_hidden

    self.image_size = load.image_size
    self.num_channels = 1

    self.graph = tf.Graph()
    self.tf_train_samples = None
    self.tf_train_labels = None
    self.tf_test_samples = None
    self.tf_test_labels = None

def define_graph(self):
    with self.graph.as_default():
        self.tf_train_samples = tf.placeholder( tf.float32, shape=(self.batch_size, image_size, image_size, num_channels)  )
        #self.tf_train_samples = tf.placeholder( tf.float32, shape=(self.batch_size, image_size, image_size, num_channels) )
        self.tf_train_labels = tf.placeholder( tf.float32, shape=(self.batch_size, num_labels)  )
        self.tf_test_samples = tf.placeholder( tf.float32, shape=(self.test_batch_size, image_size, image_size, num_channels)  )

        fc1_weights = tf.Variable( tf.truncated_normal([image_size*image_size, self.num_hidden], stddev=0.1)  )
        fc1_biases = tf.Variable( tf.constant(0.1, shape=[self.num_hidden])  )

        fc2_weights = tf.Variable( tf.truncated_normal([self.num_hidden, num_labels], stddev=0.1)  )
        fc2_biases = tf.Variable( tf.constant(0.1, shape=[num_labels])  )

        def model(data):
            shape = data.get_shape().as_list()
            print('this is model() ============================================')
            print(data.get_shape(), shape)
            reshape = tf.reshape(data, [shape[0], shape[1]*shape[2]*shape[3]])
            print(reshape.get_shape(), fc1_weights.get_shape(), fc1_biases.get_shape())
            print('model ended ================================================')
            hidden = tf.nn.relu( tf.matmul(reshape, fc1_weights) + fc1_biases  )

            return tf.matmul(hidden, fc2_weights) + fc2_biases

        logits = model(self.tf_train_samples)
        self.loss = tf.reduce_mean( tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=self.tf_train_labels) )
        self.optimizer = tf.train.GradientDescentOptimizer(0.0001).minimize(self.loss)

        self.train_prediction = tf.nn.softmax(logits)
        self.test_prediction = tf.nn.softmax( model(self.tf_test_samples)  )

def run(self):
    self.session = tf.Session(graph=self.graph)
    with self.session as session:
        tf.initialize_all_variables().run()
        #tf.global_variables_initializer().run()

        print('start Training')
        for i, samples, labels in get_chunk( train_samples, train_labels, chunkSize=self.batch_size  ):
            _, l, predictions = session.run(
                    [self.optimizer, self.loss, self.train_prediction],
                    feed_dict={ self.tf_train_samples:samples, self.tf_train_labels:labels}    
            )
            accuracy, _ = self.accuracy(predictions, labels)
            if i % 50 == 0:
                print('accuracy:'+str(accuracy))
       
def accuracy(self, predictions, labels, need_confusion_matrix = False):
    #pass
    _predictions = np.argmax(predictions, 1)
    _labels = np.argmax(labels, 1)
    cm = confusion_matrix(_labels, _predictions) if need_confusion_matrix else None

    accuracy = (100.0*np.sum(_predictions == labels) / print.shape[0])
    return accuracy, cm

if name == 'main':
train_samples, train_labels = load._train_samples, load._train_labels
test_samples, test_labels = load._test_samples, load._test_labels

print('Training set', train_samples.shape, train_labels.shape)
print('    Test set', test_samples.shape, test_labels.shape)

image_size = load.image_size
num_labels = load.num_labels
num_channels = 1#load.channels
net = Network(num_hidden=128, batch_size=100)
net.define_graph()
net.run()

when i run it the error created asTraceback (most recent call last):
File "bp.py", line 109, in
net.run()
File "bp.py", line 82, in run
feed_dict={ self.tf_train_samples:samples, self.tf_train_labels:labels}
File "/home/panda/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 778, in run
run_metadata_ptr)
File "/home/panda/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 961, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (0, 10) for Tensor 'Placeholder_1:0', which has shape '(100, 10)'
`
i can solve it, can you help me to find where is the problem.

运行20节 出现错误

Traceback (most recent call last):

File "C:\Program Files\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2898, in run_code
self.showtraceback()

File "C:\Program Files\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 1826, in showtraceback
self._showtraceback(etype, value, stb)

File "C:\Program Files\Anaconda3\lib\site-packages\ipykernel\zmqshell.py", line 522, in _showtraceback
dh.parent_header, ident=topic)

File "C:\Program Files\Anaconda3\lib\site-packages\jupyter_client\session.py", line 673, in send
to_send = self.serialize(msg, ident)

File "C:\Program Files\Anaconda3\lib\site-packages\jupyter_client\session.py", line 576, in serialize
content = self.pack(content)

File "C:\Program Files\Anaconda3\lib\site-packages\jupyter_client\session.py", line 95, in
ensure_ascii=False, allow_nan=False,

File "C:\Program Files\Anaconda3\lib\site-packages\zmq\utils\jsonapi.py", line 43, in dumps
s = s.encode('utf8')

UnicodeEncodeError: 'utf-8' codec can't encode character '\udcd5' in position 215: surrogates not allowed

卷积神经网络可视化

每层卷积之后显示图片的可视化怎么做？只显示部分就行。

视频20链接有误

视频20链接到第19了

该从何入手呢？

该从何入手呢？需要一些什么只是储备呢？ Thx

关于 tf.Variable 被训练的问题

up主你好！我是从 bilibili过来的！讲的非常好~~
但是 tensorflow有一个问题一直让我很困惑
line 132-133 训练了 fc1_weights/bias, fc2_weights/bias, 这些都是 tf.Variables。为什么 line 147 测试的时候， fc1_weights/bias, fc2_weights/bias 用的是训练后的值呢？tensorflow 对于 Variable 的训练的效果在什么 scope内是有效的？不知道我的问题表达清楚了没有。

创建图表问题

		# Graph Related
		self.graph = tf.Graph()
		self.tf_train_samples = None
		self.tf_train_labels = None
		self.tf_test_samples = None
		self.tf_test_labels = None
		self.tf_test_prediction = None

		# 统计
		self.merged = None

		# 初始化
		self.define_graph()
		self.session = tf.Session(graph=self.graph)
		self.writer = tf.train.SummaryWriter('./board', self.graph)

报以下的错
Traceback (most recent call last):
File "/home/xjk/PycharmProjects/HelloTF/dp2.py", line 202, in
net = Network(num_hidden=128, batch_size=100)
File "/home/xjk/PycharmProjects/HelloTF/dp2.py", line 67, in init
self.writer = tf.train.SummaryWriter('./board', self.graph)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/summary_io.py", line 82, in init
self.add_graph(graph_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/summary_io.py", line 128, in add_graph
event = event_pb2.Event(wall_time=time.time(), graph_def=graph_def)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 519, in init
_ReraiseTypeErrorWithFieldName(message_descriptor.name, field_name)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 450, in _ReraiseTypeErrorWithFieldName
six.reraise(type(exc), exc, sys.exc_info()[2])
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 517, in init
copy.MergeFrom(new_val)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 1208, in MergeFrom
"expected %s got %s." % (cls.name, type(msg).name))
TypeError: Parameter to MergeFrom() must be instance of same class: expected GraphDef got Graph. for field Event.graph_def

Process finished with exit code 1

請問是否能實地測試

請問能不能給我們展示一下
將一張待辨識的圖片
導入我們最後訓練完成的資料內 作判別
並輸出tensorflow所預測的值
學習完全不 卻不太懂怎麼直接導入圖片!

关于实现label embedding

你好，我想问一下我打算做label embedding是应该先用语料库先训练出word embedding，然后提取与label的相关的word embedding组成label embedding；还是在label input后面加一层神经元个数小于label数量的隐藏层就能通过训练得到label embedding啊？

求tensorboard相关教程

tensorflow里面的数据可视化功能很强大，但用起来总是很困惑。不知道什么时候该写namescope，也不知道什么时候该写summary，用什么类型的summary。请问能否推出一期相关教程？

运行20报错，求解

我运行环境是Windows8.1、python3.5、tensorflow0.12.1（gpu版）。

运行16-19的代码是oK的，但是运行20提示有错，貌似是由saver引起的，错误提示如下：

Caused by op 'save/RestoreV2_16', defined at:
File "main.py", line 68, in
net.define_model()
File "C:\Users\shang\Desktop\TensorFlow-Chinese-Tutorial-master\20\dp.py", lin
e 205, in define_model
self.saver = tf.train.Saver(tf.all_variables())
File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\training
saver.py", line 1000, in init
self.build()
File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\training
saver.py", line 1030, in build
restore_sequentially=self._restore_sequentially)
File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\training
saver.py", line 624, in build
restore_sequentially, reshape)
File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\training
saver.py", line 361, in _AddRestoreOps
tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\training
saver.py", line 200, in restore_op
[spec.tensor.dtype])[0])
File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_i
o_ops.py", line 441, in restore_v2
dtypes=dtypes, name=name)
File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\framework
\op_def_library.py", line 759, in apply_op
op_def=op_def)
File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\framework
\ops.py", line 2240, in create_op
original_op=self._default_original_op, op_def=op_def)
File "C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\framework
\ops.py", line 1128, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Unsuccessful TensorSliceReader c
onstructor: Failed to get matching files on model/default.ckpt: Not found: FindF
irstFile failed for: model : \u03f5\u0373\udcd5\u04b2\udcbb\udcb5\udcbd\u05b8\ud
cb6\udca8\udcb5\udcc4·\udcbe\udcb6\udca1\udca3

     [[Node: save/RestoreV2_16 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:

localhost/replica:0/task:0/cpu:0"](recv_save/Const_0, save/RestoreV2_16/tensor
names, save/RestoreV2_16/shape_and_slices)]]
[[Node: save/RestoreV2_7/_61 = _Recvclient_terminated=false, recv_devi
ce="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:
0/task:0/cpu:0", send_device_incarnation=1, tensor_name="edge_254_save/RestoreV2
_7", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]]

GPU显存问题

你好！我也是从 bilibili过来的(为什么要说‘也’？( •̀ ω •́ )y)！灰常感谢你的视频讲解！
现在有个问题是，我也是在用笔记本电脑学习tensorflow，2G显存，so。。。也是经常遇到显存不够的问题，有什么类似swap一样的东西可以增加显存的吗？

TensorFlow compatibility problems

Hi @twiceyuan @whyscience @CreatCodeBuild ,

I observed some TensorFlow api usage in you code(Season2/1 Word2Vec/word2vec_tf.py) that might lead to serious compatibility problems(like crash).

For tf.global_variables_initializer appears in tf v0.12.0-rc0 and this will induced a crash if used in earlier version.

These problems can be fixed in pr #37

I judged the version and made some patches to make it compatible on more versions.
Look forward to your reply!
Thanks!

关于如何卸载老版本的TF

关于TF的卸载问题。因为在版本为0.8的时候装过一次TF，并且成功运行了，之后有事情耽搁了，现在运行0.8不知道为什么运行不起来 ，官方0.11出了，想卸了0.8，装0.11的，请问大神，应该怎么卸，官方文档没找到，或者有是我眼瞎没看到，大神请指导

Hi 有个跟哔哩哔哩教程无关的数据处理问题

Hi,哲的王

我目前在做Android实时识别这块的APP，目前拿MNIST练手，发现理想环境下使用test data去评估模型，有99%的准确率，然后放在Android上实时识别或者拍照之后再识别，准确度不高，所以我想在train之前做data agumentation，然后发现

• tf.image.random_flip_left_right
• tf.image.random_brightness
• tf.image.random_contrast
• tf.image.per_image_standardization
  这些方法都是接受3维矩阵，MNIST读出来的是4维的，这个我该要怎么处理。

Best Regards
Jcme Ls

導入圖片實地測試

請問能不能給我們展示一下
將一張待辨識的圖片
導入我們最後訓練完成的資料內 作判別
並輸出tensorflow所預測的值
學習完全不 卻不太懂怎麼直接導入圖片!

不知道怎么预测

刚学tensorflow，看到代码生成了一个模型，但最开始不是要先找一张有数据的图片吗，把图片输入模型，识别数字

linux集成开发环境

请问linux下学习tensorflow用啥IDE比较好呢？请推荐一个?谢谢～视频讲的很好奥．
[email protected]

Can we parameterize filter_map for image_summary?

In 12-15, filter map reshape is hard-coded by (self.conv1_depth, 32, 32, 1). Why these values 32,32,1? Can we make them variables?

tensorFolw Win10安装出问题

看到第七节时，发现自己在windows10下还没安装
在win10控制台下输入pip命令安装时，
pip install --upgrade https://storage.googleapis.com/tensorflow/windows/gpu/tensorflow_gpu-0.12.0-cp35-cp35m-win_amd64.whl
弹出了下面的错误，说是这个文件类型不被支持，结果就悲剧了
tensorflow_gpu-0.12.0-cp35-cp35m-win_amd64.whl is not a supported wheel on this platform.

查了好一会了，还没有解决方法，请问您知道怎么解决么？
系统是win10x64，anaconda2，anaconda3都安装了，并且将64bit anaconda3设为默认，python3.5.4

B站追过来的

向UP主学习23333

one_hot_label

one_hot_label 的时候可以用zeros函数

L1 & L2 Quora 答案超链接

16-19/README.md里面 Quora 答案的超链接写成论文的了
是不是应该替换成这个: https://www.quora.com/What-is-the-difference-between-L1-and-L2-regularization

Please attatch the video link

Hi, I have watched your video tutorial for times, I really learn a lots. Does this section video tutorial has accomplished? For 'Bilibili' returns only the session one when i search 'TF-Grirls'. Can you please attach your video link? that's will be really thankful.

tensorboard

进入tensorboad后，用的chrom,
网址一栏显示出了tensorboard,但是页面是空白页面，什么都没有显示，没有报错
tensorboard --logdir=board启动正常，
请问这是什么问题啊？

优达学城深度学习中文教程

那个链接点进去之后，注册登录完成之后，有个隐私协议，点击同意没反应，一直卡在那个界面，up主还有没有别的深度学习视频推荐？