《深度学习：一起玩转TensorLayer》资源分享、讨论
欢迎补充，感谢！我们也希望更多新的作者加入，请加入微信群讨论
和官方例子(https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_generate_text.py) 相比, 少了一部分// sequence_length。
这里的x好像应该有j下标吧？
书中给出类似 https://tensorlayer.readthedocs.io/en/latest/modules/iterate.html?highlight=ptb_iterator#tensorlayer.iterate.ptb_iterator 的例子。
当batch_size=2, num_steps=3时，结果是:
print(x, y)
... [[ 0 1 2] <---x 1st subset/ iteration
... [10 11 12]]
... [[ 1 2 3] <---y
... [11 12 13]]
...
... [[ 3 4 5] <--- 1st batch input 2nd subset/ iteration
... [13 14 15]] <--- 2nd batch input
... [[ 4 5 6] <--- 1st batch target
... [14 15 16]] <--- 2nd batch target
...
... [[ 6 7 8] 3rd subset/ iteration
... [16 17 18]]
... [[ 7 8 9]
... [17 18 19]]
为什么(x, y)的pair直接从1st subset ([0, 1, 2], [1, 2, 3])跳到 2nd subset ([3, 4, 5], [4, 5, 6])，中间的([1, 2, 3], [2, 3, 4])和([2, 3, 4], [3, 4, 5])呢？
我尝试使用不同的batch_size和num_steps，还是没有办法得到中间跳过的subset.
比如batch_size=1, num_steps=3, 结果为:
x: [[0 1 2]]
y: [[1 2 3]]
x: [[3 4 5]]
y: [[4 5 6]]
x: [[6 7 8]]
y: [[7 8 9]]
x: [[ 9 10 11]]
y: [[10 11 12]]
x: [[12 13 14]]
y: [[13 14 15]]
x: [[15 16 17]]
y: [[16 17 18]]
书中说batch_size设为1，所有的样本就会被一个一个放入网络。但其实即使batch_size为1，中间还是有数据被跳过和丢失了（比如x:[1, 2, 3], y: [2, 3, 4]).
看起来问题在于num_steps既控制window_size, 又控制stride，而像cnn的kernel里面这是由两个参数一起控制的。
应该是is_fix为True时，不保存在all_drop吧？
这样会造成tl.utils.dict_to_one(network.all_drop)和feed_dict.update(dp_dict)的技巧无效。这样理解对吗？
import tensorflow as tf
import tensorlayer as tl
print("You can see me")
with tl.ops.suppress_stdout():
print("You can't see me") # build your graphs here
print("You can see me")
Above code doesn't work in pycharm.
Is there any prerequisite for this to work?
I also raised in the tricks's own repo (wagamamaz/tensorlayer-tricks#3). Just to put here as well, because here's the source I tried.
all_images在书中和官网都找不到，输入图片如何得到？
第一印157页，* 代表element-wise multiplicatin, 需要去掉 ，158盒159页的*是对的。
当is_train为False, DropoutLayer会被自动忽略。
这里的set_keep似乎没有定义.
feed_dict[set_keep['denoising1']] = 1
NameError: name 'set_keep' is not defined
我理解all_layers用index获得特定层，那么为什么取得的结果是tensor而不是layer呢？
可视化第一层W错误时运行错误，没有tf.visualize.W这个函数?
Error:
tl.visualize.W(network.all_params[0].eval(), second=10, saveable=True, shape=[28, 28],
AttributeError: module 'tensorlayer.visualize' has no attribute 'W'
这张图似乎没有代码.
如果是的话，可以加点注释，
# 伪代码开始
伪代码...
...
# 伪代码结束
不然有一点confusing，因为虽然是伪代码，但其实看起来也都符合python的语法
tl.files.save_npz(emb_net.all_params, name=model_file_name + '.npz') 给出错误. 尝试用官方的例子https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_word2vec_basic.py, 貌似也有类似的错误。
Save model, data and dictionaries!!!!!!!!!!
Traceback (most recent call last):
File "/home/ly/src/tl-play/ch5_word2vec.py", line 165, in
tl.files.save_npz(emb_net.all_params, name=model_file_name + '.npz')
File "/home/ly/anaconda3/envs/learning/lib/python3.6/site-packages/tensorlayer/files.py", line 1176, in save_npz
np.savez(name, params=save_list_var)
File "/home/ly/anaconda3/envs/learning/lib/python3.6/site-packages/numpy/lib/npyio.py", line 595, in savez
_savez(file, args, kwds, False)
File "/home/ly/anaconda3/envs/learning/lib/python3.6/site-packages/numpy/lib/npyio.py", line 693, in _savez
val = np.asanyarray(val)
File "/home/ly/anaconda3/envs/learning/lib/python3.6/site-packages/numpy/core/numeric.py", line 544, in asanyarray
return array(a, dtype, copy=False, order=order, subok=True)
ValueError: could not broadcast input array from shape (50000,128) into shape (50000)
小问题, 除法和print的使用是python 2.x的.
$ ./tensorflow_model_server --port=9000 --model_base_path=./rnn_model --model_name=antispam
找不到server, #tensorlayer/text-antispam#1
bash: ./tensorflow_model_server: No such file or directory
官方例子似乎也没有定义
可训练参数
我的理解是每一层特征量，应该不需要，也没标注是内存空间，不需要算为KiB。
参数内存占用（KB）
这里应该是kernelsizeinputchannelsoutputchannels+outputchannels乘以4除以1024.
但是书中这个表 感觉上表头和数据计算不严谨
probs = network.outputs还不是概率吧，probs的名字有点迷惑。softmax以后成为概率，却叫sampling_prob. 还是我理解的不同，在增强学习中叫sampling有什么特别的意义吗？
似乎有点重复，为什么同样的reuse值需要在两个地方设置呢？
with tf.variable_scope("MLP", reuse=reuse):
tl.layers.set_name_reuse(reuse)
我输入书中的代码跑出来的结果初始cost非常小(0.45~)，而且没有下降。按照官方例子，初始cost应该在183~,并且不断下降。
debug很久也没找到我输入的和书中哪里不同（除去对书中代码错误或者省略的修正和补充），和官方的有哪里不同（除去书中和官方本身的不同）。又一个例子，应该上传书中的完整代码，可以让debug容易很多。
我输入的代码在https://gist.github.com/arisliang/9fd2bdf554a0ccae1ce7a1dfd19b11d4.
希望下个版本的书能够包括. seq2seq还是挺基本和重要的.
不知道有没有下个版本的计划?
为什么在官方例子(https://github.com/tensorlayer/tensorlayer/blob/master/example/tutorial_generate_text.py) 没有tl.layers.set_name_reuse(reuse)测试也可以跑？
假设设置variable_scope (say 'ae')，denoise_name仍然会用本身的名字denoising1，而不是ae/denoising1. 不然有KeyError。如果有不同scope相同denoise_name的情况下，是不是就没法区别呢？
with tf.variable_scope("ae", reuse=reuse):
...
recon_layer1.pretrain(sess, x=x, X_train=X_train, X_val=X_val, denoise_name='ae/denoising1', n_epoch=n_epochs, batch_size=batch_size, print_freq=print_interval, save=True, save_name='w1pre_')
logging.info(" denoising layer keep: %f" % self.all_drop[set_keep[denoise_name]])
KeyError: 'ae/denoising1'
我尝试显示图3.10，但有错误。
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value test_recon_layer1/b
在line 41使用了新的图模型test_recon_layer1，似乎没有使用已保存的模型，在新的session下面运行就给出了错误。
我运行的代码在https://gist.github.com/arisliang/59a49faf5fe5772ff4ab71b916035b34, 有稍微refactoring改动，但基本应该是一样的。如果书中的代码有放在github或者哪里，重跑和书中完全一样的代码会更精确。
Traceback (most recent call last):
File "/home/ly/src/tl-play/ch5_word2vec.py", line 134, in
_, loss_val = sess.run([train_op, cost], feed_dict=feed_dict)
File "/home/ly/anaconda3/envs/learning/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 905, in run
run_metadata_ptr)
File "/home/ly/anaconda3/envs/learning/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1137, in _run
feed_dict_tensor, options, run_metadata)
File "/home/ly/anaconda3/envs/learning/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1355, in _do_run
options, run_metadata)
File "/home/ly/anaconda3/envs/learning/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1374, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: AttrValue must not have reference type value of float_ref
for attr 'tensor_type'
; NodeDef: word2vec_layer/embeddings/Adagrad/_61 = _Recv[_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_480_word2vec_layer/embeddings/Adagrad", tensor_type=DT_FLOAT_REF, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^Adagrad/learning_rate, ^Adagrad/update_word2vec_layer/embeddings/UnsortedSegmentSum, ^Adagrad/update_word2vec_layer/embeddings/Unique); Op<name=_Recv; signature= -> tensor:tensor_type; attr=tensor_type:type; attr=tensor_name:string; attr=send_device:string; attr=send_device_incarnation:int; attr=recv_device:string; attr=client_terminated:bool,default=false; is_stateful=true>
[[Node: word2vec_layer/embeddings/Adagrad/_61 = _Recv[_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_480_word2vec_layer/embeddings/Adagrad", tensor_type=DT_FLOAT_REF, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^Adagrad/learning_rate, ^Adagrad/update_word2vec_layer/embeddings/UnsortedSegmentSum, ^Adagrad/update_word2vec_layer/embeddings/Unique)]]
如果换成AdamOptimizer，就没有这个错误.
代码在https://gist.github.com/arisliang/a197b17b6330a86a56e500907dcd07c5
可能是AdagradOptimizer的问题.
遇到几个这种问题了，有的还没法解决。就是按照书中的例子输入了代码，但结果不正确。但由于有的书中代码与官方代码有很大区别，这就导致自己按照书中的例子输入的代码，很难与官方代码进行比较，错误也很难找出。
如果能够把书中的代码上传到github，相信自己按照书中输入的代码就比较容易比较了。
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.