snowkylin / tensorflow-handbook Goto Github PK
View Code? Open in Web Editor NEW简单粗暴 TensorFlow 2 | A Concise Handbook of TensorFlow 2 | 一本简明的 TensorFlow 2 入门指导教程
Home Page: https://tf.wiki
简单粗暴 TensorFlow 2 | A Concise Handbook of TensorFlow 2 | 一本简明的 TensorFlow 2 入门指导教程
Home Page: https://tf.wiki
基本上比较全面的介绍了 Node.js 和 Browser 里面如何使用 Tensorflow.js ,同时提供了一个完整的 Seq2Seq 模型供读者更深度的了解和上手。
我的开发子 Repo: https://github.com/huan/javascript-concise-chitchat
cc: @wangtz
hi, the android sample is be removed , could you push the code again ?
thanks.
您好,有幸读到您的手册,我有一点小小的疑问,求解答!
同源问题:https://ask.csdn.net/questions/907153
我的运行环境:
pycharm2019.2.3
python 3.7
TensorFlow 2.0
代码如下
import tensorflow as tf
import numpy as np
class DataLoader():
def __init__(self):
path = tf.keras.utils.get_file('nietzsche.txt',
origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')
with open(path, encoding='utf-8') as f:
self.raw_text = f.read().lower()
self.chars = sorted(list(set(self.raw_text)))
self.char_indices = dict((c, i) for i, c in enumerate(self.chars))
self.indices_char = dict((i, c) for i, c in enumerate(self.chars))
self.text = [self.char_indices[c] for c in self.raw_text]
def get_batch(self, seq_length, batch_size):
seq = []
next_char = []
for i in range(batch_size):
index = np.random.randint(0, len(self.text) - seq_length)
seq.append(self.text[index:index+seq_length])
next_char.append(self.text[index+seq_length])
return np.array(seq), np.array(next_char) # [batch_size, seq_length], [num_batch]
class RNN(tf.keras.Model):
def __init__(self, num_chars, batch_size, seq_length):
super().__init__()
self.num_chars = num_chars
self.seq_length = seq_length
self.batch_size = batch_size
self.cell = tf.keras.layers.LSTMCell(units=256)
self.dense = tf.keras.layers.Dense(units=self.num_chars)
def call(self, inputs, from_logits=False):
inputs = tf.one_hot(inputs, depth=self.num_chars) # [batch_size, seq_length, num_chars]
state = self.cell.get_initial_state(batch_size=self.batch_size, dtype=tf.float32)
for t in range(self.seq_length):
output, state = self.cell(inputs[:, t, :], state)
logits = self.dense(output)
if from_logits:
return logits
else:
return tf.nn.softmax(logits)
num_batches = 10
seq_length = 40
batch_size = 50
learning_rate = 1e-3
data_loader = DataLoader()
model = RNN(num_chars=len(data_loader.chars), batch_size=batch_size, seq_length=seq_length)
optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)
for batch_index in range(num_batches):
X, y = data_loader.get_batch(seq_length, batch_size)
with tf.GradientTape() as tape:
y_pred = model(X)
loss = tf.keras.losses.sparse_categorical_crossentropy(y_true=y, y_pred=y_pred)
loss = tf.reduce_mean(loss)
print("batch %d: loss %f" % (batch_index, loss.numpy()))
grads = tape.gradient(loss, model.variables)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables))
def predict(self, inputs, temperature=1.):
batch_size, _ = tf.shape(inputs)
logits = self(inputs, from_logits=True)
print(logits)
print(temperature)
prob = tf.nn.softmax(logits / temperature).numpy()
return np.array([np.random.choice(self.num_chars, p=prob[i, :])
for i in range(batch_size.numpy())])
X_, _ = data_loader.get_batch(seq_length, 1)
for diversity in [0.2, 0.5, 1.0, 1.2]:
X = X_
print("diversity %f:" % diversity)
for t in range(400):
y_pred = model.predict(X, diversity)
print(data_loader.indices_char[y_pred[0]], end='', flush=True)
X = np.concatenate([X[:, 1:], np.expand_dims(y_pred, axis=1)], axis=-1)
print("\n")
报错:
runfile('F:/pyth/pj3/study3.py', wdir='F:/pyth/pj3')
batch 0: loss 4.044459
batch 1: loss 4.025946
batch 2: loss 4.001545
batch 3: loss 3.980800
batch 4: loss 3.945248
batch 5: loss 3.867068
batch 6: loss 3.684950
batch 7: loss 3.236459
batch 8: loss 3.574704
batch 9: loss 3.551273
diversity 0.200000:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "D:\Program Files\JetBrains\PyCharm 2019.2.3\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "D:\Program Files\JetBrains\PyCharm 2019.2.3\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "F:/pyth/pj3/study3.py", line 93, in <module>
y_pred = model.predict(X, diversity)
File "D:\ProgramData\Anaconda3\envs\kingtf2\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 909, in predict
use_multiprocessing=use_multiprocessing)
File "D:\ProgramData\Anaconda3\envs\kingtf2\lib\site-packages\tensorflow_core\python\keras\engine\training_arrays.py", line 722, in predict
callbacks=callbacks)
File "D:\ProgramData\Anaconda3\envs\kingtf2\lib\site-packages\tensorflow_core\python\keras\engine\training_arrays.py", line 362, in model_iteration
batch_ids = index_array[batch_start:batch_end]
TypeError: slice indices must be integers or None or have an __index__ method
WARNING:tensorflow:Tensor._shape is private, use Tensor.shape instead. Tensor._shape will eventually be removed.
WARNING:tensorflow:Tensor._shape is private, use Tensor.shape instead. Tensor._shape will eventually be removed.
https://www.tensorflow.org/guide/keras/save_and_serialize
# Save the model
model.save('path_to_my_model.h5')
# Recreate the exact same model purely from the file
new_model = keras.models.load_model('path_to_my_model.h5')
感谢作者撰写此学习教程,受益匪浅。现在在学习过程中时常会发现一些问题,但又没有场合提问。请问能否创建一个中文讨论区,以供学习者交流?谢谢!
TPU 章节计划包括以下几部分内容:
目前看来,第一版可能来不及涵盖,所以计划在第一版中不包括 TPU 部分内容。(如果之后书出版之前还有时间补充,可以补充最基本的 Google Cloud TPU 配置方法)
大家看这样是否可以? @snowkylin @dpinthinker
原文中是1/2,应该是1/n吧
我在学习“TensorFlow 模型导出”的如下概念时:
使用继承 tf.keras.Model 类建立的 Keras 模型同样可以以相同方法导出,唯须注意 call 方法需要以 @tf.function 修饰,以转化为 SavedModel 支持的计算图
根据网站提供的代码,我执行相应的代码时出现这样的Error:
ValueError: Shape mismatch: The shape of labels (received (50,)) should equal the shape of logits except for the last dimension (received (0, 10)).
任何人能给出一点建议和帮助吗,感谢!
代码是完全拷贝于网站给出的代码(地址)。完整代码如下:(或从我的GitHub下载,地址)
"""
#---------------------------------使用 SavedModel 完整导出模型---------------------------
#Keras 模型均可方便地导出为 SavedModel 格式。
#不过需要注意的是,因为 SavedModel 基于计算图,所以对于使用继承 tf.keras.Model 类建立的 Keras 模型,
#其需要导出到 SavedModel 格式的方法(比如 call )都需要使用 @tf.function 修饰( @tf.function 的使用方式见 前文 )。
Created on Sat Dec 28 20:44:51 2019
@author:
"""
import numpy as np
import tensorflow as tf
#-----------------------------数据获取及预处理------------------------------
#tf.keras.datasets
class MNISTLoader():
def init(self):
mnist = tf.keras.datasets.mnist
(self.train_data, self.train_label), (self.test_data, self.test_label) = mnist.load_data()
# MNIST中的图像默认为uint8(0-255的数字)。以下代码将其归一化到0-1之间的浮点数,并在最后增加一维作为颜色通道
self.train_data = np.expand_dims(self.train_data.astype(np.float32) / 255.0, axis=-1) # [60000, 28, 28, 1]
self.test_data = np.expand_dims(self.test_data.astype(np.float32) / 255.0, axis=-1) # [10000, 28, 28, 1]
self.train_label = self.train_label.astype(np.int32) # [60000]
self.test_label = self.test_label.astype(np.int32) # [10000]
self.num_train_data, self.num_test_data = self.train_data.shape[0], self.test_data.shape[0]
def get_batch(self, batch_size):
# 从数据集中随机取出batch_size个元素并返回
index = np.random.randint(0, np.shape(self.train_data)[0], batch_size)
return self.train_data[index, :], self.train_label[index]
#-----------------------------模型的构建------------------------------
#: tf.keras.Model 和 tf.keras.layers
class MLP(tf.keras.Model):
def init(self):
super().init()
self.flatten = tf.keras.layers.Flatten() # Flatten层将除第一维(batch_size)以外的维度展平
self.dense1 = tf.keras.layers.Dense(units=100, activation=tf.nn.relu)
self.dense2 = tf.keras.layers.Dense(units=10)
#使用继承 tf.keras.Model 类建立的 Keras 模型同样可以以相同方法导出,唯须注意 call 方法需要以 @tf.function 修饰,
#以转化为 SavedModel 支持的计算图,代码如下:
@tf.function
def call(self, inputs): # [batch_size, 28, 28, 1]
x = self.flatten(inputs) # [batch_size, 784]
x = self.dense1(x) # [batch_size, 100]
x = self.dense2(x) # [batch_size, 10]
output = tf.nn.softmax(x)
return output
#-----------------------------定义一些模型超参数------------------------------
num_epochs = 1
batch_size = 50
learning_rate = 0.01
#-----------------------------实例化模型和数据读取类------------------------------
#并实例化一个 tf.keras.optimizer 的优化器(这里使用常用的 Adam 优化器)
model = MLP()
data_loader = MNISTLoader()
optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)
#-----------------------------迭代------------------------------
#然后迭代进行以下步骤:
#从 DataLoader 中随机取一批训练数据;
#将这批数据送入模型,计算出模型的预测值;
#将模型预测值与真实值进行比较,计算损失函数(loss)。这里使用 tf.keras.losses 中的交叉熵函数作为损失函数;
#计算损失函数关于模型变量的导数;
#将求出的导数值传入优化器,使用优化器的 apply_gradients 方法更新模型参数以最小化损失函数
num_batches = int(data_loader.num_train_data // batch_size * num_epochs)
for batch_index in range(num_batches):
X, y = data_loader.get_batch(batch_size)
with tf.GradientTape() as tape:
start_index, end_index = batch_index * batch_size, (batch_index + 1) * batch_size
#模型导入并测试性能的过程也相同,唯须注意模型推断时需要显式调用 call 方法,即使用:
y_pred = model.call(data_loader.test_data[start_index: end_index])
loss = tf.keras.losses.sparse_categorical_crossentropy(y_true=y, y_pred=y_pred)
loss = tf.reduce_mean(loss)
print("batch %d: loss %f" % (batch_index, loss.numpy()))
grads = tape.gradient(loss, model.variables)
optimizer.apply_gradients(grads_and_vars=zip(grads, model.variables))
#----------------------------------------------模型的评估------------------------------
#tf.keras.metrics
sparse_categorical_accuracy = tf.keras.metrics.SparseCategoricalAccuracy()
num_batches = int(data_loader.num_test_data // batch_size)
for batch_index in range(num_batches):
start_index, end_index = batch_index * batch_size, (batch_index + 1) * batch_size
y_pred = model.predict(data_loader.test_data[start_index: end_index])
sparse_categorical_accuracy.update_state(y_true=data_loader.test_label[start_index: end_index], y_pred=y_pred)
print("test accuracy: %f" % sparse_categorical_accuracy.result())
#---------------------------保存模型-------------------------------------------
tf.saved_model.save(model, "saved\2\")
#---------------------------载入模型-------------------------------------------
model = tf.saved_model.load("saved\2\")
#---------------------------预测-------------------------------------------
data_loader = MNISTLoader()
sparse_categorical_accuracy = tf.keras.metrics.SparseCategoricalAccuracy()
num_batches = int(data_loader.num_test_data // batch_size)
for batch_index in range(num_batches):
start_index, end_index = batch_index * batch_size, (batch_index + 1) * batch_size
y_pred = model(data_loader.test_data[start_index: end_index])
sparse_categorical_accuracy.update_state(y_true=data_loader.test_label[start_index: end_index], y_pred=y_pred)
print("test accuracy: %f" % sparse_categorical_accuracy.result())
尊敬的作者:
请问0.4版本的PDF能否下载?
source/_static
目录,或huan/concise-chit-chat
做成git sub module这个章节我的开发 Repo 在这里: https://github.com/huan/concise-chit-chat
Tensorflow
-> TensorFlow
这就是
: Replace with better wordsy = ax + b
-> LaTex renderSaved Model
is best practice对Keras或者tfjs原生的层模型
<- const saveResult = await model.save('localstorage://my-model-1')
TensorFlow.js Chapter: #6
设置显存按需使用的 code 报错。
gpus = tf.config.experimental.list_physical_devices(device_type='GPU')
for gpu in gpus:
tf.config.experimental.set_memory_growth(device=gpu, True)
显示 SyntaxError: positional arguments follows keyword argument
第三行需要修改一下才能运行
tf.config.experimental.set_memory_growth(gpu, True)
或
tf.config.experimental.set_memory_growth(device=gpu, enable=True)
您好,感谢分享, 请教一个多机训练问题。
使用keras和tf.distribute.experimental.MultiWorkerMirroredStrategy()进行多机训练时,batch_size=batch_size * worker_num,那么输入的数据(tfrecord格式)需要切分成worker_num份吗?如果不切分,每个worker都读取所有的数据,这种分布式策略会不会自动对数据进行分发呢?
tensorflow-handbook/docs/_static/code/zh/model/rl/rl.py
Lines 38 to 46 in b955fc1
以上源码中43行应该不是再用固定的initial_epsilon
,而应该是epsilon
。
RT,虽然语法看上去很动态、又可以调用其他动态语言,但Swift本身还是一个静态强类型语言(这个错误看上去还挺基础的hmmmmmm)
def compile(self, learning_rate, momentum):
"""Gets the model ready for training. Adds losses, regularization, and
metrics. Then calls the Keras compile() function.
"""
# Optimizer object
optimizer = keras.optimizers.SGD(
lr=learning_rate, momentum=momentum,
clipnorm=self.config.GRADIENT_CLIP_NORM)
# Add Losses
# First, clear previously set losses to avoid duplication
self.keras_model._losses = []
self.keras_model._per_input_losses = {}
loss_names = [
"rpn_class_loss", "rpn_bbox_loss",
"mrcnn_class_loss", "mrcnn_bbox_loss", "mrcnn_mask_loss"]
for name in loss_names:
layer = self.keras_model.get_layer(name)
if layer.output in self.keras_model.losses:
continue
loss = (
tf.reduce_mean(layer.output, keepdims=True)
* self.config.LOSS_WEIGHTS.get(name, 1.))
self.keras_model.add_loss(loss)
# Add L2 Regularization
# Skip gamma and beta weights of batch normalization layers.
reg_losses = [
keras.regularizers.l2(self.config.WEIGHT_DECAY)(w) / tf.cast(tf.size(w), tf.float32)
for w in self.keras_model.trainable_weights
if 'gamma' not in w.name and 'beta' not in w.name]
self.keras_model.add_loss(tf.add_n(reg_losses))
# Compile
self.keras_model.compile(
optimizer=optimizer,
loss=[None] * len(self.keras_model.outputs))
# Add metrics for losses
for name in loss_names:
if name in self.keras_model.metrics_names:
continue
layer = self.keras_model.get_layer(name)
self.keras_model.metrics_names.append(name)
loss = (
tf.reduce_mean(layer.output, keepdims=True)
* self.config.LOSS_WEIGHTS.get(name, 1.))
self.keras_model.metrics_tensors.append(loss)
这个是mask rcnn的代码,这部分要怎么修改才能在tensorflow 2.0, tf.keras下运行呢
将代码:
def get_batch(self, batch_size):
# 从数据集中随机取出batch_size个元素并返回
index = np.random.randint(0, np.shape(self.train_data)[0], batch_size)
return self.train_data[index, :], self.train_label[index]
改为:
def get_batch(self, batch_size):
# 从数据集中随机取出batch_size个元素并返回
# index = np.random.randint(0, np.shape(self.train_data)[0], batch_size)
index = np.random.choice(np.shape(self.train_data)[0], batch_size, replace=False)
return self.train_data[index, :], self.train_label[index]
可避免每次获取的数据中不存在重复项。
Thanks for your great work in advance!
I have some questions about the following description:
Thanks again.
n = 'string'
is not grammatical because single quotes are not valid. Should be double-quotes.matmul
. The bullet operator is highly subject to change.Xihan,下次更新文档的时候麻烦抽空把这俩 404 解决一下呗,谢谢啦 ;-)
404 - https://tf.wiki/en/deployment/en_savedmodel
(found at https://tf.wiki/en/deployment/serving.html)
404 - https://tf.wiki/py-modindex.html
(found at https://tf.wiki/)
在CPU上需要时间为97ms
where is these numbers coming from?TensorFlow.js Chapter: #6
计划 Swift 部分和 JavaScript 部分,并列为 Tensorflow 的两大跨语言脚本。
原计划:本书第一版完成 JavaScript 和 Swift
新计划:将 Swift 部分,顺延到本书第二版进行修订。第一版中暂不包含 Swift 章节。
大家看这样是否可以? @snowkylin @dpinthinker
UPDATE(17, Mar) 经过和锡涵讨论,TF2.0正式发布之前还能有一些时间,所以决定原计划不变。撰写一个最基本的版本,5-10页
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.