TensorSpace.js
Present Tensor in Space
English | 中文
TensorSpace is a neural network 3D visualization framework built by TensorFlow.js, Three.js and Tween.js. TensorSpace provides Keras-like APIs to build deep learning layers, load pre-trained models, and generate a 3D visualization in the browser. From TensorSpace, it is intuitive to learn what the model structure is, how the model is trained and how the model predicts the results based on the intermediate information. After preprocessing the model, TensorSpace supports to visualize pre-trained model from TensorFlow, Keras and TensorFlow.js.
Fig.1 - Interactive LeNet created by TensorSpace
Table of Content
Motivation
TensorSpace is a neural network 3D visualization framework designed for not only showing the basic model structure, but also presenting the processes of internal feature abstractions, intermediate data manipulations and final inference generations.
By applying TensorSpace API, it is more intuitive to visualize and understand any pre-trained models built by TensorFlow, Keras, TensorFlow.js, etc. TensorSpace introduces a way for front end developers to be involved in the deep learning ecosystem. As an open source library, TensorSpace team welcomes any further development on visualization applications.
- Interactive -- Use Keras-like API to build interactive model in browsers.
- Intuitive -- Visualize the information from intermediate inferences.
- Integrative -- Support pre-trained models from TensorFlow, Keras, TensorFlow.js.
Getting Start
Install
- Step 1: Download TensorSpace.js
There are three ways to download TensorSpace.js: npm, yarn, or official website
Option 1: NPM
npm install tensorspaceOption 2: Yarn
yarn add tensorspaceOption 3: Website download page
- Step 2: Install Dependency
Include TensorFlow.js, Three.js, Tween.js, TrackballControl.js in html file before include TensorSpace.js
<script src="tf.min.js"></script>
<script src="three.min.js"></script>
<script src="tween.min.js"></script>
<script src="TrackballControls.js"></script>- Step 3: Install TensorSpace.js
Include TensorSpace.js into html file:
<script src="tensorspace.min.js"></script>Preprocessing
For presenting multiple intermediate outputs, we need to preprocess the pre-trained model.
Based on different training libraries, we provide different tutorials: TensorFlow model preprocessing, Keras model preprocessing and TensorFlow.js model preprocessing.
Usage
If installed TensorSpace and preprocessed the pre-trained deep learning model successfully, let's create an interactive 3D TensorSpace model.
For the convenience, feel free to use the resources from our HelloWorld directory.
We will use the preprocessed TensorSpace compatible LeNet model and sample input data ("5"). All source code can be found from helloworld.html.
First, we need to new a TensorSpace model instance:
let container = document.getElementById( "container" );
let model = new TSP.models.Sequential( container );Next, based on the LeNet structure: Input + 2 X (Conv2D & Maxpooling) + 3 X (Dense), we build the structure of the model:
model.add( new TSP.layers.GreyscaleInput({ shape: [28, 28, 1] }) );
model.add( new TSP.layers.Padding2d({ padding: [2, 2] }) );
model.add( new TSP.layers.Conv2d({ kernelSize: 5, filters: 6, strides: 1 }) );
model.add( new TSP.layers.Pooling2d({ poolSize: [2, 2], strides: [2, 2] }) );
model.add( new TSP.layers.Conv2d({ kernelSize: 5, filters: 16, strides: 1 }) );
model.add( new TSP.layers.Pooling2d({ poolSize: [2, 2], strides: [2, 2] }) );
model.add( new TSP.layers.Dense({ units: 120 }) );
model.add( new TSP.layers.Dense({ units: 84 }) );
model.add( new TSP.layers.Output1d({
units: 10,
outputs: ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]
}) );Last, we should load our preprocessed TensorSpace compatible model and use init() method to create the TensorSpace model:
model.load({
type: "tfjs",
url: './lenetModel/mnist.json'
});
model.init(function(){
console.log("Hello World from TensorSpace!");
});We can get the following Fig. 2 model in the browser if everything looks good.
Fig. 2 - LeNet model without any input data
We provide a extracted file which is a handwritten "5" as the input of our model: (online demo)
model.init(function() {
model.predict( image_5 );
});
We put the predict( image_5 ) method in the callback function of init() to ensure the prediction is after the initialization complete.
Click the CodePen logo to try it in CodePen:
Fig. 3 - LeNet model with input data "5"
Example
- LeNet
Fig.4 - Interactive LeNet created by TensorSpace
- AlexNet
Fig.5 - Interactive AlexNet created by TensorSpace
- Yolov2-tiny
Fig.6 - Interactive Yolov2-tiny created by TensorSpace
- ResNet-50
Fig.7 - Interactive ResNet-50 created by TensorSpace
- Vgg16
Fig.8 - Interactive Vgg16 created by TensorSpace
- ACGAN
Fig.9 - Interactive ACGAN created by TensorSpace
- MobileNetv1
Fig.10 - Interactive MobileNetv1 created by TensorSpace
View models locally
As some models above are extremely large, view them locally may be a good choice.
- Step 1:
cloneTensorSpace Repo
git clone https://github.com/tensorspace-team/tensorspace.git- Step 2:
buildlatest TensorSpace.js
cd tensorspace
npm run build- Step 3:
Open "html" file in examples folder in local web server.
Documentation
- For a quick start, checkout Getting Start
- To download/install, see Download
- To learn more about the Basic Concepts
- To process a deep learning model, checkout Model Preprocessing
- To learn core components: Models, Layers and Merge Function
- Checkout the official website TensorSpace.org for more about TensorSpace.
Changelog
Contributors
Thanks goes to these wonderful people (emoji key):
 syt123450 💻 🎨 📖 💡 |
 Chenhua Zhu 💻 🎨 ✅ 💡 |
 YaoXing Liu 💻 🎨 ✅ 💡 |
 Qi(Nora) 💻 🎨 |
|---|
Contact
If you have any issue or doubt, feel free to contact us by:
- Email: [email protected]
- GitHub Issues: create issue
- Slack: #questions
- Gitter: #Lobby
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent