d2l-ai / d2l-pytorch-sagemaker-studio-lab Goto Github PK

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-10-07T18:39:36.505187Z",
     "start_time": "2021-10-07T18:39:36.404141Z"
    }
   },
   "source": [
    "# Installing `d2l` Enviroment\n",
    "\n",
    "The `d2l` environment contains required libraries for running all the D2L notebooks.\n",
    "\n",
    "To check whether the `d2l` kernel exists, you can open the terminal (click \"+\" on the top left) and enter the command `$ conda-env list`.\n",
    "If you don't see the `d2l` environment in your existing kernels in the above list, \n",
    "you can change directory with the command: \n",
    "\n",
    "`$ cd /home/studio-lab-user/d2l-pytorch-sagemaker-studio-lab`\n",
    "\n",
    "\n",
    "and create a new environment using:\n",
    "\n",
    "`$ conda env create --name d2l --file environment.yml` \n",
    "\n",
    "\n",
    "\n",
    "# Running with CPU or GPU Instances\n",
    "\n",
    "\n",
    "You may use CPU instances to run most notebooks in the first six chapters. Running other computationally heavy notebooks can be more efficient on GPU instances.\n",
    "\n",
    "You can switch between CPU and GPU instances by clicking `Stop instance` and restarting a new instance. Your (edited) notebooks from the old instance will be saved and relaunched with the new instance.\n",
    "\n",
    "\n",
    "# About This Book\n",
    "\n",
    "[Dive into Deep Learning (D2L)](https://www.d2l.ai/) is an open-source, interactive book that teaches the ideas, the mathematical theory, and the code that powers deep learning. It provides a comprehensive overview of deep learning principles and a state-of-the-art introduction to deep learning in computer vision and natural language processing.  \n",
    "\n",
    "D2L has been adopted for teaching by 300 universities from 55 countries, including:\n",
    "\n",
    "![](static/img/univ-logos.png)\n",
    "\n",
    "![](static/img/map.png)\n",
    "\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "This book is for students (undergraduate or graduate),\n",
    "engineers, and researchers, who seek a solid grasp\n",
    "of the practical techniques of deep learning.\n",
    "Because we explain every concept from scratch,\n",
    "no previous background in deep learning or machine learning is required.\n",
    "Fully explaining the methods of deep learning\n",
    "requires some mathematics and programming,\n",
    "but we will only assume that you come in with some basics,\n",
    "including modest amounts of linear algebra, \n",
    "calculus, probability, and Python programming.\n",
    "Just in case you forget the basics,\n",
    "the Appendix chapter *Mathematics for Deep Learning*\n",
    "provides a refresher \n",
    "on most of the mathematics \n",
    "you will find in this book.\n",
    "Most of the time, we will prioritize \n",
    "intuition and ideas\n",
    "over mathematical rigor.\n",
    "\n",
    "## Content and Structure\n",
    "\n",
    "This book can be roughly divided into three parts \n",
    "(corresponding to three different colors):\n",
    "preliminaries, deep learning techniques,\n",
    "and advanced topics with real world applications (as shown in the figure below).\n",
    "\n",
    "![Book structure](https://d2l.ai/_images/book-org.svg)\n",
    "\n",
    "\n",
    "* The first part (chapter 1-4) covers basics and preliminaries.\n",
    "Chapter *1. Introduction* offers \n",
    "an overview of deep learning.\n",
    "Next, Chapter *2. Preliminaries* \n",
    "teaches the prerequisites required\n",
    "for hands-on deep learning, \n",
    "such as how to store and manipulate data,\n",
    "and how to apply various numerical operations \n",
    "based on basic concepts from linear algebra, \n",
    "calculus, and probability.\n",
    "Then, Chapter *3. Linear Neural Networks* and \n",
    "Chapter *4. Multilayer Perceptrons* \n",
    "cover the most basic concepts \n",
    "and techniques in deep learning,\n",
    "including regression and classification;\n",
    "linear models and multilayer perceptrons;\n",
    "and overfitting and regularization.\n",
    "\n",
    "* The next six chapters focus on modern deep learning techniques.\n",
    "Chapter *5. Deep Learning Computation* describes \n",
    "the key computational components \n",
    "of deep learning systems\n",
    "and lays the groundwork\n",
    "for our subsequent implementations\n",
    "of more complex models.\n",
    "Next, Chapter *6. Convolutional Neural Networks* and \n",
    "Chapter *7. Modern Convolutional Neural Networks* \n",
    "demonstrate convolutional neural networks (CNNs), \n",
    "powerful tools that form the backbone \n",
    "of most modern computer vision (CV) systems.\n",
    "Similarly, Chapter *8. Recurrent Neural Networks*  and \n",
    "Chapter *9. Modern Recurrent Neural Networks* \n",
    "illustrate recurrent neural networks (RNNs), \n",
    "models that exploit sequential (e.g., temporal) \n",
    "structure in data and are commonly used\n",
    "for natural language processing (NLP) \n",
    "and time series prediction.\n",
    "In Chapter *10. Attention Mechanisms*, \n",
    "we introduce a relatively new class of models\n",
    "based on attention and transformer mechanisms, \n",
    "which is the dominant architecture\n",
    "for most NLP tasks.\n",
    "These chapters will bring you up to speed \n",
    "on the most powerful and general tools\n",
    "that are widely used by deep learning practitioners.\n",
    "\n",
    "* Part three discusses scalability, efficiency, and applications.\n",
    "First, in Chapter *11. Optimization Algorithms*,\n",
    "we discuss common optimization methods\n",
    "used to train deep learning models.\n",
    "The next Chapter *12. Computational Performance* \n",
    "examines several key factors\n",
    "that influence the computational performance \n",
    "of your deep learning code.\n",
    "In Chapter *13. Computer Vision*,\n",
    "we illustrate major applications \n",
    "of deep learning in CV.\n",
    "In Chapter *14. Natural Language Processing: Pretraining* \n",
    "and Chapter *15. Natural Language Processing: Applications*,\n",
    "we show how to pretrain language representation models \n",
    "and apply them to NLP tasks.\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# Table of contents\n",
    "\n",
    "## Front Matter\n",
    "\n",
    "* [Preface](chapter_preface/index.ipynb)\n",
    "* [Installation](chapter_installation/index.ipynb)\n",
    "* [Notation](chapter_notation/index.ipynb)\n",
    "\n",
    "## Part 1: Basics and Preliminaries\n",
    "\n",
    "* **Chapter 1:** [Introduction](chapter_introduction/index.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 2:** [Preliminaries](chapter_preliminaries/index.ipynb)\n",
    "    * [Data Manipulation](chapter_preliminaries/ndarray.ipynb)\n",
    "    * [Data Preprocessing](chapter_preliminaries/pandas.ipynb)\n",
    "    * [Linear Algebra](chapter_preliminaries/linear-algebra.ipynb)\n",
    "    * [Calculus](chapter_preliminaries/calculus.ipynb)\n",
    "    * [Automatic Differentiation](chapter_preliminaries/autograd.ipynb)\n",
    "    * [Probability](chapter_preliminaries/probability.ipynb)\n",
    "    * [Documentation](chapter_preliminaries/lookup-api.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 3:** [Linear Neural Networks](chapter_linear-networks/index.ipynb)\n",
    "    * [Linear Regression](chapter_linear-networks/linear-regression.ipynb)\n",
    "    * [Linear Regression Implementation from Scratch](chapter_linear-networks/linear-regression-scratch.ipynb)\n",
    "    * [Concise Implementation of Linear Regression](chapter_linear-networks/linear-regression-concise.ipynb)\n",
    "    * [Softmax Regression](chapter_linear-networks/softmax-regression.ipynb)\n",
    "    * [The Image Classification Dataset](chapter_linear-networks/image-classification-dataset.ipynb)\n",
    "    * [Implementation of Softmax Regression from Scratch](chapter_linear-networks/softmax-regression-scratch.ipynb)\n",
    "    * [Concise Implementation of Softmax Regression](chapter_linear-networks/softmax-regression-concise.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 4:** [Multilayer Perceptrons](chapter_multilayer-perceptrons/index.ipynb)\n",
    "    * [Multilayer Perceptrons](chapter_multilayer-perceptrons/mlp.ipynb)\n",
    "    * [Implementation of Multilayer Perceptrons from Scratch](chapter_multilayer-perceptrons/mlp-scratch.ipynb)\n",
    "    * [Concise Implementation of Multilayer Perceptrons](chapter_multilayer-perceptrons/mlp-concise.ipynb)\n",
    "    * [Model Selection, Underfitting, and Overfitting](chapter_multilayer-perceptrons/underfit-overfit.ipynb)\n",
    "    * [Weight Decay](chapter_multilayer-perceptrons/weight-decay.ipynb)\n",
    "    * [Dropout](chapter_multilayer-perceptrons/dropout.ipynb)\n",
    "    * [Forward Propagation, Backward Propagation, and Computational Graphs](chapter_multilayer-perceptrons/backprop.ipynb)\n",
    "    * [Numerical Stability and Initialization](chapter_multilayer-perceptrons/numerical-stability-and-init.ipynb)\n",
    "    * [Environment and Distribution Shift](chapter_multilayer-perceptrons/environment.ipynb)\n",
    "    * [Predicting House Prices on Kaggle](chapter_multilayer-perceptrons/kaggle-house-price.ipynb)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "## Part 2: Modern Deep Learning Techniques\n",
    "\n",
    "* **Chapter 5:** [Deep Learning Computation](chapter_deep-learning-computation/index.ipynb)\n",
    "    * [Layers and Blocks](chapter_deep-learning-computation/model-construction.ipynb)\n",
    "    * [Parameter Management](chapter_deep-learning-computation/parameters.ipynb)\n",
    "    * [Custom Layers](chapter_deep-learning-computation/custom-layer.ipynb)\n",
    "    * [File I/O](chapter_deep-learning-computation/read-write.ipynb)\n",
    "    * [GPUs](chapter_deep-learning-computation/use-gpu.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 6:** [Convolutional Neural Networks](chapter_convolutional-neural-networks/index.ipynb)\n",
    "    * [From Fully-Connected Layers to Convolutions](chapter_convolutional-neural-networks/why-conv.ipynb)\n",
    "    * [Convolutions for Images](chapter_convolutional-neural-networks/conv-layer.ipynb)\n",
    "    * [Padding and Stride](chapter_convolutional-neural-networks/padding-and-strides.ipynb)\n",
    "    * [Multiple Input and Multiple Output Channels](chapter_convolutional-neural-networks/channels.ipynb)\n",
    "    * [Pooling](chapter_convolutional-neural-networks/pooling.ipynb)\n",
    "    * [Convolutional Neural Networks (LeNet)](chapter_convolutional-neural-networks/lenet.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 7:** [Modern Convolutional Neural Networks](chapter_convolutional-modern/index.ipynb)\n",
    "    * [Deep Convolutional Neural Networks (AlexNet)](chapter_convolutional-modern/alexnet.ipynb)\n",
    "    * [Networks Using Blocks (VGG)](chapter_convolutional-modern/vgg.ipynb)\n",
    "    * [Network in Network (NiN)](chapter_convolutional-modern/nin.ipynb)\n",
    "    * [Networks with Parallel Concatenations (GoogLeNet)](chapter_convolutional-modern/googlenet.ipynb)\n",
    "    * [Batch Normalization](chapter_convolutional-modern/batch-norm.ipynb)\n",
    "    * [Residual Networks (ResNet)](chapter_convolutional-modern/resnet.ipynb)\n",
    "    * [Densely Connected Networks (DenseNet)](chapter_convolutional-modern/densenet.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 8:** [Recurrent Neural Networks](chapter_recurrent-neural-networks/index.ipynb)\n",
    "    * [Sequence Models](chapter_recurrent-neural-networks/sequence.ipynb)\n",
    "    * [Text Preprocessing](chapter_recurrent-neural-networks/text-preprocessing.ipynb)\n",
    "    * [Language Models and the Dataset](chapter_recurrent-neural-networks/language-models-and-dataset.ipynb)\n",
    "    * [Recurrent Neural Networks](chapter_recurrent-neural-networks/rnn.ipynb)\n",
    "    * [Implementation of Recurrent Neural Networks from Scratch](chapter_recurrent-neural-networks/rnn-scratch.ipynb)\n",
    "    * [Concise Implementation of Recurrent Neural Networks](chapter_recurrent-neural-networks/rnn-concise.ipynb)\n",
    "    * [Backpropagation Through Time](chapter_recurrent-neural-networks/bptt.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 9:** [Modern Recurrent Neural Networks](chapter_recurrent-modern/index.ipynb)\n",
    "    * [Gated Recurrent Units (GRU)](chapter_recurrent-modern/gru.ipynb)\n",
    "    * [Long Short-Term Memory (LSTM)](chapter_recurrent-modern/lstm.ipynb)\n",
    "    * [Deep Recurrent Neural Networks](chapter_recurrent-modern/deep-rnn.ipynb)\n",
    "    * [Bidirectional Recurrent Neural Networks](chapter_recurrent-modern/bi-rnn.ipynb)\n",
    "    * [Machine Translation and the Dataset](chapter_recurrent-modern/machine-translation-and-dataset.ipynb)\n",
    "    * [Encoder-Decoder Architecture](chapter_recurrent-modern/encoder-decoder.ipynb)\n",
    "    * [Sequence to Sequence Learning](chapter_recurrent-modern/seq2seq.ipynb)\n",
    "    * [Beam Search](chapter_recurrent-modern/beam-search.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 10:** [Attention Mechanisms](chapter_attention-mechanisms/index.ipynb)\n",
    "    * [Attention Cues](chapter_attention-mechanisms/attention-cues.ipynb)\n",
    "    * [Attention Pooling: Nadaraya-Watson Kernel Regression](chapter_attention-mechanisms/nadaraya-watson.ipynb)\n",
    "    * [Attention Scoring Functions](chapter_attention-mechanisms/attention-scoring-functions.ipynb)\n",
    "    * [Bahdanau Attention](chapter_attention-mechanisms/bahdanau-attention.ipynb)\n",
    "    * [Multi-Head Attention](chapter_attention-mechanisms/multihead-attention.ipynb)\n",
    "    * [Self-Attention and Positional Encoding](chapter_attention-mechanisms/self-attention-and-positional-encoding.ipynb)\n",
    "    * [Transformer](chapter_attention-mechanisms/transformer.ipynb)\n",
    "\n",
    "\n",
    "## Part 3: Scalability, Efficiency, and Applications\n",
    "\n",
    "\n",
    "* **Chapter 11:** [Optimization Algorithms](chapter_optimization/index.ipynb)\n",
    "    * [Optimization and Deep Learning](chapter_optimization/optimization-intro.ipynb)\n",
    "    * [Convexity](chapter_optimization/convexity.ipynb)\n",
    "    * [Gradient Descent](chapter_optimization/gd.ipynb)\n",
    "    * [Stochastic Gradient Descent](chapter_optimization/sgd.ipynb)\n",
    "    * [Minibatch Stochastic Gradient Descent](chapter_optimization/minibatch-sgd.ipynb)\n",
    "    * [Momentum](chapter_optimization/momentum.ipynb)\n",
    "    * [Adagrad](chapter_optimization/adagrad.ipynb)\n",
    "    * [RMSProp](chapter_optimization/rmsprop.ipynb)\n",
    "    * [Adadelta](chapter_optimization/adadelta.ipynb)\n",
    "    * [Adam](chapter_optimization/adam.ipynb)\n",
    "    * [Learning Rate Scheduling](chapter_optimization/lr-scheduler.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 12:** [Computational Performance](chapter_computational-performance/index.ipynb)\n",
    "    * [Compilers and Interpreters](chapter_computational-performance/hybridize.ipynb)\n",
    "    * [Asynchronous Computation](chapter_computational-performance/async-computation.ipynb)\n",
    "    * [Automatic Parallelism](chapter_computational-performance/auto-parallelism.ipynb)\n",
    "    * [Hardware](chapter_computational-performance/hardware.ipynb)\n",
    "    * [Training on Multiple GPUs](chapter_computational-performance/multiple-gpus.ipynb)\n",
    "    * [Concise Implementation for Multiple GPUs](chapter_computational-performance/multiple-gpus-concise.ipynb)\n",
    "    * [Parameter Servers](chapter_computational-performance/parameterserver.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 13:** [Computer Vision](chapter_computer-vision/index.ipynb)\n",
    "    * [Image Augmentation](chapter_computer-vision/image-augmentation.ipynb)\n",
    "    * [Fine-Tuning](chapter_computer-vision/fine-tuning.ipynb)\n",
    "    * [Object Detection and Bounding Boxes](chapter_computer-vision/bounding-box.ipynb)\n",
    "    * [Anchor Boxes](chapter_computer-vision/anchor.ipynb)\n",
    "    * [Multiscale Object Detection](chapter_computer-vision/multiscale-object-detection.ipynb)\n",
    "    * [The Object Detection Dataset](chapter_computer-vision/object-detection-dataset.ipynb)\n",
    "    * [Single Shot Multibox Detection](chapter_computer-vision/ssd.ipynb)\n",
    "    * [Region-based CNNs (R-CNNs)](chapter_computer-vision/rcnn.ipynb)\n",
    "    * [Semantic Segmentation and the Dataset](chapter_computer-vision/semantic-segmentation-and-dataset.ipynb)\n",
    "    * [Transposed Convolution](chapter_computer-vision/transposed-conv.ipynb)\n",
    "    * [Fully Convolutional Networks](chapter_computer-vision/fcn.ipynb)\n",
    "    * [Neural Style Transfer](chapter_computer-vision/neural-style.ipynb)\n",
    "    * [Image Classification (CIFAR-10) on Kaggle](chapter_computer-vision/kaggle-cifar10.ipynb)\n",
    "    * [Dog Breed Identification (ImageNet Dogs) on Kaggle](chapter_computer-vision/kaggle-dog.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 14:** [Natural Language Processing: Pretraining](chapter_natural-language-processing-pretraining/index.ipynb)\n",
    "    * [Word Embedding (word2vec)](chapter_natural-language-processing-pretraining/word2vec.ipynb)\n",
    "    * [Approximate Training](chapter_natural-language-processing-pretraining/approx-training.ipynb)\n",
    "    * [The Dataset for Pretraining Word Embeddings](chapter_natural-language-processing-pretraining/word-embedding-dataset.ipynb)\n",
    "    * [Pretraining word2vec](chapter_natural-language-processing-pretraining/word2vec-pretraining.ipynb)\n",
    "    * [Word Embedding with Global Vectors (GloVe)](chapter_natural-language-processing-pretraining/glove.ipynb)\n",
    "    * [Subword Embedding](chapter_natural-language-processing-pretraining/subword-embedding.ipynb)\n",
    "    * [Word Similarity and Analogy](chapter_natural-language-processing-pretraining/similarity-analogy.ipynb)\n",
    "    * [Bidirectional Encoder Representations from Transformers (BERT)](chapter_natural-language-processing-pretraining/bert.ipynb)\n",
    "    * [The Dataset for Pretraining BERT](chapter_natural-language-processing-pretraining/bert-dataset.ipynb)\n",
    "    * [Pretraining BERT](chapter_natural-language-processing-pretraining/bert-pretraining.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter 15:** [Natural Language Processing: Applications](chapter_natural-language-processing-applications/index.ipynb)\n",
    "    * [Sentiment Analysis and the Dataset](chapter_natural-language-processing-applications/sentiment-analysis-and-dataset.ipynb)\n",
    "    * [Sentiment Analysis: Using Recurrent Neural Networks](chapter_natural-language-processing-applications/sentiment-analysis-rnn.ipynb)\n",
    "    * [Sentiment Analysis: Using Convolutional Neural Networks](chapter_natural-language-processing-applications/sentiment-analysis-cnn.ipynb)\n",
    "    * [Natural Language Inference and the Dataset](chapter_natural-language-processing-applications/natural-language-inference-and-dataset.ipynb)\n",
    "    * [Natural Language Inference: Using Attention](chapter_natural-language-processing-applications/natural-language-inference-attention.ipynb)\n",
    "    * [Fine-Tuning BERT for Sequence-Level and Token-Level Applications](chapter_natural-language-processing-applications/finetuning-bert.ipynb)\n",
    "    * [Natural Language Inference: Fine-Tuning BERT](chapter_natural-language-processing-applications/natural-language-inference-bert.ipynb)\n",
    "\n",
    "\n",
    "## Appendix\n",
    "\n",
    "\n",
    "* **Chapter Appendix:** [Mathematics for Deep Learning](chapter_appendix-mathematics-for-deep-learning/index.ipynb)\n",
    "    * [Geometry and Linear Algebraic Operations](chapter_appendix-mathematics-for-deep-learning/geometry-linear-algebraic-ops.ipynb)\n",
    "    * [Eigendecompositions](chapter_appendix-mathematics-for-deep-learning/eigendecomposition.ipynb)\n",
    "    * [Single Variable Calculus](chapter_appendix-mathematics-for-deep-learning/single-variable-calculus.ipynb)\n",
    "    * [Multivariable Calculus](chapter_appendix-mathematics-for-deep-learning/multivariable-calculus.ipynb)\n",
    "    * [Integral Calculus](chapter_appendix-mathematics-for-deep-learning/integral-calculus.ipynb)\n",
    "    * [Random Variables](chapter_appendix-mathematics-for-deep-learning/random-variables.ipynb)\n",
    "    * [Maximum Likelihood](chapter_appendix-mathematics-for-deep-learning/maximum-likelihood.ipynb)\n",
    "    * [Distributions](chapter_appendix-mathematics-for-deep-learning/distributions.ipynb)\n",
    "    * [Naive Bayes](chapter_appendix-mathematics-for-deep-learning/naive-bayes.ipynb)\n",
    "    * [Statistics](chapter_appendix-mathematics-for-deep-learning/statistics.ipynb)\n",
    "    * [Information Theory](chapter_appendix-mathematics-for-deep-learning/information-theory.ipynb)\n",
    "\n",
    "\n",
    "* **Chapter Appendix:** [Tools for Deep Learning](chapter_appendix-tools-for-deep-learning/index.ipynb)\n",
    "    * [Using Amazon SageMaker](chapter_appendix-tools-for-deep-learning/sagemaker.ipynb)\n",
    "    * [Selecting Servers and GPUs](chapter_appendix-tools-for-deep-learning/selecting-servers-gpus.ipynb)\n",
    "    * [Contributing to This Book](chapter_appendix-tools-for-deep-learning/contributing.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "conda-env-.conda-default-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}