d2l-ai / d2l-en

Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 500 universities from 70 countries including Stanford, MIT, Harvard, and Cambridge.

D2L.ai: Interactive Deep Learning Book with Multi-Framework Code, Math, and Discussions

Book website | STAT 157 Course at UC Berkeley

The best way to understand deep learning is learning by doing.

This open-source book represents our attempt to make deep learning approachable, teaching you the concepts, the context, and the code. The entire book is drafted in Jupyter notebooks, seamlessly integrating exposition figures, math, and interactive examples with self-contained code.

Our goal is to offer a resource that could

1. be freely available for everyone;
2. offer sufficient technical depth to provide a starting point on the path to actually becoming an applied machine learning scientist;
3. include runnable code, showing readers how to solve problems in practice;
4. allow for rapid updates, both by us and also by the community at large;
5. be complemented by a forum for interactive discussion of technical details and to answer questions.

Universities Using D2L

If you find this book useful, please star (★) this repository or cite this book using the following bibtex entry:

@book{zhang2023dive,
    title={Dive into Deep Learning},
    author={Zhang, Aston and Lipton, Zachary C. and Li, Mu and Smola, Alexander J.},
    publisher={Cambridge University Press},
    note={\url{https://D2L.ai}},
    year={2023}
}

Endorsements

"In less than a decade, the AI revolution has swept from research labs to broad industries to every corner of our daily life. Dive into Deep Learning is an excellent text on deep learning and deserves attention from anyone who wants to learn why deep learning has ignited the AI revolution: the most powerful technology force of our time."

— Jensen Huang, Founder and CEO, NVIDIA

"This is a timely, fascinating book, providing with not only a comprehensive overview of deep learning principles but also detailed algorithms with hands-on programming code, and moreover, a state-of-the-art introduction to deep learning in computer vision and natural language processing. Dive into this book if you want to dive into deep learning!"

— Jiawei Han, Michael Aiken Chair Professor, University of Illinois at Urbana-Champaign

"This is a highly welcome addition to the machine learning literature, with a focus on hands-on experience implemented via the integration of Jupyter notebooks. Students of deep learning should find this invaluable to become proficient in this field."

— Bernhard Schölkopf, Director, Max Planck Institute for Intelligent Systems

"Dive into Deep Learning strikes an excellent balance between hands-on learning and in-depth explanation. I've used it in my deep learning course and recommend it to anyone who wants to develop a thorough and practical understanding of deep learning."

— Colin Raffel, Assistant Professor, University of North Carolina, Chapel Hill

Contributing (Learn How)

This open source book has benefited from pedagogical suggestions, typo corrections, and other improvements from community contributors. Your help is valuable for making the book better for everyone.

Dear D2L contributors, please email your GitHub ID and name to d2lbook.en AT gmail DOT com so your name will appear on the acknowledgments. Thanks.

License Summary

This open source book is made available under the Creative Commons Attribution-ShareAlike 4.0 International License. See LICENSE file.

The sample and reference code within this open source book is made available under a modified MIT license. See the LICENSE-SAMPLECODE file.

Chinese version | Discuss and report issues | Code of conduct

d2l-en's Issues

IPv6 support

The website lacks IPv6 support.

Since it seems to use CloudFront, fixing this should be as easy as enabling IPv6 on the CloudFront distribution and creating an AAAA ALIAS record pointing to it.

wrong equations in 10.3. Gradient Descent

In 10.3.3.2. Convergence Analysis
equations (10.3.10) is wrong, the denominator in right equations should be second order derivative

Little understanding issue in 8.1

I am a litte confused by the following paragraph

It is much harder to predict tomorrow’s stock prices than to fill in the blanks for a stock price we missed yesterday, even though both are just a matter of estimating one number. After all, hindsight is so much easier than foresight. In statistics the former is called extrapolation whereas the latter is called interpolation.

Isn't the former (hindsight) called intrapolation and the latter (foresight) called extrapolation. So, the other way around?

What infrastructure do you use to build this site?

Hey there 👋 - this is a really amazing resource! I work on an open source project called Jupyter Book that builds books similar to this one, and I am really curious what your build / website infrastructure looks like! Trying to see if I can find some pieces of inspiration in your infra :-)

What happened to the discussion website?

It seems that all the past threads in https://discuss.d2l.ai/ are gone?

Also, discussion sections of some chapters become Error Embedding in my browser. e.g https://www.d2l.ai/chapter_linear-networks/linear-regression.html

Error: module 'd2l' has no attribute 'train_ch5'

I was following the Feb 2020 version of the book, tried running alexnet.ipynb (chapter_convolutional-modern) and got this error: module 'd2l' has no attribute 'train_ch5'.
This error is from

d2l.train_ch5(net, train_iter, test_iter, num_epochs, lr)

It seems the d2l module does not include the train_ch5 function.

[BUG] NameError: name 'DATA_URL' is not defined

When I follow the example in Chapter 2.4 Calculus, the first block can not be executed.

%matplotlib inline

import d2l
from IPython import display
from mxnet import np, npx
npx.set_np()

def f(x):
    return 3 * x ** 2 - 4 * x

Which prompts the following error:

/usr/local/lib/python3.6/dist-packages/d2l/d2l.py in <module>()
    359 # Defined in file: ./chapter_multilayer-perceptrons/kaggle-house-price.md
    360 DATA_HUB['kaggle_house_train'] = (
--> 361     DATA_URL+'kaggle_house_pred_train.csv',
    362     '585e9cc93e70b39160e7921475f9bcd7d31219ce')
    363 

NameError: name 'DATA_URL' is not defined

https://github.com/d2l-ai/d2l-en/blob/master/d2l/d2l.py#L361

The DATA_URL is defined in chapter_multilayer-perceptrons/kaggle-house-price.md, but not in the d2l.py.

Why AddNorm() in the Transformer 9.3.3 (line: `self.norm(self.dropout(Y) + X)` ) is different from the Original one (line :` x + self.dropout(sublayer(self.norm(x))) ` ) ?

In The Annotated Transformer the residual connection is implemented like that : x + self.dropout(sublayer(self.norm(x)))
However, in the section 9.3.3, it is implemented in this way:
self.norm(self.dropout(Y) + X) . I think it should be X + self.norm(self.dropout(Y)) am I correct?

Add build instruction

For contributors, please describe how to build the book (html, pdf, or live server).

solutions manual

Any plans to publish solutions to the exercises?

Data leakage in Kaggle House Prices example

In 4.10.4. Data Preprocessing, normalization is applied to all_features, therefore information from the test set is being leaked to the training set.

I believe you should instead fit the normalization to the training set, and then apply the fitted transform to both sets, for example:

from sklearn.preprocessing import StandardScaler
scalar = StandardScaler()
X_train = scalar.fit_transform(X_train)
test = scalar.transform(test)

Ref: https://stackoverflow.com/questions/49444262/normalize-data-before-or-after-split-of-training-and-testing-data

I assume this may be intentional to simplify the example, but I think if that's the case a note should be made that this incurs leakage.

Typo in Figure d2l-en/chapter_attention-mechanisms/seq2seq-attention.md

The content should be context and also the outputs of decoder are coded as encoder outputs.

AutoMl

Can a chapter on AutoMl for deep learning on cell and network architecture search included on image segmentation

Undefined name: 'vocab' in mxnet.py

@astonzhang https://github.com/d2l-ai/d2l-en/blob/master/d2l/mxnet.py#L2222-L2223 on these lines vocab is an undefined name.

flake8 testing of https://github.com/d2l-ai/d2l-en on Python 3.8.3

$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./d2l/mxnet.py:2222:24: F821 undefined name 'vocab'
    premise = np.array(vocab[premise], ctx=d2l.try_gpu())
                       ^
./d2l/mxnet.py:2223:27: F821 undefined name 'vocab'
    hypothesis = np.array(vocab[hypothesis], ctx=d2l.try_gpu())
                          ^
2     F821 undefined name 'vocab'
2

https://flake8.pycqa.org/en/latest/user/error-codes.html

On the flake8 test selection, this PR does not focus on "style violations" (the majority of flake8 error codes that psf/black can autocorrect). Instead these tests are focus on runtime safety and correctness:

• F82 tests are almost always undefined names which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a NameError is raised which will halt/crash the script on the user.
• E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
• F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where a == b is True but a is b is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
• F7 tests logic errors and syntax errors in type hints

d2l setup.py should include mxnet as depency with appropriate version

After setting up a conda environment with all the required dependencies, running d2lbook build html fails with the error message:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-ccff59c9a15f> in <module>
      1 get_ipython().run_line_magic('matplotlib', 'inline')
----> 2 import d2l
      3 from IPython import display
      4 import numpy as np
      5 

~/Desktop/workspace/d2l-en/d2l/__init__.py in <module>
----> 1 from .d2l import *
      2 
      3 __version__ = '0.11.1'

~/Desktop/workspace/d2l-en/d2l/d2l.py in <module>
     12 import math
     13 from matplotlib import pyplot as plt
---> 14 from mxnet import autograd, context, gluon, image, init, np, npx
     15 from mxnet.gluon import nn, rnn
     16 import os

ModuleNotFoundError: No module named 'mxnet'
ModuleNotFoundError: No module named 'mxnet'

Attempting to solve this problem with pip install mxnet doesn't work, because d2l isn't using the stable released version of mxnet

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-ccff59c9a15f> in <module>
      1 get_ipython().run_line_magic('matplotlib', 'inline')
----> 2 import d2l
      3 from IPython import display
      4 import numpy as np
      5 

~/Desktop/workspace/d2l-en/d2l/__init__.py in <module>
----> 1 from .d2l import *
      2 
      3 __version__ = '0.11.1'

~/Desktop/workspace/d2l-en/d2l/d2l.py in <module>
     12 import math
     13 from matplotlib import pyplot as plt
---> 14 from mxnet import autograd, context, gluon, image, init, np, npx
     15 from mxnet.gluon import nn, rnn
     16 import os

ImportError: cannot import name 'np' from 'mxnet' (/Users/.../miniconda3/envs/d2lai/lib/python3.7/site-packages/mxnet/__init__.py)
ImportError: cannot import name 'np' from 'mxnet' (/Users/.../miniconda3/envs/d2lai/lib/python3.7/site-packages/mxnet/__init__.py)

I think mxnet should be added to the setup.py of d2l as a dependency. The appropriate mxnet version should be specified as well to avoid this issue in the future.

What do you think?

PS: installing pre released mxnet version solves the problem.

pip install mxnet --pre

Thanks.

error in last code block of 8.5 (RNN from scratch)?

If you run notebook 8.5 and you arrive at the last code block, a model should be trained where state is re-initialized at the start of each epoch:

train_ch8(model, train_iter, vocab, lr, num_epochs, ctx, use_random_iter=True)

However if you run the code as-is, it will take the model generated from the previous code block:

num_epochs, lr = 500, 1
train_ch8(model, train_iter, vocab, lr, num_epochs, ctx)

and continue training that one. Is this an error or was this as it was meant to be? I'm afraid I don't understand which one it should be from the text, and either is insightful (as the model output does change in a meaningful manner if you continue training the same model). If I had to choose one I'd rather compare a fresh model with re-initialized state per epoch to a fresh model without, in which case the last code block could read:

model = RNNModelScratch(len(vocab), num_hiddens, ctx, get_params,
                        init_rnn_state, rnn)
train_ch8(model, train_iter, vocab, lr, num_epochs, ctx, use_random_iter=True)

Inconsistent learning curves of two NiN trainings

When our team trained the NiN network in chapter 7 more than one time, we got different learning curves:
First time:

Second time:

Maybe there is a problem with the initialization? Should the random_seed be set or what can we do to solve the problem?

How can this be used in AWS SageMaker?

Cannot figure out how to use this code in a AWS SageMaker notebook instance.
The standard conda_mxnet_p36 kernel does not support,

from mxnet import np, npx
npx.set_np()

Get the message cannot import np

Typo in chapter Preliminaries in Probability part

Hello, I am using the book for refreshing my mind.
I have found a typo: "Say that we cast a DICE...." not a die

it has been repeated 22 times in the chapter.

Thank you for your effort for making ML free to learn.

Possible typo in 3.4.3

In 3.4.3, under Cross-Entropy Revisited, it says:

"Relating this back to our classification objective, even if we get the best possible predictions, if the best possible possible, then we will never be perfect."

I can't interpret what this means, so I think it may be a typo.

chapter_computational-performance/hybridize.ipynb has

In the mentioned notebook, there's a code snippet not deleted?

# Delete this since symbol is not supposed to be exposed directly to users?
#x = sym.np.var('data')  # FIXME
#net(x)

I suppose these should be deleted?

Incomplete sentence in section 6.1.2

The last sentence in section 6.1.2 ("Constraining the MLP") is currently incomplete.

This, in a nutshell is the convolutional layer. When the local region (also called a receptive field) is small, the difference as compared to a fully-connected network can be dramatic. While previously, we might have required billions of parameters to represent just a single layer in an image-processing network, we now typically need just a few hundred. The price that we pay for this drastic modification is that our features will be translation invariant and that our layer can only take local information into account. All learning depends on imposing inductive bias. When that bias agrees with reality, we get sample-efficient models that generalize well to unseen data. But of course, if those biases do not agree with reality, e.g. if images turned out not to be translation invariant, [the sentence ends here]

I suppose the sentence should say something like "the model will fail to generalize"

Chapter 2.2 typo

Section on norms, paragraph three, final sentence:

That may seem like a mouthful, but if you digest it then you probably have grepped the important concepts here.

I assume grepped is supposed to be grokked.

For future reference, do you prefer issues or pull requests for this sort of thing?

Correct method of making changes?

Hi, first time contributor here. I've made some changes to my local (forked) repo on the MLP chapter, and then committed these changes to my fork and pushed it.

When I opened a pull request, it seems to have confused github as I see large diffs between our versions, and it fails some of the CI tests. I just reworded a couple sentences and fixed some typos

How do I avoid this situation?

Single quote is wrongly spaced in html output

For example:

The pdf output of d2lbook build pdf works fine.

Fix graphic [d2l-en/img/vgg.svg]

The VGG network uses a MaxPooling layer with a 2x2 kernel not a 3x3 kernel.

yolo is not talked about

yolo is one important method of object detection and had better to be known in the book.

German translation

Is a german translation already in the progress somewhere?

I've started translating a few sentences of the preface to german and wondered if this is something already pursued by someone else.

[RFC] Code structure refactor

Background

Most d2l chapters follows the data/network/loss/training/prediction structure. Codes are reused by saving into the d2l package. However, APIs are not consistent and some functions have a lot of arguments. Some users are using the d2l package in their products after learning the materials. So it's good to have d2l to be a mini deep learning framework, and also improves its readability.

Design Principals

• We want object oriented, instead of a bunch of functions. But we should avoid over-designing
• The API should be framework agonistic, as we are adding PT and TF implementations
• Using Python 3 features, including f-string and typing for better readability

API design

Inspired by pytorch-lightning, we can create an API Trainer class:

class Trainer:
  def __init__(self, net, num_epochs, batch_size, ...):
    self.net = net
    #...
  def train_dataloader(self):
    raise NotImplemented()
  def optimizer(self):
    raise NotImplemented()
  def train_step(self, batch, batch_idx):
    raise NoteImplemented()
  def fit():
    optimizer = self.optimizer()
    for epoch in range(self.num_epochs):
      for batch_idx, batch in enumerate(self.train_dataloader()):
        loss = self.train_step(batch, batch_idx)
        loss.backward()
        optimizer.step()
  

Now implement a particular algorithm

class FashionMnistTrainer(d2l.Trainer):
  def train_dataloader(self):
    return DataLoader(...)
  def optimizer(self):
    return SGD()
  def train_step(self, batch, batch_idx):
    X, y = batch
    y_hat = self.net(X)
    return loss(y, y_hat)
  

Plugin a particular network

lenet = ...
trainer = FashionMnistTrainer(lenet, num_epochs=5)
trainer.fit()

Expand the class definition into multiple code cells

One main reason we didn't use class too much is that the implementation of a class is long, we want to break it into multiple code cells in jupyter.

Recently I found a way to do it:

Cell 1:

class MyClass():
  def __init__(self):
    pass

Cell 2:

class MyClass(MyClass):
  def run(self):
    return 1

Cell 3:

a = MyClass()
a.run()

Python 3 features

• Using typing

from typing import List

def my_func(a: str, b: int) -> List[str]:
    return [a, str(b)]

Later on, we can use mypy to check the type. It may be too verbose to add typing for every function, but we should enforce it for key APIs

• f-string

We rewrite "loss %.3f, name %s".format(loss, name) into f"loss {loss:.3f}, name {name}".

Open Questions

• There are tiny difference between MX/PT/TF on data loader, optimizer, .... If we can share an almost identical Trainer definition. Especially for TF 2.0, Keras already provide a high-level fit function. If or not it's necessary

• We now maintain two libraries, d2l for mxnet implementation and d2l_pytorch for pytorch. If or not we want to put them into a single library. Such as in mxnet, we save code by "Save into the d2l.mxnet package". To use it, just from d2l import mxnet as d2l

http://en.diveintodeeplearning.org/ is down

In the past, http://en.diveintodeeplearning.org/ would redirect users into https://d2l.ai/. The latter site is now down.

Number of parameters in Transformer

Hi, All

I want to count the number of trainable parameters in Transformer.
In PyTorch, there is a function (e.g. encoder.parameters()) to get the number of parameter.
But it seems that d2l.EncoderDecoder(TransformerEncoder, TransformerDecoder) doesn't provide this kind of function.
The reference model is following Transformer.
https://d2l.ai/chapter_attention-mechanisms/transformer.html

Anyone know how to count the parameter number in Transformer?

A glitch in section 2.4.11

You might notice that norms sound a lot like measures of distance. And if you remember Euclidean distances (think Pythagoras’ theorem) from grade school, then the concepts of non-negativity and the triangle inequality might ring a bell. In fact, the Euclidean distance is a norm: specifically it is the ℓ2 norm. Suppose that the elements in the 𝑛 -dimensional vector 𝐱 are 𝑥1,…,𝑥𝑛 . The :math:ell_2 norm of 𝐱 is the square root of the sum of the squares of the vector elements:

ell_2 is not displayed properly here.

ERROR : pip install git+https://github.com/d2l-ai/d2l-en

When I enter 'pip install git+https://github.com/d2l-ai/d2l-en', it shows that

"Cloning https://github.com/d2l-ai/d2l-en to /private/var/folders/42/wvdnbdv97t36hl2tbpl7qnhm0000gn/T/pip-req-build-3ifoib73 Running command git clone -q https://github.com/d2l-ai/d2l-en /private/var/folders/42/wvdnbdv97t36hl2tbpl7qnhm0000gn/T/pip-req-build-3ifoib73 error: RPC failed; curl 18 transfer closed with outstanding read data remaining fatal: the remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed ERROR: Command errored out with exit status 128: git clone -q https://github.com/d2l-ai/d2l-en /private/var/folders/42/wvdnbdv97t36hl2tbpl7qnhm0000gn/T/pip-req-build-3ifoib73 Check the logs for full command output.

what's wrong?

Wrong citation in Chapter recommender system

Hi d2l authors,

I found a minor mistake in the chapter of recommender system:
The citation link for Caser is wrong:

Caser :cite:Sedhain.Menon.Sanner.ea.2015

,which should be:

Caser :cite:Tang.Wang.2018

Thanks for providing this high-quality content. Also, as a author of the mentioned method, I really appreciate it could be introduced in this valuable book.

ssl cert isn't working for the website

https://gluon.ai gives a security warning.

Reading group via online video chat

We would take turns reading the material and discuss after every passage to help with understanding and also attempt to solve exercises together.

Undefined names: missing imports?

flake8 testing of https://github.com/d2l-ai/d2l-en on Python 3.8.0

$ flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics

./d2l/d2l.py:361:5: F821 undefined name 'DATA_URL'
    DATA_URL+'kaggle_house_pred_train.csv',
    ^
./d2l/d2l.py:515:13: F821 undefined name 'read_time_machine'
    lines = read_time_machine()
            ^
./d2l/d2l.py:761:27: F821 undefined name 'read_data_nmt'
    text = preprocess_nmt(read_data_nmt())
                          ^
./d2l/d2l.py:1481:17: F821 undefined name 'read_ptb'
    sentences = read_ptb()
                ^
./d2l/d2l.py:1590:34: F821 undefined name 'read_data_ml100k'
    data, num_users, num_items = read_data_ml100k()
                                 ^
5     F821 undefined name 'DATA_URL'
5

https://flake8.pycqa.org/en/latest/user/error-codes.html

On the flake8 test selection, this PR does not focus on "style violations" (the majority of flake8 error codes that psf/black can autocorrect). Instead these tests are focus on runtime safety and correctness:

• E9 tests are about Python syntax errors usually raised because flake8 can not build an Abstract Syntax Tree (AST). Often these issues are a sign of unused code or code that has not been ported to Python 3. These would be compile-time errors in a compiled language but in a dynamic language like Python they result in the script halting/crashing on the user.
• F63 tests are usually about the confusion between identity and equality in Python. Use ==/!= to compare str, bytes, and int literals is the classic case. These are areas where a == b is True but a is b is False (or vice versa). Python >= 3.8 will raise SyntaxWarnings on these instances.
• F7 tests logic errors and syntax errors in type hints
• F82 tests are almost always undefined names which are usually a sign of a typo, missing imports, or code that has not been ported to Python 3. These also would be compile-time errors in a compiled language but in Python a NameError is raised which will halt/crash the script on the user.

eifjccinlfvugtgvejbvubvccubnjkccbgcffejvcivd

Issue with plot titles in Section 18.6.1.6

In section 18.6.1.6. Standard Deviations the three different plots with different probabilities show the same title i.e. p>1/8.

This should be resolved to p>1/8, p=1/8, p<1/8 for the 3 plots respectively.
We need to update plt.title for this. I'll send in a PR to fix this issue.

[Question] in 4.4.2 Validation Dataset

In the section Validation Dataset it says,

"The uncertainty in our estimates can be shown to be of the order of $\mathcal{O}(n^{\frac{-1}{2}})$"

Would it be possible to expand on how this is derived or add a citation?

MXNet Nightly Builds Moved to S3

Hi,

MXNet nightly builds have been moved from PyPI to S3 (pypi/support#243). If this project requires the nightly builds for CI validation, please make sure that your CI pipeline now takes the nightly builds from https://dist.mxnet.io/python. Furthermore, as MXNet 1.7 and 2.0 are currently in progress simultaneously, we are releasing nightly builds for both 1.x and 2.x. Take mxnet-cu102 for example:

If you need the nightly builds for 1.x:

pip install --pre "mxnet-cu102<2" -f https://dist.mxnet.io/python

If you need the nightly builds for 2.x:

pip install --pre "mxnet-cu102>=2" -f https://dist.mxnet.io/python

Feel free to close this issue if the necessary changes have been made, or If the project doesn't rely on MXNet nightly builds. Thanks.

Questions about how to track the change

Hi, all d2l developers

Thanks for bringing d2l, a fantastic learning book for machine learning. While reading the book, I found myself a little lost, and I am not very used to Github, so could you guys give me some help?

I started to read the book in October, and the way I use the book is to download the PDF, thus the version I was read then was the version released by Sep 25, 2019. And recently I download the book again, I noticed the release date changed to Nov 08, 2019. Today, I download another version, it changed to Nov 10, 2019.

The major change I noticed is in the installation, it now started to use Conda to create virtual env.

I understand this is how the Open Source project works, it will change at a very fast speed.

My questions are:

1. How was the PDF generated？Is the PDF released date at the bottom right controlled by a human being or automatically?
   To be more clear, I am wondering, every time when I click "PDF" from the d2l.ai, to download the whole book. Was I downloading an already existed PDF that was uploaded there, or it is just a "generator", say, what it does is going back to the Github (or somewhere else) to find the latest version of each section, and then generate a PDF by some magic code, and finally add the release date based on the latest "merged" date (or maybe some other mechanism to determine the date)?

2. As the change is so fast, is there any way I can keep track of the major change?
   I tried to have a look at all the recent merge, but it makes me even more confused. I noticed that the latest book switched to Numpy. I assume this would be a major change, but as a rookie to Python, I have no idea what would be the consequence of this switch.
   So, I am wondering would there be any release note telling the readers why you guys made this change, and what would be affected? Like, can we still use the old NDarray (I remember that the NDarray was mentioned multiple times, as there is some difference between the NDarray from MXnet and Numpy)?

Simplify

y.grad == 4 * x is more straightforward than y.grad - 4 * x

explain: yield

in first lin reg section, plot training process, split train vs test

simplify true_w - w.reshape(true_w.shape)

Question about the table of contents

Hi,

I have a question about how to show the materials as the table of contents of the book in the Jupyter Notebook.

I saw there is an index.ipynb in most folders, I assume this is to show the materials as the order of the table of contents of the book. So I am wondering how can we take advantage of index.ipynb to sort other things? As now all the files can only be sorted by time, name, or size.

Thanks in advance.

Broken links

Seems like there is some issue with internal cross references:
The hash id link errors are due to a bug in the muffet tool.

$ go/bin/muffet -t 60 -e "discuss.mxnet.io|localhost" -c 4 https://d2l.ai/chapter_multilayer-perceptrons/numerical-stability-and-init.html
https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html
	id #id1 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id1
	id #id11 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id11
	id #id13 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id13
	id #id15 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id15
	id #id17 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id17
	id #id19 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id19
	id #id21 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id21
	id #id23 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id23
	id #id25 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id25
	id #id27 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id27
	id #id29 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id29
	id #id3 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id3
	id #id5 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id5
	id #id7 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id7
	id #id9 not found	https://d2l.ai/chapter_appendix-tools-for-deep-learning/d2l.html#id9
https://d2l.ai/index.html
	id #contribute-learn-how not found	https://github.com/d2l-ai/d2l-en#contribute-learn-how
https://d2l.ai/chapter_computational-performance/hardware.html
	404	https://static.googleusercontent.com/media/research.google.com/en//people/jeff/Stanford-DL-Nov-2010.pdf
https://d2l.ai/chapter_optimization/sgd.html
	x509: certificate signed by unknown authority	https://www.stat.cmu.edu/~ryantibs/convexopt-F15/lectures/26-nonconvex.pdf

kindly update the python code in chapter_preliminaries/ndarray.md for (Python 3.7.4+ and mxnet 1.5.0+)

following code in chapter_preliminaries/ndarray.md is not working in
Python 3.7.4 (win64) and mxnet 1.5.0 (mxnet-1.5.0-py2.py3-none-win_amd64.whl )

from mxnet import np, npx
npx.set_np()

---

i installed mxnet in python virtual-environment using

python3 -m venv venv 
venv\Scripts\activate.bat
pip install mxnet

which installed following python packages:

Collecting mxnet                                                                                                                      
  Using cached https://files.pythonhosted.org/packages/a4/3e/07f6d61d8e0a5d50c6f128791a617d7b222224c52dacba04855a0b6ecfe6/mxnet-1.5.0-
py2.py3-none-win_amd64.whl                                                                                                            
Collecting numpy<1.17.0,>=1.8.2 (from mxnet)                                                                                          
  Using cached https://files.pythonhosted.org/packages/03/25/d525fd3da596a4564497e1323d3e3c63c02bd911cdbd53dc180f15aae009/numpy-1.16.6
-cp37-cp37m-win_amd64.whl                                                                                                             
Collecting graphviz<0.9.0,>=0.8.1 (from mxnet)                                                                                        
  Using cached https://files.pythonhosted.org/packages/53/39/4ab213673844e0c004bed8a0781a0721a3f6bb23eb8854ee75c236428892/graphviz-0.8
.4-py2.py3-none-any.whl                                                                                                               
Collecting requests<2.19.0,>=2.18.4 (from mxnet)                                                                                      
  Using cached https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.1
8.4-py2.py3-none-any.whl                                                                                                              
Collecting idna<2.7,>=2.5 (from requests<2.19.0,>=2.18.4->mxnet)                                                                      
  Using cached https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2
.py3-none-any.whl                                                                                                                     
Collecting urllib3<1.23,>=1.21.1 (from requests<2.19.0,>=2.18.4->mxnet)                                                               
  Using cached https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22
-py2.py3-none-any.whl                                                                                                                 
Collecting chardet<3.1.0,>=3.0.2 (from requests<2.19.0,>=2.18.4->mxnet)                                                               
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.
4-py2.py3-none-any.whl                                                                                                                
Collecting certifi>=2017.4.17 (from requests<2.19.0,>=2.18.4->mxnet)                                                                  
  Using cached https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019
.11.28-py2.py3-none-any.whl                                                                                                           
Installing collected packages: numpy, graphviz, idna, urllib3, chardet, certifi, requests, mxnet                                      
Successfully installed certifi-2019.11.28 chardet-3.0.4 graphviz-0.8.4 idna-2.6 mxnet-1.5.0 numpy-1.16.6 requests-2.18.4 urllib3-1.22

i'm getting following error

from mxnet import np,npx
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'np' from 'mxnet' (venv\lib\site-packages\mxnet\__init__.py)

[Website] Table of contents incorrectly displayed

https://d2l.ai/chapter_computer-vision/image-augmentation.html

https://d2l.ai/chapter_computer-vision/rcnn.html

As you can see from the images, table of contents gets incorrectly aligned at the right side on Firefox browser
Confirmed same issue exists on Chrome & Safari as well.

d2l.py installation issue

I have installed d2l-0.11.2 (the latest version so far) using pip as it was suggested in Installation. When I attempted to run d2l.Vocab from Machine Translation and the Dataset, I got

TypeError: __init__() got an unexpected keyword argument 'reserved_tokens'

I turned to the source code of d2l package installed in my conda environment and compared it with github repository of d2l. While both __init__.py files look the same, there are significant differences between d2l.py files. Specifically, d2l.Vocab in my local environment is not updated and it still accepts an old use_special_tokens parameter. So the modifications from 431305a were not committed. Also, many other newer objects (e.g. from d235bf1) are not included.
I tried to uninstall the previous version of d2l and reinstall the newest one, but it did not help. I had no issues with previous versions. Is there something wrong with repository/package configurations?

How to start a new translation?

I would like to start translating d2l-ai book to Arabic. I see that you have a repository per language, could you create one for Arabic? If not what's the procedure? Thanks.

chapter_computational-performance/multiple-gpus-gluon.md failure

Upon running on latest master

61f01e3c (HEAD -> master, upstream/master, origin/master, origin/HEAD) add chapter author photos

Command

d2l-en bapac$ d2lbook build html

It fails with IndexError: list index out of range

Stack Trace

[d2lbook:build.py:L118] INFO   [31/165, 00:09:15] Evaluating ./chapter_computational-performance/multiple-gpus-gluon.md, save as _build/eval/chapter_computational-performance/multiple-gpus-gluon.ipynb
[d2lbook:execute.py:L404] INFO   Executing notebook with kernel: python
Traceback (most recent call last):
  File "/Users/bapac/anaconda3/bin/d2lbook", line 8, in <module>
    sys.exit(main())
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/d2lbook/main.py", line 22, in main
    commands[args.command[0]]()
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/d2lbook/build.py", line 35, in build
    getattr(builder, cmd)()
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/d2lbook/build.py", line 45, in warp
    func(self)
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/d2lbook/build.py", line 243, in html
    self.rst()
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/d2lbook/build.py", line 45, in warp
    func(self)
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/d2lbook/build.py", line 222, in rst
    self.eval()
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/d2lbook/build.py", line 45, in warp
    func(self)
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/d2lbook/build.py", line 122, in eval
    default_tab=self.config.default_tab)
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/d2lbook/build.py", line 385, in process_and_eval_notebook
    notedown.run(nb, timeout)
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/notedown/notedown.py", line 48, in run
    notebook, resources = executor.preprocess(notebook, resources={})
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 405, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "/Users/bapac/anaconda3/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 448, in preprocess_cell
    raise CellExecutionError.from_cell_and_msg(cell, out)
nbconvert.preprocessors.execute.CellExecutionError: An error occurred while executing the following cell:
------------------
x = np.random.uniform(size=(4, 1, 28, 28))
x_shards = gluon.utils.split_and_load(x, ctx)
net(x_shards[0]), net(x_shards[1])
------------------

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-4-823c783a14ce> in <module>
      1 x = np.random.uniform(size=(4, 1, 28, 28))
      2 x_shards = gluon.utils.split_and_load(x, ctx)
----> 3 net(x_shards[0]), net(x_shards[1])

IndexError: list index out of range
IndexError: list index out of range