fastai / course-v3

The 3rd edition of course.fast.ai

Welcome to fastai

Installing

You can use fastai without any installation by using Google Colab. In fact, every page of this documentation is also available as an interactive notebook - click “Open in colab” at the top of any page to open it (be sure to change the Colab runtime to “GPU” to have it run fast!) See the fast.ai documentation on Using Colab for more information.

You can install fastai on your own machines with conda (highly recommended), as long as you’re running Linux or Windows (NB: Mac is not supported). For Windows, please see the “Running on Windows” for important notes.

We recommend using miniconda (or miniforge). First install PyTorch using the conda line shown here, and then run:

conda install -c fastai fastai

To install with pip, use: pip install fastai.

If you plan to develop fastai yourself, or want to be on the cutting edge, you can use an editable install (if you do this, you should also use an editable install of fastcore to go with it.) First install PyTorch, and then:

git clone https://github.com/fastai/fastai
pip install -e "fastai[dev]"

Learning fastai

The best way to get started with fastai (and deep learning) is to read the book, and complete the free course.

To see what’s possible with fastai, take a look at the Quick Start, which shows how to use around 5 lines of code to build an image classifier, an image segmentation model, a text sentiment model, a recommendation system, and a tabular model. For each of the applications, the code is much the same.

Read through the Tutorials to learn how to train your own models on your own datasets. Use the navigation sidebar to look through the fastai documentation. Every class, function, and method is documented here.

To learn about the design and motivation of the library, read the peer reviewed paper.

About fastai

fastai is a deep learning library which provides practitioners with high-level components that can quickly and easily provide state-of-the-art results in standard deep learning domains, and provides researchers with low-level components that can be mixed and matched to build new approaches. It aims to do both things without substantial compromises in ease of use, flexibility, or performance. This is possible thanks to a carefully layered architecture, which expresses common underlying patterns of many deep learning and data processing techniques in terms of decoupled abstractions. These abstractions can be expressed concisely and clearly by leveraging the dynamism of the underlying Python language and the flexibility of the PyTorch library. fastai includes:

• A new type dispatch system for Python along with a semantic type hierarchy for tensors
• A GPU-optimized computer vision library which can be extended in pure Python
• An optimizer which refactors out the common functionality of modern optimizers into two basic pieces, allowing optimization algorithms to be implemented in 4–5 lines of code
• A novel 2-way callback system that can access any part of the data, model, or optimizer and change it at any point during training
• A new data block API
• And much more…

fastai is organized around two main design goals: to be approachable and rapidly productive, while also being deeply hackable and configurable. It is built on top of a hierarchy of lower-level APIs which provide composable building blocks. This way, a user wanting to rewrite part of the high-level API or add particular behavior to suit their needs does not have to learn how to use the lowest level.

Migrating from other libraries

It’s very easy to migrate from plain PyTorch, Ignite, or any other PyTorch-based library, or even to use fastai in conjunction with other libraries. Generally, you’ll be able to use all your existing data processing code, but will be able to reduce the amount of code you require for training, and more easily take advantage of modern best practices. Here are migration guides from some popular libraries to help you on your way:

• Plain PyTorch
• Ignite
• Lightning
• Catalyst

Windows Support

Due to python multiprocessing issues on Jupyter and Windows, num_workers of Dataloader is reset to 0 automatically to avoid Jupyter hanging. This makes tasks such as computer vision in Jupyter on Windows many times slower than on Linux. This limitation doesn’t exist if you use fastai from a script.

See this example to fully leverage the fastai API on Windows.

We recommend using Windows Subsystem for Linux (WSL) instead – if you do that, you can use the regular Linux installation approach, and you won’t have any issues with num_workers.

Tests

To run the tests in parallel, launch:

nbdev_test

For all the tests to pass, you’ll need to install the dependencies specified as part of dev_requirements in settings.ini

pip install -e .[dev]

Tests are written using nbdev, for example see the documentation for test_eq.

Contributing

After you clone this repository, make sure you have run nbdev_install_hooks in your terminal. This install Jupyter and git hooks to automatically clean, trust, and fix merge conflicts in notebooks.

After making changes in the repo, you should run nbdev_prepare and make additional and necessary changes in order to pass all the tests.

Docker Containers

For those interested in official docker containers for this project, they can be found here.

course-v3's Issues

initialisation of collab_learner object on lesson4-collab notebook, use_nn=True section is broken

Can be fixed as
learn = collab_learner(data, use_nn=True, layers=[10,10])

[guide] GCE preemptive instance deployment

This weekend, I just spend some time with GCE and trying to make some great fastai dev environment.
That's with preemptive instances.

This is a set of scripts which I tested many times for different cases.
May be this is useful for others setup a powerful fast.ai node for a very low price.

Guide: https://arunoda.me/blog/ideal-way-to-creare-a-fastai-node

Some features of this:

• create a node with the Tesla v100 GPU for just $0.8/hour
• create a node with the Tesla K80 GPU for just $0.2/hour
• create a node with No GPU for just $0.01/hour
• switch between these nodes whenever needed
• install new tools and save data (won't get deleted when switching)
• run notebooks by just starting the server (no SSH needed)
• create a password protected jupyter notebooks environment

ImageItemList gives wrong type target tensors

This is somewhat related with:

fastai/fastai#839

and:

https://forums.fast.ai/t/error-pytorch-expected-object-of-scalar-type-long-but-got-scalar-type-float-for-argument-2-other/33778/3

Complete debug message:

RuntimeError                            Traceback (most recent call last)
<ipython-input-82-47df9df23a95> in <module>
----> 1 learn.fit_one_cycle(5)

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/train.py in fit_one_cycle(learn, cyc_len, max_lr, moms, div_factor, pct_start, wd, callbacks, **kwargs)
     20     callbacks.append(OneCycleScheduler(learn, max_lr, moms=moms, div_factor=div_factor,
     21                                         pct_start=pct_start, **kwargs))
---> 22     learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks)
     23 
     24 def lr_find(learn:Learner, start_lr:Floats=1e-7, end_lr:Floats=10, num_it:int=100, stop_div:bool=True, **kwargs:Any):

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
    170         callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks)
    171         fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,
--> 172             callbacks=self.callbacks+callbacks)
    173 
    174     def create_opt(self, lr:Floats, wd:Floats=0.)->None:

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
     92     except Exception as e:
     93         exception = e
---> 94         raise e
     95     finally: cb_handler.on_train_end(exception)
     96 

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
     87             if not data.empty_val:
     88                 val_loss = validate(model, data.valid_dl, loss_func=loss_func,
---> 89                                        cb_handler=cb_handler, pbar=pbar)
     90             else: val_loss=None
     91             if cb_handler.on_epoch_end(val_loss): break

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/basic_train.py in validate(model, dl, loss_func, cb_handler, pbar, average, n_batch)
     52             if not is_listy(yb): yb = [yb]
     53             nums.append(yb[0].shape[0])
---> 54             if cb_handler and cb_handler.on_batch_end(val_losses[-1]): break
     55             if n_batch and (len(nums)>=n_batch): break
     56         nums = np.array(nums, dtype=np.float32)

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/callback.py in on_batch_end(self, loss)
    237         "Handle end of processing one batch with `loss`."
    238         self.state_dict['last_loss'] = loss
--> 239         stop = np.any(self('batch_end', not self.state_dict['train']))
    240         if self.state_dict['train']:
    241             self.state_dict['iteration'] += 1

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/callback.py in __call__(self, cb_name, call_mets, **kwargs)
    185     def __call__(self, cb_name, call_mets=True, **kwargs)->None:
    186         "Call through to all of the `CallbakHandler` functions."
--> 187         if call_mets: [getattr(met, f'on_{cb_name}')(**self.state_dict, **kwargs) for met in self.metrics]
    188         return [getattr(cb, f'on_{cb_name}')(**self.state_dict, **kwargs) for cb in self.callbacks]
    189 

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/callback.py in <listcomp>(.0)
    185     def __call__(self, cb_name, call_mets=True, **kwargs)->None:
    186         "Call through to all of the `CallbakHandler` functions."
--> 187         if call_mets: [getattr(met, f'on_{cb_name}')(**self.state_dict, **kwargs) for met in self.metrics]
    188         return [getattr(cb, f'on_{cb_name}')(**self.state_dict, **kwargs) for cb in self.callbacks]
    189 

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/callback.py in on_batch_end(self, last_output, last_target, **kwargs)
    272         if not is_listy(last_target): last_target=[last_target]
    273         self.count += last_target[0].size(0)
--> 274         self.val += last_target[0].size(0) * self.func(last_output, *last_target).detach().cpu()
    275 
    276     def on_epoch_end(self, **kwargs):

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/metrics.py in error_rate(input, targs)
     45 def error_rate(input:Tensor, targs:Tensor)->Rank0Tensor:
     46     "1 - `accuracy`"
---> 47     return 1 - accuracy(input, targs)
     48 
     49 

~/anaconda3/envs/dmx1/lib/python3.7/site-packages/fastai/metrics.py in accuracy(input, targs)
     28     input = input.argmax(dim=-1).view(n,-1)
     29     targs = targs.view(n,-1)
---> 30     return (input==targs).float().mean()
     31 
     32 

RuntimeError: Expected object of scalar type Long but got scalar type Float for argument #2 'other'

How I acquired the data, created my Learner:

src = (ImageItemList.from_csv(path, 'stuff.csv', folder='.')
       .random_split_by_pct(0.2)
       .label_from_df(cols=colz))

data = (src.transform(tfms, size=299)
        .databunch().normalize(imagenet_stats)
       )

learn = create_cnn(data, models.resnet18, metrics=error_rate, ps=0.70, loss_func=nn.BCEWithLogitsLoss())

The error pops up right after the first epoch, just as the first validation cycle starts.
Note that I implemented the suggestion provided by S. Gugger in issue #839

Implement image pre-check method

If there is a corrupt image in the train, validation or test set PIL.Image which is used by pytorch dataloader will raise an exception when it encounters the image. For example, if there is a zero byte image.

It may be worth implementing a pre-check method that checks validity of images before training.

Happy to implement it if you think it is a worthwhile idea.

Gradient Notebook Container -- No longer available

In the setup page for the first deep learning course, the recommended base container in the docs is no longer available, and there are 2 possible fast.ai containers.

Recommended in the docs:

Fast.ai 1.0 / PyTorch 1.0 BETA

Available as of January 27, 2019

Paperspace + Fast.AI 1.0 (V3)
Paperspace + Fast.AI 0.7.x

I'm assuming the 1.0(v3) is the right one, but I wanted to ask before I started another machine.

What's the correct one?

Lesson 3 7zip: wrong package suggested

The 7zip package that is suggested to install via apt (sudo apt install p7zip) doesn't actually include the command used in the subsequent cell, 7za). For this to work, the suggested/installed package should be p7zip-full.

Zeit deployment can be deleted

Zeit doesn't allow new accounts to use v1 of their cloud anymore. This means that docker builds are not supported anymore. And the python builder that they have does not work with fastai/pytorch right now. So I guess we have remove the Zeit deployment article for now.

Source: https://spectrum.chat/?t=d206a0d4-5835-4af5-bb5c-a17f0171fb25

download-images.ipynb calls 'models' directory before it is created

A recent update to download-images includes the line:
(path/'models').ls()

This is in the second block under View Data. But the models directory isn't created until ConvLearner is instantiated in the Train Model section, which happens after View Data. As it is, calling ls() on models/ throws an error.

Issues with lesson 1 data.show_batch

data.show_batch(rows=3, figsize=(7,6))

RuntimeError Traceback (most recent call last)
in ()
----> 1 data.show_batch(rows=3, figsize=(7,6))

C:\ProgramData\Anaconda3\lib\site-packages\fastai\basic_data.py in show_batch(self, rows, ds_type, **kwargs)
158 def show_batch(self, rows:int=5, ds_type:DatasetType=DatasetType.Train, **kwargs)->None:
159 "Show a batch of data in ds_type on a few rows."
--> 160 x,y = self.one_batch(ds_type, True, True)
161 if self.train_ds.x._square_show: rows = rows ** 2
162 xs = [self.train_ds.x.reconstruct(grab_idx(x, i)) for i in range(rows)]

C:\ProgramData\Anaconda3\lib\site-packages\fastai\basic_data.py in one_batch(self, ds_type, detach, denorm, cpu)
141 w = self.num_workers
142 self.num_workers = 0
--> 143 try: x,y = next(iter(dl))
144 finally: self.num_workers = w
145 if detach: x,y = to_detach(x,cpu=cpu),to_detach(y,cpu=cpu)

C:\ProgramData\Anaconda3\lib\site-packages\fastai\basic_data.py in iter(self)
68 def iter(self):
69 "Process and returns items from DataLoader."
---> 70 for b in self.dl:
71 #y = b[1][0] if is_listy(b[1]) else b[1] # XXX: Why is this line here?
72 yield self.proc_batch(b)

C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in next(self)
635 self.reorder_dict[idx] = batch
636 continue
--> 637 return self._process_next_batch(batch)
638
639 next = next # Python 2 compatibility

C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in _process_next_batch(self, batch)
656 self._put_indices()
657 if isinstance(batch, ExceptionWrapper):
--> 658 raise batch.exc_type(batch.exc_msg)
659 return batch
660

RuntimeError: Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 138, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 138, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\data_block.py", line 526, in getitem
x = x.apply_tfms(self.tfms, **self.tfmargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\vision\image.py", line 113, in apply_tfms
else: x = tfm(x)
File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\vision\image.py", line 499, in call
return self.tfm(x, *args, **{**self.resolved, **kwargs}) if self.do_run else x
File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\vision\image.py", line 446, in call
if args: return self.calc(*args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\vision\image.py", line 451, in calc
if self._wrap: return getattr(x, self._wrap)(self.func, *args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\vision\image.py", line 167, in coord
self.flow = func(self.flow, *args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\vision\transform.py", line 238, in _symmetric_warp
return _do_perspective_warp(c, targ_pts, invert)
File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\vision\transform.py", line 225, in _do_perspective_warp
return _apply_perspective(c, _find_coeffs(_orig_pts, targ_pts))
File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\vision\transform.py", line 206, in _find_coeffs
return torch.gesv(B,A)[0][:,0]
RuntimeError: b should have at least 2 dimensions, but has 1 dimensions instead

Colab setup out of date

https://github.com/fastai/course-v3/blob/master/docs/setup/colab#L4

You no longer need to install pytorch nightly since pytorch is now on the 1.0 branch. In fact, when you install fastai later on it re-installs pytorch over this install.

issue with lesson 1 image processing (or maybe just display)

When running nbs/dl1/lesson1-pets.ipynb (on Google Cloud Compute, default setup as documented at https://course.fast.ai/start_gcp.html , only think I installed as extra Jupyter notebook extensions https://github.com/ipython-contrib/jupyter_contrib_nbextensions and pyformat required by it), on executing cell with code:

interp.plot_top_losses(9, figsize=(15,11))

I see "burned up" images displayed in my notebook:

It's weird because accuracies and all don't seem affected, though I imagine there's something wrong with image preprocessing maybe.

For reference, I'm running image pytorch-latest-gpu on n1-highmem-8 with GPU nvidia-tesla-p4 in region europe-west4-b - in case there's some WTF GPU/driver bug someone may know smth about or anything esoteric like that.

Ref: commit 6b8e518237a0446e53251bd63ab4e644f4618bbf

Question: Any way to develop on my side, push to github then run it in GPU cloud?

I saw that for the course there are many options for GPU cloud processing.

Which of them supports live updates from GitHub? Crestle, Floyd, Gradient, Collab?
Price is not an issue.

I want to dev in Pycharm, run my notebooks locally, push to github, and only train on the cloud to benefit of GPU acceleration.

lesson2 - defaults.device = torch.device('cpu')

Following lesson2, I export a model trained in GPU (I don't need to export data):
learn.export()

Then I upload export.pkl in my laptop (it doesn't have GPU):

defaults.device = torch.device('cpu')
l = load_learner(path=Path('./'))

I get:
RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False.

Instead everything is ok if I move model to CPU before exporting it:

learn.model = learn.model.cpu()
learn.export()

Then I load export.pkl in my laptop (no need to specify defaults.device ):
l = load_learner(path=Path('./'))

Size mismatch when loading `fit_head` in `lesson3-imdb`

Hi !
I am using Google Cloud Platform for fastai v3 courses.
I am running 'lesson3-imdb.ipynb' notebook.
Error of size mismatch when loading 'fit_head'.
Here is the command

learn.load('fit_head');

I have executed till the following command

learn.recorder.plot(skip_end=15)

after that I am trying to load the fit_head

Lesson 1: Compiling `pat` Regular Expression

In the Jupyter notebook "lesson1-pets.ipynb", there is the line:

pat = re.compile(r'/([^/]+)_\d+.jpg$')

This is then passed to the function "from_name_re". The type signature of this function is:

@classmethod
def from_name_re(cls, path:PathOrStr, fnames:FilePathList, pat:str, valid_pct:float=0.2, **kwargs):

The first command in from_name_re recompiles pat.

pat = re.compile(pat)

Technically, objects of type Pattern seem not to be an instance of type str.

I understand that having the code as shown brings particular attention to it being a regular expression. It may create confusion around types. Totally understand if you close this.

"learn = ConvLearner(...)" missing in lesson1 notebook

The notebook lacks this line (or something like this?) and thus does not work out of the box...

learn = ConvLearner(data, models.resnet34, metrics=error_rate)

Might have accidentally been deleted in one of the last commits?

sagemaker setup

Maybe it will be good to mention about mounting efs disk possibility instead of adding 50Gb to every notebook instance. Can be useful for share datasets etc. between all aws instances. https://aws.amazon.com/blogs/machine-learning/mount-an-efs-file-system-to-an-amazon-sagemaker-notebook-with-lifecycle-configurations/

Runtime error with pytorch nightly

I followed the installation instructions for AWS EC2 but on my local machine. I installed pytorch using conda install -c pytorch pytorch-nightly cuda92. After everything was installed I fired up the first notebook but got a runtime error when running the cell that shows images from a batch: data.show_batch(rows=3, figsize=(7,6)): "RuntimeError: B should have at least 2 dimensions, but has 1 dimensions instead"

I fixed it by uninstalling pytorch nightly and just installed the latest release of pytorch instead. I ran into this issue when doing an install on the 26th of January, so it could very well work now. I would suggest changing the installation instructions for AWS EC2 to not use pytorch nightly as that could break functionality.

Lesson 3 IMDB example : KeyError: 'unsup'

data_clas = (TextList.from_folder(path, vocab=data_lm.vocab)
#grab all the text files in path
.split_by_folder(valid='test')
#split by train and valid folder (that only keeps 'train' and 'test' so no need to filter)
.label_from_folder(classes=['neg', 'pos'])
.filter_missing_y() #the error is the same even comments this line
#label them all with their folders
.databunch(bs=bs))

data_clas.save('tmp_clas')

KeyError Traceback (most recent call last)
/usr/anaconda/envs/pytorch10/lib/python3.7/site-packages/fastai/data_block.py in process_one(self, item)
277 def process_one(self,item):
--> 278 try: return self.c2i[item] if item is not None else None
279 except:

KeyError: 'unsup'

...
exception: Your validation data contains a label that isn't present in the training set, please fix your data.

lesson3-camvid.ipynb

size = src_size//2

should be changed to

size = (src_size//4).tolist()

Attaching the screenshot:

Related discussion on the forum:
https://forums.fast.ai/t/error-in-lesson-3-camvid-with-colab/35329/9?u=vettukal

Lesson 1 - What's your pet

when i run lesson1 in colab,
data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224, bs=bs).normalize(imagenet_stats)

when it comes to this line,the colab output following errors.
`AssertionError Traceback (most recent call last)
in ()
----> 1 data = ImageDataBunch.from_name_re(path_img, fnames, pat, ds_tfms=get_transforms(), size=224, bs=bs).normalize(imagenet_stats)

/usr/local/lib/python3.6/dist-packages/fastai/vision/data.py in from_name_re(cls, path, fnames, pat, valid_pct, **kwargs)
155 assert res,f'Failed to find "{pat}" in "{fn}"'
156 return res.group(1)
--> 157 return cls.from_name_func(path, fnames, _get_label, valid_pct=valid_pct, **kwargs)
158
159 @staticmethod

/usr/local/lib/python3.6/dist-packages/fastai/vision/data.py in from_name_func(cls, path, fnames, label_func, valid_pct, **kwargs)
143 def from_name_func(cls, path:PathOrStr, fnames:FilePathList, label_func:Callable, valid_pct:float=0.2, **kwargs):
144 "Create from list of fnames in path with label_func."
--> 145 src = ImageList(fnames, path=path).random_split_by_pct(valid_pct)
146 return cls.create_from_ll(src.label_from_func(label_func), **kwargs)
147

/usr/local/lib/python3.6/dist-packages/fastai/data_block.py in random_split_by_pct(self, valid_pct, seed)
194 if valid_pct==0.: return self.no_split()
195 if seed is not None: np.random.seed(seed)
--> 196 rand_idx = np.random.permutation(range_of(self))
197 cut = int(valid_pct * len(self))
198 return self.split_by_idx(rand_idx[:cut])

mtrand.pyx in mtrand.RandomState.permutation()

mtrand.pyx in mtrand.RandomState.shuffle()

/usr/local/lib/python3.6/dist-packages/numpy/core/_internal.py in init(self, array, ptr)
294 # get a void pointer to the buffer, which keeps the array alive
295 self._data = _get_void_ptr(array)
--> 296 assert self._data.value == ptr
297 else:
298 # fake a pointer-like object that holds onto the reference

AssertionError: `

What's the version of fastai in these notebooks?

I tried 1.0.39,1.0.34 and 1.0.27, but none of them could match all the code in notebooks, can anybody tell me the right version?

Official dockerfile for the course

Hi!
Is there a need for an official dockerfile for the course? I've already created one on the forum, but maybe it's a good idea to have it in the repo?
Obviously, I'm willing to help with that.

lesson1-pets.ipynb now gives 'expecting number but found string'

Lesson 1 worked fine for me last week, when the state of the file was commit 7346e87 "Merge pull request #14 from miwojc/correct-regex-link" .

Now, if one grabs a completely fresh clone of the entire course-v3 repo, then the Lesson 1 notebook will produce the error (in Jupyter):

"Notebook failed to load
The error was: [sprintf] expecting number but found string
See the error console for details."

(and the error console just directs you to a line 3004 in notebook.js which says the same thing).

Going down through the history of commits for this file, I find that versions prior to commit ceaca31cab5e2909ac102718216f271cf6203688 - "nbs are now stripped out (outputs preserved)" by @stas00 do not produce this error, but from that commit onward (and including it), all of the versions of lesson1-pets.ipynb produce that same error -- including the most recent commit 8ff043f - "camvid".

AWS Sagemaker CloudFormation: Error Template validation error: Internal Failure

As the title says, the template does not launch. Tried in any of the provided regions.

The template, however, works when manually loaded through s3 link

Solution to out of memory error on Google Colab (using more num_workers on Google Colab)

In Google Colab doc (https://github.com/fastai/course-v3/blob/master/docs/google_colab_tutorial.md), out of memory error (using num_workers=0) is mentioned. I think I have found a workaround/solution to this problem. It is mentioned in detail here - https://towardsdatascience.com/ok-google-how-to-do-speech-recognition-f77b5d7cbe0b in the section "Setup tweak to reduce training time". The solution is explained here - https://github.com/jayeshsaita/Speech-Commands-Recognition/blob/master/training.ipynb in the section "Getting more juice from Google Colab". I was able to use 32 to 64 num_workers (batch size of 64 images) in Google Colab with this trick. Can someone verify if this solution is correct? If it is correct, how can I add this solution to the fastai documentation?

Namespace issue: "data"

the notebooks use "data" as a variable name for storing databunches.
If after setting up the variable the initial import cell is run again (from fastai.vision import *)
the data reference gets "overwritten". data now points to the "fastai.vision.data" module. This leads to unexpected behaviour/errors. Running the import cell again is quite common I think (adding additional imports, reloading after library update...)

This could be solved in several ways on the imports side (preferably) or by renaming all "data" variables to e.g. "databunch", "datab", "db" or similar. But as the variable name "data" is also used in all docs maybe there is a better solution?

CUDA error: all CUDA-capable devices are busy or unavailable

_When I run the following command there is an error that is coming. I am unable to resolve as there is no ready made solution
Note in Cell 11 I executed this command

data.show_batch(rows=3, figsize=(7,6))

---

RuntimeError Traceback (most recent call last)
in
----> 1 data.show_batch(rows=3, figsize=(7,6))

~/anaconda3/lib/python3.7/site-packages/fastai/basic_data.py in show_batch(self, rows, ds_type, **kwargs)
157 def show_batch(self, rows:int=5, ds_type:DatasetType=DatasetType.Train, **kwargs)->None:
158 "Show a batch of data in ds_type on a few rows."
--> 159 x,y = self.one_batch(ds_type, True, True)
160 if self.train_ds.x._square_show: rows = rows ** 2
161 xs = [self.train_ds.x.reconstruct(grab_idx(x, i)) for i in range(rows)]

~/anaconda3/lib/python3.7/site-packages/fastai/basic_data.py in one_batch(self, ds_type, detach, denorm, cpu)
140 w = self.num_workers
141 self.num_workers = 0
--> 142 try: x,y = next(iter(dl))
143 finally: self.num_workers = w
144 if detach: x,y = to_detach(x,cpu=cpu),to_detach(y,cpu=cpu)

~/anaconda3/lib/python3.7/site-packages/fastai/basic_data.py in iter(self)
69 def iter(self):
70 "Process and returns items from DataLoader."
---> 71 for b in self.dl: yield self.proc_batch(b)
72
73 @classmethod

~/anaconda3/lib/python3.7/site-packages/fastai/basic_dat--------------------------------+----------------------+----------------------a.py in proc_batch(self, b)
63 def proc_batch(self,b:Tensor)->Tensor:
64 "Proces batch b of TensorImage."
---> 65 b = to_device(b, self.device)
66 for f in listify(self.tfms): b = f(b)
67 return b

~/anaconda3/lib/python3.7/site-packages/fastai/torch_core.py in to_device(b, device)
103 "Recursively put b on device."
104 device = ifnone(device, defaults.device)
--> 105 if is_listy(b): return [to_device(o, device) for o in b]
106 return b.to(device)
107
data.show_batch(rows=3, figsize=(7,6))
~/anaconda3/lib/python3.7/site-packages/fastai/torch_core.py in (.0)
103 "Recursively put b on device."
104 device = ifnone(device, defaults.device)
--> 105 if is_listy(b): return [to_device(o, device) for o in b]
106 return b.to(device)
107

~/anaconda3/lib/python3.7/site-packages/fastai/torch_core.py in to_device(b, device)
104 device = ifnone(device, defaults.device)
105 if is_listy(b): return [to_device(o, device) for o in b]
--> 106 return b.to(device)
107
108 def data_collate(batch:ItemsList)->Tensor:

RuntimeError: CUDA error: all CUDA-capable devices are busy or unavailable
My output for nividia-smi is as under
NVIDIA-SMI 384.130 Driver Version: 384.130--------------------------------+----------------------+---------------------- |
|------------------------------------------------------+---------------------------------------+--------------------------------------+
| GPU Name Persistence -M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Quadro P4000 Off | 00000000:73:00.0 On | N/A |
| 46% 38C P8 7W / 105W | 492MiB / 8105MiB | 0% Default |
+------------------------------------------------------+---------------------------------------+-------------------------------------+

+-------------------------------------------------------------------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1637 G /usr/lib/xorg/Xorg 217MiB |
| 0 3036 G compiz 25MiB |
| 0 5315 G ...quest-channel-token=8285522418909966745 62MiB |
| 0 16078 C /home/anaconda3/bin/python 155MiB |
| 0 18196 G /home/anaconda3/bin/python 26MiB |

Bug report: lesson3-imdb.ipynb NameError: name ‘AWD_LSTM’ is not defined

Hi!

I believe learn = language_model_learner(data_lm, AWD_LSTM, drop_mult=0.3) should be learn = language_model_learner(data_lm, "AWD_LSTM", drop_mult=0.3). With parentheses around AWD_LSTM.

This fix the problem.

lesson-1 DataLoader worker (pid 8362) is killed by signal: Illegal instruction.

When I try to run lesson-1 jupyter-notebook, I have a problem in runing learn.fit_one_cycle(4)
The detailed debug info is as follows:

RuntimeError Traceback (most recent call last)
in
----> 1 learn.fit_one_cycle(4)

~/anaconda3/envs/step/lib/python3.6/site-packages/fastai/train.py in fit_one_cycle(learn, cyc_len, max_lr, moms, div_factor, pct_start, wd, callbacks, **kwargs)
20 callbacks.append(OneCycleScheduler(learn, max_lr, moms=moms, div_factor=div_factor,
21 pct_start=pct_start, **kwargs))
---> 22 learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks)
23
24 def lr_find(learn:Learner, start_lr:Floats=1e-7, end_lr:Floats=10, num_it:int=100, stop_div:bool=True, **kwargs:Any):

~/anaconda3/envs/step/lib/python3.6/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
170 callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks)
171 fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,
--> 172 callbacks=self.callbacks+callbacks)
173
174 def create_opt(self, lr:Floats, wd:Floats=0.)->None:

~/anaconda3/envs/step/lib/python3.6/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
92 except Exception as e:
93 exception = e
---> 94 raise e
95 finally: cb_handler.on_train_end(exception)
96

~/anaconda3/envs/step/lib/python3.6/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
80 cb_handler.on_epoch_begin()
81
---> 82 for xb,yb in progress_bar(data.train_dl, parent=pbar):
83 xb, yb = cb_handler.on_batch_begin(xb, yb)
84 loss = loss_batch(model, xb, yb, loss_func, opt, cb_handler)

~/anaconda3/envs/step/lib/python3.6/site-packages/fastprogress/fastprogress.py in iter(self)
63 self.update(0)
64 try:
---> 65 for i,o in enumerate(self._gen):
66 yield o
67 if self.auto_update: self.update(i+1)

~/anaconda3/envs/step/lib/python3.6/site-packages/fastai/basic_data.py in iter(self)
68 def iter(self):
69 "Process and returns items from DataLoader."
---> 70 for b in self.dl: yield self.proc_batch(b)
71
72 @classmethod

~/anaconda3/envs/step/lib/python3.6/site-packages/torch/utils/data/dataloader.py in next(self)
629 while True:
630 assert (not self.shutdown and self.batches_outstanding > 0)
--> 631 idx, batch = self._get_batch()
632 self.batches_outstanding -= 1
633 if idx != self.rcvd_idx:

~/anaconda3/envs/step/lib/python3.6/site-packages/torch/utils/data/dataloader.py in _get_batch(self)
608 # need to call .task_done() because we don't use .join().
609 else:
--> 610 return self.data_queue.get()
611
612 def next(self):

~/anaconda3/envs/step/lib/python3.6/multiprocessing/queues.py in get(self, block, timeout)
92 if block and timeout is None:
93 with self._rlock:
---> 94 res = self._recv_bytes()
95 self._sem.release()
96 else:

~/anaconda3/envs/step/lib/python3.6/multiprocessing/connection.py in recv_bytes(self, maxlength)
214 if maxlength is not None and maxlength < 0:
215 raise ValueError("negative maxlength")
--> 216 buf = self._recv_bytes(maxlength)
217 if buf is None:
218 self._bad_message_length()

~/anaconda3/envs/step/lib/python3.6/multiprocessing/connection.py in _recv_bytes(self, maxsize)
405
406 def _recv_bytes(self, maxsize=None):
--> 407 buf = self._recv(4)
408 size, = struct.unpack("!i", buf.getvalue())
409 if maxsize is not None and size > maxsize:

~/anaconda3/envs/step/lib/python3.6/multiprocessing/connection.py in _recv(self, size, read)
377 remaining = size
378 while remaining > 0:
--> 379 chunk = read(handle, remaining)
380 n = len(chunk)
381 if n == 0:

~/anaconda3/envs/step/lib/python3.6/site-packages/torch/utils/data/dataloader.py in handler(signum, frame)
272 # This following call uses waitid with WNOHANG from C side. Therefore,
273 # Python can still get and update the process status successfully.
--> 274 _error_if_any_worker_fails()
275 if previous_handler is not None:
276 previous_handler(signum, frame)

RuntimeError: DataLoader worker (pid 8362) is killed by signal: Illegal instruction.

Google Cloud Platform-->lesson3-imdb.ipynb: Error: "untar_data is not defined"

Hi !
I am running fast.ai course v3 on GCP(Google Cloud Platform) after following the instruction from here:
[Server setup Google Cloud] (https://course-v3.fast.ai/start_gcp.html).
When I am executing the following line path = untar_data(URLs.IMDB_SAMPLE) , I am getting following error:
NameError: name 'untar_data' is not defined

Lesson 2, ImageCleaner(ds, idxs, path) does not display the expected window

I instead get an output like,
HBox(children=(VBox(children=(Image(value=b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x01\x00d\x00d\x00\x00\xff…
Button(button_style='primary', description='Next Batch', layout=Layout(width='auto'), style=ButtonStyle())

NameError: name 'TextLMDataBunch' is not defined

My list of environments is
Package Version

---

backcall 0.1.0
Bottleneck 1.2.1
certifi 2018.11.29
chardet 3.0.4
cycler 0.10.0
cymem 2.0.2
cytoolz 0.9.0.1
dataclasses 0.6
decorator 4.3.0
dill 0.2.8.2
fastai 1.0.0
fastprogress 0.1.18
idna 2.8
ipython 7.2.0
ipython-genutils 0.2.0
jedi 0.13.2
kiwisolver 1.0.1
matplotlib 3.0.2
msgpack 0.5.6
msgpack-numpy 0.4.3.2
murmurhash 1.0.1
numexpr 2.6.9
numpy 1.15.4
nvidia-ml-py3 7.352.0
packaging 18.0
pandas 0.23.4
parso 0.3.1
pexpect 4.6.0
pickleshare 0.7.5
Pillow 5.4.1
pip 18.1
plac 0.9.6
preshed 2.0.1
prompt-toolkit 2.0.7
ptyprocess 0.6.0
Pygments 2.3.1
pyparsing 2.3.0
python-dateutil 2.7.5
pytz 2018.9
PyYAML 3.13
regex 2018.1.10
requests 2.21.0
scipy 1.2.0
setuptools 40.6.3
six 1.12.0
spacy 2.0.16
thinc 6.12.0
toolz 0.9.0
torch 1.0.0
torch-nightly 1.0.0.dev20190104
torchvision 0.2.1
torchvision-nightly 0.2.1
tqdm 4.29.0
traitlets 4.3.2
typing 3.6.6
ujson 1.35
urllib3 1.24.1
wcwidth 0.1.7
wheel 0.32.3
wrapt 1.10.11

error:
NameError: name 'TextLMDataBunch' is not defined

lesson 7 superres cuda error

When trying to run Lesson 7 superres

learn.lr_find

in the train section gives:

RuntimeErrorTraceback (most recent call last)
<ipython-input-20-d81c6bd29d71> in <module>()
----> 1 learn.lr_find()

/usr/local/lib/python3.6/site-packages/fastai/train.py in lr_find(learn, start_lr, end_lr, num_it, stop_div, **kwargs)
     29     cb = LRFinder(learn, start_lr, end_lr, num_it, stop_div)
     30     a = int(np.ceil(num_it/len(learn.data.train_dl)))
---> 31     learn.fit(a, start_lr, callbacks=[cb], **kwargs)
     32 
     33 def to_fp16(learn:Learner, loss_scale:float=512., flat_master:bool=False)->Learner:

/usr/local/lib/python3.6/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
    164         callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks)
    165         fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,
--> 166             callbacks=self.callbacks+callbacks)
    167 
    168     def create_opt(self, lr:Floats, wd:Floats=0.)->None:

/usr/local/lib/python3.6/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
     92     except Exception as e:
     93         exception = e
---> 94         raise e
     95     finally: cb_handler.on_train_end(exception)
     96 

/usr/local/lib/python3.6/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
     82             for xb,yb in progress_bar(data.train_dl, parent=pbar):
     83                 xb, yb = cb_handler.on_batch_begin(xb, yb)
---> 84                 loss = loss_batch(model, xb, yb, loss_func, opt, cb_handler)
     85                 if cb_handler.on_batch_end(loss): break
     86 

/usr/local/lib/python3.6/site-packages/fastai/basic_train.py in loss_batch(model, xb, yb, loss_func, opt, cb_handler)
     16     if not is_listy(xb): xb = [xb]
     17     if not is_listy(yb): yb = [yb]
---> 18     out = model(*xb)
     19     out = cb_handler.on_loss_begin(out)
     20 

/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    487             result = self._slow_forward(*input, **kwargs)
    488         else:
--> 489             result = self.forward(*input, **kwargs)
    490         for hook in self._forward_hooks.values():
    491             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/site-packages/fastai/layers.py in forward(self, x)
    113         for l in self.layers:
    114             res.orig = x
--> 115             nres = l(res)
    116             # We have to remove res.orig to avoid hanging refs and therefore memory leaks
    117             res.orig = None

/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    487             result = self._slow_forward(*input, **kwargs)
    488         else:
--> 489             result = self.forward(*input, **kwargs)
    490         for hook in self._forward_hooks.values():
    491             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/site-packages/torch/nn/modules/container.py in forward(self, input)
     90     def forward(self, input):
     91         for module in self._modules.values():
---> 92             input = module(input)
     93         return input
     94 

/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    487             result = self._slow_forward(*input, **kwargs)
    488         else:
--> 489             result = self.forward(*input, **kwargs)
    490         for hook in self._forward_hooks.values():
    491             hook_result = hook(self, input, result)

/usr/local/lib/python3.6/site-packages/torch/nn/modules/conv.py in forward(self, input)
    318     def forward(self, input):
    319         return F.conv2d(input, self.weight, self.bias, self.stride,
--> 320                         self.padding, self.dilation, self.groups)
    321 
    322 

RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:405

this is with 1.0.37 has anyone else gotten this error?

Lesson Titles along with lesson number for Lessons Panel

In course-v3/fastai-video-browser/src/components/LessonsPanel.jsx
Do you think it would be good idea to also give lesson name like it has been done for Notes Panel,
'Lesson 1 : Image Classification'
instead of
'Lesson 1'

ImageImageList

planet notebook - kaggle download

The note book should also include the following comments:

To make the kaggle download work you have to :
-go to the kaggle competition and click “Late submission” and confirm by an sms code
-go to the datapage and accept the rule (the dialog box

You shoud add missing string with import fastai

from fastai.text import *
is not enough.

Thou should also add:
from fastai import *

Documentation for setting up own server

I have a computer that has a GPU and I managed to install everything mostly following the AWS EC2 guide.

Some people may already have decent rigs and it might be good to have a page to help guide people who want to do this? Perhaps some warnings on what to expect when running models e.g. if you have < 10GB VRAM make your batch sizes smaller.

Also some suggestions on minimum hardware requirements?

Idea: Make branches with adjustments for different python notebook services

I started working with fast.ai course - but to make it through on colab (which I still thing is the easiest to start) I have to do a lot of adjustments.

It would be cool if you created branch colab where we could propose pull request to fix these issues.
I think everyone going through the course there will have to do such adjustments so it would be nice if you accepted them upstream and suggested fastai/course-vs colab branch instead of master in https://course.fast.ai/start_colab.html

Install with Mac instructions for Google Cloud start_gcp.md

Thank you for the install guides!

for start_gcp.md

In Step2: Install Google CLI

The page I was referred to in the guide got me lost in python version land and I wasn't able to install from it.

To get to work on a Mac I found it easiest to follow Google's instructions at https://cloud.google.com/sdk/docs/downloads-interactive.

      curl https://sdk.cloud.google.com | bash
      exec -l $SHELL

Could I submit a pull request to change?

notebook 1 needs ``from fastai import *`` for untar_data

It got removed in 615585c

Windows: pytorch 1.0 lesson1-pets

'1.0.35.dev0'
When running lesson1-pets

c:\users\gerar\fastai\fastai\vision\data.py in _get_label(fn)
152 "Create from list of fnames in path with re expression pat."
153 pat = re.compile(pat)
--> 154 def _get_label(fn): return pat.search(str(fn)).group(1)
155 return cls.from_name_func(path, fnames, _get_label, valid_pct=valid_pct, **kwargs)
156

AttributeError: 'NoneType' object has no attribute 'group'

lesson2-camvid.ipynb experiences cuda runtime error (59)

At

lr_find(learn)

cuda runtime error (59) : device-side assert triggered at /opt/conda/conda-bld/pytorch-nightly_1540500315858/work/aten/src/THC/generic/THCTensorCopy.cpp:20

Such an error does not occur for other notebooks like examples/vision.ipynb

Leads me to believe it is not pytorch-nightly issue

Lesson 1: Regular Expression ImageDataBunch Error

I'm not sure if anyone else has had this issue but while getting the labels from the image files running ImageDataBunch.from_name_re()

I was getting:

AttributeError: 'NoneType' object has no attribute 'group'

I'm running on Windows 10 Pro and this seems to occur when iterating through the Windows Path objects in 'fnames' .

The solution I found is to alter the re to:

pat = r'\\.+\\([^/]+)_\d+.jpg$'

It then matches the classes correctly and runs.

Just wanted to post in case others had the same error. Feel free to close.

Should we always reload packages in colab

I am starting to use Google colab for fastai Part 1 as instructed here. As per my understanding for every new notebook, we have to install all the packages again every time I load the notebook? Can we make colab to somehow refer to any docker like image instead of reinstalling every time?

Import missing in lesson2-planet. add this to fix: "from fastai.datasets import Config"

A small error in lesson2-planet:

The Config class is not imported. Fixed by adding

from fastai.datasets import Config

Lesson 3 IMDB example

With this line of code:

data_clas = (TextList.from_folder(path, vocab=data_lm.vocab)
             #grab all the text files in path
             .split_by_folder(valid='test')
             #split by train and valid folder (that only keeps 'train' and 'test' so no need to filter)
             .label_from_folder(classes=['neg', 'pos'])
             #remove docs with labels not in above list (i.e. 'unsup')
             .filter_missing_y()
             #label them all with their folders
             .databunch(bs=bs))
data_clas.save('tmp_clas')

I get:

TypeErrorTraceback (most recent call last)
<ipython-input-36-ef1d6c6e4867> in <module>()
      3              .split_by_folder(valid='test')
      4              #split by train and valid folder (that only keeps 'train' and 'test' so no need to filter)
----> 5              .label_from_folder(classes=['neg', 'pos'])
      6              #remove docs with labels not in above list (i.e. 'unsup')
      7              .filter_missing_y()

TypeError: 'bool' object is not callable

I have checked in the path: PosixPath('/root/.fastai/data/imdb') and in the test folder, neg and pos do exist?

Lesson 3 - Unable to untar IMDB urls

I am unable to untar the IMDB urls on a Kaggle kernal

• URLs.IMDB_SAMPLE : http://files.fast.ai/data/examples/imdb_sample
• URLs.IMDB : https://s3.amazonaws.com/fast-ai-nlp/imdb

from fastai.text import *
imdb = untar_data(URLs.IMDB)

Results in
ConnectionError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /fast-ai-nlp/imdb.tgz (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f82298ee5c0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

It seems that the links are broken. Or is there something that I am missing?

"Search Transcript" hides the play and volume button on "FastAI Video Viewer"