GithubHelp home page GithubHelp logo

fastai / course22p2 Goto Github PK

View Code? Open in Web Editor NEW
429.0 429.0 220.0 101.63 MB

course.fast.ai 2022 part 2

Home Page: https://course.fast.ai/Lessons/part2.html

License: Apache License 2.0

Jupyter Notebook 99.76% CSS 0.01% Python 0.24%

course22p2's Introduction

Welcome to fastai

CI PyPI Conda (channel only) docs

Installing

You can use fastai without any installation by using Google Colab. In fact, every page of this documentation is also available as an interactive notebook - click “Open in colab” at the top of any page to open it (be sure to change the Colab runtime to “GPU” to have it run fast!) See the fast.ai documentation on Using Colab for more information.

You can install fastai on your own machines with conda (highly recommended), as long as you’re running Linux or Windows (NB: Mac is not supported). For Windows, please see the “Running on Windows” for important notes.

We recommend using miniconda (or miniforge). First install PyTorch using the conda line shown here, and then run:

conda install -c fastai fastai

To install with pip, use: pip install fastai.

If you plan to develop fastai yourself, or want to be on the cutting edge, you can use an editable install (if you do this, you should also use an editable install of fastcore to go with it.) First install PyTorch, and then:

git clone https://github.com/fastai/fastai
pip install -e "fastai[dev]"

Learning fastai

The best way to get started with fastai (and deep learning) is to read the book, and complete the free course.

To see what’s possible with fastai, take a look at the Quick Start, which shows how to use around 5 lines of code to build an image classifier, an image segmentation model, a text sentiment model, a recommendation system, and a tabular model. For each of the applications, the code is much the same.

Read through the Tutorials to learn how to train your own models on your own datasets. Use the navigation sidebar to look through the fastai documentation. Every class, function, and method is documented here.

To learn about the design and motivation of the library, read the peer reviewed paper.

About fastai

fastai is a deep learning library which provides practitioners with high-level components that can quickly and easily provide state-of-the-art results in standard deep learning domains, and provides researchers with low-level components that can be mixed and matched to build new approaches. It aims to do both things without substantial compromises in ease of use, flexibility, or performance. This is possible thanks to a carefully layered architecture, which expresses common underlying patterns of many deep learning and data processing techniques in terms of decoupled abstractions. These abstractions can be expressed concisely and clearly by leveraging the dynamism of the underlying Python language and the flexibility of the PyTorch library. fastai includes:

  • A new type dispatch system for Python along with a semantic type hierarchy for tensors
  • A GPU-optimized computer vision library which can be extended in pure Python
  • An optimizer which refactors out the common functionality of modern optimizers into two basic pieces, allowing optimization algorithms to be implemented in 4–5 lines of code
  • A novel 2-way callback system that can access any part of the data, model, or optimizer and change it at any point during training
  • A new data block API
  • And much more…

fastai is organized around two main design goals: to be approachable and rapidly productive, while also being deeply hackable and configurable. It is built on top of a hierarchy of lower-level APIs which provide composable building blocks. This way, a user wanting to rewrite part of the high-level API or add particular behavior to suit their needs does not have to learn how to use the lowest level.

Layered API

Migrating from other libraries

It’s very easy to migrate from plain PyTorch, Ignite, or any other PyTorch-based library, or even to use fastai in conjunction with other libraries. Generally, you’ll be able to use all your existing data processing code, but will be able to reduce the amount of code you require for training, and more easily take advantage of modern best practices. Here are migration guides from some popular libraries to help you on your way:

Windows Support

Due to python multiprocessing issues on Jupyter and Windows, num_workers of Dataloader is reset to 0 automatically to avoid Jupyter hanging. This makes tasks such as computer vision in Jupyter on Windows many times slower than on Linux. This limitation doesn’t exist if you use fastai from a script.

See this example to fully leverage the fastai API on Windows.

We recommend using Windows Subsystem for Linux (WSL) instead – if you do that, you can use the regular Linux installation approach, and you won’t have any issues with num_workers.

Tests

To run the tests in parallel, launch:

nbdev_test

For all the tests to pass, you’ll need to install the dependencies specified as part of dev_requirements in settings.ini

pip install -e .[dev]

Tests are written using nbdev, for example see the documentation for test_eq.

Contributing

After you clone this repository, make sure you have run nbdev_install_hooks in your terminal. This install Jupyter and git hooks to automatically clean, trust, and fix merge conflicts in notebooks.

After making changes in the repo, you should run nbdev_prepare and make additional and necessary changes in order to pass all the tests.

Docker Containers

For those interested in official docker containers for this project, they can be found here.

course22p2's People

Contributors

ekami avatar faisito avatar johnowhitaker avatar jph00 avatar pcuenca avatar seem avatar tmabraham avatar williamsk91 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

course22p2's Issues

Performance difference in lin_grad function (elementwise multiplication vs. dot product)

In the notebook 03_backprop lin_grad function for calculating gradient of the linear layer, gradient calculation for w.g = (inp.unsqueeze(-1) * out.g.unsqueeze(1)).sum(0) seems to way slower than dot production version w.g = inp.t() @ out.g

Time complexity wise, both seem to have the same: O(m x n x p).

Is the performance gain due to the way both are implemented?

Note: This is not an actual issue, but it confused me a lot because of the huge performance difference in both the implementation; that's the reason I raised this as an issue.

image
image

Release date in sight?

Hi my lovely peoples,
i was just wondering whether there is a specific date for the release of the MOOC already in sight? I am waiting patiently.
Best,
Finn

Missing train attribute in WandBCB

In notebook 21, the WandBCB assumes the presence of a train attribute in _log, but this is not present.
A possible fix would be to replace it with d["train"] == "train".

If what I said is correct and you agree with the solution, I can do a PR to fix the bug.

Notebook 4 - one s too many in the fit function before the random sampling section

Hi,

I was trying to reimplement the course material for my use case which is 1D as homework, and I had a bug running the fit function when using the PyTorch dataloaders, and I noticed that our fit function implemented right before the sampling chapter uses a variable called preds in the report function, however the predictions are stored in pred during the loop, and this lead to me having inconsistent dimensions, removing the s worked out.

Cheers

cannot install

Hi, I try to install this repo using the command:
pip install course22p2

but I got the error:
ERROR: Could not find a version that satisfies the requirement course22p2 (from versions: none)
ERROR: No matching distribution found for course22p2

I am using mac and tried to install it in a conda env.

Using Activationstats consuming Ram (A lot)

Using astats while traning model is consuming memory.

Screenshot from 2024-03-26 12-37-05

  • In the above picture i was traning a resnet model using this callback as the model sees more mini-batches it is comsuming more ram

RecorderCB plt not defined error

Hi,

I was trying to use plot from the RecorderCB in miniai/sgd.py, but it gave me plt is not defined error.
Can anyone fix this issue please? I think it just needs matplotlib.pyplot as plt in one of the exported block in the file course22p2/nbs/12_accel_sgd.ipynb.

Thank you.

MetricsCB assumes n_inp=1

I'm playing around with using miniai on different types of tasks. Currently, a siamese network (same/not same) with two input images (and will probably be trying out object detection next which will require multiple y values).

I'm using TrainCB, which allows you to specify n_inp=2. But no support for n_inp > 1 in MetricsCB. I'd be happy to do a pull request but wanted to bounce a few ideas around.

Option 1) just push n_inp into learner and MetricsCB could be modified to honor that
Option 2) add get_x and get_y methods to Learner; TrainCB could patch those (before_fit) for n_inp>1; and MetricsCB would just do *learn.get_y() to evaluate loss

option 1 is probably a reasonable quick workaround for now.
option 2 is a few more changes to the core learner, but probably better?

Thoughts?
Thanks
John

Two definitions of conv imported in resnet.py

In resnet.py there are (more than) two import *:

from .conv import *
from .init import *

Both contain a conv def. It took me a moment to figure out where the one that was actually used was (given the name I would have said in conv), maybe it would be helpful a refactoring or to avoid the "import *"? I think this is a classic case where import * can lead to unexpected behavior, what do you think?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.