fastai / course22p2 Goto Github PK

In the notebook 03_backprop lin_grad function for calculating gradient of the linear layer, gradient calculation for w.g = (inp.unsqueeze(-1) * out.g.unsqueeze(1)).sum(0) seems to way slower than dot production version w.g = inp.t() @ out.g

Time complexity wise, both seem to have the same: O(m x n x p).

Is the performance gain due to the way both are implemented?

Note: This is not an actual issue, but it confused me a lot because of the huge performance difference in both the implementation; that's the reason I raised this as an issue.

Using astats while traning model is consuming memory.

• In the above picture i was traning a resnet model using this callback as the model sees more mini-batches it is comsuming more ram

I'm playing around with using miniai on different types of tasks. Currently, a siamese network (same/not same) with two input images (and will probably be trying out object detection next which will require multiple y values).

I'm using TrainCB, which allows you to specify n_inp=2. But no support for n_inp > 1 in MetricsCB. I'd be happy to do a pull request but wanted to bounce a few ideas around.

Option 1) just push n_inp into learner and MetricsCB could be modified to honor that
Option 2) add get_x and get_y methods to Learner; TrainCB could patch those (before_fit) for n_inp>1; and MetricsCB would just do *learn.get_y() to evaluate loss

option 1 is probably a reasonable quick workaround for now.
option 2 is a few more changes to the core learner, but probably better?

Thoughts?
Thanks
John

In notebook 21, the WandBCB assumes the presence of a train attribute in _log, but this is not present.
A possible fix would be to replace it with d["train"] == "train".

If what I said is correct and you agree with the solution, I can do a PR to fix the bug.

Hi, I try to install this repo using the command:
pip install course22p2

but I got the error:
ERROR: Could not find a version that satisfies the requirement course22p2 (from versions: none)
ERROR: No matching distribution found for course22p2

I am using mac and tried to install it in a conda env.

Hi,

I was trying to use plot from the RecorderCB in miniai/sgd.py, but it gave me plt is not defined error.
Can anyone fix this issue please? I think it just needs matplotlib.pyplot as plt in one of the exported block in the file course22p2/nbs/12_accel_sgd.ipynb.

Thank you.

Hi my lovely peoples,
i was just wondering whether there is a specific date for the release of the MOOC already in sight? I am waiting patiently.
Best,
Finn

Hi,

I was trying to reimplement the course material for my use case which is 1D as homework, and I had a bug running the fit function when using the PyTorch dataloaders, and I noticed that our fit function implemented right before the sampling chapter uses a variable called preds in the report function, however the predictions are stored in pred during the loop, and this lead to me having inconsistent dimensions, removing the s worked out.

Cheers

In resnet.py there are (more than) two import *:

from .conv import *
from .init import *

Both contain a conv def. It took me a moment to figure out where the one that was actually used was (given the name I would have said in conv), maybe it would be helpful a refactoring or to avoid the "import *"? I think this is a classic case where import * can lead to unexpected behavior, what do you think?