uwsampl / pytorch Goto Github PK

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

Expected behavior

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

• PyTorch Version (e.g., 1.0):
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

Additional context

🚀 Feature

Motivation

Pitch

Alternatives

Additional context

🚀 Feature

Motivation

Pitch

Alternatives

Additional context

import torch
from torch import tensor

a = torch.randn(4, 4).checkpoint()
a = torch.tensor([-1, -2, 3], dtype=torch.int8).checkpoint()
torch.bitwise_and(a, a)

ultimately this is due to bitwise_and initiate a tensor and call mutate on it internally.

two ways to fix this
0: I can allow checkpoint_function to take in raw tensor and mutate them. this I can do quite easily but is unsafe - while I can reflect change on cptensor back to raw tensor, I cant do the reverse without modifying the old tensors.
1: I can change bitwise_and. the obvious problem is that this is non-extendable. But why can bitwise_and initialize tensor? it must know whether the input is sparse/dense and initalize likewise. we can find this hopefully generic code and add to it.

I will go with route 1 for now.

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

Expected behavior

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

• PyTorch Version (e.g., 1.0):
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

Additional context

get a vector of cptc for clear_checkpointpool, instead of reusing pool.
tensors in pool maybe incomplete and out of order due to eviction, so getting a seprate pool will fix this.

my system friend tell me to use https://github.com/oneapi-src/oneTBB
we can also roll our own producer-consumer.

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

Expected behavior

Environment

Please copy and paste the output from our
environment collection script
(or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

• PyTorch Version (e.g., 1.0):
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

Additional context

🚀 Feature

Motivation

Pitch

Alternatives

Additional context

Reformer
NAS inference
http://pyro.ai/examples/dmm.html
https://github.com/facebookresearch/clevr-iep
this will bring our total model count into 12, which we can layout 3*4 manner in the paper.
However, I am also thinking of Meta Learning with:
CNN
LSTM.
maybe we should replace a normal CNN/LSTM example with those.
MAML itself use CNN.
https://github.com/twitter/meta-learning-lstm
https://github.com/cflamant/neural-stack
Adaptive Computation Time
https://github.com/kentsommer/pytorch-value-iteration-networks/blob/master/model.py
https://github.com/ikostrikov/pytorch-meta-optimizer/blob/master/meta_optimizer.py