mikonvergence / diffusionfastforward

DiffusionFastForward: a free course and experimental framework for diffusion-based generative models

My name is Mikolaj Czerkawski, I am a Research Fellow at the European Space Agency.

My research interests involve computer vision, signal processing, and machine learning.

A recurring theme in my works is the context of learning in data-limited settings. The topics I tend to deal with include:

• 🍱 Multi-Modal Learning
• 🎨 Generative Models
• 🖼 Image Synthesis and Manipulation
• 🔬 Image Super-Resolution
• 🌆➡🌃Image-to-Image Translation
• 🔎 Model Robustness Assessment
• 🛰 Computer Vision for Remote Sensing Applications
• 🔊 Computer Vision for Radar Signal Processing

My research involves applying computer vision techniques to real-world applications where (i) the datasets are small or (ii) high risk of poor generalization exists. So far, this has primarily been done with short-range radar data and with satellite imagery.

diffusionfastforward's Issues

TypeError: 'NoneType' object is not iterable

Thanks for the tutorial.

I am getting this error in this 02-Pixel-Diffusion.ipynb when i am running this trainer.fit(model)

TypeError: 'NoneType' object is not iterable
During handling of the above exception, another exception occurred:
.
.
.
TypeError: An invalid dataloader was returned from `PixelDiffusion.val_dataloader()`. Found None.

missing notes/ files

Hi, the notes files linked in the "Notes" section of the README.md don't appear to exist (except for 01-Diffusion-Theory.md. Am I missing something or these files just haven't been created yet?

For example this one is missing:
https://github.com/mikonvergence/DiffusionFastForward/blob/master/notes/02-Pixel-Diffusion.md

Thanks for creating this awesome youtube series and notebooks!

error running colab examples

Hello, thanks for the DIffusionFastForward course and colab examples.
I'm getting the following error running 02-Pixel-Diffusion-colab as well as 03-Conditional-Pixel-Diffusion. Would you mind taking a look and giving me some pointers?

trainer.fit(model)

INFO:pytorch_lightning.accelerators.cuda:LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
INFO:pytorch_lightning.callbacks.model_summary:
| Name | Type | Params

0 | model | DenoisingDiffusionProcess | 56.6 M

56.6 M Trainable params
0 Non-trainable params
56.6 M Total params
226.463 Total estimated model params size (MB)
Sanity Checking:
0/? [00:00<?, ?it/s]

TypeError Traceback (most recent call last)
/usr/local/lib/python3.9/dist-packages/pytorch_lightning/trainer/connectors/data_connector.py in _check_dataloader_iterable(dataloader, source, trainer_fn)
382 try:
--> 383 iter(dataloader) # type: ignore[call-overload]
384 except TypeError:

TypeError: 'NoneType' object is not iterable

During handling of the above exception, another exception occurred:

TypeError Traceback (most recent call last)
10 frames
/usr/local/lib/python3.9/dist-packages/pytorch_lightning/trainer/connectors/data_connector.py in _check_dataloader_iterable(dataloader, source, trainer_fn)
397 f" def {source.name}(self): in your LightningModule/LightningDataModule."
398 )
--> 399 raise TypeError(
400 f"An invalid dataloader was returned from {type(source.instance).__name__}.{source.name}()."
401 f" Found {dataloader}."

TypeError: An invalid dataloader was returned from PixelDiffusion.val_dataloader(). Found None.

Multiple images

Hi! Thank you so much for providing the code. The video course is amazing, really helpful.

I have a question: is it possible to modify the code so that we can use more than one image as input and condition (target) data? In other words, can we do, e.g., 2images-to-3images, taking 2 images to predict 3 images.

Thanks again!

some latex formula in 01-Diffusion-Theory.md is broken

broken link

from

a question about LatentDiffusionConditional

Hey, thanks for the great tutorials!

Just a question in this file: where is LatentDiffusionConditional from? It is called in the cell, but I don't see it is imported or implemented in the notebook. Thanks!

FR: table with all significant symbols defined

E.g.