Comments (8)
Great! Thanks! BTW, I see that you upload some codes for upsampler, however it looks like it is a simple Unet architecture with ResNet blocks. Do you plan to build the unet_upsampler with gigaGAN blocks recently?
from gigagan-pytorch.
Thanks! Yeah I agree the info in the paper is insufficient especially for the upsampler part. But my two cents on the potential architecture:
- Since in table A.2, the configuration of upsampler model has mapping network, w dimension, etc, I think definitely there is latent variable in the upsampler, similar to the base model. So the mapping network with concatenation of noise and global text embedding as input, as well as the adaptive filter with style modulation should be there. However, in the main text, it mentioned "residual blocks", so I guess it is more like a resnet block, but with conv layers replaced by adaptive kernel conv.
- Since for text-to-image upsampler, the multi-scale loss is also True, there should be multi-scale outputs (i.e., toRGB layers). It makes no sense to have toRGB layers on resolutions lower than the input, so I guess you can have toRGB layers on the last few Unet blocks where the resolution is larger or equal to the input resolution.
from gigagan-pytorch.
@XavierXiao oh yes 🤦♂️ thank you!
from gigagan-pytorch.
@XavierXiao i'm kind of confused by what they used for upsampling
they have no architectural diagram, and only said it was a traditional unet, so that's what i'm going to start off with, paired with the GigaGAN discriminator. i was going to modify the unet to also output the rgb, like the style/gigagan generator
from gigagan-pytorch.
@XavierXiao open to pointers and suggestions, if you have any insights
from gigagan-pytorch.
@XavierXiao yes what you say makes sense Xavier! will build it exactly how you said!
from gigagan-pytorch.
@XavierXiao b22ecfc let me know if we are on the same page after this commit
from gigagan-pytorch.
Great! Looks good to me! Only inconsistency to paper that I noticed is that the paper mentioned (in 3.4) the Unet is asymmetric (i.e., 3 down blocks and 6 up blocks for a 8x up-sampler), and the skip connection of the unet is, of course, only at matched resolutions of up and down blocks. Here your Unet seems to be symmetric. Not sure if it matters though.
from gigagan-pytorch.
Related Issues (20)
- The training code not deal with paired data yet? HOT 2
- [Question] About the upscaler HOT 2
- Multi GPU training HOT 4
- Multi GPU with gradient accumulation
- [Request] Please provide a replicate.com version
- Confused about this project?
- NaN losses after hours of training (UPSAMPLER) HOT 16
- How to implement this model to enhance my input images? Do I have to train the model to use? HOT 2
- Weights of Gigagan Upscaler HOT 1
- Turn on/off gradients computation between generator/discriminator HOT 2
- Wrong order of resolutions list HOT 1
- to_rgb branch has only 1 learnable kernel HOT 7
- Gradient Penalty is very high in the start HOT 10
- How to use this model for SR ?
- Has Anyone Trained This Model Yet? HOT 2
- The text-to-image tasks
- Config to reproduce paper
- question about code in unet_upsampler.py HOT 1
- the loss became nan after a few train steps HOT 2
- [News] Videogigagan is published. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gigagan-pytorch.