taited / sgdiff Goto Github PK

Thanks for the great work! When will the dataset be available?

Thank you for an outstanding job！

When do you plan to release the training code? In particular, does the perceptual loss in the paper use StableDiffusionPipeline to obtain the generated image after each noise estimation?

Looking forward for your reply.

Hello, I am very interested in your work. When can you release training code?

i use this picture

and use Vincent van Gogh’s Starry Night as text prompt
the result is not as good as expected

The SCA moudle adopt semantic features form clip and text features form clip, the Q is only from text, K and V are added by text and image. And your picture in the paper shows that the Q, K, V have different shape, so how to do Q@K? Is this is your SCA code?

def forward(self, img, text_emb): if self.clip_norm: img = (img + 1) / 2 img = F.batch_norm(img, self.mean.to(self.device), self.std.to(self.device)) img = F.interpolate(img, (224, 224)) image_features = self.model(img) if self.last_layer_proj: image_features = torch.einsum('bld,ds->bls', image_features, self.model.proj) if self.cross_attn is not None: emb_features = self.cross_attn( text_emb.permute(0, 2, 1), image_features.permute(0, 2, 1)).permute(0, 2, 1) if self.skip_module is not None: if self.learned_length: residual = self.skip_module(emb_features.permute(0, 2, 1)) residual = residual.permute(0, 2, 1) else: residual = self.skip_module(emb_features) text_emb += residual return text_emb