zyinghua / uncond-image-generation-ldm Goto Github PK

i trained a model by following the instructions provided but when i try to run the inference code i get an error saying that there are keys missing

ValueError: Cannot load <class 'diffusers.models.unets.unet_2d.UNet2DModel'> from /home/david/testing/uncond-image-generation-ldm/output/vae because the following keys are missing:
down_blocks.2.resnets.0.norm1.bias, up_blocks.2.resnets.2.time_emb_proj.bias, up_blocks.2.resnets.2.norm1.bias, down_blocks.1.resnets.1.norm2.weight, mid_block.resnets.0.time_emb_proj.weight, down_blocks.2.resnets.0.norm1.weight, up_blocks.1.resnets.2.norm2.weight, up_blocks.1.resnets.1.conv1.bias, down_blocks.2.resnets.1.conv1.weight, down_blocks.1.resnets.1.conv1.weight, mid_block.attentions.0.to_q.weight, down_blocks.0.resnets.0.norm2.bias, up_blocks.2.resnets.0.norm2.bias, up_blocks.0.resnets.1.norm2.bias, up_blocks.0.resnets.2.conv1.weight, up_blocks.0.resnets.0.norm1.bias, conv_out.weight, up_blocks.0.resnets.0.time_emb_proj.bias, down_blocks.0.downsamplers.0.conv.weight, mid_block.attentions.0.to_k.weight, mid_block.resnets.1.norm1.bias, conv_out.bias, up_blocks.1.resnets.2.conv1.bias, up_blocks.0.resnets.0.time_emb_proj.weight, down_blocks.2.resnets.1.norm2.weight, up_blocks.1.upsamplers.0.conv.bias, up_blocks.2.resnets.1.time_emb_proj.weight, mid_block.attentions.0.to_v.weight, up_blocks.2.resnets.1.conv2.weight, down_blocks.1.resnets.1.conv2.weight, up_blocks.1.resnets.2.conv2.bias, mid_block.attentions.0.to_v.bias, down_blocks.1.resnets.0.conv_shortcut.bias, up_blocks.0.resnets.1.norm2.weight, up_blocks.0.resnets.2.norm1.weight, down_blocks.1.downsamplers.0.conv.weight, up_blocks.1.resnets.0.norm2.weight, up_blocks.1.resnets.0.conv2.weight, mid_block.resnets.0.norm1.weight, up_blocks.0.resnets.1.conv2.bias, down_blocks.1.resnets.0.norm1.weight, down_blocks.0.resnets.0.norm2.weight, down_blocks.1.resnets.0.norm2.weight, mid_block.resnets.0.conv2.bias, up_blocks.0.resnets.0.norm2.weight, up_blocks.1.resnets.2.norm2.bias, mid_block.resnets.0.conv2.weight, down_blocks.0.resnets.1.norm2.bias, down_blocks.2.resnets.0.conv1.bias, up_blocks.0.resnets.0.conv2.bias, up_blocks.2.resnets.1.norm2.weight, down_blocks.0.downsamplers.0.conv.bias, up_blocks.2.resnets.1.norm1.bias, time_embedding.linear_2.bias, up_blocks.1.resnets.1.conv2.weight, up_blocks.1.resnets.2.norm1.weight, down_blocks.0.resnets.0.conv2.bias, up_blocks.2.resnets.2.norm1.weight, conv_norm_out.weight, up_blocks.2.resnets.2.norm2.weight, down_blocks.0.resnets.1.conv2.weight, mid_block.attentions.0.group_norm.weight, up_blocks.0.resnets.1.conv1.bias, up_blocks.2.resnets.0.time_emb_proj.bias, up_blocks.1.upsamplers.0.conv.weight, mid_block.resnets.0.norm2.bias, up_blocks.1.resnets.0.conv_shortcut.bias, up_blocks.0.resnets.2.norm2.bias, up_blocks.0.resnets.2.conv1.bias, up_blocks.1.resnets.1.norm2.bias, mid_block.resnets.1.norm2.weight, up_blocks.1.resnets.0.norm1.weight, up_blocks.1.resnets.1.time_emb_proj.weight, down_blocks.1.resnets.0.conv1.bias, down_blocks.2.resnets.0.conv2.weight, up_blocks.0.resnets.2.time_emb_proj.weight, up_blocks.2.resnets.1.norm1.weight, up_blocks.1.resnets.1.conv1.weight, down_blocks.2.resnets.0.conv2.bias, up_blocks.0.resnets.0.conv_shortcut.weight, up_blocks.0.resnets.2.norm1.bias, up_blocks.0.resnets.2.conv2.bias, up_blocks.2.resnets.0.norm1.bias, down_blocks.0.resnets.0.conv1.weight, conv_in.weight, down_blocks.0.resnets.0.norm1.bias, mid_block.resnets.1.norm1.weight, down_blocks.1.resnets.1.conv1.bias, down_blocks.2.resnets.1.norm2.bias, up_blocks.1.resnets.1.norm1.weight, mid_block.resnets.0.norm2.weight, time_embedding.linear_2.weight, up_blocks.2.resnets.0.conv1.bias, down_blocks.2.resnets.1.conv1.bias, mid_block.attentions.0.to_out.0.weight, up_blocks.2.resnets.0.norm2.weight, up_blocks.2.resnets.1.conv1.bias, down_blocks.0.resnets.1.norm1.weight, up_blocks.2.resnets.2.time_emb_proj.weight, time_embedding.linear_1.weight, up_blocks.2.resnets.1.conv1.weight, down_blocks.1.downsamplers.0.conv.bias, up_blocks.0.resnets.1.time_emb_proj.weight, down_blocks.0.resnets.0.norm1.weight, down_blocks.1.resnets.0.norm1.bias, mid_block.resnets.0.conv1.weight, up_blocks.2.resnets.1.conv2.bias, down_blocks.2.resnets.1.conv2.bias, up_blocks.0.resnets.0.conv_shortcut.bias, up_blocks.1.resnets.1.time_emb_proj.bias, up_blocks.1.resnets.2.time_emb_proj.weight, mid_block.resnets.1.conv2.weight, up_blocks.0.upsamplers.0.conv.weight, down_blocks.0.resnets.1.conv1.bias, down_blocks.1.resnets.1.norm1.bias, up_blocks.0.resnets.0.conv1.bias, mid_block.resnets.1.time_emb_proj.weight, down_blocks.1.resnets.0.conv2.bias, up_blocks.0.resnets.1.norm1.weight, up_blocks.2.resnets.0.time_emb_proj.weight, up_blocks.2.resnets.2.conv2.weight, mid_block.resnets.1.conv1.bias, time_embedding.linear_1.bias, up_blocks.1.resnets.0.conv_shortcut.weight, mid_block.attentions.0.to_q.bias, up_blocks.1.resnets.0.conv1.bias, down_blocks.1.resnets.1.conv2.bias, down_blocks.1.resnets.0.conv2.weight, down_blocks.0.resnets.1.norm2.weight, up_blocks.1.resnets.2.time_emb_proj.bias, up_blocks.0.resnets.2.time_emb_proj.bias, up_blocks.1.resnets.1.conv2.bias, up_blocks.0.resnets.0.conv1.weight, mid_block.attentions.0.to_k.bias, down_blocks.0.resnets.0.conv1.bias, up_blocks.2.resnets.0.conv1.weight, up_blocks.0.resnets.0.norm2.bias, mid_block.resnets.1.conv1.weight, mid_block.resnets.1.time_emb_proj.bias, down_blocks.0.resnets.1.norm1.bias, mid_block.resnets.1.conv2.bias, up_blocks.0.upsamplers.0.conv.bias, up_blocks.2.resnets.2.conv2.bias, up_blocks.0.resnets.1.norm1.bias, down_blocks.1.resnets.1.norm1.weight, down_blocks.2.resnets.0.conv_shortcut.bias, down_blocks.1.resnets.0.conv_shortcut.weight, down_blocks.2.resnets.0.norm2.bias, up_blocks.0.resnets.1.conv1.weight, down_blocks.0.resnets.1.conv1.weight, up_blocks.1.resnets.2.conv1.weight, up_blocks.0.resnets.2.norm2.weight, up_blocks.1.resnets.2.norm1.bias, mid_block.attentions.0.group_norm.bias, up_blocks.2.resnets.2.norm2.bias, down_blocks.2.resnets.1.norm1.weight, mid_block.resnets.0.time_emb_proj.bias, mid_block.attentions.0.to_out.0.bias, up_blocks.0.resnets.0.norm1.weight, up_blocks.0.resnets.1.time_emb_proj.bias, up_blocks.0.resnets.0.conv2.weight, down_blocks.1.resnets.0.norm2.bias, conv_norm_out.bias, up_blocks.1.resnets.0.norm2.bias, down_blocks.1.resnets.1.norm2.bias, conv_in.bias, up_blocks.0.resnets.1.conv2.weight, mid_block.resnets.0.conv1.bias, down_blocks.2.resnets.0.conv1.weight, up_blocks.2.resnets.1.norm2.bias, down_blocks.2.resnets.0.norm2.weight, down_blocks.1.resnets.0.conv1.weight, down_blocks.2.resnets.1.conv2.weight, up_blocks.2.resnets.0.conv2.weight, up_blocks.1.resnets.0.conv2.bias, down_blocks.2.resnets.1.norm1.bias, up_blocks.1.resnets.0.conv1.weight, up_blocks.0.resnets.2.conv2.weight, up_blocks.1.resnets.0.time_emb_proj.weight, up_blocks.2.resnets.2.conv1.bias, up_blocks.1.resnets.1.norm1.bias, down_blocks.2.resnets.0.conv_shortcut.weight, up_blocks.2.resnets.0.conv2.bias, up_blocks.2.resnets.0.norm1.weight, up_blocks.1.resnets.0.norm1.bias, up_blocks.1.resnets.1.norm2.weight, up_blocks.2.resnets.1.time_emb_proj.bias, down_blocks.0.resnets.0.conv2.weight, down_blocks.0.resnets.1.conv2.bias, mid_block.resnets.1.norm2.bias, mid_block.resnets.0.norm1.bias, up_blocks.1.resnets.0.time_emb_proj.bias, up_blocks.1.resnets.2.conv2.weight, up_blocks.2.resnets.2.conv1.weight.
Please make sure to pass low_cpu_mem_usage=False and device_map=None if you want to randomly initialize those weights or else make sure your checkpoint file is correct.

Can you please explain if how are you loading the captions data with images?

Hi, I appreciate that you release the code here.
I wonder if you have test the code in training process?