yunxinli / ndcr Goto Github PK

when training the model, I encountered the bug below:

Exception has occurred: AttributeError
'NoneType' object has no attribute 'shape'
File "/home/NDCR/OFA/transformers/src/transformers/models/ofa/modeling_ofa.py", line 1901, in forward
~encoder_outputs.padding_mask, encoder_hidden_states.dtype, decoder_input_ids.shape[-1]
File "/home/NDCR/OFA_encoder_Divide_and_Conquer.py", line 204, in forward
gen = self.OFA(input_ids_context, patch_images=global_image, decoder_input_ids=None)
File "/home/NDCR/OFA_encoder_Divide_and_Conquer.py", line 742, in
contextual_clip(images, text, pos_mask, None, str(img_dir), text_=None, input_ids=input_ids)
AttributeError: 'NoneType' object has no attribute 'shape'

from extras import convert_sents_to_features, BertLayer, BartAttention

Hello author,
I encountered the following bugs when I tried to reproduce the results of the paper. It seems that the size of checkpoint(pretrain_BART_generator_coldstart_OFA) you provided on huggingface doesn't match the ''current model''.

Traceback (most recent call last):
File "/datasata0/cloud-wuzhengyuan/lxj/NDCR/OFA_encoder_Divide_and_Conquer.py", line 426, in
contextual_clip.load_state_dict(checkpoint['model_state_dict'], False)
File "/root/miniconda3/envs/blip/lib/python3.9/site-packages/torch/nn/modules/module.py", line 2152, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for ContextualCLIP:
size mismatch for text_encoder.model.shared.weight: copying a param with shape torch.Size([50265, 768]) from checkpoint, the shape in current model is torch.Size([50265, 1024]).
size mismatch for text_encoder.model.encoder.embed_tokens.weight: copying a param with shape torch.Size([50265, 768]) from checkpoint, the shape in current model is torch.Size([50265, 1024]).
size mismatch for text_encoder.model.encoder.embed_positions.weight: copying a param with shape torch.Size([1026, 768]) from checkpoint, the shape in current model is torch.Size([1026, 1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn.k_proj.weight: copying a param with shape torch.Size([768, 768]) from checkpoint, the shape in current model is torch.Size([1024, 1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn.k_proj.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn.v_proj.weight: copying a param with shape torch.Size([768, 768]) from checkpoint, the shape in current model is torch.Size([1024, 1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn.v_proj.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn.q_proj.weight: copying a param with shape torch.Size([768, 768]) from checkpoint, the shape in current model is torch.Size([1024, 1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn.q_proj.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn.out_proj.weight: copying a param with shape torch.Size([768, 768]) from checkpoint, the shape in current model is torch.Size([1024, 1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn.out_proj.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn_layer_norm.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]).
size mismatch for text_encoder.model.encoder.layers.0.self_attn_layer_norm.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]).
......