cvlab-tohoku / dense-coattention-network Goto Github PK

(running on Mac OS X)

Could you guide me as from where the attention weights are obtained for visualizing the results obtained?

Hi @kienduynguyen , there is a problem when I ran the program and the link is:
https://github.com/cvlab-tohoku/Dense-CoAttention-Network/blob/master/train.py#L47

The error information is:
img, img_mask = img_info
ValueError: too many values to unpack (expected 2)

My torch version is 1.1.0.
Did you meet this problem before or can you please tell me how to solve it? Thank you!

hello, I try to run your train.py. However, "from dense_coattn.data import VQADataset". Maybe you forget updating your code in dense_coattn.data.

Hi, I trained the model followed train.py and want to get the answers from answer.py, but when I ran it, I got an error like:

Initializing Dynamic LSTM...
Freezing resnet152 ...
Loading model from checkpoint at /home/annie/Dense-CoAttention-Network/VQA/model/DCN1.pt
Traceback (most recent call last):
  File "answer.py", line 156, in <module>
    main(args)
  File "answer.py", line 109, in main
    model.load_state_dict(checkpoint["model"])
  File "/usr/local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 522, in load_state_dict
    .format(name))
KeyError: 'unexpected key "lang_extract.rnn.weight_ih_l1" in state_dict'

Could you tell me what's wrong with it?

Building model...
Loading pretrained word vectors from /home/prudhvik/data/glove_840B.pt
Loading model from checkpoint at /home/prudhvik//models/trained.pt
Traceback (most recent call last):
File "test_dcn.py", line 202, in
model, idx2ans, word2idx = load_pretrained_model(opt)
File "test_dcn.py", line 121, in load_pretrained_model
model.cuda(opt['gpus'][0])
AttributeError: 'dict' object has no attribute 'cuda'

I am unable to resolve it.

hello,I have some questions after running your answer.py. and ensemble.py. I have got the .json file with question_id and answer. But there's nothing like the percentage that you get after running train.py. What can I do for getting the percentage result? look forward to hearing from you.

I only find load_features.ipynb for loading pre-trained features from Faster RCNN, but I cannot find preprocess files to load pre-trained features from resnet 152.
So, how can I generate h5 files of pre-trained features from resnet 152, if I want to train the model with resnet 152?
Thanks!

I read your code here, it seems that you predict a list of scores based barely on the image and question feature, find the index of the largest score, and use this index to find the answer in the answer list.
However, since you didn't use the answer feature, I think you may predict the same scores even if you shuffle the answer list randomly.

Did I miss something in the code? I will be happy if you can give me an answer :)

Hello,can you provide the pretrained model to me? I want to test the model directly.Thank you!

Hello, I want to know where the "rcnn_trainval.pt","rcnn_trainval.h5","rcnn_test.pt","rcnn_test.h5" from the class RCNNDataset from dataset.py(dense_coattn/data/). Are they from 'prepross/load_features.ipynb'? But in this program they are named trainval_images.h5,trainval_image.pt,test_images.h5,test_images.pt.
I have renamed them. However, when I run the train.py, it will get stack on “for I, batch in enumerate(dataloader)” from def trainEpoch.
I think it might be a dataset problem. I hope you can help me. Thanks for your reading.