Dimension issue on RAFT about mmflow

[ckcraig01]

with pytorch 1.9 and mmcv 1.4.3

[MeowZheng]

I try the same command line

python tools/test.py configs/raft/raft_8x2_100k_flyingchairs_368x496.py  /ckpt/raft_8x2_100k_flyingchairs.pth --eval EPE

but didn't meet the issue.

May you add breakpoints at the following line, and check the input image shape before extract_feat, and the valid shape is N, 6, H, W .
File "root/framework/optical_flow/mmflow/mmflow/models/flow_estimators/raft.py", line 145, in forward_test
feat1, feat2, h_feat, cxt_feat = self.extract_feat(imgs)

[ckcraig01]

@MeowZheng Thank you for the promptly feedback

I have added to print image shape based on your suggestion
print(imgs.shape):
Result: torch.Size([1, 1, 6, 384, 512])

I have added to print
print(data['imgs']) at
https://github.com/open-mmlab/mmflow/blob/master/mmflow/apis/test.py#L47

May you help to let me know where I shall continue to investigate? Thanks.

DataContainer([tensor([[[[-0.8353, -0.8588, -0.8667, ..., -0.2863, -0.2863, -0.2784],
[-0.8980, -0.8824, -0.8431, ..., -0.2863, -0.2863, -0.2863],
[-0.8510, -0.8824, -0.8745, ..., -0.2392, -0.2549, -0.2549],
...,
[-0.9686, -0.9608, -0.9529, ..., -0.8902, -0.8824, -0.8510],
[-0.9451, -0.9608, -0.9686, ..., -0.8902, -0.8745, -0.8588],
[-0.9294, -0.9608, -0.9765, ..., -0.8980, -0.8824, -0.8745]],

     [[-0.8667, -0.8902, -0.8980,  ..., -0.1294, -0.1294, -0.1294],
      [-0.9059, -0.8902, -0.8510,  ..., -0.1294, -0.1294, -0.1294],
      [-0.8588, -0.8902, -0.8824,  ..., -0.0745, -0.0980, -0.1059],
      ...,
      [-0.9608, -0.9529, -0.9529,  ..., -0.9216, -0.9373, -0.9216],
      [-0.9373, -0.9529, -0.9686,  ..., -0.9216, -0.9451, -0.9294],
      [-0.9216, -0.9529, -0.9765,  ..., -0.9294, -0.9529, -0.9451]],

     [[-0.8275, -0.8510, -0.8588,  ..., -0.0353, -0.0275, -0.0275],
      [-0.8902, -0.8745, -0.8196,  ..., -0.0353, -0.0353, -0.0275],
      [-0.8275, -0.8588, -0.8510,  ...,  0.0275, -0.0039,  0.0039],
      ...,
      [-0.9765, -0.9686, -0.9529,  ..., -0.8824, -0.9137, -0.8980],
      [-0.9529, -0.9686, -0.9686,  ..., -0.8824, -0.9216, -0.9059],
      [-0.9373, -0.9686, -0.9765,  ..., -0.8902, -0.9294, -0.9216]],

     [[-0.8353, -0.8588, -0.8667,  ..., -0.2706, -0.2706, -0.2627],
      [-0.8980, -0.8824, -0.8431,  ..., -0.2706, -0.2706, -0.2627],
      [-0.8510, -0.8824, -0.8745,  ..., -0.2784, -0.2784, -0.2706],
      ...,
      [-0.9686, -0.9608, -0.9529,  ..., -0.8902, -0.8824, -0.8510],
      [-0.9451, -0.9608, -0.9686,  ..., -0.8902, -0.8745, -0.8588],
      [-0.9294, -0.9608, -0.9765,  ..., -0.8980, -0.8824, -0.8745]],

     [[-0.8667, -0.8902, -0.8980,  ..., -0.1216, -0.1137, -0.1059],
      [-0.9059, -0.8902, -0.8510,  ..., -0.1216, -0.1137, -0.1059],
      [-0.8588, -0.8902, -0.8824,  ..., -0.1216, -0.1216, -0.1137],
      ...,
      [-0.9608, -0.9529, -0.9529,  ..., -0.9216, -0.9373, -0.9216],
      [-0.9373, -0.9529, -0.9686,  ..., -0.9216, -0.9451, -0.9294],
      [-0.9216, -0.9529, -0.9765,  ..., -0.9294, -0.9529, -0.9451]],

     [[-0.8275, -0.8510, -0.8588,  ..., -0.0196, -0.0196, -0.0118],
      [-0.8902, -0.8745, -0.8196,  ..., -0.0196, -0.0196, -0.0118],
      [-0.8275, -0.8588, -0.8510,  ..., -0.0196, -0.0196, -0.0118],
      ...,
      [-0.9765, -0.9686, -0.9529,  ..., -0.8824, -0.9137, -0.8980],
      [-0.9529, -0.9686, -0.9686,  ..., -0.8824, -0.9216, -0.9059],
      [-0.9373, -0.9686, -0.9765,  ..., -0.8902, -0.9294, -0.9216]]]])])

[ckcraig01]

Hi, solved!

I have installed a CPU-version of pytorch and cause this issue:

Have resolved with following installation flow

- conda create -n mmflow python=3.8 conda -y
- conda activate mmflow
- conda install pytorch torchvision cudatoolkit=11.3 -c pytorch
- pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html
- pip install -r requirements/build.txt
- pip install -v -e . 

But I found another issue
(1)

mmflow/mmflow/apis/test.py

Line 57 in 40caf06

write_flow(osp.join(out_dir, f'flow_{i:03d}.flo'), r)

(2)

mmflow/mmflow/datasets/utils/flow_io.py

Line 41 in 40caf06

def write_flow(flow: np.ndarray, flow_file: str) -> None:

The order seem to be reverse? Thanks

[MeowZheng]

Many thanks for you finding the bug of write_flow. Community contributions are more than welcome in OpenMMLab repos and we hope to cooperate deeply with the community, so we very much appreciate it if you create a PR to fix the bug. Here is a tutorial for creating PR https://github.com/open-mmlab/mmcv/blob/master/docs/en/community/pr.md