open-mmlab / mmflow Goto Github PK

View Code? Open in Web Editor NEW

941.0 7.0 113.0 22.21 MB

OpenMMLab optical flow toolbox and benchmark

Home Page: https://mmflow.readthedocs.io/en/latest/

License: Apache License 2.0

Python 99.56% Dockerfile 0.21% Shell 0.23%

optical-flow openmmlab pytorch

mmflow's People

Contributors

Stargazers

Watchers

Forkers

zhouzaida meowzheng gaotongxiao rangilyu xiaokekeke zhuangpeng97 haochenye matrixgame2018 ohhandsome wumengqi1997 hellock hologerry 459737087 czm369 dachunkai yeahtech zhanghua7099 jianshijim yangxue0827 9p15p astarchen fasladodo pppppm sycasia xujinglin zytx121 galymzhanabdimanap sxty4170160 yanziwei unexpectedjourney plyfager yshuo-li bm21 lhao0301 sheffieldcao abdurad deepbehavier dlut-yyc chenjiewang jafariparizi iuk standardgalactic deppmeng anjaeju williamcorsel iizhb betterhalfwzm ishrat-tl inoculate23 zachary-66 pingponglabs kp-forks delinqu sushlokshah qilei123 wangyixiang 8639sung chauminhnguyen zhouyue000 mr-brillianter fc-idris techthiyanes daimacode singlezombie ymfiubyf salvatore-tech lwcyjq kaleidozhouyn ice-tong robotseye a99user renkaijiu pedrohuang123 shercoo ajunlonglive ucalyptus2 eagle-chase jaehoon-zx theomoutakanni cgsaxner masterhow shanjiangemugen zenx0x chenmin00 hengle zikai1 minghuiwsw xiexinch hjy0120 whuhxb wangqiang1588 sam-37 nickalbergo liuhaochuan79 crazysteeaam sanyaade-projects open-mmlab-12 spidartist chrisindris taherehzarratehsan

mmflow's Issues

[Contribution] CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Driving

I was a member of OpenMMLab and published an optical flow estimation model CSFlow (paper: https://arxiv.org/abs/2202.00909). The work was received by IV 2022 (2022 33rd IEEE Intelligent Vehicles Symposium). I want to add this model to MMFlow.

I am not familiar with the workflow to contibute a model. Could you check if my to-do list is complete ? Can I create a PR directly so that we can discuss on the PR?

My to-do list:

Model implementation with docstring.
Configs.
Benchmark and paper introduction on README.
Pre-trained checkpoints.
Unit tests.

Thank you!

Dimension issue on RAFT

Thanks for your great work!

Test input script:

python tools/test.py configs/raft/raft_8x2_100k_flyingchairs_368x496.py  /ckpt/raft_8x2_100k_flyingchairs.pth --eval EPE

The error message is as follow:

[ ] 0/640, elapsed: 0s, ETA:Traceback (most recent call last):
File "tools/test.py", line 178, in
main()
File "tools/test.py", line 171, in main
f'In {dataset_name} '
File "root/framework/optical_flow/mmflow/mmflow/core/evaluation/evaluation.py", line 38, in online_evaluation
model, data_loader, metric=metric, **kwargs)
File "root/framework/optical_flow/mmflow/mmflow/core/evaluation/evaluation.py", line 68, in single_gpu_online_evaluation
batch_results = model(test_mode=True, **data)
File "root/miniconda3/envs/mmflow/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "root/miniconda3/envs/mmflow/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 48, in forward
return self.module(*inputs[0], **kwargs[0])
File "root/miniconda3/envs/mmflow/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "root/framework/optical_flow/mmflow/mmflow/models/flow_estimators/base.py", line 61, in forward
return self.forward_test(*args, **kwargs)
File "root/framework/optical_flow/mmflow/mmflow/models/flow_estimators/raft.py", line 145, in forward_test
feat1, feat2, h_feat, cxt_feat = self.extract_feat(imgs)
File "root/framework/optical_flow/mmflow/mmflow/models/flow_estimators/raft.py", line 72, in extract_feat
feat1 = self.encoder(img1)
File "root/miniconda3/envs/mmflow/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "root/framework/optical_flow/mmflow/mmflow/models/encoders/raft_encoder.py", line 293, in forward
x = self.conv1(x)
File "root/miniconda3/envs/mmflow/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "root/miniconda3/envs/mmflow/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 443, in forward
return self._conv_forward(input, self.weight, self.bias)
File "root/miniconda3/envs/mmflow/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 440, in _conv_forward
self.padding, self.dilation, self.groups)
RuntimeError: Expected 4-dimensional input for 4-dimensional weight [64, 3, 7, 7], but got 5-dimensional input of size [1, 1, 6, 384, 512] instead

May you help me on this?

Broken hyperlink in intro.md - Documentation

Describe the bug
The hyperlink in intro.md that should open the "get_started" page is not valid

Reproduction

Go to https://github.com/open-mmlab/mmflow/blob/master/docs/en/intro.md
In the section "How to Use this Guide", step 2, click on "get_started" hyperlink
It gives 404 page not found

Bug fix
In intro.md file, change the hyperlink from get_started to https://github.com/open-mmlab/mmflow/blob/master/docs/en/get_started.md

Great project, i'm going to study how to use it

The support of C++ project

Hi! Thank you for your good work!

Will this work support the use of C++? If not, could you give me some advice about the deployment method for the C++ project?

How should I calculate the forward and backward flow?

Thanks for your great work!

I am new to optical flow. Now I want to use occlusion_estimation() defined in occlusion_estimation.py. I see the annotations "flow_fw (Tensor): The forward flow with shape (N, 2, H, W)". Since now I get 25 extracted frames each video, I am wondering how to calculate the forward flow and backward flow? Do I need to iteratively rum image_demo.py and then concatenate all the results on the first dimension?

Thanks again for your great work.

Config file error in metafile

mmflow/configs/pwcnet/metafile.yml

Line 117 in 203ea55

Config: configs/pwcnet/.py

little bug

during dataset download wegt https://lmb.informatik.uni-freiburg.de/data/FlyingChairs/FlyingChairs.zip,
remember to change wegt to wget

in mmflow/docs/dataset_prepare.md

Problem with tensor shapes in gma_plus-p_8x2_50k_kitti2015_288x960.pth

Hi!

Bug description
I have got a problem with gma_plus-p_8x2_50k_kitti2015_288x960.pth model. Config and checkpoint was downloaded from github manually.

Reproduction
I use code from basic inference tutorial:

config_file = '/home/aa_siv/projects/flow/mmflow/configs/gma/gma_plus-p_8x2_50k_kitti2015_288x960.py'
checkpoint_file = '/home/aa_siv/projects/flow/notebooks/modelzoo_mmflow/gma_plus-p_8x2_50k_kitti2015_288x960.pth'

model = init_model(config_file, checkpoint_file, device='cuda:1')
result = inference_model(model, img1_path, img2_path)
write_flow(result, flow_file='flow_gma.flo')
flow_map = visualize_flow(result, save_file='flow_map_gma.png')

This code worked well with FlowNetC models, RAFT, but it is crushing with GMA plus p model.

It returns error:
Error traceback

RuntimeError                              Traceback (most recent call last)
/home/aa_siv/projects/flow/notebooks/test_opencv_flow.ipynb Cell 24' in <module>
      [7](vscode-notebook-cell://ssh-remote%2B86.62.87.164/home/aa_siv/projects/flow/notebooks/test_opencv_flow.ipynb#ch0000022vscode-remote?line=6) model = init_model(config_file, checkpoint_file, device='cuda:1')
      [9](vscode-notebook-cell://ssh-remote%2B86.62.87.164/home/aa_siv/projects/flow/notebooks/test_opencv_flow.ipynb#ch0000022vscode-remote?line=8) # test image pair, and save the results
---> [10](vscode-notebook-cell://ssh-remote%2B86.62.87.164/home/aa_siv/projects/flow/notebooks/test_opencv_flow.ipynb#ch0000022vscode-remote?line=9) result = inference_model(model, img1_path, img2_path)
     [11](vscode-notebook-cell://ssh-remote%2B86.62.87.164/home/aa_siv/projects/flow/notebooks/test_opencv_flow.ipynb#ch0000022vscode-remote?line=10) # save the optical flow file
     [12](vscode-notebook-cell://ssh-remote%2B86.62.87.164/home/aa_siv/projects/flow/notebooks/test_opencv_flow.ipynb#ch0000022vscode-remote?line=11) write_flow(result, flow_file='flow_gma.flo')

File ~/projects/flow/mmflow/mmflow/apis/inference.py:151, in inference_model(model, img1s, img2s, valids)
    [149](file:///home/aa_siv/projects/flow/mmflow/mmflow/apis/inference.py?line=148) # forward the model
    [150](file:///home/aa_siv/projects/flow/mmflow/mmflow/apis/inference.py?line=149) with torch.no_grad():
--> [151](file:///home/aa_siv/projects/flow/mmflow/mmflow/apis/inference.py?line=150)     results = model(test_mode=True, **data)
    [153](file:///home/aa_siv/projects/flow/mmflow/mmflow/apis/inference.py?line=152) if valid_masks[0] is not None:
    [154](file:///home/aa_siv/projects/flow/mmflow/mmflow/apis/inference.py?line=153)     # filter the output flow map
    [155](file:///home/aa_siv/projects/flow/mmflow/mmflow/apis/inference.py?line=154)     for result, valid in zip(results, valid_masks):

File ~/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py:1102, in Module._call_impl(self, *input, **kwargs)
   [1098](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1097) # If we don't have any hooks, we want to skip the rest of the logic in
   [1099](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1098) # this function, and just call forward.
   [1100](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1099) if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   [1101](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1100)         or _global_forward_hooks or _global_forward_pre_hooks):
-> [1102](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1101)     return forward_call(*input, **kwargs)
   [1103](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1102) # Do not call functions when jit is used
   [1104](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1103) full_backward_hooks, non_full_backward_hooks = [], []

File ~/projects/flow/mmflow/mmflow/models/flow_estimators/base.py:61, in FlowEstimator.forward(self, test_mode, *args, **kwargs)
     [59](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/base.py?line=58)     return self.forward_train(*args, **kwargs)
     [60](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/base.py?line=59) else:
---> [61](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/base.py?line=60)     return self.forward_test(*args, **kwargs)

File ~/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py:151, in RAFT.forward_test(self, imgs, flow_init, img_metas)
    [148](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=147) if flow_init is None:
    [149](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=148)     flow_init = torch.zeros((B, 2, H, W), device=feat1.device)
--> [151](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=150) results = self.decoder.forward_test(
    [152](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=151)     feat1=feat1,
    [153](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=152)     feat2=feat2,
    [154](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=153)     flow=flow_init,
    [155](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=154)     h_feat=h_feat,
    [156](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=155)     cxt_feat=cxt_feat,
    [157](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=156)     img_metas=img_metas)
    [158](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=157) # recover iter in train
    [159](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/flow_estimators/raft.py?line=158) self.decoder.iters = train_iter

File ~/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py:504, in RAFTDecoder.forward_test(self, feat1, feat2, flow, h_feat, cxt_feat, img_metas)
    [482](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=481) def forward_test(self,
    [483](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=482)                  feat1: torch.Tensor,
    [484](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=483)                  feat2: torch.Tensor,
   (...)
    [487](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=486)                  cxt_feat: torch.Tensor,
    [488](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=487)                  img_metas=None) -> Sequence[Dict[str, np.ndarray]]:
    [489](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=488)     """Forward function when model training.
    [490](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=489) 
    [491](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=490)     Args:
   (...)
    [502](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=501)             with the same size of images before augmentation.
    [503](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=502)     """
--> [504](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=503)     flow_pred = self.forward(feat1, feat2, flow, h_feat, cxt_feat)
    [506](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=505)     flow_result = flow_pred[-1]
    [507](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/raft_decoder.py?line=506)     # flow maps with the shape [H, W, 2]

File ~/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py:283, in GMADecoder.forward(self, feat1, feat2, flow, h, cxt_feat)
    [280](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=279) upflow_preds = []
    [281](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=280) delta_flow = torch.zeros_like(flow)
--> [283](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=282) attention = self.attn(cxt_feat)
    [285](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=284) for _ in range(self.iters):
    [286](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=285)     flow = flow.detach()

File ~/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py:1102, in Module._call_impl(self, *input, **kwargs)
   [1098](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1097) # If we don't have any hooks, we want to skip the rest of the logic in
   [1099](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1098) # this function, and just call forward.
   [1100](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1099) if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   [1101](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1100)         or _global_forward_hooks or _global_forward_pre_hooks):
-> [1102](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1101)     return forward_call(*input, **kwargs)
   [1103](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1102) # Do not call functions when jit is used
   [1104](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1103) full_backward_hooks, non_full_backward_hooks = [], []

File ~/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py:135, in Attention.forward(self, x)
    [132](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=131) self_similarity = torch.zeros((B, self.heads, H * W, H * W)).to(q)
    [134](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=133) if self.max_pos_size is not None:
--> [135](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=134)     self_similarity += self.pos_emb(q)
    [137](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=136) if not self.position_only:
    [138](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=137)     # self_similarity shape is (B, heads, HxW, HxW)
    [139](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=138)     self_similarity += torch.matmul(
    [140](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=139)         q.view(B, self.heads, -1, self.head_channels),
    [141](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=140)         k.view(B, self.heads, self.head_channels, -1)) * self.scale

File ~/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py:1102, in Module._call_impl(self, *input, **kwargs)
   [1098](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1097) # If we don't have any hooks, we want to skip the rest of the logic in
   [1099](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1098) # this function, and just call forward.
   [1100](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1099) if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   [1101](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1100)         or _global_forward_hooks or _global_forward_pre_hooks):
-> [1102](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1101)     return forward_call(*input, **kwargs)
   [1103](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1102) # Do not call functions when jit is used
   [1104](file:///home/aa_siv/projects/flow/venv/lib/python3.9/site-packages/torch/nn/modules/module.py?line=1103) full_backward_hooks, non_full_backward_hooks = [], []

File ~/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py:45, in RelPosEmb.forward(self, q)
     [41](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=40) B, heads, H, W, head_channels = q.shape
     [43](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=42) height_emb = self.rel_height(self.rel_ind[:H, :H].reshape(-1)).reshape(
     [44](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=43)     H, H, -1, head_channels)
---> [45](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=44) width_emb = self.rel_width(self.rel_ind[:W, :W].reshape(-1)).reshape(
     [46](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=45)     W, -1, W, head_channels)
     [48](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=47) height_score = torch.einsum('b h x y d, x u v d -> b h x y u v', q,
     [49](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=48)                             height_emb)
     [50](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=49) width_score = torch.einsum('b h x y d, y u v d -> b h x y u v', q,
     [51](file:///home/aa_siv/projects/flow/mmflow/mmflow/models/decoders/gma_decoder.py?line=50)                            width_emb)

RuntimeError: shape '[240, -1, 240, 128]' is invalid for input of size 3276800

I haven't done any code modifications.
I used 2 images with shape 1920, 1080.

Environment
torch 1.10.2+cu113
torchaudio 0.10.2+cu113
torchvision 0.11.3+cu113
mmcv-full 1.5.3
mmflow 0.4.2 /home/aa_siv/projects/flow/mmflow

PyTorch was installed via pip with command:

pip install torch==1.10.2+cu113 torchvision==0.11.3+cu113 torchaudio==0.10.2+cu113 -f https://download.pytorch.org/whl/torch_stable.html

Bug fix
I am new to optical flow and i can't help with bugfix :(

Tutorials Docs Translation: 0_config_.md

Motivation

Update the zh_cn docs file 0_config_.md.

Modification

Translate original english file into Chinese.

BC-breaking (Optional)

Does the modification introduce changes that break the back-compatibility of the downstream repos?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

Pre-commit or other linting tools are used to fix the potential lint issues.
The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
The documentation has been modified accordingly, like docstring or example tutorials.

Basic question about optical flow

I am new to the optical flow and have a basic question.

To learn to predict optical flow, we need the ground truth for supervision. My understanding is, for example, the TV-L1. But since the groud truth optical flow can be computed by a non-CNN algorith,, why we should use a CNN to estimate the optical flow? Is that much faster than the non-CNN method?

1, 'Ground truth length doesn\'t match video frames' AssertionError: Ground truth length doesn't match video frames

1, 'Ground truth length doesn't match video frames'
AssertionError: Ground truth length doesn't match video frames

Input resolution error

I found some model cannot handle input resolution even divided by 16, e.g. 224 or 240.

An example:

import torch
from mmflow.apis import init_model

# Specify the path to model config and checkpoint file
config_file = 'configs/pwcnet_plus_8x1_750k_sintel_kitti2015_hd1k_320x768.py'
checkpoint_file = 'checkpoints/pwcnet_plus_8x1_750k_sintel_kitti2015_hd1k_320x768.pth'

# build the model from a config file and a checkpoint file
model = init_model(config_file, checkpoint_file, device='cuda:0')
with torch.no_grad():
    out = model(imgs=torch.rand(1, 6, 224, 224).cuda())

Error:

Traceback (most recent call last):
  File "/home/louis/PycharmProjects/mmflow/memory_and_speed.py", line 21, in <module>
    out = model(imgs=torch.rand(1, 6, 224, 224).cuda())
  File "/home/louis/miniconda3/envs/mmflow/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/louis/miniconda3/envs/mmflow/lib/python3.10/site-packages/mmflow/models/flow_estimators/base.py", line 61, in forward
    return self.forward_test(*args, **kwargs)
  File "/home/louis/miniconda3/envs/mmflow/lib/python3.10/site-packages/mmflow/models/flow_estimators/pwcnet.py", line 90, in forward_test
    return self.decoder.forward_test(feat1, feat2, H, W, img_metas)
  File "/home/louis/miniconda3/envs/mmflow/lib/python3.10/site-packages/mmflow/models/decoders/pwcnet_decoder.py", line 320, in forward_test
    flow_pred = self.forward(feat1, feat2)
  File "/home/louis/miniconda3/envs/mmflow/lib/python3.10/site-packages/mmflow/models/decoders/pwcnet_decoder.py", line 258, in forward
    corr_feat = torch.cat((corr_feat_, _feat1, upflow, upfeat),
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 7 but got size 8 for tensor number 2 in the list.

Images with side of 240 and 224 result in error while 256 does not.

Finetune with downstream task

When I would like to finetune with other downstream task, it is not so easy to finish it on this repo? I have no good idea about that.

Any plan to support unsupervised optical flow

Hi, thanks for your great work. I am just wondering if you have any plan to support unsupervised optical flow algorithms.
There are also many works for unsueprvised optical flow, like uflow. and the performance of these algorithms are impressive considering the unsupervised learning paradigm. So i think it is a good idea to support some unsupervised optical flow algorithms.

the default grad_clip=None,so when we should set optimizer_config=dict(grad_clip=dict(max_norm=1.0))?

Thanks for your error report and we appreciate it a lot.

Checklist

I have searched related issues but cannot get the expected help.
I have read the FAQ documentation but cannot get the expected help.
The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.

Reproduction

What command or script did you run?

A placeholder for the command.

Did you make any modifications on the code or config? Did you understand what you have modified?
What dataset did you use?

Environment

Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

A placeholder for trackback.

Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!

Train PWC-Net with two gpus

Thanks for your wonderful code!

I have two gpus and to reproduce the PWC-Net, I only need to set the samples_per_gpu=4 HERE and keep other settings unchanged.
And then, run
bash ./tools/dist_train.sh configs/pwcnet/pwcnet_8x1_slong_flyingchairs_384x448.py 2.

Are these okay?

fix links please

some links are failed.

config --> configs for some links, please check.

Invalid weights URL in metafile.yml

mmflow/configs/gma/metafile.yml

Line 99 in 0514e24

 Weights: https://download.openmmlab.com/mmflow/gma/ gma_8x2_120k_mixed_368x768.pth 

Broken URLs

Hello. Some of the URLs in the model zoo, i.e.
https://github.com/open-mmlab/mmflow/blob/master/configs/gma/README.md
have spaces in that break the URL, i.e.

https://download.openmmlab.com/mmflow/gma/ gma_8x2_120k_mixed_368x768.py

instead of

https://download.openmmlab.com/mmflow/gma/gma_8x2_120k_mixed_368x768.py

Is AutoFlow Dataset support planned?

Hi,

first of all cool library. :)
I wanted to know if you have seen the papers on the AutoFlow Dataset, and if support and pretrained models are planned in the near future?

Best,
Markus

pwcnet to onnx

import os
import sys
import argparse
import torch
from mmflow.apis import inference_model, init_model
from mmflow.datasets import visualize_flow, write_flow

def parser_func():
    parser = argparse.ArgumentParser()
    parser.add_argument('--config',default='configs/pwcnet/pwcnet_ft_4x1_300k_kitti_320x896.py',  help='Config file')
    parser.add_argument('--checkpoint', type=str, default='models/pwc/pwcnet_ft_4x1_300k_kitti_320x896.pth')
    parser.add_argument('--out_path', type=str, default='models/pwc/pwcnet_ft_4x1_300k_kitti_320x896.onnx')
    parser.add_argument(
        '--device', default='cuda:0', help='Device used for inference')
    parser.add_argument('--batch_size', type=int, default=1)
    args = parser.parse_args()
    os.makedirs(os.path.dirname(args.out_path), exist_ok=True)
    return args

def export_onnx():
    model = init_model(args.config, args.checkpoint, device=args.device)

    input_names = ['encoder.layers.0.layers.0.conv']
    output_names = ['decoder.decoders.level6.upfeat_layer']

    dummy_input = torch.randn(args.batch_size, 6, 320, 896).cuda()

    torch.onnx.export(model, dummy_input, args.out_path,
                      input_names=input_names, output_names=output_names, opset_version=11,
                      verbose=True)

if __name__ == '__main__':
    args = parser_func()
    export_onnx()

RuntimeError: Only tuples, lists and Variables are supported as JIT inputs/outputs. Dictionaries and strings are also accepted, but their usage is not recommended. Here, received an input of unsupported type: numpy.ndarray

How to extract optical flow of a folder of frame images

if I have already extracted video data into the frames folder， how to extract optical flow from the frames?

Run with tensorRT with dynamic input shape

When deploy (pytorch->onnx->tensorrt) the flow model on TensorRT backend with dynamic shape(e.g., height and width), how to config the ops with parameter depending on input's shape, which is a python int type.
Take the one below for example

mmflow/mmflow/ops/corr_lookup.py

Line 114 in 14f8d8f

xx = torch.arange(0, W, device=flow.device)

Runtime error gradient computation when train.py

Describe the bug
Good evening, i wanted to lauch train.py using a default config file for RAFT on a standard dataset (KITTI_2015).
I followed the instruction to install MMFlow from source successfully.

Reproduction

python tools/train.py configs/raft/raft_8x2_50k_kitti2015_and_Aug_288x960.py \
--load-from /home/s.starace/FlowNets/mmflow/checkpoints/raft/raft_8x2_100k_mixed_368x768.pth

Did you make any modifications on the code or config? Did you understand what you have modified?
I just changed the name of symlink that i created under /data (uppercase)
What dataset did you use?
KITTI_2015

Environment
I launched the command on my PC and also on a little cluster and the output error is the same.

Error traceback
See log attached: slurm-53090.out.txt

Bug fix
Not sure about it, could either be a configuration issue in Encoder/Decoder or a regression.
I'll try the train.py using other models as well and update the report if i understand better the problem.

Any plan to add the training codes?

Hi,

Do you have any plan to add the codes of training from scratch?

Random seed matters?

I cannot get the desired results(epe = 0.78 on flyingchairs), I check the log file provided and find that the seed is "null", so what does the null value means? Doesn't the function "init_random_seed" generate a seed?

Norm the output of correlation_block

In this repo, the correlation output is not normalized(scale=False always), which is different from that in PWC & FlowNet repo. They normalize it by dimension. Hence, is it make little of importance?

mmflow/mmflow/models/utils/correlation_block.py

Lines 61 to 68 in 2f98716

 if self.scaled: 

 if 'sqrt' in self.scale_mode: 

 scale_factor = sqrt(float(C * self.kernel_size**2)) 

 else: 

 scale_factor = float(C * self.kernel_size**2) 

 corr = self.corr_block[0](feat1, feat2) / scale_factor

Why does GaussianNoise in data enhancement cause gradient explosion and grad_norm value is large

Thanks for your error report and we appreciate it a lot.

Checklist

I have searched related issues but cannot get the expected help.
I have read the FAQ documentation but cannot get the expected help.
The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.

Reproduction

What command or script did you run?

A placeholder for the command.

Did you make any modifications on the code or config? Did you understand what you have modified?
What dataset did you use?

Environment

Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

A placeholder for trackback.

Multiple nodes slurm training

Good morning,
I have read the documentation about finetuning and i'd like to lauch train.py to finetune PwcNet on my dataset loading a checkpoint file.
I have created a script to load the dataset under /configs/base/datasets and used in the config script under /configs/pwcnet.

I'd like to reduce the training time, using all the resource i have available on a small cluster with 4 nodes and 4 GPU (NVLINK) NVIDIA Tesla V100 32GB SXM2.
Do you see any rooms for improvements with the following command?
srun -p xgpu --job-name=pwc_kitti --gres=gpu:4 --ntasks=16 --ntasks-per-node=4 --cpus-per-task=2 --kill-on-bad-exit=1 python -u tools/train.py $MMFLOW/configs/pwcnet/pwcnet_ft_4x1_300k_kitti_320x896.py --work-dir=$MMFLOW/work_dir/pwckitti --launcher=slurm

It estimates about 1 day to complete finetuning, do you think i'm using all 4 nodes correctly?
If so, can i reduce the training iterations to require less time?
Thanks in advance!

run video demo，video w 1280 h 720

parser.add_argument('--config',default='configs/pwcnet/pwcnet_plus_8x1_750k_sintel_kitti2015_hd1k_320x768.py',  help='Config file')
parser.add_argument('--checkpoint', default='models/pwc/pwcnet_plus_8x1_750k_sintel_kitti2015_hd1k_320x768.pth', help='Checkpoint file')

Traceback (most recent call last):
File "/home/wangzhaoming/workspace/code/mmflow/demo/video_demo.py", line 142, in
main(args)
File "/home/wangzhaoming/workspace/code/mmflow/demo/video_demo.py", line 82, in main
result = inference_model(model, img1, img2)
File "/disk/workspace/wangzhaoming/code/mmflow/mmflow/apis/inference.py", line 134, in inference_model
results = model(test_mode=True, **data)
File "/disk/conda/envs/python37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/disk/workspace/wangzhaoming/code/mmflow/mmflow/models/flow_estimators/base.py", line 61, in forward
return self.forward_test(*args, **kwargs)
File "/disk/workspace/wangzhaoming/code/mmflow/mmflow/models/flow_estimators/pwcnet.py", line 90, in forward_test
return self.decoder.forward_test(feat1, feat2, H, W, img_metas)
File "/disk/workspace/wangzhaoming/code/mmflow/mmflow/models/decoders/pwcnet_decoder.py", line 320, in forward_test
flow_pred = self.forward(feat1, feat2)
File "/disk/workspace/wangzhaoming/code/mmflow/mmflow/models/decoders/pwcnet_decoder.py", line 259, in forward
dim=1)
RuntimeError: Sizes of tensors must match except in dimension 2. Got 24 and 23 (The offending index is 0)

input image size question

My use case involves 180x360 image frames (satellite earth data). Can I configure mmflow models to take that type of input sizes?

Thanks,
Alex

RuntimeError: Sizes of tensors must match except in dimension 2. Got 46 and 45 (The offending index is 0)

1、故障描述（Description）

运行video_demo推理demo.mp4生成demo_gt.mp4没有问题，推理自己的视频就有问题。
报错：RuntimeError: Sizes of tensors must match except in dimension 2. Got 46 and 45 (The offending index is 0)

2、复现步骤（Reproduction procedure）

git clone https://github.com/open-mmlab/mmflow.git
cd mmflow/
conda activate mmflow
cd checkpoints/
wget https://download.openmmlab.com/mmflow/pwcnet/pwcnet_ft_4x1_300k_sintel_final_384x768.pth
python demo/video_demo.py "/home/thhicv/视频/flow/2_数字_蜡烛_红外光源.mp4" configs/pwcnet/pwcnet_ft_4x1_300k_sintel_final_384x768.py checkpoints/pwcnet_ft_4x1_300k_sintel_final_384x768.pth demo/2_数字_蜡烛_红外光源.mp4

出现错误

(mmflow) thhicv@ThinkStation-P720:~/mmflow$ python demo/video_demo.py "/home/thhicv/视频/flow/2_数字_蜡烛_红外光源.mp4" configs/pwcnet/pwcnet_ft_4x1_300k_sintel_final_384x768.py checkpoints/pwcnet_ft_4x1_300k_sintel_final_384x768.pth demo/2_数字_蜡烛_红外光源.mp4
Use load_from_local loader
Traceback (most recent call last):
  File "demo/video_demo.py", line 141, in <module>
    main(args)
  File "demo/video_demo.py", line 81, in main
    result = inference_model(model, img1, img2)
  File "/home/thhicv/program/mmflow/mmflow/apis/inference.py", line 134, in inference_model
    results = model(test_mode=True, **data)
  File "/home/thhicv/anaconda3/envs/mmflow/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/thhicv/program/mmflow/mmflow/models/flow_estimators/base.py", line 61, in forward
    return self.forward_test(*args, **kwargs)
  File "/home/thhicv/program/mmflow/mmflow/models/flow_estimators/pwcnet.py", line 91, in forward_test
    return self.decoder.forward_test(feat1, feat2, H, W, img_metas)
  File "/home/thhicv/program/mmflow/mmflow/models/decoders/pwcnet_decoder.py", line 321, in forward_test
    flow_pred = self.forward(feat1, feat2)
  File "/home/thhicv/program/mmflow/mmflow/models/decoders/pwcnet_decoder.py", line 259, in forward
    dim=1)
RuntimeError: Sizes of tensors must match except in dimension 2. Got 46 and 45 (The offending index is 0)

涉及的文件

python tools/misc/print_cfg.py configs/pwcnet/pwcnet_ft_4x1_300k_sintel_final_384x768.py

dataset_type = 'Sintel'
data_root = 'data/Sintel'
img_norm_cfg = dict(
    mean=[0.0, 0.0, 0.0], std=[255.0, 255.0, 255.0], to_rgb=False)
crop_size = (384, 768)
global_transform = dict(
    translates=(0.05, 0.05),
    zoom=(1.0, 1.5),
    shear=(0.86, 1.16),
    rotate=(-10.0, 10.0))
relative_transform = dict(
    translates=(0.00375, 0.00375),
    zoom=(0.985, 1.015),
    shear=(1.0, 1.0),
    rotate=(-1.0, 1.0))
train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='LoadAnnotations', with_occ=True),
    dict(
        type='ColorJitter',
        brightness=0.5,
        contrast=0.5,
        saturation=0.5,
        hue=0.5),
    dict(type='RandomGamma', gamma_range=(0.7, 1.5)),
    dict(
        type='Normalize',
        mean=[0.0, 0.0, 0.0],
        std=[255.0, 255.0, 255.0],
        to_rgb=False),
    dict(type='RandomFlip', prob=0.5, direction='horizontal'),
    dict(type='RandomFlip', prob=0.5, direction='vertical'),
    dict(
        type='RandomAffine',
        global_transform=dict(
            translates=(0.05, 0.05),
            zoom=(1.0, 1.5),
            shear=(0.86, 1.16),
            rotate=(-10.0, 10.0)),
        relative_transform=dict(
            translates=(0.00375, 0.00375),
            zoom=(0.985, 1.015),
            shear=(1.0, 1.0),
            rotate=(-1.0, 1.0))),
    dict(type='RandomCrop', crop_size=(384, 768)),
    dict(type='DefaultFormatBundle'),
    dict(
        type='Collect',
        keys=['imgs', 'flow_gt'],
        meta_keys=[
            'img_fields', 'ann_fields', 'filename1', 'filename2',
            'ori_filename1', 'ori_filename2', 'filename_flow',
            'ori_filename_flow', 'ori_shape', 'img_shape', 'img_norm_cfg'
        ])
]
test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='LoadAnnotations'),
    dict(type='InputResize', exponent=4),
    dict(
        type='Normalize',
        mean=[0.0, 0.0, 0.0],
        std=[255.0, 255.0, 255.0],
        to_rgb=False),
    dict(type='TestFormatBundle'),
    dict(
        type='Collect',
        keys=['imgs'],
        meta_keys=[
            'flow_gt', 'filename1', 'filename2', 'ori_filename1',
            'ori_filename2', 'ori_shape', 'img_shape', 'img_norm_cfg',
            'scale_factor', 'pad_shape'
        ])
]
sintel_final_train = dict(
    type='Sintel',
    pipeline=[
        dict(type='LoadImageFromFile'),
        dict(type='LoadAnnotations', with_occ=True),
        dict(
            type='ColorJitter',
            brightness=0.5,
            contrast=0.5,
            saturation=0.5,
            hue=0.5),
        dict(type='RandomGamma', gamma_range=(0.7, 1.5)),
        dict(
            type='Normalize',
            mean=[0.0, 0.0, 0.0],
            std=[255.0, 255.0, 255.0],
            to_rgb=False),
        dict(type='RandomFlip', prob=0.5, direction='horizontal'),
        dict(type='RandomFlip', prob=0.5, direction='vertical'),
        dict(
            type='RandomAffine',
            global_transform=dict(
                translates=(0.05, 0.05),
                zoom=(1.0, 1.5),
                shear=(0.86, 1.16),
                rotate=(-10.0, 10.0)),
            relative_transform=dict(
                translates=(0.00375, 0.00375),
                zoom=(0.985, 1.015),
                shear=(1.0, 1.0),
                rotate=(-1.0, 1.0))),
        dict(type='RandomCrop', crop_size=(384, 768)),
        dict(type='DefaultFormatBundle'),
        dict(
            type='Collect',
            keys=['imgs', 'flow_gt'],
            meta_keys=[
                'img_fields', 'ann_fields', 'filename1', 'filename2',
                'ori_filename1', 'ori_filename2', 'filename_flow',
                'ori_filename_flow', 'ori_shape', 'img_shape', 'img_norm_cfg'
            ])
    ],
    data_root='data/Sintel',
    test_mode=False,
    pass_style='final')
sintel_clean_test = dict(
    type='Sintel',
    pipeline=[
        dict(type='LoadImageFromFile'),
        dict(type='LoadAnnotations'),
        dict(type='InputResize', exponent=4),
        dict(
            type='Normalize',
            mean=[0.0, 0.0, 0.0],
            std=[255.0, 255.0, 255.0],
            to_rgb=False),
        dict(type='TestFormatBundle'),
        dict(
            type='Collect',
            keys=['imgs'],
            meta_keys=[
                'flow_gt', 'filename1', 'filename2', 'ori_filename1',
                'ori_filename2', 'ori_shape', 'img_shape', 'img_norm_cfg',
                'scale_factor', 'pad_shape'
            ])
    ],
    data_root='data/Sintel',
    test_mode=True,
    pass_style='clean')
sintel_final_test = dict(
    type='Sintel',
    pipeline=[
        dict(type='LoadImageFromFile'),
        dict(type='LoadAnnotations'),
        dict(type='InputResize', exponent=4),
        dict(
            type='Normalize',
            mean=[0.0, 0.0, 0.0],
            std=[255.0, 255.0, 255.0],
            to_rgb=False),
        dict(type='TestFormatBundle'),
        dict(
            type='Collect',
            keys=['imgs'],
            meta_keys=[
                'flow_gt', 'filename1', 'filename2', 'ori_filename1',
                'ori_filename2', 'ori_shape', 'img_shape', 'img_norm_cfg',
                'scale_factor', 'pad_shape'
            ])
    ],
    data_root='data/Sintel',
    test_mode=True,
    pass_style='final')
data = dict(
    train_dataloader=dict(
        samples_per_gpu=1,
        workers_per_gpu=5,
        drop_last=True,
        persistent_workers=True),
    val_dataloader=dict(samples_per_gpu=1, workers_per_gpu=5, shuffle=False),
    test_dataloader=dict(samples_per_gpu=1, workers_per_gpu=5, shuffle=False),
    train=[
        dict(
            type='Sintel',
            pipeline=[
                dict(type='LoadImageFromFile'),
                dict(type='LoadAnnotations', with_occ=True),
                dict(
                    type='ColorJitter',
                    brightness=0.5,
                    contrast=0.5,
                    saturation=0.5,
                    hue=0.5),
                dict(type='RandomGamma', gamma_range=(0.7, 1.5)),
                dict(
                    type='Normalize',
                    mean=[0.0, 0.0, 0.0],
                    std=[255.0, 255.0, 255.0],
                    to_rgb=False),
                dict(type='RandomFlip', prob=0.5, direction='horizontal'),
                dict(type='RandomFlip', prob=0.5, direction='vertical'),
                dict(
                    type='RandomAffine',
                    global_transform=dict(
                        translates=(0.05, 0.05),
                        zoom=(1.0, 1.5),
                        shear=(0.86, 1.16),
                        rotate=(-10.0, 10.0)),
                    relative_transform=dict(
                        translates=(0.00375, 0.00375),
                        zoom=(0.985, 1.015),
                        shear=(1.0, 1.0),
                        rotate=(-1.0, 1.0))),
                dict(type='RandomCrop', crop_size=(384, 768)),
                dict(type='DefaultFormatBundle'),
                dict(
                    type='Collect',
                    keys=['imgs', 'flow_gt'],
                    meta_keys=[
                        'img_fields', 'ann_fields', 'filename1', 'filename2',
                        'ori_filename1', 'ori_filename2', 'filename_flow',
                        'ori_filename_flow', 'ori_shape', 'img_shape',
                        'img_norm_cfg'
                    ])
            ],
            data_root='data/Sintel',
            test_mode=False,
            pass_style='final')
    ],
    val=dict(
        type='ConcatDataset',
        datasets=[
            dict(
                type='Sintel',
                pipeline=[
                    dict(type='LoadImageFromFile'),
                    dict(type='LoadAnnotations'),
                    dict(type='InputResize', exponent=4),
                    dict(
                        type='Normalize',
                        mean=[0.0, 0.0, 0.0],
                        std=[255.0, 255.0, 255.0],
                        to_rgb=False),
                    dict(type='TestFormatBundle'),
                    dict(
                        type='Collect',
                        keys=['imgs'],
                        meta_keys=[
                            'flow_gt', 'filename1', 'filename2',
                            'ori_filename1', 'ori_filename2', 'ori_shape',
                            'img_shape', 'img_norm_cfg', 'scale_factor',
                            'pad_shape'
                        ])
                ],
                data_root='data/Sintel',
                test_mode=True,
                pass_style='clean'),
            dict(
                type='Sintel',
                pipeline=[
                    dict(type='LoadImageFromFile'),
                    dict(type='LoadAnnotations'),
                    dict(type='InputResize', exponent=4),
                    dict(
                        type='Normalize',
                        mean=[0.0, 0.0, 0.0],
                        std=[255.0, 255.0, 255.0],
                        to_rgb=False),
                    dict(type='TestFormatBundle'),
                    dict(
                        type='Collect',
                        keys=['imgs'],
                        meta_keys=[
                            'flow_gt', 'filename1', 'filename2',
                            'ori_filename1', 'ori_filename2', 'ori_shape',
                            'img_shape', 'img_norm_cfg', 'scale_factor',
                            'pad_shape'
                        ])
                ],
                data_root='data/Sintel',
                test_mode=True,
                pass_style='final')
        ],
        separate_eval=True),
    test=dict(
        type='ConcatDataset',
        datasets=[
            dict(
                type='Sintel',
                pipeline=[
                    dict(type='LoadImageFromFile'),
                    dict(type='LoadAnnotations'),
                    dict(type='InputResize', exponent=4),
                    dict(
                        type='Normalize',
                        mean=[0.0, 0.0, 0.0],
                        std=[255.0, 255.0, 255.0],
                        to_rgb=False),
                    dict(type='TestFormatBundle'),
                    dict(
                        type='Collect',
                        keys=['imgs'],
                        meta_keys=[
                            'flow_gt', 'filename1', 'filename2',
                            'ori_filename1', 'ori_filename2', 'ori_shape',
                            'img_shape', 'img_norm_cfg', 'scale_factor',
                            'pad_shape'
                        ])
                ],
                data_root='data/Sintel',
                test_mode=True,
                pass_style='clean'),
            dict(
                type='Sintel',
                pipeline=[
                    dict(type='LoadImageFromFile'),
                    dict(type='LoadAnnotations'),
                    dict(type='InputResize', exponent=4),
                    dict(
                        type='Normalize',
                        mean=[0.0, 0.0, 0.0],
                        std=[255.0, 255.0, 255.0],
                        to_rgb=False),
                    dict(type='TestFormatBundle'),
                    dict(
                        type='Collect',
                        keys=['imgs'],
                        meta_keys=[
                            'flow_gt', 'filename1', 'filename2',
                            'ori_filename1', 'ori_filename2', 'ori_shape',
                            'img_shape', 'img_norm_cfg', 'scale_factor',
                            'pad_shape'
                        ])
                ],
                data_root='data/Sintel',
                test_mode=True,
                pass_style='final')
        ],
        separate_eval=True))
optimizer = dict(
    type='Adam', lr=3e-05, weight_decay=0.0004, betas=(0.9, 0.999))
optimizer_config = dict(grad_clip=None)
lr_config = dict(
    policy='MultiStage',
    by_epoch=False,
    gammas=[0.5, 0.5],
    milestone_lrs=[3e-05, 2e-05],
    milestone_iters=[0, 150000],
    steps=[[
        45000, 65000, 85000, 95000, 97500, 100000, 110000, 120000, 130000,
        140000
    ],
           [
               195000, 215000, 235000, 245000, 247500, 250000, 260000, 270000,
               280000, 290000
           ]])
runner = dict(type='IterBasedRunner', max_iters=300000)
checkpoint_config = dict(by_epoch=False, interval=50000)
evaluation = dict(interval=50000, metric='EPE')
log_config = dict(
    interval=50,
    hooks=[dict(type='TextLoggerHook'),
           dict(type='TensorboardLoggerHook')])
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = 'https://download.openmmlab.com/mmflow/pwcnet/pwcnet_ft_4x1_300k_sintel_384x768.pth'
resume_from = None
workflow = [('train', 1)]
model = dict(
    type='PWCNet',
    encoder=dict(
        type='PWCNetEncoder',
        in_channels=3,
        net_type='Basic',
        pyramid_levels=[
            'level1', 'level2', 'level3', 'level4', 'level5', 'level6'
        ],
        out_channels=(16, 32, 64, 96, 128, 196),
        strides=(2, 2, 2, 2, 2, 2),
        dilations=(1, 1, 1, 1, 1, 1),
        act_cfg=dict(type='LeakyReLU', negative_slope=0.1)),
    decoder=dict(
        type='PWCNetDecoder',
        in_channels=dict(
            level6=81, level5=213, level4=181, level3=149, level2=117),
        flow_div=20.0,
        corr_cfg=dict(type='Correlation', max_displacement=4, padding=0),
        warp_cfg=dict(type='Warp', align_corners=True, use_mask=True),
        act_cfg=dict(type='LeakyReLU', negative_slope=0.1),
        scaled=False,
        post_processor=dict(type='ContextNet', in_channels=565),
        flow_loss=dict(
            type='MultiLevelEPE',
            p=1,
            q=0.4,
            eps=0.01,
            reduction='sum',
            weights=dict(
                level2=0.005,
                level3=0.01,
                level4=0.02,
                level5=0.08,
                level6=0.32))),
    train_cfg=dict(),
    test_cfg=dict(),
    init_cfg=dict(
        type='Kaiming',
        nonlinearity='leaky_relu',
        layer=['Conv2d', 'ConvTranspose2d'],
        mode='fan_in',
        bias=0))

demo.mp4 demo_gt.mp4 2_数字_蜡烛_红外光源.mp4 在下面

https://wss1.cn/f/76qh0hkwv7y

3、环境

python mmflow/utils/collect_env.py

sys.platform: linux
Python: 3.7.11 (default, Jul 27 2021, 14:32:16) [GCC 7.5.0]
CUDA available: True
CUDA_HOME: /usr/local/cuda
NVCC: Build cuda_11.2.r11.2/compiler.29618528_0
GPU 0: Quadro RTX 4000
GCC: gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
PyTorch: 1.8.0
PyTorch compiling details: PyTorch built with:
  - GCC 7.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2021.3-Product Build 20210617 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v1.7.0 (Git Hash 7aed236906b1f7a05c0917e5257a1af05e9ff683)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 10.2
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_37,code=compute_37
  - CuDNN 7.6.5
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=10.2, CUDNN_VERSION=7.6.5, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.8.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, 

TorchVision: 0.9.0
OpenCV: 4.5.3
MMCV: 1.3.15
MMFlow: 0.1.0+607e1ed
MMCV Compiler: GCC 7.3
MMCV CUDA Compiler: 10.2

Is it possible to get optical flow from the converted RGB video?

After visualizing optical flow as RGB video by visualize_flow(), is it possible to get optical flow from the converted RGB video?

Operation Error [Assertion failed) !dsize.empty() in function 'resize']

运行如下官方的demo是不报错的，证明不是环境问题

Demo running as follows is not an error, prove is not an environmental problem

from mmflow.apis import inference_model, init_model config_file = 'configs/pwcnet/pwcnet_ft_4x1_300k_sintel_final_384x768.py' checkpoint_file = 'checkpoints/pwcnet_ft_4x1_300k_sintel_final_384x768.pth' device = 'cuda:0' model = init_model(config_file, checkpoint_file, device=device) inference_model(model, 'demo/frame_0001.png', 'demo/frame_0002.png')

使用以下命令调用video demo是没问题的

It’s OK to call the video demo with the following command

python demo/video_demo.py "/home/thhicv/program/mmflow/demo/demo.mp4" 'configs/pwcnet/pwcnet_ft_4x1_300k_sintel_final_384x768.py' 'checkpoints/pwcnet_ft_4x1_300k_sintel_final_384x768.pth' /home/thhicv/program/mmflow/demo/xx1x.mp4

当我改变了视频文件就会报错

When I change the video file, it’s an error

/home/thhicv/anaconda3/envs/mmflow/bin/python /home/thhicv/program/mmflow/demo/video_demo.py /home/thhicv/视频/flow/溪丁灯.mp4 configs/pwcnet/pwcnet_ft_4x1_300k_sintel_final_384x768.py checkpoints/pwcnet_ft_4x1_300k_sintel_final_384x768.pth /home/thhicv/视频/flow/溪丁灯.mp4_flow.mp4

报错如下：

LOG：

Traceback (most recent call last): File "/home/thhicv/program/mmflow/demo/video_demo.py", line 143, in <module> main(args) File "/home/thhicv/program/mmflow/demo/video_demo.py", line 83, in main result = inference_model(model, img1, img2) File "/home/thhicv/program/mmflow/mmflow/apis/inference.py", line 115, in inference_model data = test_pipeline(data) File "/home/thhicv/program/mmflow/mmflow/datasets/pipelines/compose.py", line 42, in __call__ data = t(data) File "/home/thhicv/program/mmflow/mmflow/datasets/pipelines/transforms.py", line 428, in __call__ imgs, scale_factor = self._resize_img(imgs) File "/home/thhicv/program/mmflow/mmflow/datasets/pipelines/transforms.py", line 451, in _resize_img img_ = mmcv.imresize(img, (newW, newH), return_scale=False) File "/home/thhicv/anaconda3/envs/mmflow/lib/python3.7/site-packages/mmcv/image/geometric.py", line 89, in imresize img, size, dst=out, interpolation=cv2_interp_codes[interpolation]) cv2.error: OpenCV(4.5.3) /tmp/pip-req-build-l1r0y34w/opencv/modules/imgproc/src/resize.cpp:3688: error: (-215:Assertion failed) !dsize.empty() in function 'resize'

Good job

Good job, grab the couch!!!

Good work but there still some space to improve your code

Good job, but the coder needs to practice programming skills. You could refer to Ao Yile ([email protected])
for your programming improvement. AYL yyds

How can I replace conv2d with depthwise convolution?

where file is about this part?
thank you very much.

Some new issues for single GPU training

with this command

python tools/train.py configs/raft/raft_8x2_100k_flyingchairs_368x496.py --gpu-ids 0

The error is RuntimeError: Default process group has not been initialized, please make sure to call init_process_group.

Then I try this command

 python tools/train.py configs/raft/raft_8x2_100k_flyingchairs_368x496.py --gpu-ids 0 --launcher pytorch

The error becomes:

rank = int(os.environ['RANK'])
  File "/home/miniconda3/envs/mmflow/lib/python3.8/os.py", line 675, in __getitem__
    raise KeyError(key) from None
KeyError: 'RANK'

Then I add following code to train.py:

if 'RANK' not in os.environ:
        os.environ['RANK'] = str(args.local_rank)

The error is:
ValueError: Error initializing torch.distributed using env:// rendezvous: environment variable WORLD_SIZE expected, but not set

Then I add following code to train.py:

if 'WORLD_SIZE' not in os.environ:
        os.environ['WORLD_SIZE'] = str(1)

The error is:
ValueError: Error initializing torch.distributed using env:// rendezvous: environment variable MASTER_ADDR expected, but not set

Then I have setup the MASTER_ADDR and MASTER PORT in similar way,

I got this error:

Traceback (most recent call last):
  File "tools/train.py", line 199, in <module>
    main()
  File "tools/train.py", line 188, in main
    train_model(
  File "/home/framework/optical_flow/mmflow/mmflow/apis/train.py", line 212, in train_model
    runner.run(data_loaders, cfg.workflow)
  File "/home/miniconda3/envs/mmflow/lib/python3.8/site-packages/mmcv/runner/iter_based_runner.py", line 134, in run
    iter_runner(iter_loaders[i], **kwargs)
  File "/home/miniconda3/envs/mmflow/lib/python3.8/site-packages/mmcv/runner/iter_based_runner.py", line 67, in train
    self.call_hook('after_train_iter')
  File "/home/miniconda3/envs/mmflow/lib/python3.8/site-packages/mmcv/runner/base_runner.py", line 309, in call_hook
    getattr(hook, fn_name)(self)
  File "/home/miniconda3/envs/mmflow/lib/python3.8/site-packages/mmcv/runner/hooks/optimizer.py", line 56, in after_train_iter
    runner.outputs['loss'].backward()
  File "/home/miniconda3/envs/mmflow/lib/python3.8/site-packages/torch/_tensor.py", line 307, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/miniconda3/envs/mmflow/lib/python3.8/site-packages/torch/autograd/__init__.py", line 154, in backward
    Variable._execution_engine.run_backward(
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2, 128, 46, 62]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

Hope I have addressed the issue clear enough. Thank for your help

where is model()'s output?

i need that:
img = torch.rand(1, 6, 640,640).cuda()
output1 = model(img)
but the output is a dict,where can i modify this to a tensor?

run demo

parser.add_argument('--config',default='configs/liteflownet2/liteflownet2_ft_4x1_500k_kitti_320x896.py',  help='Config file')
parser.add_argument('--checkpoint', default='models/liteflownet2/liteflownet2_ft_4x1_500k_kitti_320x896.pth', help='Checkpoint file')

Traceback (most recent call last):
File "/home/wangzhaoming/workspace/code/mmflow/demo/image_demo.py", line 44, in
main(args)
File "/home/wangzhaoming/workspace/code/mmflow/demo/image_demo.py", line 35, in main
result = inference_model(model, 'demo/frame_0001.png', 'demo/frame_0002.png')
File "/disk/workspace/wangzhaoming/code/mmflow/mmflow/apis/inference.py", line 115, in inference_model
data = test_pipeline(data)
File "/disk/workspace/wangzhaoming/code/mmflow/mmflow/datasets/pipelines/compose.py", line 42, in call
data = t(data)
File "/disk/workspace/wangzhaoming/code/mmflow/mmflow/datasets/pipelines/loading.py", line 146, in call
results = self._load_sparse_flow(results)
File "/disk/workspace/wangzhaoming/code/mmflow/mmflow/datasets/pipelines/loading.py", line 190, in _load_sparse_flow
filenames = list(results['ann_info'].keys())
KeyError: 'ann_info'

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

PyTorch: 1.11
CudaToolKits: 11.3.1

Error occur while running this command

python tools/train.py configs/raft/raft_8x2_50k_kitti2015_288x960.py

Complete stacktrace

/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /opt/conda/conda-bld/pytorch_1646755953518/work/aten/src/ATen/native/TensorShape.cpp:2228.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/autograd/__init__.py:175: UserWarning: Error detected in ReluBackward0. Traceback of forward call that caused the error:
  File "tools/train.py", line 209, in <module>
    main()
  File "tools/train.py", line 205, in main
    meta=meta)
  File "/media/exthdd/laizeqiang/lzq/projects/misc/mmflow/mmflow/apis/train.py", line 238, in train_model
    runner.run(data_loaders, cfg.workflow)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/mmcv/runner/iter_based_runner.py", line 134, in run
    iter_runner(iter_loaders[i], **kwargs)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/mmcv/runner/iter_based_runner.py", line 61, in train
    outputs = self.model.train_step(data_batch, self.optimizer, **kwargs)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/mmcv/parallel/data_parallel.py", line 75, in train_step
    return self.module.train_step(*inputs[0], **kwargs[0])
  File "/media/exthdd/laizeqiang/lzq/projects/misc/mmflow/mmflow/models/flow_estimators/base.py", line 90, in train_step
    losses = self(**data, test_mode=False)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/media/exthdd/laizeqiang/lzq/projects/misc/mmflow/mmflow/models/flow_estimators/base.py", line 59, in forward
    return self.forward_train(*args, **kwargs)
  File "/media/exthdd/laizeqiang/lzq/projects/misc/mmflow/mmflow/models/flow_estimators/raft.py", line 107, in forward_train
    feat1, feat2, h_feat, cxt_feat = self.extract_feat(imgs)
  File "/media/exthdd/laizeqiang/lzq/projects/misc/mmflow/mmflow/models/flow_estimators/raft.py", line 74, in extract_feat
    cxt_feat = self.context(img1)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/media/exthdd/laizeqiang/lzq/projects/misc/mmflow/mmflow/models/encoders/raft_encoder.py", line 296, in forward
    x = res_layer(x)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/nn/modules/container.py", line 141, in forward
    input = module(input)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/media/exthdd/laizeqiang/lzq/projects/misc/mmflow/mmflow/models/utils/res_layer.py", line 88, in forward
    out = _inner_forward(x)
  File "/media/exthdd/laizeqiang/lzq/projects/misc/mmflow/mmflow/models/utils/res_layer.py", line 76, in _inner_forward
    out = self.relu(out)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/nn/modules/activation.py", line 98, in forward
    return F.relu(input, inplace=self.inplace)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/nn/functional.py", line 1442, in relu
    result = torch.relu(input)
 (Triggered internally at  /opt/conda/conda-bld/pytorch_1646755953518/work/torch/csrc/autograd/python_anomaly_mode.cpp:104.)
  allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
Traceback (most recent call last):
  File "tools/train.py", line 209, in <module>
    main()
  File "tools/train.py", line 205, in main
    meta=meta)
  File "/media/exthdd/laizeqiang/lzq/projects/misc/mmflow/mmflow/apis/train.py", line 238, in train_model
    runner.run(data_loaders, cfg.workflow)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/mmcv/runner/iter_based_runner.py", line 134, in run
    iter_runner(iter_loaders[i], **kwargs)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/mmcv/runner/iter_based_runner.py", line 67, in train
    self.call_hook('after_train_iter')
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/mmcv/runner/base_runner.py", line 309, in call_hook
    getattr(hook, fn_name)(self)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/mmcv/runner/hooks/optimizer.py", line 56, in after_train_iter
    runner.outputs['loss'].backward()
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/_tensor.py", line 363, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
  File "/home/laizeqiang/miniconda3/envs/openmmlab/lib/python3.7/site-packages/torch/autograd/__init__.py", line 175, in backward
    allow_unreachable=True, accumulate_grad=True)  # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2, 128, 36, 120]], which is output 0 of ReluBackward0, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

unsupervised fine tuning

Hi
I wonder if it's possible to fine tune a pre-trained model on a dataset w/o flow labels, i.e., mixing supervised optical flow with unsupervised learning?

Thanks,
Alex

Reopening - links broken again?

Originally posted by @jhlusko in #12 (comment)

Any plans on scene flow?

Would you provide support for scene flow algorithms like FlowNet3D, RAFT-3D, HPLFlowNet, PointPWC, etc?

Transfer learning: feature extraction

Describe the bug
Good evening, i was wondering if some high level functions are provided to perform feature extraction on a model as RAFT.
I'd like to load the pretrained model on Things3d+Chairs and update only the final layer of it.

[GLCC] GitLink Code Camp (首届 CCF GitLink 开源编程夏令营)

This issue will participate in the chinese GLCC activity.

题目名称

基于MMFlow，实现 flow1d 算法（ICCV 2021 oral）

题目描述

题目简介

MMFlow是一款基于 PyTorch 和 MMCV 的光流估计开源工具箱，提供了多个 SOTA 光流估计算法，并支持光流领域主流学术数据集，以及光流可视化和评估方法。

通过在 MMFlow 中实现 flow1d 算法模型，你将可以了解光流领域的前沿知识，熟练掌握 MM 系列框架的整体流程，并积累深度学习相关的学术及工程经验。

编码任务

在 MMFlow 中实现flow1d的算法模型，并复现其训练精度。

技能要求和编程语言

Python
一定的论文阅读能力
光流估计算法的相关经验

预期成果

复现flow1d论文精度
作为算法库新特性发版

导师及联系方式

GitHub id: @MeowZheng
Email: [email protected]
对活动感兴趣的同学欢迎 OpenMMLab x GitLink 编程夏令营群聊

error info:
ImportError: cannot import name 'Correlation' from 'mmcv.ops'

test

Thanks for your error report and we appreciate it a lot.

Checklist

I have searched related issues but cannot get the expected help.
I have read the FAQ documentation but cannot get the expected help.
The bug has not been fixed in the latest version.

Describe the bug
A clear and concise description of what the bug is.

Reproduction

What command or script did you run?

A placeholder for the command.

Did you make any modifications on the code or config? Did you understand what you have modified?
What dataset did you use?

Environment

Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback
If applicable, paste the error trackback here.

A placeholder for trackback.

	if self.scaled:

	if 'sqrt' in self.scale_mode:
	scale_factor = sqrt(float(C * self.kernel_size**2))
	else:
	scale_factor = float(C * self.kernel_size**2)

	corr = self.corr_block[0](feat1, feat2) / scale_factor

open-mmlab / mmflow Goto Github PK

mmflow's People

Contributors

Stargazers

Watchers

Forkers

mmflow's Issues

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

1、故障描述（Description）

2、复现步骤（Reproduction procedure）

出现错误

涉及的文件

demo.mp4 demo_gt.mp4 2_数字_蜡烛_红外光源.mp4 在下面

3、环境

python mmflow/utils/collect_env.py

运行如下官方的demo是不报错的，证明不是环境问题

Demo running as follows is not an error, prove is not an environmental problem

使用以下命令调用video demo是没问题的

It’s OK to call the video demo with the following command

当我改变了视频文件就会报错

When I change the video file, it’s an error

报错如下：

LOG：

题目名称

题目描述

题目简介

编码任务

技能要求和编程语言

预期成果

导师及联系方式

Recommend Projects

Recommend Topics

Recommend Org

Jobs