Hi @shangwei5 , sorry to disturb you again. I have some questions remaining. Could you offer me some explanations?

• Q1: how to use events as input in BiLSTM detector?
  As shown in here, the first conv layer has 3 channels, which seems that it is designed for RGB frame. However, BiLSTM_resnet152 has a 20-channel-input conv layer, I wonder if I could use this layer to take event as input.
• Q2: the representation of events in BiLSTM detector
  It seems that the representation code that you offered has 40 channels. I am not sure if it is the representation for Event-BiLSTM detector.
  Thanks for your time.

Hi @shangwei5 , nice work for image deblurring. It is a clever idea to utilize existing adjacent sharp frame near to the blurry image.

However, there are a few questions confusing me a lot. I hope that you can provide me some advice thx.

• a mismatch between equation 4 and your warping code

According to the equation 4 in your paper, by using forward optical flow , which means the displacement from NSF to blurry img , you align with . It did make sense. However, as far as I know, most warping functions use backward warping, which means using the optical flow from to and warp to the coordinate of target frame .

From my perspective, the warping function in your code is backward warping, https://github.com/shangwei5/D2Net/blob/76a44beab13c2b8e0d31aef89b3a3a60e691f84f/code/model/flow_pwc.py#L56 however, the optical flows are not the corresponding ones. So is equation 4 in your paper. Chances are that I have got it wrong. Please help me to understand it better, THX!

• a question of in equation 4

It seems that it should be in the second row of equation 4. I don't know if there is something wrong with my pdf reader.

• a question about synthesizing blur

In your setting of deblurring, the number of sharp images to average varies from 1 to 15 in GoPro. the generated frame is still sharp despite the number of averaging frames being 5. However, it seems that you didn't interpolate imgs between current sharp frames. I don't know if the generated frame keeps sharp after averaging the 5 original sharp frames and the interpolated imgs between them. If following this setting, I wonder if D2net can still alleviate the neighboring sharp frame.

Some outputs: when i execute python main_d2net_event.py:

DataSet INPUT path: /home/featurize/data/event_deblur/test/blur
DataSet label path: /home/featurize/data/event_deblur/test/label
Number of videos to load: 11
Number of frames to load: 1633
Using Trainer-D2Net
Now training
/environment/miniconda3/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:134: UserWarning: Detected call of lr_scheduler.step() before optimizer.step(). In PyTorch 1.1.0 and later, you should call them in the opposite order: optimizer.step() before lr_scheduler.step(). Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
"https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
/environment/miniconda3/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:370: UserWarning: To get the last learning rate computed by the scheduler, please use get_last_lr().
"please use get_last_lr().", UserWarning)
Epoch 1 with Lr 1.00e-4
Traceback (most recent call last):
File "main_d2net_event.py", line 21, in
t.train()
File "/home/featurize/work/D2Net/code/trainer/trainer_d2net_event.py", line 36, in train
for batch, (input, gt, event, label, _) in enumerate(self.loader_train):
ValueError: not enough values to unpack (expected 5, got 4)

but when i try main_d2net.py, I can train it good. So I guess there's something wrong in event_d2net's dataloader?

Hello!
Thanks for the nice work!
I really appreciate it.
If it doesn't bother you, can you share the dataset, model files, ... in Google drive format, not the Baidu link?
I'm from South Korea, and I have difficulties in downloading the uploaded files.
Thank you!

Hi @shangwei5 , sorry to disturb you again.
As for Table 1 and Table 2, the first one is the comparison at normal setting, which means the input frames are all blurry as same as the previous works, and the latter table is the comparison at non-consecutively blurry setting, which means the input frames consist of sharp frames and blurry frames.
However, in spite of training all models in non-consecutively blurry dataset, the performance in Table2 is not as good as the performance in Table1. The performance gap between D2NET and other models is smaller at non-consecutively blurry setting, as shown in the fig below .

Chances are that at non-consecutively blurry setting, debluring is more challenging. Hope you can help me to understand this phenomenon better.

Hi, it's a good job!
would you please provide the D2Net pretrain model?

Thanks

Hi, Thanks for your contribution and your great work. I am curious about the Blur-DVS dataset, an excellent real scene event camera dataset. Do you have any plan to release the Blur-DVS dataset? Thanks first.

Hi, do you have the event data for the consecutively blurry GOPRO dataset.