AttributeError: 'Tensor' object has no attribute 'depth_map' about panoptic-reconstruction

[macromogic]

I changed depth.depth_map to depth and, yes, the code did complain. 🙂 Below are the problems I met and solutions I adopted for myself:

1. Shape mismatch when calculating loss in DepthPrediction.forward() (in file lib/modeling/depth_prediction.py)

   The author might not realized that the parameter depth_target has been shadowed by the local variable. In short there is no need to do another unsqueezing for valid_masks.

   Just modify the following lines:

   ## Before:
   valid_masks = torch.stack([(depth.depth_map != 0.0).bool() for depth in depth_target], dim=0)
   valid_masks.unsqueeze_(1)
   
   ## After:
   valid_masks = torch.stack([(depth != 0.0).bool() for depth in depth_target], dim=0)
   # valid_masks.unsqueeze_(1)

2. Device mismatch in the RPN module in GeneralizedRCNN.forward() (in file lib/modeling/detector/generalized_rcnn.py):

   This error occurs because the instance ground truth is not put on the GPU. Change the following line:

   ## Before:
   bounding_boxes_gt = [target.get_field("instance2d") for target in targets]
   
   ## After:
   bounding_boxes_gt = [target.get_field("instance2d").to(config.MODEL.DEVICE) for target in targets]

3. TypeError: 'module' object is not callable (for function smooth_l1_loss from module lib.layers):

   This function is defined in lib/layers/smooth_l1_loss.py, but the author forgot to export it in lib/layers/__init__.py. Adding the following line in the latter file solves the problem:

   from .smooth_l1_loss import smooth_l1_loss

4. Shape mismatch when performing instance masking in UNetBlockHybridSparse.forward() (in file lib/modeling/backbone/unet_sparse.py):

   In my case, the network raised the error here:

   instance_prediction = torch.masked_fill(instance_prediction, frustum_mask.squeeze() == False, 0.0)

   Then I figured out that the shape of instance_prediction is [batch_size, config.MODEL.INSTANCE2D.MAX+1, 64, 64, 64], but for frustum_mask, it is [batch_size, 64, 64, 64]. If batch_size does not equal to 1 or config.MODEL.INSTANCE2D.MAX+1 (16 by default), the broadcasting fails. I resolved it by setting batch_size (i.e., config.DATALOADER.IMS_PER_BATCH) to 1. Other feasible solutions might be setting it to 16, or make a temporary unsqueezing for frustum_mask on dimension 1.

Hope my information helps!

[unlugi]

I couldn't train the 2D part of the network from scratch either. Observed the exact error.
The provided code does not work for training the all parts of the pipeline.

Maybe changing depth.depth_map to depth might work. But then the rest of the code might complain.

[xheon]

Hi all, sorry for the delay - I just returned from travelling. I will have a look at this issue the next days.

Thanks