Black cropped image is output when detect object in object detection about acap-native-sdk-examples

[Corallo]

Hello @takuya-takeuchi
It has been a while, but we are working on it. We are going to publish a fix probably this week.

[Corallo]

Hello @takuya-takeuchi
We need to make more tests before merging the fix as we noticed other problems that we need to investigate.
In the meantime, I created a branch that you can try out. it should have the issue with the black cropped image solved.

[pataxis]

Hi @takuya-takeuchi , the problem should have been fixed and is now on main branch. Thanks for reporting!

[takuya-takeuchi]

I confirmed issue is gone away.
Thank you for your kindly support

[pataxis]

Hi @takuya-takeuchi , thanks for your report, we'll get back to you when we had time to look at this issue.

[Corallo]

Hi @takuya-takeuchi

Before we dig into this issue, it would be good if you can try with a newer firmware.
Between 10.12 and 11.0 there have been some breaking changes, and the examples are meant to work on the latest firmware.

Search here:
https://www.axis.com/support/firmware
For the firmware for your Q1656 and try with the latest (should be 11.5).

[takuya-takeuchi]

@Corallo
Thank you for your quick response and advice.
I updated firmeware to 11.5.64 and rebuild app on axisecp/acap-native-sdk:1.9-aarch64-ubuntu22.04.
But problem is not resolved.

This is object detection's log.

----- Contents of SYSTEM_LOG for 'object_detection' -----


2023-07-19T08:42:53.671+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Starting /usr/local/packages/object_detection/object_detection
2023-07-19T08:42:54.005+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'buffer.strategy': <uint32 3>
2023-07-19T08:42:54.005+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'channel'--------: <uint32 1>
2023-07-19T08:42:54.005+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'format'---------: <uint32 3>
2023-07-19T08:42:54.005+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'height'---------: <uint32 300>
2023-07-19T08:42:54.005+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'width'----------: <uint32 480>
2023-07-19T08:42:54.005+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Creating VDO image provider and creating stream 480 x 300
2023-07-19T08:42:54.005+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Dump of vdo stream settings map =====
2023-07-19T08:42:54.005+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: chooseStreamResolution: We select stream w/h=480 x 300 based on VDO channel info.
2023-07-19T08:42:54.348+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'buffer.strategy': <uint32 3>
2023-07-19T08:42:54.348+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'channel'--------: <uint32 1>
2023-07-19T08:42:54.348+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'format'---------: <uint32 3>
2023-07-19T08:42:54.348+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'height'---------: <uint32 1080>
2023-07-19T08:42:54.348+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 'width'----------: <uint32 1920>
2023-07-19T08:42:54.348+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Creating VDO raw image provider and stream 1920 x 1080
2023-07-19T08:42:54.348+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Dump of vdo stream settings map =====
2023-07-19T08:42:54.695+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Calculate crop image
2023-07-19T08:42:54.695+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Create larod models
2023-07-19T08:42:54.695+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Create preprocessing maps
2023-07-19T08:42:54.695+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Crop VDO image X=90 Y=0 (300 x 300)
2023-07-19T08:42:54.695+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Setting up larod connection with chip axis-a8-dlpu-tflite, model /usr/local/packages/object_detection/model/converted_model.tflite and label file /usr/local/packages/object_detection/label/labels.txt
2023-07-19T08:42:54.746+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Available chip IDs:
2023-07-19T08:42:54.746+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Chip: axis-a8-dlpu-native
2023-07-19T08:42:54.746+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Chip: axis-a8-dlpu-proc
2023-07-19T08:42:54.746+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Chip: axis-a8-dlpu-tflite
2023-07-19T08:42:54.746+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Chip: axis-a8-gpu-proc
2023-07-19T08:42:54.746+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Chip: axis-ace-proc
2023-07-19T08:42:54.746+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Chip: cpu-proc
2023-07-19T08:42:54.746+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Chip: cpu-tflite
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Allocate memory for input/output buffers
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Create input/output tensors
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Determine tensor buffer sizes
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Loading preprocessing model with chip cpu-proc
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: createAndMapTmpFile: Setting up a temp fd with pattern /tmp/crop.test-XXXXXX and size 6220800
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: createAndMapTmpFile: Setting up a temp fd with pattern /tmp/larod.in.test-XXXXXX and size 270000
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: createAndMapTmpFile: Setting up a temp fd with pattern /tmp/larod.out1.test-XXXXXX and size 320
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: createAndMapTmpFile: Setting up a temp fd with pattern /tmp/larod.out2.test-XXXXXX and size 80
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: createAndMapTmpFile: Setting up a temp fd with pattern /tmp/larod.out3.test-XXXXXX and size 80
2023-07-19T08:43:29.091+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: createAndMapTmpFile: Setting up a temp fd with pattern /tmp/larod.pp.test-XXXXXX and size 216000
2023-07-19T08:43:29.093+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Connect tensors to file descriptors
2023-07-19T08:43:29.093+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Create job requests
2023-07-19T08:43:29.093+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Found 1 input tensors and 4 output tensors
2023-07-19T08:43:29.093+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Set input tensors
2023-07-19T08:43:29.093+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Set output tensors
2023-07-19T08:43:29.093+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Start fetching video frames from VDO
2023-07-19T08:43:29.093+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: createAndMapTmpFile: Setting up a temp fd with pattern /tmp/larod.out4.test-XXXXXX and size 4
2023-07-19T08:43:29.100+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Converted image in 9 ms
2023-07-19T08:43:29.120+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 1: Object 0: Classes: person - Scores: 0.839844 - Locations: [0.418787,0.001991,0.887607,0.304403] [3,452,580,506]
2023-07-19T08:43:29.120+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Ran inference for 19 ms
2023-07-19T08:43:29.145+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: buffer_to_jpeg
2023-07-19T08:43:29.145+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: rawWidth: 1920, rawHeight: 1080, args.quality: 80
2023-07-19T08:43:29.271+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 2: Object 1: Classes: person - Scores: 0.617188 - Locations: [0.003676,0.022635,0.996324,0.763701] [43,3,1422,1072]
2023-07-19T08:43:29.271+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: free
2023-07-19T08:43:29.271+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: jpeg_to_file
2023-07-19T08:43:29.372+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Converted image in 3 ms
2023-07-19T08:43:29.391+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 3: Object 0: Classes: person - Scores: 0.722656 - Locations: [0.012660,0.038245,0.987340,0.765896] [73,13,1397,1052]
2023-07-19T08:43:29.391+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: Ran inference for 19 ms
2023-07-19T08:43:29.485+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: buffer_to_jpeg
2023-07-19T08:43:29.485+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: rawWidth: 1920, rawHeight: 1080, args.quality: 80
2023-07-19T08:43:29.602+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: 4: Object 1: Classes: person - Scores: 0.558594 - Locations: [0.397288,0.001052,0.910682,0.267301] [2,429,511,554]
2023-07-19T08:43:29.602+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: free
2023-07-19T08:43:29.602+09:00 axis-b8a44f7b9f0b [ INFO    ] object_detection[8172]: jpeg_to_file

This has some moification to output yuv (480x300) and original image (1920x1080) but it should not affect to this issue.
yuv is correct but original and cropped images are blacks.

[takuya-takeuchi]

FYI
I just tried original object detection but detected image is still black.
A revison of source code is dbb91e9.

[takuya-takeuchi]

@Corallo
Hi, do you have any information to resolve issue?
Thank you for your cooperations.

[marbali8]

Hello,
we will look into the issue as soon as we can and let you know :)

[takuya-takeuchi]

Any update?