capture_set_color_image with K4A_IMAGE_FORMAT_COLOR_MJPG about pyk4a

[Westerby]

I got it to work by modifying the numpy_to_k4a_image to support K4A_IMAGE_FORMAT_COLOR_MJPG datatype.
For my usage width_pixels and height_pixels parameters are just hardcoded there for MJPG data in the snippet below (requires refactor).

k4a_result_t numpy_to_k4a_image(PyArrayObject *img_src, k4a_image_t *img_dst, k4a_image_format_t format) {
  
  int width_pixels = img_src->dimensions[1];
  int height_pixels = img_src->dimensions[0];
  int pixel_size;
  int stride_bytes; 
  int buffer_size; 

  switch (format) {
  case K4A_IMAGE_FORMAT_DEPTH16:
  case K4A_IMAGE_FORMAT_CUSTOM16:
  case K4A_IMAGE_FORMAT_IR16:
    pixel_size = (int)sizeof(uint16_t);
    stride_bytes = width_pixels * pixel_size;
    buffer_size = width_pixels * height_pixels * pixel_size;
    break;
  case K4A_IMAGE_FORMAT_COLOR_BGRA32:
    pixel_size = (int)sizeof(uint32_t);
    stride_bytes = width_pixels * pixel_size;
    buffer_size = width_pixels * height_pixels * pixel_size;
    break;
  case K4A_IMAGE_FORMAT_CUSTOM8:
    pixel_size = (int)sizeof(uint8_t);
    stride_bytes = width_pixels * pixel_size;
    buffer_size = width_pixels * height_pixels * pixel_size;
    break;
  case K4A_IMAGE_FORMAT_COLOR_MJPG:
    pixel_size = (int)sizeof(uint8_t);
    stride_bytes = 0;
    buffer_size = width_pixels * height_pixels * pixel_size;
    width_pixels = 1920;
    height_pixels = 1080;

    break;
  default:
    // Not supported
    return K4A_RESULT_FAILED;
  }

  return k4a_image_create_from_buffer(format, width_pixels, height_pixels, stride_bytes,
                                      (uint8_t *)img_src->data, buffer_size, NULL, NULL,
                                      img_dst);
}

I think that the best way to pass them down would be to add another parameter to k4a_module.capture_set_color_image call in color.setter for color_resolution. That will require further change to numpy_to_k4a_image - as in case of K4A_IMAGE_FORMAT_COLOR_MJPG color resolution is not the same as img_src->dimensions.

Tested the setter for BGRA and MJPG data.

Would any of the authors be interested in checking if that's the right approach?

[rajkundu]

I'm not sure if this helps, but perhaps you need to decode the compressed MJPG data into BGRA32 format first, e.g., using array_color = cv2.cvtColor(cv2.imdecode(capture.color, cv2.IMREAD_COLOR), cv2.COLOR_BGR2BGRA)? I ran into similar issues with MJPG vs. BGRA32, so perhaps this comment/issue could help: #164 (comment)