pinto0309 / headposeestimation-whenet-yolov4-onnx-openvino

WHENet - ONNX, OpenVINO, TFLite, TensorRT, EdgeTPU, CoreML, TFJS, YOLOv4/YOLOv4-tiny-3L

HeadPoseEstimation-WHENet-yolov4-onnx-openvino

ONNX, OpenVINO, TFLite, TensorRT, EdgeTPU, CoreML, TFJS, YOLOv4/YOLOv4-tiny-3L

• PINTO Special Custom Model https://github.com/PINTO0309/DMHead

  184784102-089a82b9-765a-4431-bf33-43370b5c8174.mp4

1. Usage

$ git clone https://github.com/PINTO0309/HeadPoseEstimation-WHENet-yolov4-onnx-openvino
$ cd HeadPoseEstimation-WHENet-yolov4-onnx-openvino
$ wget https://github.com/PINTO0309/HeadPoseEstimation-WHENet-yolov4-onnx-openvino/releases/download/v1.0.3/saved_model_224x224.tar.gz
$ tar -zxvf saved_model_224x224.tar.gz && rm saved_model_224x224.tar.gz
$ wget https://github.com/PINTO0309/HeadPoseEstimation-WHENet-yolov4-onnx-openvino/releases/download/v1.0.4/whenet_1x3x224x224_prepost.onnx
$ mv whenet_1x3x224x224_prepost.onnx saved_model_224x224/

$ python3 demo_video.py

usage: demo_video.py \
[-h] \
[--whenet_mode {onnx,openvino}] \
[--device DEVICE] \
[--height_width HEIGHT_WIDTH]

optional arguments:
  -h, --help
      show this help message and exit
  --whenet_mode {onnx,openvino}
      Choose whether to infer WHENet with ONNX or OpenVINO. Default: onnx
  --device DEVICE
      Path of the mp4 file or device number of the USB camera. Default: 0
  --height_width HEIGHT_WIDTH
      {H}x{W} Default: 480x640

2. Reference

1. https://github.com/Ascend-Research/HeadPoseEstimation-WHENet
2. https://github.com/AlexeyAB/darknet
3. https://github.com/jkjung-avt/yolov4_crowdhuman
4. https://github.com/linghu8812/tensorrt_inference/tree/master/Yolov4
5. https://github.com/Tianxiaomo/pytorch-YOLOv4
6. https://github.com/PINTO0309/PINTO_model_zoo
7. https://github.com/PINTO0309/openvino2tensorflow
8. Exporting WHENet
9. Darknet to ONNX to OpenVINO to TensorFlow to TFLite and Others
10. Dual model head pose estimation. Fusion of SOTA models. 360° 6D HeadPose detection. All pre-processing and post-processing are fused together, allowing end-to-end processing in a single inference. 6DRepNet+WHENet

3. Special Custom Model Structure

headposeestimation-whenet-yolov4-onnx-openvino's Issues

invalid dimensions

I tried to transform keras model to onnx, and this is my code,

onnx_model = keras2onnx.convert_keras(self.model, self.model.name)
temp_model_file = './mymodel.onnx'
onnx.save_model(onnx_model, temp_model_file)

However, when I tried to run my onnx model, I got a crash and error code is like this,

onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: input_1 for the following indices
 index: 1 Got: 3 Expected: 224
 index: 3 Got: 224 Expected: 3
 Please fix either the inputs or the model.

I also noticed your onnx model runs successfully, coud you please tell me how do you solve this problem?

yaw will output nan or inf after some infers with trt model

Hello ,Sir,

I have tested, the input of network has no problem.
With one single image, this trt model infers with different results of yaw angle on different batches, while the roll and pitch keep the same.
When this model infers on webcam, the first almost ten batches are fully ok, but after that, the yaw results turn out inf or nan, while the roll and pitch also are ok.

This question might be somewhat too specific, I would appreciate a lot if you can give me some hints about this.

Thank you!

trainning

Could you provide the training code for this model?

Thank you

any script to convert to trt with dynamic batch size?

The accuracy that can be achieved at present

thank you