rizwanmunawar / yolov7-pose-estimation

YOLOv7 Pose estimation using OpenCV, PyTorch

yolov7-pose-estimation

New Features

• Added Support for Comparision of (FPS & Time) Graph
• How to run Code in Google Colab
• Code can run on Both (CPU & GPU)
• Video/WebCam/External Camera/IP Stream Supported

Coming Soon

• Development of streamlit dashboard for Pose-Estimation

Steps to run Code

• If you are using google colab then you will first need to mount the drive with mentioned command first, (Windows or Linux users) both can skip this step.

from google.colab import drive
drive.mount("/content/drive")

• Clone the repository.

git clone https://github.com/RizwanMunawar/yolov7-pose-estimation.git

• Goto the cloned folder.

cd yolov7-pose-estimation

• Create a virtual envirnoment (Recommended, If you dont want to disturb python packages)

### For Linux Users
python3 -m venv psestenv
source psestenv/bin/activate

### For Window Users
python3 -m venv psestenv
cd psestenv
cd Scripts
activate
cd ..
cd ..

• Upgrade pip with mentioned command below.

pip install --upgrade pip

• Install requirements with mentioned command below.

pip install -r requirements.txt

• Download yolov7 pose estimation weights from link and move them to the working directory {yolov7-pose-estimation}

• Run the code with mentioned command below.

python pose-estimate.py

#if you want to change source file
python pose-estimate.py --source "your custom video.mp4"

#For CPU
python pose-estimate.py --source "your custom video.mp4" --device cpu

#For GPU
python pose-estimate.py --source "your custom video.mp4" --device 0

#For View-Image
python pose-estimate.py --source "your custom video.mp4" --device 0 --view-img

#For LiveStream (Ip Stream URL Format i.e "rtsp://username:pass@ipaddress:portno/video/video.amp")
python pose-estimate.py --source "your IP Camera Stream URL" --device 0 --view-img

#For WebCam
python pose-estimate.py --source 0 --view-img

#For External Camera
python pose-estimate.py --source 1 --view-img

• Output file will be created in the working directory with name ["your-file-name-without-extension"+"_keypoint.mp4"]

RESULTS

Football Match Pose-Estimation	Cricket Match Pose-Estimation	FPS and Time Comparision	Live Stream Pose-Estimation

References

• https://github.com/WongKinYiu/yolov7
• https://github.com/augmentedstartups/yolov7
• https://github.com/augmentedstartups
• https://learnopencv.com/yolov7-object-detection-paper-explanation-and-inference/
• https://github.com/ultralytics/yolov5

My Medium Articles

• https://medium.com/augmented-startups/yolov7-training-on-custom-data-b86d23e6623
• https://medium.com/augmented-startups/roadmap-for-computer-vision-engineer-45167b94518c
• https://medium.com/augmented-startups/yolor-or-yolov5-which-one-is-better-2f844d35e1a1
• https://medium.com/augmented-startups/train-yolor-on-custom-data-f129391bd3d6
• https://medium.com/augmented-startups/develop-an-analytics-dashboard-using-streamlit-e6282fa5e0f

yolov7-pose-estimation's Issues

try to convert the pytorch model to tflite

i tried to convert the pytorch model to onnx model but i am not able to load the architecture and not able to convert it into onnx and tflite..........

[Modifications] Plotting x, y displacement and replacing bounding box with a circle ?

Hi, first of all great work done for combining pose estimation and object tracking. I am trying to modify the implementation to basically plot

1. Heat map of few of the tracked objects to have some kind of the birds eye view . The idea was to plot if per frame and to have it visualized on top of the video stream (like realtime analysis) . Something like attached in the image itself .
2. Is it possible to draw instead of a rectangular bounding box , some kind of a circular disc on the foot of the players for better and easier visualization?

Could you please provide some guidelines on it ?

Result streaming is very slow while running rtps live camera

I have been running that yolov7 pose estimation and the model is working fine and the keypoints are marked properly but while streaming the result the video is very much slow

Can't seem to find information on keypoint landmark order

I want to use specific body landmark keypoints such as only the left and right shoulders from the predicted keypoints. However, I am unable to find consistent information anywhere on what is the order in which landmarks are arranged. They change depending on which website I refer to. Can anyone please point me to the right reliable resource for it?

Thanks.

Not working for me with other video files

KeyError: 'nkpt'

!python pose-estimate.py --source "football1.mp4" --device 0

Optimizer stripped from yolov7-w6-person.pt, 280.0MB
Fusing layers...
IDetect.fuse
Frame 0 Processing

KeyError Traceback (most recent call last)
Input In [7], in <cell line: 167>()
164 device='cpu' #'cpu/0,1,2,3(gpu)'
166 strip_optimizer(device,poseweights)
--> 167 run(poseweights,source, device)

File E:\anaconda3\envs\sleap\lib\site-packages\torch\autograd\grad_mode.py:28, in _DecoratorContextManager.call..decorate_context(*args, **kwargs)
25 @functools.wraps(func)
26 def decorate_context(*args, **kwargs):
27 with self.class():
---> 28 return func(*args, **kwargs)

Input In [7], in run(poseweights, source, device)
94 output, _ = model(image)
96 #Apply non max suppression
---> 97 output = non_max_suppression_kpt(output, 0.25, 0.65, nc=model.yaml['nc'], nkpt=model.yaml['nkpt'], kpt_label=True)
98 output = output_to_keypoint(output)
99 im0 = image[0].permute(1, 2, 0) * 255

KeyError: 'nkpt'

Potentially redundant code in image-> tensor conversion?

I was a bit confused by the lines 62,63 in your code line 62

Seems like you are converting from numpy to tensor, back to numpy, and then back to tensor. Wouldn't there by an easier way to do this?

    image = transforms.ToTensor()(image)
    image = torch.tensor(np.array([image.numpy()]))

Would this achieve the same?

image = torch.tensor(np.expand_dims(np.rollaxis(image,2)/255.0, -1))

Also is the first dimension of the image, batch? Could we technically pass multiple images here?

About the training dataset and code

Amazing repo!
But I can't find the training dataset and code, only the inferenece code,
How long will be the dataset and training code are released?
Thanks~

how to train our own pose classification model?

how to convert the coco format into our custom keypoint classification format?

can i train myself datasets？

can i train myself datasets？
not person
it about airplane and have 6 keypoint

can't view img

there is an error happended at imshow in pose-estimate.py line 105

change a video will get a error, but its works fine with the original video

(yolov5) user@user-SYS-7048GR-TR:~/ztj_files/yolov7-pose-estimation-main$ python pose-estimate.py --source smoking_test.mp4
Optimizer stripped from yolov7-w6-pose.pt, 161.1MB
Fusing layers...
/home/user/anaconda3/envs/yolov5/lib/python3.7/site-packages/torch/functional.py:445: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1639180594101/work/aten/src/ATen/native/TensorShape.cpp:2157.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
Frame 1 Processing
Traceback (most recent call last):
File "pose-estimate.py", line 161, in
main(opt)
File "pose-estimate.py", line 156, in main
run(**vars(opt))
File "/home/user/anaconda3/envs/yolov5/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
return func(*args, **kwargs)
File "pose-estimate.py", line 71, in run
output_data, _ = model(image)
File "/home/user/anaconda3/envs/yolov5/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/user/ztj_files/yolov7-pose-estimation-main/models/yolo.py", line 514, in forward
return self.forward_once(x, profile) # single-scale inference, train
File "/home/user/ztj_files/yolov7-pose-estimation-main/models/yolo.py", line 540, in forward_once
x = m(x) # run
File "/home/user/anaconda3/envs/yolov5/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/user/ztj_files/yolov7-pose-estimation-main/models/common.py", line 62, in forward
return torch.cat(x, self.d)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 39 but got size 40 for tensor number 1 in the list.

Loading output data to other usage

Dear Rizwan,

Thank you for this supertight piece of code!
It was great to get it running in notime.

I want to load out the capturepoints of the skeleton and use them in touchdesigner for a small testproject.
So I'm deciphering the shape and steps of the output array.

Any chance you can provide some hints?

Thanks a lot!

How can I train model with my own dataset?

The performance of this model is so eye-catching. I wanna finetune the model by using my own dataset.So, How can I train the model with my own dataset?

能提供tiny版本的權重檔嗎?

问题标题 File Not Found

there is no file named "yolov7-w6-pose.pt"(in pose-estimate.py line 15)

Keypoints extraction

How do i extract keypoints file . I intend to extract the keypoints for each ID and use these keypoints for action recognition using a classifier

hello. I use web cam have some probleam. how can i adjust

VideoCapture resize error

I resized the frame size in pose-estimate.py as below.

orig_image = cv2.resize(frame, dsize=(480, 480))

But, the following error occured.

$ python3 pose-estimate.py --device 0
Optimizer stripped from yolov7-w6-pose.pt, 161.1MB
Fusing layers... 
Frame 0 Processing
Traceback (most recent call last):
  File "pose-estimate.py", line 172, in <module>
    main(opt)
  File "pose-estimate.py", line 167, in main
    run(**vars(opt))
  File "/home/ishihara/.local/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "pose-estimate.py", line 96, in run
    output = model(image)
  File "/home/ishihara/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ishihara/yolov7-pose-estimation/models/yolo.py", line 514, in forward
    return self.forward_once(x, profile)  # single-scale inference, train
  File "/home/ishihara/yolov7-pose-estimation/models/yolo.py", line 540, in forward_once
    x = m(x)  # run
  File "/home/ishihara/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ishihara/yolov7-pose-estimation/models/common.py", line 111, in fuseforward
    return self.act(self.conv(x))
  File "/home/ishihara/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/ishihara/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 443, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/ishihara/.local/lib/python3.8/site-packages/torch/nn/modules/conv.py", line 439, in _conv_forward
    return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Input type (torch.cuda.HalfTensor) and weight type (torch.cuda.FloatTensor) should be the same

RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 19 but got size 20 for tensor number 1 in the list.

Hi. I'm trying to get your repo to work, but all videos I pass in (except football1.mp4) fail with the following error:

$ cd yolov7-pose-estimation && python pose-estimate.py --source ../1411739551.mp4 --device cpu
Optimizer stripped from yolov7-w6-pose.pt, 321.8MB
Fusing layers... 
/usr/local/lib/python3.8/dist-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3190.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Frame 1 Processing
Traceback (most recent call last):
  File "pose-estimate.py", line 160, in <module>
    main(opt)
  File "pose-estimate.py", line 155, in main
    run(**vars(opt))
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "pose-estimate.py", line 70, in run
    output_data, _ = model(image)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/yolov7-pose-estimation/models/yolo.py", line 514, in forward
    return self.forward_once(x, profile)  # single-scale inference, train
  File "/content/yolov7-pose-estimation/models/yolo.py", line 540, in forward_once
    x = m(x)  # run
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/yolov7-pose-estimation/models/common.py", line 62, in forward
    return torch.cat(x, self.d)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 19 but got size 20 for tensor number 1 in the list.

I'm running this on a Colab instance. The setup seems correct, because it does work for mp4.

I've tried reencoding a video to .png files and back to .mp4, but to no avail.

Any ideas?

Could you share your code for training?

The Pose-estimation result is amazing! Could you share your code for training? such as train.py, etc.

No such file or directory: 'yolov7-w6-pose.pt'

Dear Razwan,

Thank again. And this time... a true small error.
As I said, probably on my side, but I wouldn't know where to begin.
It seems to be an issue with loading the weights into torch.

Here the full log:

File "C:\Users\josse\yolov7-pose-estimation\pose-estimate.py", line 202, in
strip_optimizer(opt.device,opt.poseweights)
File "C:\Users\josse\yolov7-pose-estimation\utils\general.py", line 797, in strip_optimizer
with _open_file_like(f, 'rb') as opened_file:
File "C:\Users\josse\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 231, in _open_file_like
return _open_file(name_or_buffer, mode)
File "C:\Users\josse\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\serialization.py", line 212, in init
super(_open_file, self).init(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'yolov7-w6-pose.pt'

I hope you have a clue!

Can I resize video before model(image)?

Multi class detection with variable number of keypoints (nkpt)

Hi,

I am researching how to custom train yolov7-pose on my own dataset with 5 different classes but one of them is person class which have 17 keypoints and in the other ones we want only one keypoint or none keypoint,

is it possible? How have I to modify .cfg file to do so?

Thank you.

FileNotFoundError: [Errno 2] No such file or directory: 'yolov7-w6-pose.pt'

Where should the pose estimation model be put for it to work ?