wufan-tb / yolo_slowfast Goto Github PK

View Code? Open in Web Editor NEW

427.0 4.0 53.0 7.21 MB

Yolov5+SlowFast: Realtime Action Detection Based on PytorchVideo

Python 100.00%

yolov5 slowfast

yolo_slowfast's Introduction

Yolov5+SlowFast: Realtime Action Detection

A realtime action detection frame work based on PytorchVideo.

Here are some details about our modification:

we choose yolov5 as an object detector instead of Faster R-CNN, it is faster and more convenient
we use a tracker(deepsort) to allocate action labels to all objects(with same ids) in different frames
our processing speed reached 24.2 FPS at 30 inference batch size (on a single RTX 2080Ti GPU)

Relevant infomation: FAIR/PytorchVideo; Ultralytics/Yolov5

Demo comparison between original(<-left) and ours(->right).

Update Log:

2023.03.31 fix some bugs(maybe caused by yolov5 version upgrade), support real time testing(test on camera or video stearm).
2022.01.24 optimize pre-process method(no need to extract video to image before processing), faster and cleaner.

Installation

clone this repo:

git clone https://github.com/wufan-tb/yolo_slowfast
cd yolo_slowfast

create a new python environment (optional):

conda create -n {your_env_name} python=3.7.11
conda activate {your_env_name}

install requiments:
```
pip install -r requirements.txt
```
download weights file(ckpt.t7) from [deepsort] to this folder:
```
./deep_sort/deep_sort/deep/checkpoint/
```
test on your video/camera/stream:
```
python yolo_slowfast.py --input {path to your video/camera/stream}
```
The first time execute this command may take some times to download the yolov5 code and it's weights file from torch.hub, keep your network connection.

set --input 0 to test on your local camera, set --input {stream path, such as "rtsp://xxx" or "rtmp://xxxx"} to test on viewo stream.

References

Thanks for these great works:

[1] Ultralytics/Yolov5

[2] ZQPei/deepsort

[3] FAIR/PytorchVideo

[4] AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions. paper

[5] SlowFast Networks for Video Recognition. paper

Citation

If you find our work useful, please cite as follow:

{   yolo_slowfast,
    author = {Wu Fan},
    title = { A realtime action detection frame work based on PytorchVideo},
    year = {2021},
    url = {\url{https://github.com/wufan-tb/yolo_slowfast}}
}

Stargazers over time

yolo_slowfast's People

Contributors

Stargazers

Watchers

yolo_slowfast's Issues

how to train custom weights and detection by using it?

Thank you for creating yolo_slowfast.
I want to use the weights that I learned from the custom dataset.
I'd like you to tell me how, thanks.

如何换成自己的slowfast权重测试呢

from torch.hub import load_state_dict_from_url都是从这里自动下载权重

测试视频时长30秒，运行输出保存的只有1、2秒要怎么解决

请问我复现出来的输出视频为啥是紫色的呢，我要怎么操作才能把BGR格式改成原来的，输出视频正常色调，求解答

Unable to open the camera.

When I try to use the camera for recognition, I cannot see the image. How can I resolve this issue?

我用YOLOv8替换了YOLOv5，为什么行为检测就一直是Unknow呢？

我用YOLOv8替换了YOLOv5，为什么行为检测就一直是Unknow呢？这有什么影响吗？

how can i find output.mp4?

where can i find my result video??
i can't find output.mp4.....

support higher slowfast model？

for example use slowfast_r101_detection replace slowfast_r50_detection？？？

temp.pbtxt里面的80和11都是stand，重复了

根据默认模型是基于AVA2.2的原因，应该把temp.pbtxt改成 ava_action_list.pbtxt

请问下如何设置slowfast动作检测的类别？

请问下如何设置slowfast动作检测的类别？如何设置或者修改代码使得，slowfast只检测指定的某些动作，谢谢！

这个网络可以用最新的VIT的网络吗，就是facebook上面的那个？

这个网络可以用最新的VIT的网络吗，就是facebook最新出的那个？

Unknown urllib problem

Run“yolo_slowfast.py”reported during operation：“urllib.error.URLError: ”

Have you sloved this question?

Traceback (most recent call last):
File "D:\yolo_slowfast-master\yolo_slowfast.py", line 201, in
main(config)
File "D:\yolo_slowfast-master\yolo_slowfast.py", line 127, in main
video_model = slowfast_r50_detection(True).eval().to(device)
File "D:\Anaconda3\envs\cnslowfast\lib\site-packages\pytorchvideo\models\hub\slowfast.py", line 178, in slowfast_r50_detection
**kwargs,
File "D:\Anaconda3\envs\cnslowfast\lib\site-packages\pytorchvideo\models\hub\slowfast.py", line 30, in _slowfast
checkpoint_path, progress=progress, map_location="cpu"
File "D:\Anaconda3\envs\cnslowfast\lib\site-packages\torch\hub.py", line 528, in load_state_dict_from_url
return torch.load(cached_file, map_location=map_location)
File "D:\Anaconda3\envs\cnslowfast\lib\site-packages\torch\serialization.py", line 585, in load
with _open_zipfile_reader(opened_file) as opened_zipfile:
File "D:\Anaconda3\envs\cnslowfast\lib\site-packages\torch\serialization.py", line 242, in init
super(_open_zipfile_reader, self).init(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: [enforce fail at ..\caffe2\serialize\inline_container.cc:145] . PytorchStreamReader failed reading zip archive: failed finding central directory
RuntimeError: [enforce fail at ..\caffe2\serialize\inline_container.cc:145] . PytorchStreamReader failed reading zip archive: failed finding central directory
你好，在pycharm运行，出现上述报错这个错误表明在尝试从zip文件中读取模型状态字典时出现问题。这可能是由于zip文件损坏或不完整导致的。请问怎么解决呢？麻烦您抽空给出回答

Detection on custom yolo weights

I have a custom trained yolo weights and there the person class is not having 0 class index. How to handle that?
As far as I understood you are taking the 0th index and passing it to slowfast_r50, since person class is in 0th index at coco dataset

Real time testing

Hai @wufan-tb

How to test with real-time videos from video surveillance. is that possible and how to test the modal with multiple video surveillance cameras

Thanks