GithubHelp home page GithubHelp logo

iit-pavis / social-distancing Goto Github PK

View Code? Open in Web Editor NEW
136.0 13.0 56.0 5.58 MB

Code for estimating social distances from RGB cameras.

License: Other

Python 100.00%
ai social-distancing computer-vision openpose

social-distancing's Introduction

Social-Distancing

Social-Distancing is an open-source project for automatically estimating interpersonal distance from uncalibrated RGB cameras. The software can be freely used for any non-commercial applications to assess compliance with safe distances. The code is open and can be improved with your support, please contact us at [email protected] if you would like to help us.

output

What's New

[December 18th, 2020]

[November 5th, 2020]

  • Our work have been accepted at WACV 2021! Here are the paper and the arXiv.

[November 4th, 2020]

  • Alghorithm updates with better distance evaluation and computational speed up
  • Fast ellipses intersections check with Shapely
  • Added masking support to select interesting areas
  • Streaming support
  • Acquisition from Jetson nano camera
  • Ubuntu 20.04 with Cuda 10.1 support

[April 24th, 2020]

  • Code for live camera acquisition and video processing.
  • New video samples in the samples folder.

Description

Given a frame captured from a scene, the algorithm first detects visible people in the scene using an off-the-shelf body pose detector and estimates the height of the people through measuring the distance from their body joints. In the second step, the algorithm estimates an area of one meter around all the detected people. This distance is roughly estimated proportional to a typical human body height of 160 cm and can be used to draw a circle centered in human position in the scene. In the third step, the Homography of the scene is estimated given two parameters which essentially map the rectangular bird’s view model for the scene to the trapezoidal perspective view of the scene. These two parameters need to be manually tuned to estimate best the scene perspective. According to the Homography matrix, the safe circular distance for each person is converted to ellipsoids in perspective view. The people are considered to be staying in safe distance from each other if their ellipsoids do not collide. Conversely, if ellipsoids of two people collide, those people are considered as being in risk and their ellipsoids will be shown in red.

If you use this code as part of your research, please cite our work.

@inproceedings{vsd2021,
   title={Single Image Human Proxemics Estimation for Visual Social Distancing},
   author={Aghaei, Maya and Bustreo, Matteo and Wang, Yiming and  Bailo, Gian Luca and Morerio, Pietro and Del Bue, Alessio},
   booktitle={IEEE Winter Conference on Applications of Computer Vision (WACV)},
   year={2021}
}

Installation steps

Code is developed in Python3 and tested on Ubuntu 20.04 with NVidia driver, Cuda 10.1 and Cudnn 7.6.5.

  • Install the requirements
    To run this code, you need to install:

    • OpenPose 1.6.0:
      Please follow the instruction in the repository gitHub and install OpenPose in social-distancing/openpose/ folder.
      In case you prefer to use a different OpenPose installation folder, you can pass it using the --openpose_folder argument.

    • OpenCV:
      apt-get install python3-opencv
      pip3 install opencv-python

    • PyTurboJPEG:
      pip3 install PyTurboJPEG

    • Shapely: pip3 install Shapely

    • Itertools: pip3 install itertools

    • Numpy: pip3 install numpy

Usage

python3 process_source.py -h  #help

Images

python3 process_source.py --image_in <path to the input image> --image_out <path to the result image to be saved> -- background_in <path to the background> --horizontal_ratio 0.7 --vertical_ratio 0.7

Videos

python3 process_source.py --video enabled --stream_in [path to the input video] --stream_out [path to the result video] --horizontal_ratio 0.7 --vertical_ratio 0.7

Network stream

python3 process_source.py --preview disabled --streaming enabled --video_port [port] --js_port [js_port] --stream_in [ address ]

Dataset

The dataset can now be downloaded from the following link: social_distancing_dataset (24.5 GB).

Disclaimer

Information provided by the software is to be intended as an indication of safe distance compliance. It is not intended to measure the actual metric distance among people.

The software is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors, PAVIS or IIT be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software.

LICENSE

This project is licensed under the terms of the MIT license.

This project incorporates material from the projects listed below (collectively, "Third Party Code"). This Third Party Code is licensed to you under their original license terms. We reserves all other rights not expressly granted, whether by implication, estoppel or otherwise.

  1. OpenPose
  2. OpenCV

iit-pavis-logo

social-distancing's People

Contributors

aledelb avatar dexmac221 avatar lorenzoprincipi avatar matteobustreo avatar mayaghaei avatar pmorerio avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

social-distancing's Issues

Jetson nano support

@pmorerio
@mayaghaei
@AleDelB

thanks for your work.

I have tried on my jetson nano but got very poor fps (0.2)

also, I checked TensorRT_pose detection giving acceptable fps.

Could you post the sample code regarding how to integrate the existing (openpose) code into tensorRT code on jetson nano.

help appreciated.

How do you get the coordinates of the corresponding points between src and dst?

rationed_hight = im_size[1] * self.vertical_ratio
rationed_width = im_size[0] * self.horizontal_ratio
src = np.array([[0, 0], [0, im_size[1]],
[im_size[0], im_size[1]],
[im_size[0], 0]])
dst = np.array([[0+rationed_width/2, 0+rationed_hight],
[0, im_size[1]], [im_size[0],im_size[1]],
[im_size[0]-rationed_width/2, 0+rationed_hight]], np.int32)
图片
图片

I do not understand how to get dst points coordinates from your description in paper

[ERROR:0] global /io/opencv/modules/videoio/src/cap.cpp (116) open VIDEOIO(CV_IMAGES): raised OpenCV exception: OpenCV(4.2.0) /io/opencv/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): ./input_stream.avi in function 'icvExtractPattern' Error: Opening video stream or file ./input_stream.avi

When I run the below command:

python3 social-distancing.py --image_in ./samples/286.jpg --image_out output.jpg --horizontal_ratio 0.7 --vertical_ratio 0.7 

I run into this issue below which is weird because I am testing an image not video...

[ERROR:0] global /io/opencv/modules/videoio/src/cap.cpp (116) open VIDEOIO(CV_IMAGES): raised OpenCV exception:

OpenCV(4.2.0) /io/opencv/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): ./input_stream.avi in function 'icvExtractPattern'


Error: Opening video stream or file ./input_stream.avi

Question about the license

Hello I want to thank you and congratulate you on the project, I can only imagine the great effort and dedication dedicated to the project, this message is to tell you that I was thinking of implementing it in a web application which makes it scalable and technically accessible, however I have a doubt, the license is MIT, but it has the following limitation "The software can be used for any non-commercial application to assess compliance with safe distances", the application has a high consumption of computational resources, which it would have costs. I want to know if to make the application viable I can use it for commercial purposes.

Thank you in advance for the response and your time.

Error: OpenPose library could not be found. Did you enable `BUILD_PYTHON`in CMake and have this Python script in the right folder?

Hi,

Thanks for your contribution. This is really helpful for my local community to practise social distancing.

While setting up, I am encountering an issue as shown in the title. I have followed the prerequisites by downloading openpose (https://github.com/esemeniuc/openpose-docker). I have also ran checks(https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/doc/modules/python_module.md) to ensure that openpose is properly installed by using the following code:

# From command line
cd build/examples/tutorial_api_python

# Python 3 (default version)
python3 01_body_from_image.py

After running the code, there is no exception showing that the library is missing(see image)
Screenshot 2020-05-14 at 8 12 17 PM

I suspect that there is a missing pyopenpose.py file since the error is sourced here from openpose import pyopenpose as op. I just could not find the pyopenpose.py file in my openpose folder even though I have passed the test(see the image above).

Can you help me out?

Having issue with video

No issue with image output but I am having issue with video output. So I can safely rule out any issue with the pre-requisite installations and environments.

This is the code I ran python3 social-distancing.py --video enabled --stream_in /test.avi --stream_out ./output/1.avi --horizontal_ratio 0.7 --vertical_ratio 0.7 --openpose_folder /openpose/models/

The output is

Starting OpenPose Python Wrapper...
: cannot connect to X server 

The file 1.avi is 0 size which does not make any sense. Care to help?

total 0
-rw-r--r-- 1 root root 0 May 20 10:24 1.avi

Issues setting the horizontal_ratio and the vertical_ratio

Hi! I am enjoying this excellently written code. Congratulations!

Using videos with different perspectives and the horizontal_ratio and vertical_ratio as default, sometimes the distance estimation does not work well. My problem is that I don't know how to properly set the horizontal_ratio and vertical_ratio.

I have found this description in the code:

parser.add_argument("--horizontal_ratio", default="0.7",
                    help="Ratio between the closest horizontal line of the scene to the furthest visible. It must be a float value in (0,1)")

parser.add_argument("--vertical_ratio", default="0.7",
                    help="Ratio between the height of the trapezoid wrt the rectangular bird’s view scene (image height). It must be a float value in (0,1)")

and read your paper (https://arxiv.org/abs/2005.04813) but still I am not sure how to set these values correctly.
Could you be so kind and explain using some of your sample videos (samples in the repo), how would you best set the horizontal_ratio and vertical_ratio for the particular video and why?

Thank you in advance

inference time

could you please give me some insights about the inference time? Will it work in realtime using CPU?

Implementation on Jetson Nano

Hi there! Thank you for your wonderful work on this.
This is what the world needs right now!

Can it supports Jetson Nano if I were to implement it on there ?

  • Do you mind explain how to install open pose?

Appreciate your help!

Corrupted output video and image error

Hi! First I want to congratulate for this amazing project.
I am using an azure virtual machine with Ubuntu 18.04, Python 3.7.7, Cuda 10.0 and CuDNN 7.6.5. My GPU is Nvidia Tesla V100 PCI-E.
I managed to build opencv from source for GPU using the tutorial at https://www.pyimagesearch.com/2020/02/03/how-to-use-opencvs-dnn-module-with-nvidia-gpus-cuda-and-cudnn/. What I did differently from the tutorial was that I used the master branch (as suggested of bieng faster) of opencv and opencv_contrib which installed opencv version 4.3dev instead of 4.2 release that was used in that tutorial.
Then I used this opencv's build for building openpose from source, finishing without errors. I also installed turbojpeg using:
sudo apt-get update
sudo apt-get install libturbojpeg
pip install -U git+git://github.com/lilohuang/PyTurboJPEG.git

I am using this command for processing videos (I processed the videos in samples directory of this repo):
"python social-distancing.py --video enabled --stream_in ./samples/video-example3.mp4 --stream_out ./output/video-example3-out.mp4 --horizontal_ratio 0.7 --vertical_ratio 0.7 --openpose_folder /home/adminvdi/sw/openpose/models/
"
It gives me the following warning:
"OpenCV: FFMPEG: tag 0x44495658/'XVID' is not supported with codec id 13 and format 'mp4 / MP4 (MPEG-4 Part 14)'
OpenCV: FFMPEG: fallback to use tag 0x7634706d/'mp4v'
Starting OpenPose Python Wrapper...
[ WARN:0] global /home/adminvdi/opencv/modules/core/src/matrix_expressions.cpp (1334) assign OpenCV/MatExpr: processing of multi-channel arrays might be changed in the future: opencv/opencv#16739
"
and it does show the output video on my screen. But the output video saved as video-example3-out.mp4 cannot be played even with VLC player. I also tried to reformat the output video using the ffmpeg commands but ffmpeg throws an error "Invalid data found when processing input" and cannot convert it either.

It is also interesting that processing the images using the images from the samples directory of this repo and the command:
"python social-distancing.py --image_in ./samples/286.jpg --image_out ./output/286-out.jpg --horizontal_ratio 0.7 --vertical_ratio 0.7 --openpose_folder /home/adminvdi/sw/openpose/models/
"
does not show nor does save any image at all, and it outputs this error:
"[ERROR:0] global /home/adminvdi/opencv/modules/videoio/src/cap.cpp (142) open VIDEOIO(CV_IMAGES): raised OpenCV exception:
OpenCV(4.3.0-dev) /home/adminvdi/opencv/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): ./input_stream.avi in function 'icvExtractPattern'
Error: Opening video stream or file ./input_stream.avi
"

I would really appreciate your help resolving this issue.

In streaming mode social-distancing.py gets terminated by an error

Hi,

When using streaming mode with the following parameters (streaming='enabled', video='enabled', image='disabled', preview='disabled', etc.) and the RTSP from a network camera, many times the execution of the social-distancing.py gets terminated by some error after an arbitrary time of running, i.e., from 1 second up to 10 hours and in three different ways:

  1. without any error printed on the terminal
  2. error like "[h264 @ 0x55d380821600] error while decoding MB 41 30, bytestream -17" on the terminal
  3. the following printed on the terminal (rarely):
    [NULL @ 0x55f60ce2e100] missing picture in access unit with size 159
    [h264 @ 0x55f60ce7a240] no frame!

Could you please suggest what can I do to prevent the social-distancing.py to get terminated by an error?

Many thanks in advance!!!

installiation

hi, I liked your work and have a few doubts on how to install the basics and how to get the program running. I tried the method you had told me but still, need clarification please help, please reply

Custom data

Hello @mayaghaei ,

Awesome and interesting work! This has given a breather for some applications which will otherwise require caliberation

How do i run this for my custom data. If i understand correctly,In your paper you have mentioned about a tool that you have used to get the ratios visually and this accuracy is closer to the grid search way. Can u also upload the code for the same for us to experiment with? In my situation the dataset size is small to estimate with a grid search.

Error when trying to open a video file

Hello all,

I passed the following command onto Ubuntu terminal:

python3 process_source.py --image disabled --video enabled --stream_in ./samples/video.mp4 --stream_out /samples/video_out.avi --horizontal_ratio 0.7 --vertical_ratio 0.7 --openpose_folder /home/user/Desktop/Social-Distancing/openpose/models

However, this is what I got:

Starting OpenPose Python Wrapper...
[ERROR:0] global /tmp/pip-req-build-ms668fyv/opencv/modules/videoio/src/cap.cpp (561) open VIDEOIO(CV_IMAGES): raised OpenCV exception:

OpenCV(4.5.1) /tmp/pip-req-build-ms668fyv/opencv/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): /samples/video_out.avi in function 'icvExtractPattern'


Error: Unable to load background image (flag enabled)

Only received the error message ^ and nothing else happened.

For some context, I have a video file (video.mp4) in the samples folder of the Social-Distancing folder, and that is it. I am ob Ubuntu 20.04, with CUDA 10.1 CuDNN 7.6.5, and my GPU is Nvidia GTX 1650 Max-Q.

Please advise on what I should do, thanks!

Issue with TurboJPEG

Hello there! I am facing issues trying to run the program. I can show the error log below. Can't seem to find what is the problem.

`python3 social-distancing.py --image_in /home/upm/Desktop/Projects/Social-Distancing/samples/286.jpg --image_out /home/upm/Desktop/Projects/Social-Distancing/samples/286_out.jpg --horizontal_ratio 0.7 --vertical_ratio 0.7
Starting OpenPose Python Wrapper...

Error:
Prototxt file not found: /home/dexmac/openpose/models/pose/body_25/pose_deploy.prototxt.
Possible causes:
1. Not downloading the OpenPose trained models.
2. Not running OpenPose from the same directory where the model folder is located.
3. Using paths with spaces.

Coming from:

  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/net/netCaffe.cpp:ImplNetCaffe():58
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/net/netCaffe.cpp:ImplNetCaffe():94
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/pose/poseExtractorCaffe.cpp:addCaffeNetOnThread():106
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():196
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/pose/poseExtractorNet.cpp:initializationOnThread():102
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/pose/poseExtractor.cpp:initializationOnThread():34
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/include/openpose/pose/wPoseExtractor.hpp:initializationOnThread():57
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/include/openpose/thread/worker.hpp:initializationOnThreadNoException():77
    Traceback (most recent call last):
    File "social-distancing.py", line 715, in
    social_distance = SocialDistancing(args)
    File "social-distancing.py", line 254, in init
    self.jpeg = TurboJPEG()
    File "/home/upm/.virtualenvs/dl4cv/lib/python3.6/site-packages/turbojpeg.py", line 147, in init
    self.__find_turbojpeg() if lib_path is None else lib_path)
    File "/home/upm/.virtualenvs/dl4cv/lib/python3.6/site-packages/turbojpeg.py", line 424, in __find_turbojpeg
    'Unable to locate turbojpeg library automatically. '
    RuntimeError: Unable to locate turbojpeg library automatically. You may specify the turbojpeg library path manually.
    e.g. jpeg = TurboJPEG(lib_path)

Error occurred on a thread. OpenPose closed all its threads and then propagated the error to the main thread. Error description:

Prototxt file not found: /home/dexmac/openpose/models/pose/body_25/pose_deploy.prototxt.
Possible causes:
1. Not downloading the OpenPose trained models.
2. Not running OpenPose from the same directory where the model folder is located.
3. Using paths with spaces.

Coming from:

  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/net/netCaffe.cpp:ImplNetCaffe():58
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/net/netCaffe.cpp:ImplNetCaffe():94
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/pose/poseExtractorCaffe.cpp:addCaffeNetOnThread():106
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/pose/poseExtractorCaffe.cpp:netInitializationOnThread():196
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/pose/poseExtractorNet.cpp:initializationOnThread():102
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/pose/poseExtractor.cpp:initializationOnThread():34
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/include/openpose/pose/wPoseExtractor.hpp:initializationOnThread():57
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/include/openpose/thread/worker.hpp:initializationOnThreadNoException():77
  • [All threads closed and control returned to main thread]
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/src/openpose/utilities/errorAndLog.cpp:checkWorkerErrors():280
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/include/openpose/thread/threadManager.hpp:stop():243
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/include/openpose/wrapper/wrapper.hpp:stop():455
  • /home/upm/Desktop/Projects/Social-Distancing/openpose/include/openpose/wrapper/wrapper.hpp:~WrapperT():282
  • [Error occurred in a destructor or in the OpenPose Unity Plugin, so no std::exception has been thrown. Returning with exit status 0]
    `

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.