iitzco / faced

🚀 😏 Near Real Time CPU Face detection using deep learning

faced

🚀 😏 CPU (Near) Real Time face detection

How to install

$ pip install git+https://github.com/iitzco/faced.git

Soon to be available on PyPI.

How to use

As library

import cv2

from faced import FaceDetector
from faced.utils import annotate_image

face_detector = FaceDetector()

img = cv2.imread(img_path)
rgb_img = cv2.cvtColor(img.copy(), cv2.COLOR_BGR2RGB)

# Receives RGB numpy image (HxWxC) and
# returns (x_center, y_center, width, height, prob) tuples. 
bboxes = face_detector.predict(rgb_img, thresh)

# Use this utils function to annotate the image.
ann_img = annotate_image(img, bboxes)

# Show the image
cv2.imshow('image',ann_img)
cv2.waitKey(0)
cv2.destroyAllWindows()

As command-line program

# Detection on image saving the output
$ faced --input imgs/demo.png --save

or

# Live webcam detection
$ faced --input webcam

or

# Detection on video with low decision threshold
$ faced --input imgs/demo.mp4 --threshold 0.5

See faced --help for more information.

Examples

Performance

CPU (i5 2015 MBP)	GPU (Nvidia TitanXP)
~5 FPS	> 70 FPS

Comparison with Haar Cascades

Haar Cascades are one of the most used face detections models. Here's a comparison with OpenCV's implementation showing faced robustness.

faced	Haar Cascade

About faced

faced is an ensemble of 2 deep neural networks (implemented using tensorflow) designed to run at Real Time speed in CPUs.

Stage 1:

A custom fully convolutional neural network (FCNN) implementation based on YOLO. Takes a 288x288 RGB image and outputs a 9x9 grid where each cell can predict bounding boxes and probability of one face.

Stage 2:

A custom standard CNN (Convolutions + Fully Connected layers) is used to take a face-containing rectangle and predict the face bounding box. This is a fine-tunning step. (outputs of Stage 1 model is not so accurate by itself, this is a corrector step that takes the each bouding box predicted from the previous step to improve bounding box quality.)

Why not just perform transfer learning on trained YOLO (or MobileNet+SSD) ?

Those models were designed to support multiclass detection (~80 classes). Because of this, these networks have to be powerfull enough to capture many different low and high level features that allow them to understand the patterns of very different classes. Powerful in this context means large amount of learnable parameters and hence big networks. These big networks cannot achieve real time performance on CPUs. [1]

This is an overkill for the simple task of just detecting faces. This work is a proof of concept that lighter networks can be designed to perform simpler tasks that do not require relatively large number of features.

[1] Those models cannot perform Real Time on CPU (YOLO at least). Even tiny-yolo version cannot achieve 1 fps on CPU (tested on 2015 MacBook Pro with 2.6 GHz Intel Core i5).

How was it trained?

Training was done with WIDER FACE dataset on Nvidia Titan XP GPU.

If you are interested in the training process and/or data preprocessing, just raise an issue and we'll discuss it there.

How to run on GPU?

Just install tensorflow-gpu instead of tensorflow.

Status

🚧 Work in progress 🚧 Models will be improved and uploaded.

This is not a Production ready system. Use it at your own risk.

faced's Issues

Memory leakage in .predict - Repeatedly calling .predict(...) results in memory leak

Im trying to call face_detector.predict(rgb_img, thresh) just about 100 times/second and each time it causes memory increment just around 1.3-8.0 Mb.

    def crop_faces_from_numpy_image(self, numpy_rgb):
        bboxes = self.face_detector.predict(numpy_rgb, self.thresh)
        cropped_images_array = []

        for bbox in bboxes:
            xc = bbox[0]
            yc = bbox[1]
            w = bbox[2]
            h = bbox[3]

            face = self.cv2_image[yc - h:yc + h, xc - w:xc + w]
            if len(face) > 0:
                resized = cv2.resize(face, dim)
                cropped_images_array.append(resized)

        return cropped_images_array

    def numpy_image_from_bytes(self, data):
        cv2img = cv2.imdecode(np.frombuffer(data, np.uint8), -1)
        self.cv2_image = cv2img
        np_rgb = cv2.cvtColor(cv2img.copy(), cv2.COLOR_BGR2RGB)
        return np_rgb

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
    23    796.0 MiB    796.0 MiB           1   @profile
    24                                         def crop_and_save(barr_img):
    25    797.3 MiB      1.3 MiB           1       np_arr = fd_worker.numpy_image_from_bytes(barr_img)
    26    805.3 MiB      8.0 MiB           1       cropped_arr = fd_worker.crop_faces_from_numpy_image(np_arr) - here is leakage
    27    805.5 MiB      0.2 MiB           1       fd_worker.save_images(cropped_arr)
    28    805.5 MiB      0.0 MiB           1       del np_arr
    29    805.5 MiB      0.0 MiB           1       del cropped_arr
    30    805.5 MiB      0.0 MiB           1       gc.collect()

Python 3.8

Strange result

Hello!

I'm trying to run faced, and use the code from the example. But it gives out a strange wrong result.
Does this software require image preprocessing before putting them to the input?

www.imgur.com/3XUoU3d

How to train on new generic object class

How was faced trained on the the WIDER FACE dataset and how was the dataset prepared?

unable to install it in windows 10

Hi everyone.
I could n't install faced on my Windows , I got this error and didn't understand it .
I appreciate if you could help me to install and use it

""
C:\Users\user1\Desktop\31Dec2018\faced>pip install git+https://github.com/iitzco/faced.git
Collecting git+https://github.com/iitzco/faced.git
Cloning https://github.com/iitzco/faced.git to c:\users\user1\appdata\local\temp\pip-req-build-ei3jp_f7
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\user1\AppData\Local\Temp\pip-req-build-ei3jp_f7\setup.py", line 14, in
long_description=readme(),
File "C:\Users\user1\AppData\Local\Temp\pip-req-build-ei3jp_f7\setup.py", line 6, in readme
return f.read()
File "C:\Users\user1\Downloads\WinPython-64bit-3.6.2.0Qt5\python-3.6.2.amd64\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 19: character maps to
""

Thank you

AttributeError: 'NoneType' object has no attribute 'shape'

Hi!

Thank you for sharing this tool.
I was having this issue while running faced with an .mp4 video.

This is the terminal output:

Traceback (most recent call last):
  File "../bin/faced", line 143, in <module>
    run_video(path, t, args.save)
  File "../bin/faced", line 103, in run_video
    if frame.shape[0] == 0:
AttributeError: 'NoneType' object has no attribute 'shape'

The segment of the code that is pointing in faced is:

      ... 
      cap = cv2.VideoCapture(path)
       ...
       # Capture frame-by-frame
        ret, frame = cap.read()
        if frame.shape[0] == 0:
            break

Have you got any idea what can be happening here?
I am using a python3 conda enviroment, with tensorflow-gpu installed.

Thanks!

Installation Error : UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 21: character maps to <undefined>

OS: Windows 10
Python v3.6.x

E:\clients\Assets\FR_Infogen_v3\IGL\code>pip install git+https://github.com/iitzco/faced.git
Collecting git+https://github.com/iitzco/faced.git
  Cloning https://github.com/iitzco/faced.git to c:\users\diksha\appdata\local\temp\pip-req-build-5ibodt8c
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\Diksha\AppData\Local\Temp\pip-req-build-5ibodt8c\setup.py", line 14, in <module>
        long_description=readme(),
      File "C:\Users\Diksha\AppData\Local\Temp\pip-req-build-5ibodt8c\setup.py", line 6, in readme
        return f.read()
      File "D:\ai\envs\face3\lib\encodings\cp1252.py", line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0]
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 21: character maps to <undefined>

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\Diksha\AppData\Local\Temp\pip-req-build-5ibodt8c\

faced is not recognized

Hi, how can I solve problem faced is not recognized as an internal command?

How to train Faced

How can I train it on a dataset I have?

Preparing dataset

Faced is really awesome and light! Can you tell me how is dataset shape and structure? i would like to train my yolo into just 5 classs

Does there any plan to opensource training code on wideface dataset?

Hi, thanks for this cpu real time face detect algorithm, is there any plan to open source training code?

Have you tested this on Jetson TX2?

From the looks of it, nothing should prevent it from running on Jetson. But, I am curious if have some performance numbers on Jetson. Thanks

how to train on new data and how to use on GPU environments?

i am beginner of deep learning and i know the previous issue solution about which is how to train on your blog but i have a question that your explanation of auxilary network is not clear. Is auxilary network concatenated by main network? or independtly? so i want to know how to train main architecture and auxilary network clearly and i have one more question that how to use on GPU environments when inference step?

pipeline.config file

Please share the pipeline configuration file.

could you please upload the training scripts

Can you tell me how to extract the weights of your trained dataset?
please upload the training scripts i would like to create my yolo for eye detection.

Detection Score

Hi, thanks a lot for open-sourcing faced, this is awesome!

I have two questions:

1. In the predict method, do you have any particular reason for not returning the probability as well?

`

def predict(self, frame, thresh=0.85):
    input_img = cv2.resize(frame, (YOLO_SIZE, YOLO_SIZE)) / 255.
    input_img = np.expand_dims(input_img, axis=0)

    pred = self.sess.run([self.prob, self.x_center, self.y_center, self.w, self.h], feed_dict={self.training: False, self.img: input_img})

    bboxes = self._absolute_bboxes(pred, frame, thresh)
    bboxes = self._correct(frame, bboxes)
    bboxes = self._nonmax_supression(bboxes)

    return bboxes`

2. Related to question 1, is there a convenient way to get the prob/detection score for the predicted image?

Thank you so much.

'faced' is not recognized as an internal or external command, operable program or batch file.

'faced' is not recognized as an internal or external command,
operable program or batch file.

why ???????

add the names for the person

this was my code when i used cascade file

`
import cv2
import numpy as np
import os

def f_s():

trainer = r'C:\Users\User\Documents\jarvis\face recognition\trainer'

recognizer = cv2.face.LBPHFaceRecognizer_create()
recognizer.read(trainer + '/trainer.yml')
cascadePath = r'C:\Users\User\Documents\jarvis\cascades\data\haarcascade_frontalface_default.xml'
faceCascade = cv2.CascadeClassifier(cascadePath)

font = cv2.FONT_HERSHEY_SIMPLEX

#iniciate id counter
id = 0

# names related to ids: example ==> mark : id=1,  etc
names = ['None','Mark'] 

# Initialize and start realtime video capture
cam = cv2.VideoCapture(0)
cam.set(3, 640) # set video widht
cam.set(4, 480) # set video height

# Define min window size to be recognized as a face
minW = 0.1*cam.get(3)
minH = 0.1*cam.get(4)



while True:
    ret, img =cam.read()
    img = cv2.flip(img, 1) # Flip vertically
    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    
    faces = faceCascade.detectMultiScale( 
        gray,
        scaleFactor = 1.2,
        minNeighbors = 5,
        minSize = (int(minW), int(minH)),
    )

    for(x,y,w,h) in faces:
        cv2.rectangle(img, (x,y), (x+w,y+h), (0,255,0), 2)
        id, confidence = recognizer.predict(gray[y:y+h,x:x+w])

        # Check if confidence is less them 100 ==> "0" is perfect match 
        if (confidence < 100):
            id = names[id]
            confidence = "  {0}%".format(round(100 - confidence))
        else:
            id = "unknown"
            confidence = "  {0}%".format(round(100 - confidence))
        
                    
        if 'Mark' == id:
            A = '18'
        else:
            print('age: unknon')

        cv2.putText(img, str(id), (x+5,y-5), font, 1, (255,255,255), 2)
        cv2.putText(img, str(confidence), (x+65,y-5), font, 1, (255,255,0), 1)   
        cv2.putText(img, str(A), (x+25,y+h+23), font, 1, (255,255,0), 1)

    cv2.imshow('face recognition',img) 
    
    
    k = cv2.waitKey(10) & 0xff # Press 'ESC' for exiting video
    if k == 27:
        break

# Do a bit of cleanup
print("\n [INFO] Exiting Program and cleanup stuff")
cam.release()
cv2.destroyAllWindows()

if name == "main":
f_s()
`

is there a way of me adding my name the way i did above in the face code thing

Change Webcam

Is there a way to change the camera(like where's the videocapture)?

Model in Keras or ONNX format?

Hi! This looks really promising and interesting, good work! Would it be possible to get the actual trained model either in Keras format (.h5) or in ONNX? I'd like to use the model in MATLAB, but I can only import either Keras or ONNX formats.

TensorFlow 2.0

For use with TensorFlow 2.0 I changed first line of faced/detector.py from :

import tensorflow as tf

to

import tensorflow.compat.v1 as tf

and it works. I know it's not ideal, but it works.

For low resolution and dark contrast images.

Hi Ivan,

I used your face_detection model on some low resolution and dark contrast images such as the ones taken through cctv feed. The results aren't good on such images, Can you suggest resources which would be good for such kind of images, I'm okay with gpu usage.

Thanks,
Aditya

Accuracy? Is there a paper?

This work is very nice, and I was wondering if there is a paper attached with it so I can read it in more detail. Also, what was the testing accuracy on the Wider face dataset after reducing YOLO to your network?

ValueError: No op named IteratorFromStringHandle in defined operations.

Hello, I've tried to load model for face_yolo.pb
However, it shows the following messages:

ValueError: No op named IteratorFromStringHandle in defined operations.

Could you please help comment it?
Thanks

Actual Code for FaceDetector and FaceCorrector models?

Unless im mistaken thus repository only contains the pretrained tensorflow1.x models of the the FaceDetector (face-yolo.pb) and FaceCorrector. i was wonderign if i could have a look at the actual code, and dataset used to traij these models

Problem on installation

Hi I am trying to use this but I ran into a problem that when I use 'python setup.py install ' in cmd and it shows finished successfully and I can import faced in my python. However when I try to use faced directly in cmd it cannot recognize it as an valid command. I am confused about that issue. I am very grateful if anyone can help me. Thx!

deleted

please delete this queestion

can we use intelMKL instead of openBlas ?

Hi,

can we use intelMKL instead of openBlas ? its little bit faster than openblas

Best

How to train ?

I would like to train this model for Person detection.
Please help with the steps.

Faces misaligned?

Hi all,

Great package - much faster than many of the other options.

I'm running into an issue where, for some images, the boxes are drawn in the incorrect place. The number of boxes is correct, it just looks as though the boxes have been shifted left. See an example below.

(The image is just a random photo from the internet. The original image is here)

I've been over the code and I cannot see why this would occur. I've also tried resizing the image, including changing the aspect ratio, and the result is the same. Has anyone had a similar experience?

Edit: To be clearer - it only occurs on some images, not all. So I don't think it's a problem with the install.

GPU requirements

Hello!
Thanks for the project, it works for me.
But I have strange results with my GTX 1080 Ti.
For some reason when I install Tensorflow with CUDA support and check GPU usage in MSI Afterburner, it shows that 10.6 of 11 Gb video memory is used. And this is not a training mode - I just use these already trained models.
Does that mean that it really needs so much video memory or this program just fills all the memory a video card provides?
Will this work on some GT 730 4G GPU as well? Or it will be not enough video memory?
How much video memory does the module require?
Thanks!

Installation Error

Faiz  localhost  ~  GitRepos  Faced  🔥  pip3 install git+https://github.com/iitzco/faced.git
Collecting git+https://github.com/iitzco/faced.git
Cloning https://github.com/iitzco/faced.git to /tmp/pip-req-build-cqyujtbf
Collecting numpy (from faced==0.1)
Downloading https://files.pythonhosted.org/packages/3d/10/62224c551acfd3a3583ad16d1e0f1c9e9c333e74479dc51977c31836119c/numpy-1.16.0-cp37-cp37m-manylinux1_x86_64.whl (17.3MB)
100% |████████████████████████████████| 17.3MB 443kB/s
Collecting tensorflow (from faced==0.1)
Could not find a version that satisfies the requirement tensorflow (from faced==0.1) (from versions: )
No matching distribution found for tensorflow (from faced==0.1)

Can this library be used on raspberry pi 3 B+?

Awesome Stuff!

Hello good sir, I hope you are having a great day. I have this idea that I wish to share, any chance we can exchange emails? 😄

Mine is [email protected]. Cheers!

Porting to 🦀Rust!

Hey @iitzco

I think this is an awesome project and great example of using tensorflow in a fast, reliable and CPU backed way.

In an effort to better understand how to create Tensorflow + Rust apps I started to port faced to Rust here: https://github.com/drbh/faced-rs

I have most of the core logic completed (load and run the two models) but need to finish writing the non_max and IOU functions.

Just wanted to share the repo and thank you for the awesome project! Also I would love to hear any feedback you have - or ways that faced-rs can be improved 😁