oarriaga / face_classification

Real-time face detection and emotion/gender classification using fer2013/imdb datasets with a keras CNN model and openCV.

This repository is deprecated for at TF-2.0 rewrite visit:

https://github.com/oarriaga/paz

---

Face classification and detection.

Real-time face detection and emotion/gender classification using fer2013/IMDB datasets with a keras CNN model and openCV.

• IMDB gender classification test accuracy: 96%.
• fer2013 emotion classification test accuracy: 66%.

For more information please consult the publication

Emotion/gender examples:

Guided back-prop

Real-time demo:

B-IT-BOTS robotics team :)

Instructions

Run real-time emotion demo:

python3 video_emotion_color_demo.py

Run real-time guided back-prop demo:

python3 image_gradcam_demo.py

Make inference on single images:

python3 image_emotion_gender_demo.py <image_path>

e.g.

python3 image_emotion_gender_demo.py ../images/test_image.jpg

Running with Docker

With a few steps one can get its own face classification and detection running. Follow the commands below:

• docker pull ekholabs/face-classifier
• docker run -d -p 8084:8084 --name=face-classifier ekholabs/face-classifier
• curl -v -F image=@[path_to_image] http://localhost:8084/classifyImage > image.png

To train previous/new models for emotion classification:

• Download the fer2013.tar.gz file from here

• Move the downloaded file to the datasets directory inside this repository.

• Untar the file:

tar -xzf fer2013.tar

• Run the train_emotion_classification.py file

python3 train_emotion_classifier.py

To train previous/new models for gender classification:

• Download the imdb_crop.tar file from here (It's the 7GB button with the tittle Download faces only).

• Move the downloaded file to the datasets directory inside this repository.

• Untar the file:

tar -xfv imdb_crop.tar

• Run the train_gender_classification.py file

python3 train_gender_classifier.py

face_classification's Issues

Feature request: Docker environment

Is there any plan to include a docker/nvidia-docker environment?
Would you be interested in a pull request if I build one?

how to train for your own images

Face next to frame edge problems.

The code works great, I've used the python part as is.

One issue that comes up is when a face bounding box comes close to one of the frame edges. This will raise :

OpenCV Error: Assertion failed (ssize.width > 0 && ssize.height > 0) in resize, file /home/user/opencv-3.2.0/modules/imgproc/src/imgwarp.cpp, line 3492

Another would be when the bounding box comes near the bottom edge of the frame it will detect a 'female' regarding the actual gender.

I can make some adjustments so that faces near frame edges be ignored, but this would reduce the camera's field of vision.

question about requirements

Hi@aorriaga
Thanks for your sharing!
I used this code but I have a question.What is the statistics in your requirements?Is it statsmodel in python or others?When I used it ,it will raise:

File "video_emotion_color_demo.py", line 1, in

   from statistics import mode
  ImportError: No module named statistics  ###

I found some way,but it didn't work.
Thanks
Lifei

Finding the face bounding box dynamically

I have tried getting the offset dynamically. Used the following code to do so. It works sometimes but it crashes attimes as KeyError. Im sharing the code with you so you can consider using it since it crashes when I try it

    detections = face_cascade.detectMultiScale(
        gray,
        scaleFactor=1.1,
        minNeighbors=5,
        minSize=(30, 30),
        flags=cv2.CASCADE_SCALE_IMAGE
    )

    faces = np.empty((len(detections), 48, 48, 3))
    grayf = np.empty((len(detections), 48, 48, 3))
    img_h, img_w, _ = np.shape(frame)
    for i in range(len(detections)):
        face_i = detections[i]
        gray_i = detections[i]
        x, y, w, h = face_i

        x1 = x
        y1 = y
        x2 = x+w
        y2 = y+w

        xw1 = max(int(x1 - 0.4 * w), 0)
        yw1 = max(int(y1 - 0.4 * h), 0)
        xw2 = min(int(x2 + 0.4 * w), img_w - 1)
        yw2 = min(int(y2 + 0.4 * h), img_h - 1)

        faces[i, :, :, :] = cv2.resize(frame[yw1:yw2 + 1, xw1:xw2 + 1, :], (48, 48))
        grayf[i, :, :, :] = cv2.resize(frame[yw1:yw2 + 1, xw1:xw2 + 1, :], (48, 48))

problem with runs code in other platform

hello orriaga...codes works so great for me in ubuntu os...
when i run video_emotion_gender_demo in raspberry pi that have raspbian stretch as os ,then i got some errors ,did you know i how overcome the errors
thanks in advance

memory leak

wuchen@ubuntu-All-Series-Invalid-entry-length-16-Fixed-up-to-11:~/download/face_classification/src$ python3 image_emotion_gender_demo.py
/home/wuchen/download/face_classification/images/test_image.jpg
Using TensorFlow backend.
2018-01-29 12:43:57.028096: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2018-01-29 12:43:57.028123: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2018-01-29 12:43:57.028131: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2018-01-29 12:43:57.028136: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2018-01-29 12:43:57.028142: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
swig/python detected a memory leak of type 'int64_t *', no destructor found.

Please tell me how to fix it, thanks

Issue with training dataset

Do I have to let it train 10000 times really?

About your model when emotion classification

impressive job! You have mentioned "fer2013 emotion classification test accuracy: 66%.". To my knowledge, this result is close to the state of art. But I wanna know some information about your training data. Is it fer2013 train or some other data. And the training data total number is ?
Thank you very much, it is helpful for me~

can not run vdeo_emotion_color_demo.py

Hi, I can not start your project.
TF, Keras, OpenCV and other libs installed
What i do
source ./tensorflow/bin/activate //activate TF
(tensorflow) gubiniliya@Lenovo-Z50-75:~/Documents/face_classification-master/src$ python3 video_emotion_color_demo.py
And i have an error

Using TensorFlow backend.
/home/gubiniliya/Documents/face_classification-master/src/tensorflow/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
  return f(*args, **kwds)
2017-11-30 11:15:39.289068: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX FMA
OpenCV Error: Assertion failed (scn == 3 || scn == 4) in cvtColor, file /io/opencv/modules/imgproc/src/color.cpp, line 10638
Traceback (most recent call last):
  File "video_emotion_color_demo.py", line 39, in <module>
    gray_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2GRAY)
cv2.error: /io/opencv/modules/imgproc/src/color.cpp:10638: error: (-215) scn == 3 || scn == 4 in function cvtColor

Okay, i see on line 39 =)
Yep
gray_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2GRAY)
rgb_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2RGB)
and i found this issue on stackoverflow there they advised us to write the full path to the picture, but we also get video from the camera
bgr_image = video_capture.read()[1]
ok, for test i change bgr_image to "/home/gubiniliya/Images/image.jpg"
And where i do it i have a new error

(tensorflow) gubiniliya@Lenovo-Z50-75:~/Documents/face_classification-master/src$ python3 video_emotion_color_demo.py
Using TensorFlow backend.
/home/gubiniliya/Documents/face_classification-master/src/tensorflow/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
  return f(*args, **kwds)
2017-11-30 11:28:17.264266: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX FMA
Traceback (most recent call last):
  File "video_emotion_color_demo.py", line 39, in <module>
    gray_image = cv2.cvtColor("/home/gubiniliya/Images/image.jpg", cv2.COLOR_BGR2GRAY)
TypeError: src is not a numpy array, neither a scalar

What i need to do, for fix my problem and run projectwith cam

Documentation

Should the second python file be training for the gender?

Download the fer2013.tar.gz file from here

Move the downloaded file to the datasets directory inside this repository.

Untar the file:

tar -xzf fer2013.tar
Run the train_emotion_classification.py file
python3 train_emotion_classifier.py
To train previous/new models for gender classification:

Download the imdb_crop.tar file from here (It's the 7GB button with the tittle Download faces only).

Move the downloaded file to the datasets directory inside this repository.

Untar the file:

tar -xfv imdb_crop.tar
Run the train_emotion_classification.py file
python3 train_emotion_classifier.py

File "/scratch/anaconda3/envs/deep_emotion/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 701, in __del__ TypeError: 'NoneType' object is not callable

When I ran the code for a new face, I got this error. Can you guide how to fix? I have shared the image as well.

(deep_emotion) [jalal@goku src]$  python  image_emotion_gender_demo.py ../images/test_image1.jpg 
Using TensorFlow backend.
2017-11-20 21:32:30.919332: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-20 21:32:30.919370: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-20 21:32:30.919379: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-11-20 21:32:30.919394: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-20 21:32:30.919401: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-11-20 21:32:31.231494: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties: 
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.6705
pciBusID 0000:05:00.0
Total memory: 10.91GiB
Free memory: 10.23GiB
2017-11-20 21:32:31.467503: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x2718450 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
2017-11-20 21:32:31.468249: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 1 with properties: 
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.6705
pciBusID 0000:06:00.0
Total memory: 10.91GiB
Free memory: 10.75GiB
2017-11-20 21:32:31.468973: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 1 
2017-11-20 21:32:31.468991: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y Y 
2017-11-20 21:32:31.469008: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 1:   Y Y 
2017-11-20 21:32:31.469021: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:05:00.0)
2017-11-20 21:32:31.469032: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:06:00.0)
2017-11-20 21:32:32.484971: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:05:00.0)
2017-11-20 21:32:32.485010: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:06:00.0)
Exception ignored in: <bound method BaseSession.__del__ of <tensorflow.python.client.session.Session object at 0x7ff4db550c88>>
Traceback (most recent call last):
  File "/scratch/anaconda3/envs/deep_emotion/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 701, in __del__
TypeError: 'NoneType' object is not callable

PROBLEM in running the train gender classiffier

I follow the instrument and I faces this error:

Using TensorFlow backend.
2017-11-11 13:25:44.338135: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Traceback (most recent call last):
File "train_gender_classifier.py", line 33, in
ground_truth_data = data_loader.get_data()
File "/home/neo/Desktop/GenderCl/face_classification-master/utils/datasets.py", line 29, in get_data
ground_truth_data = self._load_imdb()
File "/home/neo/Desktop/GenderCl/face_classification-master/utils/datasets.py", line 38, in _load_imdb
dataset = loadmat(self.dataset_path)
File "/home/neo/.virtualenvs/dl4cv/lib/python3.5/site-packages/scipy/io/matlab/mio.py", line 141, in loadmat
MR, file_opened = mat_reader_factory(file_name, appendmat, **kwargs)
File "/home/neo/.virtualenvs/dl4cv/lib/python3.5/site-packages/scipy/io/matlab/mio.py", line 64, in mat_reader_factory
byte_stream, file_opened = _open_file(file_name, appendmat)
TypeError: 'NoneType' object is not iterable

Please help me how can i solve this?

How to retrain the model using saved weights?

I have been facing trouble when I try using the saved weights to retrain the train_emotion_classifier model.The model starts from the beginning,no matter what.I tried using the : model.load_model('fer2013_mini_XCEPTION.100-0.65.hdf5') for my 100th epoch.I added this line of code before the model.compile( ) line.

Can you please tell me what changes need to be made if I want to use a weight file from emotion_models folder to retrain the existing model.

A camera photo predicted in a time frame

I see that in motion video, the prediction varies a lot and it is hard to have a fixed value compare to when it is, in fact, a static image photo with facial recognition and accurate prediction.

Hence, for motion video, is there a way to have a camera photo that is predicted in a time frame?

The Strategy of facial expression recogition

Hi , It's a pleasure to see such good open source code。
in face, i am also doing the research of facial expression recognition .
i want to know what effective skills have you taken.
i see the CNN structure is ordinary except depth.
and what's the effect when the real test is done.

Feature request: Documentation

Very cool, not too familiar with python - just trying to work out the entry point to run the code. How would I plug this in to an image stream?

feature request - python2.7 support

is python3 strictly required for this? I ran the demo with python2.7 , no errors but no output. The net seems to load but nothing more.

jeremy@jr:~/projects/face_classification/src$ python image_emotion_gender_demo.py ../images/test_image.jpg 
Using TensorFlow backend.
...
2017-11-02 09:09:55.707876: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
jeremy@jr:~/projects/face_classification/src$ 

save detected result and video file as a input

Hi @oarriaga ,

I am wondering is there a way, by which we can save the detected result e.g. emotion detected after running the script video_emotion_color_demo.py . Also can we run this script on video files (mp4) saved on local system for emotion detection or gender classification ? If yes can you modify the code or tell me how to do it ?

Happy != Smiling

I would change the "happy" labels to "smiling" labels. They would make them more accurate.
You can't conclude that because a man/woman is smiling it is happy.
And what about an ironic smile?

Specifications regarding the training data

1. What are the dimensions of the training dataset images? Or do they have to be a specific size?
2. Assuming my dataset are photographs instead of "mug-shots", how can I extract the faces from the photos for training using the CNN?
3. Does this project use the same methodology as these two projects, in terms of machine learning and library usage?
   • https://github.com/ageitgey/face_recognition
   • https://cmusatyalab.github.io/openface/

can't change trained models...

hi dear friends,
if i run the gender recognition by default train model "simple_CNN.81-0.96.hdf5"
it recognizes all the person as man
after i myself train the imdb dataset 5 times and create new hdf5 files and change the model with "gender_mini_XCEPTION.01-0.95.hdf5"

i got this problem:
ValueError: Error when checking : expected input_1 to have shape (None, 64, 64, 1) but got array with shape (1, 64, 64, 3)

please help me i absolutely mix up

Emotion labels

Hi,

in the utils/get_labels() there is a code:

return {0:'angry',1:'disgust',2:'sad',3:'happy',
                    4:'sad',5:'surprise',6:'neutral'}

that looks like typo (2:'sad' and 4:'sad' too). Or that was done on purpose?

How about a env.yml file for setting up dependencies with conda :-)

It would be helpful :-)

Add Statistics, h5py, and others to requirements

Hi,

I'm not sure if this was a module I'm supposed to have but I think the statistics and h5py module should be added to the REQUIREMENTS.txt
sudo pip install statistics h5py pydot

Real Time Error

Hi all,

When I try running the image_gradcam_demo.py script I am getting the following error:

Traceback (most recent call last):
  File "image_gradcam_demo.py", line 22, in <module>
    image_path = sys.argv[1]
IndexError: list index out of range

I am currently running a conda environment with python 3.6.

Gender for woman is recognized mostly as man Any help regarding positioning of camera

A problem when running "image_gradcam_demo.py"

Traceback (most recent call last):
File "image_gradcam_demo.py", line 73, in
predicted_class, 'conv2d_7')
File "/home/zengyiqi/project/face_classification/src/utils/grad_cam.py", line 115, in compile_gradient_function
gradients = normalize(K.gradients(loss, conv_output)[0])
File "/home/zengyiqi/project/face_classification/src/utils/grad_cam.py", line 31, in normalize
return x / (K.sqrt(K.mean(K.square(x))) + 1e-5)
File "/home/zengyiqi/anaconda2/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 1417, in square
return tf.square(x)
File "/home/zengyiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 449, in square
return gen_math_ops.square(x, name=name)
File "/home/zengyiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 4567, in square
"Square", x=x, name=name)
File "/home/zengyiqi/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 528, in _apply_op_helper
(input_name, err))
ValueError: Tried to convert 'x' to a tensor and failed. Error: None values not supported.

when I ran the commad of "python image_gradcam_demo.py ../images/test_image.jpg"，
there is a problem with the errer log above.
How can I solve this problem?
I would appreciate it very much if you could help me！

NotImplementedError when use fer2013_mini_XCEPTION.110-0.65.hdf5

When I use model like...XCEPTION...hdf5, it will raise NotImplementedError.

Traceback (most recent call last):
  File "/Users/Name/Documents/face_classification-master/src/image_emotion_gender_demo.py", line 35, in <module>
    gender_classifier = load_model(gender_model_path, compile=False)
  File "/Users/Name/Documents/face_classification-master/FaceDector/lib/python3.6/site-packages/keras/models.py", line 239, in load_model
    model = model_from_config(model_config, custom_objects=custom_objects)
  File "/Users/Name/Documents/face_classification-master/FaceDector/lib/python3.6/site-packages/keras/models.py", line 313, in model_from_config
    return layer_module.deserialize(config, custom_objects=custom_objects)
  File "/Users/Name/Documents/face_classification-master/FaceDector/lib/python3.6/site-packages/keras/layers/__init__.py", line 54, in deserialize
    printable_module_name='layer')
  File "/Users/Name/Documents/face_classification-master/FaceDector/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 139, in deserialize_keras_object
    list(custom_objects.items())))
  File "/Users/Name/Documents/face_classification-master/FaceDector/lib/python3.6/site-packages/keras/engine/topology.py", line 2497, in from_config
    process_node(layer, node_data)
  File "/Users/Name/Documents/face_classification-master/FaceDector/lib/python3.6/site-packages/keras/engine/topology.py", line 2454, in process_node
    layer(input_tensors[0], **kwargs)
  File "/Users/Name/Documents/face_classification-master/FaceDector/lib/python3.6/site-packages/keras/engine/topology.py", line 602, in __call__
    output = self.call(inputs, **kwargs)
  File "/Users/Name/Documents/face_classification-master/FaceDector/lib/python3.6/site-packages/keras/layers/convolutional.py", line 1220, in call
    padding=self.padding)
  File "/Users/Name/Documents/face_classification-master/FaceDector/lib/python3.6/site-packages/keras/backend/theano_backend.py", line 1954, in separable_conv2d
    raise NotImplementedError
NotImplementedError

Keras 1 models

I was wondering if you still might have your Keras 1 models that you used for this project? And would you be interested in sharing? I'm currently building out a new model zoo for Deeplearning4j, and our importer works for Keras v1 (v2 won't be arriving for 2 months). Thanks!

cuda gpu support ?

As title. Did you have support gpu command on training ?

does it can real-time in cpu?

thanks for sharing the code. does it can run real-time in cpu?

can you write this in java?

I am devoloper of android, this project is important for me. can you write this in java

can not run

can not run? way?
ValueError: Optimizer weight shape (16,) not compatible with provided weight shape (7, 7, 1, 16)

simple_CNN

hi, I am interested in your project and trained a "simple_CNN" model, but I only got 63.5% accuracy. I also trained your "mini_XCEPTION", I got 65.3%. Anything is wrong with me?

Retrain the pretrained model

How can I be able to retrain the existing model with new images?

ValueError: Error when checking

/home/hadoop/anaconda3/bin/python3.6 /home/hadoop/naresh/face_classification-master/src/video_emotion_gender_demo.py
Using TensorFlow backend.
2017-09-18 10:39:57.354480: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-09-18 10:39:57.354537: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
GLib-GIO-Message: Using the 'memory' GSettings backend. Your settings will not be saved or shared with other applications.
Traceback (most recent call last):
File "/home/hadoop/naresh/face_classification-master/src/video_emotion_gender_demo.py", line 72, in
gender_prediction = gender_classifier.predict(rgb_face)
File "/home/hadoop/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1499, in predict
check_batch_axis=False)
File "/home/hadoop/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 140, in _standardize_input_data
str(array.shape))
ValueError: Error when checking : expected input_1 to have shape (None, 64, 64, 1) but got array with shape (1, 64, 64, 3)

Process finished with exit code 1

Incompatible optimizer weight

Newest commit fires the following error

  File "C:/face_classification-master/face_classification-master/src/video_emotion_color_demo.py", line 26, in <module>
    emotion_classifier = load_model(emotion_model_path, custom_objects=False)
  File "C:\ProgramData\Anaconda3\lib\site-packages\keras\models.py", line 272, in load_model
    model.optimizer.set_weights(optimizer_weight_values)
  File "C:\ProgramData\Anaconda3\lib\site-packages\keras\optimizers.py", line 79, in set_weights
    'provided weight shape ' + str(w.shape))
ValueError: Optimizer weight shape (8,) not compatible with provided weight shape (64,)

New commit can not recognize gender

why is that the newest commit is not able to recognise the gender?

how use web folder in project

hello guys,,how i use web examples created in web folder,could you answer me??
thank you in advance

how to convert keras model to caffe model?

@oarriaga Thanks for share your code. If I want to train the model by caffe, how I get the prototxt from your model?

Using dlib for detection

Will using dlib for detection make accuracy better? The accuracy is low during real time prediction because of the offset

Why whoever is predicted to "man" ?

When I run your model，I get this result：

rather than your example：

create json file

hello everyone...i want to produce json file of video_emotion_gender_demo.py online..do you give me any help??thanks in advance

Decrease in accuracy with multiple people in the image.

Accuracy is not good when there are multiple people

Improve accuracy for emotion recognition module

The work done in the repo, is outstanding, and the reults are pretty good as well,, it's accuracy is much more than what you have mentioned in README,, I guess it's arround 78% or more,, but am really interested to know how I can contribute to improve the overall accuracy of the project, so will changing the dataset affect more or by using some other CNN module parameters will work, or instead can we make use of caffe module directly instead of tensorflow as backend,, so let me know, it will be fun to contribute this repo.

Thanks & Regards,
Swap

Emotion recognition model

HELLO,Oarriaga! I want to know the architecture about the following CNN Models.Would you like to introducing architecture of these models. What's differences between two models?
simple_CNN.530-0.65.hdf5
simple_CNN.985-0.66.hdf5

Feature Request: Automated Testing

Travis-CI? Or something similar?

error in train_emotion_classifier

I do have all required dependencies installed and working in other projects but when I run 'train_emotion_classifier.py' I get following traceback::

Traceback (most recent call last):
File "C:/Users/barot/Desktop/ml work/face_classification-master/src/train_emotion_classifier.py", line 11, in
from keras.preprocessing.image import ImageDataGenerator
File "C:\Users\barot\AppData\Local\Programs\Python\Python35-32\lib\site-packages\keras\preprocessing\image.py", line 10, in
from scipy import linalg
File "C:\Users\barot\AppData\Local\Programs\Python\Python35\Lib\site-packages\scipy_init_.py", line 114, in
from scipy._lib._ccallback import LowLevelCallable
File "C:\Users\barot\AppData\Local\Programs\Python\Python35\Lib\site-packages\scipy_lib_ccallback.py", line 1, in
from . import _ccallback_c
File "C:\Users\barot\AppData\Local\Programs\Python\Python35\Lib\site-packages\scipy_lib_ccallback_c.py", line 7, in
bootstrap()
File "C:\Users\barot\AppData\Local\Programs\Python\Python35\Lib\site-packages\scipy_lib_ccallback_c.py", line 6, in bootstrap
imp.load_dynamic(name,file)
File "C:\Users\barot\AppData\Local\Programs\Python\Python35\Lib\imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.

I have been trying to understand but I can't and I am preparing a bigger project of which this is an important part and I am close to my deadline.I am student so I don't really have lot of experience so I will appreciate your help and time.Peace

no inference was made

can you please let me know what I should do? Nothing gets shown:

(tensorflow) [jalal@goku src]$ python  image_emotion_gender_demo.py ../images/test_image.jpg
Using TensorFlow backend.
2017-11-07 11:33:38.133391: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-07 11:33:38.133428: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-07 11:33:38.133440: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-11-07 11:33:38.133449: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-11-07 11:33:38.133460: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-11-07 11:33:38.429151: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 0 with properties:
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.6705
pciBusID 0000:05:00.0
Total memory: 10.91GiB
Free memory: 10.00GiB
2017-11-07 11:33:38.668576: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0x40e5650 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that.
2017-11-07 11:33:38.669351: I tensorflow/core/common_runtime/gpu/gpu_device.cc:955] Found device 1 with properties:
name: GeForce GTX 1080 Ti
major: 6 minor: 1 memoryClockRate (GHz) 1.6705
pciBusID 0000:06:00.0
Total memory: 10.91GiB
Free memory: 10.75GiB
2017-11-07 11:33:38.670061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:976] DMA: 0 1
2017-11-07 11:33:38.670082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 0:   Y Y
2017-11-07 11:33:38.670123: I tensorflow/core/common_runtime/gpu/gpu_device.cc:986] 1:   Y Y
2017-11-07 11:33:38.670151: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080 Ti, pci bus id: 0000:05:00.0)
2017-11-07 11:33:38.670177: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1045] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080 Ti, pci bus id: 0000:06:00.0)
(tensorflow) [jalal@goku src]$