Results Using OpenCV DNN Module about face_segmentation

[YuvalNirkin]

The 300x300 model has only two output channels.

Does that mean the output should only be a 2x300x300 and not a 21x300x300? Then, in the following block of code from face_seg.py, are you simply taking the maximum of the two channels?

# run net and take argmax for prediction
net.forward()
out = net.blobs['score'].data[0].argmax(axis=0)

The end result must be a binary mask, but all of the outputs I'm seeing in the 21x300x300 array I get are float values, many of which are even negative. I'm wondering if the output of net.forward() may be different in the actual Caffe library and in openCV's readFromCaffe. I'm not sure though...

For reference, here is my code:

import numpy as np
import cv2

image = cv2.imread('Alison_Lohman_0001.jpg')

# Define prototext and caffemodel paths, and create model
caffeModel = "face_seg_fcn8s.caffemodel"
prototextPath = "face_seg_fcn8s_deploy.prototxt"
net = cv2.dnn.readNetFromCaffe(prototextPath,caffeModel)

# Resize to 300x300
image = cv2.resize(image,(300,300))
# blobImage convert RGB (104.00698793,116.66876762,122.67891434)
blob = cv2.dnn.blobFromImage(image,1.0,(300,300),(104.00698793,116.66876762,122.67891434))

# Passing blob through network
net.setInput(blob)
output = net.forward()

output is a 1x21x300x300 float32

Ah I see what you mean about the 300x300 model. It produces a 1x2x300x300 output.

I'm still unsure of what to do with the two resulting images. After normalization, they look like this:

for the Alison_Lohman_0001.jpg image

[kiralygomba]

Did you find the answer to this? I'm trying to use it in openCV as well, same problems.

@kiralygomba no unfortunately not.

[charleswg]

@kiralygomba looks like the only thing missing is the last operation:
mask=output[0].argmax(axis=0)
mask=1*(mask>0)

Though I must say it doesn't quite work as I thought