treb1en / insightface_pytorch

Pytorch0.4.1 codes for InsightFace

InsightFace_Pytorch

Pytorch0.4.1 codes for InsightFace

---

1. Intro

• This repo is a reimplementation of Arcface(paper), or Insightface(github)
• For models, including the pytorch implementation of the backbone modules of Arcface and MobileFacenet
• Codes for transform MXNET data records in Insightface(github) to Image Datafolders are provided
• Pretrained models are posted, include the MobileFacenet and IR-SE50 in the original paper

---

2. Pretrained Models & Performance

IR-SE50 @ BaiduNetdisk, IR-SE50 @ Onedrive

LFW(%)	CFP-FF(%)	CFP-FP(%)	AgeDB-30(%)	calfw(%)	cplfw(%)	vgg2_fp(%)
0.9952	0.9962	0.9504	0.9622	0.9557	0.9107	0.9386

Mobilefacenet @ BaiduNetDisk, Mobilefacenet @ OneDrive

LFW(%)	CFP-FF(%)	CFP-FP(%)	AgeDB-30(%)	calfw(%)	cplfw(%)	vgg2_fp(%)
0.9918	0.9891	0.8986	0.9347	0.9402	0.866	0.9100

3. How to use

• clone

  git clone https://github.com/TropComplique/mtcnn-pytorch.git
  

3.1 Data Preparation

3.1.1 Prepare Facebank (For testing over camera or video)

Provide the face images your want to detect in the data/face_bank folder, and guarantee it have a structure like following:

data/facebank/
        ---> id1/
            ---> id1_1.jpg
        ---> id2/
            ---> id2_1.jpg
        ---> id3/
            ---> id3_1.jpg
           ---> id3_2.jpg

3.1.2 download the pretrained model to work_space/model

If more than 1 image appears in one folder, an average embedding will be calculated

3.2.3 Prepare Dataset ( For training)

download the refined dataset: (emore recommended)

• emore dataset @ BaiduDrive, emore dataset @ Dropbox
• More Dataset please refer to the original post

Note: If you use the refined MS1M dataset and the cropped VGG2 dataset, please cite the original papers.

• after unzip the files to 'data' path, run :

  python prepare_data.py
  

  after the execution, you should find following structure:

faces_emore/
            ---> agedb_30
            ---> calfw
            ---> cfp_ff
            --->  cfp_fp
            ---> cfp_fp
            ---> cplfw
            --->imgs
            ---> lfw
            ---> vgg2_fp

---

3.2 detect over camera:

• 1. download the desired weights to model folder:

• IR-SE50 @ BaiduNetdisk

• IR-SE50 @ Onedrive

• Mobilefacenet @ BaiduNetDisk

• Mobilefacenet @ OneDrive

• 2 to take a picture, run

  python take_pic.py -n name
  

  press q to take a picture, it will only capture 1 highest possibility face if more than 1 person appear in the camera

• 3 or you can put any preexisting photo into the facebank directory, the file structure is as following:

- facebank/
         name1/
             photo1.jpg
             photo2.jpg
             ...
         name2/
             photo1.jpg
             photo2.jpg
             ...
         .....
    if more than 1 image appears in the directory, average embedding will be calculated

• 4 to start

  python face_verify.py 
  

---

3.3 detect over video:

​```
python infer_on_video.py -f [video file name] -s [save file name]
​```

the video file should be inside the data/face_bank folder

• Video Detection Demo @Youtube

3.4 Training:

​```
python train.py -b [batch_size] -lr [learning rate] -e [epochs]

# python train.py -net mobilefacenet -b 200 -w 4
​```

4. References

• This repo is mainly inspired by deepinsight/insightface and InsightFace_TF

PS

• PRs are welcome, in case that I don't have the resource to train some large models like the 100 and 151 layers model
• Email : [email protected]

insightface_pytorch's Issues

Fine tuning on the model

Hi

Thanks for sharing your work , i have a few face pair's myself, i mean data looks like,
image1 , image2, match
image3 , image4, nomatch...

Could i fine tune the model using this dataset,

Thanks in advance.

pre-trained head .pth?

Hello,

Is there a pre-trained 'head' corresponding to the model_mobilefacenet.pth that the author shared?

Thank you.

Maybe a bug

Hi @TreB1eN , after reading the code, I found the follow codes may be a bug of your Arcloss.
cond_mask = cond_v > 0
keep_val = (cos_theta - self.mm) # when theta not in [0,pi], use cosface instead
cos_theta_m[cond_mask] = keep_val[cond_mask]
I think it should be : cond_mask = cond_v <= 0 . In fact, you implement a special Am-softmax with margin = mm, instead of ArcLoss.

What dataset was used for training IR-SE50 model?

Hi! What dataset was used for training IR-SE50 model? Is it VGGFace2?

Question about training

Hi, thanks for your great work. I would like to test if ArcFace works really well since eventually I need to use different network instead of ResNet/mobilenet. I would like to ask some questions, wish someone can answer me because I'm so confused now.

1. I downloaded the MS1M_85k_5.8M dataset, I am wondering if there's any preprocess steps such as MTCNN to crop the face or just directly feed the images into training?

2. I use my own face detector to crop the faces(remove the forehead) in both MS1M dataset(20k identities, 1.2M images) and LFW dataset off. When I used ResNet34(directly from torchvision.models) + ArcFace(s64, m=0.5, directly used yours). I only got 80% validation acc. However, when I trained on ResNet34+Center loss, I got 98% validation acc. Does it mean that Arcface can only perform well on full face?

Please help me out and if someone wants to make the test I please contact me and I will upload my dataset. ([email protected]).

Thanks in advance

内存问题

InsightFace_Pytorch/utils.py

Line 34 in 1e0bea0

names = ['Unknown']

原始代码会保留梯度，浪费大量内存，建议在这里加上torch.no_grad()，应该可以部分解决问题

Some errors about image color

Hi,
Thanks for your great contribute on this open source work! But I have a few questions!
Firstly, I downloaded the faces_webface_112x112.zip from InsightFace Dataset zoo. Then I unzipped
the data and followed your instruction to generate training imgs but I got such images!


It looks like an issue about RGB channel and BGR channel. So i wonder whether its color is correct.
Thank you!

No such file or directory: 'work_space/save/model_final.pth'

Hi there,

When I issue "python face_verify.py"
there will be the following error message:

$ python face_verify.py
mtcnn loaded
{'batch_size': 100, 'emore_folder': PosixPath('data/faces_emore'), 'test_transform': Compose(
    ToTensor()
    Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
), 'embedding_size': 512, 'face_limit': 10, 'data_mode': 'emore', 'ms1m_folder': PosixPath('data/faces_ms1m_112x112'), 'work_path': PosixPath('work_space'), 'log_path': PosixPath('work_space/log'), 'net_depth': 50, 'facebank_path': PosixPath('data/facebank'), 'net_mode': 'ir_se', 'drop_ratio': 0.6, 'vgg_folder': PosixPath('data/faces_vgg_112x112'), 'device': device(type='cuda', index=0), 'save_path': PosixPath('work_space/save'), 'min_face_size': 30, 'threshold': 1.5, 'model_path': PosixPath('work_space/models'), 'use_mobilfacenet': False, 'input_size': [112, 112], 'data_path': PosixPath('data')}
ir_se_50 model generated
Traceback (most recent call last):
  File "face_verify.py", line 31, in <module>
    learner.load_state(conf, 'final.pth', True, True)
  File "/home/robert/workingbox/InsightFace_Pytorch/Learner.py", line 82, in load_state
    self.model.load_state_dict(torch.load(save_path/'model_{}'.format(fixed_str)))
  File "/home/robert/inf1/lib/python3.5/site-packages/torch/serialization.py", line 356, in load
    f = open(str(f), 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'work_space/save/model_final.pth'

My system info as follows:

OS: Ubuntu 16.04.5 LTS x86-64
CUDA: release 9.0, V9.0.176
Python: 3.5.2
torch: 0.4.1
torchvision: 0.2.1
mxnet: 1.3.1
mxnet_cu90: 1.3.1
tensorboardX: 1.4
opencv-python: 3.4.4.19

How can I generate or get the file "model_final.pth" ?

Thanks

Maybe a bug in mtcnn.py

class MTCNN():
def init(self):
self.pnet = PNet().to(device)
self.rnet = RNet().to(device)
self.onet = ONet().to(device)
self.onet.eval()
self.rnet.eval()
self.onet.eval()
self.refrence = get_reference_facial_points(default_square= True)

Why u run onet.eval() twice?Should it be pnet.eval()?

the reference to mobileFaceNet

Hi,

the architecture of MobileFacenet is from this paper(https://arxiv.org/abs/1804.07573)?

how to train on multi-GPUs

I want to train with multiple GPUs, where should I change?

What version of python ?

Hi

I am facing some difficulties in running the path module, i just wanted to know what version of python are you using ?

Why pre-trained model has lower performance than that from MXNET?

Hi, thanks for your great work.

I noticed that the pretrained model has general lower performance than that from MXNET. May I ask did you train from scratch or did you convert the pretrained model from MXNET directly to PyTorch please?

which is model folder ?

which is model folder ?

NameError: name 'module' is not defined

Traceback (most recent call last):
File "train.py", line 28, in
learner = face_learner(conf)
File "/home/InsightFace_Pytorch-master/Learner.py", line 37, in init
paras_only_bn, paras_wo_bn = separate_bn_paras(self.model)
File "/home/InsightFace_Pytorch-master/utils.py", line 16, in separate_bn_paras
modules = [*module.modules()]
NameError: name 'module' is not defined
how to slove it ?thanks~

Environment setting problem(POSIXPATH error)

I've clone this code and run pip3 -r requirement.txt

But I met some problem due to invalid arguments

ex:
os.path.join didn't support POSIXPATH

my environment is
ubuntu 16.0.4
python 3.5.1

Have you ever transferred your pytorch model into caffemodel?

Have you ever transferred your pytorch model into caffemodel? Or is there any way we can do that?

why trans.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])

hi, you really did a great job. Here I have a question.
why trans.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5]) not mean = [0.485, 0.456, 0.406]
stdv = [0.229, 0.224, 0.225] ?
can you help me?

code seems do not support resume training from a saved weight?

I was training a model using CASIA-Webface and it stopped in some where of total epochs accidentally. So I've add some lines in Learner.py and tried to resume training but got failed.
here is my resuming code:

 def train(self, conf, epochs,resume=False,fixed_str=None):
        self.model.train()
        conf.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
        self.model = nn.DataParallel(self.model)
        self.head = nn.DataParallel(self.head)
        self.model.cuda()
        self.head.cuda()
        start_epoch=0
        if resume==True:
            if not fixed_str:
                raise ValueError('must input fixed_str parameter!')
            self.load_state(conf,fixed_str)
            self.step = int(fixed_str.split('_')[-2].split(':')[1])+1
            start_epoch = self.step//len(self.loader)
            self.step = start_epoch*len(self.loader)+1
            print('loading model at epoch {} done!'.format(start_epoch))
            print(self.optimizer)
        running_loss = 0.
        dc_loss = 0.
        bceloss_func = nn.BCELoss()
        for e in range(start_epoch,epochs):
            print('epoch {} started'.format(e))
            if e == self.milestones[0]:
                self.schedule_lr()
            if e == self.milestones[1]:
                self.schedule_lr()
        #nothing changed below

I've changed nothing below.
The wired thing is that when I load whichever the weights of all of model, head and optimizer and then continue training, I got a very high CELoss. I tested it in a ipython notebook. When I random initialize a learner, I got CELoss around 45, but when I load a weights(which get a 93% acc on LFW) for the learner I got CELoss around 77.
I think the problem lies in the logic of class Arcface in Learner.py but I am not sure.
If anyone could help me figure out the issue?

Do you need image alignment during training or evalution?

Thanks.

Do huge batch size necessary?

Hi, I have tried to retrained the model with batch size 50, But it seems the model do not converge. I am wondering if the batch size is needed to be huge (more than 100)?

No such file or directory: 'data/facebank/facebank.pth'

When I try to run the 3.3, python infer_on_video.py. I encounter the above problem. Anyone can help me out?

AttributeError: module 'torch._C' has no attribute '_cuda_getDevice'

Why is the verification using l2 distance?

Why is the code for testing using l2 distance instead of cosine distance? Thank you.

        embed1_train = embeddings1[train_set]
        embed2_train = embeddings2[train_set]
        _embed_train = np.concatenate((embed1_train, embed2_train), axis=0)
        # print(_embed_train.shape)
        pca_model = PCA(n_components=pca)
        pca_model.fit(_embed_train)
        embed1 = pca_model.transform(embeddings1)
        embed2 = pca_model.transform(embeddings2)
        embed1 = sklearn.preprocessing.normalize(embed1)
        embed2 = sklearn.preprocessing.normalize(embed2)
        # print(embed1.shape, embed2.shape)
        diff = np.subtract(embed1, embed2)
        dist = np.sum(np.square(diff), 1)

margin weight?

Thank to repo!
You can share margin weight?

Training

HI, @TreB1eN Thanks for sharing the code of this repo.

I'm just wondering do we have a model training from scratch using this pipeline?

Download link about mentioned 'emore' dataset in README.md

Hello, thanks for your pytorch implementation of insight-face!!

In the setcion: 3.2.3 Prepare Dataset ( For training)
u mentioned using so called emore dataset for trainning,
but the dataset download link is the same as the VGGFace2 dataset.

#### 3.2.3 Prepare Dataset ( For training)
download the refined dataset from original post: (emore recommended)
* [Refined-MS1M@BaiduDrive](https://pan.baidu.com/s/1nxmSCch), [Refined-MS1M@GoogleDrive](https://drive.google.com/file/d/1XRdCt3xOw7B3saw0xUSzLRub_HI4Jbk3/view)
* [VGGFace2@BaiduDrive](https://pan.baidu.com/s/1c3KeLzy), [VGGFace2@GoogleDrive](https://drive.google.com/open?id=1KORwx_DWyIScAjD6vbo4CSRu048APoum)
* [emore dataset @ BaiduDrive](https://pan.baidu.com/s/1c3KeLzy), [emore dataset @ OneDrive](https://pan.baidu.com/s/1c3KeLzy)

Maybe a bug in pytorch 1.0 in computing the cos_theta in model.py

Hi,thanks for your contribution. I've read your code carefully and found that there are few lines of code in model.py that really puzzled me. On line 275 in model.py, you wrote label = label.view(-1, 1), then on line 277, you wrote output[torch.range(0,nB-1).to(torch.long),label] = cos_theta_m[torch.range(0,nB-1).to(torch.long),label]. Here label is a 2-rank matrix, and cos_theta_m[torch.range(0, nB-1).to(torch.long), label] will choose the whole column of the orginial matrix instead of the very single element of the ground truth value. So it might overwrite other entries of the cos matrix whose label is NOT the column id.

No such file or directory: 'data/facebank/names.npy

python face_verify.py occur a error:

about test_transform params for model_ir_se50.pth

Hi, i use test_transform in config , and the model performs poorly on my own test data sets .the accuracy is about 10~15 percents lower than the same net structure trained on mxnet with ms1m.

is bgr format image necessary in training and validating? thankyou

NameError: name 'module' is not defined

Traceback (most recent call last):
File "train.py", line 28, in
learner = face_learner(conf)
File "/home/jy/qianchen/InsightFace_Pytorch-master/Learner.py", line 37, in init
paras_only_bn, paras_wo_bn = separate_bn_paras(self.model)
File "/home/jy/qianchen/InsightFace_Pytorch-master/utils.py", line 16, in separate_bn_paras
modules = [*module.modules()]
Traceback (most recent call last):
File "train.py", line 28, in
learner = face_learner(conf)
File "/home/jy/qianchen/InsightFace_Pytorch-master/Learner.py", line 37, in init
paras_only_bn, paras_wo_bn = separate_bn_paras(self.model)
File "/home/jy/qianchen/InsightFace_Pytorch-master/utils.py", line 16, in separate_bn_paras
modules = [*module.modules()]
NameError: name 'module' is not defined

how to solve it? thanks

why do you divide CE loss by batch_size?

Hello, at the code below:

InsightFace_Pytorch/Learner.py

Line 203 in b0e21f6

running_loss += loss.item()/conf.batch_size

,
I guess the loss computed by nn.CrossEntropy() is already element-wise averaged.
So div by conf.batch_size seems not to be necessary, right?
Thank you.

I get the acc 56% by 5 epoch which make me saaaaaaaaaaad

with epoch = 4 , batch_size = 256 , number workers = 3 , others parameter is the same as author.
and with 4 GPUs training ms1m in my server .
finally , get the accuracy rate = 56% , it's a very sad result. I don't know where I got wrong.
I don't believe the cause is the such short epoch ~~~

---

[email protected]

Disordered RGB Channels after Running "python prepare_data.py" (fixed).

After running "python prepare_data.py" I can get the image data as described in your repo. However, I found that the images may have disordered RGB channels. The faces are all blue and nearly all images have this problem. I have found the issue causing this, which can be solved by simply inserting "img = cv2.cvtColor(img, cv2.COLOR_RGB2BGR)" to line 92 right before "img = Image.fromarray(img)" in the script ./data/data_pipe.py

Learning rate

Using the default learning rate 0.001 will not work on my side.

Just let you know that I tried 0.01 and that number worked very well. :)

maybe few differences with mxnet

Hi, @TreB1eN , I'm learning your codes for few days ,it's very nice work, but compared the network with mxnet version: 2 key differences found as following:
1 . in bottleneck, res layer, there might be a bn between first conv and prelu as mxnet
2 . your method of short_cut :

       if depth == in_channel:
           self.shortcut_layer = MaxPool2d(1, stride)
       else:
           self.shortcut_layer = Sequential(
               Conv2d(in_channel, depth, (1, 1), stride ,bias=False), BatchNorm2d(depth))

lead the first get_block, first unit's short_cut become MaxPool2d(1,2) but not Conv+BN because of the input data channel=64=depth, that's diffferent with mxnet codes. maybe use :

          if stride == 1:
                ....
          else:
                ....

I don't know whether it would be the reason of low megaface results, and other differences like : mxnet seemed no use se model to train res50, why do we use? and the drop_ratio = 0.4 in mxnet but 0.6 in torch.
do you have any suggestions?

I will change the above and train

Did you reproduce megaface ?

Hi

Did you reproduce megaface result on pytorch?

thank you

Run on CPU?

Hi

Thanks for sharing Your work, I was wondering if i could run this only on the CPU?

Pre-trained Model performs poorly on MegaFace challenge-1

Hi, @TreB1eN thank you for your excellent work. I've got a problem as below. Since you leave your ipython notebook results here I believe that the pre-trained model you provided IR-SE50 @BaiduNetdisk has already converged so I did not test it on LFW or other verification dataset. However I got a very poor result on MegaFace challenge-1 (only 22% acc on 10 distractor) which is a Identification task. Now I am working on back to testing the model on face-emore but the data processing is slow so I don't have the result yet. I wonder have you ever test your model on identification task? How does it perform?

Do you know where is the source of vgg2_fp?

Do you know where is the source of VGG2_FP? what I can find is the aligned vgg2_fp, not original vgg2_fp.

training paras for mobilefacenet

Hi @TreB1eN ,

I found this line in the config.py file:
conf.milestones = [3,4,5] # mobildefacenet
but the milestones is not used during training, i guess that means learning rate is not decay.
Learner.py +line 225:
seems like self.schedule_lr() should be called during training according to the paper for mobilefacenet.

BTW, would you please share the training paras for mobilefacenet to reproduce your acc?
batch size,init learning rate.

Thanks for your help!

how to load pre-trained model and continue training?

How to change your code to be able to load and train the model, then continue training ? I tried to modify 'Learner.py' as follows, but it didn't work.

def train(self, conf, epochs):
self.load_state(conf, '××××××××××××××.pth', True, True)

Error message is:
AttributeError: 'PosixPath' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.

Can anybody here give me some good advice?

Save generated .pth value to Database

Is this feasible to store the 512 dimension embedding to python database?

GPU memory required to train r100

Thank you for your great work! However, I met a problem about limitation of GPU memory. Can you give me some suggestion about the minimum GPU memory required to train r100 ArcFace by your code?

NameError: name 'module' is not defined

Traceback (most recent call last):
File "train.py", line 28, in
learner = face_learner(conf)
File "/home/InsightFace_Pytorch-master/Learner.py", line 37, in init
paras_only_bn, paras_wo_bn = separate_bn_paras(self.model)
File "/home/InsightFace_Pytorch-master/utils.py", line 16, in separate_bn_paras
modules = [*module.modules()]
NameError: name 'module' is not defined
how to solve it ? thanks

MobileFaceNet is not right

The Flatten function in here https://github.com/TreB1eN/InsightFace_Pytorch/blob/master/model.py:

class Flatten(Module):
    def forward(self, input):
        return input.view(input.size(0), -1)

Will not guarantee the output of this layer has 512 channels which used by:

self.linear = Linear(512, embedding_size, bias=False)

in the definition of MobileFaceNet:
https://github.com/TreB1eN/InsightFace_Pytorch/blob/master/model.py#L209

Really appreciate your work!

about"TypeError: coercing to Unicode: need string or buffer, PosixPath found"?

hello,thanks for sharing your work.I try to train "ir_se" network but the error of "TypeError: coercing to Unicode: need string or buffer, PosixPath found" appeared. My running environment is "pytorch-0.4.1+python3.6", and also run environment of "pytorch-0.4.1+python2.7", but the problem appeared all the time .Can you tell me what I should do or can you help me?

Saved opt.pth is very large and seems overlap with head.pth and model.pth?

While saving state with model_only option False, I found the size of opt.pth is exactly the sum of head.pth and model.pth? Is it normal?

Train on Own Dataset

How can I train on my own dataset?