leaderj1001 / mobilenetv3-pytorch

Implementing Searching for MobileNetV3 paper using Pytorch

Implementing Searching for MobileNetV3 paper using Pytorch

• The current model is a very early model. I will modify it as a general model as soon as possible.

Paper

• Searching for MobileNetV3 paper
• Author: Andrew Howard(Google Research), Mark Sandler(Google Research, Grace Chu(Google Research), Liang-Chieh Chen(Google Research), Bo Chen(Google Research), Mingxing Tan(Google Brain), Weijun Wang(Google Research), Yukun Zhu(Google Research), Ruoming Pang(Google Brain), Vijay Vasudevan(Google Brain), Quoc V. Le(Google Brain), Hartwig Adam(Google Research)

Todo

• Experimental need for ImageNet dataset.
• Code refactoring

MobileNetV3 Block

Experiments

• For CIFAR-100 data, I experimented with resize (224, 224).
  

Datasets	Model	acc1	acc5	Epoch	Parameters
CIFAR-100	MobileNetV3(LARGE)	70.44%	91.34%	80	3.99M
CIFAR-100	MobileNetV3(SMALL)	67.04%	89.41%	55	1.7M
IMAGENET	MobileNetV3(LARGE) WORK IN PROCESS				5.15M
IMAGENET	MobileNetV3(SMALL) WORK IN PROCESS				2.94M

Usage

Train

python main.py

• If you want to change hyper-parameters, you can check "python main.py --help"

Options:

• --dataset-mode (str) - which dataset you use, (example: CIFAR10, CIFAR100), (default: CIFAR100).
• --epochs (int) - number of epochs, (default: 100).
• --batch-size (int) - batch size, (default: 128).
• --learning-rate (float) - learning rate, (default: 1e-1).
• --dropout (float) - dropout rate, (default: 0.3).
• --model-mode (str) - which network you use, (example: LARGE, SMALL), (default: LARGE).
• --load-pretrained (bool) - (default: False).
• --evaluate (bool) - Used when testing. (default: False).
• --multiplier (float) - (default: 1.0).

Test

python main.py --evaluate True

• Put the saved model file in the checkpoint folder and saved graph file in the saved_graph folder and type "python main.py --evaluate True".
• If you want to change hyper-parameters, you can check "python test.py --help"

Options:

• --dataset-mode (str) - which dataset you use, (example: CIFAR10, CIFAR100), (default: CIFAR100).
• --epochs (int) - number of epochs, (default: 100).
• --batch-size (int) - batch size, (default: 128).
• --learning-rate (float) - learning rate, (default: 1e-1).
• --dropout (float) - dropout rate, (default: 0.3).
• --model-mode (str) - which network you use, (example: LARGE, SMALL), (default: LARGE).
• --load-pretrained (bool) - (default: False).
• --evaluate (bool) - Used when testing. (default: False).
• --multiplier (float) - (default: 1.0).

Number of Parameters

import torch

from model import MobileNetV3

def get_model_parameters(model):
    total_parameters = 0
    for layer in list(model.parameters()):
        layer_parameter = 1
        for l in list(layer.size()):
            layer_parameter *= l
        total_parameters += layer_parameter
    return total_parameters

tmp = torch.randn((128, 3, 224, 224))
model = MobileNetV3(model_mode="LARGE", multiplier=1.0)
print("Number of model parameters: ", get_model_parameters(model))

Requirements

• torch==1.0.1

mobilenetv3-pytorch's Issues

about this function:_make_divisible()

Hi!!!
What does this function do? ---->>_make_divisible(), Both channels and exp_size are processed by this function， why? It seems that the original paper did not mention？

performance on CIFAR100

Hi,
Use your code on CIFAR100, only after 34 epoch, I got acc1:70.43, acc5:91.08 on CIFAR100_test_data with batch_size=64.
You use 80 epoch with batch_size=512 just got acc1:70.44 acc5:91.34?

Note:model_mode = 'LARGE'

1280 out features

MobileNetV3-Pytorch/model.py

Line 229 in 2cf3efa

out_conv2_out = _make_divisible(1280 * multiplier)

According to the paper in the 'small' model they use 1024 as an output features unlike the 'large' one where it is in deed 1280.

there is something wrong with load_data, in mode"CIFAR10",you load CIFAR100

数据集制作

请问如果用自己的数据训练大佬写的程序，数据集怎么制作了，1，不用再另外制作label，文件夹就是标签，有多少个文件夹就有多少类别；2，需要另外制作标签；

# Use .reshape(-1) instead of .view(-1) to ensure compatibility

I got an error 'view(-1)':
for k in topk:
correct_k = correct[:k].view(-1).float().sum(0, keepdim=True)
res.append(correct_k.mul_(100.0 / batch_size))

then I change the code and can run
for k in topk:
# Use .reshape(-1) instead of .view(-1) to ensure compatibility
correct_k = correct[:k].reshape(-1).float().sum(0, keepdim=True)
res.append(correct_k.mul_(100.0 / batch_size))

Some confusion about the data preprocessing.

I found that the "preocessing.py" codes use a simple transforms.RandomResizedCrop(224) to preprocess the input training data. Why don't you utilize aspect preserving resize before cropping? Thanks.

Identity is not the same thing as equality in Python

See #4

Any progress on ImageNet model training?

Hi, Thanks for your contribution.

I'm working training mobileNet v3 on ImageNet.

Have you finished trained the model? If so, could you share the model on ImageNet?

Confusing parm

In your large model you missed a 1414112 layer, please add it

dropout

It doesn't look like the dropout parameter is implemented. Have you trained with dropout (as mentioned in the paper).

Your checkpoint have some problem, how to use? Do you have a demo?

Incomplete git merge

https://github.com/leaderj1001/MobileNetV3-Pytorch/blob/master/model.py#L84-L89

MobileBlock中通道数没有扩张

`class MobileBlock(nn.Module):
def init(self, in_channels, out_channels, kernal_size, stride, nonLinear, SE, exp_size):
super(MobileBlock, self).init()
self.out_channels = out_channels
self.nonLinear = nonLinear
self.SE = SE
padding = (kernal_size - 1) // 2
self.use_connect = stride == 1 and in_channels == out_channels
if self.nonLinear == "RE":
activation = nn.ReLU
else:
activation = h_swish
self.conv = nn.Sequential(
nn.Conv2d(in_channels, exp_size, kernel_size=1, stride=1, padding=0, bias=False),
nn.BatchNorm2d(exp_size),
activation(inplace=True)
)
self.depth_conv = nn.Sequential(
nn.Conv2d(exp_size, exp_size, kernel_size=kernal_size, stride=stride, padding=padding, groups=exp_size),
nn.BatchNorm2d(exp_size),
)
if self.SE:
self.squeeze_block = SqueezeBlock(exp_size)

    self.point_conv = nn.Sequential(
        nn.Conv2d(exp_size, out_channels, kernel_size=1, stride=1, padding=0),
        nn.BatchNorm2d(out_channels),
        activation(inplace=True)
    )`

加粗的地方是不是应该是in_channel * exp_size?

model脚本的52行

your code is not correct

there must be some mistakes in your code, because when i use your code as backbone to train my model the loss is not decreasing but maintaining. And when i change to https://github.com/kuan-wang/pytorch-mobilenet-v3 it do decreased again! so, check your code, please.