matrixgame2018 / senet-tensorflow

Simple Tensorflow implementation of "Squeeze and Excitation Networks" using Cifar10 (ResNeXt, Inception-v4, Inception-resnet-v2)

SENet-Tensorflow

Simple Tensorflow implementation of Squeeze Excitation Networks using Cifar10

I implemented the following SENet

• ResNeXt paper
• Inception-v4, Inception-resnet-v2 paper

If you want to see the original author's code, please refer to this link

Requirements

• Tensorflow 1.x
• Python 3.x
• tflearn (If you are easy to use global average pooling, you should install tflearn)

Issue

Image_size

• In paper, experimented with ImageNet
• However, due to image size issues in Inception network, so I used zero padding for the Cifar10

input_x = tf.pad(input_x, [[0, 0], [32, 32], [32, 32], [0, 0]]) # size 32x32 -> 96x96

NOT ENOUGH GPU Memory

• If not enough GPU memory, Please edit the code

with tf.Session() as sess : NO
with tf.Session(config=tf.ConfigProto(allow_soft_placement=True)) as sess : OK

Idea

What is the "SE block" ?

def Squeeze_excitation_layer(self, input_x, out_dim, ratio, layer_name):
    with tf.name_scope(layer_name) :
        squeeze = Global_Average_Pooling(input_x)

        excitation = Fully_connected(squeeze, units=out_dim / ratio, layer_name=layer_name+'_fully_connected1')
        excitation = Relu(excitation)
        excitation = Fully_connected(excitation, units=out_dim, layer_name=layer_name+'_fully_connected2')
        excitation = Sigmoid(excitation)

        excitation = tf.reshape(excitation, [-1,1,1,out_dim])

        scale = input_x * excitation

        return scale

How apply ? (Inception, Residual)

 

How "Reduction ratio" should I set?

• original refers to ResNet-50

ImageNet Results

Benefits against Network Depth

Incorporation with Modern Architecture

Comparison with State-of-the-art

Cifar10 Results

Will be soon

Related works

• Densenet-Tensorflow
• ResNeXt-Tensorflow
• ResNet-Tensorflow

Reference

• Inception_korean

Author

Junho Kim