kentsommer / keras-inceptionv4 Goto Github PK

Firstly, thanks for the implementation. I was using this model for some experiments and noticed that it fills up the K80 while training. This seems to be logical since the network has ~170 layers, but it is limiting the batch size to very small numbers (8 samples per batch).

I had two questions:

• Am I doing something wrong that it fills up 11GB of the K80's memory while training. This means that the model wont even load on smaller GPUs?
• Is multi-GPU the only way to get past the batch-size problem? I was also looking to use the InceptionV4 in a multi-tower architecture. Seems that this will require multi-GPU too.

Was your work based on titu1994's code: https://github.com/titu1994/Inception-v4 ? It's published under MIT license. Can you include it to your repository as well?

ValueError: Layer #1 (named "conv2d_1"), weight <tf.Variable 'conv2d_1/kernel:0' shape=(3, 3, 32, 32) dtype=float32, numpy=
array[...] has shape (3, 3, 32, 32), but the saved weight has shape (32, 3, 3, 3)

Hi! How do you extract the AveragePooling2D layer?

In keras for inceptionV3, it is:
model = Model(inputs=base_model.input, outputs=base_model.get_layer('avg_pool').output)

Hello,

when running under TensorFlow version: 1.2.0 the output of evaluate_image.py is:

Class is: African elephant, Loxodonta africana
Certainty is: 0.775282

There are slight changes to how concat works in tf in comparison to keras. Is this a problem?

I have now evaluated imagenet and it resulted in a @1error of 0.26176. Do you have any suggestions or a code that achieves the numbers from the paper?

Probably not code specific, but:
What is the reason for not having the auxiliary output?

First of all, thank for your code, it is so helpful to me.
I am a freshman of inception model and cv. I hope to learn the model from your project. I run the project but the answer always be "sandbar" "dog" or something else and the "Certainty" is around 0.001. I change nothing and I don't konw why I could not get the right answer.
By the way ,could you explain the use of "preprocess_input"? The number all be negative after processed by it.

Would be good to have a "nice to have feature" of Include_top = True / False similar to inception v3 implementation in Keras.

Hi!

I found that after switching to Keras-2, training kind of pauses after every epoch for
a noticeable amount of time.

This doesn't happen when training network from scratch for example.

Didn't observe such behaviour when using previous version of inception_v4 with keras-1.

Why after run evaluate_image.py, i got
Class is: African elephant, Loxodonta africana
Certainty is: 0.7752825
it seems not match Class is: African elephant, Loxodonta africana
Certainty is: 0.868498

I am trying to train the model on custom dataset but I got this error. The expected shape and found shape are the same. I don't understand

WARNING:tensorflow:`period` argument is deprecated. Please use `save_freq` to specify the frequency in number of batches seen.
Epoch 1/5
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-43-c72e19685f40>](https://localhost:8080/#) in <module>
     21   train_ds,
     22   validation_data=val_ds,
---> 23   epochs=epochs,
     24 )

1 frames
[/usr/local/lib/python3.7/dist-packages/keras/engine/training.py](https://localhost:8080/#) in tf__train_function(iterator)
     13                 try:
     14                     do_return = True
---> 15                     retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
     16                 except:
     17                     do_return = False

ValueError: in user code:

    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1051, in train_function  *
        return step_function(self, iterator)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1040, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 1030, in run_step  **
        outputs = model.train_step(data)
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/training.py", line 889, in train_step
        y_pred = self(x, training=True)
    File "/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py", line 67, in error_handler
        raise e.with_traceback(filtered_tb) from None
    File "/usr/local/lib/python3.7/dist-packages/keras/engine/input_spec.py", line 264, in assert_input_compatibility
        raise ValueError(f'Input {input_index} of layer "{layer_name}" is '

    ValueError: Input 0 of layer "model_2" is incompatible with the layer: expected shape=(None, 299, 299, 3), found shape=(None, 229, 229, 3)

Here is how I create the model

base_model = create_model(num_classes=2, dropout_prob=0.2, weights="imagenet", include_top=False)

x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation='relu')(x)
predictions = Dense(2, activation='softmax')(x)

model = Model(inputs=base_model.input, outputs=predictions)

for layer in base_model.layers:
    layer.trainable = False

opt = Adam(learning_rate=0.001)
model.compile(optimizer=opt,loss='sparse_categorical_crossentropy',metrics=['accuracy'])
model.summary()

#Train the model
epochs = 5
history = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=epochs,
)

IMAGE_SIZE = (229, 229)
batch_size = 32
train_ds = tf.keras.preprocessing.image_dataset_from_directory(
    dataset_dir,
    validation_split=0.2,
    subset="training",
    seed=1337,
    image_size=IMAGE_SIZE,
    batch_size=batch_size,
)

val_ds = tf.keras.preprocessing.image_dataset_from_directory(
    dataset_dir,
    validation_split=0.2,
    subset="validation",
    seed=1337,
    image_size=IMAGE_SIZE,
    batch_size=batch_size,
)

This is preprocessing function of your inception v4.

def preprocess_input(x):
    x = np.divide(x, 255.0)
    x = np.subtract(x, 0.5)
    x = np.multiply(x, 2.0)
    return x

why It is totally different from other models preprocessing?
1. Why there is no mean subtraction?
2. Why there is no RGB to BGR? Instead you used RGB
3. Mapping between [-1,1] or [-x, +x] is normal for all inception models?

This is preprocessing function of VGG and ResNet in Keras:

def preprocess_input(x, data_format=None):
    if data_format is None:
        data_format = K.image_data_format()
    assert data_format in {'channels_last', 'channels_first'}

    if data_format == 'channels_first':
        # 'RGB'->'BGR'
        x = x[:, ::-1, :, :]
        # Zero-center by mean pixel

        x[:, 0, :, :] -= 103.939
        x[:, 1, :, :] -= 116.779
        x[:, 2, :, :] -= 123.68
    else:
        # 'RGB'->'BGR'
        x = x[:, :, :, ::-1]
        # Zero-center by mean pixel
        x[:, :, :, 0] -= 103.939
        x[:, :, :, 1] -= 116.779
        x[:, :, :, 2] -= 123.68
    return x

Also Caffe models use mean subtraction and RGB to BGR.

The current repository does not work with keras 2
Could you please create a version that is suitable for it?

Many thanks!

sorry, my mistake

First, I try to create the model
base_model = create_model(weights="imagenet")
output:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-4-6e62a586cf6e>](https://localhost:8080/#) in <module>
----> 1 base_model = create_model(weights="imagenet")

3 frames
[/usr/local/lib/python3.7/dist-packages/keras/saving/hdf5_format.py](https://localhost:8080/#) in load_weights_from_hdf5_group_by_name(f, model, skip_mismatch)
    842             continue
    843           raise ValueError(
--> 844               f'Shape mismatch in layer #{k} (named {layer.name}) for weight '
    845               f'{symbolic_weights[i].name}. '
    846               f'Weight expects shape {expected_shape}. Received saved weight '

ValueError: Shape mismatch in layer #1 (named conv2d_1) for weight conv2d_1/kernel:0. Weight expects shape (3, 3, 32, 32). Received saved weight with shape (32, 3, 3, 3)

After, I run it again without restart runtime.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-5-6e62a586cf6e>](https://localhost:8080/#) in <module>
----> 1 base_model = create_model(weights="imagenet")

3 frames
[/usr/local/lib/python3.7/dist-packages/keras/saving/hdf5_format.py](https://localhost:8080/#) in load_weights_from_hdf5_group_by_name(f, model, skip_mismatch)
    842             continue
    843           raise ValueError(
--> 844               f'Shape mismatch in layer #{k} (named {layer.name}) for weight '
    845               f'{symbolic_weights[i].name}. '
    846               f'Weight expects shape {expected_shape}. Received saved weight '

ValueError: Shape mismatch in layer #479 (named conv2d_149) for weight conv2d_149/kernel:0. Weight expects shape (3, 3, 3, 32). Received saved weight with shape (256, 1536, 1, 1)

Then, I run it again. There is no error occur again. Is it suppose to raise an error for shape mismatch or something?

In version 2.1, loading the model like this:
inception_v4.create_model(include_top=False, weights='imagenet')
throws an exception:

Traceback (most recent call last):
...
File "/models/inception_v4/inception_v4.py", line 268, in inception_v4
  model.load_weights(weights_path, by_name=True)
File "/venv/lib/python3.6/site-packages/keras/engine/network.py", line 1177, in load_weights
  reshape=reshape)
File "/venv/lib/python3.6/site-packages/keras/engine/saving.py", line 1018, in load_weights_from_hdf5_group_by_name
  str(weight_values[i].shape) + '.')
ValueError: Layer #305 (named "conv2d_98"), weight <tf.Variable 'conv2d_98/kernel:0' shape=(3, 3, 64, 96) dtype=float32_ref> has shape (3, 3, 64, 96), but the saved weight has shape (192, 1024, 1, 1).

//edit: Seems like the issue occurs only if I have InceptionV3 model used along with InceptionV4 in the same script.

A parameter of preprocessing seems different from the TensorFlow version.
In this Keras implementation, you subtract 1.0 from the image (https://github.com/kentsommer/keras-inceptionV4/blob/master/evaluate_image.py#L11).
On the other hand, in the TensorFlow implementation, they subtract 0.5 (https://github.com/tensorflow/models/blob/master/slim/preprocessing/inception_preprocessing.py#L273).

Why did you change this value?

Hi

When I try to run this line of code:
x = create_model(num_classes=2, weights='imagenet', include_top=False)

it says :

Layer #1 (named "conv2d_1"), weight
<tf.Variable 'conv2d_1/kernel:0' shape=(3, 3, 32, 32) dtype=float32, numpy=
array([[[[-0.04279904, -0.03153341, 0.13684416, ...,
has shape (3, 3, 32, 32), but the saved weight has shape (32, 3, 3, 3).

Am I doing something wrong or it is the weight file problem?

In Inception V3 at https://github.com/fchollet/deep-learning-models/blob/master/inception_v3.py#L391, preprocess is defined as

def preprocess_input(x):
    x /= 255.
    x -= 0.5
    x *= 2.
    return x

But in Inception V4 at https://github.com/kentsommer/keras-inceptionV4/blob/keras-2/evaluate_image.py#L9

def preprocess_input(x):
    x = np.divide(x, 255.0)
    x = np.subtract(x, 1.0)
    x = np.multiply(x, 2.0)
    return x

I wonder why it becomes subtraction by 1 instead of 0.5? I thought 0.5 was easier to understand as it scaled the range to be between [-1, 1] after x2. Could you please point out where is the reference in the paper for the preprocessing steps? I couldn't find it at https://arxiv.org/pdf/1602.07261v1.pdf. Thanks!

FYI, I tried both values for the included elephant image. Their certainties are

x = np.subtract(x, 1.0): 0.852957
x = np.subtract(x, 0.5): 0.775283

The second is lower, but still pretty high.

Since Tensorflow and Theano implement convolutions in different ways it is necessary to port a new set of weights for use with the Theano backend with TH dim ordering.

If I remember the paper correctly Inception-ResNet-v2 has comparable accuracy but trains faster.

BTW I think the paper URL formatting in README needs some fixing.

In the released version 2.1, if This release targets Keras-2. These weights work with either Theano or Tensorflow (in either dim ordering), isn't it better to rename the weights file as

inception-v4_weights_tf_th_dim_ordering_tf_th_kernels-for_keras_2
inception-v4_weights_tf_th_dim_ordering_tf_th_kernels_notop-for_keras_2

to make them more "newbie-friendly" ?

After downloading the code, and running evaluate_image.py on elephant.jpg (everything default, without changing a single line of code or the image) i get:

Loaded Model Weights!
Class is: sandbar, sand bar
Certainty is: 0.00110015

The same (sandbar) appears for different elephant images I have found.
The ordering is TF, the backend is TF as well.

Hi, I was trying to remove fully connected layer and only use pre-trained convolution layers to do fine tuning. However, the result was pretty bad. I saw in evaluate_image.py you applied:
def preprocess_input(x): x = np.divide(x, 255.0) x = np.subtract(x, 1.0) x = np.multiply(x, 2.0) return x

It seems like we need to normalize input image to range [-2, 0]. Then I followed this procedure, it do improved a little bit, but still very bad.

I used pretrained ResNet 50 from Kreas to fine tune the model, and got 92% accuracy on testing data, but when I applied the same way, but with Inception-v4 net, I only got 70% accuracy. Something must be wrong, but I couldn't figure out.

3/18 edit:

I found that someone said (http://stackoverflow.com/a/39597537/7555390) for inception the input has to be normalized to [-1, 1], but your code seems like to make it [-2, 0], is that a typo?

thanks for your amazing work!
i just want to know how can i transfer origin inceptionv4 slim model to keras model
i run get_inception_weights.py firstly, and convert_weights_to_keras.py secondly（note that i have #if "dense" in layer.name:
# continue
）,finally i get inception-v4_weights_tf_dim_ordering_tf_kernels.h5. But it is different from inception-v4_weights_tf_dim_ordering_tf_kernels.h5 from https://github.com/kentsommer/keras-inceptionV4/releases/download/2.1/inception-v4_weights_tf_dim_ordering_tf_kernels.h5, could you help me ? i am looking forward for your reply

@kentsommer you should submit a PR to get this merged into Keras's Application module

Hi ,

Is it possible to fine tune the model with pretrained weights on ImageNet for custom dataset with 3 classes ?

Thanks
Rahul

I use Tensorflow backend and Keras 2.0.3.
When I run this code by:
-- python evaluate_image.py
without changing anything from the command line.
I got:
Class is: sandbar, sand bar
Certainty is: 0.00110018

When I try with another image, the result is the same.
I don't know what the reason of this.

Hi,

It's probably not the correct place to pose this question, but I'd like to modify the InceptionV4 model you provided (default) in a way to meet the requirements of a project I am working on. Since you've written the InceptionV4 code you can tell me perhaps how to branch off.

What I want to achieve is the following:

In order to get a feature map of 71x71x384, branch off before the last pooling layer within the stem and append one 'Inception-A' block.

Thereafter, successively branch off after the "Inception-A' blocks for a 35x35x384 feature map, after the 'Inception-B' blocks for a 17x17x384 feature map and after the 'Inception-C' blocks for a 9x9x1536 map.

Also extend the network with two more parts: first a 'Reduction-B' followed by two 'Inception-C- blocks to output a 5x5x1024 map. Second, one 'Reduction-B' and one 'Inception-C' to produce 3x3x1024 map.

This text itself is not really understandable as I am looking at the InceptionV4 graph and can't seem to understand how to branch off exactly as it is written. I've "branched off" within the stem before the last pooling layer (as it is said in the text):

branch_0 = conv2d_bn(net, 192, 3, 3, strides=(2, 2), padding='valid')
branch_a =  block_inception_a(net)
branch_1 = MaxPooling2D((3, 3), strides=(2, 2), padding='valid')(net)

net = concatenate([branch_0, branch_1 , branch_a], axis=channel_axis)

but got an error:

ValueError: A Concatenatelayer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 35, 35, 192), (None, 35, 35, 192), (None, 71, 71, 384)]
which clearly means I did not branch off properly. What does "branch off for a AxBxC feature map" mean in this context? To add another conv2d_bn function with the A,B,C parameters?

Am I misunderstanding the text or is there something wrong with the branching strategy of the text? Could you give me some insights if possible? (Gonna close the issue afterwards as this is not a bug or anything to worth an issue).

Thanks in advance.