getcrest / all-conv-keras Goto Github PK

hi vibrantabhi19 :

Thank you for sharing your code! That's very helpful for me to understand All-CNN.

In addition, I've trained it last with your model night with 350 epochs, however found its accuracy (i.e. val_acc) became stable (about 0.81) after epoch 49 and remained the same to the end

Any ideas? :) 👍

The model I used:

`
model = Sequential()

model.add(Conv2D(96, (3, 3), padding="same", input_shape=(32, 32, 3)))
model.add(Activation('relu'))
model.add(Conv2D(96, (3, 3), padding="same"))
model.add(Activation('relu'))
model.add(Conv2D(96, (3, 3), padding="same", strides=2))
model.add(Dropout(0.5))

model.add(Conv2D(192, (3, 3), padding="same"))
model.add(Activation('relu'))
model.add(Conv2D(192, (3, 3), padding="same"))
model.add(Activation('relu'))
model.add(Conv2D(192, (3, 3), padding="same", strides=2))
model.add(Dropout(0.5))

model.add(Conv2D(192, (3, 3), padding="same"))
model.add(Activation('relu'))
model.add(Conv2D(192, (1, 1), padding="valid"))
model.add(Activation('relu'))
model.add(Conv2D(10, (1, 1), padding="valid"))

model.add(GlobalAveragePooling2D())
model.add(Activation('softmax'))

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])`

Thanks for the code. Without trying to look for reasons, it seems to fail for newer tensorflow/keras versions - though that could also be specific to my setup.

Anyway, I get:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimension 0 in both shapes must be equal, but are 1 and 3. Shapes are [1] and [3].

Full trace:

Using TensorFlow backend.
C:\Users\Deeplearning.keras\datasets\cifar-10-batches-py
X_train shape: (50000, 32, 32, 3)
50000 train samples
10000 test samples
(32, 32, 3)
C:/Users/Deeplearning/Desktop/DeepRepo2/greendatamining/DeepLearn/DeepCodeOwnNetwork/simplenet.py:78: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(96, (3, 3), input_shape=(3, 32, 32..., padding="same")
model.add(Convolution2D(96, 3, 3, border_mode='same', input_shape=(3, 32, 32)))
C:/Users/Deeplearning/Desktop/DeepRepo2/greendatamining/DeepLearn/DeepCodeOwnNetwork/simplenet.py:80: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(96, (3, 3), padding="same")
model.add(Convolution2D(96, 3, 3, border_mode='same'))
C:/Users/Deeplearning/Desktop/DeepRepo2/greendatamining/DeepLearn/DeepCodeOwnNetwork/simplenet.py:82: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(96, (3, 3), strides=(2, 2), padding="same")
model.add(Convolution2D(96, 3, 3, border_mode='same', subsample=(2, 2)))
C:/Users/Deeplearning/Desktop/DeepRepo2/greendatamining/DeepLearn/DeepCodeOwnNetwork/simplenet.py:85: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(192, (3, 3), padding="same")
model.add(Convolution2D(192, 3, 3, border_mode='same'))
C:/Users/Deeplearning/Desktop/DeepRepo2/greendatamining/DeepLearn/DeepCodeOwnNetwork/simplenet.py:87: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(192, (3, 3), padding="same")
model.add(Convolution2D(192, 3, 3, border_mode='same'))
C:/Users/Deeplearning/Desktop/DeepRepo2/greendatamining/DeepLearn/DeepCodeOwnNetwork/simplenet.py:89: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(192, (3, 3), strides=(2, 2), padding="same")
model.add(Convolution2D(192, 3, 3, border_mode='same', subsample=(2, 2)))
C:/Users/Deeplearning/Desktop/DeepRepo2/greendatamining/DeepLearn/DeepCodeOwnNetwork/simplenet.py:92: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(192, (3, 3), padding="same")
model.add(Convolution2D(192, 3, 3, border_mode='same'))
C:/Users/Deeplearning/Desktop/DeepRepo2/greendatamining/DeepLearn/DeepCodeOwnNetwork/simplenet.py:94: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(192, (1, 1), padding="valid")
model.add(Convolution2D(192, 1, 1, border_mode='valid'))
C:/Users/Deeplearning/Desktop/DeepRepo2/greendatamining/DeepLearn/DeepCodeOwnNetwork/simplenet.py:96: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(10, (1, 1), padding="valid")
model.add(Convolution2D(10, 1, 1, border_mode='valid'))
Traceback (most recent call last):
File "C:\Users\Deeplearning\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1576, in _create_c_op
c_op = c_api.TF_FinishOperation(op_desc)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimension 0 in both shapes must be equal, but are 1 and 3. Shapes are [1] and [3].
From merging shape 0 with other shapes. for 'tower_0/lambda_1/concat/concat_dim' (op: 'Pack') with input shapes: [1], [3].

The MaxPooling2D layers perform similar nonlinearity as ReLu so there is no need for activation function.
By replacing them with strided Conv2D we lose the nonlinearity effect and should add ReLu activation or the layer is basically useless (could be merged with next layer, because linear system).

Also the paper indicates that all Conv2D layers have ReLu activation.
@marcj maybe this gives your missing performance #4

hi vibrantabhi19 :

I think your article is great, so I try to execute the code you've provided. But I encountered some errors when I executed the code.

The last error message is as follows:
ValueError: Dimension 0 in both shapes must be equal, but are 1 and 3
From merging shape 0 with other shapes. for 'lambda_1/concat/concat_dim' (op: 'Pack') with input shapes: [1], [3].

So I would like to ask you may cause the wrong reason.You mentioned in the project environmental requirements are as follows:
Requirements:
keras with Tensorflow backend (keras version 1.0.4 or later)
h5py (if you want to save your model)
numpy
pandas (if you want to save the logs of your model)
cv2 (for image resizing)

And my operating environment is as follows:
Python 3.5.2 :: Anaconda 4.2.0 (64-bit)
TensorFlow Version:1.1.0
Keras Version:2.0.3
Using TensorFlow backend.

Is this the cause of the error?
Please help me deal with this error.
Thank you.