broadinstitute / keras-resnet

Keras package for deep residual networks

Keras-ResNet

Keras-ResNet is the Keras package for deep residual networks. It's fast and flexible.

A tantalizing preview of Keras-ResNet simplicity:

>>> import keras

>>> import keras_resnet.models

>>> shape, classes = (32, 32, 3), 10

>>> x = keras.layers.Input(shape)

>>> model = keras_resnet.models.ResNet50(x, classes=classes)

>>> model.compile("adam", "categorical_crossentropy", ["accuracy"])

>>> (training_x, training_y), (_, _) = keras.datasets.cifar10.load_data()

>>> training_y = keras.utils.np_utils.to_categorical(training_y)

>>> model.fit(training_x, training_y)

Installation

Installation couldn’t be easier:

$ pip install keras-resnet

Contributing

1. Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug. There is a Contributor Friendly tag for issues that should be ideal for people who are not very familiar with the codebase yet.
2. Fork the repository on GitHub to start making your changes to the master branch (or branch off of it).
3. Write a test which shows that the bug was fixed or that the feature works as expected.
4. Send a pull request and bug the maintainer until it gets merged and published. :) Make sure to add yourself to AUTHORS.

keras-resnet's Issues

ResNeXt-C block

Implement the block diagramed in figure 3a. from “Aggregated Residual Transformations for Deep Neural Networks.”

Starter code in README

I just tried running the starter code in the README (training ResNet50 on cifar10) to get a feel for the package. I got an error when constructing a model:

'>>> model = keras_resnet.classifiers.ResNet50(x, classes=classes)
Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'keras_resnet' has no attribute 'classifiers'`

(residual) Inception-v4 blocks (and models)

Very deep convolutional networks have been central to the largest advances in image recognition performance in recent years. One example is the Inception architecture that has been shown to achieve very good performance at relatively low computational cost. Recently, the introduction of residual connections in conjunction with a more traditional architecture has yielded state-of-the-art performance in the 2015 ILSVRC challenge; its performance was similar to the latest generation Inception-v3 network. This raises the question of whether there are any benefit in combining the Inception architecture with residual connections. Here we give clear empirical evidence that training with residual connections accelerates the training of Inception networks significantly. There is also some evidence of residual Inception networks outperforming similarly expensive Inception networks without residual connections by a thin margin. We also present several new streamlined architectures for both residual and non-residual Inception networks. These variations improve the single-frame recognition performance on the ILSVRC 2012 classification task significantly. We further demonstrate how proper activation scaling stabilizes the training of very wide residual Inception networks. With an ensemble of three residual and one Inception-v4, we achieve 3.08 percent top-5 error on the test set of the ImageNet classification (CLS) challenge

https://arxiv.org/abs/1602.07261

The freeze_bn parameter does not work for TimeDistributed models

I get the following error:

    net = keras_resnet.models.TimeDistributedResNet18(input_set, include_top=False, freeze_bn=True)
  File "/home/ubuntu/.pyenv/versions/3.6.2/lib/python3.6/site-packages/keras_resnet/models/_time_distributed_2d.py", line 125, in TimeDistributedResNet18
    return TimeDistributedResNet(inputs, blocks, block=keras_resnet.blocks.time_distributed_basic_2d, include_top=include_top, classes=classes, *args, **kwargs)
  File "/home/ubuntu/.pyenv/versions/3.6.2/lib/python3.6/site-packages/keras_resnet/models/_time_distributed_2d.py", line 87, in TimeDistributedResNet
    return keras.models.Model(inputs=inputs, outputs=outputs, *args, **kwargs)
  File "/home/ubuntu/.pyenv/versions/3.6.2/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
    return func(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'freeze_bn'

Also, the default value for freeze_bn is different in for 2D and TimeDistributed models.

Wide Residual Networks

@0x00b1 Do you think it would be interesting to have Wide Residual Networks

Shape inference

Mistaken channel axis?

Should a_shape[3] == b_shape[3] at block.py#L110 be a_shape[1] == b_shape[1]?

Please cut a release that includes 1D ResNet.

I realized that while the master branch includes a 1DResnet, the latest release does not. This has led to confusion among users (namely, myself) as I'm not in a position to be cutting local installations of this repo and am looking to use it via pip install.

Thanks!!

Travis CI integration

Let’s use Travis CI to run tests for new commits.

Publishing this on conda

Hey, I am using conda for my development but I could not install keras-resnet using conda install keras-resnet. So I was thinking, could you add keras-resnet as a conda package? The current process for a package on pypi but not on conda is a headache.

A small suggestion because of the problem I was facing.

Error with broadcasting in ResNet18

My input data size is (112, 112, 1), but I receive this error when trying to run the Resnet18 model on my data.

Traceback (most recent call last):
  File "/src/sar_optical_matching.py", line 183, in <module>
    if __name__=="__main__": main()
  File "/src/sar_optical_matching.py", line 146, in main
    model = model_def.build(input_shape, verbose=args.verbose, one_hot=args.one_hot)
  File "/src/models/resnet_siamese.py", line 21, in build
    y_opt = keras_resnet.models.ResNet18(x_opt)
  File "/opt/conda/envs/custom/lib/python3.6/site-packages/keras_resnet/models.py", line 103, in __init__
    super(ResNet18, self).__init__(inputs, [2, 2, 2, 2], block)
  File "/opt/conda/envs/custom/lib/python3.6/site-packages/keras_resnet/models.py", line 61, in __init__
    x = block(features, strides, j == 0 and k == 0)(x)
  File "/opt/conda/envs/custom/lib/python3.6/site-packages/keras_resnet/block/__init__.py", line 48, in f
    y = _shortcut(x, y)
  File "/opt/conda/envs/custom/lib/python3.6/site-packages/keras_resnet/block/__init__.py", line 121, in _shortcut
    return keras.layers.add([a, b])
  File "/opt/conda/envs/custom/lib/python3.6/site-packages/keras/layers/merge.py", line 455, in add
    return Add(**kwargs)(inputs)
  File "/opt/conda/envs/custom/lib/python3.6/site-packages/keras/engine/topology.py", line 560, in __call__
    self.build(input_shapes)
  File "/opt/conda/envs/custom/lib/python3.6/site-packages/keras/layers/merge.py", line 84, in build
    output_shape = self._compute_elemwise_op_output_shape(output_shape, shape)
  File "/opt/conda/envs/custom/lib/python3.6/site-packages/keras/layers/merge.py", line 55, in _compute_elemwise_op_output_shape
    str(shape1) + ' ' + str(shape2))
ValueError: Operands could not be broadcast together with shapes (7, 7, 512) (4, 4, 512)

Transfer Learning

I generated the docs as I saw something on transfer learning, however, the docs don't seem complete. Is there a way to load the ReNet18 with weights based on ImageNet or CIFAR? Or has this not yet been implemented?

ResNet-1001

An implementation of ResNet-1001 from He, et al.’s “Identity Mappings in Deep Residual Networks” would be useful for completeness.

He’s implementation is available.

1D has 2D kernels

Seems 1D models are set to use 2D kernels. I went to use a 1D model and got the error:

ValueError: The kernel_size argument must be a tuple of 1 integers. Received: (7, 7)

Use Sphinx to auto-generate documentation

It’d be nice if keras-resenet, was, y’know, documented.

Upate this repo as it is still using old implementation of keras

How should I load the trained model with keras？

When I try to load a trained model, I get the error:
ValueError: Unknown layer: ResNet2D50
I tried add custom objects while loading:
{'ResNet2D50': keras_resnet.models.ResNet2D50}
but I find it useless
@hgaiser @bzamecnik

ResNet-200

keras-resnet is missing the ResNet-200 implementation from He, et al.’s “Deep Residual Learning for Image Recognition.” I believe this is the only missing model from that paper.

README example should have flatten layer before dense layer

Syntax warning due to comparison of literals using is in Python 3.8

find . -iname '*.py' | grep -Ev 'vendor|example|doc|tools|sphinx' | xargs -P4 -I{} python3.8 -Wall -m py_compile {}
./keras_resnet/benchmarks/__init__.py:74: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if benchmark is "MNIST":

Args to Conv3D were given in incorrect format.

keras-resnet/keras_resnet/models/_3d.py

Line 77 in 898a1ee

x = keras.layers.Conv3D(64, (7, 7), strides=(2, 2), use_bias=False, name="conv1")(x)

keras-resnet/keras_resnet/models/_3d.py

Line 80 in 898a1ee

x = keras.layers.MaxPooling3D((3, 3), strides=(2, 2), padding="same", name="pool1")(x)

Kernel_size of a Conv3D can be a single integer or a tuple of 3 integers!

And similarly with the strides too.

expected average_pooling2d_1 to have 4 dimensions

import keras_resnet.models
import keras

shape, classes = (32, 32, 3), 10

x = keras.layers.Input(shape)

print(x)

model = keras_resnet.ResNet50(x)
model.summary()
model.compile("adam", "categorical_crossentropy", ["accuracy"])

(x, y), (_, _) = keras.datasets.cifar10.load_data()

y = keras.utils.np_utils.to_categorical(y)

model.fit(x, y)
+++++++++++++++++++++++++++++++++
I ran the above test code in Python 2.7.9 and but I got:

ValueError: Error when checking model target: expected average_pooling2d_1 to have 4 dimensions, but got array with shape (50000, 10)

NameError: name 'ryx' is not defined

Looks like a bug in the file /usr/local/lib/python3.6/site-packages/keras_resnet/blocks/_time_distributed_2d.py

Maybe it's supposed to by y instead of ryx?

At this code:

def f(x):
    y = keras.layers.TimeDistributed(keras.layers.ZeroPadding2D(padding=1), name="padding{}{}_branch2a".format(stage_char, block_char))(x)
    y = keras.layers.TimeDistributed(keras.layers.Conv2D(filters, kernel_size, strides=stride, use_bias=False, **parameters), name="res{}{}_branch2a".format(stage_char, block_char))(ryx)

E NameError: name 'ryx' is not defined

Exploding memory usage on TimeDistributedResNet

I'm trying to use TimeDistributedResNet50 in my project and the memory usage explodes (used up 64GB RAM and 35+GB Swap memory before my computer dies) with a sequence length of only 3. Can you confirm if this is expected?

I tried TimeDistributed(VGG16) and the memory usage is only 1.9GB at sequence length 20.

TensorFlow 2.0: Problem with tf.keras and keras

Hello,

is there a version for "tf.keras"? Because I used "tf.keras" in the codes, however the keras-resnet is in "keras" version. Thus, I got an error caused by a version mismatch between Keras installation and Tensorflow.

Thus, I am wondering is there a version for "tf.keras"?

Layer names

Currently it appears the layers are not named in keras-resnet. Naming the layers makes it easier to debug, because it is easier to identify layers (Tensorboard for instance). In addition, using names identical to those defined in https://github.com/fchollet/keras/blob/master/keras/applications/resnet50.py allows to transfer weights by name from the Keras ResNet50 pretrained network (transfer learning, as in #22).

Is this something that is being worked on? Is it desired? If it is not worked on and desired, I can pick it up.

Refactor blocks into sub-classes of keras.model.Model

By sub-classing keras.models.Model we avoid the ceremony of the existing block pattern.

Use a sensible default kernel initializer

keras-resnet uses glorot_uniform by default and there is no easy way to use a more appropriate initializer. We should either use a more robust initalizer (like He normal) or make this easily configurable.

Where can we download the pretrained weight?

Add Wide Residual Networks support

Wide Residual Networks (Zagoruyko & Komodakis) have proven well in image classification. They prioritize width instead of the traditional thin and deep structure of classical resnets, thus allowing a faster training due to GPU or TPU parallelization.

That would increase the speed of research since less time would be consumed to train the Networks. Those models are already implemented in Keras @
https://github.com/EricAlcaide/keras-wrn and the code ca just be copy-pasted and added to this package.

How can we add or change a layer

Please guide me on how can I change the in btw layer or last layer

shape, classes = (150, 150, 3), 11
x = keras.layers.Input(shape)

model = keras_resnet.models.ResNet50(x, classes=classes)
model.add(Dense(11, activation = "softmax")) <----- How can I add this layer? or make it last layer
model.compile(Adam(lr=0.0001), loss='binary_crossentropy', metrics=['accuracy'])

Thanks,
Arindam

Training ResNet50 on ImageNet does not achieve reasonable performance

I previously had no problems training the ResNet50 implementation bundled with Keras in keras.applications.resnet50 to 70% validation accuracy on the ILSVRC 2012 dataset.

Now I wanted to switch to the keras_resnet implementation, but was not able to get validation accuracy above 30%. Right after the first epoch, accuracy of keras_resnet is about 1%, while the bundled ResNet50 already achieves 11%.

I am creating the ResNet like this:

input_ = keras.layers.Input((3, None, None)) if K.image_data_format() == 'channels_first' else 
keras.layers.Input((None, None, 3))
rn = keras_resnet.models.ResNet50(input_, include_top = True, classes = 1000, freeze_bn = False)

I've already tried different learning rate schedules and optimizers, but nothing worked.

Is there anything special I have to take care of?

load model?

Hi @0x00b1, when I was trying to load the saved resnet18, got the error: "ResNet18' object has no attribute 'load' . Can we have the load function implemented?

Difference with 0.2.0 vs 0.1.0

The network architecture in 0.2.0 is no longer identical to that in the original Caffe implementation. I am using the ResNet50 imagenet weights available here, which were generated using the caffe weights and this tool.

On some Tabby Cat image I get the following predictions from 0.1.0:

[[('n02123045', 'tabby', 0.4307788), ('n02124075', 'Egyptian_cat', 0.32408533), ('n02123159', 'tiger_cat', 0.18477823), ('n02127052', 'lynx', 0.008598777), ('n03443371', 'goblet', 0.0048426227)]]

On 0.2.0:

[[('n02123045', 'tabby', 0.39859685), ('n02124075', 'Egyptian_cat', 0.26450825), ('n02123159', 'tiger_cat', 0.22655205), ('n02127052', 'lynx', 0.012361869), ('n03443371', 'goblet', 0.00793589)]]

If I find some more time, I will also show the results using Caffe. But for now, it is clear that there is an unintended difference between 0.2.0 and 0.1.0. I believe this difference comes from 5005c37 . @0x00b1 do you know why that commit was applied?

Unable to serialize a resnet model to a saved model

The newest keras version allow us to save models not only as .h5 keras files, but also as saved model.

However, custom layers must be correctly declared to support this.

I.e the BatchNormalization layer should define its signature for "call" properly as:
def call(self, inputs, **kwargs):
instead of

def call(self, *args, **kwargs):

You can read more about this issue here:
tensorflow/tensorflow#38384

Issue with the 3D networks

Hello, I can not make 3D resnets, it throughs an exception. The reason is that in the _3d.py file, the functions like Conv3D or MaxPooling3D are given 2D inputs. e.g., kernel=(7,7) while it should be either 7 or (7,7,7). The same for /blocks._3d.py.

allow prefixes to the layer names

If one wants to have two ResNets in the same model it doesn't seem to be possible since the two ResNets will have layers with the same name. It could be possible to get around this by adding optional prefixes to the layers.

Test whether networks have the correct number of layers

In addition to testing whether a network has the correct number of parameters, we should test whether the network has the correct number of layers.

1D ResNet?

I've been trying to use this resnet on a 1D feature tensor..DNA-sequence! but I get errors using the default ResNet object since I do not have enough dimensions. I tried writing my own ResNet1D using the framework that you have going here, but have failed, thus far.

A few pointers would be appreciated!

error 1d load model

When load a model

`
# save model
model_json = model.to_json()
with open("model.json", "w") as json_file:
json_file.write(model_json)

# load json and create model
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
model = model_from_json(loaded_model_json, custom_objects={
    'ResNet1D18': keras_resnet.models.ResNet1D18,
    'BatchNormalization': keras_resnet.layers.BatchNormalization })

`

I get this error:

Shape must be rank 3 but is rank 4 for 'padding_conv1_2/Pad'

Travis CI has changed their pricing structure for open source projects

Travis has just announced a change in their pricing structure, and no longer offer free builds for open source projects. https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing

If those changes are incompatible with the needs of this project, it might be necessary to switch to another CI service.