pplonski / keras2cpp

This is a bunch of code to port Keras neural network model into pure C++.

keras2cpp

This is a bunch of code to port Keras neural network model into pure C++. Neural network weights and architecture are stored in plain text file and input is presented as vector<vector<vector<float> > > in case of image. The code is prepared to support simple Convolutional network (from MNIST example) but can be easily extended. There are implemented only ReLU and Softmax activations.

It is working with the Theano backend.

Usage

1. Save your network weights and architecture.
2. Dump network structure to plain text file with dump_to_simple_cpp.py script.
3. Use network with code from keras_model.h and keras_model.cc files - see example below.

Example

1. Run one iteration of simple CNN on MNIST data with example/mnist_cnn_one_iteration.py script. It will produce files with architecture example/my_nn_arch.json and weights in HDF5 format example/my_nn_weights.h5.
2. Dump network to plain text file python dump_to_simple_cpp.py -a example/my_nn_arch.json -w example/my_nn_weights.h5 -o example/dumped.nnet.
3. Compile example g++ -std=c++11 keras_model.cc example_main.cc - see code in example_main.cc.
4. Run binary ./a.out - you shoul get the same output as in step one from Keras.

Testing

If you want to test dumping for your network, please use test_run.sh script. Please provide there your network architecture and weights. The script do following job:

1. Dump network into text file.
2. Generate random sample.
3. Compute predictions from keras and keras2cpp on generated sample.
4. Compare predictions.

Similar repositories

• Keras to C++ with usage of Tensorflow C API https://github.com/aljabr0/from-keras-to-c

keras2cpp's Issues

run binary error

Hi when I run binary, I have got the following error:

Reading model from ./dumped.nnet
Layers 221
Layer 0 InputLayer
Layer is empty, maybe it is not defined? Cannot define network.
Segmentation fault (core dumped)

could you please let me know what is the problem.

When the top layer must be excluded in Keras...

Hello?

Thanks for nice upload and I have a question about this situation.

In Keras code,

VGG16(weights='imagenet',
          include_top=False,
          input_shape=(224, 224, 3))

In this case the top layer is excluded and there also is an external weight for it.
I wonder this will work without any issue.

Thanks,

issue while testing the test_run.sh file

Dear Expert,
I followed these steps and completed the first four steps now I want to test dumping for your network, and use test_run.sh script but i do not understand how it will run and how to get the C++ code file which is used in fluent. please describe it.

Support for multi-input and merge layer?

Hi, I find this project very useful and interesting! Thanks a lot!
Will you add support for multi-input and merge layer?

Thanks

Traceback (most recent call last): File "keras2cpp/dump_to_simple_cpp.py", line 33, in <module> fout.write('layer ' + str(ind) + ' ' + l['class_name'] + '\n') TypeError: string indices must be integers

Hi I have got another error here:

Traceback (most recent call last):
File "keras2cpp/dump_to_simple_cpp.py", line 33, in
fout.write('layer ' + str(ind) + ' ' + l['class_name'] + '\n')
TypeError: string indices must be integers

could you please let me know why?

ValueError: Dimension 1 in both shapes must be equal, but are 3 and 4 for 'Assign_564' (op: 'Assign') with input shapes: [2048,3], [2048,4].

Hi pplonski,

 I try to dump network to plain text file, I have pre generated model.json and hdf5 file, when I used your command I get the following errors:

ValueError: Dimension 1 in both shapes must be equal, but are 3 and 4 for 'Assign_564' (op: 'Assign') with input shapes: [2048,3], [2048,4].

Could you help me resolve this error, thank you

Network dumping error !

Hi, i am trying to use your keras2cpp functions to import my Keras model into my cpp project. However, when I run the python script to dump the model into text file, I realize that all Conv2D layers dont have the matrix values displayed in the text file.
I can send you my json and h5 files if you want. Thank you !!!

Compute_output and memory management

When computing the output of a model with
KerasModel::compute_output(keras::DataChunk *dc)
the input DataChunk (dc) is deleted. Since this DataChunk is created somewhere else, I think it is better to let the code that created the object to free it. To do this, a single line should be added to the code to prevent this first input to be deleted, here.

Add a condition to
delete inp;
like
if( inp != dc ) delete inp;

does it support 'Model' model ?

I see your example is ‘Sequential’ model.
but it looks like not support 'Model' model.
thanks

custom model architecture

hello

I want to use this model with my python architecture, do I need to rewrite my model architecture in c++ file? or modify keras_model.cc?

Dump model in binary file

Hello,
First of all I'd like to thank you for making this library.
It has been very handy for me.
However, I have a ~550MB Keras model(weigths) that turns into ~1.6GB plain text model to use with keras2cpp.
Is it too much difficult to make the library be able to read a binary file?
If I make it, I'll add a pull request.
Thanks in advance.

Compatible for python3 and new keras

app crash in example

Running Environment Setting for the Code

After a half-day attempt, I think i should share my running environment for the code.
Something following may help to avoid potential issues:

#1 Ubuntu 16.04 (If you want to run your model with GPU and python=2.7)
#2 Keras = 1.2.2 (as pplonski suggested, there is something wrong with Keras 2.x)
#3 theano = 0.9
#4 Make sure your .keras.json as follows which is different from Keras 2.X

{
    "image_dim_ordering": "th", 
    "epsilon": 1e-07, 
    "floatx": "float32", 
    "backend": "theano"
}

Ps:
ImportError: cannot import name inplace_increment
↓
Open your Teminal and input theano-cache purge

Sincerely thanks for your great work @pplonski 😃

Looking forward to the Keras 2.X methods and examples since it's so popular.

Looking for a way to receive MSCOCO's Architecture JSON File

Hi,
I would like to use your solution for convertion of the MS-COCO RCNN to CPP
I have the following
http://academictorrents.com/details/f993c01f3c268b5d57219a38f8ec73ee7524421a
which contains MS-COCO H5 Weights file
but I am not sure how should i get its Architecutre JSON in order to continue and convert it to cpp
any ideas?

Right Keras and Tensorflow version?

Hello Freund, Grate work! I run your test with Python 2.7 , Keras 1.2.2 Tensorflow 1.0.1 and have wrong result:

.......
Compile keras2cpp code
Run predictions with dumped network and random data sample from step 2
Testing network from test_cnn.dumped on data from test_random_input.dat
DataChunk2D 1x28x28
DataChunkFlat values:
0.0631625 0.092902 0.0489044 0.0912403 0.0523701 0.128736 0.0515021 0.180591 0.192843 0.0977486
Test, step 4
Compare Keras and Keras2cpp outputs
Test: [ERROR]
The output from Keras and Keras2cpp are different.
Difference value: 0.154017945
Cleaning after test

What version of this items do you use ?

Skip dropout rate error

Hi,

I have noticed a potential issue in the following code:

keras2cpp/keras_model.cc

Lines 431 to 433 in ce407cc

	} else if(layer_type == "Dropout") {
	continue; // we dont need dropout layer in prediciton mode
	}

Are you sure that we do not need to include dropout layer in prediction mode? In Figure 2 of Srivastava et al. (2014), they say that in training, the weights are randomly set to 0 with probably equal to the dropout rate. In prediction mode, the dropout rate is still there but is simply multiplied to all weights in the layer - which disagrees with the code.

Additionally, I have noticed major differences in my python keras models vs keras2cpp models with dropout when using the default keras_model.cc. Then, when the weights are multiplied by dropout rate, the error goes away.

Reference
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I. and Salakhutdinov, R., 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1), pp.1929-1958.

LSTM to CPP, who has good method?

Cant verify results

I am able to recover the result in C++ as in your comment of mnist_cnn_one_iteration.py

# on my pc I got:
#[[ 0.03729606 0.00783805 0.06588034 0.21728528 0.01093729 0.34730983
# 0.01350389 0.02174525 0.26624694 0.01195715]]

Yet my python script yields a totally different prediction:

$ python mnist_cnn_one_iteration.py 
Using Theano backend.
X_train shape: (60000, 1, 28, 28)
60000 train samples
10000 test samples
Train on 60000 samples, validate on 10000 samples
Epoch 1/1
60000/60000 [==============================] - 20s - loss: 1.8239 - val_loss: 1.1273
Prediction on saved sample:
[[ 0.01161582  0.1554343   0.13277179  0.30984354  0.02740803  0.12834443
   0.00520513  0.02088989  0.13061689  0.07787018]]

My pc configs:

• ubuntu 16.04
• keras 1.2
• theano 1.0

Pls help...

Convolution2D with border_mode='same' incorrect

The Convolution2D layer with border_mode='same' does not give the same results as the Keras/Theano version (border_mode='valid' is ok in my test).

memory error while running example

I'm able to walk through the entire example until I get to running the dumped file. When I run:
./a.out

I get this error:

Layers 12
Layer 0 Convolution2D
Layer 0 Convolution2D
Layer 0 Convolution2D
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)

Any idea what's going on?

Issue dumping the network to .nnet file

Hey, I have issue while dumping the network into dat file.
Basically, 2nd step :
2. Dump network to dat file python dump_to_cpp.py -a arch.json -w weights.h5 -o dumped_nn.dat.

while, I run this script dump_to_cpp.py , it throws an error:

Using TensorFlow backend.
Read architecture from arch.json
Read weights from weights.h5
Writing to converted_to_cpp.dat
0 name
Traceback (most recent call last):
File "dump_to_cpp.py", line 36, in
fout.write('layer ' + str(ind) + ' ' + l['class_name'] + '\n')
TypeError: string indices must be integers

The issue is, l should have been of data type 'list' but in our case it is a 'string'.

can you please rectify this issue and provide a working solution.

TypeError: string indices must be integers

Thank you for your work! I got the following error by running:
python dump_to_simple_cpp.py -a net.json -w net.h5 -o dumped.nnet

Writing to dumped.nnet
Traceback (most recent call last):
File "dump_to_simple_cpp.py", line 33, in
fout.write('layer ' + str(ind) + ' ' + l['class_name'] + '\n')
TypeError: string indices must be integers

The error occurred when ind=0. I checked my 'arch["config"]', which is a very long dict and I only show a small part of it here:

{'layers': [{'class_name': 'InputLayer', 'config': {'batch_input_shape': [None, 1, 160, 128], 'input_dtype': 'float32', 'sparse': False, 'name': 'input_1'}, 'inbound_nodes': [], 'name': 'input_1'}, {'class_name': 'Convolution2D', 'config': {'activity_regularizer': None, 'trainable': True, 'dim_ordering': 'th', 'bias': True, 'nb_row': 3, 'b_constraint': None, 'name': 'convolution2d_1', 'W_constraint': None, 'nb_col': 3, 'subsample': [1, 1], 'init': 'glorot_uniform', 'nb_filter': 32, 'border_mode': 'same', 'b_regularizer': None, 'W_regularizer': None, 'activation': 'relu'}, 'inbound_nodes': [[['input_1', 0, 0]]], 'name': 'convolution2d_1'}, {'class_name': 'Convolution2D', 'config': {'activity_regularizer': None, 'trainable': True, 'dim_ordering': 'th', 'bias': True, 'nb_row': 3, 'b_constraint': None, 'name': 'convolution2d_2', 'W_constraint': None, 'nb_col': 3, 'subsample': [1, 1], 'init': 'glorot_uniform', 'nb_filter': 32, 'border_mode': 'same', 'b_regularizer': None, 'W_regularizer': None, 'activation': 'relu'}, 'inbound_nodes': [[['convolution2d_1', 0, 0]]], 'name': 'convolution2d_2'}, ...

It seems the term 'l['class_name']' caused the problem. But I have no clue how to fix it. Could you please give me some suggestions? Thanks!

TypeError: ('Keyword argument not understood:', 'border_mode')

I get this error while running mnist_cnn_one_iteration.py. Kindly shed some clarity on how to remove this error

sample for prediction

Hi, I'm trying to use your code for use the neural net in cpp and I already have my dumped.nnet but I not completely understood de sample file for prediction and how can I use images for this, if you don't mind, can you explain me this?

ValueError: The shape of the input to "Flatten" is not fully defined

Hello @pplonski,

Thank you for this tutorial to convert simple keras to cpp.

when I run the first command to generate ".h5 " and ".json" file I get following error:

I tried changing backends to both Theano and Tensorflow in keras.json file but still could not resolve this error.

$ python example/mnist_cnn_one_iteration.py

error:

Using Theano backend.
X_train shape: (60000, 1, 28, 28)
60000 train samples
10000 test samples
example/mnist_cnn_one_iteration.py:49: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(4, (3, 3), padding="same", input_shape=(1, 28, 28...)
model.add(Convolution2D(nb_filters, nb_conv, nb_conv, border_mode='same',input_shape=(1, img_rows, img_cols)))
example/mnist_cnn_one_iteration.py:51: UserWarning: Update your Conv2D call to the Keras 2 API: Conv2D(4, (3, 3), padding="same")
model.add(Convolution2D(nb_filters, nb_conv, nb_conv, border_mode='same'))
Traceback (most recent call last):
File "example/mnist_cnn_one_iteration.py", line 56, in
model.add(Flatten())
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 476, in add
output_tensor = layer(self.outputs[0])
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 615, in call
output_shape = self.compute_output_shape(input_shape)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/core.py", line 481, in compute_output_shape
'(got ' + str(input_shape[1:]) + '. '
ValueError: The shape of the input to "Flatten" is not fully defined (got (0, 14, 4). Make sure to pass a complete "input_shape" or "batch_input_shape" argument to the first layer in your model.

My keras.json file looks like:

{
"image_dim_ordering": "th",
"epsilon": 1e-07,
"floatx": "float32",
"backend": "theano"
}

Please let me know about it.

Thank you.

Regards,
Bhushan

How to port Keras neural network model into C# ?

It's Great !

But now,i have a problem
I wanna make C# version from your c++ model,
but I can't do that..
If I pay for it,would you help me do that?

Support for keras2

hello!
I am using Keras2 and there is a compatibility problem due to conv2d layer which was changed in current version of Keras. Actually, the way it is stored in json file has also changed and as a result the current c++ code does not work as it is.

I made some changes in the code that I attach and now it works, however c++ and python do not provide the same results when conv2d is used. (If these two conv2D layers are omitted the results are the same) Even though the two outcomes are comparable, there are not the same so I would like to ask you if there is something more that needs to be changed.

The main change is the following:
in keras1.x convolutiond2D layer's weights are stored and their shape is [nb_filter][depth][row][cols], which gives for the first layer of the example the numbers [4][1][3][3]
In contrary, in Keras 2.x conv2D layer's weights are stored and their shape is now [row][cols][depth][nb_filters], which gives the numbers [3][3][1][4] (you can see this in the dumped file created).
So, in order to use current cpp I created m_rowsVec() in class keras::LayerConv2D : public Layer.
Then, I read the weights from dumped file and i store them there. Afterwards I make the transition to the old format and I store them in m_kernels(). (see lines 71 - 111 in keras_model.cc). The following code remains the same.

The code attached works fine for new Keras2.0.6 but the prediction results are not the same as when I run them in python. Any suggestions? Should they be the same?
keras2cpp(keras2 version).zip

Compatibility with normal multilayer perceptron

Hi @pplonski Thank you so much for this amazing keras converter.
I have MLP trained using keras and want to use in C++.

Is this working with MLP which has relu and linear as an activation function?
I noticed that you mentioned you focused on CNN.

Thank you very much in advance.
Masaki

no support to AveragePooling2D, BatchNormalization and Merge

Compile error in keras_model.cc

k1_size and k2_size are not declared in conv_single_depth_valid and conv_single_depth_same

newbie asking

how can I use colored images inputs?

how can I use your implementation with colored inputs, rgb images for example?

model with function API

Hi this is really great work!
I just want to provide something I find might be useful to others.
I find that in order to dump model correctly. We need to build model with sequential model and add Activation layer separately.
For example, the second method will get dumped correctly. While for the first method, dumped model has no Activation layer.

from keras.models import Sequential, Model
from keras.layers import Input, Dense, Dropout, Activation
def get_model_by_sequential():
        model = Sequential()
        model.add(Dense(64,input_dim=15,init = 'uniform',activation='relu'))
        model.add(Dense(128,init = 'uniform',activation='relu'))
        model.add(Dense(256,init = 'uniform',activation='relu'))
        model.add(Dense(1,activation='sigmoid'))
        return model

def get_model_by_sequential_with_separate_activation():
        model = Sequential()
        model.add(Dense(64,input_dim=15,init = 'uniform'))
        model.add(Activation('relu'))
        model.add(Dense(128,init = 'uniform'))
        model.add(Activation('relu'))
        model.add(Dense(256,init = 'uniform'))
        model.add(Activation('relu'))
        model.add(Dense(1))
        model.add(Activation('sigmoid'))
        return model

def get_model_by_functional_API():
        a = Input(shape=(15,))
        b = Dense(64,input_dim=15,init = 'uniform',activation='relu')(a)
        b = Dense(128,init = 'uniform',activation='relu')(b)
        b = Dense(256,init = 'uniform',activation='relu')(b)
        b = Dense(1,activation='sigmoid')(b)
        model = Model(input=a, output=b)
        return model

terminate called after throwing an instance of 'std::bad_alloc'

Test, step 3
Compile keras2cpp code
Run predictions with dumped network and random data sample from step 2
Testing network from test_cnn.dumped on data from test_random_input.dat
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc

I'm using it on win10 and i got this when trying to run test_bin.
What should i do?

# TypeError: softmax() got an unexpected keyword argument 'axis'
# Keras Version 2.1.6
#
# terminate called after throwing an instance of 'std::bad_alloc'
#   what():  std::bad_alloc
# Keras Version 2.0.1 -2.1.5
#
# TypeError: ('Keyword argument not understood:', 'input_dtype')
# collect2.exe: error: ld returned 1 exit status
# DataChunk2D 0x
# Keras Version 2.0.0

simpler c++ code

Hi guys, could we create a simpler c++ code?

maybe a plain C code and avoid external libs and file read

my idea is a fixed code with hardcoded parameters values, and only predicting "one row", no batch prediction and no GPU support, something very very simple

if possible don't use vector<> just use fixed arrays, something like treelite do with xgboost

only one function to predict + inline functions to activation functions sigmoid, relu, conv, etc:

double predict(double *data){
}

data is an array of input "features"

Prediction on model in cpp

How exactly do you return a prediction for the keras model in the cpp code? Thanks for the help!

Support for recurrent layers

Great work, thanks a lot!
I have trained a recurrent net using keras and am looking for ways to make predictions in a c++ application.
Will support for recurrent layers be added?
Thanks

Conv1D layer support

Hello,

Do you plan to add support for Conv1D layer? I don't currently see the handling for that.

Thanks.
Paddy