ayush1999 / keras.jl

Run keras models with a Flux backend

Keras.jl

Load Keras models in Julia.

This is not a wrapper around Keras. This is built on top of Flux, to directly load Keras models into Flux. [W.I.P]

How?

Loading a model in Flux is fairly simple. Clone this repository into ~/.julia/v0.6. Make sure you have all dependencies installed. In order to load a model, you need to have two files:

1. The model.json file. This stores the structure of the model. This can be obtained from any Keras model using the model.to_json() method.
2. The weights.h5 file. This stores the weights associated with different layers of the pre-trained Keras model. This file can be produced from a Keras model using Keras.save_weights(weight_file_name).

(The files can have any other name (as long as they are in the correct format). I'm using model.json and weights.h5 as an example here)

Keras models can broadly be divided into two categories:

1. The models using the sequential API.
2. The models using the functional API. (Also called Model API)

Due to subtle differences in their structure and functioning, you need to follow different steps to run these models in Flux. You can check the type of the model by:

>>> using Keras

>>> Keras.check_modeltype("model.json")

Running Sequential Models

>>> using Keras

>>> model = Keras.load("model.json", "weights.h5")

model is now the corresponding model in Flux. This can be used directly as:

>>> model(rand(28,28,1,1))

Another straight-forward way of running such models is:

>>> using Keras

>>> Keras.load("model.json", "weights.h5", ip)

Where ip is our input. This directly returns the models output.

Running Functional Models.

Functional models can be tricky as they may consist of a number of sub-graphs within themselves. Running such models is similar to the second way of running Sequential models mentioned above.

>>> using Keras

>>> Keras.load("model.json", "weight.h5", ip)

Where ip is the input to our model. This directly returns the output. (Note: Currently there is no other way of running functional API models).

Intermediate outputs

Keras.jl also allows you to get the intermediate outputs of a model. Suppose your model contains m layers, and you need the output after n layers (m > n).

>>> model[1:n](ip)

Should give you the output after exactly n layers.

Insight

The process of loading and running a Keras model in Flux mainly consists of two parts:

1. Converting all Keras operators to Flux ops.
2. Generating the computation graph from the Flux operators obtained.

In order to get correct results, make sure that the value of mode parameter is set to 0 (here). It's default value is 0, so if you haven't played around with NNlib.jl, you're good to go!

Issues

Since this is currently under development, feel free to open any issue you encounter. You can post your queries on Julia Slack, generally on the #machine-learning channel.

Current Impediments:

Lambda layers cannot be handled at this moment. This is because we'd need to handle the Python AST, for parsing it as JSON.

keras.jl's Issues

Errors recompiling Keras on Julia 1.3.1

I see that the package is a bit stale and has not been upgraded to work on Julia 1, however I was able to install it seamlessly in Julia 1.3.1 by executing Pkg.add(Pkg.PackageSpec(url="https://github.com/ayush1999/Keras.jl", rev="master")). Executing using Keras returned errors I wasn't able to fix. The full output is below. Any suggestion is greatly appreciated.

[ Info: Precompiling Keras [031d7243-eb6a-5793-a290-e3c72503329c]
WARNING: importing deprecated binding Compat.Libdl into Blosc.
WARNING: importing deprecated binding Compat.Libdl into Blosc.
WARNING: Method definition iterate(DataFlow.ObjectArraySet{T} where T, Any...) in module DataFlow at /home/adude/.julia/packages/Lazy/mAoZN/src/macros.jl:285 overwritten at /home/adude/.julia/packages/Lazy/mAoZN/src/macros.jl:285.
  ** incremental compilation may be fatally broken for this module **

ERROR: LoadError: LoadError: LoadError: syntax: "..." expression outside call
Stacktrace:
 [1] top-level scope at /home/adude/.julia/packages/Keras/IoYuA/src/ops.jl:13
 [2] include at ./boot.jl:328 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1105
 [4] include at ./Base.jl:31 [inlined]
 [5] include(::String) at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:1
 [6] top-level scope at /home/adude/.julia/packages/Keras/IoYuA/src/graph.jl:2
 [7] include at ./boot.jl:328 [inlined]
 [8] include_relative(::Module, ::String) at ./loading.jl:1105
 [9] include at ./Base.jl:31 [inlined]
 [10] include(::String) at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:1
 [11] top-level scope at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:4
 [12] include at ./boot.jl:328 [inlined]
 [13] include_relative(::Module, ::String) at ./loading.jl:1105
 [14] include(::Module, ::String) at ./Base.jl:31
 [15] top-level scope at none:2
 [16] eval at ./boot.jl:330 [inlined]
 [17] eval(::Expr) at ./client.jl:425
 [18] top-level scope at ./none:3
in expression starting at /home/adude/.julia/packages/Keras/IoYuA/src/ops.jl:13
in expression starting at /home/adude/.julia/packages/Keras/IoYuA/src/graph.jl:2
in expression starting at /home/adude/.julia/packages/Keras/IoYuA/src/Keras.jl:4
ERROR: Failed to precompile Keras [031d7243-eb6a-5793-a290-e3c72503329c] to /home/adude/.julia/compiled/v1.3/Keras/RBgJ9_OVTEM.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1283
 [3] _require(::Base.PkgId) at ./loading.jl:1024
 [4] require(::Base.PkgId) at ./loading.jl:922
 [5] require(::Module, ::Symbol) at ./loading.jl:917

ERROR: UndefVarError: weight not defined

julia> using Kerasjulia> model, weight = Keras.load("/home/phil/Downloads/keras_structure.json", "/home/phil/Downloads/keras_weights.h5")
ERROR: UndefVarError: weight not defined
Stacktrace:
[1] (::Keras.##3#4)(::Keras.new_type) at /home/phil/.julia/v0.6/Keras/src/ops.jl:9
[2] get_ops(::Array{Any,1}) at /home/phil/.julia/v0.6/Keras/src/graph.jl:38
[3] load(::String, ::String) at /home/phil/.julia/v0.6/Keras/src/graph.jl:61
I looked at ops.jl:9:

ops = Dict{Symbol, Any}()

#weight = Keras.weights()

ops[:Input] = function(a)

    return vcall(:.+, a, 0)

end

ops[:Conv] = function(a)

    activation = a.fields["activation"]

    kernel_weight = permutedims(weight[a.fields["name"]][a.fields["name"]]["kernel:0"], (4,3,2,1))

    kernel_bias = weight[a.fields["name"]][a.fields["name"]]["bias:0"]

    strides = (a.fields["strides"]...)

    pads = (0,0)

    return vcall(:Conv, Symbol(activation), kernel_weight, kernel_bias, strides, pads)

end

Maybe weight shouldn't have been commented out there on line 2?

Name of the package

Should this be called KerasLoader or something along those lines? There is this: https://github.com/invenia/Keras.jl , which IS a wrapper around Keras.