off99555 / superkeras

:rocket: A bunch of Keras utilities and paper implementations written under TensorFlow backend

superkeras

A bunch of Keras utilities and paper implementations written under TensorFlow backend

It is made to work nicely with tensorflow.keras and TensorFlow 2.0.

How to use

Download this repository and put it inside your project as a folder named superkeras.

One of the nice option is to do git submodule add https://github.com/offchan42/superkeras in order to download this repo and track it as a part of your Git repository because this repo is frequently updated.

Then simply import superkeras to use all the functionalities provided. See more info below.

General utilities/helpers/extras

• import superkeras.layers to use use functions and classes in the file. Contains some useful functions like:
  • repeat_layers for creating multiple layers with the same type
  • apply_residual_block function for building ResNet-like architecture, making the network able to learn without degradation when the depth is very deep
  • BlurPool antialiased layer to make ConvNet shift-invariant, also increase accuracy. See https://github.com/adobe/antialiased-cnns
  • Arithmetic layer for performing simple arithmetic operations on a trainable weight
  • NormalizeQuaternion layer for normalizing Quaternion data to have magnitude of 1.
  • and couple more utilities
• import superkeras.losses to use loss functions (which can also be used as metrics)
  • r2_score for computing r-squared score for regression problem.
  • mean_euclidean_distance or mean_euclidean_distance_squared for computing distance between 2 positions.
  • mean_quat_distance, mean_quat_angle, mean_quat_angle_deg, and mean_sqr_quat_angle are for computing Quaternion difference, must be used with normalized quaternion (output of NormalizeQuaternion layer).
  • dice_loss, iou_coef, and dice_coef are losses/metrics for image segmentation problems.
• import superkeras.datautil to use utilities made for tf.data module
• import superkeras.utils to use some helper functions not related to keras e.g. make_xy_3d for converting a time-series DataFrame into a 3D data for ConvNets or LSTM.
• import superkeras.imgutils to use helper functions for manipulating images e.g. cropping with zero padding. Please check Rect class which helps you a lot in bounding box related manipulation.

Paper implementations

• import superkeras.permutational_layer to use PermutationalLayer model implemented accurately following the paper Permutation-equivariant neural networks applied to dynamics prediction. This layer is for modeling problems that the order of the input objects are not important and that you can swap/permute/re-order them and the prediction should stay the same, preserving Permutation Invariance property. You can think of this as modeling a Set data structure for the inputs of neural networks.

  To use it without the need to understand too much details, you can use PermutationalModule.

  You can run the main code in the module to understand how it works intuitively.

• import superkeras.pointnet to use PointNet architecture from the PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation paper. PointNet also allows modeling of permutation invariance problems, it's computationally cheaper than PermutationalLayer for large amount of point instances.

  You can run the main code in the module to understand how it practically works.

Tips

You can run pytest to run test files that have name starting with test_.

For any functions that are not mentioned here but exist in the file, you can use them. All of the functions that are supposed to be usable usually have documentation written very good on them. So check that!

Troubleshooting

• ValueError: Unknown metric function:mean_quat_angle_deg:

  This error is caused by not providing the function to the model loader. It usually happens when you save the Keras model to disk and trying to load it using keras.models.load_model function.

  To fix this, you need to provide custom_objects dictionary with string key pointing to the function reference. Example:

  from superkeras.losses import mean_quat_angle_deg
  from keras.models import load_model
  model = load_model('model_path.h5', custom_objects=dict(mean_quat_angle_deg=mean_quat_angle_deg))

  There are many possible metric functions that this error can indicate. Most of the functions live in losses and layers module. So you must provide all of the unknown functions into custom_objects.

FAQ

How do I run the example main code in permutational_layer.py or pointnet.py?

You can run each file as a module of superkeras package by changing directory to be above superkeras and then run python -m superkeras.pointnet for example. You cannot run python pointnet.py because this kind of run is not working with relative imports.

Why is this library named superkeras?

When I was searching for a unique name for my keras helper utilities like keras helpers, keras utils, keras extras, etc, I've found that all of those names were already chosen by someone else. I don't want to repeat the name so I thought of something cool and just name superkeras for ease in remembering.

superkeras's Issues

Traing

Dear off99555:

If I want to use this code for classification, can you show me how to train your permutation model?

Thanks

Add permutational layer to LSTM

Dear @off99555

I am a student new to this problem.

I'm trying to build LSTM with input feature permutation invariant, for example, the feature is a set of position, how can I use this module to deal with LSTM?

Could you help me with this?

Best regards

LSTM and additional non-permutational-invariant features

Hi,

I have two questions:

1. is it possible to have additional features besides the permutational invariant?
2. is it possible to use LSTM-Layers since those require themselves 3D inputs?

example code crashes (incompatible shape)

Hi,

Trying to run the untouched test-permutational-layer.ipynb, I have this error:

WARNING:tensorflow:Model was constructed with shape (None, 4) for input Tensor("x1_9:0", shape=(None, 4), dtype=float32), but it was called on an input with incompatible shape (None, 1, 4).

AssertionError Traceback (most recent call last)
in
2 [[4,3,2,1]],
3 [[1,2,3,4]]]
----> 4 predictions = perm_layer.predict(x)

Thanks for any help

Damien

A little help...

Dear @off99555
I am interested in applying the permutational layer to my model that consists of:
30 individual features for element A, 30 individual features for elment B and 9 common characteristics.

I would like to train my model using permutational_layer.py

Could you guide me using you project?

Regards

Training a permutation invariant model to learn set partitions

@off99555 first of all, thank you for the implementation of the permutational module, it´s very cool.

I'm trying to find a way to build a model that could learn to find partitions within a set of data points in a supervised way. The criteria of how to group the elements has to be learned from the training data that I could represent as a boolean adjacency matrix, for example (where A[i, j] = 1 means element i and element j are in the same group).

So far, the best way I found to model the problem was treating the set as a sequence, feeding it to a Bidirectional RNN (many to many) and then predicting each bit of the adjacency matrix as a binary classification problem (using a Dense layer with sigmoid activation). But there isn't really a good way of ordering the points (similar to the point cloud problem), so I don't really like this approach.
The number of points in the set and the number of groups they could form are variable, and there could even be outliers that don't belong to any group. I'm keeping the input and output size constant by using 0 padding.

I'm wondering if this module would make more sense for this task... in that case I guess I would need to call the module like this:

perm = PermutationalModule(
(features,),
n_objects,
[repeat_layers(Dense, [n_cells1, n_cells2], activation="relu"), repeat_layers(Dense, [n_objects], activation="sigmoid")]
)

And each output should be compared against the row of the adjacency matrix of it's input using binary crossentropy...
does it make sense? I'd appreciate any help.

Thank you!