Basenji for keras about basenji

[davek44]

Hey Jacob, I've been moving everything to the TF 2.0 compatible tf.keras layers and eager mode in the tf2 branch of the repo. I recently converted the latest models to tf.keras here: https://github.com/calico/basenji/tree/tf2/manuscripts/biorxiv2019. I'll work on converting the previous one, too.

[jmschrei]

Hi David

I got the human model using wget https://storage.googleapis.com/basenji_barnyard/model_human.h5 from https://github.com/calico/basenji/blob/tf2/manuscripts/biorxiv2019/get_models.sh. However, when I try to load it using

from tensorflow.keras.models import load_model
load_model("model_human.h5")

I get the following error:

Traceback (most recent call last):
  File "load_bassenji.py", line 4, in <module>
    load_model("model_human.h5")
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/save.py", line 146, in load_model
    return hdf5_format.load_model_from_hdf5(filepath, custom_objects, compile)
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/hdf5_format.py", line 168, in load_model_from_hdf5
    custom_objects=custom_objects)
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/saving/model_config.py", line 55, in model_from_config
    return deserialize(config, custom_objects=custom_objects)
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/layers/serialization.py", line 102, in deserialize
    printable_module_name='layer')
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/utils/generic_utils.py", line 191, in deserialize_keras_object
    list(custom_objects.items())))
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py", line 906, in from_config
    config, custom_objects)
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py", line 1842, in reconstruct_from_config
    process_layer(layer_data)
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/network.py", line 1824, in process_layer
    layer = deserialize_layer(layer_data, custom_objects=custom_objects)
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/layers/serialization.py", line 102, in deserialize
    printable_module_name='layer')
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/utils/generic_utils.py", line 180, in deserialize_keras_object
    config, module_objects, custom_objects, printable_module_name)
  File "/net/noble/vol1/home/jmschr/anaconda/lib/python3.7/site-packages/tensorflow_core/python/keras/utils/generic_utils.py", line 165, in class_and_config_for_serialized_keras_object
    raise ValueError('Unknown ' + printable_module_name + ': ' + class_name)
ValueError: Unknown layer: StochasticShift

Do you know why I get this? It doesn't appear to be a layer in TF or in this repo.

Thanks!

[davek44]

Hmm I've never tried using load_model. I've been using load_weights, e.g. https://github.com/calico/basenji/blob/tf2/basenji/seqnn.py#L263. I'll have to learn more about this.

The layer that you're looking for is here: https://github.com/calico/basenji/blob/tf2/basenji/layers.py#L176

[jmschrei]

Thanks for the reply. After fussing with Tensorflow and cudnn for a little bit I was able to get it working. Apparently things work best when you don't use versions that are several years out of date.