shamangary / loupe_keras

Rewrite the LOUPE library (https://github.com/antoine77340/LOUPE) into Keras version. Many learnable pooling or differentiable aggregation methods are covered. (NetVLAD, NetRVLAD, SoftDBoW, NetFV, CG)

Academic Kickstart

Academic makes it easy to create a beautiful website for free using Markdown, Jupyter, or RStudio. Customize anything on your site with widgets, themes, and language packs. Check out the latest demo of what you'll get in less than 10 minutes, or view the showcase.

Academic Kickstart provides a minimal template to kickstart your new website.

• Get Started
• View the documentation
• Ask a question
• Request a feature or report a bug
• Updating? View the Update Guide and Release Notes
• Support development of Academic:
  • Donate a coffee
  • Become a backer on Patreon
  • Decorate your laptop or journal with an Academic sticker
  • Wear the T-shirt

Install

You can choose from one of the following four methods to install:

• one-click install using your web browser (recommended)
• install on your computer using Git with the Command Prompt/Terminal app
• install on your computer by downloading the ZIP files
• install on your computer with RStudio

Then personalize your new site.

Ecosystem

• Academic Admin: An admin tool to import publications from BibTeX or import assets for an offline site
• Academic Scripts: Scripts to help migrate content to new versions of Academic

License

Copyright 2017-present George Cushen.

Released under the MIT license.

loupe_keras's Issues

Issue when using Keras NetVLAD with functional api

Hi,
First of all, thanks for this repository.
I'm trying to stack your NetVLAD implementation in keras on the top of a ResNet50, but i'm having some issue doing it.

That's my code:

# instantiating the resnet50
resnet = ResNet50(weights='imagenet', include_top=False, pooling=False, 
input_shape=input_shape)
# deleting the default Input layer
resnet.layers.pop(0)
input_q = Input(shape=(224, 224, 3))
# stacking the resnet on a new input (seems useless but i need this to having multiple inputs)
resnet_q = resnet(input_q)
# permuting the tensor from resnet last layer shape which is (None, 7,7,2048) to (None, 2048, 7,7)
transpose = Permute((3,1,2), input_shape=(7,7,2048))(resnet_p)
# reshaping from (2048, 7,7) to (2048, 7*7), in the form (max_samples, feature_size) like requested from NetVLAD
reshape = Reshape((2048, 7*7))(transpose)
# stacking netvlad
netvlad = NetVLAD(feature_size=7*7, max_samples=2048, cluster_size=64, output_dim=1024)(reshape)

But i'm getting this error:

 ---------------------------------------------------------------------------
 
 ValueError                                Traceback (most recent call last)
 
 <ipython-input-20-7cb139e75b78> in <module>
      19 transpose = Permute((3,1,2), input_shape=(7,7,2048))(resnet_p)
      20 reshape = Reshape((2048, 7*7))(transpose)
 ---> 21 netvlad = NetVLAD(feature_size=7*7, max_samples=2048, cluster_size=64, output_dim=1024)(reshape)
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in symbolic_fn_wrapper(*args, **kwargs)
      73         if _SYMBOLIC_SCOPE.value:
      74             with get_graph().as_default():
 ---> 75                 return func(*args, **kwargs)
      76         else:
      77             return func(*args, **kwargs)
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)
     461                                          'You can build it manually via: '
     462                                          '`layer.build(batch_input_shape)`')
 --> 463                 self.build(unpack_singleton(input_shapes))
     464                 self.built = True
     465 
 
 ~/Documenti/netvlad/loupe_keras.py in build(self, input_shape)
      81         self.cluster_weights = self.add_weight(name='kernel_W1',
      82                                       shape=(self.feature_size, self.cluster_size),
 ---> 83                                       initializer=tf.random_normal_initializer(stddev=1 / math.sqrt(self.feature_size)),
      84                                       trainable=True)
      85         self.cluster_biases = self.add_weight(name='kernel_B1',
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/keras/engine/base_layer.py in add_weight(self, name, shape, dtype, initializer, regularizer, trainable, constraint)
     280                             dtype=dtype,
     281                             name=name,
 --> 282                             constraint=constraint)
     283         if regularizer is not None:
     284             with K.name_scope('weight_regularizer'):
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py in variable(value, dtype, name, constraint)
     618     """
     619     v = tf_keras_backend.variable(
 --> 620         value, dtype=dtype, name=name, constraint=constraint)
     621     if hasattr(value, 'tocoo'):
     622         v._keras_shape = value.tocoo().shape
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/tensorflow_core/python/keras/backend.py in variable(value, dtype, name, constraint)
     809       dtype=dtypes_module.as_dtype(dtype),
     810       name=name,
 --> 811       constraint=constraint)
     812   if isinstance(value, np.ndarray):
     813     v._keras_shape = value.shape
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/tensorflow_core/python/ops/variables.py in __call__(cls, *args, **kwargs)
     258       return cls._variable_v1_call(*args, **kwargs)
     259     elif cls is Variable:
 --> 260       return cls._variable_v2_call(*args, **kwargs)
     261     else:
     262       return super(VariableMetaclass, cls).__call__(*args, **kwargs)
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/tensorflow_core/python/ops/variables.py in _variable_v2_call(cls, initial_value, trainable, validate_shape, caching_device, name, variable_def, dtype, import_scope, constraint, synchronization, aggregation, shape)
     252         synchronization=synchronization,
     253         aggregation=aggregation,
 --> 254         shape=shape)
     255 
     256   def __call__(cls, *args, **kwargs):
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/tensorflow_core/python/ops/variables.py in <lambda>(**kws)
     233                         shape=None):
     234     """Call on Variable class. Useful to force the signature."""
 --> 235     previous_getter = lambda **kws: default_variable_creator_v2(None, **kws)
     236     for _, getter in ops.get_default_graph()._variable_creator_stack:  # pylint: disable=protected-access
     237       previous_getter = _make_getter(getter, previous_getter)
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/tensorflow_core/python/ops/variable_scope.py in default_variable_creator_v2(next_creator, **kwargs)
    2554       synchronization=synchronization,
    2555       aggregation=aggregation,
 -> 2556       shape=shape)
    2557 
    2558 
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/tensorflow_core/python/ops/variables.py in __call__(cls, *args, **kwargs)
     260       return cls._variable_v2_call(*args, **kwargs)
     261     else:
 --> 262       return super(VariableMetaclass, cls).__call__(*args, **kwargs)
     263 
     264 
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-ackages/tensorflow_core/python/ops/resource_variable_ops.py in __init__(self, initial_value, trainable, collections, validate_shape, caching_device, name, dtype, variable_def, import_scope, constraint, distribute_strategy, synchronization, aggregation, shape)
    1404           aggregation=aggregation,
    1405           shape=shape,
 -> 1406           distribute_strategy=distribute_strategy)
    1407 
    1408   def _init_from_args(self,
 
 ~/Documenti/netvlad/venv/lib/python3.6/site-packages/tensorflow_core/python/ops/resource_variable_ops.py in _init_from_args(self, initial_value, trainable, collections, caching_device, name, dtype, constraint, synchronization, aggregation, distribute_strategy, shape)
    1487     if isinstance(initial_value, ops.Tensor) and hasattr(
    1488         initial_value, "graph") and initial_value.graph.building_function:
 -> 1489       raise ValueError("Tensor-typed variable initializers must either be "
    1490                        "wrapped in an init_scope or callable "
    1491                        "(e.g., `tf.Variable(lambda : "
 
 ValueError: Tensor-typed variable initializers must either be wrapped in an init_scope or callable (e.g., `tf.Variable(lambda : tf.truncated_normal([10, 40]))`) when building functions. Please file a feature request if this restriction inconveniences you.

Do you have any idea what could be the issue here? I really can't figure out, i looked at your code and i did some variation trying to spot the bug but nothing changed.
Thank you!

Why NetFV class has NetVLAD annotation

class NetFV(layers.Layer):
"""Creates a NetVLAD class.
"""
Is it NetFV code or NetVLAD code?

如何安装

请问该如何安装loupe_keras