ardiya / siamesenetwork-tensorflow

Using siamese network to do dimensionality reduction and similar image retrieval

Siamese Network Tensorflow

Siamese network is a neural network that contain two or more identical subnetwork. The objective of this network is to find the similarity or comparing the relationship between two comparable things. Unlike classification task that uses cross entropy as the loss function, siamese network usually uses contrastive loss or triplet loss.

Siamese network has a lot of function, this repository is trying to use Siamese network to do a dimensionality reduction and image retrieval.

This project follows Hadsell-et-al.'06 [1] by computing the Euclidean distance on the output of the shared network and by optimizing the contrastive loss (see paper for more details). The contastive loss is defined as follows

The is the distance of between the output of the network with the input and the input .

The similarity function is defined as . This function will be activated when the Label equal to 1 and deactivated when is equal to 0. The goal of this function is to minimize the distance of the pairs.

The dissimilarity function is defined as . This function will be activated when the Label is equal to 0 and deactivated when is equal to 1. The goal of this function is to give a penalty of the pairs when the distance is lower than margin .

[1] "Dimensionality Reduction by Learning an Invariant Mapping" http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf

Model

The input of these will be image_left, image_right and . Our model uses 5 layer of convolutional layer and pooling followed. We do not use fully convolutonal net because convolution operation is faster on GPU(especially using CUDNN). See http://cs231n.github.io/convolutional-networks/#convert for more information on converting FC layer to Conv layer.

Run

Train the model

git clone https://github.com/ardiya/siamesenetwork-tensorflow
python train.py

Tensorboard Visualization(After training)

tensorboard --logdir=train.log

Updates

• Update the API to 1.0
• Cleanup the old code

Dimensionality reduction

The images below shows the final Result on MNIST test dataset. By only using 2 features, we can easily separate the input images.

The gif below shows some animation until it somehow converges.

Image retrieval

Image retrieval uses the trained model to extract the features and get the most similar image using cosine similarity. See here

Retrieving similar test image from trainset

• Select id 865 in test image

• Retrieved top n similar image from train data with ids of [53144 47864 11074 51561 41350 34215 48182] from train data

siamesenetwork-tensorflow's Issues

Using another dataset

I want to train this model for a set of objects, and not MNIST. Any suggestions on how I could do that ?

# IOError: [Errno 2] No such file or directory: u'img/50.jpg

IOError: [Errno 2] No such file or directory: u'img/50.jpg
when i trying to run the train.py , the erro occur ... so, how can i fix it

Why an extra axis is added in dataset.py for class MNISTDataset

Hi ,
i have a question regarding the dataset,py. I am trying to create my own dataset to test the model and in the dataset.py in

class MNISTDataset(Dataset):
    def __init__(self):
	print("===Loading MNIST Dataset===")
	(self.images_train, self.labels_train), (self.images_test, self.labels_test) = mnist.load_data()
	self.images_train = np.expand_dims(self.images_train, axis=3) / 255.0
	self.images_test = np.expand_dims(self.images_test, axis=3) / 255.0

Can anyone explain why we are adding a new axis? I am trying to create a dataset witg RGB channels. And i dont know if the MNIST dataset is in grayscale?

I would like to ask if the final output matching degree Y is 0 and 1, or before 0-1, and if the labels of the two pictures I made can be 0.9, similar to pencils and pens.

I would like to ask if the final output matching degree Y is 0 and 1, or before 0-1, and if the labels of the two pictures I made can be 0.9, similar to pencils and pens.

Need suggestions

I have written the code for a similar problem involving RGB images on a broader dataset but the results are not that good.
The repo link is given below:

https://github.com/ArkaJU/Sketch-Retrieval---Siamese

Any help will be gratefully appreciated.

inference

hello, how to test the trained model??

Running on non-MNIST greyscale images

Hi ardiya!

Thanks for putting together this repository. Could you tell me what would need to be done to run this on non-MNIST greyscale images?

Thanks so much in advance!
Lilly

how to choose the margin parameter?

In your code the margin is 0.2 ,but in other code,I see margin is much bigger than yours（e.g 16）

Error use_nesterov

I got this error while trying to train.

File "train.py", line 40, in
train_step = tf.train.MomentumOptimizer(0.01, 0.99, use_nesterov=True).minimize(loss, global_step=global_step)
TypeError: init() got an unexpected keyword argument 'use_nesterov'

Creating batches

I noticed that you pass FLAGS.batch_size to the gen.next_batch function but in the next_batch function, you don't actually use the batch_size variable at all. Can you please explain how you make them into batches?