multifeature_resnet50.py

DiVA: Including auxiliary and self-supervised features in Deep Metric Learning

What can I find here?

This repository contains implementations for

DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning

accepted to ECCV 2020.

Link: https://arxiv.org/abs/2004.13458

The majority of this codebase is built upon research and implementations provided in Paper: https://arxiv.org/abs/2002.08473 Repo: https://github.com/Confusezius/Revisiting_Deep_Metric_Learning_PyTorch

This repository contains the DiVA extension to standard Deep Metric Learning by extending auxiliary and self-supervised features for improved generalization to unseen test classes:

Contact: Karsten Roth, [email protected]

Suggestions are always welcome!

Architecturally, running DiVA extends DML like this:

Some Notes:

If you use this code in your research, please cite

@misc{milbich2020diva,
    title={DiVA: Diverse Visual Feature Aggregation for Deep Metric Learning},
    author={Timo Milbich and Karsten Roth and Homanga Bharadhwaj and Samarth Sinha and Yoshua Bengio and Björn Ommer and Joseph Paul Cohen},
    year={2020},
    eprint={2004.13458},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

Requirements & Datasets:

PyTorch 1.2.0+ & Faiss-Gpu
Python 3.6+
pretrainedmodels, torchvision 0.3.0+

for a detailed installation script and download links to all benchmarks, please visit https://github.com/Confusezius/Revisiting_Deep_Metric_Learning_PyTorch

Training:

Update 14/09/21: There was a slight error in the uploaded fast_moco.py script resulting in negative reweighting different to that mentioned in the paper. As such, we have introduced the --diva_fixed-flag which can be used to run the corrected fast momentum contrast/nce objective. Performance however remains effectively the same, so it is not necessarily needed. Thanks to BrandonHanx for noticing!

Training is done by using diva_main.py, with settings available in parameters.py. Exemplary runs are provided in SampleRuns/ECCV2020_DiVA_SampleRuns.sh. Logging via Weights and Biases follows the same setup as in https://github.com/Confusezius/Revisiting_Deep_Metric_Learning_PyTorch.

Generally, a run can look like this:

python diva_main.py --dataset cub200 --log_online --project DiVA_SampleRuns --group CUB_DiVA-R50-512
                    --diva_ssl fast_moco --source_path $datapath --tau 55 --gamma 0.2
                    --diva_alpha_ssl 0.3 --diva_alpha_shared 0.3 --diva_alpha_intra 0.3
                    --diva_rho_decorrelation 1500 1500 1500 --diva_features discriminative selfsimilarity shared intra
                    --diva_sharing random --evaltypes all --diva_moco_temperature 0.01  --diva_moco_n_key_batches 30
                    --n_epochs 350 --seed 0 --gpu 0 --samples_per_class 2
                    --loss margin --batch_mining distance --arch resnet50_frozen_normalize
                    --embed_dim 128 (--diva_fixed)

We apply DiVA on CUB (--dataset cub200) and Margin Loss (--loss margin) with distance-based batchmining (--batch_mining distance).

We use the standard discriminative features, as well as self-supervised features selfsimilarity based on "fast Momentum Contrast" (--diva_ssl fast_moco), corresponding to the DaNCE objective in the original paper, as well as shared and intra_class features (intra), all of which we note in --diva_features discriminative selfsimilarity shared intra.

Each auxiliary loss is weighted by 0.3 (see --diva_alpha_ssl, --diva_alpha_shared, --diva_alpha_intra). Each decorrelation (discriminative to ...) is weighted by 1500 (see --diva_rho_decorrelation). Finally, some self-supervision/MoCo specific hyperparameters (--diva_moco_temperature, --diva_moco_n_key_batches), corresponding to the temperature and number of key batches used in the NCE objective.

Finally, each embedding space has dimensionality --embed_dim 128, meaning that during testing, we evaluate on a ResNet50 with a total dimensionality of 4 x 128 = 512.

We also note that other self-supervision methods can be included as well (see criteria/__init__.py and parameters.py), e.g. Deep Clustering:

python diva_main.py --source_path $datapath --log_online --project DiVA_Experiments --diva_rho_decorrelation 500
                    --diva_features discriminative dc --evaltypes all --diva_dc_update_f 2 --diva_dc_ncluster 300
                    --n_epochs 200 --seed 0 --gpu $gpu --bs 104 --samples_per_class 2 --loss margin
                    --batch_mining distance --arch bninception_normalize --embed_dim 256

	#Find negatives that violate triplet constraint semi-negatives
	neg_mask = np.logical_and(neg,d<d[np.where(pos)[0]].max())
	#Find positives that violate triplet constraint semi-hardly
	pos_mask = np.logical_and(pos,d>d[np.where(neg)[0]].min())

	for i in range(bs):
	neg = labels!=labels[i]; pos = labels==labels[i]

	if np.sum(pos)>1:
	anchors.append(i)
	q_d_inv = self.inverse_sphere_distances(batch, distances[i], labels, labels[i])
	#Sample positives randomly
	pos[i] = 0
	positives.append(np.random.choice(np.where(pos)[0]))
	#Sample negatives by distance
	negatives.append(np.random.choice(bs,p=q_d_inv))

	distance_weighting = ((2.0 - float(dim)) * torch.log(distances) - (float(dim-3) / 2) * torch.log(1.0 - 0.25 * (distances.pow(2))))
	distance_weighting = torch.exp(distance_weighting - torch.max(distance_weighting))
	distance_weighting[distances>self.upper_cutoff] = 0
	distance_weighting = distance_weighting.clamp(min=1e-45)
	distance_weighting = distance_weighting/torch.sum(distance_weighting, dim=0)

	for layerblock in self.layer_blocks:
	x = layerblock(x)

confusezius / eccv2020_diva_multifeature_dml Goto Github PK

eccv2020_diva_multifeature_dml's Introduction

DiVA: Including auxiliary and self-supervised features in Deep Metric Learning

What can I find here?

Some Notes:

Requirements & Datasets:

Training:

eccv2020_diva_multifeature_dml's People

Contributors

Stargazers

Watchers

Forkers

eccv2020_diva_multifeature_dml's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs