liuem607 / expvae

Visually Explainable VAE

ExpVAE

Overview

This repository provides training and testing code and data for CVPR 2020 paper:

"Towards Visually Explaining Variational Autoencoders", Wenqian Liu, Runze Li, Meng Zheng, Srikrishna Karanam, Ziyan Wu, Bir Bhanu, Richard J. Radke, and Octavia Camps

Further information please contact Wenqian Liu at [email protected], Runze Li at [email protected] .

Requirements

• PyTorch 1.0
• Python 3.7
• Cuda 9.0

Running codes

• Start trainning a one-class VAE with default settings by call:

python train_expVAE.py 

other options are:

[-h] [--result_dir DIR] [--ckpt_dir DIR] [--batch_size N] [--epochs N] 
     [--seed S] [--resume PATH] [--latent_size N] [--one_class N]

Here result_dir saves validating visualizations; ckpt_dir saves best model after each epoch; latent_size decides VAE's latent vector shape; one_class decides which digit to be used as inlier to train on.

• Start testing with default settings by call:

python test_expVAE.py

other options are:

[-h] [--result_dir DIR] [--batch_size N] [--seed S]
     [--latent_size N] [--model_path DIR] [--one_class N]

Here one_class decides the outlier digit, so that expVAE will generate anomaly localization attention maps, in which high response regions denote the difference between the outlier digit and the inlier digit.

Pretrained models for MNIST can be found here

License and Citation

The use of this software is RESTRICTED to non-commercial research and educational purposes.

@article{liu2019towards,
  title={Towards Visually Explaining Variational Autoencoders},
  author={Liu, Wenqian and Li, Runze and Zheng, Meng and Karanam, Srikrishna and Wu, Ziyan and Bhanu, Bir and Radke, Richard J and Camps, Octavia},
  journal={arXiv preprint arXiv:1911.07389},
  year={2019}
}

expvae's Issues

test can't run

I run:
python test_expVAE.py
And get error as follows:

cuda available
Traceback (most recent call last):
  File "test_expVAE.py", line 99, in <module>
    main()
  File "test_expVAE.py", line 77, in main
    gcam.backward(mu, logvar, mu_avg, logvar_avg)
  File "/mnt/data/home/x/expVAE/code/gradcam.py", line 60, in backward
    self.score_fc.backward(gradient=one_hot, retain_graph=True)
  File "/mnt/data/home/x/anaconda3/envs/py38pytorch151/lib/python3.8/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/mnt/data/home/x/anaconda3/envs/py38pytorch151/lib/python3.8/site-packages/torch/autograd/__init__.py", line 93, in backward
    grad_tensors = _make_grads(tensors, grad_tensors)
  File "/mnt/data/home/x/anaconda3/envs/py38pytorch151/lib/python3.8/site-packages/torch/autograd/__init__.py", line 25, in _make_grads
    raise RuntimeError("Mismatch in shape: grad_output["
RuntimeError: Mismatch in shape: grad_output[0] has a shape of torch.Size([128, 32]) and output[0] has a shape of torch.Size([]).

A question about 'encode_one_hot_batch' function on gradcam.py

I'm the one who wanted to use this model with different datasets.
However, I'm having trouble getting an anomaly attention map, so I want to ask for advice.

I have a question about the function in gradcam.py python file.
Like below, the function 'encode_one_hot_bath' just return mu, not using one_hot_batch.
Is this configured as intended? or not developed yet?

set the target class as one others as zero. use this vector for back prop added by Lezi
def encode_one_hot_batch(self, z, mu, logvar, mu_avg, logvar_avg):
--one_hot_batch = torch.FloatTensor(z.size()).zero_()
--return mu

Plus, if this function is implemented as intended, I want to ask which part of the code conducted the (4) equation of the paper which generating anomaly attention.

Thanks,

I have some question.

Hello sir.
I have interesting about XAI and VAE.
so i am happy to find your site.

I tested your code.
But i found strange results that i couldn't understand.

When inference your code,
If digit 3, the red area should not appear, i think.
But the red area did appear.
How do you think about it?

If you agree my opinion, how can modify your code?
Thank you.

MVTec code can not be viewed.

MVTec code can not be viewed.