cloud-cv / grad-cam

:rainbow: :camera: Gradient-weighted Class Activation Mapping (Grad-CAM) Demo

Grad-CAM: Gradient-weighted Class Activation Mapping

Grad-CAM uses the class-specific gradient information flowing into the final convolutional layer of a CNN to produce a coarse localization map of the important regions in the image. It is a novel technique for making CNN more 'transparent' by producing visual explanations i.e visualizations showing what evidence in the image supports a prediction. You can play with Grad-CAM demonstrations at the following links:

Arxiv Paper Link: https://arxiv.org/abs/1610.02391

Grad-CAM VQA Demo: http://gradcam.cloudcv.org/vqa

Grad-CAM Classification Demo: http://gradcam.cloudcv.org/classification

Grad-CAM Captioning Demo: http://gradcam.cloudcv.org/captioning

Installing / Getting started

We use RabbitMQ to queue the submitted jobs. Also, we use Redis as backend for realtime communication using websockets.

All the instructions for setting Grad-CAM from scratch can be found here

Note: For best results, its recommended to run the Grad-CAM demo on GPU enabled machines.

Interested in Contributing?

Cloud-CV always welcomes new contributors to learn the new cutting edge technologies. If you'd like to contribute, please fork the repository and use a feature branch. Pull requests are warmly welcome.

if you have more questions about the project, then you can talk to us on our Gitter Channel.

Acknowledgements

• VQA_LSTM_CNN
• HieCoAttenVQA
• NeuralTalk2
• PyTorch

grad-cam's Issues

Image not loading if there is any special character in caption in the captioning demo

If I enter any special character in the caption input field, then the image generated is not rendered. need to encode the the image url at the backend before rendering it at the frontend.

For reference, see this http://stackoverflow.com/questions/5607551/how-to-urlencode-a-querystring-in-python

Links broken in Navigation bar dropdown

The links in the navigation bar are broken and needs to be prepended by http://cloudcv.org/

Add link to arxiv paper in the main page

We want to add a link to arxiv paper https://arxiv.org/abs/1610.02391 in the main page of Grad-CAM project.

Decouple django-channels with RabbitMQ worker

Setup continuous integration

Put all terminal messages in constants file

After choosing new image it showing previous predicted answer

Write unittests

For running tests, we would like to use python-coverage. For more details, see this https://pypi.python.org/pypi/coverage

Add capability to upload images using URLs

For now, one can upload images by choosing one image from their computer. We want the user to upload the image using an image URL.

Change directory to save results

Currently, the resultant images are stored in the demo images folder and hence causing the problem of showing the result images as demo images.

Setup Docker containers for easy deployment

Setup Postgres for storing the job related information

Classification demo not working

Hi,

The classification demo (http://gradcam.cloudcv.org/classification) doesn't seem to work for me, where the results don't show up. Could you please take a look?

Thanks!

Update README