Error in Class Activation map for hidden layers about keras-vis

[wendywangwwt]

I got similar constant heatmap (but mine are covered by a blue layer). #27 mentioned this problem as well and I think it's not fully fixed?

[raghakot]

Can you guys try this again in latest? There were a couple of big changes with the latest commit.

[wendywangwwt]

@raghakot I'm using the latest version on PyPI (0.3.1).

from keras.preprocessing.image import ImageDataGenerator
from keras.models import load_model

batch_size = 2
test_datagen = ImageDataGenerator(rescale=1./255)
validation_generator = test_datagen.flow_from_directory(
        'validation',
        target_size=(200, 200),
        batch_size=batch_size,
        class_mode='binary',
        shuffle=False)

model = load_model('weights.hdf5')
res_cnn = 1- best.predict_generator(validation_generator,steps=1)

res_cnn_class = [0 if x < 0.5 else 1 for x in res_cnn[:,0]]

import numpy as np
import matplotlib
import matplotlib.pyplot as plt

from vis.utils import utils
from vis.visualization import visualize_cam

model = model

fig = plt.figure(figsize=(20,20))
i = -1
for fn in validation_generator.filenames:
    i += 1
    subplot = fig.add_subplot(1,12,i+1)
    
    if i < 1:
            subplot.set_title('Class A')
    else:
        subplot.set_title('Class B')
            
    seed_img = utils.load_img('validation/'+fn, target_size=(200,200))

    heatmap = visualize_cam(best, 7, [res_cnn_class[i]], seed_img)
    plt.imshow(heatmap, cmap=plt.get_cmap('gray'))
    plt.axis('off')

plt.show()

Attached zip file contains model weights, python notebook of the code, and test images to reproduce the problem.
debug.zip

[gprabaha]

@raghakot I reinstalled keras-vis from source, and the new error that is now showing up is,

Traceback (most recent call last):
  File "keras-vis-cam.py", line 38, in <module>
    heatmap = visualize_cam(model, layer_idx, [pred_class], seed_img)
  File "/home/yx96/anaconda2/lib/python2.7/site-packages/keras_vis-0.3.1-py2.7.egg/vis/visualization.py", line 371, in v
isualize_cam
    penultimate_output = penultimate_layer.output
AttributeError: 'numpy.ndarray' object has no attribute 'output'

It seems to be failing to get the output from the penultimate layer! This error is showing, of course, for all acceptable layer_name, like, 'predictions', or 'block2_conv2'.

[raghakot]

Hi, The API has changed. There is now visualize_class_cam and visualize_regression_cam with visualize_cam being more general that works on custom loss functions. You might want to use visualize_class_cam now. API docs should be updated. I am still working on completely overhauling the docs and examples.

[wendywangwwt]

@raghakot Thanks for the update. After updating the package to 0.3.2 I'm able to import the new function visualize_class_cam. However now the output are pure blue squares (I just simply replaced visualize_cam with visualize_class_cam) :

[raghakot]

It is very likely the case that the penultimate_layer has a spatial resolution of (1, 1). You might have to specify a pre Conv layer manually via penultimate_layer_idx. I added examples for MNIST in examples/. Highly recommend checking that out to understand grad-CAM.

Is your code the same as this?

[wendywangwwt]

@raghakot The MNIST example runs well on my machine but my model still gets these blue squares even after I set the penultimate layer idx (now the idx corresponds to a 48x48x64 maxpooling layer) for CAM. visualize_class_saliency also gives me these blue squares. My input size is 200x200 and the output size of last maxpooling layer is 48x48x64.
The code for both CAM and saliency map I used is included in a notebook in this compressed file, along with two test images and the model weights.
debug.zip

[raghakot]

@wanting0wang It appears that the gradients are all 0's with respect to the input. This could mean that your model is overfitting or not trained properly since none of the input pixels have an effect on the output. How was this trained? It also seems too shallow. You probably need more conv layers and reduce spatial resolution down to (7, 7) or (5, 5) before applying Dense layer.

Also, this is unrelated to MNIST dataset, so you should create a separate issue for it and avoid cluttering this thread. I will help with debugging on the new thread.

[raghakot]

I am closing this due to lack of correspondence. Feel free to reopen the issue if you're still encountering problems.

[wendywangwwt]

@raghakot Hi sorry for the late reply.. I was on holiday. I think my problem is because of the high spatial resolution (and that increases the parameters a lot and leads to overfit). I trained another model with lower spatial resolution in the last maxpooling layer (about 9x9) and everything looks great. Thank you for the suggestions!

[raghakot]

Great. Glad it worked out :)