GithubHelp home page GithubHelp logo

Comments (29)

JesperChristensen89 avatar JesperChristensen89 commented on July 26, 2024 16

Try building your model without using a sequential model.
E.g. for VGG-16 do this;

# get base model
base_model = applications.VGG16(weights='imagenet', include_top=False,input_shape = (img_width, img_height, 3))

# build top model
x = Flatten(name='flatten')(base_model.output)
x = Dense(4096, activation='relu', name='fc1')(x)
x = Dense(4096, activation='relu', name='fc2')(x)
x = Dense(3, activation='softmax', name='predictions')(x)

# stitch together
model = Model(inputs= base_model.input, outputs=x)

# inspect
model.summary()

This builds the model as one unified network rather than two cascading ones.

from keras-vis.

itsnamgyu avatar itsnamgyu commented on July 26, 2024 13

@yuval-harpaz 's hack seems to work for me. I'm using a cascaded model with a Sequential model on top of base MobileNetV2 (alpha=0.25).

I had a similar error for a network with two nodes of output, dense_1_1/Relu:0 and sequential_2/dense_1/Relu:0 . The solution for me was to go to losses.py and change layer_output = self.layer.output to layer_output = self.layer.get_output_at(-1).

For those of you who want to try out this method, I'll reformat the code:

Line 76 of losses.py

    #layer_output = self.layer.output
    layer_output = self.layer.get_output_at(-1)

from keras-vis.

Rubenkl avatar Rubenkl commented on July 26, 2024 5

Resnet5 same problem..

from keras-vis.

yuval-harpaz avatar yuval-harpaz commented on July 26, 2024 2

I had a similar error for a network with two nodes of output, dense_1_1/Relu:0 and sequential_2/dense_1/Relu:0 . The solution for me was to go to losses.py and change layer_output = self.layer.output to layer_output = self.layer.get_output_at(-1).

from keras-vis.

MathiasKahlen avatar MathiasKahlen commented on July 26, 2024 1

Any updates on this? I don't understand the answer from @8enmann - how do I implement that? I am trying to do the same with visualize_cam and visualize_saliency but get the multiple inbound nodes error. Hope you can help.

from keras-vis.

raghakot avatar raghakot commented on July 26, 2024

Can you post a gist of your code? It looks as though the output has two inbound inputs. I have to make some changes to support multi input/outptut. Having a gist would help me verify as well.

from keras-vis.

dgorissen avatar dgorissen commented on July 26, 2024

Thanks for the response. I will try refactor/clean into a self contained example but fundamentally Im not doing anything complicated. Essentially this, adding a new Sequential Model on top of a pre-trained base: https://gist.github.com/fchollet/7eb39b44eb9e16e59632d25fb3119975

from keras-vis.

dgorissen avatar dgorissen commented on July 26, 2024

Ok, here is is an example. Im guessing you would need to add support for visualising models which have been stacked in this way by recursing.

from __future__ import division, print_function
from os import path
import numpy as np
from vis.utils import utils
from vis.visualization import visualize_activation, get_num_filters
from keras.models import Model, Sequential
from keras.layers import Dropout, Flatten, Dense, Input
from keras import applications
from PIL import Image

# Base model
input_tensor = Input(shape=(100, 100, 3))
base_model = applications.VGG16(weights='imagenet',
                                include_top=False,
                                input_tensor=input_tensor)

# New top layer(s)
input_shape = base_model.output_shape[1:]
top_model = Sequential()
top_model.add(Flatten(input_shape=input_shape))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(5, activation='softmax', name='predictions'))

# Combine base and top
model = Model(inputs=[base_model.input],
              outputs=[top_model(base_model.output)])

# Print summary
model.summary()

##### This works
layer_name = 'block5_conv1'

##### How to visualise the top softmax?
# This layer exists but needs to be unpacked so it fails
#layer_name = "sequential_1"

# Ideally you one can do this:
#layer_name = "sequential_1/predictions/Softmax"

layer, layer_idx = [(l, idx) for idx, l in enumerate(model.layers) if l.name == layer_name][0]

tot_filters = get_num_filters(layer)

filters = np.arange(tot_filters)
vis_images = []
for idx in filters[:2]:
    img = visualize_activation(model, layer_idx, filter_indices=[idx])
    img = utils.draw_text(img, str(idx))
    vis_images.append(img)

fn = "filters_%s.png" % layer_name
stitched = utils.stitch_images(vis_images, cols=8)
im = Image.fromarray(stitched)
im.save(fn)

I thought I could work around this by flattening the layers:


def flatten_model(model):
    layers_flat = []
    for layer in model.layers:
        try:
            layers_flat.extend(layer.layers)
        except AttributeError:
            layers_flat.append(layer)

    model_flat = Sequential(layers_flat)
    return model_flat

But this, unfortunately, also gives errors.

from keras-vis.

dgorissen avatar dgorissen commented on July 26, 2024

Any suggested way forward on this?

from keras-vis.

raghakot avatar raghakot commented on July 26, 2024

Sorry, I haven't looked at this yet. Will get to it this week,

from keras-vis.

dgorissen avatar dgorissen commented on July 26, 2024

No worries at all. I started editing to create a patch but didnt find enough time to complete properly.

from keras-vis.

8enmann avatar 8enmann commented on July 26, 2024

I'm seeing the same issue with a sequential model

from keras-vis.

8enmann avatar 8enmann commented on July 26, 2024

This let the code run:

def visualize_activation(model, layer_idx, filter_indices=None,
                         seed_input=None, input_range=(0, 255),
                         backprop_modifier=None, grad_modifier=None,
                         act_max_weight=1, lp_norm_weight=10, tv_weight=10,
                         **optimizer_params):
    if backprop_modifier is not None:
        modifier_fn = get(backprop_modifier)
        model = modifier_fn(model)

    if len(model.inbound_nodes) > 1:
        input_ = model.get_input_at(0)
    else:
        input_ = model_.input

    losses = [
        (ActivationMaximization(model.layers[layer_idx], filter_indices), act_max_weight),
        (LPNorm(input_), lp_norm_weight),
        (TotalVariation(input_), tv_weight)
    ]

    # Add grad_filter to optimizer_params.
    optimizer_params = utils.add_defaults_to_kwargs({
        'grad_modifier': grad_modifier
    }, **optimizer_params)

    return visualize_activation_with_losses(input_, losses, seed_input, input_range, **optimizer_params)

from keras-vis.

JesperChristensen89 avatar JesperChristensen89 commented on July 26, 2024

Any updates on this @raghakot? I'm seeing the same thing trying to visualize a MobileNet with custom top.

from keras-vis.

swapb94 avatar swapb94 commented on July 26, 2024

I also got the same error while trying to use layer.output. Even i used the MobileNet pre-trained model.

from keras-vis.

pcatattacks avatar pcatattacks commented on July 26, 2024

A hacky fix I found was going into the source code in losses.py, line 76 and changing

layer_output = self.layer.output

to

layer_output = self.layer.outputs[i]

where i is a valid index in the list self.layer.outputs. I'm trying to do something similar to what you are and having trouble too - it seems like you have to refer to the self.layer.outputs list for the specific output you want when you stack models. It's redundant since the list only has one element in it (for me at least) but hopefully this can serve as a temporary fix.

from keras-vis.

keisen avatar keisen commented on July 26, 2024

Hi @pcatattacks,
I seem, your problem different this issue.
Actually, I modified `losses.py' as you written, but could not work around.

I think, the cause of this issue is cascading model, thus the solution of this issue is that ,as @JesperChristensen89 said, it's to build the model as one unified network.

from keras-vis.

SahuH avatar SahuH commented on July 26, 2024

Tried using trick mentioned by @pcatattacks and @yuval-harpaz, but neither of them worked. I too am using cascaded model, a Sequential() model on top of base InceptionV3 model. Really would not prefer to re-build (as mentioned by @JesperChristensen89 ) and re-train the model.

from keras-vis.

martin-etchart avatar martin-etchart commented on July 26, 2024

I have the same issue. I managed to make it work by saving my model arch to json and loading it from there instead of sequentially assembling it. I suspect the json save cleans up the inbound node thing... anyhow it worked.

from keras-vis.

devjaynemorais avatar devjaynemorais commented on July 26, 2024

Could someone help me solve the following problem?

Environment: Keras==1.1.0 Theano==1.0.2 numpy==1.15.1 scipy==1.3.0

I created a fine tuning and frozen all layers except layer [2] because I want to get the activation values only from layer [2].

Network summary before freezing:

Layer (type) | Output Shape | Param # | Connected to


dense_1 (Dense) (None, 512) 2097664 dense_input_1[0][0]


dropout_1 (Dropout) (None, 512) 0 dense_1[0][0]
dense_1[0][0]


dense_2 (Dense) (None, 32) 16416 dropout_1[0][0]
dropout_1[1][0]


dropout_2 (Dropout) (None, 32) 0 dense_2[0][0]
dense_2[1][0]


dense_3 (Dense) (None, 1) 33 dropout_2[0][0]
dropout_2[1][0]


Total params: 2114113

Freezing layers:

for layer in model.layers[0:]:
------ layer.trainable = False
model.layers[2].trainable = True

Network summary after freezing:

`Layer (type) | Output Shape | Param # | Connected to


dense_1 (Dense) (None, 512) 0 dense_input_1[0][0]


dropout_1 (Dropout) (None, 512) 0 dense_1[0][0]


dense_2 (Dense) (None, 32) 16416 dropout_1[1][0]


dropout_2 (Dropout) (None, 32) 0 dense_2[1][0]


dense_3 (Dense) (None, 1) 0 dropout_2[1][0]


Total params: 16416`

To print layer output [2]:

OutFunc = keras.backend.function([model2.input], [model2.layers[2].get_output_at(0)])
out_val = OutFunc([inputs])[0]
print(out_val)

Returns the following output error:

MissingInputError Traceback (most recent call last)
in
1 #OutFunc = keras.backend.function([model2.input], [model2.layers[0].output])
----> 2 OutFunc = keras.backend.function([model2.input], [model2.layers[2].get_output_at(0)])
3
4
5 out_val = OutFunc([inputs])[0]

~/anaconda3/lib/python3.7/site-packages/keras/backend/theano_backend.py in function(inputs, outputs, updates, **kwargs)
725 return T.clip(x, min_value, max_value)
726
--> 727
728 def equal(x, y):
729 return T.eq(x, y)

~/anaconda3/lib/python3.7/site-packages/keras/backend/theano_backend.py in init(self, inputs, outputs, updates, **kwargs)
711
712 def pow(x, a):
--> 713 return T.pow(x, a)
714
715

~/anaconda3/lib/python3.7/site-packages/theano/compile/function.py in function(inputs, outputs, mode, updates, givens, no_default_updates, accept_inplace, name, rebuild_strict, allow_input_downcast, profile, on_unused_input)
315 on_unused_input=on_unused_input,
316 profile=profile,
--> 317 output_keys=output_keys)
318 return fn

~/anaconda3/lib/python3.7/site-packages/theano/compile/pfunc.py in pfunc(params, outputs, mode, updates, givens, no_default_updates, accept_inplace, name, rebuild_strict, allow_input_downcast, profile, on_unused_input, output_keys)
484 accept_inplace=accept_inplace, name=name,
485 profile=profile, on_unused_input=on_unused_input,
--> 486 output_keys=output_keys)
487
488

~/anaconda3/lib/python3.7/site-packages/theano/compile/function_module.py in orig_function(inputs, outputs, mode, accept_inplace, name, profile, on_unused_input, output_keys)
1837 on_unused_input=on_unused_input,
1838 output_keys=output_keys,
-> 1839 name=name)
1840 with theano.change_flags(compute_test_value="off"):
1841 fn = m.create(defaults)

~/anaconda3/lib/python3.7/site-packages/theano/compile/function_module.py in init(self, inputs, outputs, mode, accept_inplace, function_builder, profile, on_unused_input, fgraph, output_keys, name)
1485 # OUTPUT VARIABLES)
1486 fgraph, additional_outputs = std_fgraph(inputs, outputs,
-> 1487 accept_inplace)
1488 fgraph.profile = profile
1489 else:

~/anaconda3/lib/python3.7/site-packages/theano/compile/function_module.py in std_fgraph(input_specs, output_specs, accept_inplace)
179
180 fgraph = gof.fg.FunctionGraph(orig_inputs, orig_outputs,
--> 181 update_mapping=update_mapping)
182
183 for node in fgraph.apply_nodes:

~/anaconda3/lib/python3.7/site-packages/theano/gof/fg.py in init(self, inputs, outputs, features, clone, update_mapping)
173
174 for output in outputs:
--> 175 self.import_r(output, reason="init")
176 for i, output in enumerate(outputs):
177 output.clients.append(('output', i))

~/anaconda3/lib/python3.7/site-packages/theano/gof/fg.py in import_r(self, variable, reason)
344 # Imports the owners of the variables
345 if variable.owner and variable.owner not in self.apply_nodes:
--> 346 self.import(variable.owner, reason=reason)
347 elif (variable.owner is None and
348 not isinstance(variable, graph.Constant) and

~/anaconda3/lib/python3.7/site-packages/theano/gof/fg.py in import(self, apply_node, check, reason)
389 "for more information on this error."
390 % (node.inputs.index(r), str(node)))
--> 391 raise MissingInputError(error_msg, variable=r)
392
393 for node in new_nodes:

MissingInputError: Input 0 of the graph (indices start from 0), used to compute InplaceDimShuffle{x,x}(keras_learning_phase), was not provided and not given a value. Use the Theano flag exception_verbosity='high', for more information on this error.

Backtrace when that variable is created:

File "", line 219, in _call_with_frames_removed
File "/home/jayne/anaconda3/lib/python3.7/site-packages/keras/backend/init.py", line 61, in
from .theano_backend import *
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/jayne/anaconda3/lib/python3.7/site-packages/keras/backend/theano_backend.py", line 23, in
_LEARNING_PHASE = T.scalar(dtype='uint8', name='keras_learning_phase') # 0 = test, 1 = train

from keras-vis.

kweweli avatar kweweli commented on July 26, 2024

I get similar error when I used Model(inputs=..., outputs=...), I usually work around this by explicitly iterating over the layers of the model i want to transfer-learn from. something like this:

# first create a dictionary to hold layer names and indices. This is not necessary
#but I like to have two ways of getting to a layer (using the index or layer name). 
#And the method layers doesn't take # a string or layer name.

model_layer   = {v.name: i for i, v in enumerate(model.layers)}
model_temp = Sequential(name = "trash2")
        
# counter for getting layer index
count = -1   
# pass the layer name to the dict to get the index     
for i in model.layers[ : model_layer[last_layer] + 1]:
       # add layers to new model using old model layers
       model_temp.add(i)
       count +=1
      # set weights to old model's weights
       model_temp.layers[count].set_weights(i.get_weights())

You could then add additional layers to your new model (model_temp) as usual. This always works without the error.

from keras-vis.

somz22 avatar somz22 commented on July 26, 2024

I had the same problem, I tried may techniques but converting sequential model to functional worked for me.

Workaround to convert sequential model to functional

from keras import layers, models

input_layer = layers.Input(batch_shape=model.layers[0].input_shape)
prev_layer = input_layer
for layer in model.layers:
    layer._inbound_nodes = []
    prev_layer = layer(prev_layer)

funcModel = models.Model([input_layer], [prev_layer])

from keras-vis.

Purav-Zumkhawala avatar Purav-Zumkhawala commented on July 26, 2024

@itsnamgyu @pcatattacks
I see a lot of people saying the workaround is to edit the losses.py but where can I find the losses.py. If it is in the site-packages inside Keras, I cannot find the text "layer_output = self.layer.output" in that losses.py file. Is this because of Keras's version?

I am currently using Keras 2.3.1. If I can solve this issue by degrading my Keras version which version should I install?

Please advise

from keras-vis.

yuval-harpaz avatar yuval-harpaz commented on July 26, 2024

don't you get the path in the error message?

from keras-vis.

Purav-Zumkhawala avatar Purav-Zumkhawala commented on July 26, 2024
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-18-a12f24aa46b4> in <module>
     33 while count<8:
     34     print(count)
---> 35     model = pop(model,count)
     36     count+=1
     37 op.reverse()

<ipython-input-18-a12f24aa46b4> in pop(model, i)
     24         else:
     25             model.layers[-1].outbound_nodes = []
---> 26             model.outputs = [model.layers[-i].output]
     27             op.append(model.layers[-i].output)
     28         model.built = True

~\anaconda3\envs\vision\lib\site-packages\keras\engine\base_layer.py in output(self)
    843         if len(self._inbound_nodes) > 1:
    844             raise AttributeError('Layer ' + self.name +
--> 845                                  ' has multiple inbound nodes, '
    846                                  'hence the notion of "layer output" '
    847                                  'is ill-defined. '

AttributeError: Layer vgg16 has multiple inbound nodes, hence the notion of "layer output" is ill-defined. Use `get_output_at(node_index)` instead.

@yuval-harpez This is my error log

from keras-vis.

yuval-harpaz avatar yuval-harpaz commented on July 26, 2024

I don't have the package now, but try dig in, got to keras\engine and try locate losses.py

from keras-vis.

Purav-Zumkhawala avatar Purav-Zumkhawala commented on July 26, 2024

There is no losses.py that contains that text in my whole system. I think that code has changed and that workaround does not work for the latest version.

from keras-vis.

yuval-harpaz avatar yuval-harpaz commented on July 26, 2024

dig in further, surch inside your files for self.layer.get_output_at, don't give up. Every bug has a fix. On Linux I use grep -lr, I don't know the equivalent for you

from keras-vis.

fengcong1992 avatar fengcong1992 commented on July 26, 2024

There is no losses.py that contains that text in my whole system. I think that code has changed and that workaround does not work for the latest version.

I have the same issue with you. losses.py and any other files in Keras2.3.1 does not contain this line: "layer_output = self.layer.output". Any idea?

from keras-vis.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.