szagoruyko / pytorchviz

A small package to create visualizations of PyTorch execution graphs

PyTorchViz

A small package to create visualizations of PyTorch execution graphs and traces.

Installation

Install graphviz, e.g.:

brew install graphviz

Install the package itself:

pip install torchviz

Usage

Example usage of make_dot:

model = nn.Sequential()
model.add_module('W0', nn.Linear(8, 16))
model.add_module('tanh', nn.Tanh())
model.add_module('W1', nn.Linear(16, 1))

x = torch.randn(1, 8)
y = model(x)

make_dot(y.mean(), params=dict(model.named_parameters()))

Set show_attrs=True and show_saved=True to see what autograd saves for the backward pass. (Note that this is only available for pytorch >= 1.9.)

model = nn.Sequential()
model.add_module('W0', nn.Linear(8, 16))
model.add_module('tanh', nn.Tanh())
model.add_module('W1', nn.Linear(16, 1))

x = torch.randn(1, 8)
y = model(x)

make_dot(y.mean(), params=dict(model.named_parameters()), show_attrs=True, show_saved=True)

Acknowledgements

The script was moved from functional-zoo where it was created with the help of Adam Paszke, Soumith Chintala, Anton Osokin, and uses bits from tensorboard-pytorch. Other contributors are @willprice, @soulitzer, @albanD.

pytorchviz's Issues

RecursionError: maximum recursion depth exceeded in comparison

I have a loss function and I'm trying to make a graph visualization on the calculation of the loss:

class MVP_Loss(Module):
    def __init__(self, crit=None, model=None):
        self.crit = ifnone(crit, MSELossFlat())
        self.model = model
        self.mask = slice(None)

    def forward(self, preds, target):
        # print(self.mask)
        loss_forward = self.crit(preds, target)
        print("Making dot...")
        dot = make_dot(loss_forward, params=dict(self.model.named_parameters()))
        dot.format = "png"
        dot.render("mseloss_dot")
        print("Made dot.")
        return loss_forward
        # return self.crit(preds[self.mask], target[self.mask])

I see this output:

Making dot...
Traceback (most recent call last):
  File "/home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/torchviz/dot.py", line 154, in add_base_tensor
    add_base_tensor(var._base, color='darkolivegreen3')
  File "/home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/torchviz/dot.py", line 154, in add_base_tensor
    add_base_tensor(var._base, color='darkolivegreen3')
  File "/home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/torchviz/dot.py", line 154, in add_base_tensor
    add_base_tensor(var._base, color='darkolivegreen3')
  [Previous line repeated 991 more times]
  File "/home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/torchviz/dot.py", line 149, in add_base_tensor
    dot.node(str(id(var)), get_var_name(var), fillcolor=color)
  File "/home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/graphviz/dot.py", line 131, in node
    attr_list = self._attr_list(label, attrs, _attributes)
  File "/home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/graphviz/lang.py", line 136, in attr_list
    content = a_list(label, kwargs, attributes)
  File "/home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/graphviz/lang.py", line 107, in a_list
    result = ['label=%s' % quote(label)] if label is not None else []
  File "/home/xander/anaconda3/envs/my_model/lib/python3.7/site-packages/graphviz/lang.py", line 75, in quote
    return '"%s"' % escape_unescaped_quotes(identifier)
  File "/home/xander/anaconda3/envs/my_model/lib/python3.7/re.py", line 311, in _subx
    template = _compile_repl(template, pattern)
RecursionError: maximum recursion depth exceeded in comparison

Python 3.7. torch 1.7.7. torchviz 0.0.2.

Do you have any thoughts on what might be causing this? If it's a blackbox I can try to make a minimal reproducing example.

How does one save the image as a file?

How does one save the image as a file?

LogSoftmaxBackward node seemed to be dupulicated?

I created following model:

class Net(torch.nn.Module):
    def __init__(self, n_feature, n_output):
        super(Net, self).__init__()
        self.l1 = torch.nn.Linear(n_feature, n_hidden)
        self.l2 = torch.nn.Linear(n_hidden, n_hidden)
        self.l3 = torch.nn.Linear(n_hidden, n_output)

    def forward(self, x):
        h1 = self.l1(x)
        h2 = torch.sigmoid(h1)
        h3 = self.l2(h2)
        h4 = torch.sigmoid(h3)
        h5 = self.l3(h4)
        h6 = F.log_softmax(h5, dim=0)
        return h6
 
 net = Net(cols, n_output).to(device)
 optimizer = optim.Adam(net.parameters(), lr=lr)
 criterion = nn.CrossEntropyLoss()       

And I got attached output.
Would you please tell me why LogSoftmaxBackward node seemed to be dupulicated?

Graph terminates in small horizontal bar

Is this a display issue (if so, how can I see the whole graph?) Or is it showing me that the gradients have stopped propagating at that point?

Getting "NoneType" for my graph.

What does this mean? A more informative error/warning message would be cool aswell if nothing is detected. Great library though, this can really help many people!

graphviz.backend.CalledProcessError: Command '['dot', '-Tpng', '-O', 'torchviz-sample']' returned non-zero exit status 1. [stderr: b'Format: "png" not recognized. Use one of:\r\n']

I used this to test,but get an error,how can I fix it? TY
model = nn.Sequential()
model.add_module('W0', nn.Linear(8, 16))
model.add_module('tanh', nn.Tanh())
model.add_module('W1', nn.Linear(16, 1))

x = torch.randn(1, 8)

dot = make_dot(model(x), params=dict(model.named_parameters()))
dot.format = 'png'
dot.render('torchviz-sample')

AttributeError: 'torch._C.Value' object has no attribute 'uniqueName'

when using torch1.3, I use make_dot_from_trace() function to generate the achitecture graph, it incurs an error:'AttributeError: 'torch._C.Value' object has no attribute 'uniqueName''. Any advice?

function object is not iterable error in master

Running master 0.4.0a0+2df578a, get this while running notebook:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-4-fe552e8dc282> in <module>()
----> 1 trace, _ = torch.jit.trace(model, args=(x,))
      2 make_dot_from_trace(trace)

TypeError: 'function' object is not iterable

JIT trace does not work with PyTorch 1.0

ERROR: test_mlp_make_dot_from_trace (__main__.TestTorchviz)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test.py", line 27, in test_mlp_make_dot_from_trace
    dot = make_dot_from_trace(trace)
  File "/Users/szagoruyko/anaconda3/lib/python3.6/site-packages/torchviz/dot.py", line 133, in make_dot_from_trace
    list_of_nodes = parse(graph)
  File "/Users/szagoruyko/anaconda3/lib/python3.6/site-packages/torchviz/dot.py", line 99, in parse
    inputs = [replace(i.uniqueName(), scope) for i in n.inputs()]
  File "/Users/szagoruyko/anaconda3/lib/python3.6/site-packages/torchviz/dot.py", line 99, in <listcomp>
    inputs = [replace(i.uniqueName(), scope) for i in n.inputs()]
  File "/Users/szagoruyko/anaconda3/lib/python3.6/site-packages/torchviz/dot.py", line 80, in replace
    return '/'.join([scope[name], name])
KeyError: 'input.1'

the solution is probably somewhere in https://github.com/lanpa/tensorboardX/blob/master/tensorboardX/pytorch_graph.py#L188

Add variable names to nodes

This is neat and by solving an NP-hard problem in my head I can tell which node corresponds to a given other tensor in my code. Could you add the names of each tensor to its node in the graph? Perhaps using https://pypi.org/project/varname/ or by modifying Tensor.__init__ to extract variable names from declarations in the traceback, then saving them in grad_fn.

Page beyond pdf interface

The front part of the network is truncated, is there any way to get the complete network structure?

my code is:
prediction = model.forward(sample) g = make_dot(prediction,params=dict(model.named_parameters())) g.render('res', format='pdf', view=False)

TypeError: unhashable type: 'list'

import torch
import torchvision
from torchviz import make_dot

model = torchvision.models.detection.retinanet_resnet50_fpn(pretrained=True)
model = model
model.eval()
x = [torch.rand(3, 300, 400)]
predictions = model(x)
print(predictions)

g = make_dot(predictions)
g.render('espnet_model', view=False)

Tests missing

TODO: add tests for make_dot and make_dot_from_trace

Option to show number of parameters in each node

torchsummary has the capability to show the number of parameters in each node, and the shape of the tensor. Could you add this functionality too?

Error when running examples.ipynb

When I run examples.ipynb, notebook shows me the following error message:

trace, _ = torch.jit.trace(model, args=(x,))
make_dot_from_trace(trace)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-95b4d4bfd5bf> in <module>()
      1 trace, _ = torch.jit.trace(model, args=(x,))
----> 2 make_dot_from_trace(trace)

/opt/anaconda-py/anaconda3/lib/python3.6/site-packages/torchviz/dot.py in make_dot_from_trace(trace)
    108     >>> dot = make_dot_from_trace(trace)
    109     """
--> 110     torch.onnx._optimize_trace(trace, False)
    111     graph = trace.graph()
    112     list_of_nodes = parse(graph)

TypeError: _optimize_trace() takes 1 positional argument but 2 were given

How to solve the problem?

'builtin_function_or_method' object has no attribute 'grad_fn'

Thanks for your sharing!
But I meet this question about

` File "/home/meichen/PycharmProjects/CBAM /cbam+resnet/MODELS/model_resnet.py", line 282, in
make_dot(output.mean, params=dict(net.named_parameters()))
File "/home/meichen/anaconda3/envs/reidbaseline/lib/python3.6/site-packages/torchviz/dot.py", line 37, in make_dot
output_nodes = (var.grad_fn,) if not isinstance(var, tuple) else tuple(v.grad_fn for v in var)

AttributeError: 'builtin_function_or_method' object has no attribute 'grad_fn'
`

Pls help. Thank you very much.

How do i visualize networks with more then one input

Hi,

for example let's say i'm visualizing a GAN architecture for style transfer which has inputes

lets call the network Gen and it recieves two images from two styles image_a and image_b.

once i try to do as in the exmple i get the next error

AttributeError: 'tuple' object has no attribute 'grad_fn'

Inconsistent arrow directions for saved variables

For saved variables created in a custom autograd function, the direction SavedVariable -> Node is used.
(cf. https://github.com/szagoruyko/pytorchviz/blob/master/torchviz/dot.py#L141)

For the other ones, no direction is used (cf. https://github.com/szagoruyko/pytorchviz/blob/master/torchviz/dot.py#L110 and https://github.com/szagoruyko/pytorchviz/blob/master/torchviz/dot.py#L116)

Do not support pytorch>=0.4.1?

@szagoruyko This will be failed: assert all(isinstance(p, Variable) for p in params.values())

Error while doing make_dot_from_trace

When I run the make_dot_from_trace function with argument as the output from torch.jit.trace function; I get the following error

How to draw a rnn graph?

Does not output any figure

Hello,
I was running a simple code that is provided in examples. However, nothing showed up on the screen.
Can anyone help me?

Not every modal has a grad_fn

I personally think that sometimes self-defined modal might not have grad_fn, so maybe could consider this more general case.

'NetworkCIFAR' object has no attribute 'grad_fn'

children module may cause a problem

i made a mistake

`KeyError` in `name = param_map[id(u)]`

When I run:

import torch
from torch import nn
import torchviz

h1 = nn.Linear(1, 1)

inputs = torch.zeros(1, 1, requires_grad=True)
inputs.data = torch.rand(1, 1)
print('inputs', inputs)
outputs = h1(inputs)
print('outputs', outputs)
torchviz.make_dot(outputs, params={'inputs': inputs})

I get:

inputs tensor([[0.3559]], requires_grad=True)
outputs tensor([[0.4822]], grad_fn=<ThAddmmBackward>)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-4-a36a1735c26c> in <module>()
     10 outputs = h1(inputs)
     11 print('outputs', outputs)
---> 12 torchviz.make_dot(outputs, params={'inputs': inputs})

/toknas/hugh/condap4/lib/python3.6/site-packages/torchviz/dot.py in make_dot(var, params)
     68             add_nodes(v.grad_fn)
     69     else:
---> 70         add_nodes(var.grad_fn)
     71 
     72     resize_graph(dot)

/toknas/hugh/condap4/lib/python3.6/site-packages/torchviz/dot.py in add_nodes(var)
     57                     if u[0] is not None:
     58                         dot.edge(str(id(u[0])), str(id(var)))
---> 59                         add_nodes(u[0])
     60             if hasattr(var, 'saved_tensors'):
     61                 for t in var.saved_tensors:

/toknas/hugh/condap4/lib/python3.6/site-packages/torchviz/dot.py in add_nodes(var)
     57                     if u[0] is not None:
     58                         dot.edge(str(id(u[0])), str(id(var)))
---> 59                         add_nodes(u[0])
     60             if hasattr(var, 'saved_tensors'):
     61                 for t in var.saved_tensors:

/toknas/hugh/condap4/lib/python3.6/site-packages/torchviz/dot.py in add_nodes(var)
     45             elif hasattr(var, 'variable'):
     46                 u = var.variable
---> 47                 name = param_map[id(u)] if params is not None else ''
     48                 node_name = '%s\n %s' % (name, size_to_str(u.size()))
     49                 dot.node(str(id(var)), node_name, fillcolor='lightblue')

KeyError: 140051140404712

collectenv output:

Collecting environment information...
PyTorch version: 0.4.1
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.5 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.0.176
GPU models and configuration: GPU 0: Tesla M60
Nvidia driver version: 384.130
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.5.1.10
/usr/lib/x86_64-linux-gnu/libcudnn.so.6.0.21
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.0.3
/usr/lib/x86_64-linux-gnu/libcudnn_static_v5.a
/usr/lib/x86_64-linux-gnu/libcudnn_static_v6.a
/usr/lib/x86_64-linux-gnu/libcudnn_static_v7.a

Versions of relevant libraries:
[pip] numpy (1.14.3)
[pip] pytorchviz (0.0.1)
[pip] torch (0.4.1)
[pip] torchtext (0.2.3)
[pip] torchvision (0.2.1)
[conda] pytorch                   0.4.1           py36_cuda9.0.176_cudnn7.1.2_1    pytorch
[conda] pytorchviz                0.0.1                     <pip>
[conda] torchtext                 0.2.3                     <pip>
[conda] torchvision               0.2.1                    py36_1    pytorch

i got an empty pdf

first i meet this problem: AttributeError: 'list' object has no attribute 'grad_fn', then i concatenate the output:

    model = models.__dict__['load_arc_face']()
    net = model.model

    from torchviz import make_dot
    x = torch.rand(8, 3, 112, 112).cuda()
    y = net(x)
    for i in range(len(y)):
        if i == 0: c = torch.cat((y[0], y[1]), 0)
        elif i >= 2 and i <= len(y)-2:
            c = torch.cat((c, y[i+1]), 0)
    g = make_dot(c)
    g.render('net_arch', view=False)  

but at last it only saved an empty PDF for me, why ?
net_arch.pdf

save problem

how can I save the output as png or jpg?

how about an example that triggers fillcolor='orange' in dot.py?

In dot.py there is a case in which a node will be plotted with fillcolor='orange'. However, none of the example networks in examples.ipynb show orange boxes - only blue and gray. Under what conditions are you supposed to get orange boxes? Can you tweak one of the example networks to illustrate this?

I've been playing with the code in dot.py, but I haven't been able to define a network that produces orange boxes... I don't think "torch.is_tensor(var)" ever evaluates to true... at least in no network I could come up with. (However, "torch.is_tensor(var.variable)" does sometimes evaluate true.)

Thanks! (Very useful tool!)

torchviz images not showing in vs code jupyuter

https://github.com/microsoft/vscode-python/issues/10168

Pass figure size as argument for displaying huge graphs

Hi,

Can the output figure size be set as an argument instead of hard-coding it to (12, 12) here?

It helps when printing huge graphs.

How to use the package with Google Colaboratory?

When using the package on https://colab.research.google.com, I get the following error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/graphviz/backend.py in pipe(engine, format, data, quiet)
    153             stdout=subprocess.PIPE, stderr=subprocess.PIPE,
--> 154             startupinfo=STARTUPINFO)
    155     except OSError as e:

/usr/lib/python3.6/subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, encoding, errors)
    708                                 errread, errwrite,
--> 709                                 restore_signals, start_new_session)
    710         except:

/usr/lib/python3.6/subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session)
   1343                             err_msg += ': ' + repr(err_filename)
-> 1344                     raise child_exception_type(errno_num, err_msg, err_filename)
   1345                 raise child_exception_type(err_msg)

FileNotFoundError: [Errno 2] No such file or directory: 'dot': 'dot'

During handling of the above exception, another exception occurred:

ExecutableNotFound                        Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/IPython/core/formatters.py in __call__(self, obj)
    336             method = get_real_method(obj, self.print_method)
    337             if method is not None:
--> 338                 return method()
    339             return None
    340         else:

/usr/local/lib/python3.6/dist-packages/graphviz/files.py in _repr_svg_(self)
    104 
    105     def _repr_svg_(self):
--> 106         return self.pipe(format='svg').decode(self._encoding)
    107 
    108     def pipe(self, format=None):

/usr/local/lib/python3.6/dist-packages/graphviz/files.py in pipe(self, format)
    123         data = text_type(self.source).encode(self._encoding)
    124 
--> 125         outs = backend.pipe(self._engine, format, data)
    126 
    127         return outs

/usr/local/lib/python3.6/dist-packages/graphviz/backend.py in pipe(engine, format, data, quiet)
    155     except OSError as e:
    156         if e.errno == errno.ENOENT:
--> 157             raise ExecutableNotFound(args)
    158         else:  # pragma: no cover
    159             raise

ExecutableNotFound: failed to execute ['dot', '-Tsvg'], make sure the Graphviz executables are on your systems' PATH

Should we resolve by doing:

import os
os.environ["PATH"] += 'whatever/the/path/should/be'

Incorrect graph for saved variable using custom autograd function

In the following example, the saved variable for each FixedGradientFunctionBackward Node should be different, but they are merged into a single dot.node

https://colab.research.google.com/drive/1MyOV58n6oex9X_Z5HSRf-Gb87dclYNKN?usp=sharing

from torch.autograd import Function
from torchviz import make_dot

class FixedGradientFunction(Function):
    @staticmethod
    def forward(ctx, x, grad_x):
        ctx.save_for_backward(grad_x)
        return x

    @staticmethod
    def backward(ctx, grad_x):
        saved_grad_x, = ctx.saved_tensors
        return saved_grad_x, None

fn = FixedGradientFunction

x = torch.randn(1, requires_grad=True)

dense_1 = torch.rand(1)
dense_2 = torch.rand(1)

z = (fn.apply(x, dense_1) + fn.apply(x, dense_2)).sum()
# z.backward()
make_dot(z)

TypeError: unhashable type: 'list'

File "F:\Anaconda3\lib\site-packages\torchviz\dot.py", line 38, in add_nodes
if var not in seen:
TypeError: unhashable type: 'list'

AttributeError: 'list' object has no attribute 'grad_fn'

File "C:\Users\admin\AppData\Local\conda\conda\envs\pytorch\lib\site-packages\torchviz\dot.py", line 38, in make_dot
output_nodes = (var.grad_fn,) if not isinstance(var, tuple) else tuple(v.grad_fn for v in var)

AttributeError: 'list' object has no attribute 'grad_fn'

use:
x= torch.randn(1, 3, 800, 800)
y = self.model.cpu()(x)
vis_graph = make_dot(y, params=dict(list(self.model.named_parameters()) ))

Nothing is displayed

Hi,
does it work only in a Notebook?
Is there a way to save the result as an image?
Regards,

Consider making clear that adding a 'print' after 'make_dot' will clear the graph

Hi Sergey,

Cool repo :)

Works pretty nicely :) The only thing, it took me a while to figure out why it wasnt working for me. It was because I had a print statement after the make_dot. Might be worth considering pointing this out in the doc perhaps?

vs

Is this working properly with Conv2d and MaxPool2d?

I'm trying to use the library to visualize a CNN, but the output doesn't seem consistent.
Thanks

No text label displayed on Google Colab (Pytorch 1.8.1)

I run following sample code on both local anaconda and google colab.
And on google colab no text label dispkayed on a result node.

Sample Code:

import torch
from torchviz import make_dot
print(torch.__version__)
x_np = np.arange(-2, 2.1, 0.25)
x = torch.tensor(x_np, requires_grad=True)
y = 2 * x**2 + 2
z = y.sum()
params = dict(x = x)
make_dot(z, params=params)

Result 1 (local anaconda pytorch 1.7.1)

Result 2 (Google Colab pytorch 1.8.1)

add LICENSE?

Hey, would you like to add an open source license?

Issue with Multiple Networks

Hi,

I have a model which is a combination of two networks -- one's output going as input to the next one. The issues I face are:

1. For make_dot, I need to have a dictionary of all named parameters, which in turn means the layer names of one network cannot be the same as that of the second one.
2. For make_dot_from_trace, it creates some graph (which is not right), and since I want to compute the graph for the entire model end-to-end, I pass in the last network in torch.jit.get_trace_graph, but then only picks up the name of the second network and populates that name everywhere.

Any solutions around this?

float tensor variable got error

I got make_dot error in following code.
Would you please tell me what should I do?

good case:

import numpy as np
import torch
from torchviz import make_dot
x_np = np.arange(-2, 2.1, 0.25)
x = torch.tensor(x_np, requires_grad=True)
y = 2 * x**2 + 2
z = y.sum()
params = dict(x = x)
make_dot(z, params=params)

bad case:

import numpy as np
import torch
from torchviz import make_dot
x_np = np.arange(-2, 2.1, 0.25)
x = torch.tensor(x_np, requires_grad=True).float()
y = 2 * x**2 + 2
z = y.sum()
params = dict(x = x)
make_dot(z, params=params)

Compare graphs

How do we compare two graphs and show the difference?

It must be a structural comparison since the nodes have different object IDs

How to draw a graph with multiple outputs?

Hi, @szagoruyko

Thanks for your repo! pytorchviz has made network drafting and demonstration much more convenient.

From #15 , your said that multiple outputs was supported then.

But in constructing a net as the following, I keeps receiving the error like #10 .

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms
from torchviz import make_dot

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)
        self.fc1_2 = nn.Linear(320, 60)
        self.fc2_2 = nn.Linear(60, 10)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        y1 = F.relu(self.fc1(x))
        y1 = F.dropout(y1, training=self.training)
        y1 = self.fc2(y1)
        y2 = F.relu(self.fc1_2(x))
        y2 = F.dropout(y2, training=self.training)
        y2 = self.fc2_2(y2)
        # case1: works, but single output
        y = F.log_softmax(y1, dim=1) + F.log_softmax(y2, dim=1)
        return y
        # case2: fails, like issue #10 
        return F.log_softmax(y1, dim=1), F.log_softmax(y2, dim=1)

model = Net()

x = torch.randn(1, 1, 28, 28).requires_grad_(True)
# for case1
y = model(x)
g = make_dot(y, params=dict(list(model.named_parameters()) + [('x', x)]))
# for case2
y1, y2 = model(x) 
y = (y1, y2) # to form a tuple
g = make_dot(y, params=dict(list(model.named_parameters()) + [('x', x)]))
g.view()

And the traceback:

Traceback (most recent call last):
  File "net_test.py", line 39, in <module>
    g = make_dot(y, params=dict(list(model.named_parameters()) + [('x', x)]))
  File "/usr/local/lib/python3.6/site-packages/torchviz/dot.py", line 60, in make_dot
    add_nodes(var.grad_fn)
AttributeError: 'tuple' object has no attribute 'grad_fn'

Must it be running in jupyter notebook?

Or any way to export the graph to image files?

How to hide the backward node?

I plot the graph in a PDF which is a long file. Is there any method to hide the backward so that the graph is smaller and more clear ?

make_dot without backward nodes?

For a huge or complicated network, directly drawing with backward nodes seems to be lengthy and jumbled. So I'm wondering does this package support drawing model graph without showing backward nodes (but only show forward layers/nodes)?

Thanks for your attention.

RecursionError when used on subclass of Tensor

When using subclasses of Tensor, make_dot crashed with RecursionError.

class DummyTensor(torch.Tensor):
    pass

a = DummyTensor(torch.ones(5, requires_grad=True))
make_dot(a)

Output:

/usr/local/lib/python3.7/dist-packages/torchviz/dot.py in add_base_tensor(var, color)
    152             dot.edge(str(id(var.grad_fn)), str(id(var)))
    153         if var._is_view():
--> 154             add_base_tensor(var._base, color='darkolivegreen3')
    155             dot.edge(str(id(var._base)), str(id(var)), style="dotted")
    156 

RecursionError: maximum recursion depth exceeded in __instancecheck__

The reason is because a is a view, so is a._base, so is a._base._base, so is a._base._base._base, etc.

https://colab.research.google.com/drive/1VlAEQY5lOtp3MsYfgyCkrNmkjzuKoRX1?usp=sharing

make_dot_from_trace fails for very simple bivariate function

Is there a reason why the following simple example fails? I'm having a hard time figuring it out from the error message.

from torch.autograd import Variable
import torch.nn as nn
import torch
from torchviz import make_dot_from_trace

class toy(nn.Module):
    
    def __init__(self):
        super(toy, self).__init__()
        
    def forward(self, x, y):
        return x + y

f = toy()
a = Variable(torch.FloatTensor([3.0]), requires_grad=True)
b = Variable(torch.FloatTensor([2.0]), requires_grad=True)

trace, _ = torch.jit.trace(f, args=(a,b))
make_dot_from_trace(trace)

The error message from running this is:

File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torchviz/dot.py", line 110, in make_dot_from_trace torch.onnx._optimize_trace(trace, False) TypeError: _optimize_trace() takes 1 positional argument but 2 were given

How to adjust the resolution of the saved dot image?

How to adjust the resolution of the saved dot image?
My computational graph is too long...then the image seems not clear enough.

AttributeError: 'tuple' object has no attribute 'grad_fn'

My env is python2.7+pytorch0.3.1.
When I try this repo, I face the problem:
AttributeError: 'tuple' object has no attribute 'grad_fn'

Does anybody know why?
Is it because the pytorch version is not supported?