Funny fusion effects - constant as output of compilation group about tvm

[t-vi]

Trying to produce a minimal example, I got:

RuntimeError: IValue is not a Scalar
The above operation failed in interpreter, with the following stack trace:
at <ipython-input-15-e6d508d37f4e>:3:12
@torch.jit.script
def func(x)->torch.Tensor:
    return x+torch.tanh(x+x)
           ~~~~~~~~~~~~~~~~ <--- HERE

[t-vi]

So the first error seems related to prim::Constant being declared fusible.

As far as I understand, prim::Constant nodes are not usually declared fusible but instead are pulled in (but duplicated) when they are arguments to a fused node. This happens in GraphFuser::mergeNodeIntoGroup:

https://github.com/pytorch/pytorch/blob/2132ea1d8d6ca9cb2ab6c2206c7cdc9203e1908f/torch/csrc/jit/passes/graph_fuser.cpp#L359-L365

[ulyssesxxxi]

Hi @t-vi,
I'm experiencing the same issue, while trying to compile a model from maskrcnn-benchmark. Do you know if there is any workaround until the issue is fixed?
Thx.

[t-vi]

Yes, sorry, I have been meaning to send a PR, but you simply need to remove prim::Constant fusion:

tvm/torch_tvm/operators.cpp

Line 417 in 7d3fa3f

if (node->kind() == prim::Constant) {

I see, however, that #72 is moving away from the CustomGraphFuser (which ironically seems to be doing funny const handling, too, when the current approach makes fixing this simple).