Export quantized model about distiller

[nzmora]

Hi @cattpku,

We're waiting for the ONNX quantization standard, before supporting ONNX export of quantized models. See #23 for some discussion.
You can export a non-quantized model to ONNX using --summary=onnx.
Cheers
Neta

[cattpku]

Hi Neta,
I tried to export a non-quantized model to ONNX as advised, however, got a RuntimeError 'ONNX export failed: Couldn't export operator aten::softmax'.

I know this layer is added in Distiller, and I checked the corresponding 'torch/onnx/symbolic.py', the 'softmax' definition is there.

After checking Pytorch 0.4.0 doc, for 'torch.onnx' it stated 'softmax (only dim=-1 supported)', while in 'model_summaries.py', it is 'softmax = torch.nn.Softmax(dim=1)'.

Can you kindly advise? Thanks.

[nzmora]

Hi @cattpku ,

Please share your command-line, so I can try to recreate this problem.
I think that using

softmax = torch.nn.Softmax(dim=-1)

is probably more correct, but I want to see why it fails for you before making changes to the code.

BTW, you can also try to change it to -1 and report if this fixes the fail you see.

Thanks
Neta

[cattpku]

Hi Neta,
I am not using command-line, as I put the whole code in Pycharm to run it. I just set the following args:
--data: my own dataset path
--arch: my own network definition
--summary: default = 'onnx'

I tried to change it to -1 as suggested, it went well, no error occurred.

[nzmora]

Thanks, for classification tasks the softmax input has shape==(1, num_classes) so

softmax = torch.nn.Softmax(dim=1)

and

softmax = torch.nn.Softmax(dim=-1)

are the same. But as you point out, for the general case we should follow the PyTorch ONNX limitations.