alibaba / tinyneuralnetwork Goto Github PK
View Code? Open in Web Editor NEWTinyNeuralNetwork is an efficient and easy-to-use deep learning model compression framework.
License: MIT License
TinyNeuralNetwork is an efficient and easy-to-use deep learning model compression framework.
License: MIT License
Execute a python file like below:
import torch
from torch import nn
from tinynn.converter import TFLiteConverter
from tinynn.graph.quantization.quantizer import QATQuantizer
from tinynn.graph.tracer import model_tracer
from tinynn.util.train_util import DLContext, get_device, train
class DummyNet(nn.Module):
def __init__(self, num_classes=4):
super(DummyNet, self).__init__()
self.input_channel = 1
self.base_channel = 4
def conv_bn(inp, oup, stride):
return nn.Sequential(
nn.Conv2d(inp, oup, 3, stride, 1, bias=False),
nn.BatchNorm2d(oup),
nn.ReLU(inplace=True)
)
def conv_dw(inp, oup, stride):
return nn.Sequential(
nn.Conv2d(inp, inp, 3, stride, 1, groups=inp, bias=False),
nn.Conv2d(inp, oup, 1, 1, 0, bias=False),
nn.BatchNorm2d(oup),
nn.ReLU(inplace=True)
)
self.model = nn.Sequential(
conv_bn(self.input_channel, self.base_channel, 2),
conv_dw(self.base_channel, self.base_channel * 2, 1),
conv_dw(self.base_channel * 2, self.base_channel * 4, 2),
conv_dw(self.base_channel * 4, self.base_channel * 8, 2),
conv_dw(self.base_channel * 8, self.base_channel * 16, 2),
conv_dw(self.base_channel * 16, self.base_channel * 16, 1),
conv_dw(self.base_channel * 16, self.base_channel * 16, 1),
conv_dw(self.base_channel * 16, self.base_channel * 32, 2),
conv_dw(self.base_channel * 32, self.base_channel * 32, 1),
nn.AvgPool2d(kernel_size=(3, 6)),
nn.Flatten(),
nn.Linear(self.base_channel * 32, num_classes)
)
self.softmax = nn.Softmax(dim=1)
def forward(self, x):
x = self.model(x)
x = self.softmax(x)
return x
def predict(self, x):
x = self.forward(x)
x = torch.argmax(x, dim=1)
return x
if __name__ == '__main__':
with model_tracer():
model = DummyNet()
model.eval()
dummy_input = torch.rand((1, 1, 135, 240))
quantizer = QATQuantizer(model, dummy_input, work_dir='out')
qat_model = quantizer.quantize()
device = get_device()
qat_model.to(device=device)
with torch.no_grad():
qat_model.eval()
qat_model.cpu()
qat_model = torch.quantization.convert(qat_model)
torch.backends.quantized.engine = 'qnnpack'
converter = TFLiteConverter(qat_model, dummy_input, tflite_path='out/dummy_qat.tflite')
converter.convert()
And then I got below error:
File "/root/miniconda3/lib/python3.7/site-packages/torch/jit/_trace.py", line 744, in trace
_module_class,
File "/root/miniconda3/lib/python3.7/site-packages/torch/jit/_trace.py", line 959, in trace_module
argument_names,
File "/root/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1039, in _slow_forward
result = self.forward(*input, **kwargs) File "out/dummynet_qat.py", line 96, in forward softmax = self.softmax(model_11) File "/root/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1039, in _slow_forward
result = self.forward(*input, **kwargs)
File "/root/miniconda3/lib/python3.7/site-packages/torch/nn/modules/activation.py", line 1256, in forward
return F.softmax(input, self.dim, _stacklevel=5)
File "/root/miniconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 1679, in softmax
ret = input.softmax(dim)
NotImplementedError: Could not run 'aten::_softmax' with arguments from the 'QuantizedCPU' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::_softmax' is only available for these backends: [CPU, CUDA, MkldnnCPU, BackendSelect, Named, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, UNKNOWN_TENSOR_TYPE_ID, AutogradMLC, AutogradHPU, AutogradNestedTensor, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, Autocast, Batched, VmapMode].
CPU: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/build/aten/src/ATen/RegisterCPU.cpp:16286 [kernel]
CUDA: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/build/aten/src/ATen/RegisterCUDA.cpp:20674 [kernel]
MkldnnCPU: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/build/aten/src/ATen/RegisterMkldnnCPU.cpp:563 [kernel]
BackendSelect: fallthrough registered at /opt/conda/conda-bld/pytorch_1623448265233/work/aten/src/ATen/core/BackendSelectFallbackKernel.cpp:3 [backend fallback]
Named: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback]
ADInplaceOrView: fallthrough registered at /opt/conda/conda-bld/pytorch_1623448265233/work/aten/src/ATen/core/VariableFallbackKernel.cpp:60 [backend fallback]
AutogradOther: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradCPU: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradCUDA: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradXLA: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
UNKNOWN_TENSOR_TYPE_ID: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradMLC: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradHPU: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradNestedTensor: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradPrivateUse1: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradPrivateUse2: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
AutogradPrivateUse3: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/VariableType_0.cpp:9848 [autograd kernel]
Tracer: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/torch/csrc/autograd/generated/TraceType_0.cpp:9750 [kernel]
Autocast: fallthrough registered at /opt/conda/conda-bld/pytorch_1623448265233/work/aten/src/ATen/autocast_mode.cpp:255 [backend fallback]
Batched: registered at /opt/conda/conda-bld/pytorch_1623448265233/work/aten/src/ATen/BatchingRegistrations.cpp:1019 [backend fallback]
VmapMode: fallthrough registered at /opt/conda/conda-bld/pytorch_1623448265233/work/aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]
It looks like Softmax is not implemented for QAT. If quantized Softmax is not supported, floating Softmax is good to me.
Could you implement it?
Many Thanks
I applied QAT to a model and the final conversion to tflite fails with the following error messages:
torch.jit._trace.TracingCheckError: Tracing failed sanity checks!
ERROR: Tensor-valued Constant nodes differed in value across invocations. This often indicates that the tracer has encountered untraceable code.
Node:
%data_1 : Tensor = prim::Constantvalue= # out-2022-01-17-08-27-39/detector_qat.py:259:0
Source Location:
out-2022-01-17-08-27-39/detector_qat.py(259): forward
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py(709): _slow_forward
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py(725): _call_impl
/usr/local/lib/python3.6/dist-packages/torch/jit/_trace.py(940): trace_module
/usr/local/lib/python3.6/dist-packages/torch/jit/_trace.py(742): trace
/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+e35ef92faba445830bb6156c916b1e838801c07e-py3.6.egg/tinynn/converter/base.py(97): init_jit_graph
/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+e35ef92faba445830bb6156c916b1e838801c07e-py3.6.egg/tinynn/converter/base.py(281): convert
main.py(133): main
main.py(193):
Comparison exception: promoteTypes with quantized numbers is not handled yet; figure out what the correct rules should be, offending types: QUInt8 Float
Please refer to the attachment. Thanks.
test.zip
Why not use torch.FX module and no need a lot of manual work in latest version.
Please see the below colab that I am using to convert mobilet v2 from pytorch to tflite(int8)
https://colab.research.google.com/drive/1eW-I0RDzB3L6Zbz364t5lkI4fxgvpGbI#scrollTo=5YtQg5Ga2wmq
Getting the below erro]rs
Traceback (most recent call last):
File "./examples/converter/convert.py", line 9, in
from examples.models.cifar10.mobilenet import DEFAULT_STATE_DICT, Mobilenet
ModuleNotFoundError: No module named 'examples.models'
Hi developer,
Thank your great work. I want to use QATQuantizer
to quantize my model, but in converter, a error appears:
RuntimeError: [enforce fail at q_avgpool.cpp:369] createStatus == pytorch_qnnp_status_success. failed to create QNNPACK Average Pooling operator
I'm using Pytorch v1.10. Is this a QNNPACK issue? I try to use fbgemm and it works. Thank you!
Below is whole call stack:
18 converter = TFLiteConverter(qat_model, dummy_input, tflite_path='tflite_model/qat_model.tflite')
---> 19 converter.convert()
/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+6bdac3b8f7d9fd53d7ef4c07920414f46d6e2e62-py3.6.egg/tinynn/converter/base.py in convert(self)
332 """
333 self.init_input_transpose()
--> 334 self.init_jit_graph()
335 self.init_lowered_module()
336 self.init_common_graph()
/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+6bdac3b8f7d9fd53d7ef4c07920414f46d6e2e62-py3.6.egg/tinynn/converter/base.py in init_jit_graph(self)
120
121 with torch.no_grad():
--> 122 script = torch.jit.trace(self.model, self.dummy_input)
123
124 # Remove reference to original model to save memory
/usr/local/lib/python3.6/dist-packages/torch/jit/_trace.py in trace(func, example_inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)
748 strict,
749 _force_outplace,
--> 750 _module_class,
751 )
752
/usr/local/lib/python3.6/dist-packages/torch/jit/_trace.py in trace_module(mod, inputs, optimize, check_trace, check_inputs, check_tolerance, strict, _force_outplace, _module_class, _compilation_unit)
963 strict,
964 _force_outplace,
--> 965 argument_names,
966 )
967 check_trace_method = module._c._get_method(method_name)
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1100 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1101 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102 return forward_call(*input, **kwargs)
1103 # Do not call functions when jit is used
1104 full_backward_hooks, non_full_backward_hooks = [], []
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
1088 recording_scopes = False
1089 try:
-> 1090 result = self.forward(*input, **kwargs)
1091 finally:
1092 if recording_scopes:
/home/okt/aidea-semantic/semantic-segmentation/out/ddrnet_qat.py in forward(self, input_1)
376 spp_process3_1 = self.spp_process3_1(spp_process3_0)
377 spp_process3_2 = self.spp_process3_2(spp_process3_1)
--> 378 spp_scale4_0 = self.spp_scale4_0(add_18)
379 spp_scale4_1 = self.spp_scale4_1(spp_scale4_0)
380 spp_scale4_2 = self.spp_scale4_2(spp_scale4_1)
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1100 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1101 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102 return forward_call(*input, **kwargs)
1103 # Do not call functions when jit is used
1104 full_backward_hooks, non_full_backward_hooks = [], []
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in _slow_forward(self, *input, **kwargs)
1088 recording_scopes = False
1089 try:
-> 1090 result = self.forward(*input, **kwargs)
1091 finally:
1092 if recording_scopes:
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/pooling.py in forward(self, input)
615 def forward(self, input: Tensor) -> Tensor:
616 return F.avg_pool2d(input, self.kernel_size, self.stride,
--> 617 self.padding, self.ceil_mode, self.count_include_pad, self.divisor_override)
618
619
RuntimeError: [enforce fail at q_avgpool.cpp:369] createStatus == pytorch_qnnp_status_success. failed to create QNNPACK Average Pooling operator
Use TFLiteConvert to convert a quantized model containing InstanceNorm2D like below
converter = TFLiteConverter(model, dummy_input, tflite_path=args.output)
converter.convert()
And the error occured like below
ERROR (tinynn.converter.base) Unsupported ops: quantized::instance_norm
Traceback (most recent call last):
File "export/export_model_to_tflite.py", line 131, in <module>
main()
File "export/export_model_to_tflite.py", line 117, in main
converter.convert()
File "/root/miniconda3/lib/python3.7/site-packages/tinynn/converter/base.py", line 269, in convert
raise Exception("Cannot continue due to fatal error")
Exception: Cannot continue due to fatal error
Look like op "quantized::instance_norm" is unsupported. Could you support "quantized::instance_norm"?
Many Thanks
Hi, sorry to bother. This issue may not be directly related to tinynn. The scenario on my side is described as following:
My target is to keep the weights/bias the same with the original model for the frozen layers after dequantization. Not sure if it's feasible. As you know, the official float32 model performs well for most cases and I just want to fine tune a few layers. Meanwhile, considering the inference speed on some edge devices, I have to perform QAT since the post-training quantization is really worse.
Thanks for your time.
Assertion fails during conversion to tflite for u8 QAT with the following errors:
converter.convert()
File "/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+a60ddb1432bfd81eebd2d8d232e3a67bb6d2c434-py3.6.egg/tinynn/converter/base.py", line 264, in convert
self.init_operations()
File "/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+a60ddb1432bfd81eebd2d8d232e3a67bb6d2c434-py3.6.egg/tinynn/converter/base.py", line 231, in init_operations
converter.parse(node, attrs, args, self.common_graph)
File "/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+a60ddb1432bfd81eebd2d8d232e3a67bb6d2c434-py3.6.egg/tinynn/converter/operators/torch/aten.py", line 479, in parse
self.elementwise_unary(tfl.QuantizeOperator, graph_converter)
File "/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+a60ddb1432bfd81eebd2d8d232e3a67bb6d2c434-py3.6.egg/tinynn/converter/operators/torch/base.py", line 252, in elementwise_unary
outputs = self.to_tfl_tensors(self.output_names, self.output_tensors)
File "/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+a60ddb1432bfd81eebd2d8d232e3a67bb6d2c434-py3.6.egg/tinynn/converter/operators/torch/base.py", line 155, in to_tfl_tensors
t = tfl.Tensor(t, n, has_buffer=non_existent_as_buffer, asymmetric=self.asymmetric)
File "/usr/local/lib/python3.6/dist-packages/TinyNeuralNetwork-0.1.0+a60ddb1432bfd81eebd2d8d232e3a67bb6d2c434-py3.6.egg/tinynn/converter/operators/tflite/base.py", line 180, in init
assert tensor.q_zero_point() == 128, "As for symmetric quantization, "
AssertionError: As for symmetric quantization, the zero point of the u8 tensors should be 128. This could happen if you didn't train the model after QAT preparation.
Actually, I remember u8 QAT works in a certain older tinynn version. The quantizer code of mine is written like this:
quantizer = QATQuantizer(model, dummy_input, work_dir='out-', config={'backend': "qnnpack", 'force_overwrite': True, 'asymmetric': True, 'per_tensor': True, 'rewrite_graph': True})
Thanks for your excellent work. when converting the model from Pytorch to TFLite, it does not support the following operations:
ERROR (TinyNeuralNetwork.tinynn.converter.base) Unsupported ops: aten::remainder, aten::where
Hope you can support these ops later. Thanks.
非常棒的Repo,但我想使用convert.py
将PyTorch训练好的3DCNN分类模型转换为.tflite
模型,发生了错误:
问题1:x = F.avg_pool3d(x, x.data.size()[-3:])
不支持
ERROR (tinynn.converter.base) Unsupported ops: aten::avg_pool3d
问题2:我将Temporal
通道Squeeze掉了,但还是发生了错误:
Traceback (most recent call last):
File "/home/leovin/Dynamic Gesture Detection/3D-CNNs/TinyNeuralNetwork/examples/converter/convert_for_3DCNNs.py", line 50, in <module>
main_worker()
File "/home/leovin/Dynamic Gesture Detection/3D-CNNs/TinyNeuralNetwork/examples/converter/convert_for_3DCNNs.py", line 46, in main_worker
converter.convert()
File "/home/leovin/anaconda3/envs/dgr/lib/python3.9/site-packages/TinyNeuralNetwork-0.1.20220310142810-py3.9.egg/tinynn/converter/base.py", line 347, in convert
optimizer.optimize()
File "/home/leovin/anaconda3/envs/dgr/lib/python3.9/site-packages/TinyNeuralNetwork-0.1.20220310142810-py3.9.egg/tinynn/converter/operators/optimize.py", line 1377, in optimize
self.transform_graph()
File "/home/leovin/anaconda3/envs/dgr/lib/python3.9/site-packages/TinyNeuralNetwork-0.1.20220310142810-py3.9.egg/tinynn/converter/operators/optimize.py", line 210, in transform_graph
op.transform(self.graph, mapping)
File "/home/leovin/anaconda3/envs/dgr/lib/python3.9/site-packages/TinyNeuralNetwork-0.1.20220310142810-py3.9.egg/tinynn/converter/operators/tflite/transformable.py", line 235, in transform
assert False, "Only Conv[Transpose]1d/2d is supported"
AssertionError: Only Conv[Transpose]1d/2d is supported
Process finished with exit code 1
Only Conv[Transpose]1d/2d is supported
请问是因为不支持Conv3d
吗?
问题3:如果支持Conv3d
,我该怎么做?
期待您的回答!
class Net(nn.Module):
def __init__(self):
super().__init__()
self.register_parameter("x", nn.Parameter(torch.ones(2, 3)))
def forward(self, data):
out = [data]
for i in range(2):
y1 = torch.cat([out[-1], self.x])
y2 = torch.cat([out[-1], self.x])
y = y1 + y2
out.append(y)
return out
class Net(torch.nn.Module):
def __init__(self):
super().__init__()
self.register_parameter("tensor_1", torch.nn.Parameter(torch.empty((2, 3), dtype=torch.float32)))
self.register_parameter("tensor_2", torch.nn.Parameter(torch.empty((2, 3), dtype=torch.float32)))
self.register_parameter("tensor_3", torch.nn.Parameter(torch.empty((2, 3), dtype=torch.float32)))
self.register_parameter("tensor_4", torch.nn.Parameter(torch.empty((2, 3), dtype=torch.float32)))
def forward(self, input_1):
cat_1 = torch.cat([input_1, self.tensor_1])
cat_2 = torch.cat([input_1, self.tensor_2])
add_1 = cat_1.__add__(cat_2)
cat_3 = torch.cat([add_1, self.tensor_3])
cat_4 = torch.cat([add_1, self.tensor_4])
add_2 = cat_3.__add__(cat_4)
return input_1, add_1, add_2
There is a kind of quantization that quantizes model with dynamic fixed point int16. Is there any chance to support int16 quantization for QAT and mixed QAT?
when using converter.convert()
File "/home/shanshaojie/project/TinyNeuralNetwork/tinynn/converter/base.py", line 356, in convert
self.init_jit_graph()
File "/home/shanshaojie/project/TinyNeuralNetwork/tinynn/converter/base.py", line 143, in init_jit_graph
script = torch.jit.trace(self.model, self.dummy_input)
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/jit/_trace.py", line 742, in trace
_module_class,
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/jit/_trace.py", line 928, in trace_module
module = make_module(mod, _module_class, _compilation_unit)
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/jit/_trace.py", line 560, in make_module
return _module_class(mod, _compilation_unit=_compilation_unit)
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/jit/_trace.py", line 1040, in init
submodule, TracedModule, _compilation_unit=None
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/jit/_trace.py", line 560, in make_module
return _module_class(mod, _compilation_unit=_compilation_unit)
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/jit/_trace.py", line 1040, in init
submodule, TracedModule, _compilation_unit=None
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/jit/_trace.py", line 560, in make_module
return _module_class(mod, _compilation_unit=_compilation_unit)
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/jit/_trace.py", line 1040, in init
submodule, TracedModule, _compilation_unit=None
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/jit/_trace.py", line 549, in make_module
elif torch._jit_internal.module_has_exports(mod):
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/_jit_internal.py", line 505, in module_has_exports
item = getattr(mod, name)
File "/home/shanshaojie/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 779, in getattr
type(self).name, name))
torch.nn.modules.module.ModuleAttributeError: 'ConvModule' object has no attribute 'norm'
How to debug the problem? thanks
Installation 说明python版本3.6及以上即可,为什么执行安装的时候却要求>=3.8
可否与torch的新特性fx结合,为量化、剪枝提供新的解决方法。
Thanks for your excellent work. It will definitely help when converting the model from Pytorch to TFLite.
When I try to convert the MoveNet (https://github.com/lee-man/movenet-pytorch) to TFLite using tinynn converter, the log reported some unsupported ops: argmax, expand and gather. Hope you can support these ops later:)
Hi author,
For a certain model, oneshotpruner fails when there's channel padding with following error message:
ERROR (tinynn.graph.modifier) All node's sparsity in one subgraph must be the same
Please let me know if you need the model description file. Thanks.
Hi, I got a problem when I use OneShotChannelPruner() to prune my model. I checked the example for mobilenet, it was fine when we use reshape or view like :
x = x.reshape(-1,2049) or
view_1 = model_14.view(shape_1[0], -1)
However, when I reshape to higher dimensions like: x = x.reshape(-1, batch_size, 1024), I will get error:
AttributeError: 'TraceNode' object has no attribute 'modifier'
Did I do something wrong? How can I solve the problem? Thanks.
提示不支持这个
Hi, I notice that the default value to parameter 'asymmetric' is True, which indicates that the quantization should be done as uint8. But the following codes make me a little confused:
if not self.asymmetric:
sym_fq = torch_q.FakeQuantize.with_args(observer=torch_q.MovingAverageMinMaxObserver, quant_min=0, quant_max=255,
dtype=torch.quint8, qscheme=torch.per_tensor_symmetric, reduce_range=False)
qconfig = torch_q.QConfig(sym_fq, qconfig.weight)
where, even users set the 'asymmetric' to False, the codes still force to use uint8. Any suggestions? Thanks a lot.
您好,当我使用工具转换时,报错:AssertionError: Model is in training model,详细信息如下:
2022-04-08 13:56:40.738033: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
Making model...
Load the model from ../float/SESR_S/model/model_best_rep.pt
Traceback (most recent call last):
File "convert_and_compare.py", line 170, in <module>
main_worker()
File "convert_and_compare.py", line 72, in main_worker
converter.convert()
File "anaconda3/envs/torchenv/lib/python3.7/site-packages/TinyNeuralNetwork-0.1.0.20220407153809-py3.7.egg/tinynn/converter/base.py", line 357, in convert
self.init_lowered_module()
File "anaconda3/envs/torchenv/lib/python3.7/site-packages/TinyNeuralNetwork-0.1.0.20220407153809-py3.7.egg/tinynn/converter/base.py", line 191, in init_lowered_module
), 'Model is in training model'
AssertionError: Model is in training model
请问这个原因是为何,我应该如何 解决?
main_worker(args)
File "post_train_quantization.py", line 78, in main_worker
ptq_model = quantizer.quantize()
File "/home/yaoxinghua/miniconda3/envs/yolo/lib/python3.7/site-packages/TinyNeuralNetwork-0.1.0.20220512170349+d0053782b9ca90a8554d211660f82c7da1e36962-py3.7.egg/tinynn/graph/quantization/quantizer.py", line 194, in quantize
rewritten_model.load_state_dict(torch.load(model_weights_path))
File "/home/yaoxinghua/miniconda3/envs/yolo/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1407, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Stereo_qat:
Missing key(s) in state_dict: "stereo_stem_4_0.weight", "stereo_stem_4_0.bias", "stereo_stem_4_0_1.weight", "stereo_stem_4_0_1.bias", "stereo_cost_agg_conv_agg_1_1_activate.weight", "stereo_cost_agg_conv_agg_1_1_activate.bias", "stereo_cost_agg_conv_agg_1_1_activate.running_mean", "stereo_cost_agg_conv_agg_1_1_activate.running_var".
Unexpected key(s) in state_dict: "stereo_feature_block0_0_0_conv_dw.weight", "stereo_feature_block0_0.0.conv_dw.weight", "stereo_feature_block0_0.0.bn1.weight", "stereo_feature_block0_0.0.bn1.bias", "stereo_feature_block0_0.0.bn1.running_mean", "stereo_feature_block0_0.0.bn1.running_var", "stereo_feature_block0_0.0.bn1.num_batches_tracked", "stereo_feature_block0_0.0.conv_pw.weight", "stereo_feature_block0_0.0.bn2.weight", "stereo_feature_block0_0.0.bn2.bias", "stereo_feature_block0_0.0.bn2.running_mean", "stereo_feature_block0_0.0.bn2.running_var", "stereo_feature_block0_0.0.bn2.num_batches_tracked", "stereo_feature_block0_0_0_conv_dw_1.weight", "stereo_feature_block0_0_1.0.conv_dw.weight", "stereo_feature_block0_0_1.0.bn1.weight", "stereo_feature_block0_0_1.0.bn1.bias", "stereo_feature_block0_0_1.0.bn1.running_mean", "stereo_feature_block0_0_1.0.bn1.running_var", "stereo_feature_block0_0_1.0.bn1.num_batches_tracked", "stereo_feature_block0_0_1.0.conv_pw.weight", "stereo_feature_block0_0_1.0.bn2.weight", "stereo_feature_block0_0_1.0.bn2.bias", "stereo_feature_block0_0_1.0.bn2.running_mean", "stereo_feature_block0_0_1.0.bn2.running_var", "stereo_feature_block0_0_1.0.bn2.num_batches_tracked", "stereo_stem_4_0.conv.weight", "stereo_stem_4_0.bn.weight", "stereo_stem_4_0.bn.bias", "stereo_stem_4_0.bn.running_mean", "stereo_stem_4_0.bn.running_var", "stereo_stem_4_0.bn.num_batches_tracked", "stereo_stem_4_0_1.conv.weight", "stereo_stem_4_0_1.bn.weight", "stereo_stem_4_0_1.bn.bias", "stereo_stem_4_0_1.bn.running_mean", "stereo_stem_4_0_1.bn.running_var", "stereo_stem_4_0_1.bn.num_batches_tracked".
size mismatch for stereo_cost_agg_conv_skip_1_bn.weight: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([960]).
size mismatch for stereo_cost_agg_conv_skip_1_bn.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([960]).
size mismatch for stereo_cost_agg_conv_skip_1_bn.running_mean: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([960]).
size mismatch for stereo_cost_agg_conv_skip_1_bn.running_var: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([960]).
Perform below code
if __name__ == '__main__':
with model_tracer():
model = QfMobileNet()
model.eval()
model.softmax = nn.LogSoftmax(dim=1)
dummy_input = torch.rand((1, 1, 135, 240))
quantizer = QATQuantizer(
model, dummy_input, work_dir=output_dir, config={
'backend': "qnnpack",
'force_overwrite': True,
'asymmetric': False,
'per_tensor': False,
'rewrite_graph': True
}
)
qat_model = quantizer.quantize()
device = get_device()
qat_model.to(device=device)
context = DLContext()
context.device = device
context.train_loader = training_loader
context.val_loader = testing_loader
context.max_epoch = 100
context.criterion = nn.NLLLoss()
context.optimizer = torch.optim.Adam(qat_model.parameters(), lr=0.001)
if isinstance(context.criterion, nn.Module):
context.criterion = context.criterion.to(device=context.device)
qat_model(torch.randn(1, 1, 135, 240).cuda())
with torch.no_grad():
qat_model.eval()
qat_model.cpu()
qat_model.softmax = nn.Softmax(dim=1)
qat_model = torch.quantization.convert(qat_model)
torch.backends.quantized.engine = 'qnnpack'
converter = TFLiteConverter(qat_model, dummy_input, tflite_path=f'{output_dir}/qat.tflite', quantize_target_type='int16', strict_symmetric_check=True)
converter.convert()
Then I got qat.tflite but -128 is at convolution weight which input name is "input.4_te_transform_1_te_transform_2" even though setting asymmetric as False (Range shoud be [-127, 127])
qat.tflite is compressed as qat.zip
I tried examples/pruner/netadapt/netadapt_prune.py and got the following errors:
INFO (tinynn.prune.netadapt_pruner) Global Target/Initial FLOPS: 437178624/582904832
INFO (tinynn.prune.netadapt_pruner) Start iteration 1
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'OneShotChannelPruner.init..'
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/usr/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'OneShotChannelPruner.init..'
INFO (tinynn.prune.netadapt_pruner) Init pool process with cuda id 0
The process then blocks after displaying above messages.
Hi, just wonder if Vit transformer will be supported? Many vision transformer worthy to support such as VIT, Swin-transformer, or some e2e OD like DETR.
Will it plann to support in the future?
Symmetric int16 quantization is now supported by TinyNN. Any chance support fixed point int16 quantization for QAT?
Hi,
I want to apply hidden state on my LSTM model and run QAT. The model can run normally without Quantizer, but will get the error below when I apply QATQuantizer:
ERROR (tinynn.graph.tracer) inputs: ['input_1', 'input_2', 'input_3']
ERROR (tinynn.graph.tracer) forwards: []
ERROR (tinynn.graph.tracer) outputs: []
ERROR (tinynn.graph.tracer) constants: []
The attachment is an example file, thanks.
torch_lstm_test.zip
Hi, my qat training goes well based on your sample code. But the conversion to tflite fails with the following error message:
RuntimeError: Quantized copy only works with contiguous Tensors
Based on the core trace, the error comes from:
converter.convert()
My codes are almost the same with the sample code:
print("Start converting the model to TFLite")
with torch.no_grad():
qat_model.eval()
qat_model.cpu()
# The step below converts the model to an actual quantized model, which uses the quantized kernels.
qat_model = torch.quantization.convert(qat_model)
# When converting quantized models to TFLite, please ensure the quantization backend is QNNPACK.
torch.backends.quantized.engine = 'qnnpack'
# The code section below is used to convert the model to the TFLite format
converter = TFLiteConverter(qat_model, dummy_input, tflite_path='out/qat_model.tflite')
converer.convert()
By the way, I am curious how the trained qat model affect the conversion. Looks like the conversion could be done even without training. Not familiar with that, appreciate your help.Thanks.
First off, thanks for this amazing repo. Im working on a ssd model in pytorch and i want to add post processing (NMS) into the tflite, how can i add it into my model so that it gets translated to tflite's NMS OP. thanks
Thanks for your excellent work. when converting the model from Pytorch to TFLite,it do not supported copy.error follows:
ERROR (tinynn.converter.base) Unsupported ops: aten::copy_
Traceback (most recent call last):
File "export_tf.py", line 25, in
converter.convert()
File "/home/demo/miniconda3/lib/python3.8/site-packages/TinyNeuralNetwork-0.1.0-py3.8.egg/tinynn/converter/base.py", line 269, in convert
raise Exception("Cannot continue due to fatal error")
Exception: Cannot continue due to fatal error
I update to the recent version which supports per-channel QAT. It seems that some incorrect import operations are generated to the model description script, e.g:
import self.float_functional_simple_0
import self.float_functional_simple_1
import self.float_functional_simple_10
import self.float_functional_simple_11
import self.float_functional_simple_12
import self.float_functional_simple_2
import self.float_functional_simple_3
import self.float_functional_simple_4
import self.float_functional_simple_5
import self.float_functional_simple_6
import self.float_functional_simple_7
import self.float_functional_simple_8
import self.float_functional_simple_9
The training then cannot be done due to following error:
import self.float_functional_simple_0
ModuleNotFoundError: No module named 'self'
现在移动端部署接入了TFLite,需要实现Torch到TFLite的转换;建议后端接入MNN, MNN后端性能领先于TFLite, 且目前提供了torchscript -> mnn模型的转换工具。
Hi, a type error happens when I generate the qat model based on a pth model:
TypeError: init() got an unexpected keyword argument 'value'
I check the generated the py file re-written by tinynn:
torch.nn.ZeroPad2d(padding=(0, 1, 0, 1), value=0.0)
There are actually codes similar to above line. It's supposed no 'value' is needed for ZeroPad2d. Any suggestions? Thanks a lot and look forward to your help.
Hi Author,
I need to add some extra output tensors which are used for inference. These tensors are not referenced during training but just for inference after the conversion to tflite. My naive intention is to put some operations in forward as many as possible so as to relief the loading of post processing which has to be implemented by c/c++ code.
For instance, some matrix ops such as reshape/sigmoid/multiply better be done by GPU/NPU instead of with CPU.
I add some logic in forward to implement this requirement and the training goes well but the conversion to tflite fails with following error message:
File "/usr/local/lib/python3.6/dist-packages/torch/nn/quantized/modules/functional_modules.py", line 160, in mul
r = ops.quantized.mul(x, y, scale=self.scale, zero_point=self.zero_point)
RuntimeError: Mul operands should have same data type.
Is there any feasible way or workaround for this scenario?
The script attached. Thanks.
movenet_qat.zip
Hi, this is really great work, thanks!
I am able to convert the LiteHRNet model to TFLite without running into any issues. However, the outputs don't match up.
Here is the output from sending ones through the network. Output is of shape [1,17,96,72].
I am just showing here output[0,0,0]
from both pytorch and tflite:
array([6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 1.8188522e-04, 1.7515068e-04, 1.9644469e-04,
1.6027213e-04, 1.9049855e-04, 1.5419864e-04, 1.2460010e-04,
9.0751186e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05],
dtype=float32)
array([6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
1.1580180e-04, 2.3429818e-04, 3.9018277e-04, 7.7823577e-03,
1.8948119e-02, 2.8559987e-02, 3.3612434e-02, 2.5932681e-02,
1.2074142e-02, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05,
6.4367385e-05, 6.4367385e-05, 6.4367385e-05, 6.4367385e-05],
dtype=float32)
When I convert to tflite via the onnx route, the outputs do match. So my guess is that some of the transpose/reshapes for NHWC is not happening correctly but I am not sure.
Looking for some insight into what would be the best way to debug this?
因为有些模型的输入shape是不固定的,请问转换方法支持转出的TFLite模型是动态输入大小的吗?
Hi Author, there's a runtime error when I do QAT training, which reports:
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/conv.py", line 420, in _conv_forward
self.padding, self.dilation, self.groups)
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same
It's easy to tell from the information that the input and weights do not match as for data type. It's most possible that the model may not be set to cuda but I review related codes and cannot find out the root cause yet.
The original model (non-qat) trains normally
The code slice is as follows:
。。。。。。。。。。。。。。。。。。。。。。。。
device = get_device()
model.to(device=device)
。。。。。。。。。。。。。。。。。。。。。。。。。。
print("Start preparing the model for quantization")
quantizer = QATQuantizer(model, dummy_input, work_dir='out')
qat_model = quantizer.quantize()
print("Start quantization-aware training ===>", device)
qat_model.to(device=device)
context = DLContext()
context.device = device
context.val_loader = torch.utils.data.DataLoader(
Dataset(opt, 'val'),
batch_size=1,
shuffle=False,
num_workers=1,
pin_memory=True
)
context.train_loader = torch.utils.data.DataLoader(
Dataset(opt, 'train'),
batch_size=opt.batch_size,
shuffle=True,
num_workers=opt.num_workers,
pin_memory=True,
drop_last=True
)
context.max_epoch = 50
train(qat_model, context, trainer.train_one_epoch, trainer.validate, qat=True)
As you see, the original model is set to cuda before converting to quant model. And, the quant model is set to cuda before calling train(). I'm just confused what may be the reason for this issue. Any suggestions are appreciated. BTW, you guys are doing an excellent job !
Hi developer,
Thank your great work. For some reason my tflite inference environment is limited to Tensorflow 1.13.2 version. When I converted my model to tflite and inference this tflite file, there is a problem
ValueError: Didn't find op for builtin opcode 'CONV_2D' version '2' Registration failed.
But it is all ok in tf 1.15.5 and output result is perfect. How do I modify converter code to fit into tf ver 1.13.2 ? Thank you!
According to user feedback, the current supported ops are not sufficient for model deployment. This requires a major update of the existing TFLite schema. Currently, we are using the schema of TFLite 2.3.0.
After a quick skimming of the current TF operator list (as of TFLite 2.8.0), if we consider updating it to the latest schema, we will have support for the builtin ops include
We may also add support to the following custom kernels if users choose to build TFLite from source.
I got a QAT int8 per-channel tflite model. To check the accuracy, I compare the inference results between it and the de-quantized onnx model.
I'm not pretty clear about the inference flow of a QAT tflite. Not sure such situation is normal or not. It's supposed to have similar results to that of onnx model.
Attach onnx and tflite model for reference.
test.zip
我这里使用转换的代码如下:
import os
import torch
CURRENT_PATH = os.path.abspath(os.path.dirname(__file__))
print(CURRENT_PATH)
from tinynn.converter import TFLiteConverter
model=torch.jit.load("netG.pt")
model.eval()
print(type(model))
dummy_input = torch.rand((1, 3, 512, 512))
output_path = os.path.join(CURRENT_PATH, 'out', 'mbv1_512.tflite')
converter = TFLiteConverter(model, dummy_input, output_path)
converter.convert()
但是我得到的结果是
Traceback (most recent call last):
File "convert_torch_2tf.py", line 14, in <module>
converter.convert()
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/base.py", line 374, in convert
self.init_operations()
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/base.py", line 339, in init_operations
converter.parse(node, attrs, args, self.common_graph)
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/operators/torch/aten.py", line 198, in parse
inputs = [self.find_or_create_input(i, graph_converter) for i in range(5)]
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/operators/torch/aten.py", line 198, in <listcomp>
inputs = [self.find_or_create_input(i, graph_converter) for i in range(5)]
File "//AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/operators/torch/base.py", line 176, in find_or_create_input
return tfl.Tensor(tensor, name, has_buffer=True, asymmetric=self.asymmetric, q_type=self.q_type)
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/operators/tflite/base.py", line 247, in __init__
assert False, f"unrecognized tensor type {type(tensor).__name__}"
AssertionError: unrecognized tensor type NoneType
我去打印了,我的tensor得到了None,所以报错了,这个是神原因呢
Hi, I check the QAT model based on tinynn and it seems that the QAT is done with per-tensor schema. I am very concerned with less accuracy loss, so I wonder if there's any way to do per-channel QAT. Thanks.
代码如下:
import io
import os
import sys
import torch
sys.path.append('../../')
from tinynn.converter import TFLiteConverter
CURRENT_PATH = os.path.abspath(os.path.dirname(__file__))
def main_worker():
input_size = 80
seq_length = 10
hidden_size = 512
with torch.no_grad():
model = torch.nn.LSTM(input_size=input_size, hidden_size=hidden_size, bidirectional=False, bias=False, batch_first=True).cuda()
model.cpu()
model.eval()
dummy_input = torch.rand((1, 10, input_size))
output_path = os.path.join(CURRENT_PATH, 'out', 'lstm_80.tflite')
# When converting quantized models, please ensure the quantization backend is set.
torch.backends.quantized.engine = 'qnnpack'
# The code section below is used to convert the model to the TFLite format
# If you want perform dynamic quantization on the float models,
# you may pass the following arguments.
# `asymmetric=True, quantize_target_type='int8', hybrid_quantization_from_float=True, hybrid_per_channel=False`
converter = TFLiteConverter(model, dummy_input, output_path)
converter.convert()
if __name__ == '__main__':
main_worker()
请问我这种方式是否正确,如果不正确,请问有其他转换LSTM相关的参考代码吗?谢谢。
Hi, thanks for your excellent work. I've converted my pytorch model that has one input to tflite model successfully.
However, the converter seems not support pytorch models that have multiple inputs yet. Any plan for it?
Hello, I use model_convert function to convert .pth from pytorch to .tflite for outside devices. I find different erros in two models. The model1 has several errors in both abs and ref diff, the model2 has only abs diff.
I doubt whether it is caused by Resize-ops or Odd length of feature map. Model1 has a feature map - 15x43, the model2 has a feature map 15x20.
1, model 1
First bad error raise up after first resizeNearest's transpose
2, model 2
First bad error raise up before first resizeBilinear's transpose
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.