Comments (10)
peterjc123
commented on May 2, 2024
1
在 3a91b98 中已经支持了affine=False的BatchNorm
from tinyneuralnetwork.
TomatoBoy90
commented on May 2, 2024
我使用的pip环境如下：
albumentations 1.1.0
certifi 2020.6.20
cycler 0.11.0
dataclasses 0.8
decorator 4.4.2
dill 0.3.4
dominate 2.6.0
flatbuffers 2.0
futures3 1.0.0
igraph 0.9.10
imageio 2.15.0
joblib 1.1.0
kiwisolver 1.3.1
matplotlib 3.3.4
networkx 2.5.1
numpy 1.19.5
opencv-python 4.5.5.64
opencv-python-headless 4.5.5.64
Pillow 8.4.0
pip 21.2.2
pyparsing 3.0.8
python-dateutil 2.8.2
python-igraph 0.9.10
PyWavelets 1.1.1
PyYAML 6.0
qudida 0.0.4
ruamel.yaml 0.17.21
ruamel.yaml.clib 0.2.6
scikit-image 0.17.2
scikit-learn 0.24.2
scipy 1.5.4
setuptools 58.0.4
six 1.16.0
texttable 1.6.4
threadpoolctl 3.1.0
tifffile 2020.9.3
TinyNeuralNetwork 0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd
torch 1.7.1
torchaudio 0.7.2
torchvision 0.8.2
typing_extensions 4.1.1
wheel 0.37.1
系统使用的Ubuntu18.04
from tinyneuralnetwork.
peterjc123
commented on May 2, 2024
@TomatoBoy90 看起来是BatchNorm算子的问题，能看下他的在原始模型里的定义吗？
from tinyneuralnetwork.
peterjc123
commented on May 2, 2024
可能是因为affine=False？
from tinyneuralnetwork.
TomatoBoy90
commented on May 2, 2024
@TomatoBoy90 看起来是BatchNorm算子的问题，能看下他的在原始模型里的定义吗？
原模型使用过普归一化，但是我使用torch里面的remove普归一化函数去除的，我使用传统的方式，是可以成功转tflite的
from tinyneuralnetwork.
TomatoBoy90
commented on May 2, 2024
@TomatoBoy90 看起来是BatchNorm算子的问题，能看下他的在原始模型里的定义吗？
我没有在模型使用的BN
from tinyneuralnetwork.
TomatoBoy90
commented on May 2, 2024
我打印过具体的tensor，tensor刚开始是可以有数据的，后面遇到None就报错了
tensor is tensor([[[[ 0.0660, 0.0142, -0.0214],
[ 0.0055, 0.0113, -0.0357],
[-0.0785, -0.0038, 0.0184]],
[[ 0.0550, 0.0137, -0.0039],
[ 0.0151, 0.0360, -0.0105],
[-0.0546, 0.0426, 0.0536]],
[[ 0.0210, -0.0211, -0.0178],
[-0.0087, -0.0082, -0.0456],
[-0.0566, 0.0130, 0.0288]]],
[[[-0.0439, -0.0613, -0.0175],
[-0.0062, 0.0117, -0.0006],
[ 0.0417, 0.0485, -0.0038]],
[[-0.0068, -0.0395, 0.0160],
[-0.0243, -0.0186, -0.0021],
[ 0.0052, 0.0156, -0.0186]],
[[ 0.0063, -0.0410, 0.0099],
[ 0.0042, 0.0125, 0.0019],
[ 0.0327, 0.0539, -0.0110]]],
[[[-0.0374, 0.0024, 0.0197],
[-0.0650, -0.0045, 0.0747],
[-0.0406, -0.0293, 0.0173]],
[[ 0.0205, 0.0099, -0.0254],
[ 0.0330, 0.0190, -0.0214],
[ 0.0279, 0.0309, 0.0028]],
[[ 0.0226, -0.0039, -0.0418],
[ 0.0315, -0.0212, -0.0886],
[ 0.0217, 0.0161, -0.0215]]],
...,
[[[-0.0137, 0.0132, 0.0013],
[-0.0180, -0.0097, -0.0091],
[ 0.0291, -0.0961, 0.0297]],
[[ 0.0038, 0.0397, 0.0020],
[ 0.0028, 0.0405, -0.0014],
[ 0.0537, -0.0219, 0.0578]],
[[-0.0065, 0.0204, -0.0420],
[-0.0112, 0.0139, -0.0491],
[ 0.0346, -0.0352, -0.0068]]],
[[[ 0.0479, 0.0407, 0.0003],
[ 0.0295, 0.0628, 0.0389],
[-0.0010, 0.0407, -0.0131]],
[[-0.0284, -0.0176, -0.0557],
[-0.0325, 0.0136, -0.0034],
[-0.0410, 0.0148, -0.0311]],
[[-0.0185, 0.0021, -0.0382],
[-0.0281, 0.0265, 0.0006],
[-0.0228, 0.0249, -0.0270]]],
[[[-0.0055, 0.0141, -0.0324],
[ 0.0164, 0.0203, 0.0480],
[-0.0108, -0.0273, -0.0149]],
[[-0.0164, 0.0132, -0.0279],
[-0.0019, -0.0154, 0.0528],
[-0.0406, -0.0622, -0.0205]],
[[-0.0439, 0.0156, -0.0355],
[ 0.0025, 0.0103, 0.0461],
[-0.0134, 0.0497, 0.0078]]]], requires_grad=True)
tensor is tensor([-0.0006, -0.0020, 0.0047, ..., -0.0010, 0.0026, 0.0035],
requires_grad=True)
tensor is tensor([[[[ 0.0274, -0.1042, -0.0265, ..., -0.0545, -0.0163, -0.0254],
[ 0.0655, 0.0486, 0.0454, ..., 0.0186, -0.0080, 0.1129],
[ 0.0204, 0.0805, 0.0807, ..., 0.0177, -0.0666, 0.0269],
...,
[ 0.0293, -0.0231, -0.0158, ..., 0.1018, 0.0187, 0.0289],
[ 0.0419, 0.0538, 0.0296, ..., -0.0192, 0.0124, 0.0051],
[-0.0566, 0.0155, 0.0468, ..., -0.0272, -0.0075, 0.1142]],
[[ 0.0725, 0.0699, 0.0151, ..., 0.0352, 0.0398, 0.0838],
[-0.0375, -0.0885, -0.0483, ..., -0.0349, -0.0492, -0.0373],
[-0.0626, -0.0412, -0.0184, ..., 0.0552, 0.0326, -0.0373],
...,
[ 0.0347, 0.0188, 0.0361, ..., -0.0394, 0.0013, 0.0147],
[-0.0544, -0.0535, 0.0642, ..., 0.0204, 0.0190, -0.0019],
[-0.1132, -0.1318, -0.1302, ..., -0.0472, -0.0907, -0.1388]],
[[-0.0061, 0.0032, -0.0152, ..., 0.0331, 0.0102, -0.0361],
[-0.0122, 0.0533, -0.0498, ..., -0.0125, 0.0478, 0.0302],
[ 0.0143, 0.1179, -0.0315, ..., -0.0638, -0.0288, 0.0486],
...,
[ 0.0223, 0.0208, -0.1188, ..., -0.0151, 0.0319, 0.0532],
[-0.0235, 0.0325, -0.0692, ..., -0.0300, 0.0111, 0.0494],
[-0.0433, -0.0019, -0.0462, ..., -0.0869, -0.0196, 0.0336]],
...,
[[-0.0720, 0.0216, 0.0302, ..., 0.0310, 0.0152, -0.0673],
[ 0.0319, 0.0427, 0.0009, ..., -0.0575, 0.0566, -0.0008],
[ 0.0258, 0.0724, 0.0020, ..., -0.0232, 0.0251, 0.0566],
...,
[-0.0979, 0.0649, -0.1222, ..., 0.0055, 0.0056, 0.0150],
[ 0.0148, -0.0182, -0.0729, ..., 0.0550, 0.0084, 0.0358],
[ 0.0062, -0.0178, -0.0080, ..., -0.0321, -0.0073, 0.0704]],
[[ 0.1088, 0.0724, 0.0202, ..., 0.0505, 0.0514, 0.0852],
[ 0.0162, -0.0422, -0.0452, ..., -0.0058, -0.0125, 0.1181],
[ 0.0297, 0.0087, 0.0050, ..., 0.0121, -0.0268, 0.0564],
...,
[ 0.0486, 0.0373, 0.0309, ..., -0.0227, -0.0884, 0.0819],
[ 0.0683, 0.0845, 0.0782, ..., -0.0173, -0.0235, 0.0907],
[ 0.0187, 0.0472, 0.0914, ..., 0.0245, -0.0150, 0.0402]],
[[ 0.1114, 0.0304, 0.0534, ..., 0.0402, 0.0284, -0.0531],
[-0.0515, -0.1510, -0.1044, ..., 0.0019, -0.0045, -0.0310],
[ 0.0543, -0.0062, -0.0260, ..., 0.0162, 0.0150, -0.0474],
...,
[ 0.0240, -0.0100, -0.0766, ..., -0.0279, -0.0352, -0.0567],
[ 0.0003, 0.0284, -0.0558, ..., -0.0017, -0.0681, -0.0629],
[ 0.0708, 0.0364, 0.0630, ..., 0.0805, 0.0407, 0.0171]]]])
tensor is None
Traceback (most recent call last):
File "convert_torch_2tf.py", line 14, in <module>
converter.convert()
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/base.py", line 374, in convert
self.init_operations()
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/base.py", line 339, in init_operations
converter.parse(node, attrs, args, self.common_graph)
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/operators/torch/aten.py", line 198, in parse
inputs = [self.find_or_create_input(i, graph_converter) for i in range(5)]
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/operators/torch/aten.py", line 198, in <listcomp>
inputs = [self.find_or_create_input(i, graph_converter) for i in range(5)]
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/operators/torch/base.py", line 176, in find_or_create_input
return tfl.Tensor(tensor, name, has_buffer=True, asymmetric=self.asymmetric, q_type=self.q_type)
File "/AN/envs/TinyNeuralNetwork/lib/python3.6/site-packages/TinyNeuralNetwork-0.1.0.20220509160335+35e27d3e883f4b7829a994ed2563a438d1c90efd-py3.6.egg/tinynn/converter/operators/tflite/base.py", line 247, in __init__
assert False, f"unrecognized tensor type {type(tensor).__name__}"
AssertionError: unrecognized tensor type NoneType
from tinyneuralnetwork.
peterjc123
commented on May 2, 2024
但是从代码来看，198行就是bn啊
试一下设置环境变量LOGLEVEL=DEBUG，然后拉一下完整日志
from tinyneuralnetwork.
peterjc123
commented on May 2, 2024
@TomatoBoy90 看起来是BatchNorm算子的问题，能看下他的在原始模型里的定义吗？
原模型使用过普归一化，但是我使用torch里面的remove普归一化函数去除的，我使用传统的方式，是可以成功转tflite的
你是说torch.nn.utils.remove_weight_norm吗？我感觉你用的可能不对，导致没完全去掉。再说了，传统方式能正常转不代表你的weight norm就移除了啊。传统方式支持affine=False的BatchNorm，但我们目前不支持（虽然支持起来也不难，但是最好还是能查清楚为什么会有这个）
from tinyneuralnetwork.
TomatoBoy90
commented on May 2, 2024
在 3a91b98 中已经支持了affine=False的BatchNorm
-牛逼
from tinyneuralnetwork.
Related Issues (20)
- Meet Detailed error: Tensor-likes are not close! using TFLiteConverter HOT 2
- [Converter] Need transpose optimization HOT 2
- Float model failed to convert to TFLite
- [converter] map gather(+reshape) ops with seperate consecutive indices to split(unpack) ops
- tinynn.converter module not found! HOT 2
- [CI] several tests for modifier failed
- Whether to support pytorch to keras HOT 1
- TransposeConv wrong shape? HOT 15
- change input to INT8 after converting to tflite HOT 2
- [converter] implement torch's `aten::scaled_dot_product_attention` operator HOT 2
- Request: clamp would be more efficient to go to Bounded Relu than Maximum + Minimum HOT 3
- Do not support PReLU module? HOT 5
- torch.max not working HOT 2
- OneShotChannelPruner results in the miss of some operators HOT 4
- KeyError when executing quantization HOT 5
- PyTorch 转 TFLite 使用 int8 量化 HOT 4
- Does tinynn support following int16 quantization? HOT 1
- jit.trace succeed but tinynn tracer failed HOT 1
- It became larger after converting to tflite model HOT 4
- how to do Post-training integer quantization with int16 activation HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tinyneuralnetwork.