xilinx / finn-base

Open Source Compiler Framework using ONNX as Frontend and IR

Deprecation notice

This repo (finn-base) is scheduled to be deprecated soon, and should be replaced in any dependent projects with the new QONNX (https://github.com/fastmachinelearning/qonnx) repo. QONNX is nearly a drop-in replacement for finn-base, only a namespace change is needed (e.g. import finn.core.modelwrapper becomes import qonnx.core.modelwrapper).

Core Components for Quantized Neural Network Inference

finn-base is part of the FINN project and provides the core infrastructure for the FINN compiler, including:

• wrapper around ONNX models for easier manipulation
• infrastructure for applying transformation and analysis passes on ONNX graphs
• infrastructure for defining and executing custom ONNX ops (for verification and code generation)
• extensions to ONNX models using annotations, including few-bit data types, sparsity and data layout specifiers
• several transformation passes, including topological sorting, constant folding and convolution lowering
• several custom ops including im2col and multi-thresholding for quantized activations
• several utility functions, including packing for few-bit integers

Installation

finn-base can be installed via pip by following these instructions.

Documentation

You can view the documentation on readthedocs or build them locally using ./run-docker.sh docs.

Community

We have a gitter channel where you can ask questions. You can use the GitHub issue tracker to report bugs, but please don't file issues to ask questions as this is better handled in the gitter channel.

We also heartily welcome contributions to the project, please check out the contribution guidelines and the list of open issues. Don't hesitate to get in touch over Gitter to discuss your ideas.

finn-base's Issues

Can't unset tensor shape, with set_tensor_shape

It is currently not possible to easily remove the shape from a given tensor.

The attached .zip file contains an QONNX model, which has unset tensors:
CNV_2W2A.zip

Running the following:

from finn.core.modelwrapper import ModelWrapper
model = ModelWrapper("CNV_2W2A.onnx")
print(model.get_tensor_shape("55"))
model.set_tensor_shape("55", None)
print(model.get_tensor_shape("55"))

produces:

None
[]

When it should produce:

None
None

Unsetting the tensor shape is in particular useful, when one has changed some up-stream tensor and wants to run shape inference again for the rest of the down-stream tensors. Because as long as the down-stream tensors have wrong or out-dated shapes the shape inference will always fail.

This would require changes to the set_tensor_shape function here:

finn-base/src/finn/core/modelwrapper.py

Line 188 in 92f0019

def set_tensor_datatype(self, tensor_name, datatype):

Such that the function will not append a new tensor_value_info, when the argument tensor_shape is None.

fix_float64 not passed if cleanup is run

In the transform member function of ModelWrapper:

finn-base/src/finn/core/modelwrapper.py

Lines 122 to 146 in 2f10c0c

	def transform(
	self, transformation, make_deepcopy=True, cleanup=True, fix_float64=True
	):
	"""Applies given Transformation repeatedly until no more changes can be made
	and returns a transformed ModelWrapper instance.
	- make_deepcopy : operates on a new (deep)copy of model.
	- fix_float64 : DoubleToSingleFloat correction before starting
	- cleanup : execute cleanup transformations before returning
	"""
	transformed_model = self
	if make_deepcopy:
	transformed_model = copy.deepcopy(self)
	if fix_float64:
	(transformed_model, model_was_changed) = DoubleToSingleFloat().apply(
	transformed_model
	)
	model_was_changed = True
	while model_was_changed:
	(transformed_model, model_was_changed) = transformation.apply(
	transformed_model
	)
	if cleanup:
	transformed_model.cleanup()
	return transformed_model

if cleanup=True, then the transform is called with fix_float64=True during the cleanup, no matter what the transform was originally called with. I think either fix_float64 should default to False, or fix_float64 should be passed to the cleanup.

Issue with inferring shapes in example model

If I create an onnx file with this sample script and input.txt:

import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np

# Define simple MLP architecture
class MLP(nn.Module):

    def __init__(self):
        super(MLP, self).__init__()
        # Two layer MLP, ingesting a single frame of BLM data
        self.layer1 = nn.Linear(259, 128)
        self.layer2 = nn.Linear(128, 259*2)

    def forward(self, x):
        x = F.relu(self.layer1(x))
        x = torch.sigmoid(self.layer2(x))
        return x

# Training function
def run_inference() -> None:

    # Instantiate the MLP model
    model = MLP()
    # Fix random seed
    np.random.seed(0)

    # Generate weight tensors
    w1 = torch.tensor(np.random.normal(loc=0, scale=0.1, size=(128, 259)).astype(np.single))
    b1 = torch.tensor(np.random.normal(loc=0, scale=0.1, size=128).astype(np.single))

    w2 = torch.tensor(np.random.normal(loc=0, scale=0.1, size=(259*2, 128)).astype(np.single))
    b2 = torch.tensor(np.random.normal(loc=0, scale=0.1, size=259*2).astype(np.single))

    # Single inference step
    with torch.no_grad():

        # Load the fixed weights
        model.layer1.weight = nn.parameter.Parameter(w1)
        model.layer1.bias = nn.parameter.Parameter(b1)

        model.layer2.weight = nn.parameter.Parameter(w2)
        model.layer2.bias = nn.parameter.Parameter(b2)

        # Load the input data and add a batch dimension
        input_data = torch.from_numpy(np.loadtxt('input.txt', dtype=np.single)).unsqueeze(0)

        # Inference
        out = model(input_data)

        # Save in ONNX format
        torch.onnx.export(model,  # model being run
                          input_data,  # model input (or a tuple for multiple inputs)
                          "MLP.onnx")

if __name__ == '__main__':
    run_inference()

(the produced ONNX file is available at: https://drive.google.com/file/d/1wt6ub3cChvPD-XM4-7keuTy5dC5wdVZk/view?usp=sharing)

it seems that infer_shapes from the cleaning fails:

(fastml) mac-137349:validation jmitrevs$ qonnx-cleanup MLP.onnx 
(fastml) mac-137349:validation jmitrevs$ qonnx-exec MLP_clean.onnx 
Traceback (most recent call last):
  File "/Users/jmitrevs/fastml/bin/qonnx-exec", line 33, in <module>
    sys.exit(load_entry_point('qonnx', 'console_scripts', 'qonnx-exec')())
  File "/Users/jmitrevs/work/qonnx/src/qonnx/util/exec_qonnx.py", line 43, in main
    clize.run(exec_qonnx)
  File "/Users/jmitrevs/fastml/lib/python3.9/site-packages/sigtools/modifiers.py", line 158, in __call__
    return self.func(*args, **kwargs)
  File "/Users/jmitrevs/fastml/lib/python3.9/site-packages/clize/runner.py", line 363, in run
    ret = cli(*args)
  File "/Users/jmitrevs/fastml/lib/python3.9/site-packages/clize/runner.py", line 220, in __call__
    return func(*posargs, **kwargs)
  File "/Users/jmitrevs/work/qonnx/src/qonnx/util/exec_qonnx.py", line 35, in exec_qonnx
    odict = execute_onnx(model, idict)
  File "/Users/jmitrevs/work/finn-base/src/finn/core/onnx_exec.py", line 147, in execute_onnx
    raise Exception("Found unspecified tensor shapes, try infer_shapes")
Exception: Found unspecified tensor shapes, try infer_shapes

The problem is that model.get_tensor_shape('Gemm_0_param0') returns []. I do not understand the behavior.

Resolve fixed-point datatypes for Quant nodes

When querying the output datatype of Quant nodes, the current implementation only supports identifying quantization cases the corespond to integers. Everything else is set to FLOAT32.

In cases where the scale is global and is a negative power of two, we can resolve the datatype as FIXED<> instead.
See also relevant discussion: #52 (comment)_

Potential circular import problem

As documented here:
#2 (comment)

Latest protobuf version not compatible with current onnx version

finn-base[onnx] is pinned to use onnx==1.7.0, which is not compatible with protobuf>3.20. This is solved in onnx==1.12 by introducing an upper constraint on the protobuf version (onnx/onnx#4222)

Possible solutions would be:

1. Upgrade onnx to >= 1.12
2. Add protobuf<3.21 as a direct dependency

I would be happy to create a PR to implement either of these

Cannot retrieve ONNX model via URL

Problem

Trying to retrieve an ONNX model via an URL fails. The returned .onnx model is not complete and when trying to wrap, an error message is returned saying:
google.protobuf.message.DecodeError: Protobuf decoding consumed too few bytes: 84 out of 184640.

Specifically:

import urllib.request as ureq
from finn.core.modelwrapper import ModelWrapper

download_url = "https://github.com/onnx/models/tree/main/vision/classification"
download_url += "/shufflenet/model/shufflenet-9.onnx"
export_onnx_path = download_url.split("/")[-1]
ureq.urlretrieve(download_url, export_onnx_path)

model = ModelWrapper(export_onnx_path)

Desired behaviour

To be able to download the complete ONNX model without HTML markup.