Comments (5)
If you use external-data format, you can replace the data file representing external tensors with new values, as you wish.
Alternatively, you can make the weights as input parameters of the model, and then vary them as you wish for each invocation. However, this will incur a performance penalty (potentially huge) if ort has to do things like move the weight to gpu or transpose them etc. (which will be done once at session creation if the weights are not inputs).
from onnx.
@gramalingam After reading the docs and tinkering with some of those functions, I am still not sure I quite understand the purpose of the external-data format, or if it would be compatible for the onnxruntime API (as opposed to onnx). What is the purpose of the format, and could you provide psuedo code to show how to load a subset of params with onnxruntime?
from onnx.
Yes, onnxruntime also supports the external-data format, which is part of the onnx standard. The external-data format serves a couple of purposes.
First, the protobuf format has a limit of 2GB on the size of a protobuf object (in terms of the size of the serialized representation). Models which exceed this size can exploit the external-data format to get around this limitation.
Second, even if the model size is less than 2GB, weights end up dominating the size of the model representation. Hence, it is convenient and efficient to load these weights only if required. It helps analysis/optimization tools that care about the graph, and not so much about the weights.
from onnx.
Is there a way to specify which parameters in the graph to load weights into? Or this capability doesn't exist yet
from onnx.
In my application I adding an initializer with the AddInitializer
method, and then loading the session via the CreateSessionFromArray
API. I am getting the following initialization error when I call CreateSessionFromArray
[E:onnxruntime:, inference_session.cc:1935 onnxruntime::InferenceSession::Initialize::<lambda_5a23845ba810e30de3b9e7b450415bf5>::operator ()] Exception during initialization: C:\a\_work\1\s\onnxruntime\core\optimizer\initializer.cc:35 onnxruntime::Initializer::Initializer !model_path.IsEmpty() was false. model_path must not be empty. Ensure that a path is provided when the model is created or loaded.
Because I am not supplying a model path (I'm initializing from an array), does this imply the two methods or not compatible?
from onnx.
Related Issues (20)
- Using Pytorch Count of input_names provided during export not matching with session.get_inputs() HOT 1
- ONNX is broken when built with ONNX_USE_LITE_PROTO=OFF (using protobuf-24.4): File already exists in database: onnx/onnx-ml.proto HOT 5
- Schema error trying to create new version of training ops
- ERROR: Could not build wheels for onnx which use PEP 517 and cannot be installed directly HOT 2
- output different between onnx and pytorch HOT 7
- Implement DepthToSpace as a function HOT 2
- Reference implementation of ONNX Reduce sum square is mismatch with ONNX Spec when noop_with_empty_axes == 1 HOT 5
- Clarification of Reshape semantics for attribute 'allowzero' NOT set, zero volume HOT 3
- Change dynamic shape in fixed shape using C++ HOT 1
- The weekly model zoo CI has been failing HOT 12
- Incorrect input names for quantize/dequantize ONNX backend node tests HOT 1
- Fix and test numpy_helper to_array
- How to read onnx and obtain names, imgsz HOT 3
- Model zoo test failures HOT 1
- DequantizeLinear spec clarification: What happens if the subtraction overflows/underflows? HOT 1
- the model for UniqueOp in backend test is not always correct for different input
- A conflict doc abount compatibility between Onnx Version & ML Opset Version HOT 2
- [Shape Inference] Robustness of `ConstantOfShape` operator HOT 2
- Numpy 2.0.0rc2 causes `pytest` on ONNX to fail HOT 2
- How to modify Slice attribute HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onnx.