Comments (5)
@arkerman hello!
It's quite common to observe slight discrepancies in model performance when converting from a .pt
file to a .engine
file due to difference in optimization and precision handling between the two formats. To minimize such discrepancies, ensure that the precision during conversion matches (e.g., FP32 in both cases) and that all optimization settings are similar.
If the performance difference is significant and these adjustments don't help, consider reviewing the conversion logs for any warnings or error messages that could indicate what might be going wrong during the process.
Happy coding! 😊
from yolov5.
A warning will be reported during the conversion process:
"Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32."
So how should I convert the model to ensure accurate alignment?
from yolov5.
Hi @arkerman!
The warning you're seeing indicates a mismatch in data types during the conversion process from ONNX to TensorRT, where TensorRT does not support INT64 weights. To help ensure better precision and compatibility, you could manually cast the weights from INT64 to FP32 before converting to TensorRT. This is generally more aligned with TensorRT's capabilities than INT32 and helps minimize potential loss of information. Here's a quick code snippet to adjust the data types in the ONNX model:
import onnx
from onnx import numpy_helper
# Load your ONNX model
model = onnx.load('model.onnx')
# Iterate through each initializer (weight)
for initializer in model.graph.initializer:
data = numpy_helper.to_array(initializer)
if data.dtype == np.int64:
# Cast INT64 to FP32
data = data.astype(np.float32)
# Replace the initializer with the new data
new_initializer = numpy_helper.from_array(data, initializer.name)
model.graph.initializer.remove(initializer)
model.graph.initializer.append(new_initializer)
# Save the modified model
onnx.save(model, 'modified_model.onnx')
This snippet converts INT64 weights to FP32, which might help with your conversion process! 😊 Happy coding!
from yolov5.
@glenn-jocher Thanks for your help!
But it seems that the code snipped is not work.
It raised an error : "onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : Load model from modified_yolov5s.onnx failed:This is an invalid model. Type Error: Type 'tensor(float)' of input parameter (onnx::Reshape_468) of operator (Reshape) in node (Reshape_237) is invalid."
from yolov5.
Hi @arkerman!
It looks like the model is expecting a different data type for certain operations. Instead of converting all weights to FP32 indiscriminately, you might need to specifically target only those tensors that are compatible with such a conversion. Here's a refined approach to ensure you only adjust the necessary tensors:
import onnx
from onnx import numpy_helper
# Load your ONNX model
model = onnx.load('model.onnx')
# Iterate through each initializer (weight)
for initializer in model.graph.initializer:
data = numpy_helper.to_array(initializer)
if data.dtype == np.int64 and data.ndim == 0: # Target scalar int64 weights
data = data.astype(np.float32)
new_initializer = numpy_helper.from_array(data, initializer.name)
model.graph.initializer.remove(initializer)
model.graph.initializer.append(new_initializer)
# Save the modified model
onnx.save(model, 'modified_model.onnx')
This code now checks if the tensor is a scalar (0-dimensional) before converting, which might help avoid the type error you encountered. Give it a try and let us know how it goes! 😊
from yolov5.
Related Issues (20)
- cls loss HOT 1
- Problem with training for a single class HOT 4
- Issue when try to validate openvino format model HOT 3
- Is there a problem with the way I fine-tuned the YOLOv5? HOT 3
- No module named 'models' HOT 2
- Roc curve /part 2 HOT 1
- REQUIREMENTS.TXT FILE ERROR WITHIN YOLOV5 HOT 2
- Custom object detection by retaining the original classes of yolo HOT 5
- Is yolov5 sensitive to the size of defects and what structural improvements are needed to increase its sensitivity to defects? HOT 5
- Inconsistency issue with single_cls functionality and dataset class count HOT 3
- A minor query about the image channel number check using `im.shape[0] < 5` HOT 5
- Questions about mosaic and affine transformation data augmentation. HOT 6
- Does YOLO perform object detection on jp2 image format? HOT 2
- Parameter performance indicators HOT 5
- How to reduce the size of best.pt HOT 2
- Confusion Matrix HOT 6
- 🚀 Feature Request: Simplified Method for Changing Label Names in YOLOv5 Model HOT 2
- where is yolov5 v7.0 --trian in export.py? HOT 2
- MESSES MY SYSTEM HOT 4
- Per Detection class accuracy on validation set HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yolov5.