Understanding the CUDNN_STATUS_NOT_SUPPORTED warning about ultralytics

[gerrylwk]

I encountered a similar error when trying to run YOLOv8 from the official repository:

from ultralytics import YOLO
import cv2
import torch
# torch.backends.cudnn.benchmark = True
# torch.backends.cudnn.enabled=False
# torch.backends.cudnn.deterministic = True
# Load a model
model = YOLO("yolov5su.pt")  # load a pretrained model (recommended for training)

im2 = cv2.imread("test.png")
results = model.predict(source=im2, conf=0.1, half=True, classes=[0])

The fix for me was to run it on benchmark mode OR disabling cudnn (as in the commented out lines). Hope this helps! Credits to ptrblack from official Torch forums

[tripathi-genius]

@gerrylwk thanks for the solution. I faced a similar issue, but it was solved when I used
`from ultralytics import YOLO
import cv2
import torch
torch.backends.cudnn.benchmark = True
torch.backends.cudnn.deterministic = True
model = YOLO("yolov5su.pt") # load a pretrained model (recommended for training)

im2 = cv2.imread("test.png")
results = model.predict(source=im2, conf=0.1, half=True, classes=[0])`

It was on NVIDIA-RTX 3050 GPU. It seems to be a viable solution.

[aysent911]

I added only the lines :

torch.backends.cudnn.benchmark = True
torch.backends.cudnn.deterministic = True

and the warning disappeared. I was tuning resnet50 on NVIDIA-RTX 3050

[github-actions]

👋 Hello @Kaspso, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

Hello! It seems you're encountering some issues with cuDNN when using YOLOv8 for keypoint detection. The CUDNN_STATUS_NOT_SUPPORTED warning sometimes indicates a compatibility issue between your configuration and cuDNN's capabilities, particularly with specific convolution operations that your model might be using.

Even though your software versions (PyTorch, CUDA, and cuDNN) are compatible with each other, the error might be due to specific layer parameters or operations in the model that are not supported by your GPU. Here’s what you could try:

• Verify Compatibility: Ensure that all layer configurations (like kernel sizes, padding, etc.) within your model are supported by cuDNN on your particular GPU.
• Update or Roll-back Software Versions: Although your current versions should work together, sometimes slight differences in update patches might cause unexpected issues. Trying a slightly different combination may help.
• Adjust Model Architecture: It’s also possible that adjusting specific layer parameters might avoid this issue.

If none of these help, please provide more details about your model architecture and any modifications you’ve made to it. This will help in diagnosing the issue better! 😊

[glenn-jocher]

@gerrylwk hey there! Thanks for sharing your solution! 🌟 It's great to see community members helping each other out. Using torch.backends.cudnn.benchmark = True can indeed speed up the runtime by allowing cuDNN to automatically find the most efficient algorithms for your specific configuration. Disabling cuDNN can be a useful fallback when encountering compatibility issues. Both approaches are valid depending on the specific needs and setup. Keep up the great work, and happy coding with YOLOv8! 😊

[glenn-jocher]

Hello @aysent911,

Thank you for sharing your experience! It's great to hear that enabling torch.backends.cudnn.benchmark and setting torch.backends.cudnn.deterministic resolved the warning for you. These settings can indeed help optimize performance and ensure reproducibility:

• torch.backends.cudnn.benchmark = True allows cuDNN to find the most efficient algorithms for your hardware, which can speed up your model training and inference.
• torch.backends.cudnn.deterministic = True ensures that operations are deterministic, which is useful for reproducibility.

Here's a quick summary of your solution for others who might encounter a similar issue:

import torch
from ultralytics import YOLO
import cv2

# Enable cuDNN benchmark and deterministic mode
torch.backends.cudnn.benchmark = True
torch.backends.cudnn.deterministic = True

# Load a pretrained YOLO model
model = YOLO("yolov5su.pt")

# Read an image
im2 = cv2.imread("test.png")

# Run prediction
results = model.predict(source=im2, conf=0.1, half=True, classes=[0])

This approach can be particularly helpful when working with models like ResNet50 on GPUs like the NVIDIA RTX 3050. If anyone else is facing similar issues, give this a try! 😊

Thanks again for contributing to the community, and happy coding!