Getting last layer of the network by removing activation function about ultralytics

[github-actions]

👋 Hello @rohinr7, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@rohinr7 hello!

To remove the activation function from the last layer, you can modify the model's architecture directly. Here's a quick example of how you might do it:

from ultralytics import YOLO

# Load the model
model = YOLO('yolov8n.pt')

# Access the last layer and remove its activation function
model.model.model[-1].activation = nn.Identity()

# Now you can register your forward hook
target_layers = [model.model.model[-3]]
target_layers[0].register_forward_hook(self.save_activation)

This way, the last layer will no longer have an activation function. If you need further assistance, feel free to ask! 😊

Best of luck with your project!

[rohinr7]

Thankyou for the information and i need to confirm that target_layers = [model.model.model[-3]] this a last
layer of network? if not please let me know how to get the last layer of yolov8 network

[glenn-jocher]

@rohinr7 you're welcome! 😊

To clarify, model.model.model[-3] is not the last layer of the network. If you want to access the last layer of the YOLOv8 model, you should use model.model.model[-1]. Here's how you can do it:

# Access the last layer of the network
last_layer = model.model.model[-1]

This will give you the last layer of the YOLOv8 network. If you have any more questions or need further assistance, feel free to ask!

[rohinr7]

hello glenn
Thanks for your help actually i am trying to extract the last layer of yolov8 and remove the activation function and i end up a feature map which i will pass it to my algorithm but i am getting trouble here, as per the image i am getting error when i loop through the target layers below is my code

target_layers = model.model.model[-1]

activations_and_grads = ActivationsAndGradients(model, target_layers,None)
outputs = activations_and_grads(img)
activations_out = activations_and_grads.activations

and here is my class of ActivationsAndGradients

class ActivationsAndGradients:
    def __init__(self, model, target_layers, reshape_transform):
        self.model = model
        self.gradients = []
        self.activations = []
        self.reshape_transform = reshape_transform
        self.handles = []
        for target_layer in target_layers:
            self.handles.append(
                target_layer.register_forward_hook(self.save_activation))
            # Because of https://github.com/pytorch/pytorch/issues/61519,
            # we don't use backward hook to record gradients.
            self.handles.append(
                target_layer.register_forward_hook(self.save_gradient))

    def save_activation(self, module, input, output):
        activation = output

        if self.reshape_transform is not None:
            activation = self.reshape_transform(activation)
        self.activations.append(activation.cpu().detach())

    def save_gradient(self, module, input, output):
        if not hasattr(output, "requires_grad") or not output.requires_grad:
            # You can only register hooks on tensor requires grad.
            return

        # Gradients are computed in reverse order
        def _store_grad(grad):
            if self.reshape_transform is not None:
                grad = self.reshape_transform(grad)
            self.gradients = [grad.cpu().detach()] + self.gradients

        output.register_hook(_store_grad)

    def __call__(self, x):
        self.gradients = []
        self.activations = []
        return self.model(x)

    def release(self):
        for handle in self.handles:
            handle.remove()

Thanks in advance i am glad if i get outbreaking results than any other xai i can add it to ultralytics features

[glenn-jocher]

Hello @rohinr7,

Thank you for sharing the details of your implementation! It looks like you're on the right track. To address the issue you're encountering, it seems like the problem might be with how you're passing the target_layers. Since model.model.model[-1] is a single layer, you should wrap it in a list to iterate over it correctly.

Here's a refined version of your code:

target_layers = [model.model.model[-1]]  # Wrap the last layer in a list

activations_and_grads = ActivationsAndGradients(model, target_layers, None)
outputs = activations_and_grads(img)
activations_out = activations_and_grads.activations

This should resolve the error you're facing. If you have any more questions or need further assistance, feel free to ask! 😊

Best of luck with your XAI implementation!

[rohinr7]

Thankyou I have tried that part this is the error i am getting at consistent

[glenn-jocher]

Hello @rohinr7,

Thank you for the update! It looks like there's an issue with the target_layers setup. Please ensure that target_layers is a list, even if it contains only one layer. Here's a quick fix:

target_layers = [model.model.model[-1]]  # Ensure it's a list

activations_and_grads = ActivationsAndGradients(model, target_layers, None)
outputs = activations_and_grads(img)
activations_out = activations_and_grads.activations

This should resolve the error. If you continue to face issues, please share the specific error message, and we'll be happy to assist further. 😊

Best of luck with your implementation!

[rohinr7]

I am sure it was a list

[glenn-jocher]

Hello @rohinr7,

Thank you for confirming that target_layers is a list. It seems like the issue might be related to how the hooks are being registered or how the model is being called.

Let's ensure that the hooks are correctly set up and that the model is properly invoked:

1. Ensure target_layers is a list:

   target_layers = [model.model.model[-1]]  # Confirm it's a list

2. Check the ActivationsAndGradients class:

   • Ensure that the hooks are correctly registered.
   • Verify that the model is properly called within the __call__ method.

3. Debugging:

   • Add print statements or use a debugger to check if the hooks are being triggered.

Here’s a slightly refined version of your code for clarity:

class ActivationsAndGradients:
    def __init__(self, model, target_layers, reshape_transform):
        self.model = model
        self.gradients = []
        self.activations = []
        self.reshape_transform = reshape_transform
        self.handles = []
        for target_layer in target_layers:
            self.handles.append(
                target_layer.register_forward_hook(self.save_activation))
            self.handles.append(
                target_layer.register_forward_hook(self.save_gradient))

    def save_activation(self, module, input, output):
        activation = output
        if self.reshape_transform is not None:
            activation = self.reshape_transform(activation)
        self.activations.append(activation.cpu().detach())

    def save_gradient(self, module, input, output):
        if not hasattr(output, "requires_grad") or not output.requires_grad:
            return
        def _store_grad(grad):
            if self.reshape_transform is not None:
                grad = self.reshape_transform(grad)
            self.gradients = [grad.cpu().detach()] + self.gradients
        output.register_hook(_store_grad)

    def __call__(self, x):
        self.gradients = []
        self.activations = []
        return self.model(x)

    def release(self):
        for handle in self.handles:
            handle.remove()

# Usage
target_layers = [model.model.model[-1]]  # Ensure it's a list
activations_and_grads = ActivationsAndGradients(model, target_layers, None)
outputs = activations_and_grads(img)
activations_out = activations_and_grads.activations

If the issue persists, please share the specific error message you're encountering, and we'll be happy to assist further. 😊

Best regards and good luck with your implementation!

[rohinr7]

i am getting same error the problem is in the self.activations.append(activation.cpu().detach()) this particular line

[glenn-jocher]

Hello @rohinr7,

Thank you for the update! It looks like the issue is with the activation.cpu().detach() line. This might be due to the activation not being a tensor. Let's add a check to ensure it's a tensor before appending it:

def save_activation(self, module, input, output):
    activation = output
    if self.reshape_transform is not None:
        activation = self.reshape_transform(activation)
    if isinstance(activation, torch.Tensor):
        self.activations.append(activation.cpu().detach())

This should help resolve the issue. If you have any further questions, feel free to ask! 😊

Best regards and good luck with your implementation!