How to visually learn weights from a subset of 24 neurons from the first feature mixer block？ about mixvpr

[amaralibey]

Hello @wpumain,

Assuming that in_dim=400, it is likely that feature maps of size Cx20x20 are being used, which are then flattened to a size of Cx400. If mlp_ratio=1, the linear layer will have 400 outputs, which means it will have 400x400 parameters (excluding biases).
If the objective is to plot the learned parameters (not the activations), it is not necessary to feed an image and plot the outputs. Instead, we can plot the learned weights of the Linear layer.
For instance, the code snippet below shows the size of the parameters in the first Linear layer of the first Mixer block:

print(model.aggregator.mix[0].mix[1].weight.shape) # oututs torch.Size([400, 400])

To plot the weights of a single neuron, which is a line of 400 parameters from the weight matrix, it needs to be resized to 20x20 (remember that each neurone is connected to every feature map) and visualized.

all_neurones = model.aggregator.mix[0].mix[1].weight.reshape(-1, 20, 20).detach().cpu() # 400x20x20
random_idx = random.sample(list(range(400)), 1) # select a random idx from 0,399
random_neurone = all_neurones[random_idx] # 1x20x20
random_neurone = random_neurone.permute(1,2,0).numpy() #20x20x1
plt.imshow(random_neurone, cmap='RdBu', interpolation='bicubic')
plt.colorbar()
plt.axis('off')
plt.show()

[wpumain]

Think you for your help .
Illustration of learned weights from a subset of 24 neurons from the first Feature-Mixer block. Blue color corresponds to positive weights and Red corresponds to negative weights.
What are positive weights?
what are negative weights?

[amaralibey]

Weights refer to the learned parameters of the neuron. If you use the code I shared above you'll get figure similar to the following: https://ibb.co/5MLV4h9

The color bar included in the visualization is self explanatory, blue corresponds to positive weights (parameters) of the neuron, while red corresponds to negative ones.

[wpumain]

Think you for your help .