ikintosh / gabornet

GaborNet

Installation

GaborNet can be installed via pip from Python 3.7 and above:

pip install GaborNet

Getting started

import torch
import torch.nn as nn
from torch.nn import functional as F
from GaborNet import GaborConv2d

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")


class GaborNN(nn.Module):
    def __init__(self):
        super(GaborNN, self).__init__()
        self.g0 = GaborConv2d(in_channels=1, out_channels=96, kernel_size=(11, 11))
        self.c1 = nn.Conv2d(96, 384, (3,3))
        self.fc1 = nn.Linear(384*3*3, 64)
        self.fc2 = nn.Linear(64, 2)

    def forward(self, x):
        x = F.leaky_relu(self.g0(x))
        x = nn.MaxPool2d()(x)
        x = F.leaky_relu(self.c1(x))
        x = nn.MaxPool2d()(x)
        x = x.view(-1, 384*3*3)
        x = F.leaky_relu(self.fc1(x))
        x = self.fc2(x)
        return x

net = GaborNN().to(device)

Original research paper (preprint): https://arxiv.org/abs/1904.13204

This research on deep convolutional neural networks proposes a modified architecture that focuses on improving convergence and reducing training complexity. The filters in the first layer of network are constrained to fit the Gabor function. The parameters of Gabor functions are learnable and updated by standard backpropagation techniques. The proposed architecture was tested on several datasets and outperformed the common convolutional networks

Citation

Please use this bibtex if you want to cite this repository in your publications:

@misc{gabornet,
    author = {Alekseev, Andrey},
    title = {GaborNet: Gabor filters with learnable parameters in deep convolutional
               neural networks},
    year = {2019},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{https://github.com/iKintosh/GaborNet}},
}

gabornet's Issues

Difference between layers

Hello,
Could you please tell me what is the difference between "GaborSmallConv2d", "GaborConv2d" and "GaborCurvedConv2d"? from the math point of view.

Add test cases to check that in train mode params change

GaborNet/GaborNet/GaborLayer.py

Lines 52 to 60 in a104371

	self.theta = Parameter(
	(math.pi / 8)
	* torch.randint(0, 8, (out_channels, in_channels)).type(torch.Tensor),
	requires_grad=True,
	)
	self.sigma = Parameter(math.pi / self.freq, requires_grad=True)
	self.psi = Parameter(
	math.pi * torch.rand(out_channels, in_channels), requires_grad=True
	)

Vectorize kernel creation

Remove loops

GaborNet/GaborNet/GaborLayer.py

Lines 103 to 119 in a104371

	def calculate_weights(self):
	for i in range(self.conv_layer.out_channels):
	for j in range(self.conv_layer.in_channels):
	sigma = self.sigma[i, j].expand_as(self.y)
	freq = self.freq[i, j].expand_as(self.y)
	theta = self.theta[i, j].expand_as(self.y)
	psi = self.psi[i, j].expand_as(self.y)
	rotx = self.x * torch.cos(theta) + self.y * torch.sin(theta)
	roty = -self.x * torch.sin(theta) + self.y * torch.cos(theta)
	g = torch.exp(
	-0.5 * ((rotx ** 2 + roty ** 2) / (sigma + self.delta) ** 2)
	)
	g = g * torch.cos(freq * rotx + psi)
	g = g / (2 * math.pi * sigma ** 2)
	self.conv_layer.weight.data[i, j] = g

Deal with sanity_check code

sanity_check requires heavy computation (probably with GPU) so it cannot be part of test suite. However it is nice to have sanity_check case to be able to test layer performance.

Add tests cases to check that in eval mode params stays the same

GaborNet/GaborNet/GaborLayer.py

Lines 52 to 60 in a104371

	self.theta = Parameter(
	(math.pi / 8)
	* torch.randint(0, 8, (out_channels, in_channels)).type(torch.Tensor),
	requires_grad=True,
	)
	self.sigma = Parameter(math.pi / self.freq, requires_grad=True)
	self.psi = Parameter(
	math.pi * torch.rand(out_channels, in_channels), requires_grad=True
	)

Remove private class inheritance

This should be removed from implementation:

from torch.nn.modules.conv import _ConvNd
from torch.nn.modules.utils import _pair

class GaborConv2d(_ConvNd):
...

Remove codecov label from readme.md

Minor modifications

Hello,
Thank you @iKintosh for providing your code.
I'm currently trying to use your layer in my code (with torch 1.1.0) and I ran into some minor problems

I think the following lines should be updated in your code but I'm not sure so I decided to just point them out here :

1- Line 19 -> padding_mode should be added

super(GaborSmallConv2d, self).__init__(in_channels, out_channels, kernel_size, stride, padding, dilation, False, _pair(0), groups, bias, padding_mode='zeros')

2- Line 24 -> .type(torch.Tensor) should be added to make learnable parameter floating

self.theta = nn.Parameter(0.628*torch.randint(0, 6, (out_channels, in_channels)).type(torch.Tensor))

3- Line 62 -> same as #1

super(GaborConv2d, self).__init__(in_channels, out_channels, kernel_size, stride, padding, dilation, False, _pair(0), groups, bias, padding_mode='zeros')

4- Line 72 -> second parameter for meshgrid should be self.x0-1 not self.x0 (??)

y, x = torch.meshgrid([torch.linspace(-self.x0+1, self.x0, self.kernel_size[0]), torch.linspace(-self.y0+1, self.y0, self.kernel_size[1])])

5- Line 105 -> same as #1

super(GaborCurvedConv2d, self).__init__(in_channels, out_channels, kernel_size, stride, padding, dilation, False, _pair(0), groups, bias, padding_mode='zeros')

A question about making weights fixed in the eval mode

Hello, thanks for your great work!
Can you explain to me why the following code can make weights fixed in the eval mode? What is the function of the parameter self.is_calculated？ Thanks very much!

Make weights fixed in eval mode

Is there any way to make GaborLayer not to calculate weights every time in evaluate or no_grads mode?

I guess I should change something in forward method, but I have no idea what to do.

GaborNet/GaborLayers.py

Lines 71 to 93 in 435e031

	def forward(self, input):
	y, x = torch.meshgrid([torch.linspace(-self.x0+1, self.x0, self.kernel_size[0]), torch.linspace(-self.y0+1, self.y0, self.kernel_size[1])])
	x = x.to(device)
	y = y.to(device)
	weight = torch.empty(self.weight.shape, requires_grad=False).to(device)
	for i in range(self.out_channels):
	for j in range(self.in_channels):
	sigma = self.sigma[i, j].expand_as(y)
	freq = self.freq[i, j].expand_as(y)
	theta = self.theta[i, j].expand_as(y)
	psi = self.psi[i, j].expand_as(y)
	rotx = x * torch.cos(theta) + y * torch.sin(theta)
	roty = -x * torch.sin(theta) + y * torch.cos(theta)
	g = torch.zeros(y.shape)
	g = torch.exp(-0.5 * ((rotx ** 2 + roty ** 2) / (sigma + 1e-3) ** 2))
	g = g * torch.cos(freq * rotx + psi)
	g = g / (2*3.14*sigma**2)
	weight[i, j] = g
	self.weight.data[i, j] = g
	return F.conv2d(input, weight, self.bias, self.stride, self.padding, self.dilation, self.groups)

Remove numbers

Should have proper names:

• 3.14 / 2
• 1.41
• 8
• 0.5
• 1e-3
• 3.14

Questions about calculate_weights()

Hi Andrey, I have a couple of questions about calculate_weights().

1. in line 106-109, why do you need expand_as(self.y)?
2. in line 118, why do you need to divide g by (2 * math.pi * sigma ** 2)? I cannot find this in the original expression of Gabor filters.
   Thank you!

Any available pretrained weights?

Hi guys,

I was wondering if you have any available pretrained weights for the GaborLayer?

Add simple test case to test perfomance

When refactor, the performance (in this case, ability to learn proper image representation) should be tested. So it's necessary to find simple dataset and add test case to train simple CNN on it and monitor accuracy metric.

Add poetry as dependency manager