imbirik / liestationarykernels

Supplementary code for the paper "Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces"

Stationary Kernels and Gaussian Processes
on Lie Groups and their Homogeneous Spaces

This is a prototypical implementation for the methods described in Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces(part I, part II), a two-part series of papers by I. Azangulov, A. Smolensky, A. Terenin and V. Borovitskiy.

The library features (approximate) computational techniques for heat and Matérn kernels on compact Lie groups, their homogeneous spaces and non-compact symmetric spaces. It allows approximate kernel evaluation and differentiation, with positive semidefiniteness guarantees, and efficient sampling of the corresponding Gaussian process.

Example. Samples from a Gaussian process with heat kernel covariance on the torus $\mathbb{T}^2$, on the real projective plane $\mathrm{RP}^2$ and on the sphere $\mathbb{S}^2$:

Spaces of interest

The following spaces are implemented:

• Special orthogonal group $\mathrm{SO}(n)$ (n-by-n orthogonal matrices of determinant 1).
• Special unitary group $\mathrm{SU}(n)$ (n-by-n unitary matrices of determinant 1).
• Stiefel manifold $\mathrm{V}(n,m)$ (collections of m orthonormal vectors in the n-space), including hypersphere S^n.
• Grassmann manifold $\mathrm{Gr}(n,m)$ (m-dimensional subspaces in the n-space), including projective spaces P^n and oriented Grassmannians.
• Hyperbolic space $\mathbb{H}^n$.
• Symmetric positive-definite matrices $\mathrm{SPD}(n)$.

Showcase

from lie_stationary_kernels.spaces import Grassmannian
from lie_stationary_kernels.spectral_kernel import RandomPhaseKernel
from lie_stationary_kernels.spectral_measure import MaternSpectralMeasure
from lie_stationary_kernels.prior_approximation import RandomPhaseApproximation

# First of all let us choose a space
space = Grassmannian(n, m)
# Then select a spectral measure
measure = MaternSpectralMeasure(space.dim, lengthscale, nu, variance)
# Finally we create kernel and sampler
kernel = RandomPhaseKernel(measure, space)
sampler = RandomPhaseApproximation(kernel, phase_order)
# Create two sets of random points
x = space.rand(10)
y = space.rand(20)
# Then
cov = kernel(x,y) # is 10x20 matrix --- covariance matrix 
sample = sampler(x) # is 10x1 vector --- random realization at x

Correspondence between spaces and kernels/samplers

Kernels:

1. With EigenSumKernel the covariance is computed exactly up to truncation using manifold Fourier features. Works with CompactLieGroup.

2. With RandomPhaseKernel the covariance is computed using generalized random phase Fourier features. Works with CompactLieGroup and СompactHomogeneousSpace.

3. With RandomFourierKernel the covariance is computed using symmetric space random Fourier features. Works with NonCompactSymmetricSpace.

Samplers:

1. RandomPhaseApproximation is used for compact spaces (CompactHomogeneousSpace, CompactLieGroup).

2. RandomFourierApproximation is used for non-compact spaces (NonCompactSymmetricSpace).

Installation and dependencies

1. [Optionally] Create virtual environment.

2. Install PyTorch.

3. [Optionally] To use the sphere and projective space, install SphericalHarmonics following the instructions.

4. Install the library by running

pip install git+https://github.com/imbirik/LieStationaryKernels.git

5. To install in developer mode, clone the repository, enter its directory and run

pip install -e ./

liestationarykernels's Issues

Links to Papers in Readme Are Incorrect

The links to Part I / Part II of the paper in the readme don't resolve to the correct locations.

Question for the parametrization

Thank you for the clarification for question #3. However, I still feel confused. Why this is the right way? What's the difference between this paper and Matern Gaussian processes on Riemannian manifolds(just the continuous and discrete way)? In the file torus.py(in the package GeometricKernels), the parametrization is as follows:

x = B.linspace(0, 2*3.14159, 101)
xx = np.meshgrid(x,x)
X = B.stack(*xx, axis=-1).reshape((-1, 2))

Would you like to point out where do I misunderstand? Or, the codes in the LieStationaryKernels already have 2\pi in function chi(self, x) makes them different? Thank you very much!

How to define kernel on circle

I write codes for the kernel defined on circles. However, the figure is not right. I can not get the result fig. 2 in the paper. Where do I go wrong in the following codes:

import sys
import torch
from lie_stationary_kernels.spaces.torus import Torus
from lie_stationary_kernels.spectral_kernel import RandomSpectralKernel, RandomFourierFeatureKernel, RandomPhaseKernel
from lie_stationary_kernels.spectral_kernel import EigenbasisSumKernel, EigenbasisKernel
from lie_stationary_kernels.prior_approximation import RandomFourierApproximation
from lie_stationary_kernels.prior_approximation import RandomPhaseApproximation
from lie_stationary_kernels.spectral_measure import MaternSpectralMeasure, SqExpSpectralMeasure
import matplotlib.pyplot as plt
import os
import lab as B
import numpy as np

os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE"
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="1"
torch.autograd.set_detect_anomaly(True)
torch.cuda.set_device("cuda:1")
dtype = torch.float64
device = 'cuda' if torch.cuda.is_available() else 'cpu'

order = 100
space = Torus(n=1, order=order)
print(space.dim)

num = 300
theta = B.linspace(0, 2*3.14159, num).reshape(-1,1)

Y = np.sin(theta)
X = np.cos(theta)
Z = np.zeros_like(Y)

lengthscale, nu = 1, 1.0/2
measure = MaternSpectralMeasure(space.dim, lengthscale, nu)
circle_kernel = EigenbasisSumKernel(measure, space)
circle_sampler = RandomPhaseApproximation(circle_kernel)
index = 20

point = theta[index].reshape(1,-1)
print(theta.shape, point.shape)
circle_matrix = circle_kernel(torch.tensor(theta, device=device, dtype=dtype), torch.tensor(point, device=device, dtype=dtype)).detach().cpu().numpy()

off_set = np.min(circle_matrix)
print(off_set)
k_point = (circle_matrix - off_set).reshape(-1, 1)

fig = plt.figure()
ax = fig.gca(projection='3d')
ax.scatter(X, Y, Z)
ax.scatter(X[0], Y[0], Z[0],  c='r', marker = 'D', s=50)
ax.scatter(X, Y, k_point,  c='k', marker = 'o', s=50)
ax.axis('off')
plt.show()

Thank you very much!

'HomogeneousSpace' is not defined

I run the example code gpr_experiments, and it seems that the class HomogeneousSpace is not defined.
Would you like to upload the related codes? Thank you very much!