neurreps / awesome-neural-geometry

A curated collection of resources and research related to the geometry of representations in the brain, deep networks, and beyond

Awesome Neural Geometry

A curated collection of resources and research related to the geometry of representations in the brain, deep networks, and beyond, collaboratively generated on the Symmetry and Geometry in Neural Representations Slack Workspace.

This is a collaborative work-in-progress. Please contribute via PRs!

Join us on Slack!

Contents

• Educational Resources
  • Abstract Algebra + Group Theory
  • Differential Geometry + Lie Groups
  • Information Geometry
  • Topology
  • Geometric Machine Learning
  • Computational Neuroscience
• Datasets
• Software Libraries
• Conferences and Workshops

Educational Resources

Abstract Algebra + Group Theory

Textbooks & Notes

• Group Theory: A Primer (2019)
  Luciano da Fontoura Costa
• Tensors in Computations (2021)
  Lek-Heng Lim
• Aspects of Harmonic Analysis and Representation Theory (2021)
  Gallier & Quaintance
• Basic concepts of representation theory (2013)
  Amritanshu Prasad
• Representation Theory of Finite Groups (2012)
  Bemjamin Steinberg

Courses, Lectures, and Videos

• Essence of Group Theory *Beginner-Friendly
  Mathemaniac
• Abstract Algebra *Beginner-Friendly
  Socratica
• Euler's formula with introductory group theory *Intuition Building
  3blue1brown
• What is a Tensor?
  XylyXylyX
• Representation Theory
  Math Doctor Bob
• Category Theory for AI
  Online Course, October 2022

Differential Geometry + Lie Groups

Textbooks & Notes

• Lie Groups, Lie Algebras, and Representations (2003)
  Brian C. Hall
• Differential Geometry and Lie Groups: A Computational Perspective (2020)
  Gallier & Quaintance
• Introduction to Riemannian Geometry and Geometric Statistics: from basic theory to implementation with Geomstats (2022)
  Nicolas Guigui, Nina Miolane, Xavier Pannec

Courses, Lectures, and Videos

• Geometry and Topology and Symmetry *Beginner-Friendly
  Sean Carroll
• Differential Geometry for Computer Science
  Justin Solomon
• Discrete Differential Geometry
  CMU
• What is a Manifold?
  XylyXylyX
• Manifolds
  Robert Davie
• Lie Groups and Lie Algebras
  XylyXylyX
• Lectures on Geometric Anatomy of Theoretical Physics
  Frederic Schuller
• Weekend with Bernie (Riemann)
  Søren Hauberg @ DTU
• Riemann and Gauss Meet Asimov: A Tutorial on Geometric Methods in Robot Learning, Optimization, and Control
  IROS 2022

Notebooks and Blogposts

• Introduction to Differential Geometry and Machine Learning
  Geomstats Jupyter notebooks
• Differential Geometry for Machine Learning
  Roger Grosse
• Manifolds: A Gentle Introduction
  Brian Keng

Information Geometry

Primers

• The Many Faces of Information Geometry (2022)
  Frank Nielsen

Topology

Courses, Lectures, and Videos

• Computational Algebraic Topology
  Vidit Nanda

• Topological Data Analysis for Machine Learning
  Bastian Rieck

Textbooks, Notes

• Elementary Applied Topology
  Robert Ghrist

Geometric Machine Learning

Textbooks

• Group Invariance Applications in Statistics (1989)
  Morris Eaton
• Group Theoretical Methods in Machine Learning (2008)
  Risi Kondor, PhD Thesis
• Pattern Theory: The Stochastic Analysis of Real-World Signals (2010)
  David Mumford and Agnès Desolneux
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges (2021)
  Michael M. Bronstein, Joan Bruna, Taco Cohen, Petar Veličković
• Equivariant Convolutional Networks (2021)
  Taco Cohen, PhD Thesis
• An Introduction to Optimization on Smooth Manifolds (2022)
  Nicolas Boumal

Courses, Lectures, and Videos

• Geometric Deep Learning (2nd Edition)
  Michael Bronstein, Joan Bruna, Taco Cohen, Petar Veličković @ AMMI
• CSC 2547: Current Topics in Machine Learning Methods in 3D and Geometric Deep Learning (2021)
  Animesh Garg @ University of Toronto
• An Introduction to Group-Equivariant Deep Learning (2022)
  Erik Bekkers @ UvA
• Italian Summer School on Geometric Deep Learning (2022)
  Cristian Bodnar, Michael Bronstein, Francesco Di Giovanni, Pim de Haan, Maurice Weiler
• COMP760: Geometry and Generative Models (2022)
  Joey Bose and Prakash Panangaden @ MILA

Notebooks and Blogposts

• Geometric foundations of Deep Learning
  Michael Bronstein, Joan Bruna, Taco Cohen, and Petar Veličković
• What does 2022 hold for Geometric & Graph ML?
  Michael Bronstein
• Geometric Machine Learning for Shape Analysis with Jupyter Notebooks
  Nina Miolane

Computational Neuroscience

Textbooks

• Introduction to the Theory of Neural Computation (1991)
  John Hertz, Anders Krogh, Richard G Palmer
• Theoretical Neuroscience (2001)
  Peter Dayan
• Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting (2006)
  Eugene M. Izhikevich
• Principles of Neural Design (2015)
  Peter Sterling & Simon Laughlin

Books - General Audience

• Rhythms of the Brain (2006)
  Gyorgy Buzsaki
• Networks of the Brain (2010)
  Olaf Sporns
• Models of the Mind: How Physics, Engineering and Mathematics Have Shaped Our Understanding of the Brain (2021)
  Grace Lindsay

Tutorials

• Generative Models for Neural Data Analysis
  COSYNE Workshops 2023

Courses, Lectures, and Videos

• Neural Computation VS265
  Bruno Olshausen @ UC Berkeley

Datasets

Open-Source Neuroscience Datasets

• OpenNeuro
• NeuroVault
• CRCNS
• NeuroData Without Borders
• Allen Brain Atlas
• Kavli Institute for Systems Neuroscience Grid Cell Database
• The Natural Scenes Dataset

Software Libraries

• Geomstats
  • Computation, statistics, and machine learning on non-Euclidean manifolds
• Giotto TDA
  • Topological Data Analysis
• E3NN
  • E(3)-equivariant neural networks
• equivariant-MLP
  • Construct equivariant multilayer perceptrons for arbitrary matrix groups
• SHTOOLS
  • Python library for computations involving spherical harmonics
• LieConv
  • Generalizing convolutional neural networks for equivariance to Lie groups on arbitrary continuous data
• Open Neuroscience
  • A database of open-source tools and software for neuroscience
• LieTorch
  • Geometric machine learning and Lie analysis tools for PyTorch

Conferences and Workshops

• NeurIPS Workshop on Symmetry and Geometry in Neural Representations (2023)
• ICML Workshop on Topology, Algebra and Geometry in Machine Learning (2023)
• RSS Workshop on Symmetries in Robot Learning (2023)
• NeurIPS Workshop on Symmetry and Geometry in Neural Representations (2022)
• ICML Workshop on Topology, Algebra and Geometry in Machine Learning (2022)
• ICLR Workshop on Geometrical and Topological Representation Learning (2022)
• Workshop on Symmetry, Invariance and Neural Representations @ The Bernstein Conference (2022)

awesome-neural-geometry's Issues

Information geometry ?

Hi, do you think it is a good idea to add some stuff on information geometry? The repository so far focuses on the geometry of neural representation or ways to handle the fact that the data can be on some manifold i.e geometric deep learning. However, information geometry would be talking about the fact that the parameters of the model would lie on a manifold. I can make a PR for this.