lucidrains Goto Github PK

followers: 30.4K following: 0.0 repos: 282.0 gists: 27.0

Name: Phil Wang

Type: User

Bio: Working with Attention. It's all we need

Location: San Francisco

Blog: lucidrains.github.io

Phil Wang's Projects

g-mlp-pytorch

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

gated-state-spaces-pytorch

Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch

gateloop-transformer

Implementation of GateLoop Transformer in Pytorch and Jax

geometric-vector-perceptron

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

gigagan-pytorch

Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs

global-self-attention-network

A Pytorch implementation of Global Self-Attention Network, a fully-attention backbone for vision tasks

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates concepts from neural fields, top-down-bottom-up processing, and attention (consensus between columns), for emergent part-whole heirarchies from data

gradnorm-pytorch

A practical implementation of GradNorm, Gradient Normalization for Adaptive Loss Balancing, in Pytorch

graph-transformer-pytorch

Implementation of Graph Transformer in Pytorch, for potential use in replicating Alphafold2

h-transformer-1d

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning

halonet-pytorch

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

hamburger-pytorch

Pytorch implementation of the hamburger module from the ICLR 2021 paper "Is Attention Better Than Matrix Decomposition"

holodeck-pytorch

Implementation of a holodeck, written in Pytorch

hourglass-transformer-pytorch

Implementation of Hourglass Transformer, in Pytorch, from Google and OpenAI

htm-pytorch

Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

hyena-dna

Fork of HyenaDNA, a long-range genomic foundation model built with Hyena

imagen-pytorch

Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch

insertion-deletion-ddpm

Implementation of Insertion-deletion Denoising Diffusion Probabilistic Models

invariant-point-attention

Implementation of Invariant Point Attention, used for coordinate refinement in the structure module of Alphafold2, as a standalone Pytorch module

isab-pytorch

An implementation of (Induced) Set Attention Block, from the Set Transformers paper

itransformer

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group

ittr-pytorch

Implementation of the Hybrid Perception Block and Dual-Pruned Self-Attention block from the ITTR paper for Image to Image Translation using Transformers