lucidrains Goto Github PK

followers: 30.4K following: 0.0 repos: 282.0 gists: 27.0

Name: Phil Wang

Type: User

Bio: Working with Attention. It's all we need

Location: San Francisco

Blog: lucidrains.github.io

Phil Wang's Projects

einops

Deep learning operations reinvented (for pytorch, tensorflow, jax and others)

einops-exts

Implementation of some personal helper functions for Einops, my most favorite tensor manipulation library ❤️

electra-pytorch

A simple and working implementation of Electra, the fastest way to pretrain language models from scratch, in Pytorch

ema-pytorch

A simple way to keep track of an Exponential Moving Average (EMA) version of your pytorch model

en-transformer

Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network

enformer-pytorch

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

enformer-tensorflow-sonnet-training-script

The full training script for Enformer - Tensorflow Sonnet

equiformer-diffusion

Implementation of Denoising Diffusion for protein design, but using the new Equiformer (successor to SE3 Transformers) with some additional improvements

equiformer-pytorch

Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and adopted for use by EquiFold for protein folding

equinox

Callable PyTrees and filtered JIT/grad transformations => neural networks in JAX.

esbn-pytorch

Usable implementation of Emerging Symbol Binding Network (ESBN), in Pytorch

esbn-transformer

An attempt to merge ESBN with Transformers, to endow Transformers with the ability to emergently bind symbols

etsformer-pytorch

Implementation of ETSformer, state of the art time-series Transformer, in Pytorch

fast-transformer-pytorch

Implementation of Fast Transformer in Pytorch

feedback-transformer-pytorch

Implementation of Feedback Transformer in Pytorch

ffcv

FFCV: Fast Forward Computer Vision (and other ML workloads!)

flamingo-pytorch

Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

flash-attention

Fast and memory-efficient exact attention

flash-attention-jax

Implementation of Flash Attention in Jax

flash-cosine-sim-attention

Implementation of fused cosine similarity attention in the same style as Flash Attention

flash-pytorch

Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"

flexible-diffusion-modeling-videos-pytorch

Implementation of the video diffusion model and training scheme presented in the paper, Flexible Diffusion Modeling of Long Videos, in Pytorch

g-mlp-pytorch

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

gated-state-spaces-pytorch

Implementation of Gated State Spaces, from the paper "Long Range Language Modeling via Gated State Spaces", in Pytorch

geometric-vector-perceptron

Implementation of Geometric Vector Perceptron, a simple circuit for 3d rotation equivariance for learning over large biomolecules, in Pytorch. Idea proposed and accepted at ICLR 2021

gigagan-pytorch

Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs

global-self-attention-network

A Pytorch implementation of Global Self-Attention Network, a fully-attention backbone for vision tasks

An attempt at the implementation of Glom, Geoffrey Hinton's new idea that integrates concepts from neural fields, top-down-bottom-up processing, and attention (consensus between columns), for emergent part-whole heirarchies from data

graph-transformer-pytorch

Implementation of Graph Transformer in Pytorch, for potential use in replicating Alphafold2

lucidrains Goto Github PK

Phil Wang's Projects

Recommend Projects

Recommend Topics

Recommend Org

Jobs