lucidrains Goto Github PK

long-short-transformer

Implementation of Long-Short Transformer, combining local and global inductive biases for attention over long sequences, in Pytorch

lucidrains

Config files for my GitHub profile.

lucidrains.github.io

magic3d-pytorch

Implementation of Magic3D, Text to 3D content synthesis, in Pytorch

make-a-video-pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

marge-pytorch

Implementation of Marge, Pre-training via Paraphrasing, in Pytorch

med-seg-diff-pytorch

Implementation of MedSegDiff in Pytorch - SOTA medical segmentation using DDPM and filtering of features in fourier space

medical-chatgpt

Implementation of ChatGPT, but tailored towards primary care medicine, with the reward being able to collect patient histories in a thorough and efficient manner and come up with a reasonable differential diagnosis

mega-pytorch

Implementation of Mega, the Single-head Attention with Multi-headed EMA architecture that currently holds SOTA on Long Range Arena

memcnn

PyTorch Framework for Developing Memory Efficient Deep Invertible Networks

memformer

Implementation of Memformer, a Memory-augmented Transformer, in Pytorch

memorizing-transformers-pytorch

Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch

memory-compressed-attention

Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizing Long Sequences"

memory-editable-transformer

My explorations into editing the knowledge and memories of an attention network

memory-efficient-attention-pytorch

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

memory-transformer-xl

A variant of Transformer-XL where the memory is updated not with a queue, but with attention

mesh

Mesh TensorFlow: Model Parallelism Made Easier

metaformer-gpt

Implementation of Metaformer, but in an autoregressive manner

mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts

mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

mlm-pytorch

An implementation of masked language modeling for Pytorch, made as concise and simple as possible

mlp-gpt-jax

A GPT, made only of MLPs, in Jax

mlp-mixer-pytorch

An All-MLP solution for Vision, from Google AI

mogrifier

Usable implementation of Mogrifier, a circuit for enhancing LSTMs and potentially other networks, from Deepmind

molecule-attention-transformer

Pytorch reimplementation of Molecule Attention Transformer, which uses a transformer to tackle the graph-like structure of molecules

mujoco

Multi-Joint dynamics with Contact. A general purpose physics simulator.

multistream-transformers

Implementation of Multistream Transformers in Pytorch

muse-maskgit-pytorch

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

musiclm-pytorch

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

n-grammer-pytorch

Implementation of N-Grammer, augmenting Transformers with latent n-grams, in Pytorch