lucidrains Goto Github PK

followers: 30.4K following: 0.0 repos: 282.0 gists: 27.0

Name: Phil Wang

Type: User

Bio: Working with Attention. It's all we need

Location: San Francisco

Blog: lucidrains.github.io

Phil Wang's Projects

palm-jax

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways - in Jax (Equinox framework)

palm-pytorch

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways

palm-rlhf-pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

panoptic-transformer

Another attempt at a long-context / efficient transformer by me

parti-pytorch

Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch

pause-transformer

Yet another random morning idea to be quickly tried and architecture shared if it works; to allow the transformer to pause for any amount of time on any token

perceiver-ar-pytorch

Implementation of Perceiver AR, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch

perceiver-pytorch

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

perfusion-pytorch

Implementation of Key-Locked Rank One Editing, from Nvidia AI

phasic-policy-gradient

An implementation of Phasic Policy Gradient, a proposed improvement of Proximal Policy Gradients, in Pytorch

phenaki-pytorch

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

pi-gan-pytorch

Implementation of π-GAN, for 3d-aware image synthesis, in Pytorch

pixel-level-contrastive-learning

Implementation of Pixel-level Contrastive Learning, proposed in the paper "Propagate Yourself", in Pytorch

point-transformer-pytorch

Implementation of the Point Transformer layer, in Pytorch

ponder-transformer

Implementation of a Transformer that Ponders, using the scheme from the PonderNet paper

product-key-memory

Standalone Product Key Memory module in Pytorch - for augmenting Transformer models

progen

Implementation and replication of ProGen, Language Modeling for Protein Generation, in Jax

protein-bert-pytorch

Implementation of ProteinBERT in Pytorch

pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

pytorch-optimizer

torch-optimizer -- collection of optimizers for Pytorch

q-transformer

Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind

raveforce

RaveForce - An OpenAI Gym style toolkit for music generation experiments.

recurrent-interface-network-pytorch

Implementation of Recurrent Interface Network (RIN), for highly efficient generation of images and video without cascading networks, in Pytorch

recurrent-memory-transformer-pytorch

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

recurrentjs

Deep Recurrent Neural Networks and LSTMs in Javascript. More generally also arbitrary expression graphs with automatic differentiation.

reformer-pytorch

Reformer, the efficient Transformer, in Pytorch

rela-transformer

Implementation of a Transformer using ReLA (Rectified Linear Attention) from https://arxiv.org/abs/2104.07012

remixer-pytorch

Implementation of the Remixer Block from the Remixer paper, in Pytorch

res-mlp-pytorch

Implementation of ResMLP, an all MLP solution to image classification, in Pytorch

lucidrains Goto Github PK

Phil Wang's Projects

Recommend Projects

Recommend Topics

Recommend Org

Jobs