lucidrains Goto Github PK

followers: 20.8K following: 0.0 repos: 261.0 gists: 27.0

Name: Phil Wang

Type: User

Bio: Working with Attention. It's all we need

Location: San Francisco

Blog: lucidrains.github.io

Phil Wang's Projects

adamod

Adaptive and Momental Bounds for Adaptive Learning Rate Methods.

adan-pytorch

Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch

adjacent-attention-network

Graph neural network message passing reframed as a Transformer with local attention

all-normalization-transformer

A simple Transformer where the softmax has been replaced with normalization

alphafold2

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released

Prediction of key transcription factors in cell fate determination using enhancer networks. See full ANANSE documentation for detailed installation instructions and usage examples. http://anansepy.readthedocs.io

anymal-belief-state-encoder-decoder-pytorch

Implementation of the Belief State Encoder / Decoder in the new breakthrough robotics paper from ETH Zürich

aoa-pytorch

A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering

arxiv-sanity-preserver

Web interface for browsing, search and filtering recent arxiv submissions

attention

This repository will house a visualization that will attempt to convey instant enlightenment of how Attention works to someone not working in artificial intelligence, with 3Blue1Brown as inspiration

attention-tensorflow-mesh

Tools for building attention networks for Tensorflow Mesh

audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

autoregressive-linear-attention-cuda

CUDA implementation of autoregressive linear attention, with all the latest research findings

axial-attention

Implementation of Axial attention - attending to multi-dimensional data efficiently

axial-positional-embedding

Axial Positional Embedding for Pytorch

bidirectional-cross-attention

A simple cross attention that updates both the source and target in one step

big-sleep

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun