1a3orn Goto Github PK

efficientzero

Open-source codebase for EfficientZero, from "Mastering Atari Games with Limited Data" at NeurIPS 2021.

eval-possible

gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

length-generalization

Best lenght generalization test suite

metaseq

Repo for external large-scale work

mingpt

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

nanogpt

The simplest, fastest repository for training/finetuning medium-sized GPTs.

neurips2020-procgen

rainbow

Rainbow DQN implementation accompanying the paper "Fast and Data-Efficient Training of Rainbow" which reaches 205.7 median HNS after 10M frames. 🌈

rwkv-lm-downsampled

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

very-simple-moe

Extremely simple MoE implementation, mostly based off Switch Transformer