Imesh7/gpt_clone

GPT clone

★ 0Forks 0PythonGitHub ↗Compare
gpttransformer

README

GPT from Scratch

Project Description

Implemented GPT clone from scratch. Transformer based architecture with some addon features.

Key Features

  1. Dataloader
  2. Transformer (decoder only)
  3. Load pre-train weights

What's Next

  • Quantization
  • Multimodel
  • LORA
  • MOE
  • Reasoning
  • Flow matching
  • GRPO implementation

Contributors

Imesh7imesh-sp

Issues