Mellonta/flash-attention
Fast and memory-efficient exact attention
Μέλλοντα
Repositories
Mellonta/linear-attention-visualized
Educational PyTorch implementations of linear attention, covering RetNet, Mamba-2, DeltaNet, Gated DeltaNet (GDN), Gated Delta Product (GDP), Gated Linear Attention (GLA), and Kimi Delta Attention (KDA).
Mellonta/causal-conv1d
Causal depthwise conv1d in CUDA, with a PyTorch interface
Mellonta/mamba-cp
Mamba SSM architecture
Mellonta/flash-linear-attention
🚀 Efficient implementations for emerging model architectures
Mellonta/nvidia-resiliency-ext
NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the effective training time by minimizing the downtime due to failures and interruptions.
Mellonta/TransformerEngine
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.
Mellonta/Megatron-LM
Ongoing research training transformer models at scale
Mellonta/sglang
SGLang is a fast serving framework for large language models and vision language models.
Mellonta/tmp
Mellonta/danmaku
A userscript that adds danmaku to Plex Web
Mellonta/nccl
Optimized primitives for collective multi-GPU communication
Mellonta/weface
A front-end library for displaying wechat messages with faces.