WoosukKwon/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
@Inferact | @vllm-project
Repositories
WoosukKwon/retraining-free-pruning
[NeurIPS 2022] A Fast Post-Training Pruning Framework for Transformers
WoosukKwon/vmi
WoosukKwon/scatter
WoosukKwon/flashinfer
FlashInfer: Kernel Library for LLM Serving
WoosukKwon/scripts
WoosukKwon/dynamo
A Datacenter Scale Distributed Inference Serving Framework
WoosukKwon/exp
WoosukKwon/torch-xla
Enabling PyTorch on XLA Devices (e.g. Google TPU)