ZewenShen-Cohere/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
@ZewenShen
Repositories
ZewenShen-Cohere/compressed-tensors
A safetensors extension to efficiently store sparse quantized tensors on disk
ZewenShen-Cohere/llm-compressor-fork
Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM