zihanlin-ai/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
Repositories
zihanlin-ai/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
zihanlin-ai/LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
zihanlin-ai/speculators
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
zihanlin-ai/memline
zihanlin-ai/LMCache-Ascend
LMCache-Ascend is a plugin for running LMCache on the Ascend NPU.
zihanlin-ai/vllm-ascend
Community maintained hardware plugin for vLLM on Ascend
zihanlin-ai/sgl-kernel-npu
SGLang kernel library for NPU