GIREESH7963/vllm-scheduler
Adaptive admission scheduling for multi-tenant LLM inference on vLLM (T4, spec decode).
Repositories
GIREESH7963/speculators
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM