julienmancuso/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Repositories
julienmancuso/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
julienmancuso/ome
OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs)
julienmancuso/grove
Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling