robertgshaw2-redhat/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
vllm and llm-d
Repositories
robertgshaw2-redhat/llm-d-router
llm-d Router: The intelligent entry point for inference requests
robertgshaw2-redhat/fireconnect
Use Fireworks AI models in Claude Code, Codex, Cursor, VS Code, Copilot, and other coding agents.
robertgshaw2-redhat/humming
Humming is a high-performance, lightweight, and highly flexible JIT (Just-In-Time) compiled GEMM kernel library specifically designed for quantized inference.
robertgshaw2-redhat/llm-d
llm-d is a Kubernetes-native high-performance distributed LLM inference framework
robertgshaw2-redhat/litellm
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
robertgshaw2-redhat/kserve-rollout-poc
robertgshaw2-redhat/litellm-llm-d-fc-poc
robertgshaw2-redhat/inference-quality-automation
Automation for running spot checks on quality
robertgshaw2-redhat/Kimi-Vendor-Verifier
Kimi-Vendor-Verifier
robertgshaw2-redhat/llm-d-models
Model X HW X llm-d
robertgshaw2-redhat/aiperf
AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.
robertgshaw2-redhat/flow-control-experiments
Experiments
robertgshaw2-redhat/maas-demo
robertgshaw2-redhat/rhaii-neo-stack
robertgshaw2-redhat/vllm-rl
Experiments related to vllm + rl loops
robertgshaw2-redhat/rhaii-on-xks
Midstream Fork
robertgshaw2-redhat/guidellm
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
robertgshaw2-redhat/demo-ocp-llm-d
Demo - Distributed Inference on OpenShift (aka llm-d)
robertgshaw2-redhat/llm-d-demo
llm-d demo
robertgshaw2-redhat/vTeam
robertgshaw2-redhat/ig-load
robertgshaw2-redhat/j-llm-d
Justfile harness for llm-d
robertgshaw2-redhat/DeepEP
DeepEP: an efficient expert-parallel communication library
robertgshaw2-redhat/vllm_dependency_install
Scripts for installing libraries
robertgshaw2-redhat/vllm-benchmarking
Benchmarking vLLM
robertgshaw2-redhat/nim_utils
utils for NIM
robertgshaw2-redhat/vllm-trt-compare
Comparison of vLLM and TRT-LLM
robertgshaw2-redhat/llm-benchmark-helm-charts
Helm charts for benchmarking