BenjaminBraunDev/k8s.io
Code and configuration to manage Kubernetes project infrastructure, including various *.k8s.io sites
Repositories
BenjaminBraunDev/llm-d-infra
repo for CI and infrastructure required to maintain llm-d org member repos
BenjaminBraunDev/llm-d-benchmark
llm-d benchmark scripts and tooling
BenjaminBraunDev/llm-d.github.io
Website for llm-d: This repository builds the website seen at llm-d.ai
BenjaminBraunDev/llm-d-skills
BenjaminBraunDev/llm-d-router
Inference scheduler for llm-d
BenjaminBraunDev/lws
LeaderWorkerSet: An API for deploying a group of pods as a unit of replication
BenjaminBraunDev/llm-d-latency-predictor
Latency prediction service for ML-model based scoring with llm-d-inference-scheduler
BenjaminBraunDev/llm-d-async
Asynchronous Processor for Inference Gateway. Orchestrator of queues
BenjaminBraunDev/llm-d
llm-d is a Kubernetes-native high-performance distributed LLM inference framework
BenjaminBraunDev/gateway-api-inference-extension
Gateway API Inference Extension
BenjaminBraunDev/TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
BenjaminBraunDev/inference-perf
GenAI inference performance benchmarking tool
BenjaminBraunDev/tensorrtllm_backend
The Triton TensorRT-LLM Backend
BenjaminBraunDev/triton-client
Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.
BenjaminBraunDev/server-fork-orca-header
Development Fork of Triton Inference Server