VincyZhang/dynamo
A Datacenter Scale Distributed Inference Serving Framework
Repositories
VincyZhang/llm-d-benchmark
llm-d benchmark scripts and tooling
VincyZhang/LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
VincyZhang/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
VincyZhang/llm-d
Achieve state of the art inference performance with modern accelerators on Kubernetes
VincyZhang/llm-d-prism
Performance analysis for distributed inference systems
VincyZhang/llm-d-modelservice
helm charts for deploying models with llm-d
VincyZhang/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
VincyZhang/ucx
Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group)
VincyZhang/Time-HD-Lib
Time-HD-Lib: A Library for High-Dimensional Time Series Forecasting
VincyZhang/GenAIExamples
Generative AI Examples is a collection of GenAI examples such as ChatQnA, Copilot, which illustrate the pipeline capabilities of the Open Platform for Enterprise AI (OPEA) project.
VincyZhang/intel-extension-for-transformers
Extending Hugging Face transformers APIs for Transformer-based models and improve the productivity of inference deployment. With extremely compressed models, the toolkit can greatly improve the inference efficiency on Intel platforms.