Ace Eldeib

@alexeldeib · User

GitHub profile ↗ · Compare

@coreweave Cambridge, MA143 followers276 repositories

Repositories

alexeldeib/speculators

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

★ 0Forks 0

alexeldeib/Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

★ 0Forks 0

alexeldeib/aiperf

AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.

★ 0Forks 0

alexeldeib/modelexpress

Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and improve overall performance.

★ 0Forks 0

alexeldeib/dynamo

A Datacenter Scale Distributed Inference Serving Framework

★ 0Forks 0

alexeldeib/grove

Kubernetes enhancements for Network Topology Aware Gang Scheduling & Autoscaling

★ 0Forks 0

alexeldeib/jaxy

TensorStore S3 write throughput optimization benchmarks for JAX arrays

★ 0PythonForks 0

alexeldeib/mem

A markdown-based knowledge tracking CLI for projects

★ 0RustForks 0

alexeldeib/hf-mem

A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

★ 0Forks 0

alexeldeib/fireworks

Terminal fireworks show - ASCII art celebration in your terminal

★ 0PythonForks 1