TheTom/vllm-swift
vLLM Metal plugin powered by mlx-swift — high-performance LLM inference on Apple Silicon
Working on LLM inference systems, KV cache compression, and kernel-level optimizations (TurboQuant).
Repositories
TheTom/hermes-go
The agent that grows with you
TheTom/llama-cpp-turboquant
LLM inference in C/C++
TheTom/turboquant_plus
TheTom/elm327_obd_for_mac
I'm crazy and trying to make a ForScan OBD reader work on my mac.
TheTom/pascal-egpu
Driverless NVIDIA Pascal (GTX 1060) compute from macOS Apple Silicon over Thunderbolt eGPU
TheTom/macos-ci-burst-runner
Opt-in menu-bar controller for safely using any Mac as a GitHub Actions burst runner
TheTom/offlabel
TheTom/atlas-inference
TheTom/vllm-turboquant
A high-throughput and memory-efficient inference and serving engine for LLMs
TheTom/melee
SSBM GALE01 decomp progress fork (doldecomp/melee). Provide your own main.dol.
TheTom/model-serving-minefield
Community registry of LLM serving-path traps that produce confidently wrong measurements: templates, tool parsers, reasoning fields, quant kernel paths, CUDA toolchains, KV allocation, eval harnesses, versioning. Symptom-first, with the check that catches each.
TheTom/melee-decomp
A decompilation of Super Smash Bros Melee brought to you by a bunch of clever folks.
TheTom/ffai
FFAI — F*cking Fast AI. Dual-engine inference: Swift (Apple/iPhone, native Metal) + Rust (cross-platform: CUDA/Vulkan/ROCm). Shared kernels via metaltile.
TheTom/zero
TheTom/momento
Deterministic State Recovery for AI Coding Agents
TheTom/mlx
MLX: An array framework for Apple silicon
TheTom/pastors-pocket-spurgeon
Offline Spurgeon study companion — pastoral counsel, sermon prep, and Sword & Trowel sermon grading. Gemma-4-12B fine-tune served on TurboQuant llama.cpp.
TheTom/metaltile
A Rust-embedded DSL for writing Apple Metal GPU kernels. Write tile-level algorithms in Rust, get optimized Metal Shading Language out.
TheTom/ds4-turboquant_plus
Private backup of TQ+ Metal port work on antirez/ds4. Branch tom/turbo3-kv-cuda (turbo2/3/4 + Wave M3 inline-dequant + half-tile + CUDA stubs + h8 Flash WIP).
TheTom/ds4
DeepSeek 4 Flash local inference engine for Metal and CUDA
TheTom/atlas
Pure Rust Inference Engine
TheTom/longctx
Open long-context inference stack: retrieval + open weights, no closed parts. pip install longctx.
TheTom/tqkit
Unified toolkit for benchmarking and integrating TurboQuant+ KV-cache compression across llama.cpp, vLLM, MLX, and vllm-swift.
TheTom/homebrew-tap
Homebrew tap for TheTom projects (vllm-swift)
TheTom/llama.cpp-turboquant-hip
TurboQuant KV cache compression for llama.cpp — HIP/ROCm port for AMD RDNA3 (gfx1100)
TheTom/mlx-swift
Swift API for MLX
TheTom/mlx-swift-lm
LLMs and VLMs with MLX Swift
TheTom/vllm-metal
Community maintained hardware plugin for vLLM on Apple Silicon
TheTom/cs161
StemVille real-time data plotting experiment