vLLM 7/16 three-layer quality gates: CI 37 test groups / 266 jobs / perf-eval 17 model-hardware recipes / two-week release cycle

vLLM's July 16, 2026 blog post 'Keeping vLLM Production Quality' documents three layers every PR passes through: Layer 1 CI (lightweight GitHub Actions on every PR, heavier unit testing on Buildkite with 37 test groups and 266 jobs dynamically selected based on the diff, shared multi-stage container image, pip-compile lock files, 58 runner queues across AWS / Crusoe / LambdaLabs / Nebius / NVIDIA / Roblox / RunPod via Buildkite agents, MIG-slicing and autoscale-from-zero per-machine runners, custom dashboard at ci.vllm.ai, nightly CI-analyzer bot posting Slack reports with auto-revert PRs ~1.5/day at ~70% correct diagnosis); Layer 2 perf & accuracy (nightly pipeline at github.com/vllm-project/perf-eval covering 17 model-hardware recipes across H200 / B200 / MI300X / MI355X — DeepSeek V4 Pro/Flash, gpt-oss, Kimi K2.5, MiniMax M2.5/M3, Qwen3.5, GLM 5.1, Gemma 4, Nemotron 3 Super — measuring TTFT / TPOT / vllm-bench, plus accuracy via lm-eval on GSM8K / GPQA / AIME and function-calling via BFCL); Layer 3 release (two-week cadence since November 2025 — every other Monday the release manager picks the greenest main-branch commit to cut releases/vX.Y.Z; Mon-Wed cherry-pick window; candidate ships only when all three gates pass; delays accepted rather than lowering the bar; 7 Python wheels + 11 Docker images per release smoke-tested before publication).