Batched Decoding

Sample Count Is Not Enough (arXiv:2609.19499): Why Candidate-Generation Strategy Matters for LLM Test-Time Scaling Energy and Performance
arXiv:2609.19499 — Candidate count N alone does not define the systems cost of LLM test-time scaling. At fixed N=8, generation schedules like 1×8 vs 8×1 can change A100 energy by about 4.6–4.9× and P95 latency by about 5.8–6.1×.
Mobina Kashaniyan
Posted Sep 16, 2026 Last updated Sep 17, 2026 Large Language Models, High-Performance Computing, Energy Efficiency, Test-Time Scaling, Research Summary

© 2026 Mobina Kashaniyan.