Latency

Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling
arXiv:2609.19499 — At fixed candidate count N, how candidates are batched into generation calls can change LLM test-time scaling energy and latency by several times on A100 GPUs.
Mobina Kashaniyan, Ali Jannesari
Posted Nov 1, 2026 Conference Paper, Workshop Paper, Preprint, Large Language Models, High-Performance Computing, Energy Efficiency
PDF DOI PDF arXiv DOI Google Scholar Research Summary Local PDF

© 2026 Mobina Kashaniyan.