Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling

Sun, 01 Nov 2026 00:00:00 +0000

New preprint: arXiv:2609.19499 — Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling by Mobina Kashaniyan and Ali Jannesari.

This paper studies LLM test-time scaling from a systems and energy perspective. It shows that the candidate count N is not enough to describe the cost of multi-candidate inference.

In sampling-based test-time scaling, the same budget of N candidates can be generated as one large batch or as many small sequential calls. Those schedules have the same candidate count, but they can differ dramatically in latency, throughput, GPU-hours, utilization, and GPU-device energy.

We evaluate Phi-3-mini and Qwen2.5-1.5B on GSM8K, then fix N=8 and compare generation schedules 1×8, 2×4, 4×2, and 8×1. On A100 GPUs, eight serial calls use about 4.64–4.86× the energy and 5.77–6.12× the P95 latency of one batched eight-candidate call. Similar trends appear on SciQ with V100 GPUs.

This work was accepted to the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP), co-located with SC26 in Chicago, USA, and is freely available on arXiv as 2609.19499.

It is relevant to researchers working on test-time scaling, test-time compute, candidate generation, batched LLM inference, GPU energy measurement, sustainable AI, green AI, high-performance computing, distributed/parallel computing, and efficient large language model systems.

Keywords: arXiv:2609.19499, LLM test-time scaling, test-time compute, candidate-generation strategy, generation schedule, batched inference, multi-candidate sampling, self-consistency, best-of-N, GPU energy, energy-efficient AI, sustainable AI, SC26, EESP, high-performance computing, A100, GSM8K, SciQ, cs.LG, cs.DC, cs.PF.

Venue

Mobina Kashaniyan, Ali Jannesari. Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling. In Proc. of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP), co-located with SC26, Chicago, USA, pages 1–8, November 2026. arXiv:2609.19499.

How to Cite

@inproceedings{kashaniyan2026sample,
 title = {Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of {LLM} Test-Time Scaling},
 author = {Kashaniyan, Mobina and Jannesari, Ali},
 booktitle = {Proceedings of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP)},
 pages = {1--8},
 year = {2026},
 note = {arXiv:2609.19499},
 doi = {10.48550/arXiv.2609.19499},
 url = {https://arxiv.org/abs/2609.19499}
}

Kashaniyan, M., & Jannesari, A. (2026). Sample count is not enough: Candidate-generation strategy shapes the energy and performance of LLM test-time scaling. In Proceedings of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP) (pp. 1–8). IEEE/ACM. https://doi.org/10.48550/arXiv.2609.19499

GPU Inference | Mobina Kashaniyan

Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling

Links

Venue

How to Cite