Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling

Mobina Kashaniyan, Ali Jannesari
Posted Nov 1, 2026 Conference Paper, Workshop Paper, Preprint, Large Language Models, High-Performance Computing, Energy Efficiency
PDF DOI PDF arXiv DOI Google Scholar Research Summary Local PDF

New preprint: arXiv:2609.19499 — Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling by Mobina Kashaniyan and Ali Jannesari.

This paper studies LLM test-time scaling from a systems and energy perspective. It shows that the candidate count N is not enough to describe the cost of multi-candidate inference.

In sampling-based test-time scaling, the same budget of N candidates can be generated as one large batch or as many small sequential calls. Those schedules have the same candidate count, but they can differ dramatically in latency, throughput, GPU-hours, utilization, and GPU-device energy.

We evaluate Phi-3-mini and Qwen2.5-1.5B on GSM8K, then fix N=8 and compare generation schedules 1×8, 2×4, 4×2, and 8×1. On A100 GPUs, eight serial calls use about 4.64–4.86× the energy and 5.77–6.12× the P95 latency of one batched eight-candidate call. Similar trends appear on SciQ with V100 GPUs.

This work was accepted to the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP), co-located with SC26 in Chicago, USA, and is freely available on arXiv as 2609.19499.

It is relevant to researchers working on test-time scaling, test-time compute, candidate generation, batched LLM inference, GPU energy measurement, sustainable AI, green AI, high-performance computing, distributed/parallel computing, and efficient large language model systems.

Keywords: arXiv:2609.19499, LLM test-time scaling, test-time compute, candidate-generation strategy, generation schedule, batched inference, multi-candidate sampling, self-consistency, best-of-N, GPU energy, energy-efficient AI, sustainable AI, SC26, EESP, high-performance computing, A100, GSM8K, SciQ, cs.LG, cs.DC, cs.PF.

Venue

Mobina Kashaniyan, Ali Jannesari. Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling. In Proc. of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP), co-located with SC26, Chicago, USA, pages 1–8, November 2026. arXiv:2609.19499.

How to Cite

@inproceedings{kashaniyan2026sample,
  title     = {Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of {LLM} Test-Time Scaling},
  author    = {Kashaniyan, Mobina and Jannesari, Ali},
  booktitle = {Proceedings of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP)},
  pages     = {1--8},
  year      = {2026},
  note      = {arXiv:2609.19499},
  doi       = {10.48550/arXiv.2609.19499},
  url       = {https://arxiv.org/abs/2609.19499}
}

Kashaniyan, M., & Jannesari, A. (2026). Sample count is not enough: Candidate-generation strategy shapes the energy and performance of LLM test-time scaling. In Proceedings of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP) (pp. 1–8). IEEE/ACM. https://doi.org/10.48550/arXiv.2609.19499

— views · — this month

ArXiv:2609.19499 Sample Count Is Not Enough LLM Test-Time Scaling Test-Time Scaling Test-Time Compute Candidate Generation Candidate-Generation Strategy Generation Schedule Batched Inference Multi-Candidate Sampling Self-Consistency Best-of-N LLM Energy Efficiency GPU Energy Measurement GPU Inference High-Performance Computing Sustainable AI Green AI Energy-Efficient AI SC26 EESP Large Language Models Inference Performance Throughput Latency A100 GSM8K SciQ Phi-3-Mini Qwen2.5 Cs.LG Cs.DC Cs.PF

© 2026 Mobina Kashaniyan.