Self-Consistency Sampling | Mobina Kashaniyan

Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling

Sun, 01 Nov 2026 00:00:00 +0000

Preprint: arXiv:2609.19499 — Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling by Mobina Kashaniyan and Ali Jannesari. Follow-up: ATEST / RATTS (arXiv:2610.25519), adaptive resource-aware test-time scaling that also reacts to live GPU load.

This paper studies LLM test-time scaling from a systems and energy perspective. It shows that the candidate count N is not enough to describe the cost of multi-candidate inference.

In sampling-based test-time scaling, the same budget of N candidates can be generated as one large batch or as many small sequential calls. Those schedules have the same candidate count, but they can differ dramatically in latency, throughput, GPU-hours, utilization, and GPU-device energy.

We evaluate Phi-3-mini and Qwen2.5-1.5B on GSM8K, then fix N=8 and compare generation schedules 1×8, 2×4, 4×2, and 8×1. On A100 GPUs, eight serial calls use about 4.64–4.86× the energy and 5.77–6.12× the P95 latency of one batched eight-candidate call. Similar trends appear on SciQ with V100 GPUs.

This work was accepted to the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP), co-located with SC26 in Chicago, USA, and is freely available on arXiv as 2609.19499.

It is relevant to researchers working on test-time scaling, test-time compute, candidate generation, batched LLM inference, GPU energy measurement, sustainable AI, green AI, high-performance computing, distributed/parallel computing, and efficient large language model systems.

Keywords: arXiv:2609.19499, self-consistency, self consistency, LLM test-time scaling, test-time compute, candidate-generation strategy, generation schedule, batched inference, multi-candidate sampling, best-of-N, GPU energy, energy-efficient AI, sustainable AI, SC26, EESP, high-performance computing, A100, GSM8K, SciQ, cs.LG, cs.DC, cs.PF.

Self-Consistency and Generation Schedule

Venue

Mobina Kashaniyan, Ali Jannesari. Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling. In Proc. of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP), co-located with SC26, Chicago, USA, pages 1–8, November 2026. arXiv:2609.19499.

How to Cite

@inproceedings{kashaniyan2026sample,
 title = {Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of {LLM} Test-Time Scaling},
 author = {Kashaniyan, Mobina and Jannesari, Ali},
 booktitle = {Proceedings of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP)},
 pages = {1--8},
 year = {2026},
 note = {arXiv:2609.19499},
 doi = {10.48550/arXiv.2609.19499},
 url = {https://arxiv.org/abs/2609.19499}
}

Kashaniyan, M., & Jannesari, A. (2026). Sample count is not enough: Candidate-generation strategy shapes the energy and performance of LLM test-time scaling. In Proceedings of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP) (pp. 1–8). IEEE/ACM. https://doi.org/10.48550/arXiv.2609.19499

ATEST / RATTS: Adaptive Resource-Aware Test-Time Scaling for LLM Inference (arXiv:2610.25519)

Sun, 18 Oct 2026 00:00:00 +0000

New Paper: ATEST (arXiv:2610.25519)

ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference is now on arXiv.

We also call this idea RATTS: Resource-Aware Test-Time Scaling for LLM Inference.

Authors: Mobina Kashaniyan, Ali Jannesari (Iowa State University)
arXiv: 2610.25519
DOI: 10.48550/arXiv.2610.25519
PDF: arxiv.org/pdf/2610.25519

Self-Consistency and Why a Fixed Sample Count Wastes Energy

Self-consistency improves LLM answers by generating several samples and voting. Most adaptive self-consistency methods decide how many samples a question needs from difficulty or answer agreement. They treat the cost of one sample as a fixed number.

On a shared GPU, that cost is not fixed. In our measurements, the same four-sample query uses up to 4.1× more energy when other jobs load the GPU. A GSM8K query can cost 121.7 J on an idle A100 and 455.6 J on a busy one.

What ATEST Does

ATEST answers two questions at once:

Is one more sample useful for this query?
What does that sample cost on the GPU right now?

ATEST-Online (default) needs no offline calibration. It starts with one sample, checks whether the plurality answer is stable, reads the live GPU state, learns step energy online, and stops when the next sample is not worth its cost.

ATEST-Calibrated profiles a dataset once and picks the full budget per query. That mode is useful when GPU load is stable and low tail latency matters. It does not follow the load when the GPU becomes busy.

Results

On an NVIDIA A100 with Phi-3-mini, ATEST-Online answers every query, stays inside the energy budget, and matches or leads accuracy on all four datasets while using fewer samples as load rises. The same pattern holds on an NVIDIA L40S and with Qwen2.5-3B.

On GSM8K at high load:

ATEST-Online: 60.0% accuracy, no deferred queries
ATEST-Calibrated: 44.0% accuracy, because it defers 42.4% of queries

Including offline profiling, ATEST-Online uses 42× less GPU time and 48× less energy than ATEST-Calibrated.

How to Cite ATEST

@misc{kashaniyan2026atest,
 title = {ATEST: Adaptive Resource-Aware Test-Time Scaling for {LLM} Inference},
 author = {Kashaniyan, Mobina and Jannesari, Ali},
 year = {2026},
 eprint = {2610.25519},
 archivePrefix = {arXiv},
 primaryClass = {cs.LG},
 doi = {10.48550/arXiv.2610.25519},
 url = {https://arxiv.org/abs/2610.25519}
}

Kashaniyan, M., & Jannesari, A. (2026). ATEST: Adaptive resource-aware test-time scaling for LLM inference. arXiv preprint arXiv:2610.25519. https://doi.org/10.48550/arXiv.2610.25519

Paper Links

arXiv
PDF
DOI
Publication page
Related: Sample Count Is Not Enough (arXiv:2609.19499)

Resource-Aware Test-Time Scaling for Shared GPUs

Adaptive Sampling under Live GPU Load

Energy-Aware LLM Inference with ATEST and RATTS

ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

Sun, 18 Oct 2026 00:00:00 +0000

New preprint: arXiv:2610.25519 — ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference by Mobina Kashaniyan and Ali Jannesari.

ATEST is a resource-aware test-time scaling method, also called RATTS (Resource-Aware Test-Time Scaling). It targets self-consistency: the model draws several answers and keeps the plurality vote. Standard self-consistency uses a fixed sample count whether the GPU is idle or busy. ATEST reads the live GPU state and skips extra self-consistency samples when they are not worth their current energy cost.

On an NVIDIA A100, the same four-sample query uses up to 4.1× more energy when other jobs load the GPU. A GSM8K query that costs 121.7 J on an idle GPU can cost 455.6 J on a busy one.

ATEST-Online, the default mode, needs no offline calibration. It starts with one sample, checks whether the plurality answer is stable, predicts the energy of the next step, and stops when another sample is not worth the cost. ATEST-Calibrated is an offline mode for stable deployments: it profiles a dataset once and picks the whole budget per query.

On GSM8K at high load, ATEST-Online keeps 60.0% accuracy and answers every query. ATEST-Calibrated drops to 44.0% because it defers 42.4% of queries. Including profiling, ATEST-Online uses 42× less GPU time and 48× less energy than ATEST-Calibrated. The same pattern holds on an NVIDIA L40S and with Qwen2.5-3B.

This work follows our earlier result that candidate count alone is not enough to describe the systems cost of test-time scaling. ATEST asks the next question: what does one more sample cost on the GPU right now?

Keywords: ATEST, RATTS, self-consistency, self consistency, resource-aware self-consistency, adaptive test-time scaling, LLM inference, GPU load, energy-aware systems, adaptive sampling, A100, L40S, Phi-3-mini, Qwen2.5-3B, GSM8K, arXiv:2610.25519.

Self-Consistency under GPU Load

How to Cite

@misc{kashaniyan2026atest,
 title = {ATEST: Adaptive Resource-Aware Test-Time Scaling for {LLM} Inference},
 author = {Kashaniyan, Mobina and Jannesari, Ali},
 year = {2026},
 eprint = {2610.25519},
 archivePrefix = {arXiv},
 primaryClass = {cs.LG},
 doi = {10.48550/arXiv.2610.25519},
 url = {https://arxiv.org/abs/2610.25519}
}

Kashaniyan, M., & Jannesari, A. (2026). ATEST: Adaptive resource-aware test-time scaling for LLM inference. arXiv preprint arXiv:2610.25519. https://doi.org/10.48550/arXiv.2610.25519

Sample Count Is Not Enough (arXiv:2609.19499): Why Candidate-Generation Strategy Matters for LLM Test-Time Scaling Energy and Performance

Wed, 16 Sep 2026 00:00:00 +0000

New Paper on arXiv: 2609.19499

Our paper is now live on arXiv:

Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling

Authors: Mobina Kashaniyan, Ali Jannesari
arXiv: 2609.19499
DOI: 10.48550/arXiv.2609.19499
PDF: arxiv.org/pdf/2609.19499
Venue: 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP), co-located with SC26, Chicago, USA, November 2026

What Is LLM Test-Time Scaling?

LLM test-time scaling, also called test-time compute or inference-time scaling, improves large language model reasoning by spending more compute at inference time. A common approach is to generate multiple candidate answers and then aggregate them, for example with self-consistency or best-of-N selection.

Why Candidate Count Alone Is Not Enough

In sampling-based methods, the inference budget is often summarized by the candidate count N. That number says how many candidates are generated, but not how they are executed on the GPU.

The same N can be produced as:

one batched generation call with all candidates together, or
several sequential generation calls with smaller batch sizes

Those schedules have the same candidate count, but they can have very different latency, throughput, GPU-hours, utilization, and energy.

Candidate-Generation Strategy and Generation Schedules

In arXiv:2609.19499, we formalize the candidate-generation schedule and compare four fixed-budget schedules at N=8:

1×8: one call with eight candidates
2×4: two calls with four candidates each
4×2: four calls with two candidates each
8×1: eight serial calls with one candidate each

Energy and Performance Results on A100 GPUs

We evaluate Phi-3-mini and Qwen2.5-1.5B on GSM8K. Increasing N from 1 to 8 improves accuracy, as expected. But at fixed N=8, systems cost depends strongly on the schedule.

On A100 GPUs, eight serial calls use about 4.64–4.86× as much gross GPU-device energy and have about 5.77–6.12× the P95 latency of one batched eight-candidate call. Similar patterns appear across independently scheduled A100 nodes and in short-output SciQ/V100 experiments.

Practical Takeaway for Efficient LLM Inference

When candidates are independent and memory allows it, fewer generation calls with larger batch sizes are more efficient. Evaluations of multi-candidate test-time scaling should report not only candidate count and accuracy, but also:

generation schedule
calls per query
candidates per call
GPU-level systems metrics such as latency, throughput, and energy

Our Paper

In our paper, “Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling” (arXiv:2609.19499), we show that the systems cost of multi-candidate LLM inference depends on how candidates are grouped into generation calls, not only on how many candidates are generated.

This work was accepted to the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP), co-located with SC26 in Chicago, USA (pages 1–8, November 2026).

Authors: Mobina Kashaniyan and Ali Jannesari, Iowa State University.

Abstract

Test-time scaling can improve large language model reasoning by generating and combining multiple candidate responses. In sampling-based methods, the inference budget is often described by the number of generated candidates, N. However, N tells us how many candidates are generated, not how they are executed. The same candidate budget can be produced in one batched generation call or split across several sequential calls with smaller batch sizes.

We first study the effect of increasing N on reasoning accuracy using Phi-3-mini and Qwen2.5-1.5B on 500 GSM8K prompts. As expected, increasing N from 1 to 8 improves accuracy by 8.4 percentage points for Phi-3-mini and 18.4 points for Qwen2.5-1.5B. However, accuracy alone does not show the systems cost of using a larger candidate budget.

We therefore fix N=8 and compare four generation schedules: 1×8, 2×4, 4×2, and 8×1, where a×b denotes a generation calls with b candidates per call. We measure latency, throughput, GPU-hours, and gross GPU-device energy while keeping the total candidate count fixed. On A100 GPUs, eight serial calls use 4.64–4.86× as much gross GPU-device energy and have 5.77–6.12× the P95 latency of one batched call with eight candidates.

The same pattern appears across three independently scheduled A100 nodes per model and in short-output SciQ/V100 experiments. These results show that candidate count alone is not enough to describe the systems cost of multi-candidate test-time scaling. When candidates are independent and memory allows it, fewer generation calls with larger batch sizes are more efficient. Evaluations should therefore report not only candidate count and accuracy, but also generation schedule and GPU-level systems metrics.

Paper Links

Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling

Mobina Kashaniyan and Ali Jannesari

arXiv Abstract
arXiv PDF
DOI
HTML
Publication Page
Venue: 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP), co-located with SC26, Chicago, USA, November 2026

Cite This Paper

@inproceedings{kashaniyan2026sample,
 title = {Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of {LLM} Test-Time Scaling},
 author = {Kashaniyan, Mobina and Jannesari, Ali},
 booktitle = {Proceedings of the 3rd IEEE/ACM SC26 Workshop on Energy Efficiency with Sustainable Performance (EESP)},
 pages = {1--8},
 year = {2026},
 note = {arXiv:2609.19499},
 doi = {10.48550/arXiv.2609.19499},
 url = {https://arxiv.org/abs/2609.19499}
}

Kashaniyan, M., & Jannesari, A. (2026). Sample count is not enough: Candidate-generation strategy shapes the energy and performance of LLM test-time scaling. arXiv preprint arXiv:2609.19499. https://doi.org/10.48550/arXiv.2609.19499

LLM Test-Time Scaling Energy and Performance

Candidate-Generation Strategy for Multi-Candidate Inference

Batched vs Serial Generation Schedules for Test-Time Compute

Sustainable and Energy-Efficient LLM Inference on GPUs

arXiv 2609.19499 LLM Energy Efficiency SC26 EESP

Interpretable Adaptive Sampling for LLM Test-Time Scaling

Tue, 04 Aug 2026 00:00:00 +0000

This paper introduces an interpretable adaptive sampling framework for large language model test-time scaling.

Most test-time scaling approaches assign the same number of candidate generations to every prompt. However, easy and difficult prompts may require very different amounts of inference-time compute.

Our method uses a lightweight fuzzy controller that combines estimated prompt complexity and model confidence. It assigns fewer samples to easier or higher-confidence prompts and more samples to difficult or uncertain prompts.

The approach is evaluated against fixed best-of-N sampling, compute-aware scaling, and self-certainty-based methods on question-answering and mathematical reasoning tasks. The results demonstrate that adaptive fuzzy control can reduce the average number of generated samples while maintaining competitive reasoning performance.

This work is relevant to LLM test-time scaling, adaptive inference, compute-efficient AI, interpretable sampling, self-consistency, and efficient large language model reasoning.

Keywords: self-consistency, self consistency, adaptive self-consistency, LLM test-time scaling, adaptive sampling, inference-time compute, efficient LLM inference, interpretable AI, fuzzy control, best-of-N sampling, adaptive computation, mathematical reasoning.

Adaptive Self-Consistency

Citation

Kashaniyan, M., & Jannesari, A. (2026). Interpretable adaptive sampling for LLM test-time scaling. arXiv preprint arXiv:2608.03961. https://doi.org/10.48550/arXiv.2608.03961

@misc{kashaniyan2026adaptive,
 title = {Interpretable Adaptive Sampling for {LLM} Test-Time Scaling},
 author = {Kashaniyan, Mobina and Jannesari, Ali},
 year = {2026},
 eprint = {2608.03961},
 archivePrefix = {arXiv},
 primaryClass = {cs.AI},
 doi = {10.48550/arXiv.2608.03961},
 url = {https://arxiv.org/abs/2608.03961}
}

Interpretable Adaptive Sampling for LLM Test-Time Scaling and Test-Time Compute

Tue, 04 Aug 2026 00:00:00 +0000

Adaptive Self-Consistency

Self-consistency draws several answers and keeps the most common one. This paper chooses a different self-consistency sample count for each prompt instead of using one fixed budget.

What Is LLM Test-Time Scaling?

Test-time scaling, also referred to as test-time compute or inference-time scaling, improves large language model reasoning by allocating additional computation during inference.

Adaptive Sampling for Test-Time Compute

Most fixed best-of-N approaches assign the same sampling budget to every prompt. Our approach dynamically allocates inference-time compute based on prompt complexity and model confidence.

Interpretable Inference-Time Compute

A lightweight fuzzy controller determines how many candidate responses each query receives. Easier or higher-confidence prompts receive fewer samples, while more difficult or uncertain prompts receive additional test-time compute.

Efficient LLM Reasoning

Adaptive sampling can reduce unnecessary candidate generation while maintaining competitive reasoning performance across question-answering and mathematical reasoning tasks.

Comparison with Best-of-N and Self-Consistency

We compare the approach with fixed best-of-N sampling, compute-aware test-time scaling, and self-certainty-based methods.

Our Paper

Test-time scaling, also known as test-time compute or inference-time scaling, can improve large language model reasoning by generating multiple candidate answers. However, most approaches assign the same sampling budget to every prompt, even though easy and difficult questions may require different amounts of inference-time compute.

In our paper, “Interpretable Adaptive Sampling for LLM Test-Time Scaling,” we introduce an adaptive method that determines how many candidate answers should be generated for each query.

The method uses a lightweight fuzzy controller that combines interpretable signals such as prompt complexity and model confidence. It assigns fewer samples to easier or higher-confidence prompts and more samples to difficult or uncertain prompts.

Our experiments compare this approach with fixed best-of-N sampling, compute-aware scaling, and self-certainty-based methods across question-answering and mathematical reasoning tasks.

The results show that adaptive fuzzy control can reduce the average number of generated samples while maintaining competitive reasoning performance.

This research contributes to LLM test-time scaling, test-time compute, adaptive inference, compute-efficient AI, interpretable sampling, self-consistency, and efficient large language model reasoning.

Abstract

Test-time scaling improves LLM reasoning by generating and aggregating multiple candidate answers, yet many pipelines use fixed per-query budgets that spend the same compute on easy and difficult prompts. These fixed budgets are also difficult to inspect because they do not explain why a given prompt receives a particular number of samples.

We propose adaptive test-time scaling with a lightweight fuzzy controller that maps interpretable signals, including estimated prompt complexity and model confidence, to a per-query sampling budget. The controller assigns fewer samples to easier or more confident prompts and more samples to harder or less certain prompts, making inference-time compute inspectable rather than fixed or opaque.

We evaluate under a fair-alignment protocol with matched decoding settings and controlled answer selection, and compare against best-of-N, compute-aware scaling, and self-certainty-based baselines on question-answering and mathematical reasoning tasks. Across models and datasets, adaptive fuzzy control improves over several standard baselines and remains close to a selector-matched full-budget control while reducing the average number of samples.

These findings suggest that interpretable adaptive sampling is a practical direction for more efficient test-time reasoning in large language models.

Self-Consistency Sampling | Mobina Kashaniyan

Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling

Self-Consistency and Generation Schedule

Links

Venue

How to Cite

ATEST / RATTS: Adaptive Resource-Aware Test-Time Scaling for LLM Inference (arXiv:2610.25519)

New Paper: ATEST (arXiv:2610.25519)

Self-Consistency and Why a Fixed Sample Count Wastes Energy

What ATEST Does

Results

How to Cite ATEST

Paper Links

Resource-Aware Test-Time Scaling for Shared GPUs

Adaptive Sampling under Live GPU Load

Energy-Aware LLM Inference with ATEST and RATTS

ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

Self-Consistency under GPU Load

Links

How to Cite

Sample Count Is Not Enough (arXiv:2609.19499): Why Candidate-Generation Strategy Matters for LLM Test-Time Scaling Energy and Performance

New Paper on arXiv: 2609.19499

What Is LLM Test-Time Scaling?

Why Candidate Count Alone Is Not Enough

Candidate-Generation Strategy and Generation Schedules

Energy and Performance Results on A100 GPUs

Practical Takeaway for Efficient LLM Inference

Our Paper

Abstract

Paper Links

Cite This Paper

LLM Test-Time Scaling Energy and Performance

Candidate-Generation Strategy for Multi-Candidate Inference

Batched vs Serial Generation Schedules for Test-Time Compute

Sustainable and Energy-Efficient LLM Inference on GPUs

arXiv 2609.19499 LLM Energy Efficiency SC26 EESP

Interpretable Adaptive Sampling for LLM Test-Time Scaling

Adaptive Self-Consistency

Citation

Interpretable Adaptive Sampling for LLM Test-Time Scaling and Test-Time Compute

Adaptive Self-Consistency

What Is LLM Test-Time Scaling?

Adaptive Sampling for Test-Time Compute

Interpretable Inference-Time Compute

Efficient LLM Reasoning

Comparison with Best-of-N and Self-Consistency

Our Paper

Abstract

Paper

LLM Test-Time Scaling and Test-Time Compute

Adaptive Sampling for Inference-Time Compute

Interpretable Test-Time Reasoning with a Fuzzy Controller