ATEST / RATTS: Adaptive Resource-Aware Test-Time Scaling for LLM Inference (arXiv:2610.25519)

Sun, 18 Oct 2026 00:00:00 +0000

New Paper: ATEST (arXiv:2610.25519)

ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference is now on arXiv.

We also call this idea RATTS: Resource-Aware Test-Time Scaling for LLM Inference.

Authors: Mobina Kashaniyan, Ali Jannesari (Iowa State University)
arXiv: 2610.25519
DOI: 10.48550/arXiv.2610.25519
PDF: arxiv.org/pdf/2610.25519

Self-Consistency and Why a Fixed Sample Count Wastes Energy

Self-consistency improves LLM answers by generating several samples and voting. Most adaptive self-consistency methods decide how many samples a question needs from difficulty or answer agreement. They treat the cost of one sample as a fixed number.

On a shared GPU, that cost is not fixed. In our measurements, the same four-sample query uses up to 4.1× more energy when other jobs load the GPU. A GSM8K query can cost 121.7 J on an idle A100 and 455.6 J on a busy one.

What ATEST Does

ATEST answers two questions at once:

Is one more sample useful for this query?
What does that sample cost on the GPU right now?

ATEST-Online (default) needs no offline calibration. It starts with one sample, checks whether the plurality answer is stable, reads the live GPU state, learns step energy online, and stops when the next sample is not worth its cost.

ATEST-Calibrated profiles a dataset once and picks the full budget per query. That mode is useful when GPU load is stable and low tail latency matters. It does not follow the load when the GPU becomes busy.

Results

On an NVIDIA A100 with Phi-3-mini, ATEST-Online answers every query, stays inside the energy budget, and matches or leads accuracy on all four datasets while using fewer samples as load rises. The same pattern holds on an NVIDIA L40S and with Qwen2.5-3B.

On GSM8K at high load:

ATEST-Online: 60.0% accuracy, no deferred queries
ATEST-Calibrated: 44.0% accuracy, because it defers 42.4% of queries

Including offline profiling, ATEST-Online uses 42× less GPU time and 48× less energy than ATEST-Calibrated.

How to Cite ATEST

@misc{kashaniyan2026atest,
 title = {ATEST: Adaptive Resource-Aware Test-Time Scaling for {LLM} Inference},
 author = {Kashaniyan, Mobina and Jannesari, Ali},
 year = {2026},
 eprint = {2610.25519},
 archivePrefix = {arXiv},
 primaryClass = {cs.LG},
 doi = {10.48550/arXiv.2610.25519},
 url = {https://arxiv.org/abs/2610.25519}
}

Kashaniyan, M., & Jannesari, A. (2026). ATEST: Adaptive resource-aware test-time scaling for LLM inference. arXiv preprint arXiv:2610.25519. https://doi.org/10.48550/arXiv.2610.25519

Paper Links

arXiv
PDF
DOI
Publication page
Related: Sample Count Is Not Enough (arXiv:2609.19499)

Resource-Aware Test-Time Scaling for Shared GPUs

Adaptive Sampling under Live GPU Load

Energy-Aware LLM Inference with ATEST and RATTS

ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference

Sun, 18 Oct 2026 00:00:00 +0000

New preprint: arXiv:2610.25519 — ATEST: Adaptive Resource-Aware Test-Time Scaling for LLM Inference by Mobina Kashaniyan and Ali Jannesari.

ATEST is a resource-aware test-time scaling method, also called RATTS (Resource-Aware Test-Time Scaling). It targets self-consistency: the model draws several answers and keeps the plurality vote. Standard self-consistency uses a fixed sample count whether the GPU is idle or busy. ATEST reads the live GPU state and skips extra self-consistency samples when they are not worth their current energy cost.

On an NVIDIA A100, the same four-sample query uses up to 4.1× more energy when other jobs load the GPU. A GSM8K query that costs 121.7 J on an idle GPU can cost 455.6 J on a busy one.

ATEST-Online, the default mode, needs no offline calibration. It starts with one sample, checks whether the plurality answer is stable, predicts the energy of the next step, and stops when another sample is not worth the cost. ATEST-Calibrated is an offline mode for stable deployments: it profiles a dataset once and picks the whole budget per query.

On GSM8K at high load, ATEST-Online keeps 60.0% accuracy and answers every query. ATEST-Calibrated drops to 44.0% because it defers 42.4% of queries. Including profiling, ATEST-Online uses 42× less GPU time and 48× less energy than ATEST-Calibrated. The same pattern holds on an NVIDIA L40S and with Qwen2.5-3B.

This work follows our earlier result that candidate count alone is not enough to describe the systems cost of test-time scaling. ATEST asks the next question: what does one more sample cost on the GPU right now?

Keywords: ATEST, RATTS, self-consistency, self consistency, resource-aware self-consistency, adaptive test-time scaling, LLM inference, GPU load, energy-aware systems, adaptive sampling, A100, L40S, Phi-3-mini, Qwen2.5-3B, GSM8K, arXiv:2610.25519.

Self-Consistency under GPU Load

How to Cite

@misc{kashaniyan2026atest,
 title = {ATEST: Adaptive Resource-Aware Test-Time Scaling for {LLM} Inference},
 author = {Kashaniyan, Mobina and Jannesari, Ali},
 year = {2026},
 eprint = {2610.25519},
 archivePrefix = {arXiv},
 primaryClass = {cs.LG},
 doi = {10.48550/arXiv.2610.25519},
 url = {https://arxiv.org/abs/2610.25519}
}