Efficient LLM Inference | Mobina Kashaniyan

Interpretable Adaptive Sampling for LLM Test-Time Scaling

Tue, 04 Aug 2026 00:00:00 +0000

This paper introduces an interpretable adaptive sampling framework for large language model test-time scaling.

Most test-time scaling approaches assign the same number of candidate generations to every prompt. However, easy and difficult prompts may require very different amounts of inference-time compute.

Our method uses a lightweight fuzzy controller that combines estimated prompt complexity and model confidence. It assigns fewer samples to easier or higher-confidence prompts and more samples to difficult or uncertain prompts.

The approach is evaluated against fixed best-of-N sampling, compute-aware scaling, and self-certainty-based methods on question-answering and mathematical reasoning tasks. The results demonstrate that adaptive fuzzy control can reduce the average number of generated samples while maintaining competitive reasoning performance.

This work is relevant to LLM test-time scaling, adaptive inference, compute-efficient AI, interpretable sampling, self-consistency, and efficient large language model reasoning.

Keywords: LLM test-time scaling, adaptive sampling, inference-time compute, efficient LLM inference, interpretable AI, fuzzy control, best-of-N sampling, self-consistency, adaptive computation, mathematical reasoning.

Citation

Kashaniyan, M., & Jannesari, A. (2026). Interpretable adaptive sampling for LLM test-time scaling. arXiv preprint arXiv:2608.03961. https://doi.org/10.48550/arXiv.2608.03961

Interpretable Adaptive Sampling for LLM Test-Time Scaling and Test-Time Compute

Tue, 04 Aug 2026 00:00:00 +0000

What Is LLM Test-Time Scaling?

Test-time scaling, also referred to as test-time compute or inference-time scaling, improves large language model reasoning by allocating additional computation during inference.

Adaptive Sampling for Test-Time Compute

Most fixed best-of-N approaches assign the same sampling budget to every prompt. Our approach dynamically allocates inference-time compute based on prompt complexity and model confidence.

Interpretable Inference-Time Compute

A lightweight fuzzy controller determines how many candidate responses each query receives. Easier or higher-confidence prompts receive fewer samples, while more difficult or uncertain prompts receive additional test-time compute.

Efficient LLM Reasoning

Adaptive sampling can reduce unnecessary candidate generation while maintaining competitive reasoning performance across question-answering and mathematical reasoning tasks.

Comparison with Best-of-N and Self-Consistency

We compare the approach with fixed best-of-N sampling, compute-aware test-time scaling, and self-certainty-based methods.

Our Paper

Test-time scaling, also known as test-time compute or inference-time scaling, can improve large language model reasoning by generating multiple candidate answers. However, most approaches assign the same sampling budget to every prompt, even though easy and difficult questions may require different amounts of inference-time compute.

In our paper, “Interpretable Adaptive Sampling for LLM Test-Time Scaling,” we introduce an adaptive method that determines how many candidate answers should be generated for each query.

The method uses a lightweight fuzzy controller that combines interpretable signals such as prompt complexity and model confidence. It assigns fewer samples to easier or higher-confidence prompts and more samples to difficult or uncertain prompts.

Our experiments compare this approach with fixed best-of-N sampling, compute-aware scaling, and self-certainty-based methods across question-answering and mathematical reasoning tasks.

The results show that adaptive fuzzy control can reduce the average number of generated samples while maintaining competitive reasoning performance.

This research contributes to LLM test-time scaling, test-time compute, adaptive inference, compute-efficient AI, interpretable sampling, self-consistency, and efficient large language model reasoning.

Abstract

Test-time scaling improves LLM reasoning by generating and aggregating multiple candidate answers, yet many pipelines use fixed per-query budgets that spend the same compute on easy and difficult prompts. These fixed budgets are also difficult to inspect because they do not explain why a given prompt receives a particular number of samples.

We propose adaptive test-time scaling with a lightweight fuzzy controller that maps interpretable signals, including estimated prompt complexity and model confidence, to a per-query sampling budget. The controller assigns fewer samples to easier or more confident prompts and more samples to harder or less certain prompts, making inference-time compute inspectable rather than fixed or opaque.

We evaluate under a fair-alignment protocol with matched decoding settings and controlled answer selection, and compare against best-of-N, compute-aware scaling, and self-certainty-based baselines on question-answering and mathematical reasoning tasks. Across models and datasets, adaptive fuzzy control improves over several standard baselines and remains close to a selector-matched full-budget control while reducing the average number of samples.

These findings suggest that interpretable adaptive sampling is a practical direction for more efficient test-time reasoning in large language models.

Paper

Interpretable Adaptive Sampling for LLM Test-Time Scaling

Mobina Kashaniyan and Ali Jannesari

LLM Test-Time Scaling and Test-Time Compute

Adaptive Sampling for Inference-Time Compute

Interpretable Test-Time Reasoning with a Fuzzy Controller

PerfMamba: Mamba Performance, SSM Pruning, Efficient Inference, and Model Optimization

Thu, 02 Jul 2026 00:00:00 +0000

Mamba Performance and Efficient State Space Models

Mamba and selective state space models (SSMs) have emerged as promising alternatives to Transformer-based sequence models for efficient sequence modeling and long-context processing.

Unlike conventional Transformer architectures that rely heavily on attention, Mamba-style state space models are designed to provide efficient sequence processing and favorable scaling behavior. However, practical deployment still depends on understanding runtime performance, memory usage, GPU utilization, inference efficiency, and computational bottlenecks.

Our paper, “PerfMamba: Performance Analysis and Pruning of Selective State Space Models,” provides an empirical performance analysis of Mamba-1 and Mamba-2 and investigates how state pruning can improve the efficiency of selective state space models.

Performance Analysis of Mamba-1 and Mamba-2

PerfMamba benchmarks Mamba-1 and Mamba-2 across sequence lengths ranging from 64 to 16,384 tokens.

The study analyzes:

runtime performance
execution time
computation patterns
memory access behavior
memory efficiency
I/O characteristics
GPU resource utilization
sequence-length scaling
component-level performance
model throughput

This analysis helps reveal how Mamba architectures behave in practice and where computational bottlenecks occur as sequence length increases.

Mamba Inference and Runtime Profiling

The theoretical efficiency of a neural architecture does not necessarily translate directly into optimal hardware performance.

PerfMamba therefore studies Mamba inference performance at the systems level, examining how different components of the architecture contribute to runtime, memory consumption, and resource utilization.

The profiling results show that the selective state space model component accounts for an important portion of the computational workload, making it a natural target for optimization.

This makes the work relevant to researchers studying:

Mamba inference
LLM inference optimization
AI inference acceleration
efficient deep learning
GPU performance
hardware-aware machine learning
machine learning systems

State Space Model Pruning

Based on the performance profiling results, PerfMamba investigates state pruning for selective state space models.

The proposed approach identifies and removes low-activity states within the SSM component to reduce unnecessary computation.

This connects Mamba optimization with broader research on:

neural network pruning
model compression
state-space model pruning
model acceleration
memory optimization
efficient AI inference
hardware-aware optimization

The goal is to improve computational efficiency while preserving model accuracy under moderate pruning.

Mamba Pruning for Faster and More Memory-Efficient Inference

PerfMamba demonstrates that selective state pruning can improve runtime and memory efficiency.

Under moderate pruning, the method achieves:

up to 1.14× inference speedup
up to 11.5% reduction in memory usage
improved throughput across different sequence lengths
limited degradation in model quality under moderate pruning

These results show that internal state redundancy can provide an opportunity for practical optimization of Mamba-based models.

Efficient LLM Inference and Mamba

State space models such as Mamba are increasingly relevant to research on large language models and efficient LLM inference.

As language models grow in size and context length, researchers are actively exploring architectures that can process long sequences with lower computational and memory overhead.

PerfMamba contributes to this area by examining how the internal components of Mamba behave at runtime and how selective pruning can reduce inference cost.

The work is therefore relevant to:

LLM efficiency
efficient LLM inference
long-context language models
inference optimization
sequence model acceleration
memory-efficient AI
scalable machine learning systems

Mamba vs Transformer Efficiency

Mamba and other state space models are often studied as alternatives to Transformer architectures because of their sequence-processing efficiency.

PerfMamba complements architectural comparisons by asking a systems-oriented question:

Where is computation actually spent inside Mamba, and which components can be optimized to improve real-world performance?

This performance-oriented perspective is important because theoretical complexity alone does not determine actual GPU runtime, memory behavior, or inference throughput.

Long-Sequence Modeling and Scaling

One of the key motivations for state space models is their ability to process long sequences efficiently.

PerfMamba evaluates Mamba models over sequence lengths from 64 to 16,384 tokens, providing insight into how execution time, memory use, and resource utilization change as workloads scale.

This makes the study relevant to:

long-sequence modeling
long-context models
scalable sequence models
efficient sequence processing
large language model systems
high-performance AI

Hardware-Aware Mamba Optimization

PerfMamba takes a systems-oriented approach to model optimization.

Rather than evaluating only predictive accuracy, the work considers:

computation
memory access
I/O
resource utilization
execution time
throughput
sequence-length scaling

This makes the work relevant to hardware-aware AI, GPU optimization, machine learning systems, high-performance computing, and AI systems research.

What Is PerfMamba?

PerfMamba is a performance-analysis and optimization study of Mamba-1 and Mamba-2 selective state space models.

The framework combines:

Mamba benchmarking → runtime profiling → bottleneck analysis → state pruning → inference acceleration → memory reduction

The goal is to understand how Mamba models behave on real hardware and identify opportunities for more efficient inference.

Why Does Mamba Pruning Matter?

Modern deep learning models often contain redundant computation.

For selective state space models, some internal states may contribute less to model behavior than others. Identifying and pruning low-activity states can potentially reduce computation and memory requirements.

PerfMamba demonstrates that this strategy can improve the efficiency of Mamba models while maintaining competitive model quality under moderate pruning.

Applications

The findings are relevant to researchers and engineers working on:

Mamba architectures
state space models
large language models
efficient LLM inference
long-context AI
model compression
neural network pruning
AI inference acceleration
GPU optimization
scalable machine learning
machine learning systems
high-performance computing
hardware-aware AI

Paper

PerfMamba: Performance Analysis and Pruning of Selective State Space Models

Abdullah Al Asif, Mobina Kashaniyan, Sixing Yu, Juan Pablo Muñoz, Ali Jannesari

Published in International Symposium on Benchmarking, Measuring and Optimization
Springer, Singapore, 2026
Pages 27–44

DOI:
https://doi.org/10.1007/978-981-95-9694-2_3

arXiv:
https://arxiv.org/abs/2511.22849

PDF:
https://mobinakashaniyan.github.io/papers/perfmamba-performance-analysis-pruning-selective-state-space-models.pdf

Keywords

PerfMamba, Mamba, Mamba architecture, Mamba model, Mamba-1, Mamba-2, Mamba performance, Mamba benchmarking, Mamba inference, Mamba optimization, Mamba pruning, Mamba efficiency, Mamba SSM, selective state space models, state space models, state-space models, SSM, SSM pruning, SSM optimization, SSM inference, efficient state space models, sequence models, sequence modeling, long sequence modeling, long-context models, Transformer alternatives, Mamba vs Transformer, large language models, LLM efficiency, efficient LLM inference, LLM inference optimization, AI inference, inference optimization, inference acceleration, model acceleration, model optimization, model pruning, neural network pruning, state pruning, model compression, neural network compression, efficient AI, efficient machine learning, efficient deep learning, model efficiency, memory efficiency, memory optimization, runtime performance, runtime profiling, performance analysis, performance optimization, AI benchmarking, GPU performance, GPU benchmarking, GPU optimization, hardware-aware AI, hardware-aware optimization, resource utilization, memory access patterns, I/O performance, sequence-length scaling, AI systems, machine learning systems, deep learning systems, high-performance computing, HPC, scalable machine learning.

Citation

Al Asif, A., Kashaniyan, M., Yu, S., Muñoz, J. P., & Jannesari, A. (2026). PerfMamba: Performance Analysis and Pruning of Selective State Space Models. In International Symposium on Benchmarking, Measuring and Optimization (pp. 27–44). Springer, Singapore. https://doi.org/10.1007/978-981-95-9694-2_3