Efficient LLM Inference

An interpretable adaptive sampling method that dynamically allocates inference-time compute for efficient LLM test-time scaling.

Interpretable adaptive sampling for LLM test-time scaling and test-time compute. A fuzzy controller uses prompt complexity and model confidence to dynamically allocate inference-time compute for efficient LLM reasoning.

PerfMamba analyzes Mamba-1 and Mamba-2 selective state space models for efficient AI inference, runtime performance, memory efficiency, long-sequence scaling, GPU resource utilization, and state pruning. The work studies Mamba optimization, SSM pruning, model compression, and acceleration for efficient sequence models and LLM systems.