PerfMamba: Performance Analysis and Pruning of Selective State Space Models

Thu, 01 Jan 2026 00:00:00 +0000

This paper contributes to research on PerfMamba, Mamba-1, Mamba-2, selective state space models, performance analysis, model pruning, benchmarking, efficient AI, and high-performance computing by providing a systematic empirical study of the runtime behavior and optimization opportunities of Mamba-style architectures.

Recent sequence modeling research has introduced selective state space models as efficient alternatives to Transformer architectures. However, their real-world performance behavior, memory access patterns, I/O characteristics, resource utilization, and scaling properties require deeper analysis for effective deployment.

PerfMamba profiles Mamba-1 and Mamba-2 across sequence lengths from 64 to 16,384 tokens. The study analyzes computation patterns, memory behavior, I/O characteristics, and scaling trends to identify the components that dominate runtime and resource usage.

Based on these insights, the paper proposes a pruning technique that removes low-activity states within the SSM component. This supports improved throughput and reduced memory usage while maintaining accuracy under moderate pruning.

This work is relevant to researchers working on Mamba, selective state space models, state space models, large language models, sequence modeling, model pruning, model compression, runtime profiling, AI systems, neural network acceleration, and hardware-aware optimization.

Keywords: PerfMamba, Mamba, Mamba-1, Mamba-2, selective state space models, state space models, SSM, sequence modeling, Transformer alternatives, performance analysis, benchmarking, runtime profiling, resource utilization, memory access patterns, I/O characteristics, scaling analysis, model pruning, state pruning, model compression, efficient AI, high-performance computing, hardware-aware optimization, large language models, AI systems.

Citation:

Al Asif, A., Kashaniyan, M., Yu, S., Muñoz, J. P., & Jannesari, A. (2026). PerfMamba: Performance analysis and pruning of selective state space models. In International Symposium on Benchmarking, Measuring and Optimization (pp. 27–44). Springer, Singapore.

Neural Network Acceleration | Mobina Kashaniyan

PerfMamba: Performance Analysis and Pruning of Selective State Space Models