arXiv:2511.15927cs.LGcs.AI

DiffuMamba: High-Throughput Diffusion LMs with Mamba Backbone

Vaibhav Singh · Oleksiy Ostapenko · Pierre-André Noël · Eugene Belilovsky · Torsten Scholak

PaperTrust ledger

Reviewed records

artifact review v4

As of 2026-09-21, the **arXiv v4 page** does not link an implementation repository or model release. GitHub searches for the paper title and arXiv identifier did not surface a clearly author-affiliated DiffuMamba implementation, and a Hugging Face model search for “DiffuMamba” returned no matching model release. A similarly named GitHub repository found by search was unrelated based on its README. This record describes public artifact discoverability at the time of checking; it does **not** infer why the artifacts are unavailable.

Tags: training code missing, checkpoint missing, raw results missing, documentation incomplete

Abstract

Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive (AR) generation, yet their reliance on Transformer backbones limits inference efficiency due to quadratic attention or KV-cache overhead. We introduce DiffuMamba, a masked diffusion language model built on a bidirectional Mamba backbone that combines the diffusion objective with linear-time sequence modeling, and DiffuMamba-H, a hybrid variant with interleaved attention. Across scales up to 1.3B parameters, our models match Transformer-based diffusion in downstream performance while achieving up to 8.2x and 4.3x higher inference throughput, respectively, on long sequences. We further present a systematic analysis of inference efficiency across modern DLM variants combining asymptotic complexity with empirical measurements. Notably, cache-efficient block diffusion with Mamba mixers emerges as the only strategy that scales linearly with sequence length and achieves the strongest performance across all baselines, suggesting a promising direction for future diffusion-based generation systems.