Neuromodulated state space models: a dopamine-inspired selective gating framework with memory–stability trade-offs

Résumé fourni par la source

Dopamine shapes how the prefrontal cortex filters incoming information, maintains it in working memory, and releases it for action — three computational roles that closely parallel the input, state, and output stages of a sequence model. Here we ask whether formalizing this correspondence yields a sequence architecture that is both computationally competitive and mechanistically interpretable in biological terms. We introduce the Neuromodulated State Space Model (NeuSSM), which reinterprets the selective gating mechanism of modern state space models (e.g., Mamba) as three dopamine-inspired operators — sensory gating, working-memory modulation, and output relay — controlled by a single learnable gain parameter λ. We show theoretically that the classical S4 model arises as a limiting case of NeuSSM at λ = 0 (Theorem 1), and that NeuSSM’s local dynamical stability is governed by a phase-transition criterion based on the real part of the modulated state-transition eigenvalues, which partitions the model’s worst-case behavior into three qualitatively distinct regimes (Theorem 2). Mamba-style selective architectures are recovered qualitatively as λ → 1, though we present this correspondence empirically rather than as a formal approximation guarantee. Training NeuSSM on four long-range sequence benchmarks (≈ 0.45 M parameters, single consumer GPU), we find that the model learns different λ values in different layers depending on the task, with tasks requiring more flexible, long-range reasoning driving higher λ in upper layers — a pattern consistent with the proposed dynamical-stability trade-off (Theorem 2). NeuSSM reaches 98.75% on Sequential MNIST, 49.45% on LRA ListOps, 59.88% on Sequential CIFAR-10, and 76.0% on LRA Text, competitive with or approaching much larger S4 and Mamba baselines at a fraction of the parameter count. These results suggest that neuromodulatory principles from systems neuroscience can inform both the design and the interpretation of sequence models, even at modest computational scale.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.