pengzhangzhi / awesome-mamba

Awesome list of papers that extend Mamba to various applications.

Awesome-Mamba

Awesome list of Mamba papers, theory, and applications!

Please issue/pr if you have any updates :)

Mamba

Mamba is a new state-space model architecture showing promising performance on language modeling with O(N) complexity.

mamba.py 🐍 : a simple and efficient Mamba implementation

Mamba-jax

Mamba-minimal-pytorch

Mamba-minimal-in-JAX

Mamba.c: inference of Mamba models in C and CUDA

Computer Vision

PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition

Efficient Visual Representation Learning with Bidirectional State Space Model

MambaMorph: a Mamba-based Backbone with Contrastive Feature Learning for Deformable MR-CT Registration

Vivim: a Video Vision Mamba for Medical Video Object Segmentation

SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation

VMamba: Visual State Space Model

U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation

Swin-UMamba

VM-UNet

ZigMa (ECCV 2024)

I2I-Mamba: Multi-modal Medical Image Synthesis via Selective State Space Modeling [code]

SUM: Saliency Unification through Mamba for Visual Attention Modeling [code] (WACV2025)

DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesis [code]

NLP

MambaByte: Token-free Selective State Space Model

MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts

BlackMamba: Mixture of Experts for State-Space Models

ClinicalMamba: A Generative Clinical Language Model on Longitudinal Clinical Notes

Repeat After Me: Transformers are Better than State Space Models at Copying

Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks

LOCOST: State-Space Models for Long Document Abstractive Summarization

Graph

Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces

Graph Mamba: Towards Learning on Graphs with State Space Models

Theory

Universality-1

Universality-2

StableSSM

Generalization

Medical

• Computation-Efficient Era: A Comprehensive Survey of State Space Models in Medical Image Analysis [Github]

• BioMamba: A Pre-trained Biomedical Language Representation Model Leveraging Mamba

Acknowledgement

• @alxndrTL
• @GianlucaMancusi

awesome-mamba's Issues

Contributions

MambaByte: https://github.com/kyegomez/MambaByte
MambaMorph: https://github.com/Guo-Stone/MambaMorph
Vivim: https://github.com/scott-yjyang/Vivim
SegMamba: https://github.com/ge-xing/SegMamba

mamba.py: https://github.com/alxndrTL/mamba.py

The Hidden Attention of Mamba Models

Thank you for creating this excellent resource for the Mamba architecture.

Here is a recent paper investigating the interpretability of these models, analogous to the attention mechanism in Transformers. I think it is highly relevant for understanding the mechanisms of Mamba models. https://arxiv.org/pdf/2403.01590.pdf

I2I-Mamba: Multi-modal medical image synthesis via selective state space modeling

Hi,

Please consider including our recent work, I2I-Mamba, in your collection. Here are the links to the preprint and the GitHub repository:

ArXiv preprint: https://arxiv.org/abs/2405.14022
GitHub repo: https://github.com/icon-lab/I2I-Mamba

Thank you for this collection.

Repeat After Me: Transformers are Better than State Space Models at Copying

I believe it is essential also to comprehend the limits of Mamba.
This paper shows that any state-space model fails to solve the copy task unless its latent state grows linearly with the sequence length: https://arxiv.org/pdf/2402.01032.pdf