MambaVision-B-21K

The first hybrid Transformer-Mamba vision model. Similar to text-only models, they find that the combination of attention and mamba layers is superior compared to only using Mamba or attention layers. The accompanying paper goes into more detail, including ablation studies.