xLSTM-7b

In addition to the other non-transformer models in this episode, Sepp Hochreiter's NXAI has scaled up their xLSTM recipe to a 7B version. Although it outperforms other SMM models like FalconMamba or RWKV, it falls short compared to Transformer-based LLMs.