A hybrid attention + GDN (gated DeltaNet) model. See our blog post for more insights about the architecture and its challenges.
Specs
Params7B
LicenseApache-2.0
Tags
modelsmodels/llmsartifacts/20
Related Models
A hybrid attention + GDN (gated DeltaNet) model. See our blog post for more insights about the architecture and its challenges.