Of course, Qwen is also exploring different architectures, releasing a LLM with hybrid attention, consisting of Gated DeltaNet and Gated Attention. This model is trained on over 15T tokens and could be the groundwork for the next generation of Qwen models. Junyang Lin writes in a tweet about this series:
Specs
Params80B
LicenseApache-2.0
Tags
modelsmodels/llmsmodels/llms/reasoningmultilingualartifacts/15
Capability · Artificial Analysis
AA Index
13.7
Frontier lag
11.9 mo
o1-preview
12 Sep 2024
RAM score
—
Hugging Face Downloads
246.7K
last 30d
15.7M
all time
HF Likes
1K
Relative Adoption Metric contextualizes downloads against the model's size bucket.
VAIL Fingerprint
001e:002a:0044:005e:00b3:0175:0313:56b3
Explore other models with behavioral similarity to Qwen3-Next-80B-A3B-Instruct.
Related Models
