A conversion of Qwen2.5 72B into the RKWV architecture, which is done by deleting the attention layers and replacing them with RKWV layers, followed by distillation of the logits from the original, transformer-based model. The implication of this process is that the knowledge is in the FeedForward layers, not the attention layers.
Specs
Params72B
LicenseTongyi Qianwen
Tags
Similarity · VAIL
VAIL Fingerprint
0085:00b2:00e0:0137:018d:0205:04a3:4f06
Explore other models with behavioral similarity to QRWKV-72B.
Adoption · Hugging Face
RAM score
Relative Adoption Metric not scored because required parameter, download, or API metadata is not cataloged. This is not a zero score.
Hugging Face Downloads
96
last 30d
18.6K
all time
HF Likes
69
Relative Adoption Metric not scored because required parameter, download, or API metadata is not cataloged. This is not a zero score.
Related Models
