RWKV-7 "Goose"
The seventh RWKV generation: a pure-RNN, attention-free architecture with dynamic state evolution, linear time and constant memory.
About
RWKV-7, codenamed "Goose", is the 2025 generation of the RWKV line from Bo Peng (BlinkDL) and the RWKV community. It keeps the project's founding bet: a 100% recurrent, attention-free architecture that trains like a Transformer but infers with constant memory and no KV cache. The v7 change is dynamic state evolution, a generalized delta rule with vector-valued gating that the authors prove crosses an expressivity barrier (TC0) that diagonal RNNs and linear attention do not. The paper landed on arXiv in March 2025, with weights on Hugging Face spanning 0.19B up to 7.2B parameters.
RWKV is a Linux Foundation AI & Data hosted project, the weights are Apache 2.0, and the ecosystem is unusually practical for an RNN: GGUF builds, an efficient-inference project (Albatross, reporting 145+ tok/s for the 7.2B on an RTX 5090 at constant VRAM), and a mobile app for on-device runs. The tradeoff is the usual one for RWKV: it is a community-driven project, and its standard-benchmark standing against frontier Transformers is asserted more than demonstrated. For long-context and edge inference where constant memory matters, it is the most mature non-Transformer option in the open.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Authors prove RWKV-7 crosses the TC0 expressivity barrier and recognizes all regular languages at constant depth (theory result, arXiv:2503.14456)
- Third-party inference report: 145+ tok/s for RWKV-7 7.2B fp16 on RTX 5090 at constant speed and VRAM (BlinkDL Albatross project, community measurement)
- No standard LM benchmark table (MMLU-class) reproduced from the paper here
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: paper arXiv:2503.14456 published Mar 18, 2025; RWKV is a Linux Foundation AI & Data hosted project (lfaidata.foundation); HF repo BlinkDL/rwkv-7-world (Apache 2.0, 108 likes); architecture corroborated by the RWKV wiki and community inference projects. Open weights. Strengths: genuinely novel architecture with theory backing, permissive license, mature edge/mobile ecosystem, constant-memory inference. Docked for: (1) standard-benchmark standing vs frontier Transformers is asserted rather than demonstrated with comparable tables; (2) community-driven maintenance, no corporate support. License: Apache-2.0.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 6.0/10
- Maintenance · recency, releases, issue load
- 7.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · Paper arXiv:2503.14456 published Mar 18, 2025 (RWKV-7 Goose with Expressive Dynamic State Evolution)
- · RWKV is a Linux Foundation AI & Data hosted project
- · HF repo BlinkDL/rwkv-7-world; Apache 2.0; 108 likes (HF API Oct 2026)
- · Sizes 0.19B-7.2B; community reports 145+ tok/s for 7.2B fp16 on RTX 5090 (Albatross)
- · Standard-benchmark standing vs frontier Transformers asserted rather than demonstrated