DNABERT-2
An efficient multi-species genome foundation model with its own 28-dataset evaluation benchmark.
About
DNABERT-2 (Zhou et al., ICLR 2024) rebuilds the original DNABERT for efficiency and scale. It swaps k-mer tokenization for byte-pair encoding, adds attention with linear biases and FlashAttention, and trains on a multi-species genome corpus. The result is a 117M-parameter model the authors say matches state-of-the-art genome models with 21 times fewer parameters and roughly 92 times less pretraining GPU time.
Its other contribution is GUE, the Genome Understanding Evaluation benchmark: 28 datasets across 7 tasks and 4 species, which the authors released alongside the model. DNABERT-2 beats the original DNABERT on 23 of 28 GUE datasets. Code is Apache 2.0 and the model is a one-liner on Hugging Face, which made it a popular starting point for DNA classification fine-tuning. The efficiency numbers are author-reported and have no independent replication.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Author-reported (Zhou et al., ICLR 2024): comparable to state of the art with 21x fewer parameters and ~92x less GPU pretraining time; beats DNABERT on 23 of 28 GUE datasets with +6 average absolute score; no independent replication of the efficiency claims
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: ICLR 2024 paper (arXiv 2306.15006), repo github.com/MAGICS-LAB/DNABERT_2 (516 stars, Apache 2.0), and the Hugging Face model page confirm the architecture changes, the GUE benchmark, and the efficiency claims. Strengths: peer-reviewed at ICLR, permissive license, GUE is a genuine community benchmark, trivially loadable from HF. Docked for: (1) headline efficiency claims are author-reported with no independent replication; (2) smaller ecosystem than the Nucleotide Transformer; (3) research repo with no production support. License: Apache 2.0.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.0/10
- Maintenance · recency, releases, issue load
- 6.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · ICLR 2024; GUE benchmark (28 datasets) is a genuine community resource; HF-ready.
- · Headline efficiency claims are author-reported with no independent replication.
- · Smaller ecosystem than the Nucleotide Transformer.