HyenaDNA

HyenaDNA is a long-range genomic foundation model from Stanford's Hazy Research lab, built on the Hyena operator (implicit convolutions) instead of attention. Pretrained on the human reference genome, it processes up to 1 million tokens at single-nucleotide resolution — a 500x context increase over previous dense-attention genomics models — while training up to 160x faster than transformers thanks to subquadratic scaling.

On Nucleotide Transformer benchmarks it reached state of the art on 12 of 18 datasets with roughly 1500x fewer parameters and 3200x less pretraining data, and pioneered in-context learning in genomics: soft-prompt tokens let it adapt to novel tasks without weight updates. Published at NeurIPS 2023 (arXiv:2306.15794) under BSD-3-Clause with open checkpoints in tiny, small, medium, and large sizes, it remains the reference for ultra-long-context DNA modeling.