AfriBERTa
Tiny 126M-parameter multilingual model proving small data works: pretrained on under 1GB of text across 11 African languages.
About
AfriBERTa is the low-resource landmark from the Masakhane community and the University of Waterloo's Castorini group. Its premise was deliberately contrarian: can a multilingual language model trained from scratch on less than 1GB of news text, covering 11 African languages like Amharic, Hausa, Igbo, Swahili, Yoruba, and Nigerian Pidgin, compete with giants trained on terabytes? The answer, published at the 2021 Multilingual Representation Learning workshop, was yes on the tasks that matter: named entity recognition and text classification, including on languages it never saw in pretraining.
Three sizes exist, Small (97M), Base (111M), and Large (126M), and the whole thing trains on modest hardware, which is exactly the point. AfriBERTa became a rallying reference for African NLP and the proof that communities without hyperscale compute can still build their own foundation models. It is an encoder for understanding tasks rather than a translation engine, but it anchors the low-resource multilingual story no catalog should skip.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Author-reported (MRL 2021): competitive NER and text-classification F1 on African languages vs mBERT and XLM-R, including unseen languages; PuoBERTa study reports AfriBERTa at 83.2 F1 on MasakhaNER datasets
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (verified tier, metrics-v1). Evidence: peer-reviewed MRL 2021 paper (Ogueji et al.) with NER and classification results confirmed via the Hugging Face model card and reproducing repo; cited as a foundational African-NLP reference. Strengths: landmark small-data result, three size options, trains on modest hardware, strong community significance. Docked for: (1) no license declared on the model card, creating commercial-use uncertainty; (2) encoder-only, so no generation or translation capability. License: undeclared on the model card — flag for review.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 5.0/10
- Openness · license + docs
- 8.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · MRL 2021 paper (Ogueji et al., Masakhane/Castorini); Small 97M / Base 111M / Large 126M
- · Pretrained on <1GB of text across 11 African languages
- · Competitive NER/text-classification vs mBERT and XLM-R, including unseen languages