Emotion-LLaMA
Multimodal LLM for emotion recognition and reasoning — strong benchmarks, but read the validity caveats.
About
Emotion-LLaMA, accepted at NeurIPS 2024, is a multimodal large language model tuned for emotion understanding: it fuses audio, video and text through emotion-specific encoders and instruction-tunes a LLaMA backbone to both recognize emotions and explain its reasoning. It won the MER2024 challenge tracks and the follow-up Emotion-LLaMAv2 set new highs on the MMEVerse emotion benchmark.
The necessary honesty: emotion recognition is scientifically contested. A well-known line of research (Barrett et al.) argues facial expressions don't map reliably onto inner emotional states, and critics have documented validity and bias problems across the category — enough that the EU AI Act restricts emotion inference in workplaces and schools. The paper's numbers are real (EMER clue/label overlap 7.83/6.25, MER2023-SEMI F1 0.9036), but a benchmark score is not proof that 'detecting emotions' works as advertised. The code and model are BSD-3-Clause open source.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Paper-reported (arXiv 2406.11161): EMER Clue Overlap 7.83 / Label Overlap 6.25; MER2023-SEMI F1 0.9036; DFEW zero-shot UAR 45.59 / WAR 59.37
- MER2024 challenge (SZTU-CMU system): 1st place, 85.30% weighted F1 in MER-NOISE track
- Emotion-LLaMAv2: 66.63% on MMEVerse-Bench, ~12 points above the next model (author-reported)
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 6/10
Scored 6/10 (TIER metrics, metrics-v1). Evidence: peer-reviewed NeurIPS 2024 paper (arXiv 2406.11161) with EMER/MER2023/DFEW evals; BSD-3-Clause repo (627 stars); 1st place at MER2024 challenge. Strengths: open, challenge-winning, honest about multimodal difficulty. Docked for: (1) the underlying task — inferring emotions from expressions — is scientifically contested, capping real-world validity; (2) regulatory restrictions (EU AI Act) limit deployment contexts. License: BSD-3-Clause.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 5.0/10
- Maintenance · recency, releases, issue load
- 7.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 5.0/10
- · NeurIPS 2024 paper arXiv 2406.11161; EMER Clue/Label overlap 7.83/6.25; MER2023-SEMI F1 0.9036; DFEW zero-shot UAR 45.59 / WAR 59.37
- · BSD-3-Clause repo (627 stars); 1st place MER2024 challenge (85.30% weighted F1 MER-NOISE)
- · Emotion-from-expression inference is scientifically contested (Barrett et al.); EU AI Act restricts workplace/school emotion inference