ChatMusician
Open-source LLaMA2-based LLM that treats music as a second language via ABC notation: composes, answers theory questions, and chats.
About
ChatMusician takes the unusual route of teaching a language model music without any audio or specialized tokenizers: it continually pre-trains and fine-tunes LLaMA2 on ABC notation, the plain-text sheet-music format, so music becomes a second language alongside English. That lets it compose well-structured, full-length pieces conditioned on text, chords, melodies, motifs, and musical forms, while also answering music-theory questions, and the authors report it slightly beats the base LLaMA2 on MMLU too.
On the authors' MusicTheoryBench it beats LLaMA2 and GPT-3.5 on zero-shot questions by a clear margin, and its compositions reportedly surpass a GPT-4 baseline. Code, the 4B-token MusicPile corpus, and Hugging Face weights (m-a-p/ChatMusician) are public. The authors are candid about limits: much of the training data is Irish traditional music, the model hallucinates, and it is not meant for music education.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Author-reported (arXiv 2402.16153): beats LLaMA2 and GPT-3.5 on zero-shot MusicTheoryBench; MMLU slightly above base LLaMA2; composition quality beats GPT-4 baseline in author evaluations
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 7/10
Scored 7/10 (verified tier, metrics-v1). Evidence: paper (arXiv 2402.16153, Feb 2024) from Skywork AI and HKUST with MusicTheoryBench and MMLU results; code confirmed at github.com/jags111/chatmusician and weights at huggingface.co/m-a-p/ChatMusician; 4B-token MusicPile corpus released. Strengths: novel pure-text approach to music LLMs, full release of code/data/weights/benchmark, honest limitations section. Docked for: (1) repo license not explicitly confirmed in the sources checked — likely LLaMA2-derived terms; (2) training data skew toward Irish traditional music; (3) weak in-context learning per the authors. License: not explicitly confirmed — flag for review.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 6.0/10
- Maintenance · recency, releases, issue load
- 6.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · Paper arXiv 2402.16153 (Feb 2024), Skywork AI / HKUST
- · MusicTheoryBench beats LLaMA2 and GPT-3.5 zero-shot; MMLU slightly above base LLaMA2
- · Code at github.com/jags111/chatmusician; weights at huggingface.co/m-a-p/ChatMusician