EvalScope
One-stop LLM evaluation framework from the ModelScope community.
About
EvalScope is an open-source framework from the ModelScope community, backed by Alibaba, for evaluating large models including LLMs, VLMs, and AIGC systems. It aims to make evaluation a one-command operation covering model capability benchmarks, inference performance stress testing, and result visualization.
Usage is pip-based: pip install evalscope, then evalscope eval against a model, whether local or behind an OpenAI-compatible API. It supports standard datasets like GSM8K and publishes documentation in both English and Chinese, positioning it as the evaluation companion to the ModelScope model ecosystem.
What it can do
- Capability evals for LLM, VLM, and AIGC models
- Inference performance stress testing
- Result visualization dashboards
- One-command runs against local models or API endpoints
Setup
Install with pip install evalscope, then run evalscope eval pointing at your model or API endpoint.
Prerequisites
Python 3.10+, and a model to evaluate (local weights or an API endpoint).
Use cases
Works with
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 95; Trending score 68/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.