Deepchecks
Open-source testing for ML and LLMs — 50+ checks for data, drift, and model quality.
About
Deepchecks is an Apache-2.0 open-source validation suite (4,000+ GitHub stars) with 50+ built-in checks covering data integrity, distribution drift, and model performance, plus a dedicated LLM evaluation module testing for hallucination, toxicity, PII exposure, and prompt-injection vulnerabilities. It is Python-native, plays well with pytest and Jupyter, and drops into CI/CD pipelines so every prompt or model change gets validated before deploy; a commercial cloud adds production monitoring.
The honest picture: the open-source core is strongest on classic ML validation — the LLM module is newer and less battle-tested. Serious production monitoring pushes you toward the paid cloud, and writing custom checks assumes Python comfort.
What it can do
- 50+ built-in data and model checks
- LLM hallucination and toxicity testing
- Distribution drift detection over time
- CI/CD pipeline integration
- Dataset quality and bias validation
Setup
pip install deepchecks. Import checks into pytest suites or Jupyter notebooks.
Prerequisites
Python. A model or dataset to validate.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 51; Trending score 68/100 from researcher assessment.
★ 4,000 GitHub stars
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.