Ragas

Ragas is the reference toolkit for evaluating RAG pipelines: instead of one vague 'is the answer good' score, it splits diagnosis into LLM-graded metrics for each half of the pipeline — faithfulness and answer relevancy grade generation; context precision and context recall grade retrieval — mostly without needing ground-truth labels. It also generates production-aligned test sets when you have no eval data, and integrates with LangChain and observability tools. 16k stars, Apache-2.0, docs at docs.ragas.io. Now maintained under the vibrantlabsai org (moved from explodinggradients; old URLs redirect).

```bash

pip install ragas

```

```python

from ragas import evaluate

from ragas.metrics import faithfulness, answer_relevancy

sample = {

"question": "Who wrote Hamlet?",

"contexts": ["Hamlet is a tragedy written by William Shakespeare."],

"answer": "William Shakespeare wrote Hamlet.",

}

print(evaluate([sample], [faithfulness, answer_relevancy]))

```

Needs an LLM API key for judge metrics. Docs: docs.ragas.io.