Sanity Context and content APIs (opens in a new tab)
The official reference for the tools used in this lesson.
Link
Evaluating retrieval
Before blaming the model for a bad answer, check whether the right passage was ever retrieved. Recall at k tells you which half of the system to fix.
Lesson 3.312m 19sAdvanced5.4k students
Chapters
12:19- 0:00Introduction
- 0:32BLEU Explained with Examples
- 2:09ROUGE Explained with Examples
- 3:36F1 Score Explained with Examples
- 5:10RAG Needs New Metrics
- 6:07Context Precision with Examples
- 6:53Context Recall with Examples
- 7:34Answer Relevance with Examples
- 8:07Faithfulness with Examples
- 8:40Groundedness with Examples
- 9:20LLM-as-a-Judge
- 10:17What RAGAS Is and Why It Matters
- 11:25Recap
What this lesson covers
- Build a query set with known correct passages
- Measure recall and precision at k
- Separate retrieval failures from generation failures
This lesson sits in Retrieval Quality, part of Retrieval-Augmented Generation from Scratch. It assumes what came before it and leads directly into the next lesson in the module.
In this lesson you will
- Build a query set with known correct passages
- Measure recall and precision at k
- Separate retrieval failures from generation failures
Resources
Personal notes are coming soon.