Generative AI
LLM Evaluation
Measure whether your AI feature actually works. Test sets, automatic metrics, LLM-as-judge and regression testing.
- Advanced
- About 8 hours
- 3 modules
- 10 lessons
- Free
This course is not released yet. We publish courses one at a time, each one complete. The full syllabus is below.
What you will learn
- Why and WhatWhy Evaluation Is Hard, Building an Evaluation Set, Defining Success Criteria
- MethodsExact and Semantic Metrics, LLM-as-Judge, Human Review and more
- ProcessRegression Testing Prompts, Evaluation in CI, Online Evaluation and Feedback
Course syllabus
Not released yet · full syllabus below
- 1
Why and What
3 lessons- 1Why Evaluation Is HardSoon
- 2Building an Evaluation SetSoon
- 3Defining Success CriteriaSoon
- 2
Methods
4 lessons- 4Exact and Semantic MetricsSoon
- 5LLM-as-JudgeSoon
- 6Human ReviewSoon
- 7Evaluating RAG and AgentsSoon
- 3
Process
3 lessons- 8Regression Testing PromptsSoon
- 9Evaluation in CISoon
- 10Online Evaluation and FeedbackSoon