Evals Foundations: A free evals course - Braintrust
Evals
Learn how the best teams, including Ramp, Notion, and OpenAI, ship quality AI products. In this course, you'll build a customer support chatbot from scratch and improve it with evals.
LLMs are probabilistic. The same prompt can return different answers on different runs, and one prompt tweak can improve one example but make others regress. Evals are how teams measure output quality and catch regressions before users do.
In this course, you'll build a customer chatbot and learn to build evals the way top teams do to ship reliable products. Specifically, you'll work on writing deterministic and LLM-as-judge scorers, learn to compare prompt variants, and turn production traces into test cases.
14 modules across three sections — Learn, Build, Refine - taking about an hour. No prior evals experience needed.
Learn2 modules
01Why are evals important?02What is an eval?
Build7 modules
03Simple eval using the UI04Comparing experiments05Playgrounds vs experiments06Simple eval using the SDK07How to deal with nondeterminism08How to read a trace09How to analyze your eval results
Refine5 modules
10Building a multi-turn chat app11Analyzing multi-turn traces12Online scoring13Analyzing production logs14The eval improvement loop
Trace everything
Sign up
- Resources
- DocumentationComplete guide to Braintrust AI evaluation platform
- IntegrationsAI provider and SDK framework integration guides
- Eval foundationsLearn how to build evals with Braintrust
- EncyclopediaA comprehensive encyclopedia of eval terms
- CookbookCode examples and practical recipes
- ChangelogLatest updates and feature releases
- For PMsHow product managers can leverage Braintrust
-
ArticlesIn-depth articles and insights
-
Community
- GitHubOpen source libraries and tools
- DiscordJoin our developer community
- NewsletterSubscribe to our newsletter for updates
- XLatest news and updates
- YouTubeVideo tutorials and demos
- LinkedInFollow us for company updates
Copyright ©2026 Braintrust Data, Inc.