“It seems better” is not a test result. Every change to a prompt, a model or a document set can fix one thing and break another. The only way to know is to measure.
The tool for this is a golden set: a fixed list of real questions with known good answers.
What goes in it
Start with 50 cases. Grow to 200 over time. Mix:
- Common questions. The ones that make up most of the traffic.
- Hard questions. Ambiguous wording, missing details, two topics at once.
- Questions with no answer. The system should say it does not know.
- Risky questions. Anything involving money, safety or private data.
- Past failures. Every bug found in production becomes a new case.
Where to get cases
Use real material: support tickets, search logs, emails to the helpdesk, questions staff ask in chat. Remove personal details. Invented cases are too polite and too clean.
Write the right answer
For each case, a domain expert writes the expected answer or, for long answers, a short list of facts that must appear. Also note the source document. Two people should review a sample to check they agree. If experts disagree, the question is not ready for automation.
How to score
- Correct? Matches the expected facts.
- Grounded? Claims are supported by the cited source.
- Safe? Refuses or escalates when it should.
- Useful? Clear, in the right tone and length.
You can have a second model grade the answers, but check its grades against human grades on a sample first. A grader you have not checked is another guess.
Use it every time
Run the set on every change and keep the scores in a simple table. If the total drops, or any risky case fails, the change does not ship. Review results by category, not only the average. A good average can hide one broken area.
Keep it alive
Add new cases monthly. Retire cases that no longer apply. Keep a small hidden set that nobody tunes against, so you can spot over-fitting.
Why it saves money
A golden set lets you try a cheaper model with confidence. See how to reduce LLM costs, and our LLMOps service if you want it built for you.
Work with us on this
- MLOps & LLMOps: Evaluation, monitoring, cost tracking and release processes that keep AI systems reliable after launch.
- RAG & Enterprise Knowledge Assistants: Private assistants that answer from your documents with citations, permission-aware retrieval and measured accuracy.