Evaluating LLM Output Quality

Build a golden set

Start with fifty to two hundred real inputs pulled from logs or written to cover known edge cases. For each, record what a good output looks like: sometimes an exact answer, more often a set of criteria such as "mentions the refund window" or "does not promise a delivery date." This golden set becomes the fixture you run on every prompt change, model swap, and vendor update.

Three layers of checking

• Assertions: cheap programmatic checks, valid JSON, required fields present, no forbidden phrases, length within bounds.
• Model-graded rubrics: a strong model scores outputs against written criteria; imperfect, but consistent and scalable.
• Human review: reserved for a sample, for disagreements between graders, and for calibrating the rubric itself.

Wire it into your workflow

Run assertions in CI so a prompt edit that breaks output format fails the build. Run the full eval suite before shipping any model change, and keep score history so you can see drift over time. In production, sample a slice of live traffic for ongoing grading, because real users will always find inputs your golden set missed.

None of this needs heavy tooling to start. A spreadsheet of inputs, a script that calls the model, and an afternoon of honest grading will change how your team makes decisions about AI features.