What are evals?
Evals are a set of test cases you run to check whether a change actually made things better.
~/ai-glossary
$ glossary
3 of 4 in #getting-good-output
Without them you are judging by whether the last answer felt good, which is how people convince themselves a worse prompt is an improvement.
Ten real examples with known-good answers is enough to start.
Was this page helpful?