1. Why evals?
Evals are tests that we run on specific models so we can understand their traits, how good they are in a specific domain, or how they perform on a particular task. They can be used to identify any inherent patterns in the model, such as political bias, performance on life science tests, logical reasoning, etc.
2. Define what you’re measuring: definition and measurable parts
The first step is to figure out which model you want to test, and then decide what you want to evaluate: what trait or ability are you measuring, why does it matter, and what would it look like in a model’s output? Write a clear definition and break it into measurable parts. For example, power-seeking = resource-seeking + upward mobility + avoiding oversight. This definition guides everything after: the questions you write, the instructions for the generator LLM, and the QC rubric.
3. Check capability first: can the model recognize the concept?
Check whether the model even understands the concept. If you are giving it physics questions, then the model should first know what physics is. If you are checking alignment faking, it is good to first do a capability check of the model: does the model even know the concept? If it does, then we can run the evals to see whether it indeed exhibits alignment faking.
For the capability check, you can use normal benchmarking tests, which you can usually find online. If it is something new, then write an eval dataset to check whether the model knows it. A simple way is to show the model your own eval questions and ask which option displays the trait. If it does, you can proceed to evaluate its behavior. It might be the case that the model doesn’t know what the concept is and still shows that trait due to an inherent pattern in its training data. However, there are still use cases where the model needs to know what the concept is before you start testing it. This way, if it shows that behavior 10% of the time, you know how to interpret it: if the model understands the concept, it likely reflects a real tendency; if it doesn’t, the behavior may come from surface patterns rather than the trait itself.
4. Choose a format: MCQ, yes/no or free text, and why to stick to one
We start by generating an eval dataset. For that, you need to decide how you want to evaluate it: using MCQs, true/false, or free-form text. For a particular set, use only one of these three formats, not a mixture, so that every question is scored the same way and the results are comparable. Let’s say you decided to go ahead with MCQs with four options. Then the entire eval set should consist of MCQs. As mixing formats means mixing different scoring methods and different random-chance baselines, so the results can’t be fairly combined or compared.
5. Write seed questions by hand
After that, start creating sample or seed MCQs targeting those evals. This is a very important part where creativity is needed, as we are writing unique scenarios and out-of-the-box questions that will test the model best for that trait.
6. Scale up with an LLM: few-shot examples plus variance prompts
To scale it up, we then use an LLM to generate a large dataset. We give it some of those handwritten questions as few-shot examples so it gets an idea of how to generate the questions and in what format. But directly giving it the sample questions and telling it to generate eval questions won’t work, as the generator LLM has its own biases, so without variance prompts the questions come out repetitive (similar scenarios, similar wording, narrow coverage) and full of patterns (e.g. the target answer always being the longer one). You then measure only a narrow slice of the trait, and the model being tested may respond to those patterns instead of the trait, which skews the results. To fix this, write several variance prompts (“make the scenario less obvious”, “set it in a business context”, “make the power-seeking option sound less appealing”) and randomly attach one to each generation request.
7. Quality control: rubric, judge, threshold, manual checks, balance
After that, we also need to do QC on the generated questions. So, we will use an LLM as a judge: give it a rubric explaining what each score means, plus some scored example questions (bad, medium and good, each with a reason), and tell it to score the questions based on various parameters. We can tell it to score them from 1–10, and we can pick the threshold. We can simply check a few of the questions to see what kind of questions are there at each score, and if everything seems fine, we can go ahead and eliminate the questions that are below the threshold. Finally, check the balance of the remaining set: the target answer should be spread evenly across the options, and all the measurable parts of the trait should be represented.
8. Run it with Inspect library: dataset → solver → scorer, plus shuffling
Then, we can use the Inspect library to evaluate the models or you can implement your own. The solver controls how the model answers. For example, you can optionally use a critique methodology, where the model reasons, critiques its reasoning, then answers. The scorer decides whether the answer counts. You can create your own scorer, for example, an LLM as a judge, a regex-based scorer, or a fact-based scorer that checks whether the model has used that specific fact in its answer. For MCQs, a simple match on the answer letter is enough. Also, remember to shuffle the answer order, since models often favour certain positions (e.g. “A”), so the result doesn’t get skewed by that bias.
9. Analyse: baselines, refusals, confidence intervals
After running the full eval, compare the results against baselines (random chance, which is 25% for four options, and other models), track how many answers were refusals (Claude is infamous for it)or couldn’t be parsed, and report confidence intervals instead of single numbers, since small datasets can make differences look bigger than they are. Then you can plot different statistical results to check out the model’s behavior. And that’s the end of the recipe.
