Your evaluator now knows the answer

A small change to how PrepGraph grades your practice answers.

Marjuk · 2026-04-29 · 7 min read

Before PROMPT Question: How do you use JTBD to decide what to build? Hints: [3 abstract bullet points] Answer: "I start by identifying the user motivation..." EVALUATION OUTPUT Needs Work What was missing: • Could be more specific • Try to add more structure • Consider mentioning frameworks Invented standard. Vague. Inconsistent. After PROMPT Question: How do you use JTBD to decide what to build? Hints: [3 abstract bullet points] Benchmark — strong answer: "I use JTBD to shift from features to the progress users are trying to make..." Answer: "I start by identifying the user motivation..." EVALUATION OUTPUT Good What was missing: • Missed the switching moment anchor • Benchmark covers frequency x importance — your answer skipped the sizing step

Every question in PrepGraph — all 174 of them — has a hand-written model answer behind the "Show me a strong answer" button. Until this week, the AI evaluator never saw them.

What was happening

When you submitted an answer, Claude got the question, a short list of hints about what a strong answer covers, and your response. It returned a rating, a "what you got right" section, and a "what was missing" section.

It was inventing its own standard each time. The hints pointed in a direction, but a direction is not a benchmark. Feedback came out plausible but inconsistent, and "what was missing" often read like generic advice that could apply to almost any answer.

What changed

We pass the curated model answer into the evaluation prompt as calibration context, before asking Claude to grade your response. Same pattern as RAG, minus the retrieval step. The benchmark already existed per question; we just weren't sending it.

What you'll notice

"What was missing" is specific now. Instead of "try to be more structured," you'll see things like "the benchmark sizes the problem with frequency × importance, your answer skipped that step." Actionable rather than generic.

Ratings are stable. The same answer submitted twice gets the same rating, because both runs grade against the same fixed benchmark instead of improvising one.

The "Show me a strong answer" button is unchanged. It still surfaces the curated answer directly, not a generated one. The two features just point at the same source now.