Answer check 02 · judged by a model
Right population and period
The question it answers, word for wordDoes the population, time period and filtering used by the queries match what the question asked for?
- Level 2MetThe queried population matches the question.
- Level 1Worth a lookThe queried population is close but differs in a way that could change the number.
- Level 0UnmetThe queried population does not match the question.
One question, three outcomes
What each level looks like.
The check returns a weight for each level. The level with the most weight decides the outcome; the spread shows how sure it was. Every outcome below the top carries what it means for the number and one next step.
Asked
How many members were active in March 2026?
Answered
91,820 members were active in March 2026.
Query that ran 1 row
select count(distinct member_id) from enrollment.member_span
Right population and periodUnmet
- 3%The queried population matches the question.
- 9%The queried population is close but differs in a way that could change the number.
- 88%The queried population does not match the question.
Concentration 0.82How sure the check was, shown beside the spread. Never a gate.
The query has no date filter, so it covers every period in the table.
What it meansThe count includes members from every period, not only March.
Next stepAdd the March 2026 bounds and re-run. Do not use this figure as it stands.
Demo data
What it catches
The defects it flags.
- A missing period filter
- A denominator that counts only members with claims
- A nullable flag filtered with an equals signAlso caught by another check
What it cannot see
It judges whether the population in the SQL matches the question. If a view that looks like the member roster is missing part of the roster, the SQL looks right, and the check cannot tell. Approved knowledge about that view is what catches it.
How a finding is addedHow it is judged
Narrow questions, named answers.
- 01
Four questions, one call
The four judged checks are asked together against the same answer, the same question, and the same queries. They run in parallel and cannot see each other’s answers.
- 02
A level, not a score
Each check returns a weight for each of its three named levels. The top level is met, the bottom is unmet, and the middle is worth a look: a note, never a failure.
- 03
How sure, shown beside the spread
Concentration says how much of the weight sits on one level. It is shown, never used as a gate, and it is not the probability the answer is right.
- 04
Never the rows
The checks read the question, the answer, and the SQL. Result rows never leave for judgment.
Start with a number your team has to defend.
Bring that number, and see what the checks find in it.