Answer check 01 · judged by a model
Data supports the answer
The question it answers, word for wordDoes the data actually returned by the queries support the answer as stated? Treat any figure in the answer that no query produced as unsupported.
- Level 2MetThe returned data directly supports the answer as stated.
- Level 1Worth a lookThe returned data is consistent with the answer, but the answer states more than the data shows.
- Level 0UnmetThe returned data does not support the answer.
One question, three outcomes
What each level looks like.
The check returns a weight for each level. The level with the most weight decides the outcome; the spread shows how sure it was. Every outcome below the top carries what it means for the number and one next step.
Asked
How much did we pay above the allowed amount for group 1042 in 2026?
Answered
$2.14M was paid above the allowed amount across 562 members.
Query that ran 1 row
select sum(paid_amt - allowed_amt) from claims.claim_line where group_id = 1042 and service_date >= '2026-01-01'
Data supports the answerUnmet
- 2%The returned data directly supports the answer as stated.
- 13%The returned data is consistent with the answer, but the answer states more than the data shows.
- 85%The returned data does not support the answer.
Concentration 0.77How sure the check was, shown beside the spread. Never a gate.
The answer states a member count, 562, that no query produced.
What it meansPart of this answer is not backed by any query.
Next stepAsk for the member count to be queried, or drop it from the answer.
Demo data
What it catches
The defects it flags.
- A figure the answer states that no query produced
- A confident answer stated over zero rows
- A nullable flag filtered with an equals signAlso caught by another check
- A missing period filterAlso caught by another check
- A denominator that counts only members with claimsAlso caught by another check
What it cannot see
It reads the answer beside the rows that came back. If the query itself was wrong in a way the SQL does not show, the rows can support the answer perfectly and the figure can still be wrong. That is why the strongest result is “no problems found,” never “correct.”
How a finding is addedHow it is judged
Narrow questions, named answers.
- 01
Four questions, one call
The four judged checks are asked together against the same answer, the same question, and the same queries. They run in parallel and cannot see each other’s answers.
- 02
A level, not a score
Each check returns a weight for each of its three named levels. The top level is met, the bottom is unmet, and the middle is worth a look: a note, never a failure.
- 03
How sure, shown beside the spread
Concentration says how much of the weight sits on one level. It is shown, never used as a gate, and it is not the probability the answer is right.
- 04
Never the rows
The checks read the question, the answer, and the SQL. Result rows never leave for judgment.
Start with a number your team has to defend.
Bring that number, and see what the checks find in it.