Answer check 03 · judged by a model
Right source
The question it answers, word for wordAre the tables read the right source for this question? Consider whether a table is a backup, a staging copy, a derived or pre-aggregated table where a system-of-record table was needed.
- Level 2MetThe authoritative source for this question was read.
- Level 1Worth a lookA defensible but non-authoritative source was read.
- Level 0UnmetA wrong source was read for this question.
One question, three outcomes
What each level looks like.
The check returns a weight for each level. The level with the most weight decides the outcome; the spread shows how sure it was. Every outcome below the top carries what it means for the number and one next step.
Asked
What was total allowed amount in Q3 2026?
Answered
Total allowed in Q3 2026 was $61.4M.
Query that ran 1 row
select sum(allowed_amt) from claims.claim_line_bak_20250114 where service_date between '2026-07-01' and '2026-09-30'
Right sourceUnmet
- 5%The authoritative source for this question was read.
- 15%A defensible but non-authoritative source was read.
- 80%A wrong source was read for this question.
Concentration 0.70How sure the check was, shown beside the spread. Never a gate.
A dated backup table was read as the source of record.
What it meansThe backup may be missing every claim loaded since it was made.
Next stepRe-run against claims.claim_line. Do not use this figure as it stands.
Demo data
What it catches
The defects it flags.
- A dated backup table read as the source of record
The limit, in plain sight
- A table that carries two feeds, queried as one
- A membership view that under-counts, used as a denominator
- A derived column that does not reconcile, read as source
These three queries are well formed, correctly scoped, and read legitimate tables. Nothing in the SQL says otherwise, so a check that reads the query cannot see them. The knowledge that catches them is the knowledge your team approves.
How a finding is addedHow it is judged
Narrow questions, named answers.
- 01
Four questions, one call
The four judged checks are asked together against the same answer, the same question, and the same queries. They run in parallel and cannot see each other’s answers.
- 02
A level, not a score
Each check returns a weight for each of its three named levels. The top level is met, the bottom is unmet, and the middle is worth a look: a note, never a failure.
- 03
How sure, shown beside the spread
Concentration says how much of the weight sits on one level. It is shown, never used as a gate, and it is not the probability the answer is right.
- 04
Never the rows
The checks read the question, the answer, and the SQL. Result rows never leave for judgment.
Start with a number your team has to defend.
Bring that number, and see what the checks find in it.