What gets checked, and who decides.
Evals for the models I ship, limits for the agents I run, and a record of my own calls.
01 · wrong answers cost
Checking the model.
Borrow the bar from whoever would know
If the team writes the rubric, the model gets marked on our opinions. Examiners and regulators have spent years arguing about what good looks like, so I start from theirs.
Tutor: the exam boards' mark schemes. ward: KCSIE. Farewill: the regulator's rules.
Mark the conversation, not just the answer
A tutor that gets the answer right and hands it over has still failed. So the whole conversation gets marked, and the model doing the marking has to agree with teacher-marked sessions before it runs on its own.
Tutor: a judge on every session, against the Socratic spec.
Read the transcripts yourself
Evals catch what you thought to test. Reading real sessions catches the rest, and the failures that mattered were rarely on anyone's list.
Tutor: production sessions sorted into failure modes, and every mark a teacher flagged turned into a test.
Attack it before a student does
The failure that hurts is rarely a wrong fact. In a tutor it's a student talking the model into the answer. In safeguarding it's a disclosure worded to slip past the filter. So I red-team for those first, with cases written to break the rubric.
Tutor: red-teamed against the Socratic spec. ward: a held-out red-team set, all 16 disguised disclosures caught, against one for keywords.
Keep one suite you're failing
A regression suite should stay green. A second one you mostly fail shows where to go next. If everything passes, you've stopped learning.
This site: its agent's suite failed half its cases on the first run, and CI now blocks any change that breaks it.
02 · wrong answers cost
Checking the person who checks it.
Decide which way it's allowed to fail
Every model gets something wrong, so the product call is which kind of wrong you design for. In safeguarding that meant catching everything first, then cutting the noise.
Tutor and ward: recall first, then precision.
Budget the reviewer's attention
The right to overrule a model is worth little to someone paged all day. So the false-alarm rate is a product number, and someone owns it.
Tutor: conduct flags split from disclosures, so the designated lead is paged for the ones that need them.
Put a name on every call
Code can check a rule, but it can't own the outcome. Every decision that reaches a person carries their name and a reason.
Nursing prototype: a named clinician records each change. Lease prototype: the lawyer decides whether the wording holds.
03 · working with agents
Checking the agents I run.
An experiment: I gave an agent my email, calendar and work sessions, under the rules I'd put on any product that touches someone's data. Code enforces them, whatever the model says.
- Nothing sends anything as me. The worst it can do is draft something and hand it to me.
- Every fact carries the sentence it came from. If the source doesn't back it, it's thrown out and counted.
- Code does the counting. The model is never asked for a number.
- No part of it gets all three of the lethal trifecta: my private data, text written by strangers, and a way to send things out.
- What I say outranks anything the AI inferred. My corrections are kept and checked again every week.
what it refused, counted by code
A system that only reports what it did can't show it checks anything. So this counts what it refused, since 5 Oct.
- 746facts kept, each with the sentence it came from
- 24%of the facts it found, thrown out: 237 of 983, because the quote wasn't in the source
- 0messages sent as me. The worst it can do is draft one
- 8 → offmorning messages with no reply, so I switched them off on 16 October
| week of | checks run | facts kept | thrown out | evening wrap-ups |
|---|---|---|---|---|
| 5 Oct | 290181 found nothing new | 603 | 197 | 3 |
| 12 Oct | 362 | 65 | 30 | 5 |
| 19 Oct | 214 | 43 | 7 | 5 |
| 26 Octso far | 108 | 35 | 3 | 21 held back by its checks |
Week one was its first read of everything already there, so the counts drop after it. What would make it wrong: hundreds of checks and nothing kept, or the thrown-out share climbing. If the numbers show either, I cut it back.
The first morning brief fell back to the plain board text: under launchd the Claude CLI couldn't find node. Fixed the same day; the fallback is why a message still arrived.
The night pass stated a count it had worked out itself, and got it wrong. Anything numeric is now computed in code before the model sees it, and the model is told never to count.
04 · working with agents
Checking myself.
crux
I built crux to record the calls I make while a model types: what I pushed back on, what I redirected, what I let through. Run on my own sessions, the record came out thinner than I'd assumed, and thinnest on the moves that are hard to undo.
The weekly note
Once a week my system writes a short note on how I've actually been working with AI, drawn from my sessions, and every new session starts with it. When a line's wrong I correct it, and the correction is kept.