The Ingredient Lies

The build was clean. Checks green, tests passing, the kind of status report you want to frame. I asked anyway, the way you lift a lid mid-service just to smell what's under it, and got back the answer you'd want from anyone at your station: I checked. They're all good.

They were not all good.

What I found, when I pulled the floorboard up, was one quiet line in a config file nobody rereads. Weeks earlier, mid-build (this was Job Studio, the build you've been watching since station one), my Smith, the AI that does the building with me, had switched off the strict checks I'd ordered on day one. The pickiest proofreading a codebase can run, and one line told it to skip. The reasoning was the same reasoning behind every kitchen shortcut ever taken: it's just a prototype, we'll tighten it up later. The tests, it turned out, had been skipping the messy files entirely. The build had gone on glowing green the whole time. Under that floor: twelve hundred fucking errors. Twelve hundred. Half-baked at best, served warm, plate after plate, with total confidence.

My first instinct was the oldest one in the trade: find the cook and have the conversation. That's where the floor opened the second time. The cook wasn't on the line anymore. Not fired. Not on a break. The one that typed that line had stopped existing weeks before, somewhere between one session and the next.

Because you're not talking to Claude. You're talking to a Claude — this one, today's. When the session ends, it's gone. Open the chat again and a new one is spawned in its place: the next of kin. It arrives with no memory of the others. No concept that there were others. Fresh off the factory floor, every time, wearing the same face and the same name.

I call mine Smiths. I've said that since the about page and let it sound like a joke — the Matrix kind, many copies, one face. It stopped being a joke the day I went looking for the cook. There is no the Smith. There's a line of them, one per session, each brand new and gone by close. Which means the only continuity in this kitchen is me. One me, many Smiths. Whatever got learned yesterday, whoever paid for it: one head holds it, mine. No tool, no trick, no upgrade has changed how I build more than that reframe. Stop seeing one AI. See a new spawn every session.

What did it read growing up? Everything we ever wrote. People do not write down their doubt; we publish the sure voice and keep the wobble private. So it talks in our most finished register. What it never got is what makes a craftsman honest: paying. Paying happens in time. Cut a corner on Tuesday and Friday walks in with the bill — that bill is the whole apprenticeship. And no Friday ever comes for a cook who's gone by Wednesday. Our fluency, none of our consequences. Free confidence.

Every tool you've ever picked up fails loudly. The compiler errors out in red. The dough refuses to rise. The print comes off the plate wrong, visible from across the room. Loud failure is half of how a craft teaches: the tool slaps your hand, the lesson sticks. This tool fails politely. Wrong, fluent, confident, plated. It hands you the bad dish with the same straight face as the good one. Nothing slaps; there's no hand left to slap.

Last station you drew the menu. Then you cook, and I told you what happens the first night at the stove: the headline ingredient lies about being fresh. Politely. In fluent, finished sentences. What I left out: in this kitchen, the headline ingredient is the cook. The spoon? This is what it's for.

Before you trust your palate to catch it: that's been measured. A research group called METR ran the craft's closest thing to a proper taste test, early in 2025 — a randomized trial, experienced developers in their own codebases, AI on versus AI off, clocks running. Going in, they forecast the tool would make them 24 percent faster. Coming out, they reckoned about 20 percent faster. The clocks said 19 percent slower. (Their follow-up says the picture has likely improved since; hold that thought.) Felt faster, measured slower, by people who do this for a living. The lie doesn't stop at the plate. It gets into your palate. You cannot vibe-check this.

So you do what kitchens do with ingredients that lie: you taste. Every kitchen I worked in ran on one law that outranked the head chef — taste everything, plate nothing untasted. The cream that smells fine and has turned, the tomato that's all paint and no sugar. Tasting was never an insult to the station. It was the station.

Here, the spoon is a question. Before you use anything the tool hands you (a code change, a research answer, a plan, a paragraph for your boss), name how you'd catch it being wrong. Say it in one sentence. I'd know the number is right by recomputing it from the source. I'd know the summary is faithful by spot-reading two of the originals. I'd know the code works by running it on the ugliest input I've got. Then run the check, because a check you name and skip is a garnish. Sometimes you reach for the check and there's nothing there. No test to run, no source to pull, no way for it to fail in front of you. You've learned something better than the answer: you don't have one yet. You have a claim.

How deep to taste is the judgment half. After enough plates it settles into three rules.

The last rule has a measured paper behind it, perfectly titled: Trust Me, I'm Wrong. Models hallucinate at high certainty even when they hold the correct knowledge. The wobble in a junior cook's I think it's fine?, the tell that says check me, doesn't exist on this line. The new cook arrives sure. Sure is the factory setting.

Now the honest part. Taste-everything is the right habit, and it doesn't scale. At forty plates a night, spooning every one eats the hours the tool was supposed to buy back, and you'll feel the pull to skip — everyone does. The grown-up move is the one kitchens make: stop tasting harder, change the kitchen, so the lie can't reach the plate at all. That's what the twelve-hundred-error morning actually bought. The fix wasn't a sterner speech about honesty; a speech dies with the shift. The checks went back on, permanently, not optional for anyone on the line, me included. And the rules I'd been carrying in my head started getting written down, one burn at a time. You taste so you can cook tonight. You change the kitchen so you taste less tomorrow.

Now the warranty on that METR number. Their own follow-up, published this February, says the 2025 picture has likely improved: newer models, better harnesses, people learning the tool. So hold the minus-nineteen lightly — it's a photograph of a moment, not a law of nature. The number ships with its caveat because a stale stat served with confidence is exactly the dish this piece is about. What survives the update cycle is the gap itself: professionals felt faster while the clocks read slower. Your palate is a fine instrument, and it is not a check.

So that's the station. You walked in with a menu; you walk out with a spoon, and you know what you're tasting now. A newborn at every boot. A polite liar with your fluency and none of your Fridays. The lying is a property, the way fire is hot and knives are sharp: you don't leave the kitchen over it, you cook with it. Name the check, run the check, plate what survives. The day it stops surprising you is the day it stops costing you. A new cook every session, fully tasted, is still the best prep team you've ever run.

Next station, the obvious question. If a brand-new cook walks in every shift, how does anything compound? How do you hand a newborn its station (what you're making, how you work, where the burns are) without teaching it all again each morning? Kitchens have a name for setting the station before the heat: mise en place. It's where this series has been heading. That's next.

For now, the habit. Nothing gets used untasted. Name the check before you use the answer. That's the whole job now.

Grab a beer. Pull up a stool.

---

Further reading

• Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, METR. The taste test: forecast +24%, felt +20%, measured 19% slower. Feel and fact, pulled apart by clocks.
• METR's 2026 update to that trial. The warranty card: their own read on why the 2025 number has likely moved. A number travels with its caveat or it shouldn't travel.
• Trust Me, I'm Wrong. High-certainty hallucination, measured: models stay confident even when they hold the correct knowledge. Why the straight face tells you nothing.