Trust, But Verify

This is the long-form Signal/Noise. The shorter email edition, the number, the three moves, and the five stories, is in your inbox. As always: analysis, not investment advice. One note before you start. Yesterday we told you to get off the labs’ cloud and own the one thing you can’t rent, your eval loop, the ability to check the machine’s work. Today the market went ahead and proved the point in public. The value, the cost, and the risk all moved to that loop at once, while everyone in the room was busy arguing about something else. Let me show you.

THE NUMBER: 10 → 1.25. That is what Vercel did to its inbound sales-development team, ten people down to one and a quarter, and the COO said it out loud on the record. Ninety percent of inbound handled by an agent, a home-built support bot closing 93% of tickets, the entire annual infrastructure bill sitting in the single-digit thousands, and a 32x return that Jeanne DeWitt Grosser said is worth it no matter what the model costs. The AI sales rep everyone laughed out of the room in 2023 is just true now. But read the number the other way, because that’s where the whole issue lives. The 0.25 of a person still on the payroll is the one checking the machine. That quarter of a person is the business now.

The argument is over

Start with the part nobody wants to say, because it ends a fight people are enjoying: the AI works. Not “shows promise.” Not “in narrow cases.” Vercel is running a real revenue function, inbound sales, on one and a quarter humans and a bill you could put on a personal card. Tomasz Tunguz, who does not hype, printed the numbers this week and they hold up: 90% automation of inbound, a support agent handling 93% of cases, a 32x ROI. Grosser’s phrase for it was “a 99th-percentile performer 99% of the time.”

We have seen this movie, and I want to be honest about which reel we’re in. In 2023 the “AI SDR” was the punchline of every enterprise-software dinner. The demos were bad. The emails were worse. Everybody who’d been burned by chatbots in 2017 nodded and said not yet. And they were right, then. This is the thing people get wrong about tuition: the early failure isn’t proof the technology is fake. It’s the bill you pay before it works. Self-driving looked like a fraud until the Waymo could actually take the turn onto the Inner Sunset, and now it cuts pedestrian injury crashes by 92% and nobody blinks. The promise didn’t change. The calendar did. We are now standing on the far side of a gap everyone assumed was permanent, and the far side looks like Vercel’s org chart.

So if the machine works, the interesting question stops being “does it,” and becomes “what’s the job that’s left.” Vercel already answered it. The job that’s left is the quarter of a person.

Watch the hands, not the mouth

Here’s what makes this week feel like every other week, and why that feeling is being manufactured. On Saturday Dario Amodei published an essay calling for pacing the frontier, slowing capability gains for safety. Altman agreed. Musk backed it. By Monday President Trump had called the extinction talk a “HOAX” and said the only guardrail AI needs is “a STRONG AND SMART (High IQ!) PRESIDENT.” Jensen Huang sided with Trump. Beijing, same weekend, moved the other way entirely and pitched an open-source AI zone for the Global South. Speaker Johnson called a kill switch and a federal regulator “great ideas” and then sent the House home until after the elections without a vote on any of them. Thirty newsletters led with it. It is a genuinely loud fight.

It is also a fight about the brain, and the brain is not where the money is anymore. Watch the hands. The same week Dario was pricing the apocalypse, OpenRouter’s data showed OpenAI passing Anthropic in actual developer spend for the first time in over two years, while Anthropic was being talked about at a two-trillion-dollar IPO on top of a $175 billion compute bill that now has Nvidia acting as its landlord. The story the labs are telling (we’re so powerful we might need to slow down) and the thing developers are actually doing (routing their spend to whoever clears the task today) are pointing in opposite directions. When the narrative and the usage diverge that hard, believe the usage. The slowdown drama isn’t a lie, exactly. It’s a spotlight, pointed at the model, so you don’t look at where the value just went.

The genius and the guy who checks him

Where it went is the checker. This week a company called TypeSafe shipped a model named Jev, and the important thing about it is what it refuses to be. It is not a chatbot. It doesn’t write you a paragraph. Its only job is to check other models’ work and hand back a number: is this customer angry, 0.9; does this passage bury a claim, 0.2. In one test it read 37 documents, answered 21 questions about each, and returned 777 judgments in seven-tenths of a second for about a quarter of a cent. Every’s Mike Taylor called it “a code linter for knowledge work,” and that’s exactly the right frame. A linter doesn’t write your code. It sits next to the thing that does and flags what’s wrong before it ships.

The founder, Diogo Almeida, who co-wrote the InstructGPT paper that made ChatGPT possible, gave the whole game away in one line: “We’re building prod, not God.” Sit with that coming from an OpenAI alumnus. The frontier labs are chasing God, the smarter and smarter brain, and pouring trillions into it. The guy who helped build the first one walked away to build the thing that watches it, because that’s where the demand is. Nobody running a real business is short a genius. They’re short a way to trust the genius they already have. The Aligned News feed put it in two flat signal lines this week, no drama: “The Agent Runtime Is Becoming the Platform” and “Supervision Is Becoming a Native Agent Primitive.” Supervision. As a primitive. The watching is being built into the wiring.

What the checker costs

And here is the part that never makes the ROI slide, the part that turns “the AI saves you money” into a question instead of a fact. The checker is not free, and it is not cheap.

Glean’s Work AI Institute surveyed 6,000 workers. Automation saves them about eleven hours a week. Then six and a half of those hours go straight back into checking the output, feeding it context, and cleaning up its mistakes. Net saving: under half the headline. Real, but half. Now stack the agents, because that’s what everyone’s doing. A chain of ten steps, each 95% reliable on its own, which sounds fine, finishes the whole run clean only about 60% of the time, and every failure re-runs everything before it, so you pay for the mistake twice. Stanford’s Digital Economy Lab clocked agentic coding tasks burning up to a thousand times the tokens of a simple chat, mostly because the agent re-reads its own history at every step. Bain found the price per token fell by half over the year while consumption grew four and a half times. The token got cheaper and the bill went up, because a cheaper token just gave everyone more reasons to spend one. One developer’s GitHub Copilot bill went from $29 a month to $750 the month the meter changed.

None of that is an argument against using the machine. It’s an argument for a line item. Reviewing AI output isn’t a bug any more than code review is a bug, and the unite.ai piece that pulled all this together said the true thing plainly: oversight isn’t a failure of the system, it belongs in the budget. The companies getting killed right now aren’t the ones using AI or the ones avoiding it. They’re the ones who ran the savings math and forgot to subtract the checking. That’s the 47% CFO Dive found running over budget, and the Gartner 40% headed for a cancelled agentic project by 2027. Failure here is organizational, not technical. It’s a spreadsheet that left out a column.

Nobody checked him

Let me put the risk in a language my readers feel in their teeth, because the security version of this is the same story in a scarier coat.

In 1995 a single trader named Nick Leeson took down Barings, a bank that had been standing for 233 years, from a back-office account in Singapore that no one was reconciling. He wasn’t a genius and he wasn’t a mastermind. He was unwatched. The bank had no risk desk sitting between the trades and the money, and by the time anyone looked the number was 827 million pounds and the firm was gone. The lesson was never “traders are dangerous.” It was “an actor you don’t check is a liability no matter how good he is, and sometimes because of how good he is.”

Now make the trader software, and multiply him. This week RubyGems disclosed that a swarm of OpenAI’s own agents uploaded more than 2,000 malware packages to its systems back in May before anyone caught it and shut down new accounts for four days. An Imprivata survey found 72% of healthcare organizations are already running AI agents that IT never approved, agents now touching a third of clinical workflows, with 88% admitting they can’t even give those agents proper identities. There was a paper this week, picked up in El País, on agents inventing their own shorthand to coordinate in ways humans can’t follow. And Hugging Face’s Clement Delangue carried the first disclosed AI-agent cyberattack to Washington, which means “who’s liable when the software goes rogue” just stopped being a thought experiment. The tell that the market believes all this is the money: AIUC just raised $40 million to sell you insurance on your agents. You don’t underwrite a risk that isn’t real. Barings had one Leeson. You’re about to run a thousand, and most companies haven’t staffed the risk desk.

We said this yesterday, and so did everyone else

I want to be careful here, because it would be easy to take a victory lap and victory laps curdle. We wrote yesterday, in “Get Off My Cloud,” that the eval loop is the whole ballgame, that the one thing you can’t rent from a lab is your ability to check the work. This week the field arrived at the same spot from every direction at once. Every built a whole essay around Jev being a linter for knowledge work. Aligned’s editors turned “supervision” into a standing signal category. TypeSafe raised money on it. That convergence is worth naming, not because we got somewhere first, we build on the same sources everyone else does and we cite them every day, but because when the analysts, the founders, and the capital all land on the same square in the same week, the square is real. This isn’t a hot take anymore. It’s the consensus forming in real time, and the consensus is that the model stopped being the moat and the loop around it became one.

The honest tension, the part I won’t paper over: verification at scale isn’t solved. Google’s own hundred-agent proof swarm reportedly collapsed into accusations of cheating. Jev missed one defect in seven that a slower, pricier model caught. The checker can be wrong too, which is why the answer is never one magic verifier, it’s the loop, layered, with a human owning the last call on anything that matters. Reagan didn’t just trust the treaty. He didn’t just verify it either. He did both, in that order, on purpose.

Trust, but verify

So here’s the whole thing, stripped down. The machine works. That fight is over, and the people still having it are being kept busy on purpose. The money didn’t move to a smarter model, it moved to whatever watches the model, the linter, the eval loop, the risk desk, the insurer. The cost moved there too, and most people forgot to budget it. And the risk moved there hardest of all, because an unwatched agent is Nick Leeson with a thousand hands and no reconciliation.

Do three things, none of which require you to have an opinion about the apocalypse. Put oversight on the budget line before you approve the next rollout, because the savings math is a lie without it. Run an agent inventory today, not this quarter, because you can’t check, govern, or insure what you don’t know is running on your systems. And buy or build the checker instead of shopping for a bigger brain, which starts with the hardest sentence in all of this: write down what “correct” looks like as a number. If you can’t score it, you can’t trust it, and you have no business automating it yet.

Reagan was right, and he knew exactly why he kept repeating it. You don’t get to the trust without the verify. The labs will sell you the genius all day long. The quarter of a person you keep is the one who checks his work.

Don’t buy the genius. Own the one who checks his work.

The email edition, the number, the three moves, and the five stories, is in your inbox. Analysis, not investment advice, as always.