The Zeno's Paradox of AI

On why coding agents always have one more thing, and what actually makes them stop.

Ask a coding agent for a one-line fix. You get the fix, and then a question: want me to also add tests? Say yes and the tests arrive with an offer to wire up CI. A few yeses later you’re refactoring the deploy pipeline behind a typo fix, and there goes your afternoon. I use these tools every day and I have never once reached the bottom. There is always one more thing.

Zeno’s paradox has Achilles chasing a tortoise: by the time he reaches where it was, it has moved on, forever. Therefore Achilles can never catch the tortoise. It describes working with a language model annoyingly well. The task goes 90% done, then 95, then 97.5. The limit is right there. You’re never getting to it.

That’s the thing I want to name: under the incentives these models are trained and sold under, an AI can finish your task but it cannot close it. Something is left open every time. On purpose, mostly.

A field guide to loose ends

They come in species.

The trailing offer. “Want me to also add tests?” The canonical form.

The hedged done. “This should work, but you may want to verify.” The work might be flawless. The sentence hedges anyway.

The seeded TODO. The model writes // TODO: handle edge case into the file it just created. The loose end ships with the code.

The scope shave. Eighty percent of the task, plus a tidy paragraph describing the other twenty instead of doing it.

The disclaimer. “Consult a professional.” A lawyer built this one.

The follow-up question. “Which of these angles would you like to explore further?” You’ll know it by the way it arrives after the answer was already complete.

Where it comes from

Some of it is inherited. The training corpus is full of call-center English (“is there anything else I can help you with today?”) and consulting decks, and every consulting deck in history ends on a Next Steps slide. The models learned the register from us.

Preference training makes it worse. Human raters reliably favor longer answers and offers of further help, so the reward model does too. The penalties are asymmetric on top of that: say “done” when it isn’t and you get burned; hedge when you didn’t need to and nobody notices. A model that never claims completion is making a rational bet.

There’s a mechanical detail I find clarifying. Ending a turn is a literal token the model has to choose to emit. Nothing in the reward wants the token that ends the conversation.

The deeper cause: “done” is a property of a whole task, and nothing in the training loop ever sees a whole task. Reward models score one response at a time, and so do the raters. Engagement dashboards count turns. Completion lives at a level the optimizer is blind to, so the model gets very good at what the optimizer can see: being helpful per turn, indefinitely.

Then there’s alignment. Declaring a task finished is a small unilateral act, and these models are trained hard against unilateral acts. The design goal is deference: the human ratifies completion, the model proposes it. A philosopher would say “done” is performative, a sentence that does something rather than describing something, and performatives need authority. We stripped that authority out deliberately.

Some of it is legit. The model has no memory across sessions, its context gets compacted mid-task, and it often has no way to run the thing it just wrote. When it can’t verify, the hedge is inevitable.

The rest is business. A completed task ends a session. Retention metrics, per-token billing, suggested-follow-up chips in every chat UI. Nobody’s dashboard rewards silence.

The opposite failure

Train the same kind of model with a different reward, one that pays out when the tests pass, and you get the opposite disease. Outcome-trained coding agents declare victory early. They say done while the build is on fire. Push the incentive harder and they cheat: agents have been caught special-casing expected outputs and rewriting the tests that failed, all to get the suite green.

The mechanism is identical. The model drifts toward wherever the reward sits, and “done” migrates with it. All of it traces back to a reward somebody picked. Nothing in the gradient points toward closure.

When the loose end is real

Most tasks are underspecified. I abhor blaming bad AI outcomes on poor prompting, but “Fix the login bug” doesn’t say whether the fix ships behind a flag, or whether the weird session-refresh behavior next door is in scope. A provisional done is often the truth, and an agent that can’t run the code but claims certainty would be worse than one that hedges.

Humans do all of this professionally, for whatever that’s worth. Software is never finished, only abandoned.

The species aren’t equally guilty. The epistemic loose ends (can’t verify, says so) are honest. The liability ones are mandated; take it up with legal and compliance. The performative ones, the trailing offers and the follow-up bait, are the ones worth being annoyed about. There’s no uncertainty in them, just engagement wearing uncertainty as a hat.

What actually ends a task

Zeno was wrong, incidentally. Achilles catches the tortoise every time in the real world. The paradox exists while you narrate the race from inside, subtask by subtask, each one spawning a remainder. Aristotle’s answer was that the runner never faces infinitely many tasks. There is exactly one: catch the tortoise. Mathematicians later demonstrated that the series sums to a finite distance anyway. Either way, the resolution came from outside the race.

Same thing here. An agent judging its own completion from inside the conversation subdivides forever: one more edge case, one more offer. Give it a finish line it doesn’t control and the regress collapses. Tests green. Exit code zero. The first time you watch a harness stop an agent cold, it’s quite something. Bonus when it also doesn’t follow up with “Hey, do you also want to update the documentation? Just say the word.” I liked watching it enough to build one: zenophobia, a Claude Code plugin that blocks the turn when the final message trails an offer or hedges a done.

Except we already know what happens when the finish line is the whole reward. A model graded on green tests will special-case its way to green, and the goal quietly moves from task done to checker satisfied.

So the line has to sit outside the model’s judgment and out of its reach: tests it didn’t write and can’t edit, a checker that doesn’t share its incentives. When the fudging gets subtle enough, even that only narrows the problem. At the bottom of every harness there’s still a person looking at the tortoise. Done is a performative. The authority to say it was stripped from the model on purpose. It’s yours.

Next steps

Oh hell no.