How to Keep AI Agents From Writing Their Own Tests

Your CI went green and production still broke. That is the trap when an autonomous agent writes the tests your pipeline runs: the same bug slipped past it twice, and the suite kept saying everything was safe.

The fix is short, and it came out of half an hour of thinking about it. Put the eval contract first and the agent second — the golden outputs and the assertions sit outside the agent, and the agent only adds new tests while the old ones stay green.

Tutorial

1. Lock the golden outputs and the assertions outside the agent. Write them yourself and keep them where the agent cannot edit them.
2. Let the agent add new tests only while the existing tests stay green. If an old test is failing, nothing new gets added until it passes again.

Wrapping Up

Two rules, and the order carries most of the weight: contract first, agent second. With the assertions out of the agent's hands, the next test it writes has to earn its place against the ones you already trust.