A plan the specs wrote, run with an AI QA partner, one screen at a time

Every screen is built. Every report renders. The build phase is done, and now I have to find out if any of it actually works.

My first instinct is the obvious one. Open the app, click through one happy path start to finish, call it tested. I actually start doing exactly that. Then I land on one screen, the one where a session gets registered and paid, and I start counting the ways through it. Individual or a couple. Paid in cash, by transfer, or split across both. Invoiced or not, to the patient or to someone else. Paid in full, in part, or not charged at all. A credit applied. A discount. Edited afterward. Written off entirely. A dozen paths and more, on one screen alone.

A single happy path isn't testing. It's a smoke test wearing testing's clothes.

The screen that changed my mind

I keep looking at that one screen. Every one of those paths is a real thing a user of this kind of app does: a couple session billed to one partner, a payment split between cash and transfer because that's how the money actually arrived, a session marked uncollectable months later, a deposit that gets confirmed today and reconciled against a bank transfer next week. None of them are edge cases in the dismissive sense. They're the actual job.

The tool that keeps this from becoming infinite is one I'd half forgotten: equivalence partitioning. You don't test every possible dollar amount, you test one representative of each class that behaves differently (a normal charge, a zero-cost session, a split that sums under the cost, a split that sums over it). The dozen-plus paths on that one screen stop being an infinite space and become a finite, countable checklist. That's true on paper. It only becomes real once someone, or something, actually derives the list.

Two different things

Before writing a single test case, one distinction had to be nailed down, because it decides everything downstream. A flow you want to exercise is a test case. It belongs on a checklist, gets checked off, and that's the end of it. A thing that's actually broken is a bug. It belongs in the tracker, gets a priority, gets fixed.

Collapse that distinction and the tracker fills with noise. Ten screens' worth of flows, filed as tickets instead of checklist lines, would have buried the real defects under a hundred entries that are really just "verify this one path still works." Keeping the two separate meant the checklist could get long, because it needed to, without the tracker getting long too.

Letting the specs write the plan

So I fanned out. Four AI readers, in parallel, each pointed at the same set of specs: the acceptance criteria, the domain model, the workflow diagrams, the interface requirements, cross-checked against the code as it actually stands. Each one covers its own slice of the screens and pulls out every flow, every variant, every edge case, every failure mode, tagged back to the specific requirement it verifies.

Minutes later there's one plan. Around 320 test cases across ten screens, organized as happy paths, variants, edge and boundary conditions, failure paths, and a set of cross-cutting checks that apply to every screen (does it work at every viewport, is every string actually in the right language, do the totals add up across screens). Not invented from memory. Derived from documents that already existed, most of them written weeks before the first screen was even built. The plan doubles as the regression suite for whatever ships next.

The same read surfaced something I wasn't expecting: nine likely bugs, found by comparing the built code against the spec before a single click. A screen showing a raw internal identifier where a person's name should be. A billing default that's hardcoded instead of reading the setting meant to control it. A missing guard against linking a record to itself. I'm careful about how I say this: these are leads from reading code against spec, not confirmed failures. The actual passes will confirm them or clear them. Still a strange thing, having a list of likely defects before opening the app once.

Running the pass with a partner

The plan tells you what to check. Someone still has to check it. That part stays manual, on purpose. I drive the app by hand, screen by screen, clicking through the actual checklist. What changes is what happens next to what I find.

Each screen gets its own ticket and its own short, focused session with an AI partner, run from one reusable prompt where only the ticket id changes between screens. I describe what I did, what I expected, what actually happened. It decides pass or fail, and when something fails, it writes up the bug (the steps, expected versus actual, the viewport, the severity, the requirement it traces back to) and files it under that screen's ticket, tagged from a crash down to cosmetic. I keep driving. It keeps capturing, triaging, filing. Nothing gets fixed in that session. That's deliberate too. The pass is about finding out what's true, not about patching it as I go.

This is just how testing works

None of this is a new idea. A test plan as the coverage map, a ticket per area of tracked work, defects kept separate from the checklist that surfaces them, every case traced back to a requirement instead of invented from memory: that's how testing has always been supposed to work when it's done properly. What's different is the cost of doing it properly. Deriving a 320-case plan from a stack of specs used to be days of careful reading. Running a rigorous, fully documented pass on every screen used to cost more than most solo projects could afford, so people cut corners and told themselves a happy-path click-through counted as testing.

That's the actual counter to the version of this where an AI "tests the app" by clicking around and calling whatever it notices a bug. The plan doesn't come from the AI improvising. It comes from the specs, the same ones that shaped everything before this point. The AI just makes deriving it, and running it with discipline, as cheap as skipping it used to be.

The plan is done. The passes start next. I don't know yet what they'll find. I do know that whatever they find will trace back to something I wrote down weeks ago, before I'd opened the app once to test it.

#BuildInPublic #ClaudeCode #SoftwareTesting #AIAgenticEngineer