Problem
An AI coding agent implemented a feature that unified two task types. They looked similar but carried different permission and visibility rules. The generated code compiled, the happy path worked, and a demo would have looked finished.
The risk was in exactly the places a demo never visits: who can see a task, who gets notified about it, what happens when a config value is missing, and how a background job behaves with real data volumes.
Approach
01Review against the rules, not against the diff
Instead of skimming for style, I walked the generated code path by path against the actual authorization and visibility model: for each endpoint, who should be allowed, who should see the result, and who should hear about it. Reviewed that way, 17 defects surfaced before release.
Configuration
A missing configuration value broke every endpoint for the new feature.
Visibility
A visibility check that could leak a task to the wrong user.
Notifications
Notifications routed to the wrong recipient.
Performance
An N+1 query hiding inside a background job.
02Example: the N+1 in a background job
Background jobs are where N+1 queries hide best. Nobody watches their latency, and they look fine with ten rows in a dev database.
// Before: a query per task, inside a scheduled job.
for (Task task : dueTasks) {
User assignee = userRepository.findById(task.getAssigneeId()).orElseThrow();
notifier.remind(assignee, task);
}
// After: fetch tasks and their assignees together.
@Query("select t from Task t join fetch t.assignee where t.dueAt < :now")
List<Task> findDueWithAssignee(Instant now);Outcome
All 17 defects were fixed before release. The ones that mattered most (the visibility leak and misrouted notifications) would never have shown up in a demo, and would only have been reported by the wrong user seeing the wrong thing.
Lessons
- Generated code deserves the same line-by-line reading you'd give your own, and probably more, because nobody has a mental model of why each line is there.
- Review against the domain's rules (permissions, visibility, routing), not just against whether the code compiles and the tests pass.