Let's be honest with ourselves, AI review is better than human right now, especially top tier models. Not saying that for important project like Chrome, you can skip reviewing changes. Far from it. But it is probably better and more thorough than most human devs already.
It’s not better than human review. It’s good at catching low-hanging issues but it is often requested overly-defensive code to handle edge cases that don’t exist or are handled elsewhere in the stack. It generates a ton of noise with low-signal comments. It basically cannot at all identify problems that pose the highest risk, like integrations, correctness from a business perspective, etc.
To be fair it’s also worth noting that it’s much easier to find a buggy edge case with existing code than it is to write bug free code that doesn’t have any edge cases at all. It’s so much easier to read some concrete logic and find holes in it, than it is to start from nothing and end up with perfection. It’s true both for humans and agents, but agents are better are validating correctness.
My own reviews have shifted now. I tend not to examine detailed semantics anymore. The AI is as good or better than me at assessing whether a chunk of code does what the author said it was supposed to do.
Instead my job is to spot design and architecture smells, broader semantic errors, violations of unspoken business requirements, etc.
For example, I was recently reviewing code that built out a transactional flow. Part of that flow involved recording the transaction somewhere user visible and I knew that should only happen after the transaction was confirmed. AI implemented it where the transaction was posted.
That starts as an issue of underspecified requirements but that always happens in the real world. Thus that's where I can provide the most valuable insight: assessing with that context, whether business, operational, historical, or forward looking.
