Automation Bias

The tendency to trust AI output just because a computer generated it - even when it's wrong.

The simple explanation

Automation bias is a well-documented psychological tendency: people trust automated systems more than they should. It's the same instinct that makes drivers follow GPS into a lake, or pilots ignore their own instruments because the autopilot looks confident.

In AI coding, it shows up as rubber-stamping. An agent generates a diff, you glance at it, think "the AI probably got it right," and approve it without truly understanding what changed. The code looks clean, the structure seems reasonable, and you're busy - so you merge it.

The problem is that AI output looks more authoritative than it is. A well-formatted, well-structured piece of code feels trustworthy. But formatting and structure don't guarantee correctness. The AI might have made a subtle logic error, introduced a security vulnerability, or solved the wrong problem entirely.

Why it matters for agentic engineering

Automation bias is the biggest cultural risk in agentic engineering. If the point of having agents is to produce code faster, there's a natural pressure to also review code faster. But fast review + automation bias = shipping bugs.

The discipline of agentic engineering specifically requires fighting this tendency. Review AI code with the same rigor you'd apply to any code review - or arguably more rigor, because AI makes mistakes that humans typically don't (like hallucinating APIs or ignoring project-specific conventions it wasn't told about).

In practice

• Treat AI code like a junior dev's PR: You wouldn't LGTM a junior developer's first PR without careful review. Apply the same standard to AI output
• Read the diff, not the description: AI agents often produce articulate explanations of their changes. The explanation might be wrong, or might describe what the code should do rather than what it actually does. Always read the actual code
• Run the tests yourself: Don't just trust that the agent says "all tests pass." Verify it. Better yet, write tests for the specific behavior before the agent starts
• Check for what's missing: AI often produces code that handles the happy path well but misses edge cases, error states, or accessibility requirements. Actively look for gaps
• Slow down for critical code: Authentication, authorization, payment processing, data deletion - for high-stakes code, slow down your review proportionally to the risk
• Question confidence: When AI says "this is the correct approach," treat it as a hypothesis to verify, not a conclusion to accept