AI-Generated Code Contains 70% More Errors Than Human Code, New Study Finds

Despite the skyrocketing adoption of AI coding tools among software developers, new research reveals significant quality concerns with AI-generated code. According to a recent study by CodeRabbit, AI-produced code contains substantially more errors than human-written code, raising questions about the technology’s current limitations.

Key Findings on AI vs. Human Code Quality

Google’s research earlier this year found that 90% of software developers now use AI tools on the job, a dramatic increase from just 14% last year. However, CodeRabbit’s analysis of 470 pull requests revealed troubling quality issues:

AI-generated code produced an average of 10.83 issues per request, compared to just 6.45 issues in human-authored code
This represents 70% more errors in AI code than in human-written code
AI code contained a higher rate of “critical” and “major” issues
Logic and correctness errors were particularly common in AI-generated code
Code quality and readability issues were identified as the biggest weaknesses, potentially creating long-term technical debt
Serious cybersecurity vulnerabilities were found, including improper password handling

The only area where AI code excelled was spelling accuracy, with humans twice as likely to introduce spelling errors.

Growing Concerns About AI Coding Tools

This study aligns with other recent research highlighting AI code generation problems:

Bain & Company reported in September that despite early adoption, the savings from AI in programming have been “unremarkable”
Security firm Apiiro found developers using AI produce ten times more security problems
A July study showed programmers were actually slowed down by AI assistance tools

Shifting Developer Responsibilities

As AI coding tools become more prevalent, the role of human developers appears to be evolving. Rather than being replaced, programmers may increasingly need to focus on reviewing, correcting, and improving AI-generated code.

CodeRabbit AI Director David Loker noted: “AI coding tools dramatically increase output, but they also introduce predictable, measurable weaknesses that organizations must actively mitigate.”

Conclusion

While AI coding tools offer impressive productivity potential, this research reveals significant quality trade-offs. Organizations implementing these technologies should establish robust review processes to catch the higher volume of errors and security issues that AI tends to introduce. The findings suggest that rather than replacing human programmers, AI may be changing the nature of their work toward quality control and error correction.