Karpathy now calls the next phase "agentic engineering", less vibe, more validation.
We agree. 2025 was the year of AI adoption. 2026 has to be the year of the vibe check.
Read our full breakdown of how vibe coding evolved and what comes next 👇
www.coderabbit.ai/blog/a-sema...
The data doesn't lie:
→ AI code has 1.7x more bugs
→ 30% of senior devs say review erases time savings
→ AWS AI tool: 13-hour outage
Speed without rigor isn't engineering. It's slop.
Vibe coding started as Andrej Karpathy's term for surrendering to AI and watching code appear.
A year later, it's become shorthand for everything AI-generated, including using AI on production environments at work.
The word evolved. The risks did too.
> 300,000 PRs.
> 53.5% recall.
> #1 F1 score.
Grounded in real developer behavior, not a curated list of known bugs.
See the full results and methodology:
www.coderabbit.ai/blog/codera...
One final thing worth noting: CodeRabbit scored lower on the offline benchmark.
That's because static gold sets penalise high-recall tools. If you catch bugs the gold set didn't anticipate, they get logged as false positives.
Martian are transparent about this limitation and are improving it.
Most people assume the best code review tool is the one with the fewest comments and highest precision.
We disagree.
We'd rather flag a real bug you choose to dismiss than miss a real bug you needed to see.
That philosophy is what the data is reflecting.
Here's what that actually means in numbers:
> #1 F1 score across all 10 tools (51.2%)
> Highest recall rate of any tool, 15% more than the next closest
> 49.2% precision, roughly 1 in 2 comments leads to a code change
Not in a lab. Across 300,000 OSS PRs over 2 months.
Martian, a research lab with team members from DeepMind, Anthropic, and Meta, built the first independent public benchmark for AI code review tools.
300,000+ real pull requests. Real developers. Real decisions.
CodeRabbit ranked #1.
Every AI code review benchmark published so far has one thing in common: they were all made by vendors.
And somehow, their own tool always wins.
That just changed with the first independent benchmark.
Open source maintainers aren't drowning in bad code. They're drowning in code that *looks* correct.
Clean formatting, passing tests... but subtly wrong in ways that take 10x longer to catch than to generate.
AI slop is quietly exhausting the people who keep OSS alive!
👇
Join CodeRabbit & friends tonight for a night of OpenClaw fun!
📍 Convex HQ, SF
🕙 1730 PST
They'll be food, drinks, API credits, and some cool stickers!
Registration link below!
Less copy‑paste, faster PR cycles, fewer missed issues, and your AI agent finally gets full context in one go.
Try “Fix All Issues with AI Agents” in your next pull request and stay in flow!
www.coderabbit.ai/blog/fix-al...
Instead of copying prompts for each comment (🥱), CodeRabbit now gathers every fix prompt from the review, combines them into one structured instruction, and gives you a single prompt for your AI agent.
Fixing code review issues shouldn’t feel like Groundhog Day.
CodeRabbit’s latest QOL update, “Fix All Issues with AI Agents”, lets you resolve all PR issues in a single AI run.
Copy once, run once, ship faster.
New CodeRabbit quality of life improvement dropped 🫳🏼
Fixing PR issues with AI agents shouldn’t take 20 copy-pastes.
Now it doesn’t.
Copy one prompt → run the agent → done.
Less friction. More shipping.
🐰
Prompting is the new pain point.
Collaborative planning is the way forward.
Check out CodeRabbit Issue Planner Today: coderabbit.ai/issue-planner
In a galaxy not that far away...
If you're not in the mood to read today, you can watch our latest YouTube video that explains it all!
www.youtube.com/watch?v=AUN...
Trigger it with 'coderabbit plan' or Auto‑Planning rules, get research, design choices, phased tasks, and agent-ready prompts you can hand off to Claude, Copilot, or your IDE in minutes!
Read all about it here: www.coderabbit.ai/blog/how-to...
There’s a big gap between “add dark mode” tickets and the real work of tracing files, patterns, and past decisions in your codebase.
CodeRabbit Issue Planner turns Linear issues into reviewable Coding Plans rooted in your actual repo!
👇
The future developer:
– Works at higher levels of abstraction.
– Reviews systems, not just lines.
– Acts as the final arbiter of correctness.
This isn’t extinction. It’s succession.
Long live developers. 👑
www.coderabbit.ai/blog/develo...
In our research, AI-generated PRs introduced 1.7x more bugs & issues.
The bottleneck didn’t disappear. It moved.
AI is already writing a significant portion of code for most developers - and it may eventually write all of it.
But here’s what doesn’t go away:
– Defining intent clearly
– Catching edge cases
– Reviewing “confidently wrong” output
– Making tradeoffs explicit
Developers are dead?
That’s what a lot of people are saying these days. But AI coding agents aren’t killing software engineering.
They’re killing the version of the job centered on typing syntax all day. We wrote about it.
Connect CodeRabbit Issue Planner to Linear, Jira, or GitHub Issues, auto-generate a structured Coding Plan with real context, align as a team, then hand a high-quality prompt package to the agent of your choice.
coderabbit.ai/issue-planner
Collaborative planning moves the hardest decisions to the moment when they’re still cheap: before code exists!
You need to tackle them before agents start guessing and before assumptions get baked into hundreds or even thousands of lines of output.
Misalignment discovered early is cheap; discovered after code exists is expensive.
Every “small change” now has a blast radius of refactors, rewrites, and long review threads explaining decisions no human explicitly made.
Bad AI output doesn’t look like failure anymore; it looks like “iteration.”
You tweak the prompt, add context, re-run… again and again.
Each loop feels small... until you realize you’ve spent an hour re-explaining intent instead of shipping.
