Ethereum Foundation's AI Agents Found Real Bugs in Protocol Code

Ethereum News

AI agents are no longer just writing code. The Ethereum Foundation's Protocol Security team has been pointing them at Ethereum's own infrastructure, and they found real vulnerabilities.

AI Agents Found Real Bugs, but Triage Did the Heavy Lifting

The team said the finding itself was not the surprise. What caught their attention was how little time went into discovering it compared to the work required to determine whether it was real. According to the post, most candidates produced by the agents were wrong, duplicated, or out of scope. Sorting the genuine bugs from the convincing-looking false positives consumed far more of the team's effort than generating candidates in the first place.

The setup the team used runs many agents in parallel against a single target. Rather than using a central coordinator, the agents share state through the repository itself, writing claims to version control where other agents can see them. The team said it adapted this approach from Anthropic's published work on building a C compiler using a coordinated agent fleet.

Each agent takes on one of four roles depending on what the work demands. The recon role converts an attack surface into specific, testable hypotheses. The hunting role traces code paths and attempts to build a reproducer. The gap-filling role tracks what has been checked and generates the next batch of hypotheses. The validation role independently re-checks every candidate, removes duplicates, and makes the final call on whether something counts.

Before a candidate can be treated as a finding, it must meet a defined bar: a named target and entry point an attacker can actually reach, a specific property that must hold, a mechanism by which it could fail, an observable proof of failure, and a self-contained reproducer that runs against the real code for someone who did not write it.

False Positives Follow Predictable Patterns at High Volume

The reproducer requirement is the team's most important filter, because AI agents produce flawed versions just as quickly and confidently as valid ones. Three false positive types appeared repeatedly in the team's work. The first is a panic that only occurs in a debug build and disappears in production. The second is a reproducer that manually constructs an internal value no real input could ever reach. The third applies to formal verification, where a proof passes but does not actually constrain the behavior it was meant to cover. All three look credible in writing and fail only when run.

The team also mapped out where agents are genuinely useful versus where they mislead. Agents are effective at reading a specification alongside code, stating and checking real invariants, and drafting a reproducer from a one-line idea. They struggle with call chains that look reachable but are not, with accurately assessing severity, and with bugs that only appear across a sequence of valid steps where each individual action is correct. For that last category, the team said the agent's role is to suggest which sequences are worth running through a stateful test harness, not to replace the harness itself.

The team noted that acceptance rates vary significantly depending on the target. Heavily audited, mature code produces almost no survivors, which the team described as a valid result in itself. Less-explored code, and formally verified code where a machine-checked proof covers a model but the deployed bytecode is only assumed to match it, produces more. Cloudflare and Anthropic's own security research teams have described the same pattern, with Anthropic's property-based-testing agent generating roughly 1,000 candidate reports before expert review narrowed them to a top tier that held up about 86% of the time.

The post concludes that AI has not replaced the security researcher but moved where their time goes. The effort that previously went into generating and chasing hypotheses now goes into building verification infrastructure, maintaining lists of known issues, and handling disclosure. The team described this as a better location for human judgment, while being clear that it remains a bottleneck. The practices involved—reproducible failures, real oracles, careful triage—are the same ones that turned fuzzing into standard security practice over the past 15 years. The tools are new; the discipline is not.