Prysm Bug Knocks Ethereum Consensus Participation After Fusaka

Shortly after the Fusaka network upgrade, the Ethereum network saw a sharp drop in validator participation after a bug in the Prysm consensus client knocked a chunk of votes offline.

According to a Thursday Prysm announcement, version v7.0.0 of the client unnecessarily generated old states while processing outdated attestations, a flaw that Prysm core developer Terence Tsao said prevented the nodes from functioning correctly. Developers recommended that users launch the client with the โ€œ–disable-last-epoch-targetsโ€ flag as a temporary workaround.

Beaconcha.in network data shows that at epoch 411,448, the network achieved only 75% sync participation (the percentage of 512 randomly selected nodes signing chain heads) and 74.7% voting participation. Voting participation being down 25% is under 9% shy of the network losing the two-thirds supermajority needed to maintain finality and regular operation.

At the time of writing, the current Ethereum network epoch (411,712) is experiencing nearly 99% voting participation and has reached 97% sync participation, indicating that the network has recovered. Prior to the issue, epochs routinely saw well over 99% of vote participation.

The decline in vote participation roughly matches the share of validators using the Prysm consensus client, estimated at 22.71% on Wednesday, before falling to 18% after the incident. This suggests that the attestation failure was likely concentrated among Prysm validators.

Client diversity chart. Source: MigaLabs

The Ethereum Foundation and Prysm developer organization Offchain Labs had not answered Cointelegraphโ€™s request for comment by publication.

Related: Exclusive data from EigenPhi reveals that sandwich attacks on Ethereum have waned

Brushing with finality loss

If voting participation falls below two-thirds of the total staked Ether (ETH), the Ethereum network loses finality. Under Ethereumโ€™s design, blocks can still be produced in that scenario, but the chain is no longer considered finalized.

As a likely consequence of such an outage, layer-2 bridges would freeze, rollups would pause withdrawals, and exchanges would increase their block confirmation requirements amid heightened risk of chain reorganization.

A similar incident that could lead to Ethereum losing finality is not purely theoretical. In early May 2023, the Ethereum mainnet lost finality โ€” an incident that occurred twice within 24 hours due to bugs in the handling of old-target attestations in the Prysm and Teku consensus clients.

The incident could have led to much worse consequences, since Prysm was estimated by its developers to run on over two-thirds of the consensus nodes back in September 2021. Data shared in January 2022 by Michael Sproul, a developer working on the current majority consensus client, Lighthouse, showed that Prysm was running on 68.1% of nodes.

Client diversity chart. Source: Michael Sproul

Related: Fusaka goes live as Ethereum edges closer to โ€˜instant feelโ€™ UX

Client diversity is still insufficient

While Ethereum consensus client diversity has made some progress since 2022, it is still far from achieving a client count under 33%, a limit that would ensure that a bug in a single client is not enough to halt network finality. Current MigaLabs data indicates that Lighthouse alone accounts for 52.55% of consensus nodes, with Prysm in second place at 18%.

Client diversity chart. Source: MigaLabs

That represents a deterioration from before the incident, when Lighthouse was below 48.5% and Prysm around 22.71%, according to MigaLabs.

Ethereum educator Anthony Sassano noted in an X post that โ€œif Lighthouse had had the bug instead, then the network wouldโ€™ve lost finalization.โ€

Magazine: Ethereum L2s will be interoperable โ€˜within monthsโ€™: Complete guide