Skip to content

fix(kona/derive): add over-fill check in BlobSource::load_blobs#19364

Merged
ajsutton merged 6 commits intodevelopfrom
fix/lr-f04-blobsource-overfill-check
Mar 12, 2026
Merged

fix(kona/derive): add over-fill check in BlobSource::load_blobs#19364
ajsutton merged 6 commits intodevelopfrom
fix/lr-f04-blobsource-overfill-check

Conversation

@ajsutton
Copy link
Contributor

@ajsutton ajsutton commented Mar 3, 2026

Summary

BlobSource::load_blobs was missing a post-loop over-fill check. After the blob-pointer fill loop completes, if blob_index < blobs.len() the blob provider returned more blobs than were requested. Previously kona silently discarded the extra blobs. This PR adds a guard that returns ResetError::BlobsOverFill (→ PipelineErrorKind::Reset) in that case, mirroring the op-node reference.

Background

The Go op-node performs this check in fillBlobPointers in blob_data_source.go:

if blobIndex != len(blobs) {
    return fmt.Errorf("got too many blobs")
}

This error is wrapped as NewResetError at the call site, causing the pipeline to reset and retry from a clean state. Kona was missing the over-fill check; under-fill is handled separately via BlobDecodingError::InvalidLength in BlobData::fill().

When Over-fill Can Occur

  • Buggy L1 providers: A beacon/blob provider that returns more blobs than requested (not spec-compliant but observed in practice with third-party providers).
  • Rare L1 reorg scenarios: The set of blob transactions referenced by an L1 block can shift between the time the hashes are collected and the time the blobs are fetched, resulting in a count mismatch.

Changes

  • Add ResetError::BlobsOverFill(usize, usize) variant to pipeline.rs (addresses the over-fill case; under-fill is handled separately via BlobDecodingError::InvalidLength in BlobData::fill()).
  • Import ResetError in blobs.rs and add the post-loop check after the fill loop.
  • Add should_return_extra_blob flag to TestBlobProvider for testing.
  • Add test_load_blobs_overfill_triggers_reset regression test.

Testing

cargo test --package kona-derive sources::blobs

All 12 tests pass, including the new test_load_blobs_overfill_triggers_reset.

Fixes #19363

@ajsutton ajsutton requested a review from a team as a code owner March 3, 2026 05:42
@ajsutton ajsutton requested a review from axelKingsley March 3, 2026 05:42
@codecov
Copy link

codecov bot commented Mar 3, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.9%. Comparing base (6eb557f) to head (de05125).
⚠️ Report is 2 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop   #19364      +/-   ##
===========================================
+ Coverage     75.7%    75.9%    +0.1%     
===========================================
  Files          671      477     -194     
  Lines        71228    59985   -11243     
===========================================
- Hits         53974    45529    -8445     
+ Misses       17110    14456    -2654     
+ Partials       144        0     -144     
Flag Coverage Δ
cannon-go-tests-64 ?
contracts-bedrock-tests ?
unit 75.9% <100.0%> (+<0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...kona/crates/protocol/derive/src/errors/pipeline.rs 100.0% <100.0%> (ø)
...t/kona/crates/protocol/derive/src/sources/blobs.rs 89.2% <100.0%> (+1.6%) ⬆️
...es/protocol/derive/src/test_utils/blob_provider.rs 62.5% <ø> (ø)

... and 199 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ajsutton ajsutton force-pushed the fix/lr-f04-blobsource-overfill-check branch 2 times, most recently from ccf298a to 9fe6fd4 Compare March 3, 2026 08:56
Copy link
Contributor Author

@ajsutton ajsutton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove references to matching op-node.

Reduce the number and verbosity of comments. No need to explain the same thing multiple times.

@ajsutton ajsutton force-pushed the fix/lr-f04-blobsource-overfill-check branch from 9fe6fd4 to d0cc171 Compare March 3, 2026 09:54
@ajsutton ajsutton force-pushed the fix/lr-f04-blobsource-overfill-check branch from d0cc171 to 4b3f95e Compare March 8, 2026 23:07
Copy link
Contributor Author

@ajsutton ajsutton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Correct fix matching Go's fillBlobPointers over-fill check.

Two minor notes:

  1. PR body inaccuracy: The description says this is "symmetric with the existing BlobsUnderFill variant," but no BlobsUnderFill variant exists. Under-fill is handled inside BlobData::fill() via BlobDecodingError::InvalidLength. The BlobsOverFill name is fine on its own, but the symmetry claim is misleading.

  2. Comment line-number references: The comments reference specific Go line numbers (e.g., blob_data_source.go:162). These go stale quickly — consider referencing just the function name (fillBlobPointers) instead.

Copy link
Member

@theochap theochap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, two nits

@ajsutton ajsutton force-pushed the fix/lr-f04-blobsource-overfill-check branch from 78c1423 to fd64d18 Compare March 11, 2026 00:03
@ajsutton ajsutton enabled auto-merge March 11, 2026 02:23
@ajsutton ajsutton added this pull request to the merge queue Mar 11, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to a conflict with the base branch Mar 11, 2026
ajsutton and others added 4 commits March 11, 2026 23:12
After the blob-pointer fill loop completes, add a post-loop check:
if `blob_index < blobs.len()` the provider returned more blobs than
were requested. Return `ResetError::BlobsOverFill` (→
PipelineErrorKind::Reset) rather than silently discarding the extras.

This mirrors op-node's `fillBlobPointers` check at
blob_data_source.go:162-163 which returns
`fmt.Errorf("got too many blobs")` wrapped as `NewResetError`.

Over-fill can occur with buggy blob providers (e.g. third-party RPC
services that ignore the requested hash list) or in rare L1 reorg
scenarios where the blob set shifts between hash collection and fetch.

Changes:
- Add `ResetError::BlobsOverFill(usize, usize)` variant to
  `pipeline.rs` (symmetric with the existing `BlobsUnderFill` variant).
- Import `ResetError` in `blobs.rs` and add the post-loop guard.
- Add `should_return_extra_blob` flag to `TestBlobProvider` for testing.
- Add `test_load_blobs_overfill_triggers_reset` regression test.

Fixes #19363

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
#19480)

Rename the `blob_index` variable in `load_blobs` to `filled_blobs`
for clarity, as the variable tracks the number of blob placeholders
that were filled rather than serving as a traditional index.

Addresses review feedback from optimism#19364.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
@ajsutton ajsutton force-pushed the fix/lr-f04-blobsource-overfill-check branch from 823280d to d52ee33 Compare March 11, 2026 23:14
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ajsutton ajsutton enabled auto-merge March 11, 2026 23:19
@ajsutton ajsutton force-pushed the fix/lr-f04-blobsource-overfill-check branch from c9b9f3e to 4d4da58 Compare March 11, 2026 23:36
The `From<BlobProviderError> for PipelineErrorKind` impl already exists,
so `?` handles the conversion automatically. The explicit `map_err` was
redundant.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ajsutton ajsutton force-pushed the fix/lr-f04-blobsource-overfill-check branch from 4d4da58 to de05125 Compare March 11, 2026 23:40
@ajsutton ajsutton added this pull request to the merge queue Mar 12, 2026
Merged via the queue into develop with commit 7f9662a Mar 12, 2026
241 of 242 checks passed
@ajsutton ajsutton deleted the fix/lr-f04-blobsource-overfill-check branch March 12, 2026 00:33
ajsutton added a commit that referenced this pull request Mar 12, 2026
* fix(kona/derive): add over-fill check in BlobSource::load_blobs

After the blob-pointer fill loop completes, add a post-loop check:
if `blob_index < blobs.len()` the provider returned more blobs than
were requested. Return `ResetError::BlobsOverFill` (→
PipelineErrorKind::Reset) rather than silently discarding the extras.

This mirrors op-node's `fillBlobPointers` check at
blob_data_source.go:162-163 which returns
`fmt.Errorf("got too many blobs")` wrapped as `NewResetError`.

Over-fill can occur with buggy blob providers (e.g. third-party RPC
services that ignore the requested hash list) or in rare L1 reorg
scenarios where the blob set shifts between hash collection and fetch.

Changes:
- Add `ResetError::BlobsOverFill(usize, usize)` variant to
  `pipeline.rs` (symmetric with the existing `BlobsUnderFill` variant).
- Import `ResetError` in `blobs.rs` and add the post-loop guard.
- Add `should_return_extra_blob` flag to `TestBlobProvider` for testing.
- Add `test_load_blobs_overfill_triggers_reset` regression test.

Fixes #19363

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: use function names instead of line numbers in Go references

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(kona/derive): use named fields for BlobsOverFill variant

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(kona/derive): rename blob_index to filled_blobs in BlobSource (#19480)

Rename the `blob_index` variable in `load_blobs` to `filled_blobs`
for clarity, as the variable tracks the number of blob placeholders
that were filled rather than serving as a traditional index.

Addresses review feedback from optimism#19364.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove references to Go implementation in comments

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(kona/derive): remove useless BlobProviderError conversion in map_err

The `From<BlobProviderError> for PipelineErrorKind` impl already exists,
so `?` handles the conversion automatically. The explicit `map_err` was
redundant.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kona: BlobSource::load_blobs silently ignores over-fill instead of returning Reset error

2 participants