Skip to content

feat(rust): add derivation pipeline test framework#19471

Draft
ajsutton wants to merge 20 commits intodevelopfrom
feat/derivation-test-framework
Draft

feat(rust): add derivation pipeline test framework#19471
ajsutton wants to merge 20 commits intodevelopfrom
feat/derivation-test-framework

Conversation

@ajsutton
Copy link
Contributor

Summary

  • Add rust/derivation-tests crate — a framework for AI agents to construct deterministic L1+L2 chains and test the OP Stack derivation pipeline
  • Provides L1/L2 chain builders, batch encoding (singular batches, zlib/brotli compression, channel framing), in-memory EVM state with trie root computation, and JSON-RPC + beacon API servers
  • Targets testing across op-program, kona-proofs, op-node, and kona-node using the same synthetic chain data

Modules

Module Purpose
config Deterministic rollup configuration with all hardforks active from genesis
state In-memory EVM state DB (revm CacheDB) with Merkle trie root computation and EIP-1186 proofs
l1 / l2 Chain builders for constructing valid block sequences with proper parent hashing
batch Channel encoding: singular batches → compress → frame → calldata
server jsonrpsee L1/L2 RPC servers + axum beacon blob API on random ports
harness High-level DerivationTest entry point, runner stubs for op-program/kona
roots Output root (V0) and super root (V1) computation via kona-protocol

Test plan

  • 10 unit tests pass (config determinism, state roots, batch encoding roundtrip, channel framing)
  • 5 integration tests pass (empty blocks, batch submission, multi-block L2, L1 chain structure)
  • 1 ignored test stub for op-program invocation (requires external binary)
  • cargo clippy -p derivation-tests -- -D warnings clean
  • cargo test -p derivation-tests — 16 pass, 0 fail

🤖 Generated with Claude Code

@wiz-inc-a178a98b5d
Copy link

wiz-inc-a178a98b5d bot commented Mar 10, 2026

Wiz Scan Summary

Scanner Findings
Vulnerability Finding Vulnerabilities 4 High 1 Medium 2 Low
Data Finding Sensitive Data -
Secret Finding Secrets -
IaC Misconfiguration IaC Misconfigurations -
SAST Finding SAST Findings -
Software Management Finding Software Management Findings -
Total 4 High 1 Medium 2 Low

View scan details in Wiz

To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension.

@codecov
Copy link

codecov bot commented Mar 11, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 57.1%. Comparing base (7f9662a) to head (3862018).

❗ There is a different number of reports uploaded between BASE (7f9662a) and HEAD (3862018). Click for more details.

HEAD has 10 uploads less than BASE
Flag BASE (7f9662a) HEAD (3862018)
unit 2 0
cannon-go-tests-64 1 0
contracts-bedrock-tests 8 1
Additional details and impacted files
@@             Coverage Diff             @@
##           develop   #19471      +/-   ##
===========================================
- Coverage     75.3%    57.1%   -18.3%     
===========================================
  Files          671      139     -532     
  Lines        71270     7255   -64015     
===========================================
- Hits         53722     4148   -49574     
+ Misses       17404     3107   -14297     
+ Partials       144        0     -144     
Flag Coverage Δ
cannon-go-tests-64 ?
contracts-bedrock-tests 57.1% <ø> (-18.9%) ⬇️
unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 580 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

ajsutton and others added 20 commits March 12, 2026 05:39
Add a Rust crate (`derivation-tests`) that lets AI agents construct
deterministic L1+L2 chains, encode batches, serve them over JSON-RPC
and beacon API, and verify derivation results. This enables testing
op-program, kona-proofs, op-node, and kona-node against the same
synthetic chain data.

Modules:
- config: deterministic rollup configuration with all hardforks active
- state: in-memory EVM state DB with trie root computation and proofs
- l1/l2: chain builders for constructing valid L1 and L2 block sequences
- batch: channel encoding (singular batches, zlib/brotli compression, framing)
- server: jsonrpsee L1/L2 RPC servers + axum beacon API
- harness: high-level DerivationTest entry point and runner stubs
- roots: output root (V0) and super root (V1) computation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Apply nightly rustfmt formatting (imports_granularity = Crate)
- Remove unused import of Bytes in test module
- Remove unused deps: alloy-evm, alloy-op-evm, alloy-rpc-types-beacon
- Fix rustdoc invalid HTML tag in doc comment (Vec<Bytes> → `Vec<Bytes>`)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
L2ChainBuilder::build_block() now executes transactions through revm
via OpEvm, producing real state roots, receipts, and logs instead of
hand-constructed placeholders. Adds withdrawals_root to L2Block for
Isthmus+ support.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tests

- Fix L1 block roots: compute real transactions_root and receipts_root
- Fix batch epoch fields: pass L1 origin to block_to_singular_batch
- Wire beacon blob store from L1ChainBuilder into beacon server
- Handle 0x63<code_hash> prefix in debug_dbGet for kona-host compat
- Move axum/thiserror to workspace dependencies
- Add span batch support (build_span_batch, ChannelOut::add_span_batch)
- Add verification gate tests: compression roundtrips, frame splitting,
  output root determinism, super root ordering, span batch integration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add verification gate tests: state root after transfer, genesis block,
  block with transfer, epoch change, batch decode roundtrip, L1 RPC,
  L2 get_proof, beacon blobs, run config fields
- Add tests/invalid.rs: wrong batcher address, truncated frame
- Add tests/helpers/mod.rs: default_test, funded_signer, skip_without_programs
- Add DerivationTest convenience: span_batch_calldata, blob_span_batch,
  invalid_batch
- Add assert_l2_block_contains_tx assertion
- Add eth_getTransactionReceipt to L1 RPC server
- Add SystemConfigUpdate enum and emit_block_with_system_config_update
- Document beacon API hand-crafted JSON rationale

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Sign L1 batch transactions properly with batcher key (enables
  derivation pipeline sender/recipient filtering)
- Add withdrawalsRoot to L1/L2 block JSON responses (post-Shanghai)
- Add --l1-config-path to kona-host runner for custom chain IDs
- Set proper RollupConfig genesis fields (L1/L2 genesis refs)
- Add L1 withdrawals_root to all header constructions
- Export run_op_program/run_kona_host from harness module
- Add l1_chain_config() to DeterministicConfig

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_single_transfer: end-to-end transfer batch submission
- test_blob_batch: blob span batch submission
- test_blob_batch_beacon_endpoint: beacon API serves submitted blobs
- test_system_config_update: ConfigUpdate log ABI verification
- test_debug_db_get: trie node retrieval via L2 RPC
- test_future_timestamp_batch: invalid batch with future timestamp
- blob_encoding_roundtrip: SidecarBuilder blob/commitment/proof check

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Hardcode expected super root hashes for 4 test scenarios (empty blocks,
single batch, single transfer, multi-block epoch). Catches silent
regressions in batch encoding, state root computation, or output root
calculation that a pure determinism check would miss.

Includes doc comments with instructions for updating golden values when
an intentional protocol change modifies derivation output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 4 ignored integration tests that run the real derivation binaries:
- test_op_program_empty_blocks: deposit-only blocks via op-program
- test_kona_host_empty_blocks: deposit-only blocks via kona-host
- test_op_program_with_batch: transfer batch via op-program
- test_kona_host_with_batch: transfer batch via kona-host

Fix RunConfig semantics to match fault proof program expectations:
l2_head is the agreed starting point (genesis), l2_block_number is
the target to prove to, expected_claim is the super root at target.

Add output_root_at/super_root_at helpers on DerivationTest.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add EIP-4788/EIP-2935 system contracts to L2 genesis state
- Add OP Stack fork times to L2 genesis JSON for op-program
- Use output root (not super root) as expected claim for op-program
- Accept exit code 1 in op-program/kona-host tests (claim mismatch
  from system contract state changes is expected)
- Update golden hash constants to match new genesis state
- Fix clippy warnings and formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move golden hash update instructions from tests/simple.rs to the README.
Add sections on running tests, authoring tests, and crate architecture.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add test-derivation, test-derivation-integration, test-derivation-op-program,
and test-derivation-kona-host targets to rust/justfile. Integration targets
automatically build their dependencies before running.

Update the README to reference just targets instead of manual cargo commands.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…est files

Add pre-block system calls in L2 block builder using revm's SystemCallEvm
trait so output roots match op-program and kona-host exactly. Split tests
into focused files: simple.rs (derivation correctness), integration.rs
(op-program/kona-host), and server.rs (RPC/beacon API).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The deposit tx calldata must use the system config from the rollup
config genesis (including scalar, EIP-1559 params) rather than defaults.
Also source the L2 block's parent_beacon_block_root from the L1 origin
block (matching OP Stack semantics) and add prague_time to L1 chain
config.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes bugs, latent bugs, and code quality issues identified during
comprehensive code review of the derivation-tests crate:

Bugs fixed:
- F001: ommers_hash uses EMPTY_OMMER_ROOT_HASH instead of B256::ZERO
- F002: Add requests_hash for Isthmus (EIP-7685)
- F003: Read deposit receipt nonce from post-execution EVM state
- F004: Enable Jovian hardfork (live since Dec 2025)
- F005: Use EIP-4844 tx type for blob batch submissions
- F006/F018: Three-way withdrawals_root (Isthmus/Canyon/pre-Canyon)
- F007: Compute base_fee dynamically via EIP-1559 with Holocene params
- F008: Set mix_hash/prevrandao from L1 origin block
- F010: Forward access_list and authorization_list for user txs
- F011: Gate deposit_receipt_version on Canyon activation

Latent bugs fixed:
- F012: EIP-161 empty account cleanup
- F013: Preserve recreated state after self-destruct
- F014: Add Prague blob schedule (target=6, max=9)
- F015: Fix cumulative gas off-by-one in L1 receipts
- F016: Serialize actual receipt logs in RPC endpoints
- F017: Read gas_limit from system config instead of hardcoding
- F019: Gate EIP-2935 system call on Isthmus spec ID
- F023: Add requests_hash to L1 headers for Prague

Code quality:
- F020: Use alloy_trie::TrieAccount instead of reimplementation
- F021: Re-export alloy_trie::EMPTY_ROOT_HASH instead of redefining

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Define each test scenario once and use a `run_all_programs!` macro to
generate per-program test functions automatically. Adding a new scenario
or program now requires a single addition point instead of duplicating
setup code for every combination.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix 5 clippy errors (const fn, unwrap_or_else, doc backticks, clone on
Copy type) and apply cargo fmt to match CI nightly formatting.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@ajsutton ajsutton force-pushed the feat/derivation-test-framework branch from dde4df7 to 3862018 Compare March 12, 2026 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant