common/test: reproduce BOLT 12 PR #1295 marker-jump spec contradiction#9170
Open
vincenzopalazzo wants to merge 3 commits into
Open
common/test: reproduce BOLT 12 PR #1295 marker-jump spec contradiction#9170vincenzopalazzo wants to merge 3 commits into
vincenzopalazzo wants to merge 3 commits into
Conversation
…ec contradiction Add a failing C unit test that reproduces the BOLT 12 spec contradiction raised on lightning/bolts#1295. The writer rule in `12-offer-encoding.md` (lines 1041-1049, head db3eff3a54) mandates that after a `proof_omitted_tlvs` entry of `239` the next entry MUST be `1000000000`, because the writer skips past the signature/payer-proof TLV range (240..999999999). The reader rule (lines 1065-1067) only accepts an entry that is `prev + 1` or `<included TLV> + 1`, with no carve-out for the 239 -> 1000000000 jump. A reader applying the spec literally rejects a marker sequence the writer rule mandates. CLN's reader in common/bolt12_proof.c lines 339-388 is a literal transcription of the bogus reader rule (specifically the `omitted != prev_omitted + 1` branch at line 379). Fed the writer-rule-mandated sequence [1, 2, ..., 239, 1000000000] it returns an error on the 1000000000 entry. The new test in common/test/run-bolt12_proof_marker_jump.c builds the minimum tlv_payer_proof needed to reach the marker-validation loop and asserts check_payer_proof returns NULL. It is expected to fail on current master, the failure IS the reproduction. The fix belongs in the spec PR first (amend reader rule 1065) and only then mirrored into CLN. The same contradiction is reproduced on the Rust-Lightning side in lightningdevkit/rust-lightning#4297, test spec_writer_reader_rules_contradict_on_gap_jump in lightning/src/offers/merkle.rs (commit 0034c79ae). Changelog-None Link: lightning/bolts#1295 (comment) Link: lightningdevkit/rust-lightning#4297 Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
In `lightning/bolts#1295` (head db3eff3a54, 12-offer-encoding.md), the writer rule (lines 1041-1049) mandates that after a `proof_omitted_tlvs` entry of `239` the next entry is `1000000000` (jumping past the signature/payer-proof TLV range 240..999999999), but the reader rule (lines 1065-1067) only accepts `prev + 1` or `<included TLV> + 1`. A reader applying the spec literally rejects a marker sequence the writer rule mandates. The contradiction was raised on the spec PR: lightning/bolts#1295 (comment) CLN's reader at `common/bolt12_proof.c` line 379 was a literal transcription of the bogus reader rule, and CLN's writer at line 115 used `++last_type` which would have emitted `240` (invalid) after `239` if a real invoice ever reached the boundary. Fix this by collapsing both writer and reader onto a single helper `next_marker(prev)` that returns the smallest valid marker number strictly greater than `prev`. Both sides now reference the same symbol, so the writer/reader asymmetry that caused this bug is structurally impossible. The unit test `run-bolt12_proof_marker_jump` is rewritten as a real end-to-end test: it builds an invoice with the standard required fields plus 64 dummy unknown TLVs at odd extension-range types (1000000001, 1000000003, ..., 1000000127), makes a payer_proof including only the three required fields, and asserts that `check_payer_proof` returns NULL. The marker sequence produced ends with [..., 237, 238, 239, 1000000000], exercising the gap-jump end-to-end. The test also sanity-checks that 239 appears in the sequence and that 1000000000 follows it immediately, so the test cannot go green for the wrong reason. Existing `run-bolt12_proof` and `run-bolt12_proof_vectors` still pass with no changes to their behaviour. Parallel LDK reproduction: lightningdevkit/rust-lightning#4297 (test `spec_writer_reader_rules_contradict_on_gap_jump` in lightning/src/offers/merkle.rs, commit 0034c79ae). Changelog-Fixed: Reader of BOLT 12 payer_proof now accepts the writer-rule-mandated 239 -> 1000000000 marker jump. Link: lightning/bolts#1295 (comment) Link: lightningdevkit/rust-lightning#4297 Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
Add a new valid_vector `marker_jump_239_to_1000000000` to the JSON output produced by `run-bolt12_proof_vectors`. The vector encodes a real signed payer_proof whose `proof_omitted_tlvs` ends with [..., 237, 238, 239, 1000000000], i.e. it actually contains the spec gap-jump. The point is cross-implementation coverage: with this vector in the JSON corpus that CLN already emits, other BOLT 12 implementations (LDK PR ElementsProject#4297, future eclair, ...) can consume the same bech32 string through their own decoder and verifier. A reader that still has the old "must be prev + 1 or <included TLV> + 1" rule will reject the proof with an error naming `1000000000` after `239`; that surfaces the same latent bug across implementations rather than having each project rediscover it independently. The marker-jump invoice is built by `build_marker_jump_invoice`, which is a minimal invoice (same keys A-F as the existing vectors) plus 64 dummy unknown TLVs at odd extension-range types (1000000001, 1000000003, ..., 1000000127). The new `include_required_only` callback includes only the three required fields (invreq_payer_id, invoice_payment_hash, invoice_node_id), so the 64 trailing dummies and the optional invoice fields all get omitted. The writer-side `next_marker` walks the marker counter through 1..239 and then emits 1000000000 on the 64th dummy, which is the case under test. The existing 5 valid_vectors and 23 invalid_vectors are unchanged; the new entry is appended at the end of `valid_vectors[]` so the existing positions are preserved. Link: lightning/bolts#1295 (comment) Link: lightningdevkit/rust-lightning#4297 Signed-off-by: Vincenzo Palazzo <vincenzopalazzodev@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a failing C unit test in
common/test/run-bolt12_proof_marker_jump.cthat reproduces the BOLT 12 spec contradiction raised on
lightning/bolts#1295.
In
lightning/bolts#1295(headdb3eff3a54,12-offer-encoding.md),the writer rule (lines 1041-1049) mandates that after a
proof_omitted_tlvsentry of239, the next entry MUST be1000000000, because the writer needs to skip past thesignature/payer-proof TLV range (
240..999999999) which is not validfor marker numbers.
However the reader rule (lines 1065-1067) only accepts the next entry
if it is
prev + 1or<included TLV> + 1. There is no carve-out forthe
239 -> 1000000000jump, so a reader applying the spec literallywill reject a marker sequence the writer rule mandates. The issue is
still open on the spec PR.
CLN's reader in
common/bolt12_proof.clines 339-388 is a literaltranscription of the bogus reader rule. The buggy branch is at line
379, which errors out when
omitted != prev_omitted + 1andfind_tlv_num(pptlv, omitted - 1)is false:Fed the writer-rule-mandated sequence
[1, 2, ..., 239, 1000000000],this returns an error on the
1000000000entry, because it is notprev_omitted + 1 = 240and there is no included TLV at999999999for the fallback to find. That is exactly the contradiction.
The new test constructs the minimum
tlv_payer_proofneeded to reachthe marker-validation loop: required field pointers populated,
preimage matching
invoice_payment_hash,pptlv->fieldsdeliberatelyleft empty so
find_tlv_numreturns false in every branch.Then it sets
proof_omitted_tlvsto[1, 2, ..., 239, 1000000000]and asserts that
check_payer_proofreturns NULL. It does not, andthe assertion blows up:
The failure naming
[239](the slot holding1000000000) andprevious 239is exactly the spec contradiction surfaced in CLN.The test is expected to fail on current master, the failure IS the
reproduction. I am not fixing the underlying bug in this PR, the fix
belongs in the spec PR first (amend reader rule 1065 to allow the
239 -> 1000000000jump) and only then mirrored into CLN.Landing the failing test now means the eventual spec fix has a
concrete CLN-side regression target instead of a vague note about
remembering to update the validator.
Link: lightning/bolts#1295 (comment)
Link: lightningdevkit/rust-lightning#4297
Signed-off-by: Vincenzo Palazzo vincenzopalazzodev@gmail.com