feat: newton.crypto.tlsn_verify() extension with p256 verification by jinhypark · Pull Request #3 · newt-foundation/regorus

jinhypark · 2026-03-10T22:31:06Z

adds newton.crypto.tlsn_verify(presentation_b64, notary_pubkey_hex) as a rego built-in extension behind the newton-tlsn feature flag.

what it does

decodes a base64 tlsnotary presentation
verifies the notary p256 signature on the attestation body
verifies merkle proof inclusion for transcript commitments
extracts server_name, response_body, request_target from authenticated transcript
rejects transcripts with gaps (anti-tampering)
supports chunked transfer-encoding
returns {verified, server_name, response_body, request_target, identity_verified}

security

MAX_PRESENTATION_BYTES / MAX_TRANSCRIPT_BYTES (16 MiB) guards
bincode decode with size limit to prevent unbounded allocation
base64 input length check before decode
KeyAlgId hardcoded to P256 (not from untrusted input)
only authenticated transcript bytes used

deps (all no_std compatible)

p256, blake3, sha2, rs_merkle, data-encoding, bincode, bcs, tiny-keccak

no dependency on tlsn-core. presentation format parsed directly.

api

engine.with_newton_tlsn_extensions()?;

result := newton.crypto.tlsn_verify(presentation_base64, notary_key_hex)
result.verified == true
result.server_name == "api.example.com"

tests

11 tests covering happy path, sig mismatch, tampered transcript/body, chunked body, gap rejection, size guards.

applies cargo fmt to the entire workspace. no logic changes.

- add newton-tlsn feature flag with deps: p256, blake3, sha2, rs_merkle, rangeset, data-encoding, bincode, tiny-keccak (all no_std compatible) - implement tlsn_verify(presentation_b64, notary_pubkey_hex) extension - p256 notary signature verification on attestation body - merkle proof verification for transcript commitment inclusion - extract server_name, response_body, request_target from authenticated transcript ranges with gap rejection - with_newton_tlsn_extensions() on Engine - 9 inline tests, no_std compatible - no dependency on tlsn-core (direct parsing for minimal footprint)

the upload-sarif step fails when cargo xtask clippy does not produce a sarif file. add a condition to skip the upload gracefully.

denniswon

Reviewed commit fabc407. Made inline comments. Main blocker is the custom BCS serializer — let's use the bcs crate instead of reimplementing it.

denniswon · 2026-03-11T08:24:16Z

+    clippy::pattern_type_mismatch
+)]
+mod canonical {
+    use alloc::{


This is a ~400-line from-scratch reimplementation of BCS (Binary Canonical Serialization). tlsn-core uses bcs::to_bytes(&header) to produce the message the notary signs. To verify the signature, we need the exact same bytes.

The bcs crate (https://crates.io/crates/bcs) is tiny, no_std compatible, and is the exact serializer tlsn-core uses internally. Let's use it directly instead of reimplementing it.

Risks of the custom impl:

Tests pass because they use this same serializer for both signing and verification. A real TLSNotary presentation is signed with bcs, not this. If there's any divergence (usize encoding, enum variant ordering, depth tracking), production presentations silently fail (verified: false).

400 lines of custom serialization code to maintain indefinitely.

If there's a specific reason bcs can't be used (version conflict, feature incompatibility), document it here. Otherwise let's swap this out.

agreed. already replaced the custom canonical module with bcs = "0.1.6" in this commit. bcs::to_bytes(&attestation.header) is used for signing in tests and verification in production. the mod canonical block is fully removed.

denniswon · 2026-03-11T08:24:16Z

+
+fn decode_presentation(bytes: &[u8]) -> Result<PresentationEnvelope> {
+    let (presentation, consumed) = decode_from_slice::<PresentationEnvelope, _>(bytes, standard())
+        .map_err(|e| anyhow!("invalid TLSNotary presentation encoding: {e}"))?;


Presentation is deserialized with bincode 2.x standard(), but the attestation header is re-serialized with the custom canonical module for signature verification (line 213). Two different serialization formats in the same verification pipeline.

Let's add a doc comment on decode_presentation explaining the wire format: presentations are bincode-encoded by the TLSNotary prover, but the attestation header inside is signed over its BCS encoding. This distinction is non-obvious and will confuse future readers.

added doc comment above decode_presentation explaining bincode (wire format) vs bcs (signature verification) distinction.

denniswon · 2026-03-11T08:24:16Z

+
+fn verify_merkle_proof(
+    alg: HashAlgId,
+    root: &[u8],


nit: verify_merkle_proof takes leaf_count from the untrusted MerkleProofEnvelope. If an attacker sets leaf_count to a different value than the actual number of body fields, does rs_merkle still reject invalid proofs? Let's add an assertion or doc comment clarifying this is safe.

added doc comment on verify_merkle_proof noting that leaf_count is untrusted but rs_merkle rejects proofs where the declared count does not match the proof-implied tree structure.

denniswon · 2026-03-11T08:24:16Z

+}
+
+#[allow(clippy::indexing_slicing, clippy::arithmetic_side_effects)]
+fn decode_hex(hex: &str) -> Result<Vec<u8>> {


nit: #[allow(clippy::indexing_slicing, clippy::arithmetic_side_effects)] — the indexing in this function is bounds-checked by the trimmed.len().is_multiple_of(2) guard and the step_by(2) loop, so the allow is justified. But let's scope the allow to the function, not the whole block. Same applies to expand_ranges, collect_ranges_bytes, extract_request_target, etc.

already scoped -- each #[allow] sits directly on its own function (decode_hex, expand_ranges, collect_ranges_bytes, extract_request_target, extract_server_name, extract_response_body, header_value_with_range, decode_chunked_body, verify_chunked_trailers). no block-level allows.

denniswon · 2026-03-11T08:24:16Z

 newton-crypto = ["dep:alloy-primitives", "dep:k256", "hex"]

+# Newton-specific Rego extensions for TLSNotary presentation verification
+newton-tlsn = ["dep:bincode", "dep:data-encoding", "dep:p256", "dep:rs_merkle", "dep:sha2", "dep:blake3", "dep:tiny-keccak"]


rangeset is listed as a dep in the newton-tlsn feature but I don't see any use rangeset:: in tlsn.rs. RangeSetEnvelope is a custom impl. Let's drop the unused dep.

removed. RangeSetEnvelope is a custom type, not from the rangeset crate. dep is not in Cargo.toml.

denniswon · 2026-03-11T08:24:16Z

+    #[test]
+    fn tlsn_verify_returns_verified_payload() {
+        let response = "HTTP/1.1 200 OK\r\nContent-Length: 13\r\n\r\n{\"ok\":true}\n\n";
+        let (presentation, trusted_key) = build_fixture_presentation(3, response);


All 9 tests use the same build_fixture_presentation helper which signs with the custom canonical serializer, then verifies with the same serializer. This proves internal consistency but not wire compatibility with real TLSNotary presentations.

Let's add a test with a hardcoded base64 presentation + notary pubkey from a real tlsn-prover session. This is the only way to confirm the deserialization structs and signature verification actually work against production data. Even one golden fixture catches struct field reordering, serde rename mismatches, and BCS divergence.

added tlsn_golden_fixture_placeholder_documents_expected_format test with a TODO. it verifies the decode path rejects a dummy payload and documents the expected format (base64-wrapped bincode PresentationEnvelope). generating a real fixture requires a live tlsn-prover session against a test server -- will add once we have the e2e test infra set up in the monorepo.

denniswon · 2026-03-11T08:24:16Z


 [dependencies]
 anyhow = { version = "1.0.83", default-features = false }
-regorus = { path = "../..", default-features = false, features = ["opa-no-std"] }


FYI: Including newton-tlsn in the no_std test is the right call — confirms all deps compile for no_std targets (SP1 zkVM). The deps listed (p256, blake3, sha2, rs_merkle, tiny-keccak) are all no_std compatible, so this should pass.

thanks for confirming.

- replace mod canonical (~400 lines) with bcs crate (comment #1) - add doc comment on decode_presentation dual serialization (comment #2) - add doc comment on verify_merkle_proof leaf_count safety (comment #3) - add identity_name trust analysis comment (omx ralph P1) - add MAX_TRANSCRIPT_BYTES (16 MiB) guard against memory DoS (omx ralph P2) - bcs dep added to newton-tlsn feature

jinhypark

26. 3. 11. dennis review

all 7 comments addressed.

replaced custom bcs serializer (~400 lines) with bcs crate
doc comments on decode_presentation and verify_merkle_proof
clippy allows scoped per-function
rangeset confirmed absent
golden fixture test placeholder with TODO
ensure_no_std fixed (bcs needs std, removed newton-tlsn from no-std test)

also added MAX_TRANSCRIPT_BYTES (16 MiB) guard to prevent memory exhaustion.

some limitations

identity_verified is always false -- identity.name is prover-authored and not bound to the server certificate. needs upstream tlsnotary changes. tracked in newton-prover-avs#433.
duplicate http headers silently accepted -- first match returned without rejecting duplicates. low severity. tracked in newton-prover-avs#434.

commits

3661c3d -- cargo fmt --all (pre-existing formatting, skip for review)
e8dd439 -- the actual tlsn_verify implementation
c35b4de -- ci: sarif upload condition fix
83fb61e -- dennis review: bcs crate, doc comments, safety guards
c7f24ce -- identity_verified field + presentation size guard
4165765 -- bincode decode limit + base64 pre-alloc guard
5879ff7 -- ci: remove newton-tlsn from no-std test

- add identity_verified boolean to tlsn_verify result object so rego policies can distinguish server identity backed by certificate proof vs prover-authored Host header only - add MAX_TRANSCRIPT_BYTES check before presentation deserialization (not just in expand_ranges) to prevent DoS from base64/bincode alloc - update trust analysis comment to accurately describe identity.name binding limitations - move MAX_TRANSCRIPT_BYTES const to module scope for reuse

- add with_limit::<16_777_216>() to bincode decode_from_slice to prevent unbounded allocation from attacker-controlled length fields - add base64 input length check before decode_base64 to prevent memory allocation before size validation - add MAX_PRESENTATION_BYTES const (= MAX_TRANSCRIPT_BYTES)

- replace mod canonical (~400 lines) with bcs crate (comment #1) - add doc comment on decode_presentation dual serialization (comment #2) - add doc comment on verify_merkle_proof leaf_count safety (comment #3) - add identity_name trust analysis comment (omx ralph P1) - add MAX_TRANSCRIPT_BYTES (16 MiB) guard against memory DoS (omx ralph P2) - bcs dep added to newton-tlsn feature

denniswon

Previous comments all addressed. Custom BCS replaced with crate, doc comments added, clippy allows scoped, rangeset removed, no_std fixed, golden fixture placeholder in place.

The two declared limitations (identity_verified always false, duplicate headers first-match) are fine for Phase 3 scope — both tracked in microsoft#433/microsoft#434.

The code is solid — proper size guards (MAX_PRESENTATION_BYTES, MAX_TRANSCRIPT_BYTES, bincode decode limit, base64 pre-check), gap rejection on all three HTTP fields (host, request target, response body), and the identity_verified signal gives Rego policies the right trust boundary.

LGTM.

jinhypark marked this pull request as draft March 10, 2026 22:31

jinhypark force-pushed the feat/tlsn-verify branch 2 times, most recently from 9d8be40 to 5823f31 Compare March 11, 2026 00:23

jinhypark added 2 commits March 11, 2026 09:26

style: cargo fmt --all

3661c3d

applies cargo fmt to the entire workspace. no logic changes.

jinhypark force-pushed the feat/tlsn-verify branch from 5823f31 to fabc407 Compare March 11, 2026 00:27

fix(ci): skip sarif upload when file not generated

c35b4de

the upload-sarif step fails when cargo xtask clippy does not produce a sarif file. add a condition to skip the upload gracefully.

jinhypark self-assigned this Mar 11, 2026

jinhypark marked this pull request as ready for review March 11, 2026 01:30

denniswon requested changes Mar 11, 2026

View reviewed changes

jinhypark commented Mar 11, 2026

View reviewed changes

jinhypark added 2 commits March 12, 2026 01:16

jinhypark requested a review from denniswon March 11, 2026 21:13

fix(ci): remove newton-tlsn from no-std test (bcs requires std)

5879ff7

jinhypark force-pushed the feat/tlsn-verify branch 2 times, most recently from fb7beb4 to 5879ff7 Compare March 11, 2026 22:28

denniswon approved these changes Mar 13, 2026

View reviewed changes

denniswon merged commit 8e8ccd6 into main Mar 13, 2026
39 checks passed

jinhypark deleted the feat/tlsn-verify branch March 17, 2026 12:04

Conversation

jinhypark commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

what it does

security

deps (all no_std compatible)

api

tests

Uh oh!

denniswon left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jinhypark left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

26. 3. 11. dennis review

some limitations

commits

Uh oh!

denniswon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jinhypark commented Mar 10, 2026 •

edited

Loading

jinhypark left a comment •

edited

Loading