Skip to content

feat: RFC 9000 requirements manifest + MCP test report#17

Open
ElNiak wants to merge 61 commits intoproductionfrom
feat/mcp-tool-fixes-and-requirements
Open

feat: RFC 9000 requirements manifest + MCP test report#17
ElNiak wants to merge 61 commits intoproductionfrom
feat/mcp-tool-fixes-and-requirements

Conversation

@ElNiak
Copy link
Copy Markdown
Owner

@ElNiak ElNiak commented Mar 12, 2026

Summary

  • Add rfc9000_requirements.yaml with 101 extracted RFC 9000 requirements
  • Add ivy-tools-test-report.md documenting live verification of all 19 MCP tools
  • Update ivy-lsp submodule: MCP tool fixes (ISS-001/002/003/005/006/008)
  • Update panther-ivy-plugin submodule: auto-detect local ivy-lsp source

Test Plan

  • All 19 MCP tools verified via live calls (22 PASS, 4 PARTIAL)
  • 101 requirements loaded from YAML manifest
  • Full results in docs/ivy-tools-test-report.md

Summary by Sourcery

Add a structured RFC 9000 requirements manifest and document end-to-end MCP tool verification, alongside submodule updates that improve ivy MCP tooling reliability and local development workflow.

New Features:

  • Introduce a RFC 9000 YAML requirements manifest for QUIC protocol testing and traceability.

Bug Fixes:

  • Resolve multiple ivy MCP server issues affecting visualization, symbol queries, requirement loading, linting, and include graph accuracy by updating the ivy-lsp submodule.

Enhancements:

  • Enable panther ivy tools to auto-detect and use a local ivy-lsp source for easier development and testing.
  • Ensure MCP deployments install PyYAML so RFC requirement manifests can be loaded correctly.

Documentation:

  • Add an ivy-tools MCP server test and fix report documenting live verification results across all MCP tools.

- Add rfc9000_requirements.yaml with 101 extracted RFC 9000 requirements
  organized by level (MUST/SHOULD/MAY) and protocol layer
- Add ivy-tools-test-report.md documenting live verification of all 19
  MCP tools (22 PASS, 4 PARTIAL)
- Update ivy-lsp submodule: MCP tool fixes (ISS-001/002/003/005/006/008)
- Update panther-ivy-plugin submodule: auto-detect local ivy-lsp source
Copilot AI review requested due to automatic review settings March 12, 2026 21:53
@sourcery-ai
Copy link
Copy Markdown

sourcery-ai bot commented Mar 12, 2026

Reviewer's Guide

Adds a structured RFC 9000 requirements manifest and an MCP test report for ivy-tools, plus updates ivy-lsp and panther-ivy-plugin submodules to improve MCP tool behavior, dependency handling, and local source detection.

Sequence diagram for MCP QUIC requirements testing with RFC9000 manifest

sequenceDiagram
  actor Developer
  participant IvyTools as ivy_tools
  participant ProtocolTesting as protocol_testing_quic
  participant Manifest as rfc9000_requirements_yaml
  participant MCPTool as mcp_tool_quic
  participant IvyLSP as ivy_lsp

  Developer->>IvyTools: run_quic_requirements_tests()
  IvyTools->>ProtocolTesting: start_test_run()
  ProtocolTesting->>Manifest: load_requirements()
  Manifest-->>ProtocolTesting: requirements_list(101)

  loop for each_requirement
    ProtocolTesting->>MCPTool: verify_requirement(requirement)
    MCPTool->>IvyLSP: execute_quic_scenario(requirement_id)
    IvyLSP-->>MCPTool: scenario_result(pass_or_partial)
    MCPTool-->>ProtocolTesting: verification_result(status)
  end

  ProtocolTesting-->>IvyTools: aggregated_results(22_pass_4_partial)
  IvyTools-->>Developer: write_report(ivy_tools_test_report_md)
Loading

ER diagram for RFC9000 requirements manifest

erDiagram
  RequirementManifest {
    string rfc
    string title
  }

  Requirement {
    string requirement_id
    string text
    string section
    string level
    string layer
    boolean testable
  }

  RequirementManifest ||--o{ Requirement : contains
Loading

File-Level Changes

Change Details Files
Introduce a structured RFC 9000 requirements manifest used by MCP requirements/coverage tools.
  • Add YAML manifest defining 101 extracted RFC 9000 requirements with metadata such as section, level, layer, and testable flag.
  • Ensure requirement IDs follow a consistent rfc9000:
    naming convention and cover multiple QUIC layers (flow-control, connection, handshake, address-validation, migration, termination, packet, loss-recovery, packet-format, frame, etc.).
protocol-testing/quic/rfc9000_requirements.yaml
Document comprehensive live testing of all ivy-tools MCP tools and their fixes.
  • Add a detailed markdown report summarizing the test setup, metrics before/after, and a per-issue breakdown (ISS-001..009) of MCP tool bugs, fixes, and known limitations.
  • Capture live MCP verification matrix showing 22 PASS / 4 PARTIAL results across linting, include graph, semantic queries, visualization, traceability/coverage, CLI tools, and cross-references.
  • Record specific changes made in the ivy-lsp server (RequirementGraph initialization, symbol extraction, include resolution, PyYAML dependency) and plugin startup scripts for future reviewers.
docs/ivy-tools-test-report.md
Update ivy-lsp submodule with MCP fixes and dependency adjustments to support requirements manifests and visualization.
  • Pull in mcp_server changes that build a RequirementGraph via extract_requirements_light, enabling visualization tools to return modelReady=true and non-empty data.
  • Include fixes for semantic symbol extraction and include-graph handling, improving ivy_query_symbol, ivy_cross_references, ivy_include_graph, and ivy_lint behavior.
  • Ensure the ivy-lsp MCP extra includes the PyYAML dependency so YAML manifests like rfc9000_requirements.yaml can be loaded by traceability and coverage tools.
submodules/ivy-lsp
Update panther-ivy-plugin to auto-detect local ivy-lsp source instead of always using the remote GitHub version.
  • Modify the start-ivy-tools script to prefer the local ivy-lsp submodule when running inside a Panther project, falling back to GitHub only when necessary.
  • Enable local development and testing of ivy-lsp MCP changes without requiring publishing or external installation.
submodules/panther-ivy-plugin

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues, and left some high level feedback:

  • The rfc9000_requirements.yaml manifest mixes machine-oriented content with human notes (e.g., -> OK, -> Impossible to know, speculative questions) inside text fields; consider moving reviewer commentary into separate metadata fields or comments so downstream tooling can reliably treat text as the canonical requirement body.
  • In rfc9000_requirements.yaml, some fields are inconsistent in type/format (e.g., section sometimes quoted vs. numeric, requirement IDs like rfc9000:5.2 reused for different subsections, and layered text that appears truncated or merged), which could complicate automated consumers; normalizing section identifiers and ensuring each requirement entry is cleanly extracted would make the manifest more robust.
  • For the requirements marked with inline notes such as -> not possible to test or -> impossible to know, you may want to encode that constraint in structured form (e.g., testable: false or a test_notes field) rather than leaving testable: true so that test tooling and coverage reports more accurately reflect what can actually be verified.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The `rfc9000_requirements.yaml` manifest mixes machine-oriented content with human notes (e.g., `-> OK`, `-> Impossible to know`, speculative questions) inside `text` fields; consider moving reviewer commentary into separate metadata fields or comments so downstream tooling can reliably treat `text` as the canonical requirement body.
- In `rfc9000_requirements.yaml`, some fields are inconsistent in type/format (e.g., `section` sometimes quoted vs. numeric, requirement IDs like `rfc9000:5.2` reused for different subsections, and layered text that appears truncated or merged), which could complicate automated consumers; normalizing section identifiers and ensuring each requirement entry is cleanly extracted would make the manifest more robust.
- For the requirements marked with inline notes such as `-> not possible to test` or `-> impossible to know`, you may want to encode that constraint in structured form (e.g., `testable: false` or a `test_notes` field) rather than leaving `testable: true` so that test tooling and coverage reports more accurately reflect what can actually be verified.

## Individual Comments

### Comment 1
<location path="protocol-testing/quic/rfc9000_requirements.yaml" line_range="212-219" />
<code_context>
+    level: MAY
+    layer: address-validation
+    testable: true
+  rfc9000:8.2.1:
+    text: "Initiating Path Validation\n\nAn endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame\nto at least\
+      \ the smallest allowed maximum datagram size of 1200 bytes,\n**unless the anti-amplification limit for the path** does\
</code_context>
<issue_to_address>
**suggestion:** Remove embedded reviewer notes like "-> impossible to know ?" from the normative `text` field.

Mixing RFC language with inline reviewer commentary in `text` (e.g., `-> impossible to know ?`) can break tools that rely on this field containing only the requirement wording. Please move these annotations to a separate metadata field (such as `notes`) or to commit messages, and reserve `text` for the normative requirement only.

```suggestion
  rfc9000:8.2.1:
    text: "Initiating Path Validation\n\nAn endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame\nto at least\
      \ the smallest allowed maximum datagram size of 1200 bytes,\nunless the anti-amplification limit for the path does not\
      \ permit\nsending a datagram of this size."
    notes: "Previous inline reviewer note '-> impossible to know ?' was removed from the normative text and recorded here."
    section: 8.2.1
    level: MUST
    layer: address-validation
    testable: true
```
</issue_to_address>

### Comment 2
<location path="protocol-testing/quic/rfc9000_requirements.yaml" line_range="571" />
<code_context>
+    level: MUST NOT
+    layer: loss-recovery
+    testable: true
+  rfc9000:13.2.1.5:
+    text: "-> not possible to test\n\nIn order to assist loss detection at the sender, an endpoint SHOULD\t \t   In order\
+      \ to assist loss detection at the sender, an endpoint SHOULD\nsend an ACK frame immediately on receiving an ack-eliciting\
</code_context>
<issue_to_address>
**issue (testing):** Align the `testable` flag with the "not possible to test" note to avoid contradictory metadata.

The note says `-> not possible to test` while `testable` is `true`, which is inconsistent and may mislead tooling that uses `testable` to determine expected automated checks. Please either set `testable: false` or update/remove the note so they convey the same intent.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a QUIC RFC 9000 requirements manifest and an MCP “ivy-tools” live test report, while updating ivy-lsp and panther-ivy-plugin submodules to pick up MCP fixes and local-source auto-detection.

Changes:

  • Added protocol-testing/quic/rfc9000_requirements.yaml containing 101 extracted RFC 9000 requirements.
  • Added docs/ivy-tools-test-report.md documenting live verification results across 19 MCP tools.
  • Bumped ivy-lsp and panther-ivy-plugin submodule commits.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 11 comments.

File Description
submodules/panther-ivy-plugin Updates submodule pointer to a commit with local ivy-lsp source auto-detection.
submodules/ivy-lsp Updates submodule pointer to a commit with MCP tool fixes and dependency updates.
protocol-testing/quic/rfc9000_requirements.yaml Adds a new YAML requirements manifest intended to be machine-loadable by tooling.
docs/ivy-tools-test-report.md Adds a new report documenting tool verification, fixes, and known limitations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +28 to +33
text: "Issuing Connection IDs\n\nAn endpoint that initiates migration and requires non-zero-length\t An endpoint that\
\ initiates migration and requires non-zero-length\nconnection IDs SHOULD ensure that the pool of connection IDs\t \
\ connection IDs SHOULD ensure that the pool of connection IDs\navailable to its peer allows the peer to use a\
\ new connection ID on\t available to its peer allows the peer to use a new connection ID on\nmigration, as the peer\
\ will close the connection if the pool is\t migration, as the peer will be **unable** to respond if the pool is\n\
exhausted."
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The text fields appear to include extraction artifacts (duplicate sentences/columns, embedded “-> OK/Impossible to know” reviewer notes, odd backslash continuations, and inline commentary). If this manifest is consumed for traceability/coverage, these artifacts can break parsing/display and also make exact matching against RFC text unreliable. Suggestion: keep text strictly as the clean RFC requirement statement (prefer YAML block scalars | for readability), and move any commentary/feasibility notes into a separate field (e.g., notes: or rationale:) so tooling can treat requirement text as canonical.

Suggested change
text: "Issuing Connection IDs\n\nAn endpoint that initiates migration and requires non-zero-length\t An endpoint that\
\ initiates migration and requires non-zero-length\nconnection IDs SHOULD ensure that the pool of connection IDs\t \
\ connection IDs SHOULD ensure that the pool of connection IDs\navailable to its peer allows the peer to use a\
\ new connection ID on\t available to its peer allows the peer to use a new connection ID on\nmigration, as the peer\
\ will close the connection if the pool is\t migration, as the peer will be **unable** to respond if the pool is\n\
exhausted."
text: |
Issuing Connection IDs
An endpoint that initiates migration and requires non-zero-length
connection IDs SHOULD ensure that the pool of connection IDs
available to its peer allows the peer to use a new connection ID on
migration, as the peer will close the connection if the pool is
exhausted.

Copilot uses AI. Check for mistakes.
Comment on lines +89 to +90
text: "Endpoints MUST validate that received transport\t-> OK\nensures that that an attacker cannot influence the choice\
\ of\t parameters match received connection ID values."
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The text fields appear to include extraction artifacts (duplicate sentences/columns, embedded “-> OK/Impossible to know” reviewer notes, odd backslash continuations, and inline commentary). If this manifest is consumed for traceability/coverage, these artifacts can break parsing/display and also make exact matching against RFC text unreliable. Suggestion: keep text strictly as the clean RFC requirement statement (prefer YAML block scalars | for readability), and move any commentary/feasibility notes into a separate field (e.g., notes: or rationale:) so tooling can treat requirement text as canonical.

Copilot uses AI. Check for mistakes.
Comment on lines +165 to +168
text: '->> is it that now 128 bits is minimum size ?? "would be"


Servers MUST ensure that replay of tokens is prevented or limited.'
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The text fields appear to include extraction artifacts (duplicate sentences/columns, embedded “-> OK/Impossible to know” reviewer notes, odd backslash continuations, and inline commentary). If this manifest is consumed for traceability/coverage, these artifacts can break parsing/display and also make exact matching against RFC text unreliable. Suggestion: keep text strictly as the clean RFC requirement statement (prefer YAML block scalars | for readability), and move any commentary/feasibility notes into a separate field (e.g., notes: or rationale:) so tooling can treat requirement text as canonical.

Suggested change
text: '->> is it that now 128 bits is minimum size ?? "would be"
Servers MUST ensure that replay of tokens is prevented or limited.'
text: |
Servers MUST ensure that replay of tokens is prevented or limited.
notes: 'is it that now 128 bits is minimum size ?? "would be"'

Copilot uses AI. Check for mistakes.
Comment on lines +254 to +271
text: '-> OK

An endpoint MUST NOT delay transmission of a

packet containing a PATH_RESPONSE frame unless constrained by

congestion control.'
section: 8.2.2
level: MUST NOT
layer: address-validation
testable: true
rfc9000:8.2.2.3:
text: '-> Impossible to know


A PATH_RESPONSE frame MUST be sent on the network path where the

PATH_CHALLENGE frame was received.'
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The text fields appear to include extraction artifacts (duplicate sentences/columns, embedded “-> OK/Impossible to know” reviewer notes, odd backslash continuations, and inline commentary). If this manifest is consumed for traceability/coverage, these artifacts can break parsing/display and also make exact matching against RFC text unreliable. Suggestion: keep text strictly as the clean RFC requirement statement (prefer YAML block scalars | for readability), and move any commentary/feasibility notes into a separate field (e.g., notes: or rationale:) so tooling can treat requirement text as canonical.

Suggested change
text: '-> OK
An endpoint MUST NOT delay transmission of a
packet containing a PATH_RESPONSE frame unless constrained by
congestion control.'
section: 8.2.2
level: MUST NOT
layer: address-validation
testable: true
rfc9000:8.2.2.3:
text: '-> Impossible to know
A PATH_RESPONSE frame MUST be sent on the network path where the
PATH_CHALLENGE frame was received.'
text: |
An endpoint MUST NOT delay transmission of a packet containing a
PATH_RESPONSE frame unless constrained by congestion control.
notes: 'Reviewer feasibility: OK'
section: 8.2.2
level: MUST NOT
layer: address-validation
testable: true
rfc9000:8.2.2.3:
text: |
A PATH_RESPONSE frame MUST be sent on the network path where the
PATH_CHALLENGE frame was received.
notes: 'Reviewer feasibility: Impossible to know'

Copilot uses AI. Check for mistakes.
Comment on lines +45 to +47
rfc9000:5.2.2:
text: "An endpoint MUST generate a\t \t Initial, Retry, or Version Negotiation, MAY be discarded."
section: '5.2'
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requirement metadata looks inconsistent in a way that will make downstream grouping/navigation harder: (1) section sometimes doesn’t match the requirement id (e.g., rfc9000:5.2.2/5.2.3 have section: '5.2'), and (2) section values are inconsistently typed/quoted ('5.2' vs 5.2.1). Suggestion: normalize section to always be the precise subsection (e.g., 5.2.2) and always store it as a string for consistency across YAML parsers and tooling.

Copilot uses AI. Check for mistakes.
Comment on lines +58 to +61
rfc9000:5.2.1:
text: "Client Packet Handling\n\nIf a client receives a packet that has an **unsupported version**, it\t \t If a client\
\ receives a packet that uses a **different version** than it\nMUST discard that packet."
section: 5.2.1
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requirement metadata looks inconsistent in a way that will make downstream grouping/navigation harder: (1) section sometimes doesn’t match the requirement id (e.g., rfc9000:5.2.2/5.2.3 have section: '5.2'), and (2) section values are inconsistently typed/quoted ('5.2' vs 5.2.1). Suggestion: normalize section to always be the precise subsection (e.g., 5.2.2) and always store it as a string for consistency across YAML parsers and tooling.

Copilot uses AI. Check for mistakes.
Comment on lines +12 to +19
| Metric | Before | After |
|--------|--------|-------|
| Tools returning useful data | 8/19 | 19/19 |
| Visualization tools working | 0/7 | 7/7 |
| Semantic symbol queries working | 0/3 | 3/3 |
| Lint false positives (per file avg) | ~15 | 0 |
| RFC requirements loadable | 0 | 101 (after ISS-008 fix) |
| RequirementGraph nodes | 0 | 10,723 |
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tables use a leading empty column (|| ...) which renders as an extra blank column in Markdown. If that wasn’t intentional, switch to a standard table header row starting with a single | (e.g., | Metric | Before | After |) so the report renders cleanly and consistently across Markdown viewers.

Copilot uses AI. Check for mistakes.
**Symptom**: MCP log shows `WARNING: PyYAML not installed; cannot load manifest .../rfc9000_requirements.yaml`. Both traceability/coverage tools return 0 requirements. `model_summary.totals.rfcTagsTotal = 0`.
**Root cause**: `pyyaml` not listed in `pyproject.toml` `[project.optional-dependencies] mcp` extra. The `uvx` virtual environment doesn't install it.
**Fix**: Added `"pyyaml>=6.0"` to the `mcp` optional dependency list in `submodules/ivy-lsp/pyproject.toml`.
**Status**: Fix applied — requires MCP server restart to verify.
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The report is internally inconsistent about ISS-008 verification: it states the fix “requires MCP server restart to verify”, but later marks the traceability/coverage tools as PASS with 101 requirements loaded. Also ivy_model_summary() still calls out rfcTagsTotal=0 (ISS-008) even though the narrative says ISS-008 is fixed. Suggestion: align these sections to reflect the same post-fix state (e.g., either keep ISS-008 as “pending restart” and mark Step 4 as pending/partial, or update the symptom/status lines and the rfcTagsTotal note to match the verified run).

Suggested change
**Status**: Fix appliedrequires MCP server restart to verify.
**Status**: FIX VERIFIED — MCP server restarted; traceability/coverage tools now load 101 requirements and `model_summary.totals.rfcTagsTotal` reflects the loaded RFC tags.

Copilot uses AI. Check for mistakes.
Comment on lines +118 to +122
### Step 4: ISS-003+008 — ivy_traceability_matrix + ivy_requirement_coverage
| Tool | Result | Notes |
|------|--------|-------|
| `ivy_traceability_matrix()` | **PASS** | 101 total requirements loaded, 0 covered (ISS-004: bare tags) |
| `ivy_requirement_coverage()` | **PASS** | 101 total: MUST=45, MUST NOT=12, SHOULD=17, SHOULD NOT=3, MAY=24. 13 layers. |
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The report is internally inconsistent about ISS-008 verification: it states the fix “requires MCP server restart to verify”, but later marks the traceability/coverage tools as PASS with 101 requirements loaded. Also ivy_model_summary() still calls out rfcTagsTotal=0 (ISS-008) even though the narrative says ISS-008 is fixed. Suggestion: align these sections to reflect the same post-fix state (e.g., either keep ISS-008 as “pending restart” and mark Step 4 as pending/partial, or update the symptom/status lines and the rfcTagsTotal note to match the verified run).

Copilot uses AI. Check for mistakes.
### Step 6: ISS-001 — Visualization tools (core 4)
| Tool | Result | Key Metrics |
|------|--------|-------------|
| `ivy_model_summary()` | **PASS** | 486 actions, 8,128 reqs, 2,109 svars, rfcTagsTotal=0 (ISS-008) |
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The report is internally inconsistent about ISS-008 verification: it states the fix “requires MCP server restart to verify”, but later marks the traceability/coverage tools as PASS with 101 requirements loaded. Also ivy_model_summary() still calls out rfcTagsTotal=0 (ISS-008) even though the narrative says ISS-008 is fixed. Suggestion: align these sections to reflect the same post-fix state (e.g., either keep ISS-008 as “pending restart” and mark Step 4 as pending/partial, or update the symptom/status lines and the rfcTagsTotal note to match the verified run).

Suggested change
| `ivy_model_summary()` | **PASS** | 486 actions, 8,128 reqs, 2,109 svars, rfcTagsTotal=0 (ISS-008) |
| `ivy_model_summary()` | **PASS** | 486 actions, 8,128 reqs, 2,109 svars, rfcTagsTotal=101 (matches traceability tools) |

Copilot uses AI. Check for mistakes.
ElNiak added 25 commits March 13, 2026 08:57
- protocol-testing/patterns/: 13 reusable .ivy template files covering
  serdes (binary + JSON), variants, monitors, shims (UDP + TCP), modules,
  and entities (asymmetric + symmetric) patterns
- pattern_catalog.yaml: Machine-readable registry with detection heuristics
- Update ivy-lsp submodule (pattern detection engine + MCP tools)
- Update panther-ivy-plugin submodule (command + skill + enhanced workflow)
ivy-lsp: semantic edge wiring, smart suggestions fix, new MCP tools
(ivy_generate_manifest, ivy_scaffold_check, ivy_quality_gate),
workspace scoping, per-action state var filtering.

panther-ivy-plugin: latest plugin changes.
ivy-lsp: add ivy_diagnostics MCP tool, harden audit logging
panther-ivy-plugin: smart LSP source detection, MCP config improvements
…submodules

- Clean ~37 garbled requirement text fields (remove diff artifacts, tabs,
  editorial annotations like '-> OK', '-> impossible to know')
- Set testable: false for 3 untestable requirements
- Fix Section 14 layer: ecn -> datagram-size (5 requirements)
- Update ivy-lsp: ValueError fix, cache thread-safety, truncated flag, path order
- Update panther-ivy-plugin: JSON escaping, conditional --reinstall
- Create api/_shared.py so api/ package has self-contained imports
  (fixes broken `from ._shared import` in runner.py and discovery.py)
- Remove misleading "TODO not used" from PantherIvyVersion docstring
  (class IS used as VERSION_CLASS and type annotation)
- Update ivy-lsp submodule: mcp_server.py modularized into tools/ package
- Update panther-ivy-plugin submodule: all 25 MCP tools wired,
  shell scripts deduplicated
- Remove shadowed adapt_environment_paths from IvyProtocolAwareMixin (MRO dead code)
- Consolidate 3 protocol name methods into single get_protocol_name() from mixin
- Merge duplicate output pattern lists into IvyOutputPatternMixin (fix artifacts/ prefix)
- Remove dead build_tests() method (zero callers)
ivy-lsp: consolidate 25 MCP tools to 15, add counterexample parser,
per-isolate verification caching, coverage regression detection.

panther-ivy-plugin: consolidate agents (9→4), skills (14→6),
commands (6→5), soften hooks, update CLAUDE.md.
…lers + plugin tests)

- Add classify_endpoint_type() to _shared.py for directory-based test filtering
- Modify _build_ivy_model_setup_commands() to selectively copy only relevant
  test subdirectory (server_tests/, client_tests/, mim_tests/) instead of all
- Update submodules: ivy-lsp (goToImplementation + call hierarchy handlers),
  panther-ivy-plugin (55 integration tests + /nct-health command)
…ports

- Replace _extract_test_directory_from_name() body with classify_endpoint_type()
  to fix mim/attacker test directory divergence between setup and compilation
- Fix "strict subset" docstring claim (fallback copies ALL test files)
- Improve classify_endpoint_type() docstring (document priority order, attacker mapping)
- Add 3 edge case tests (empty string, mim+client precedence, attacker-only)
- Remove unused Protocol imports from panther_ivy.py and ivy_command_mixin.py
- Fix pattern_catalog.yaml: remove "variant this of" from entity markers (copy-paste),
  add note about monitor markers matching models not templates
Re-export classify_endpoint_type in api/_shared.py and rewrite
api/compiler.py::_extract_test_directory to delegate to it. Adds
attacker handling, correct mim>client>server precedence, and
role-aware fallback. Updates tests to match.
Removed exact duplicates: rfc9000:5.2.3 (=5.2.2), rfc9000:17.1.3
(=17.1.2), rfc9000:17.2.4.4 (=17.2.4.3), rfc9000:19.11.3 (=19.11.2).
Count: 101 → 97 requirements.
…misclassification

Replace bare substring matching with position-aware segment matching.
The function now splits test names by underscore and only checks segments
before the first "test" segment for endpoint keywords, preventing scenario
words like "mim" in "quic_client_test_0rtt_mim_replay" from overriding
the actual endpoint "client".

This fixes misclassification of 7+ real QUIC test files that contain
"mim" in their scenario portion but are client or server tests.
…emove duplicate -print

- Log warning when classify_endpoint_type falls back to role-based inference
- Narrow except Exception to (AttributeError, TypeError) in _process_commands()
- Remove duplicate -print from find-echo command that doubled entries in copied files list
- Fix 6 section-key mismatches in rfc9000_requirements.yaml where the
  section field was truncated relative to the key (e.g. key 5.2.2 had
  section 5.2). Affected keys: 5.2.2, 5.2.1.2, 7.3.2, 10.2.2, 14.2,
  17.2.5.3.8
- Update .ivyworkspace to version 2 with scope_detection: auto
- Update ivy-lsp submodule (gitignore fix for temp dir leakage)
- Update panther-ivy-plugin submodule (LSP env config, version bump,
  hook scoping, agent/skill improvements)
14 tests covering:
- Missing model path returns empty list with warning
- Endpoint-type-aware subdirectory selection (server, client, mim, attacker)
- Protocol name and $PYTHON_IVY_DIR references in generated commands
- Prune flag for tests directory exclusion
- Fallback warning logging when no endpoint keyword matches
- No warning for known endpoint keywords
- Different protocol names produce correct tests dir
- If/else fallback branch presence in shell commands
- Role-based inference via oppose_role when no keyword present
…commit

Re-point submodule after accidental pointer regression in previous commit.
ivy-lsp: workspaceSymbol active_filepath passthrough, test_file scoping for coverage tools
panther-ivy-plugin: nct-validate ground truth update (97 reqs), coverage scoping docs
ElNiak added 30 commits March 18, 2026 15:38
…udes, LSP guard

- Delete redundant apt_time.ivy; standardize 68 APT files on quic_time
- Fix 7 phantom 'include endpoint' → protocol-specific endpoints
- Fix 2 phantom 'include ivy_stream_data_attacker' → client+server split
- Fix 1 phantom 'include ivy_minip_attacker' → client+server split
- Remove dead 'include ivy_system_shim' from ivy_system_mim.ivy
- Update ivy-lsp submodule (diagnostics guard, prior LSP fixes)
- MCP verification tools (ivy_verify, ivy_compile, ivy_show) now prefer
  layer-specific staging dirs over flat staging for colliding basenames
- Collision guard in include resolver falls through to workspace root
  instead of returning None for unmapped files
- SessionStart hook kills stale ivy_lsp PIDs before spawning new ones
- ivy-lsp: tool consolidation, MCP HTTP sidecar, stdio↔HTTP bridge
- panther-ivy-plugin: consolidate plugins, mcp-bridge mode, skill updates
- Restructure .ivyworkspace: split monolithic 'apt' into apt_core,
  apt_quic_fork, apt_minip_fork, apt_tls_fork, apt_tests sub-layers
- Split quic into quic (core) and quic_tests layers
- Eliminates ~158 basename collisions from apt/quic fork duplicates
- Update ivy-lsp submodule (diagnostics, performance, resolution fixes)
- Update panther-ivy-plugin submodule (docs, debounce config)
- Add missing #lang ivy1.7 to 4 QUIC spec files (quality gate fix)
- Fix include naming mismatch: ivy_quic_shim_client_example_ext →
  ivy_quic_shim_client_ext_example
- Update ivy-lsp submodule with ToolTraceContext fix, visualization
  pipeline diagnostics, stdlib fallback expansion, and Docker tool guards
Remove the mcp-bridge startup mode that caused a race condition
with the lazily-started LSP sidecar. MCP now starts immediately
in standalone mode using the offline index.
…MCP)

ivy-lsp: sidecar_client module, sidecar monitor, safe_tool delegation
panther-ivy-plugin: mcp-bridge → standalone mcp mode, script cleanup
Restore 3 .ivy files that were deleted from the APT protocol fork while
still being actively referenced by other APT files:

- quic_frame.ivy + quic_frame.md (apt_protocols/quic/quic_stack/)
  deleted in 78ef4c8, referenced by 10 APT files
- apt_time.ivy (apt_network/)
  deleted in 78ef4c8, referenced via include apt_time
- ping_byte_stream.ivy (apt_protocols/minip/minip_stack/)
  deleted in 5e24328, referenced by ping_application.ivy

Revert 'include quic_time' back to 'include apt_time' in 35 .ivy
and 35 .md files — cross-protocol includes break both ivyc compilation
and ivy-lsp indexing since each workspace must be self-contained.

Eliminates all "module not found" indexing errors for the APT protocol.
…re depends_on

- Add workspace_groups field to root .ivyworkspace for logical protocol groupings (quic, apt, apt_quic, minip, bgp, coap, scaffolds)
- Remove false depends_on from apt_core layer (APT is autonomous with its own quic file copies)
- Create per-protocol .ivyworkspace markers for quic, apt, minip, bgp, coap with workspace_root_offset="../.." for auto-scoping
- CoAP retains depends_on: ["quic"] pending cleanup of copy-paste QUIC artifacts
- Add .ivy-workspace-state.json to .gitignore (session-specific, not committed)
…y fixes)

ivy-lsp: MCP crash fix, sidecar race condition, diagnostics hang,
compiler semaphore timeout, duplicate logging, diagnostics cache.

panther-ivy-plugin: health check retry logic.
- ivy-lsp: isError:true on timeouts, sidecar timeout, cancel-safe helper
- panther-ivy-plugin: health check hook timing fix (6s→3s)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants