Add filtering and pagination to reduce LLM context consumption

[key]

Background

When an LLM calls sigrok-mcp-server tools, every token in the response consumes context window budget. Several tools return large unfiltered datasets that waste tokens.

Current State

• list_supported_hardware (handlers.go:71-74) — returns ALL 160+ drivers, no filtering
• list_supported_decoders (handlers.go:76-79) — returns ALL 110+ decoders, no filtering
• list_input_formats / list_output_formats — returns complete lists (smaller, ~10-20 items)
• show_decoder_details (handlers.go:104-124) — includes full multi-line documentation text
• show_driver_details (handlers.go:126-146) — includes both parsed fields AND RawOutput
• decode_protocol (handlers.go:310-382) — returns potentially megabytes of raw decoded text
• All list tools call sigrok-cli -L and parse sections via runListSection (handlers.go:91-102)

Problem

• An LLM looking for "spi" decoder gets all 110+ decoders dumped into context
• Driver details include redundant RawOutput field alongside parsed data
• decode_protocol output can be extremely large with no truncation or summary
• No tool supports filtering by keyword, limiting results, or pagination

Proposed Changes

1. Add optional filter string parameter to list_supported_hardware and list_supported_decoders
   • Filter by substring match on id and description
2. Remove RawOutput field from DriverDetails response (parsed data is sufficient)
3. Add optional include_documentation boolean to show_decoder_details (default: false)
4. Add max_lines or limit parameter to decode_protocol for truncating large outputs
5. Include result count in all list responses so LLM knows the full dataset size

Acceptance Criteria

• list_supported_hardware and list_supported_decoders accept optional filter param
• Filtered results return only matching items with total count
• show_driver_details no longer returns RawOutput
• show_decoder_details documentation is opt-in
• decode_protocol supports output size limiting
• Existing tests updated; new tests for filter behavior

Affected Files

• internal/tools/tools.go — add filter/limit params to tool definitions
• internal/tools/handlers.go — implement filtering logic
• internal/sigrok/parser.go — update DriverDetails struct (remove RawOutput)
• internal/tools/handlers_test.go — update tests

[xXMrNidaXx]

Excellent issue — context window efficiency is critical for MCP servers. Large unbounded responses are one of the most common causes of LLM tool call failures. Here are some implementation patterns that work well:

Pagination Pattern

For large lists, a cursor-based approach lets the LLM request more if needed:

type ListDecodersInput struct {
    Filter   string `json:"filter,omitempty"`    // substring match
    Limit    int    `json:"limit,omitempty"`     // default 20
    Cursor   string `json:"cursor,omitempty"`    // base64 encoded offset
}

type ListDecodersResponse struct {
    Decoders   []Decoder `json:"decoders"`
    TotalCount int       `json:"total_count"`    // always include
    NextCursor string    `json:"next_cursor,omitempty"` // if more available
}

Key insight: Include total_count even when paginated so the LLM knows the full scope without fetching everything.

Tiered Detail Levels

For detail endpoints, offer summary vs. full modes:

type ShowDecoderInput struct {
    DecoderID string `json:"decoder_id"`
    Detail    string `json:"detail,omitempty"` // "summary" (default) | "full"
}

• summary: id, description, required_channels, annotation_classes (100-200 tokens)
• full: adds documentation, options, examples (could be 1000+ tokens)

Default to minimal; let LLM explicitly request more.

Output Truncation with Indication

For decode_protocol, truncation with clear signaling:

type DecodeProtocolResponse struct {
    Output      string `json:"output"`
    Truncated   bool   `json:"truncated"`
    TotalLines  int    `json:"total_lines"`
    ReturnedLines int  `json:"returned_lines"`
    // Optional: byte range for follow-up requests
    StartByte   int64  `json:"start_byte,omitempty"`
    EndByte     int64  `json:"end_byte,omitempty"`
}

If truncated, LLM can request specific ranges or apply additional filtering.

Filter Implementation

For hardware/decoder filtering, prefix matching often works better than substring:

func matchesFilter(item, filter string) bool {
    filter = strings.ToLower(filter)
    id := strings.ToLower(item.ID)
    desc := strings.ToLower(item.Description)
    
    // Prefix match on ID ("spi" matches "spi", "spiflash")
    if strings.HasPrefix(id, filter) {
        return true
    }
    // Substring match on description
    if strings.Contains(desc, filter) {
        return true
    }
    return false
}

General MCP Context Optimization Tips

1. Default limits everywhere — unbounded responses are bugs
2. Structured over prose — JSON fields cheaper than parsing paragraphs
3. Count before content — `{"total": 110}" helps LLM decide whether to filter
4. Stable ordering — consistent output = better LLM reasoning

The goal: an LLM should be able to explore the tool's capabilities incrementally without burning 10K tokens on the first call.

Happy to discuss specific implementation details!