Skip to content

Correlation event-ID: extend to the Squid access log + proxy_logs (5b) #107

Description

@fabriziosalmi

Follow-up to the WAF-minted correlation event-ID (5a, shipped in Wave 2): the WAF
now stamps every inspected request with an event_id, carried in the
/data/waf_traffic.jsonl record and in the block notification / websocket event
(via the alert details). This links WAF traffic log ↔ notification ↔ live feed.

5b extends the same ID to the Squid access log → proxy_logs row, so a
blocked request in the Logs page / analytics can be joined back to its WAF event
end-to-end. Deferred from 5a because it carries real, stack-specific risk that
needs live verification on CT130, not just CI:

Plan

  1. sendBlockResponse (waf-go) sets an ICAP/HTTP response header X-WAF-Event-Id
    on the 403, carrying the same eventID.
  2. proxy/startup.sh: define a custom Squid logformat that keeps the 10 stock
    fields in exact order
    and appends the event id as field 11; switch
    access_log ... squidaccess_log ... spm.
  3. log_tailer.go parseSquidLine: read the trailing field into event_id; add an
    event_id TEXT column (idempotent ALTER + index, after the column migrations
    per the PR perf(db): indexed blocked flag to replace LIKE '%DENIED%' full scans (Wave 2) #104 ordering lesson) + include it in the insert and the websocket
    broadcast.

Why it needs live testing (the risk)

  • Positional-parse fragility: parseSquidLine is positional and requires
    len(fields) >= 10. If the custom logformat reorders or drops any stock field,
    every log line fails to parse → the Logs page + analytics ingestion break.
    Must be validated against the real Squid build on CT130.
  • The correct logformat code is uncertain: X-WAF-Event-Id is added by the
    ICAP service on the response path. Whether Squid surfaces it via %{...}>h,
    %{...}<h, or an adaptation %note{...} depends on the Squid/ICAP behavior and
    must be confirmed empirically.
  • Coverage gaps to document: allowed requests (204, no modified response) carry
    no header → event_id is - for non-blocked rows (acceptable — those are the
    rows you don't need to trace). SSL-bumped CONNECT tunnels may not get a per-inner-
    request header.

Acceptance

  • A blocked request shows the SAME event_id in the WAF JSONL, the notification,
    and its proxy_logs row.
  • parseSquidLine still parses every normal line (no ingestion regression) — verified
    live on CT130 before release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions