Skip to content

Feature/wsjtx sse enrichment#900

Open
ceotjoe wants to merge 8 commits intoaccius:Stagingfrom
ceotjoe:feature/wsjtx-sse-enrichment
Open

Feature/wsjtx sse enrichment#900
ceotjoe wants to merge 8 commits intoaccius:Stagingfrom
ceotjoe:feature/wsjtx-sse-enrichment

Conversation

@ceotjoe
Copy link
Copy Markdown
Collaborator

@ceotjoe ceotjoe commented Apr 9, 2026

What does this PR do?

Summary

This PR moves the WSJTX decode enrichment intelligence from the OHC server into the local rig-bridge process, so SSE consumers receive fully enriched events even when no server relay is configured.

  • Local enrichment library (rig-bridge/lib/wsjtx-enrich.js): Maidenhead grid → lat/lon, Hz → band name, FT8/FT4 message parser (CQ/QSO type, caller, modifier, embedded grid), per-client grid cache (2 h TTL), and full enrichment helpers for DECODE, STATUS, QSO_LOGGED, and WSPR_DECODE messages.
  • HamQTH callsign lookup (Phase 5, opt-in): fire-and-forget background resolution of callsign → lat/lon via hamqth.com/dxcc.php; 24 h cache, max 5 concurrent requests. On resolution a decode-update SSE event is emitted so the frontend can live-patch map pins already on screen. Toggled via a new checkbox in the rig-bridge Integrations tab.
  • End-to-end wiring (rig-bridge.js): decode, status, qso, clear, wspr, and decode-update bus events are now all forwarded to SSE with their full enriched field sets (previously only a trimmed subset was forwarded, silently dropping all enrichment).
  • Frontend (useWSJTX.js): handles new SSE event types — clear (per-client filtering), wspr (capped ring buffer), decode-update (async HamQTH patch). Band-change detection uses the bandChanged flag from enriched status; falls back to manual comparison on the server/relay path.
  • Bug fix: wsjtx-enrich was emitting lowercase 'cq'/'qso' type strings; changed to uppercase 'CQ'/'QSO' to match the server convention and fix CQ filter + row colouring in PSKReporterPanel.
  • Bug fix: ModernLayout was not passing wsjtxWspr to PSKReporterPanel — the WSPR tab now receives data in that layout.
  • Enrichment on the relay path is unchanged — the rig-bridge relay queue still sends raw messages so the server can do its own enrichment without double-processing.

Test plan

  • Run rig-bridge in SSE-only mode (no server relay): confirm DECODE events in PSKReporterPanel show band, grid, lat/lon, CQ/QSO type and row colour
  • Enable HamQTH callsign lookup in the Integrations tab: confirm callsigns without an embedded grid eventually get lat/lon resolved and map pins appear
  • Confirm WSPR tab populates in ModernLayout (was broken before this PR)
  • Confirm CLEAR from WSJT-X wipes only the correct client's decodes
  • Run rig-bridge with a server relay configured: confirm server-side enrichment still works and no fields are duplicated or corrupted
  • Check rig-bridge status endpoint reports gridCacheSize, callsignCacheSize, and hamqthLookup flag correctly
  • Verify MSHV / JTDX / JS8Call (digital-mode-base.js) receive the same enrichment as WSJT-X

Checklist

  • App loads without console errors
  • Tested in Dark, Light, and Retro themes
  • Responsive at different screen sizes (desktop + mobile)
  • If touching server.js: caches have TTLs and size caps (we serve 2,000+ concurrent users)
  • If adding an API route: includes caching and error handling
  • If adding a panel: wired into Modern, Classic, and Dockable layouts
  • No hardcoded colors — uses CSS variables (var(--accent-cyan), etc.)
  • No .bak, .old, console.log debug lines, or test scripts included

ceotjoe and others added 6 commits April 7, 2026 21:40
Port the intelligence layer from the OHC server's WSJTX route into the
rig-bridge plugin stack so SSE consumers receive rich events even when
no server relay is configured.

New shared library — rig-bridge/lib/wsjtx-enrich.js
  • gridToLatLon()      — Maidenhead grid → lat/lon (pure math)
  • getBandFromHz()     — Hz → band name (160 m – 70 cm)
  • createGridCache()   — callsign → grid cache (2 h TTL, per-instance)
  • parseDecodeMessage()— FT8/FT4 text parser: CQ/QSO type, caller,
                          modifier, dxCall/deCall, embedded grid;
                          populates grid cache as a side-effect
  • enrichDecode()      — full decode enrichment + grid-cache fallback
  • enrichStatus()      — band name, band-change flag, DX/DE lat/lon
  • enrichQso()         — band name, dxGrid → lat/lon
  • enrichWspr()        — band name, grid → lat/lon

wsjtx-relay.js changes (Phases 1–4)
  • CLEAR message → new `clear` bus event (was silently dropped before)
  • WSPR_DECODE   → new `wspr` bus event with enriched fields
  • STATUS        → enriched with band, bandChanged, dxLat/dxLon, deLat/deLon
  • DECODE        → enriched with parsed message fields, grid → lat/lon,
                    content-based dedup ID; duplicates are suppressed
  • QSO_LOGGED    → enriched with band + lat/lon; 60 s call+freq+mode dedup
  • Grid cache pruned every 5 min; cleared per-client on CLEAR
  • getStatus() now reports gridCacheSize
  • Relay queue still receives raw (un-enriched) messages — server does
    its own enrichment on the relay path (no double-processing)

digital-mode-base.js changes (MSHV / JTDX / JS8Call)
  • Same enrichment applied via wsjtx-enrich helpers
  • CLEAR forwarded as `clear` bus event (was silently dropped)
  • STATUS/DECODE/QSO enriched identically to wsjtx-relay
  • lastBand added to getStatus() alongside existing lastMode

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…chment

Adds opt-in background callsign → lat/lon resolution via the HamQTH DXCC
API, matching the server's Phase 5 behaviour but running entirely inside
the local rig-bridge process.

wsjtx-enrich.js
  • createCallsignCache() — 24 h TTL cache for HamQTH results, separate
    from the 2 h gridCache (different source, different lifetime)
  • triggerHamqthLookup(callsign, cache, inflight, onResult) — fire-and-
    forget HTTPS GET to hamqth.com/dxcc.php; max 5 concurrent requests
    (inflightSet guard); writes result into callsignCache and calls
    onResult so callers can emit a decode-update event
  • enrichDecode() — accepts optional 5th param callsignCache; consults
    it as a third fallback tier between gridCache and giving up

wsjtx-relay.js
  • Instantiates callsignCache + hamqthInflight per plugin instance
  • Passes callsignCache to enrichDecode() when cfg.hamqthLookup is true
  • After emitting a decode with no lat/lon, calls triggerHamqthLookup
    and on resolution emits a decode-update bus event with the resolved
    callsign + coordinates for frontend live-patching
  • gridPruneInterval now also prunes callsignCache every 5 min
  • getStatus() reports callsignCacheSize, hamqthLookup flag, hamqthInflight

rig-bridge/core/server.js (Integrations tab UI)
  • New "HamQTH callsign lookup" checkbox (id=wsjtxHamqth) with help text
    explaining the internet requirement and 24 h cache
  • populateIntegrations() reads w.hamqthLookup → checkbox
  • saveIntegrations() writes checkbox → wsjtxRelay.hamqthLookup
  • Status line now shows callsign cache size when hamqthLookup is active
    and omits relay count when running in SSE-only mode

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rig-bridge/rig-bridge.js
  • decode handler was building a trimmed object that silently dropped all
    fields added by wsjtx-enrich (lat, lon, band, grid, type, caller,
    modifier, dxCall, deCall, gridSource). Now passes them all through.
  • id forwarded directly from enrichDecode (content-based) instead of
    being rebuilt; raw-decode fallback retained for non-enriched plugins.
  • status handler forwards new enriched fields: band, bandChanged,
    dxLat, dxLon, deLat, deLon.
  • qso handler forwards: band, lat, lon, frequency, myCall, myGrid.
  • New clear handler → broadcasts clientId + window to SSE.
  • New wspr handler → broadcasts full enriched WSPR decode to SSE.
  • New decode-update handler → broadcasts HamQTH-resolved callsign
    coordinates to SSE for live map-pin patching.

src/hooks/useWSJTX.js
  • status handler now stores band, bandChanged, dxLat, dxLon so DX
    target resolution and band-change detection work without server.
  • Band-change detection uses the bandChanged flag from the enriched
    status event; falls back to manual prev-band comparison for the
    server/relay path.
  • New clear event handler: filters decodes by clientId.
  • New wspr event handler: prepends to wspr array (capped at 100).
  • New decode-update event handler: patches existing decodes that share
    the resolved callsign and have no lat/lon yet (HamQTH async result).

src/layouts/ModernLayout.jsx
  • Pass wsjtxWspr={wsjtx.wspr} to PSKReporterPanel so the WSPR tab
    actually receives data (DockableApp already had this; ModernLayout
    was the only layout missing it).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nvention

PSKReporterPanel filters on d.type === 'CQ' (uppercase), matching the
OHC server's parseDecodeMessage output. wsjtx-enrich was using lowercase
'cq'/'qso', causing the CQ filter and the blue CQ row colour to never
match in local SSE mode.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ceotjoe ceotjoe requested a review from alanhargreaves April 9, 2026 21:36
@alanhargreaves
Copy link
Copy Markdown
Collaborator

I'm happy that it does what it says it does. I'm not seeing any errors from the rig-bridge node.js nor extras from the clients.

I checked out the debug stuff that was in there, and the api that had some stats on port 5555. Also played with adding in the gridSource to the spots on WorldMap to verify that we were getting grid locs from different places.

I feel I was a bit close to this one to approve it, and I'd also be more comfortable with you doing the best-practice javascript once over, given there is some claude code in there.

@ceotjoe ceotjoe marked this pull request as ready for review April 12, 2026 15:52
@ceotjoe
Copy link
Copy Markdown
Collaborator Author

ceotjoe commented Apr 12, 2026

@accius if you don't like the .gitignore leave it. I just want to avoid that this thing (Claude.md) get's into the repo accidentally.

Copy link
Copy Markdown
Owner

@accius accius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review — WSJT-X SSE enrichment

Really solid work. Local enrichment makes the SSE-only mode genuinely useful, and the separation (raw → server relay, enriched → bus) is clean. Good test plan too. A handful of issues to fix before merge:

Bugs

  1. Duplicate type key in rig-bridge.js decode forwarder:

    type: msg.message.startsWith('CQ') ? 'CQ' : 'QSO',
    ...
    type: msg.type,

    Both keys are on the same object literal — the second overwrites the first and the first line is dead (and throws TypeError if msg.message is ever undefined, which is guarded elsewhere with ?? ''). Remove the first.

  2. clientStates[msg.id]?.deCall / deGrid never populated. In both digital-mode-base.js and wsjtx-relay.js, clientStates[msg.id] is set to { band, dialFrequency, mode } only. The QSO_LOGGED handler falls back to clientStates[msg.id]?.deCall || null, which is always undefined → null. Either store deCall/deGrid from STATUS into clientStates, or drop the fallback and accept myCall/myGrid being null when WSJT-X doesn't supply them.

  3. isGrid case-sensitivity bug: the regex /^[A-R]{2}\d{2}(?:[A-Xa-x]{2})?$/ has no i flag and is tested against the original input (not the uppercased g). A 4-char lowercase grid (em28) returns false. FT8 messages are uppercase in practice, so low-impact, but trivial to fix — test against g, or add the i flag.

  4. pluginBus.on('wspr', …) forwards msg directly as data, including the internal source field and no field whitelisting — every other handler carefully selects fields. Be consistent, and be careful not to leak fields the UI shouldn't see.

Safety / robustness

  1. HamQTH XML parsed by regex (<lat>([^<]+)<\/lat>). Works for the current endpoint but brittle against whitespace/attributes. Consider a minimal XML parser or at least matching against <lat[^>]*>(-?[0-9.]+)</lat> to constrain to numerics.

  2. HamQTH public dxcc.php — confirm this endpoint permits unauthenticated automated use at our expected volume (5 concurrent × many users). If ToS requires a session token, this breaks silently. Worth documenting in the README.

  3. No persistent HamQTH cache. Every rig-bridge restart re-queries — not a correctness issue, but if we're asking users to enable this across the fleet we may want to persist a JSON cache file.

  4. Rate limiting only via HAMQTH_MAX_CONCURRENT = 5. There is no overall QPS limit; on a busy band a burst of unknown calls could trigger dozens of requests per minute. Consider a short (~2s) per-callsign cool-down plus a global QPS cap.

  5. SEEN_DECODE_MAX = 2000 — reasonable, but check memory behavior on a very active 20m FT8 session over 24h. Should be fine with FIFO eviction.

Minor

  1. .gitignore adds CLAUDE.md — harmless, but unrelated to this PR's topic. Happy to leave it.
  2. dt is stored as a pre-formatted string (toFixed(1)) in enrichDecode, but useWSJTX and the UI may want the numeric form for sorting. Worth passing both or keeping numeric and formatting at render time.
  3. Setup HTML adds hardcoded colors #1a1f2e, #2d3548, #6b7280 — matches existing style in this file, so not a regression, but a note for a future cleanup.
  4. .gitignore-ing CLAUDE.md with no comment — a one-line comment would help future readers.

Checklist: all UI-related items are unchecked — please verify at least the Modern layout WSPR tab and the CQ filter in PSKReporterPanel before merge.

Overall: fix 1–3 (real bugs) and this is ready.

ceotjoe and others added 2 commits April 14, 2026 22:09
- Remove dead duplicate `type` key in rig-bridge.js decode handler
  (could throw TypeError when msg.message is undefined)
- Whitelist fields in wspr bus handler instead of forwarding raw msg
- Fix isGrid() to test regex against uppercased input so lowercase
  grids like em28 are recognised correctly
- Tighten HamQTH XML regex to accept only numeric lat/lng values,
  rejecting arbitrary text content
- Store deCall/deGrid in clientStates from STATUS in both wsjtx-relay
  and digital-mode-base so QSO_LOGGED myCall/myGrid fallback works

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rate limiting (item 8):
- Add per-callsign 60s cooldown via lastAttemptedMap in triggerHamqthLookup
- Add global 2 req/s QPS cap via module-level sliding-window timestamp array
- Both guards are in wsjtx-enrich.js; wsjtx-relay passes lastAttemptedMap

Persistent HamQTH cache (item 7):
- Add loadCallsignCache/saveCallsignCache helpers to wsjtx-enrich.js
- createCallsignCache().set() now accepts an optional timestamp to preserve
  original expiry time when loading persisted entries
- wsjtx-relay loads cache from hamqth-cache.json at connect, flushes on
  disconnect, and does a debounced 30s save after each successful lookup

dt as numeric (item 11):
- enrichDecode/enrichWspr now store dt as float (msg.deltaTime ?? 0)
  instead of a pre-formatted string
- PSKReporterPanel formats dt at render time with sign and one decimal place;
  handles both legacy string values (server path) and new numeric values

Documentation / housekeeping:
- .gitignore: add one-line comment explaining CLAUDE.md entry (item 13)
- rig-bridge/README.md: add HamQTH callsign lookup section documenting ToS
  context, rate limits, cache behaviour, and network requirements (item 6)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants