Skip to content

Track QueryBuilder reads for AutoRefresh - #2760

Draft
mpscholten wants to merge 2 commits into
masterfrom
codex/track-autorefresh-queries
Draft

Track QueryBuilder reads for AutoRefresh#2760
mpscholten wants to merge 2 commits into
masterfrom
codex/track-autorefresh-queries

Conversation

@mpscholten

@mpscholten mpscholten commented Jul 15, 2026

Copy link
Copy Markdown
Member

Summary

  • retain structured reads from the normal IHP QueryBuilder instead of reducing every fetch to a table name
  • capture canonical primary-key values returned by row fetches, including custom and composite keys
  • distinguish row, count, and exists reads while keeping manual/raw SQL reads conservative
  • preserve the existing ModelContext.trackTableReadCallback and withTableReadTracker APIs through an internal metadata bridge
  • bound retained queries and result keys, falling back to whole-table dependencies when a safe bound or row identity is unavailable
  • register notification triggers before publishing a rerender's dependency snapshot, with retry-safe and atomic session updates

Why

This is a QueryBuilder-focused foundation for improving the approach discussed in #2305.

AutoRefresh currently records only a Set Text of table names. That is not enough to determine whether an INSERT, UPDATE, or DELETE can affect an action's rendered result. The previous smart-filtering prototype extracted row IDs and WHERE conditions into separate maps, which lost query semantics such as ordering, limits, offsets, counts, and existence checks.

This change retains the compiled SQLQuery and the canonical keys actually returned by row queries. It does not change notification filtering yet; current table-level refresh behavior remains in place while the data needed for safe relevance checks is introduced.

Impact

The public autoRefresh do ..., ModelContext.trackTableReadCallback, and withTableReadTracker APIs remain unchanged. The original AutoRefreshSession constructor also remains available for compatibility.

Normal QueryBuilder fetches are tracked automatically. Outside AutoRefresh they continue to use the existing statements. Inside AutoRefresh, row queries are wrapped in a CTE that preserves their filtering, ordering, distinctness, limit, and offset while also returning primary-key values as JSON.

Manual trackTableRead calls, tables without usable primary keys, oversized result sets, and unknown structured reads remain conservative whole-table dependencies. Tracking retains at most 64 queries per table and 1024 returned keys per query.

Trigger registration is serialized and only committed to server state after both trigger installation and PGListener subscription succeed. A failed rerender registration leaves the previous dependency snapshot intact, and development rerenders reinstall idempotent trigger SQL after make db recreates the database.

Follow-up work

  • reuse or extract DataSync's row-notification infrastructure
  • implement conservative relevance checks from the tracked SQLQuery and returned keys
  • keep complex or unsupported queries on the whole-table fallback
  • add end-to-end PostgreSQL notification relevance coverage

Validation

  • loaded the changed modules with GHC 9.10.3 and -Werror=unused-imports
  • ran the complete IHP GHCi suite: 577 examples, 0 failures (26 database-dependent examples pending)
  • ran the focused AutoRefresh suite against an isolated PostgreSQL instance: 10 examples, 0 failures
  • ran the Nix package check phase with PostgreSQL: 577 examples, 0 failures, including the composite-primary-key fetch
  • git diff --check

@github-actions

Copy link
Copy Markdown

Core Size & Compile Allocations Benchmark

Metric Baseline (master) This PR Change
Core size 10733081 bytes 10849013 bytes 1.1%
Compile allocations 27222790216 bytes 27498764704 bytes 1.0%

Core size within threshold
Compile allocations within threshold

HTTP Latency (GET /, 5000 reqs, 10 concurrent)

Metric Baseline (master) This PR Change
Mean 3.31ms 3.22ms -2.7%
p50 2.00ms 2.10ms
p99 28.80ms 22.00ms
Min 0.60ms 0.60ms
Max 202.00ms 222.30ms
Req/s 2629 2811

HTTP latency within threshold

Top 10 modules (this PR)

Module Size (bytes)
Web.Types.thr 547347
Web.Routes.thr 409719
Web.Controller.Comments.thr 396368
Web.Controller.Threads.thr 313601
Web.View.Threads.Show.thr 303761
build.Generated.Enums.thr 278101
Admin.Types.thr 264263
build.Generated.User.thr 259635
Admin.Routes.thr 255763
Web.Controller.Users.thr 254027

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant