Skip to content

Fixed lazy URL service handing out URLs for non-public resources#28890

Open
allouis wants to merge 5 commits into
mainfrom
lazy-url-non-public-resources
Open

Fixed lazy URL service handing out URLs for non-public resources#28890
allouis wants to merge 5 commits into
mainfrom
lazy-url-non-public-resources

Conversation

@allouis

@allouis allouis commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

While the lazy URL service runs alongside eager in compare mode, the lazy forward path handed out URLs for resources eager returns /404/ for — most visibly internal/private tags (the reported divergence), and also draft/scheduled/sent posts.

Root cause

Eager only builds URLs for resources that pass each type's resource-fetch filter (visibility:public for tags, status:published+type:* for posts/pages), applied at fetch time. The lazy forward path (getUrlForResource/ownsResource) only consulted the per-router filter, which is null for taxonomy routers, so any tag/post matched.

Changes (5 atomic commits)

  1. Fixed lazy URL service handing out URLs for non-public resources — lazy applies the same per-type base filter (BASE_FILTERS, an explicit inline constant duplicated from services/url/config.js for now — lazy is meant to replace eager and consolidate) in getUrlForResource/ownsResource. A resource that reaches URL generation must carry the columns its base filter reads (status/visibility); production callers always do (full models, or forced by the serializers), so a thin resource throws LAZY_URL_THIN_RESOURCE rather than silently 404 a URL eager would produce. Authors have no base filter (users.visibility is schema-pinned to 'public'). The url-service-parity integration test fed eager's cached resources (which drop those columns after fetch-time filtering) — updated it to re-add them so it exercises the shape lazy actually receives.
  2. Added getRequiredFields to the lazy URL service and facade — exposes the per-type columns the lazy backend needs (mirrors getRequiredRelations); returns [] under eager.
  3. Forced lazy URL base-filter columns in the API input serializers — under ?fields=url the selected columns are narrowed, so posts/pages/tags serializers force the base-filter columns back into the fetch. No-op under eager.
  4. Forced permalink columns for lazy URL generation under ?fields=url — lazy builds URLs by substituting resource fields into the permalink template (slug, id, published_at for date tokens), which ?fields=url also strips. getRequiredFields reports each type's permalink scalar columns; permalink relations stay covered by getRequiredRelations.
  5. Required router-filter scalar columns on the lazy forward path — a router filter's own-columns (e.g. featured:true) were neither reported by getRequiredFields nor required by the thin-check, so a thin record under ?fields=url could route a featured post to /:slug/ instead of /featured/:slug/. Now the non-relation scalar fields a router filter references are reported (so the serializers force them) and required on the forward path (relations and the page/type discriminator stay excluded).

The eager services/url/config.js is not modified.

Testing (run locally)

  • Unit: test/unit/api + test/unit/server + test/unit/frontend — all green.
  • Integration: full test/integration suite green (incl. url-service-parity.test.js).

Known remaining divergence (follow-up, not in this PR)

  • shouldHavePosts: eager omits tags/authors with zero published posts from its forward map → /404/; lazy's forward path still returns a URL (the reverse resolveUrl path already matches, via TagPublic). Needs post-count data on the resource. Low impact: nothing branches on a forward tag/author /404/, and the archive page itself still 404s via the reverse path.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Serializer inputs for authors, pages, posts, and tags now call a lazy URL column helper. LazyUrlService adds per-resource base filters, required-field calculation, and base-filter checks in URL and ownership resolution. UrlServiceFacade exposes required-field lookup. Tests were added and updated for the helper, facade delegation, service routing, ownership, and parity behavior.

Possibly related PRs

  • TryGhost/Ghost#28891: Also adjusts lazy URL-related resource payloads so router filters can evaluate against thin inputs.

Suggested reviewers

  • rob-ghost
  • vershwal
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: lazy URL generation no longer serves URLs for non-public resources.
Description check ✅ Passed The description is clearly aligned with the patch and explains the lazy URL service, serializer, and parity test changes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch lazy-url-non-public-resources

Comment @coderabbitai help to get the list of available commands.

@nx-cloud

nx-cloud Bot commented Jun 25, 2026

Copy link
Copy Markdown

🤖 Nx Cloud AI Fix

Ensure the fix-ci command is configured to always run in your CI pipeline to get automatic fixes in future runs. For more information, please see https://nx.dev/ci/features/self-healing-ci


View your CI Pipeline Execution ↗ for commit 44fd692

Command Status Duration Result
nx run ghost:test:ci:integration ✅ Succeeded 2m 39s View ↗
nx run ghost:test:integration ✅ Succeeded 2m 30s View ↗
nx run ghost:test:legacy ✅ Succeeded 2m 50s View ↗
nx run ghost:test:e2e ✅ Succeeded 2m 13s View ↗
nx run-many --target=build --projects=tag:publi... ✅ Succeeded 1s View ↗
nx run-many -t lint -p ghost ✅ Succeeded 26s View ↗
nx run-many -t test:unit -p ghost ✅ Succeeded 30s View ↗
nx run @tryghost/admin:build ✅ Succeeded 9s View ↗
Additional runs (2) ✅ Succeeded ... View ↗

💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗


☁️ Nx Cloud last updated this comment at 2026-06-25 12:12:57 UTC

@github-actions

Copy link
Copy Markdown
Contributor

E2E Tests Failed

To view the Playwright test report locally, run:

REPORT_DIR=$(mktemp -d) && gh run download 28148235613 -n playwright-report -D "$REPORT_DIR" && npx playwright show-report "$REPORT_DIR"

@allouis allouis force-pushed the lazy-url-non-public-resources branch from 0160c12 to 484df86 Compare June 25, 2026 05:31
@github-actions

Copy link
Copy Markdown
Contributor

E2E Tests Failed

To view the Playwright test report locally, run:

REPORT_DIR=$(mktemp -d) && gh run download 28149109638 -n playwright-report -D "$REPORT_DIR" && npx playwright show-report "$REPORT_DIR"

@allouis allouis force-pushed the lazy-url-non-public-resources branch 3 times, most recently from 20bd27f to b4d2de8 Compare June 25, 2026 07:44
allouis added 4 commits June 25, 2026 08:14
- eager only builds URLs for resources passing each type's resource-fetch filter
  (visibility:public for tags, status:published for posts/pages); the lazy forward
  path only consulted the per-router filter (null for taxonomy routers), so it
  handed out URLs for internal tags and draft posts where eager returns /404/
- apply the same per-type base filter (inline BASE_FILTERS, duplicated from the
  eager config for now) in getUrlForResource/ownsResource
- a resource that reaches URL generation must carry the columns its base filter
  reads (status/visibility); production callers always do (full models, or forced
  by the serializers), so a thin one throws LAZY_URL_THIN_RESOURCE rather than
  silently 404 a URL eager would produce
- the parity integration test fed eager's *cached* resources, which drop those
  columns; updated it to re-add them (everything cached is published/public) so it
  exercises the shape lazy actually receives
- the lazy URL service needs each resource's base-filter columns (status for
  posts/pages, visibility for tags/authors) to decide routability; under a
  Content API ?fields=url query those columns get stripped before the URL is
  generated, so the service would throw LAZY_URL_THIN_RESOURCE
- expose the per-type base-filter fields so the API serializers can force the
  columns to be fetched, mirroring getRequiredRelations for router relations
- returns [] with no lazy backend, so eager callers are unaffected
- under a Content API ?fields=url query the selected columns are narrowed, so a
  resource reaches URL generation without the columns the lazy service reads
  (status for posts/pages, visibility for tags) and is rejected as thin
- add forceUrlColumnsWhenLazy to the input url serializer util and call it from
  the posts/pages/tags input serializers so those columns are force-fetched when
  url is requested; the response is still trimmed to ?fields
- no-op under the eager service (getRequiredFields returns [])
- the lazy URL service builds URLs by substituting the resource's fields into the
  router permalink template (slug, id, published_at for :year/:month/:day), so it
  needs those columns present; eager never does (it looks URLs up by id)
- a Content API ?fields=url query strips them, so lazy produced broken URLs like
  /tag// — a divergence from eager
- extend getRequiredFields to also report the scalar columns each type's router
  permalinks reference, so the existing serializer column-forcing loads them
- add the authors serializer hook: authors have no base filter but their
  /author/:slug/ permalink still needs slug
- permalink relations (primary_tag/primary_author) stay covered by getRequiredRelations
@allouis allouis force-pushed the lazy-url-non-public-resources branch from b4d2de8 to 9ba39b3 Compare June 25, 2026 08:16
@allouis allouis requested a review from vershwal June 25, 2026 08:37
@allouis allouis marked this pull request as ready for review June 25, 2026 08:37

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ghost/core/core/server/services/url/lazy-url-service.ts`:
- Around line 153-173: getRequiredFields() currently only collects
permalink-derived fields and base filters, so router-specific scalar filter
fields like featured or page can be missing and cause lazy routing to
mis-evaluate. Update getRequiredFields() in lazy-url-service.ts to also inspect
each matching router config’s filter definitions and add any non-relation scalar
fields referenced by config.filter to the required field set, alongside
slug/id/published_at, so thin resources still include the values needed by
filterMatches() on the forward path.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c99cc864-7856-4ede-89ca-0075d004af53

📥 Commits

Reviewing files that changed from the base of the PR and between 0888b43 and 9ba39b3.

📒 Files selected for processing (11)
  • ghost/core/core/server/api/endpoints/utils/serializers/input/authors.js
  • ghost/core/core/server/api/endpoints/utils/serializers/input/pages.js
  • ghost/core/core/server/api/endpoints/utils/serializers/input/posts.js
  • ghost/core/core/server/api/endpoints/utils/serializers/input/tags.js
  • ghost/core/core/server/api/endpoints/utils/serializers/input/utils/url.js
  • ghost/core/core/server/services/url/lazy-url-service.ts
  • ghost/core/core/server/services/url/url-service-facade.ts
  • ghost/core/test/integration/url-service-parity.test.js
  • ghost/core/test/unit/api/canary/utils/serializers/input/utils/url.test.js
  • ghost/core/test/unit/server/services/url/lazy-url-service.test.js
  • ghost/core/test/unit/server/services/url/url-service-facade.test.js

Comment thread ghost/core/core/server/services/url/lazy-url-service.ts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
ghost/core/core/server/services/url/lazy-url-service.ts (1)

253-263: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Apply thin-resource checks before ownership matching.

ownsResource() now gates on base filters, but it still evaluates router filters on whatever fields happen to be present. A thin record missing featured, tags, etc. can skip an earlier filtered router and incorrectly let a later catch-all router claim ownership.

Suggested fix
         // router owns it. Mirrors the base-filter gate in getUrlForResource.
         const record = this._recordForFilter(resource);
+        if (this._hasRouterForType(routerType)) {
+            this._assertBaseFieldsPresent(routerType, resource);
+        }
         if (!this._baseFilterMatches(routerType, record)) {
             return false;
         }
         // Ownership is exclusive: only the first matching router of the type
         // owns the resource, matching eager's reservation.
-        const owner = this.routerConfigs.find(
-            c => c.resourceType === routerType && filterMatches(c.compiledFilter, record)
-        );
+        const owner = this.routerConfigs.find((c) => {
+            if (c.resourceType !== routerType) {
+                return false;
+            }
+            this._assertNotThin(c, resource, routerType);
+            return filterMatches(c.compiledFilter, record);
+        });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ghost/core/core/server/services/url/lazy-url-service.ts` around lines 253 -
263, Update ownsResource() in lazy-url-service.ts so thin records are rejected
before router ownership matching: after _recordForFilter(resource) and the base
filter check, add the same thin-resource validation used elsewhere before
calling routerConfigs.find/filterMatches. This should ensure missing fields like
featured or tags do not let a later catch-all router claim ownership. Reference
ownsResource(), _recordForFilter(), _baseFilterMatches(), and
routerConfigs.find() when making the change.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ghost/core/core/server/services/url/lazy-url-service.ts`:
- Around line 76-83: The field extraction logic in lazyUrlService’s filter
parsing is too broad because the regex scans the entire filter string and can
treat colon-containing values as fake fields. Update the extraction in the same
filter-handling path used by buildFilter to only match real filter-expression
field names, or otherwise anchor matches to expression boundaries before adding
to fields. Keep the existing FILTER_NON_SCALAR_FIELDS and _assertNotThin flow,
but ensure the matcher in the field-collection logic does not capture URLs,
timestamps, or other literal values.

---

Outside diff comments:
In `@ghost/core/core/server/services/url/lazy-url-service.ts`:
- Around line 253-263: Update ownsResource() in lazy-url-service.ts so thin
records are rejected before router ownership matching: after
_recordForFilter(resource) and the base filter check, add the same thin-resource
validation used elsewhere before calling routerConfigs.find/filterMatches. This
should ensure missing fields like featured or tags do not let a later catch-all
router claim ownership. Reference ownsResource(), _recordForFilter(),
_baseFilterMatches(), and routerConfigs.find() when making the change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 738b22d3-35ab-40c5-ac92-1fbc33a7c287

📥 Commits

Reviewing files that changed from the base of the PR and between 9ba39b3 and 0d86527.

📒 Files selected for processing (2)
  • ghost/core/core/server/services/url/lazy-url-service.ts
  • ghost/core/test/unit/server/services/url/lazy-url-service.test.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • ghost/core/test/unit/server/services/url/lazy-url-service.test.js

Comment thread ghost/core/core/server/services/url/lazy-url-service.ts Outdated
- getRequiredFields and the forward thin-check only covered base-filter and
  permalink columns plus router-filter *relations*; a router filter's scalar
  fields (e.g. featured:true, a custom collection filter) were neither reported
  nor required
- so under ?fields=url a thin record reached the service, filterMatches saw the
  field as undefined, and a featured post routed to /:slug/ instead of
  /featured/:slug/ (or 404'd) — a divergence from eager, whose cache keeps those
  columns
- extract the non-relation scalar fields a router filter references; report them
  from getRequiredFields (so the serializers force them) and require them in
  _assertNotThin (relations and page/type discriminator stay excluded)
@allouis allouis force-pushed the lazy-url-non-public-resources branch from 0d86527 to 44fd692 Compare June 25, 2026 12:03

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
ghost/core/core/server/services/url/lazy-url-service.ts (1)

257-264: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Apply the same thin-resource guard in ownership checks.

ownsResource() now evaluates the base filter, but unlike getUrlForResource() it does not call _assertBaseFieldsPresent() first. If a thin post/page/tag lacks status, type, or visibility, filterMatches() can return false and make a valid resource look unowned instead of raising LAZY_URL_THIN_RESOURCE.

🐛 Proposed fix
         // A resource failing its base filter is not in eager's map, so no
         // router owns it. Mirrors the base-filter gate in getUrlForResource.
         const record = this._recordForFilter(resource);
+        this._assertBaseFieldsPresent(routerType, resource);
         if (!this._baseFilterMatches(routerType, record)) {
             return false;
         }

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: fd6703fa-a871-4acc-bce7-e741548b3d79

📥 Commits

Reviewing files that changed from the base of the PR and between 0d86527 and 44fd692.

📒 Files selected for processing (2)
  • ghost/core/core/server/services/url/lazy-url-service.ts
  • ghost/core/test/unit/server/services/url/lazy-url-service.test.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • ghost/core/test/unit/server/services/url/lazy-url-service.test.js

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants