Test the deployed examples site on every PR - #956
Merged
Conversation
`res.status(303).redirect(url)` does not produce a 303. Passing only a URL to `redirect()` makes Next answer 307 and ignore the status set before it, and 307 preserves the method, so the browser re-POSTed to the page and got a blank 405. Logging in through this example has been landing on an empty page. Broken since #631. The new deployed e2e suite is what caught it.
The existing Playwright suite asserts library behaviour against a fixture on localhost. Nothing checked the site the README sends people to. That gap is why v9 shipped with every example returning an empty 500: `examples/next/src/passwords.ts` started reading the session passwords from the environment, nothing set them on the Vercel project, and because the password is read per request the build stayed green throughout. So this suite runs the visitor flows against a deployed URL given by `BASE_URL`: login, a session write, persistence across a reload, logout, the proxy redirect, the protected pages, and both magic-link paths. Two things it has to work around, both of which cost real debugging time: - The SWR examples render their login form in the server HTML and submit it through a React `onSubmit` that calls `preventDefault()`. Clicking before hydration posts to the page URL instead of the session route, so `openExample` waits for the client's session GET first. - They also pass `optimisticData`, so the logged-in UI appears before the POST that sets the cookie has answered. Asserting the rendered state proves nothing, so `login` waits for the cookie and the counter waits for its PATCH.
Vercel's git integration fires `deployment_status` when a deployment finishes, with the URL in `environment_url`. A separate workflow keyed on that event is the simplest way to test the real preview: it does not exist yet when a `pull_request` run starts, and waiting for it inside ci.yaml would mean polling the Vercel API with a token. It runs for previews and for production, so a PR is checked before merge and the live site is checked again after. Also builds the examples app in the quality job. Nothing in CI built it, so a break in it only surfaced as a failed deploy after merge.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Every preview is behind Vercel Authentication, so the whole suite failed on the first run: each request answered 302 to vercel.com/sso-api, including the home page. Previews stay private and CI passes the project's Protection Bypass for Automation secret as a header instead. The magic-link test now clears cookies on its own context rather than opening a new one. `browser.newContext()` does not inherit the config's `use` options, so the fresh context did not carry the bypass header.
The deployed suite needs BASE_URL and, on a preview, the bypass secret. Neither is guessable. The two waits it depends on look like flakiness, so they are written down before someone removes them.
vvo
marked this pull request as ready for review
August 30, 2026 22:20
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tl;dr; the live examples site has been returning 500s on every login since v9, and nothing would have caught it, so we test the real Vercel deployment on every PR.
Problem
Solution
Implementation details
e2e/examples.spec.tsruns againstBASE_URLviaplaywright.deployed.config.ts, chromium only, since the fixture suite already covers Firefox and WebKit..github/workflows/e2e-deployed.yamlkeys ondeployment_status, taking the URL fromenvironment_url. Apull_requestrun starts before the preview exists, and polling the Vercel API would need a token.onSubmit, so a click before hydration posts to the page URL.openExamplewaits for the client's session GET first.optimisticData, so the logged-in UI appears before the cookie exists.loginwaits for the cookie and the counter waits for its PATCH.res.redirect(303, url).res.status(303).redirect(url)answers 307, which keeps the method, so the browser re-POSTed to the page and got a 405.testaggregator, so it needs adding to branch protection to block merges.