Skip to content

@W-21304863: Add Tableau OAuth E2E tests#245

Open
anyoung-tableau wants to merge 74 commits intomainfrom
anyoung/oauth-e2e
Open

@W-21304863: Add Tableau OAuth E2E tests#245
anyoung-tableau wants to merge 74 commits intomainfrom
anyoung/oauth-e2e

Conversation

@anyoung-tableau
Copy link
Collaborator

@anyoung-tableau anyoung-tableau commented Mar 3, 2026

  • These changes introduce E2E tests for Tableau MCP with OAuth issuer as the Tableau AuthZ server.
  • They use Playwright to automate the Tableau sign in flow in the browser.
  • Until we have a stable test environment in Stage the tests will not run as part of CI.
  • All tool tests are basic and only test happy path. list-pulse-metric-subscriptions is skipped due to user ID missing because it's not provided in the access token for users without MFA enabled.
image

mattcfilbert and others added 30 commits February 11, 2026 16:31
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Drop explicit axios dependency now that build/tests pass via transitive usage, and reorder imports to satisfy simple-import-sort.

Co-authored-by: Cursor <cursoragent@cursor.com>
Use the pulse definitions scope for listing metrics by definition id to avoid 403s.

Co-authored-by: Cursor <cursoragent@cursor.com>
Remove token exchange tasks and questions from the plan, noting it is out of scope for initial release.

Co-authored-by: Cursor <cursoragent@cursor.com>
Drop admin scope config in favor of derived MCP scopes, enforce per-request MCP scopes with optional API scope checks, and update OAuth metadata/docs accordingly.

Co-authored-by: Cursor <cursoragent@cursor.com>
Gate API scope advertisement and validation behind a new flag for Falcon, update auth metadata and scope validation, and document the option.

Co-authored-by: Cursor <cursoragent@cursor.com>
Normalize formatting and import order after rebase conflict resolution.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@anyoung-tableau anyoung-tableau marked this pull request as ready for review March 18, 2026 18:35
Copilot AI review requested due to automatic review settings March 18, 2026 18:35
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Playwright-based E2E coverage for Tableau MCP OAuth flows where the OAuth issuer is the Tableau AuthZ server, exercising a number of MCP tools after browser-driven sign-in/consent.

Changes:

  • Introduces a Playwright test suite (tests/oauth/tableau-authz) with OAuth login/consent automation and tool-level assertions.
  • Adds a custom OAuth-capable MCP client + Playwright fixtures to authenticate once per worker and reuse across tests.
  • Adds Playwright configuration, npm script/dependency, and a composite GitHub Action stub to run these tests.

Reviewed changes

Copilot reviewed 32 out of 34 changed files in this pull request and generated 24 comments.

Show a summary per file
File Description
tests/oauth/tableau-authz/tests/testEnv.ts Helpers to resolve known Tableau content IDs from env + tests/constants.
tests/oauth/tableau-authz/tests/searchContent.test.ts E2E test for search-content.
tests/oauth/tableau-authz/tests/queryDatasource.test.ts E2E test for query-datasource using VizQL Data Service schema.
tests/oauth/tableau-authz/tests/oauth.test.ts Sanity check that all tools are listed.
tests/oauth/tableau-authz/tests/listWorkbooks.test.ts Skipped E2E test for list-workbooks.
tests/oauth/tableau-authz/tests/listViews.test.ts Skipped E2E test for list-views.
tests/oauth/tableau-authz/tests/listPulseMetricsFromMetricIds.test.ts E2E test for pulse metrics lookup by metric IDs.
tests/oauth/tableau-authz/tests/listPulseMetricsFromMetricDefinitionId.test.ts E2E test for pulse metrics lookup by definition ID.
tests/oauth/tableau-authz/tests/listPulseMetricSubscriptions.test.ts Skipped E2E test for pulse subscriptions.
tests/oauth/tableau-authz/tests/listPulseMetricDefinitionsFromDefinitionIds.test.ts E2E test for pulse metric definitions lookup by IDs.
tests/oauth/tableau-authz/tests/listDatasources.test.ts Skipped E2E test for list-datasources.
tests/oauth/tableau-authz/tests/listAllPulseMetricDefinitions.test.ts E2E test for listing all pulse metric definitions.
tests/oauth/tableau-authz/tests/getWorkbook.test.ts E2E test for get-workbook.
tests/oauth/tableau-authz/tests/getViewImage.test.ts E2E test for get-view-image.
tests/oauth/tableau-authz/tests/getViewData.test.ts Skipped E2E test for get-view-data.
tests/oauth/tableau-authz/tests/getDatasourceMetadata.test.ts E2E test for get-datasource-metadata.
tests/oauth/tableau-authz/tests/generatePulseMetricValueInsightBundleTool.test.ts E2E test for generating a pulse insight bundle.
tests/oauth/tableau-authz/tests/generatePulseInsightBriefTool.test.ts E2E test for generating a pulse insight brief (AI-enabled site).
tests/oauth/tableau-authz/tests/base.ts Playwright test fixture wiring for env + OAuth client.
tests/oauth/tableau-authz/testEnv.ts Loads/validates OAuth test credentials from .env.oauth.
tests/oauth/tableau-authz/oauthClient.ts MCP client wrapper using Streamable HTTP transport + OAuth provider + parsing helpers.
tests/oauth/tableau-authz/flows/loginFlow.ts Page-object for Tableau login steps.
tests/oauth/tableau-authz/flows/flow.ts Base page-object class.
tests/oauth/tableau-authz/flows/consentFlow.ts Page-object for granting consent when prompted.
tests/oauth/tableau-authz/fixtures/env.ts Playwright fixture to provide validated env to tests.
tests/oauth/tableau-authz/fixtures/connectOAuthClient.ts Worker fixture to run OAuth flow via Playwright and provide authenticated client.
tests/oauth/tableau-authz/env.oauth.example.list Example env file for local OAuth test credentials.
tests/e2e/getDatasourceMetadata.test.ts Improves invariant error message.
tests/constants.ts Adds server/site/content IDs for additional Tableau test environments.
playwright.config.ts Adds Playwright config and validates env for starting the MCP server.
package.json Adds test:oauth:tableau script and Playwright dev dependency.
package-lock.json Locks Playwright dependency tree.
.gitignore Ignores Playwright output directories.
.github/actions/run-oauth-tableau-authz-tests/action.yml Composite action stub to run the Playwright OAuth suite.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@anyoung-tableau anyoung-tableau changed the title Add Tableau OAuth E2E tests @W-21304863: Add Tableau OAuth E2E tests Mar 18, 2026
},
},
},
'https://test-dataplane1.tableau.sfdc-ckzqgc.svc.sfdcfc.net': {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im abit confused on how this is used

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file contains the metadata for various content keyed by Tableau pod and site. It makes it so, say, the query-datasource test doesn't first have to call the list-datasources tool to figure out what data source luid to use. Ideally these tests could run against any site, but we're not there yet.

Copy link
Contributor

@mattcfilbert mattcfilbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants