@W-21304863: Add Tableau OAuth E2E tests#245
Conversation
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Drop explicit axios dependency now that build/tests pass via transitive usage, and reorder imports to satisfy simple-import-sort. Co-authored-by: Cursor <cursoragent@cursor.com>
Use the pulse definitions scope for listing metrics by definition id to avoid 403s. Co-authored-by: Cursor <cursoragent@cursor.com>
Remove token exchange tasks and questions from the plan, noting it is out of scope for initial release. Co-authored-by: Cursor <cursoragent@cursor.com>
Drop admin scope config in favor of derived MCP scopes, enforce per-request MCP scopes with optional API scope checks, and update OAuth metadata/docs accordingly. Co-authored-by: Cursor <cursoragent@cursor.com>
Gate API scope advertisement and validation behind a new flag for Falcon, update auth metadata and scope validation, and document the option. Co-authored-by: Cursor <cursoragent@cursor.com>
Normalize formatting and import order after rebase conflict resolution. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Pull request overview
Adds Playwright-based E2E coverage for Tableau MCP OAuth flows where the OAuth issuer is the Tableau AuthZ server, exercising a number of MCP tools after browser-driven sign-in/consent.
Changes:
- Introduces a Playwright test suite (
tests/oauth/tableau-authz) with OAuth login/consent automation and tool-level assertions. - Adds a custom OAuth-capable MCP client + Playwright fixtures to authenticate once per worker and reuse across tests.
- Adds Playwright configuration, npm script/dependency, and a composite GitHub Action stub to run these tests.
Reviewed changes
Copilot reviewed 32 out of 34 changed files in this pull request and generated 24 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/oauth/tableau-authz/tests/testEnv.ts | Helpers to resolve known Tableau content IDs from env + tests/constants. |
| tests/oauth/tableau-authz/tests/searchContent.test.ts | E2E test for search-content. |
| tests/oauth/tableau-authz/tests/queryDatasource.test.ts | E2E test for query-datasource using VizQL Data Service schema. |
| tests/oauth/tableau-authz/tests/oauth.test.ts | Sanity check that all tools are listed. |
| tests/oauth/tableau-authz/tests/listWorkbooks.test.ts | Skipped E2E test for list-workbooks. |
| tests/oauth/tableau-authz/tests/listViews.test.ts | Skipped E2E test for list-views. |
| tests/oauth/tableau-authz/tests/listPulseMetricsFromMetricIds.test.ts | E2E test for pulse metrics lookup by metric IDs. |
| tests/oauth/tableau-authz/tests/listPulseMetricsFromMetricDefinitionId.test.ts | E2E test for pulse metrics lookup by definition ID. |
| tests/oauth/tableau-authz/tests/listPulseMetricSubscriptions.test.ts | Skipped E2E test for pulse subscriptions. |
| tests/oauth/tableau-authz/tests/listPulseMetricDefinitionsFromDefinitionIds.test.ts | E2E test for pulse metric definitions lookup by IDs. |
| tests/oauth/tableau-authz/tests/listDatasources.test.ts | Skipped E2E test for list-datasources. |
| tests/oauth/tableau-authz/tests/listAllPulseMetricDefinitions.test.ts | E2E test for listing all pulse metric definitions. |
| tests/oauth/tableau-authz/tests/getWorkbook.test.ts | E2E test for get-workbook. |
| tests/oauth/tableau-authz/tests/getViewImage.test.ts | E2E test for get-view-image. |
| tests/oauth/tableau-authz/tests/getViewData.test.ts | Skipped E2E test for get-view-data. |
| tests/oauth/tableau-authz/tests/getDatasourceMetadata.test.ts | E2E test for get-datasource-metadata. |
| tests/oauth/tableau-authz/tests/generatePulseMetricValueInsightBundleTool.test.ts | E2E test for generating a pulse insight bundle. |
| tests/oauth/tableau-authz/tests/generatePulseInsightBriefTool.test.ts | E2E test for generating a pulse insight brief (AI-enabled site). |
| tests/oauth/tableau-authz/tests/base.ts | Playwright test fixture wiring for env + OAuth client. |
| tests/oauth/tableau-authz/testEnv.ts | Loads/validates OAuth test credentials from .env.oauth. |
| tests/oauth/tableau-authz/oauthClient.ts | MCP client wrapper using Streamable HTTP transport + OAuth provider + parsing helpers. |
| tests/oauth/tableau-authz/flows/loginFlow.ts | Page-object for Tableau login steps. |
| tests/oauth/tableau-authz/flows/flow.ts | Base page-object class. |
| tests/oauth/tableau-authz/flows/consentFlow.ts | Page-object for granting consent when prompted. |
| tests/oauth/tableau-authz/fixtures/env.ts | Playwright fixture to provide validated env to tests. |
| tests/oauth/tableau-authz/fixtures/connectOAuthClient.ts | Worker fixture to run OAuth flow via Playwright and provide authenticated client. |
| tests/oauth/tableau-authz/env.oauth.example.list | Example env file for local OAuth test credentials. |
| tests/e2e/getDatasourceMetadata.test.ts | Improves invariant error message. |
| tests/constants.ts | Adds server/site/content IDs for additional Tableau test environments. |
| playwright.config.ts | Adds Playwright config and validates env for starting the MCP server. |
| package.json | Adds test:oauth:tableau script and Playwright dev dependency. |
| package-lock.json | Locks Playwright dependency tree. |
| .gitignore | Ignores Playwright output directories. |
| .github/actions/run-oauth-tableau-authz-tests/action.yml | Composite action stub to run the Playwright OAuth suite. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
tests/oauth/tableau-authz/tests/listPulseMetricsFromMetricDefinitionId.test.ts
Outdated
Show resolved
Hide resolved
tests/oauth/tableau-authz/tests/listPulseMetricsFromMetricIds.test.ts
Outdated
Show resolved
Hide resolved
tests/oauth/tableau-authz/tests/listPulseMetricSubscriptions.test.ts
Outdated
Show resolved
Hide resolved
| }, | ||
| }, | ||
| }, | ||
| 'https://test-dataplane1.tableau.sfdc-ckzqgc.svc.sfdcfc.net': { |
There was a problem hiding this comment.
im abit confused on how this is used
There was a problem hiding this comment.
This file contains the metadata for various content keyed by Tableau pod and site. It makes it so, say, the query-datasource test doesn't first have to call the list-datasources tool to figure out what data source luid to use. Ideally these tests could run against any site, but we're not there yet.
list-pulse-metric-subscriptionsis skipped due to user ID missing because it's not provided in the access token for users without MFA enabled.