[BUGFIX] Warn instead of raise when schema absent from get_schema_names() in test_connection (#10499)#11879
Conversation
…es() in test_connection (fivetran#10499) `TableAsset.test_connection` raised TestConnectionError immediately when the configured schema was not present in `inspector.get_schema_names()`. On SQL Server (and other backends where users lack VIEW DEFINITION permission), get_schema_names() only returns schemas the connected user can describe—not every schema they can access. This caused false-positive failures: users with only SELECT permission on a table could not call test_connection even though the table was reachable. Fix: replace the hard raise with a LOGGER.warning and let the subsequent table-level SELECT 1 query act as the definitive accessibility check. If the schema or table is genuinely absent the SQL error surfaces a clear TestConnectionError; if it is accessible the connection is confirmed. Also adds two unit tests covering both the "schema absent but table reachable" and "schema absent and table inaccessible" paths. Fixes fivetran#10499
👷 Deploy request for niobium-lead-7998 pending review.Visit the deploys page to approve it
|
|
A new contributor, HUZZAH! Welcome and thanks for joining our community. In order to accept a pull request we require that all contributors sign our Contributor License Agreement. We have two different CLAs, depending on whether you are contributing to GX in a personal or professional capacity. Please sign the one that is applicable to your situation so that we may accept your contribution: Individual Contributor License Agreement v1.0 Once you have signed the CLA, you can add a comment with the text Please reach out to the #gx-community-support channel, on our Slack if you have any questions or if you have already signed the CLA and are receiving this message in error. Users missing a CLA: creazyfrog |
for more information, see https://pre-commit.ci
|
A new contributor, HUZZAH! Welcome and thanks for joining our community. In order to accept a pull request we require that all contributors sign our Contributor License Agreement. We have two different CLAs, depending on whether you are contributing to GX in a personal or professional capacity. Please sign the one that is applicable to your situation so that we may accept your contribution: Individual Contributor License Agreement v1.0 Once you have signed the CLA, you can add a comment with the text Please reach out to the #gx-community-support channel, on our Slack if you have any questions or if you have already signed the CLA and are receiving this message in error. Users missing a CLA: creazyfrog |
|
Is this PR still relevant? If so, what is blocking it? Is there anything you can do to help move it forward? This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions 🙇 |
1 participant
Summary
Fixes #10499
TableAsset.test_connectionraisedTestConnectionErrorimmediately when the configured schema name was not returned byinspector.get_schema_names(). On SQL Server (and potentially other backends),get_schema_names()only lists schemas the connected user hasVIEW DEFINITIONpermission on. A user with onlySELECTpermission on a table in a given schema can still access the table, but the schema is absent from the inspector's list — causing false-positive failures.The error message was:
even though the table was fully accessible.
Root Cause
The pre-check on
inspector.get_schema_names()is unreliable when the connected user lacksVIEW DEFINITIONon the schema. It is redundant anyway because the subsequentSELECT 1 FROM <table> LIMIT 1query serves as a definitive accessibility test.Changes
great_expectations/datasource/fluent/sql_datasource.py— Replace the hardraise TestConnectionErrorwheneffective_schema not in schema_nameswith aLOGGER.warning. The table-level SELECT query that follows will catch any genuine connection failures and raiseTestConnectionErrorwith a clear message.tests/datasource/fluent/test_sql_datasources.py— AddTestTableAssetTestConnectionwith two unit tests:test_schema_absent_from_get_schema_names_does_not_raise: verifies that when the schema is omitted byget_schema_names()but the table query succeeds, no error is raised.test_schema_absent_and_table_inaccessible_raises: verifies that if the table query also fails,TestConnectionErroris still surfaced.Workaround (before fix)
Users had to use
add_query_asset()instead ofadd_table_asset()to bypass the schema pre-check.Test plan
TestTableAssetTestConnection::test_schema_absent_from_get_schema_names_does_not_raiseTestTableAssetTestConnection::test_schema_absent_and_table_inaccessible_raises