Skip to content

[BUGFIX] Fix exception_info nested under metric ID key instead of flat structure (#10849)#11878

Open
creazyfrog wants to merge 4 commits into
fivetran:developfrom
creazyfrog:fix/exception-info-nested-under-metric-id-key-10849
Open

[BUGFIX] Fix exception_info nested under metric ID key instead of flat structure (#10849)#11878
creazyfrog wants to merge 4 commits into
fivetran:developfrom
creazyfrog:fix/exception-info-nested-under-metric-id-key-10849

Conversation

@creazyfrog

@creazyfrog creazyfrog commented May 17, 2026

Copy link
Copy Markdown

Summary

Fixes #10849

When a metric resolution fails (e.g. a validation expectation references a non-existent column on a Spark/Databricks datasource), graph_validate passed the entire metric_exception_info dict directly as the exception_info field of ExpectationValidationResult. Because that dict is keyed by stringified metric IDs (e.g. "('column_values.nonnull.condition', '...')"), downstream consumers received a nested structure like:

# Actual (broken)
{
    "('column_values.nonnull.condition', '242ce...')": {
        "raised_exception": True,
        "exception_traceback": "...",
        "exception_message": "..."
    }
}

instead of the documented flat structure:

# Expected
{
    "raised_exception": True,
    "exception_traceback": "...",
    "exception_message": "..."
}

This broke any code that inspects validation_result["results"][i]["exception_info"] programmatically (monitoring dashboards, alerting pipelines, etc.).

Changes

  • great_expectations/validator/validator.py — In _resolve_metrics_and_identify_aborted_expectations, extract the first ExceptionInfo object from metric_exception_info and pass it directly to ExpectationValidationResult so callers always receive the flat {raised_exception, exception_traceback, exception_message} shape.
  • tests/validator/test_validator.py — Update test_validator_with_exception_info_in_result to assert the flat structure rather than the old nested-by-metric-ID structure.

How to reproduce

import great_expectations as gx

context = gx.get_context()
# add a Spark datasource ...
vd = context.validation_definitions.add(
    gx.ValidationDefinition(
        name="test",
        data=batch_definition,
        suite=gx.ExpectationSuite(
            name="test",
            expectations=[
                gx.expectations.ExpectColumnValuesToNotBeNull(column="___nonexistent___")
            ]
        )
    )
)
result = vd.run()
print(result["results"][0]["exception_info"])  # was nested under metric ID, now flat

Test plan

  • Updated test_validator_with_exception_info_in_result in tests/validator/test_validator.py to assert the flat exception_info structure
  • Manually verified that a Spark validation that raises produces a flat exception_info with the correct raised_exception, exception_traceback, and exception_message keys

@creazyfrog
[BUGFIX] Fix exception_info nested under metric ID key instead of fla…
753b56f 
…t structure (fivetran#10849)

When a metric resolution fails (e.g. referencing a non-existent column),
`graph_validate` passed the entire `metric_exception_info` dict—keyed by
stringified metric IDs like `"('column_values.nonnull.condition', '...')"` —
directly as the `exception_info` field of `ExpectationValidationResult`.

This produced a nested structure:
  {"('column_values.nonnull.condition', '...')": {"raised_exception": true, ...}}

instead of the documented flat structure:
  {"raised_exception": true, "exception_traceback": "...", "exception_message": "..."}

Fix: extract the first ExceptionInfo object from the metric exception dict and
pass it directly to ExpectationValidationResult so callers always see the
expected flat exception_info shape.

Fixes fivetran#10849
@netlify

netlify Bot commented May 17, 2026 •
edited
Loading

Copy link
Copy Markdown

👷 Deploy request for niobium-lead-7998 pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit aafe386

@gx-cla-bot

gx-cla-bot Bot commented May 17, 2026

Copy link
Copy Markdown

A new contributor, HUZZAH! Welcome and thanks for joining our community. In order to accept a pull request we require that all contributors sign our Contributor License Agreement. We have two different CLAs, depending on whether you are contributing to GX in a personal or professional capacity. Please sign the one that is applicable to your situation so that we may accept your contribution:

Individual Contributor License Agreement v1.0
Software Grant and Corporate Contributor License Agreement v1.0

Once you have signed the CLA, you can add a comment with the text @cla-bot check and the bot will update the PR status!

Please reach out to the #gx-community-support channel, on our Slack if you have any questions or if you have already signed the CLA and are receiving this message in error.

Users missing a CLA: creazyfrog

@Trivenzaa-Admin @claude
[BUGFIX] Fix integration tests using old metric-keyed exception_info …
1f2f68d 
…structure

After flattening exception_info from a metric-id-keyed dict to a plain
ExceptionInfo object, update the three integration tests that were
iterating .values() on the old nested structure to instead access
.get("raised_exception") / .get("exception_message") directly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gx-cla-bot

gx-cla-bot Bot commented May 17, 2026

Copy link
Copy Markdown

A new contributor, HUZZAH! Welcome and thanks for joining our community. In order to accept a pull request we require that all contributors sign our Contributor License Agreement. We have two different CLAs, depending on whether you are contributing to GX in a personal or professional capacity. Please sign the one that is applicable to your situation so that we may accept your contribution:

Individual Contributor License Agreement v1.0
Software Grant and Corporate Contributor License Agreement v1.0

Once you have signed the CLA, you can add a comment with the text @cla-bot check and the bot will update the PR status!

Please reach out to the #gx-community-support channel, on our Slack if you have any questions or if you have already signed the CLA and are receiving this message in error.

Users missing a CLA: creazyfrog, trivenzaa-admin

@Trivenzaa-Admin @claude
[BUGFIX] Fix tests using str(exception_info) broken by JSON-escaped q…
528c852 
…uotes

ExceptionInfo.__str__() uses json.dumps(), which escapes double quotes in
values (e.g. 'The column "b" ...' becomes 'The column \"b\" ...'). Tests
that checked for double-quoted substrings via str(result.exception_info)
therefore failed. Switch all such assertions to access
result.exception_info.get("exception_message", "") directly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gx-cla-bot

gx-cla-bot Bot commented May 17, 2026

Copy link
Copy Markdown

A new contributor, HUZZAH! Welcome and thanks for joining our community. In order to accept a pull request we require that all contributors sign our Contributor License Agreement. We have two different CLAs, depending on whether you are contributing to GX in a personal or professional capacity. Please sign the one that is applicable to your situation so that we may accept your contribution:

Individual Contributor License Agreement v1.0
Software Grant and Corporate Contributor License Agreement v1.0

Once you have signed the CLA, you can add a comment with the text @cla-bot check and the bot will update the PR status!

Please reach out to the #gx-community-support channel, on our Slack if you have any questions or if you have already signed the CLA and are receiving this message in error.

Users missing a CLA: creazyfrog, trivenzaa-admin

@gx-cla-bot

gx-cla-bot Bot commented May 17, 2026

Copy link
Copy Markdown

A new contributor, HUZZAH! Welcome and thanks for joining our community. In order to accept a pull request we require that all contributors sign our Contributor License Agreement. We have two different CLAs, depending on whether you are contributing to GX in a personal or professional capacity. Please sign the one that is applicable to your situation so that we may accept your contribution:

Individual Contributor License Agreement v1.0
Software Grant and Corporate Contributor License Agreement v1.0

Once you have signed the CLA, you can add a comment with the text @cla-bot check and the bot will update the PR status!

Please reach out to the #gx-community-support channel, on our Slack if you have any questions or if you have already signed the CLA and are receiving this message in error.

Users missing a CLA: creazyfrog, trivenzaa-admin

@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Is this PR still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity.

It will be closed if no further activity occurs. Thank you for your contributions 🙇

@github-actions github-actions Bot added the stale Stale issues and PRs label Jun 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

stale Stale issues and PRs

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Incorrect validation_result["results"]["exception_info"] structure when raised_exception == True

2 participants

@creazyfrog @Trivenzaa-Admin