issue in OpenLineage callback when table is None, moving imports up, type annotations. #60096
Description
Apache Airflow Provider(s)
Versions of Apache Airflow Providers
No response
Apache Airflow version
3.2.0
Operating System
Windows (local development)
Deployment
Official Apache Airflow Helm Chart
Deployment details
No response
What happened
-
While reviewing BigQueryUpdateTableSchemaOperator, I noticed that get_openlineage_facets_on_complete assumes _table is always populated. If the hook returns None, this can raise an exception during the OpenLineage callback.
-
Moving the Imports mentioned in the method to upwards
-
Mentioning type annotate return.
def execute(self, context: Context): to def execute(self, context: Context) -> dict | None:
What you think should happen instead
No response
How to reproduce
- Use Apache Airflow with the Google provider installed.
- Enable OpenLineage.
- Create a DAG that uses BigQueryUpdateTableSchemaOperator.
- Pass an empty list or no-op schema update to
schema_fields_updates
(for example, an empty list[]or updates that do not modify the table). - Trigger the DAG.
Example:
BigQueryUpdateTableSchemaOperator(
task_id="update_schema",
dataset_id="test_dataset",
table_id="test_table",
schema_fields_updates=[],
)
6. The operator execution completes, but update_table_schema may return None.
7. After task execution, get_openlineage_facets_on_complete is invoked and
assumes the table resource is always present.
8. This can result in an exception when calling Table.from_api_repr(self._table).
Anything else
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct