Update dependency apache-airflow to v3 [SECURITY] #5550
+1
−1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openverse-bot
added
dependencies
openverse-bot
requested review from
krysal and
obulat
and removed request for
a team
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
==2.10.4->==3.1.6GitHub Vulnerability Alerts
CVE-2025-68675
In Apache Airflow versions before 3.1.6, the proxies and proxy fields within a Connection may include proxy URLs containing embedded authentication information. These fields were not treated as sensitive by default and therefore were not automatically masked in log output. As a result, when such connections are rendered or printed to logs, proxy credentials embedded in these fields could be exposed.
Users are recommended to upgrade to 3.1.6 or later, which fixes this issue
Release Notes
apache/airflow (apache-airflow)
v3.1.6Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
is_authorized_hitl_task()method now available in auth managers(#59399).""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
This method is now available in auth managers to check whether a user is authorized to approve a HITL task
proxyandproxiesadded toDEFAULT_SENSITIVE_FIELDS(#59688)""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
proxyandproxieshave been added toDEFAULT_SENSITIVE_FIELDSin secrets_masker to treat proxy configurations as sensitive by defaultBug Fixes
^^^^^^^^^
deprecated_optionsentry fordag_file_processor_timeout(#59181) (#60162)ApprovalOperatorwithSimpleAuthManagerwhenall_admins=True(#59399) (#60116)ti_failuremetrics for tasks (#59731) (#59964)TaskInstanceHistoryon schedulerTIresets (#59639) (#59752)proxyandproxiesas sensitive fields inDEFAULT_SENSITIVE_FIELDS(#59688) (#59792)[webserver] base_url(#59659) (#59781)DagRunContext(#59714) (#59732)Content-Typeto request headers in Task SDK calls when missing (#59676) (#59687)_read_from_logs_serverwhen status_code is 403 (#59489) (#59504)run_on_latest_versiondefaulting to False instead of True (#59304) (#59328).airflowignorenegation not working in subfolders (#58740) (#59305)DagRun.queued_atnot updating when clearing (#59066) (#59177)Miscellaneous
^^^^^^^^^^^^^
Doc Only Changes
^^^^^^^^^^^^^^^^
0.3.0(#59538)Taiwanese Mandarin (#59513) (#59515), Hebrew: (#59133) (#59255), Ca: (#59216) (#60199), TR: (#59169) (#60191)]
permalinkicon (#58763)get_template_context(#59023) (#59036)v3.1.5Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
^^^^^^^^^
gc.freeze(#58934)pre-AIP-39DAG runs (#58773)dag.test()(#58266)dayjscorrectly (#57880)endDateis not null (#58435)parseStreamingLogContentfor non-string data (#58399)Miscellaneous
^^^^^^^^^^^^^
.pycand.pyofiles after building Python (#58947)Doc Only Changes
^^^^^^^^^^^^^^^^
v3.1.4Compare Source
v3.1.3Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
Fix Connection & Variable access in API server contexts (plugins, log handlers)(#56583)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
Previously, hooks used in API server contexts (plugins, middlewares, log handlers) would fail with an
ImportErrorfor
SUPERVISOR_COMMS, becauseSUPERVISOR_COMMSonly exists in task runner child processes.This has been fixed by implementing automatic context detection with three separate secrets backend chains:
Context Detection:
SUPERVISOR_COMMSpresence_AIRFLOW_PROCESS_CONTEXT=serverenvironment variableBackend Chains:
EnvironmentVariablesBackend→ExecutionAPISecretsBackend(routes to Execution API via SUPERVISOR_COMMS)EnvironmentVariablesBackend→MetastoreBackend(direct database access)EnvironmentVariablesBackendonly (+ external backends from config like AWS Secrets Manager, Vault)The fallback chain is crucial for supervisor processes (worker-side, before task runner starts) which need to access
external secrets for remote logging setup but should not use
MetastoreBackend(to maintain worker isolation).Architecture Benefits:
MetastoreBackend, maintaining strict isolationImpact:
GCSHook,S3Hooknow work correctly in log handlers and pluginsSee:
#​56120 <https://github.com/apache/airflow/issues/56120>,#​56583 <https://github.com/apache/airflow/issues/56583>,#​51816 <https://github.com/apache/airflow/issues/51816>__Remove insecure dag reports API endpoint that executed user code in API server (#56609)
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
The
/api/v2/dagReportsendpoint has been removed because it loaded user DAG files directly in the API server process,violating Airflow's security architecture. This endpoint was not used in the UI and had no known consumers.
Use the
airflow dags reportCLI command instead for DAG loading reports.Bug Fixes
^^^^^^^^^
healthchecktimeout not respecting worker-timeout CLI option (#57731) (#57854)Miscellaneous
^^^^^^^^^^^^^
Doc Only Changes
^^^^^^^^^^^^^^^^
v3.1.2Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
^^^^^^^^^
DagProcessorManagerfor bundle initialization (#57459)triggering_user_namecontext variable (#56193)ObjectStoragePath(#57156)default_args(#57397)Miscellaneous
^^^^^^^^^^^^^
XComviewer and standardize task instance columns (#57447)retryhttptotenacitylibrary (#56762)Content-Typeheader to Task SDK API requests (#57386)task_display_namealias in event log API responses (#57609)Doc Only Changes
^^^^^^^^^^^^^^^^
instance_namein UI docs (#57523)v3.1.1Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
No significant changes.
Bug Fixes
^^^^^^^^^
dag_run.confduring upgrades from earlier versions (#56729)retry_delayisNone(#56236)generate_run_idnot called for manual triggers (#56699)KeyErrorwhen accessingretry_delayonMappedOperatorwithout explicit value (#56605)task-sdkconnection error handling to matchairflow-corebehavior (#56653)get_ti_countandget_task_statesaccess in callback requests (#56860)ConnectionorVariableaccess in Server context (#56602).airflowignoreorder precedence (#56832)--dag_run_confinairflow dags backfillCLI (#56599)'root'causes blue screen on hover (#56926)Day-of-MonthandDay-of-Weekconflicts (#56255)SerializedDagModelquery optimization (#56938)url_prefix(#55262)max_retry_delaytoMappedOperatormodel (#56951)@assetdecorator when fetching the asset (#56611)Miscellaneous
^^^^^^^^^^^^^
DISTINCTfordag_version_idlookup (#56565)PodGeneratorfor deserialization (#56733)action_on_existence(#56672)CreateAssetEventsBodyto Pydantic v2ConfigDict(#56772)active_runs_limitcheck (#56922)is_favoriteto UI dags list (#56341)executor,hostname, andqueuecolumns toTaskInstancespage (#55922)XComspage (#56285)ndjson(#56480)sla_miss_callback(#56127)natsortdependency toairflow-core(#56582)babeldependency in Task SDK (#56592)dagReportsAPI endpoint (#56621)Doc Only Changes
^^^^^^^^^^^^^^^^
triggering_asset_eventretrieval documentation in DAGs (#56957)v3.1.0Compare Source
Significant Changes
^^^^^^^^^^^^^^^^^^^
Human in the Loop (HITL)
""""""""""""""""""""""""
Airflow 3.1 introduces :doc:
Human-in-the-Loop (HITL) </tutorial/hitl>functionality that enablesworkflows to pause and wait for human decision-making. This powerful feature is particularly valuable for
AI/ML workflows, content moderation, and approval processes where human judgment is essential.
HITL tasks pause execution in a
deferredstate while waiting for human input via the Airflow UI. Userswith appropriate roles can see pending tasks, review context (including
XComdata andDAGparameters), andcomplete actions through intuitive web forms. The feature also supports API-driven interactions for custom
UIs and notification integration.
For detailed usage instructions, see :doc:
/tutorial/hitl.Note: HITL operators require
apache-airflow-providers-standardpackage and Airflow 3.1+.Task SDK Decoupling for Independent Upgrades
"""""""""""""""""""""""""""""""""""""""""""""
Airflow 3.1 advances the decoupling of the Task SDK from Airflow Core through
improved DAG serialization with versioned contracts. While complete code separation is planned for Airflow 3.2.0,
the serialization foundation enables independent upgrades when components are deployed separately.
For DAG Authors: Import constructs from
airflow.sdknamespace:from airflow.sdk import DAG, task, assetFor Platform Teams: Foundation for independent upgrades:
For technical details on the serialization contract, see :doc:
/administration-and-deployment/dag-serialization.Deadline Alerts
"""""""""""""""
Deadline Alerts provide proactive monitoring for DAG execution by automatically triggering notifications
when time thresholds are exceeded. This helps ensure SLA compliance and timely completion of critical workflows.
Configure deadline monitoring by specifying:
Example use cases:
Current Limitations: Deadline Alerts currently support only asynchronous callbacks (
AsyncCallback).Support for synchronous callbacks (
SyncCallback) is planned for a future release.For configuration details and examples, see :doc:
/howto/deadline-alerts... warning::
Deadline Alerts are experimental in 3.1 and may change in future versions based on user feedback.
UI Internationalization
"""""""""""""""""""""""
Airflow 3.1 delivers comprehensive internationalization (
i18n) support, making the web interfaceaccessible to users worldwide. The React-based UI now supports 17 languages with robust translation infrastructure.
Supported Languages:
The translation system includes automated completeness checking and clear contribution guidelines for community translators.
React Plugin System (AIP-68)
"""""""""""""""""""""""""""""
Airflow 3.1 introduces a modern plugin architecture enabling rich integrations through React components and
external views. This extensibility framework allows organizations to embed custom dashboards,
monitoring tools, and domain-specific interfaces directly within the Airflow UI.
New Plugin Capabilities:
Developer Experience:
airflow-react-plugindev toolsThis plugin system replaces legacy Flask-based approaches with modern web standards, improving performance,
maintainability, and user experience.
For more details and examples, see :doc:
/howto/custom-view-plugin.Enhanced UI Views and Filtering
""""""""""""""""""""""""""""""""
Airflow 3.1 brings significant UI improvements including rebuilt Calendar and Gantt chart views for the modern React UI,
comprehensive filtering capabilities, and a refreshed visual design system.
Visual Design Improvements
The UI now features an updated color palette leveraging Chakra UI semantic tokens, providing better consistency,
accessibility, and theme support across the interface. This modernization improves readability and creates
a more cohesive visual experience throughout Airflow.
Rebuilt Views and Enhanced Filtering
The Calendar and Gantt views from Airflow 2.x have been rebuilt for the modern React UI, along with enhanced
filtering capabilities across all views. These improvements provide better performance and a more consistent
user experience with the rest of the modern Airflow interface.
DAG Dashboard Organization
Users can now pin and favorite DAGs for better dashboard organization, making it easier to find and prioritize
frequently used workflows. This feature is particularly valuable for teams managing large numbers of DAGs,
providing quick access to critical workflows without searching through extensive DAG lists.
Inference Execution (Synchronous DAGs)
""""""""""""""""""""""""""""""""""""""
Airflow 3.1 introduces a new streaming API endpoint that allows applications to watch DAG runs until completion,
enabling more responsive integration patterns for real-time and inference workflows.
New Streaming Endpoint:
The
/dags/{dag_id}/dagRuns/{dag_run_id}/waitendpoint repeatedly emits JSON updates at specified intervals until the DAG run reaches a finished state... code-block:: bash
Watch a DAG run with 2-second polling interval, including XCom results
Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Renovate Bot.