Skip to content

docs(profiling-315): py3.15 dev tooling and CPython upgrade runbook - #19273

Draft
vlad-scherbich wants to merge 8 commits into
vlad/ddtracepy-315-profiling-asyncio-monitoringfrom
vlad/315-profiling-dev-tooling
Draft

docs(profiling-315): py3.15 dev tooling and CPython upgrade runbook#19273
vlad-scherbich wants to merge 8 commits into
vlad/ddtracepy-315-profiling-asyncio-monitoringfrom
vlad/315-profiling-dev-tooling

Conversation

@vlad-scherbich

@vlad-scherbich vlad-scherbich commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

prev: #19272

Description

Delta vs #19272.

Adds docs/contributing-profiling-new-cpython.rst, docs/cpython-diffs/analysis_314_to_315.md, scripts/verify_profiler_compatibility.py, scripts/run-profiling-tests, compatibility baselines, AGENTS.md / type-annotate notes, and scripts/py315-stack/ bring-up files (including stale draft PR bodies).

No runtime profiler change.

Testing

Risks

Additional Notes

Docs and scripts only.

Contributes to #17817

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jul 24, 2026

Copy link
Copy Markdown

Pipelines  Tests

Unblock PR with BitsAI

⚠️ Warnings

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 16 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v113741238-d2b8243-manylinux2014_x86_64, 1] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | build linux serverless: [amd64, cp315-cp315, v126532274-233089d-musllinux_1_2_x86_64, 1] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-py | build linux serverless: [arm64, cp315-cp315, v113741357-d2b8243-manylinux2014_aarch64, 1] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

View all 16 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1c11f4f | Docs | View more details | Give us feedback!

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against vlad/ddtracepy-315-profiling-asyncio-monitoring using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

.claude/skills/type-annotate/SKILL.md                                   @DataDog/python-guild
AGENTS.md                                                               @DataDog/python-guild
docs/contributing-profiling-new-cpython.rst                             @DataDog/python-guild
docs/contributing-testing.rst                                           @DataDog/python-guild
docs/contributing.rst                                                   @DataDog/python-guild
docs/cpython-diffs/analysis_314_to_315.md                               @DataDog/python-guild
scripts/profiles/compatibility_baselines.json                           @DataDog/profiling-python
scripts/py315-stack/DRAFT_PRS.md                                        @DataDog/python-guild
scripts/py315-stack/MANUAL_EXISTING_PRS.md                              @DataDog/python-guild
scripts/py315-stack/PROFILING_STACK.md                                  @DataDog/python-guild
scripts/py315-stack/TRACKING.md                                         @DataDog/python-guild
scripts/py315-stack/generate-pr-urls.sh                                 @DataDog/python-guild
scripts/py315-stack/open-all-draft-prs.sh                               @DataDog/python-guild
scripts/py315-stack/pr-bodies/01-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/01-gab-315-monitoring-multiplexer.md      @DataDog/python-guild
scripts/py315-stack/pr-bodies/02-chore-315-wrapping-context.md          @DataDog/python-guild
scripts/py315-stack/pr-bodies/02-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/03-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/03-vlad-315-ci-matrix.md                  @DataDog/python-guild
scripts/py315-stack/pr-bodies/04-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/04-vlad-315-ci-autoregen-lockfiles.md     @DataDog/python-guild
scripts/py315-stack/pr-bodies/05-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/05-vlad-315-official-support.md           @DataDog/python-guild
scripts/py315-stack/pr-bodies/05-vlad-315-peripheral-compat.md          @DataDog/python-guild
scripts/py315-stack/pr-bodies/06-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/06-vlad-315-peripheral-compat.md          @DataDog/python-guild
scripts/py315-stack/pr-bodies/06-vlad-profiling-native-test-install-subdir.md  @DataDog/python-guild
scripts/py315-stack/pr-bodies/07-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/07-vlad-ddtracepy-315-profiling-native.md  @DataDog/python-guild
scripts/py315-stack/pr-bodies/07-vlad-profiling-native-test-install-subdir.md  @DataDog/python-guild
scripts/py315-stack/pr-bodies/08-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/08-vlad-ddtracepy-315-profiling-collectors.md  @DataDog/python-guild
scripts/py315-stack/pr-bodies/08-vlad-ddtracepy-315-profiling-native.md  @DataDog/python-guild
scripts/py315-stack/pr-bodies/09-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/09-vlad-ddtracepy-315-profiling-collectors.md  @DataDog/python-guild
scripts/py315-stack/pr-bodies/09-vlad-ddtracepy-315-profiling-only.md   @DataDog/python-guild
scripts/py315-stack/pr-bodies/10-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/10-vlad-ddtracepy-315-profiling-asyncio-monitoring.md  @DataDog/python-guild
scripts/py315-stack/pr-bodies/10-vlad-ddtracepy-315-profiling-only.md   @DataDog/python-guild
scripts/py315-stack/pr-bodies/11-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/11-vlad-315-profiling-dev-tooling.md      @DataDog/python-guild
scripts/py315-stack/pr-bodies/11-vlad-ddtracepy-315-profiling-asyncio-monitoring.md  @DataDog/python-guild
scripts/py315-stack/pr-bodies/12-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/12-vlad-315-lib-injection-ssi.md          @DataDog/python-guild
scripts/py315-stack/pr-bodies/12-vlad-315-profiling-dev-tooling.md      @DataDog/python-guild
scripts/py315-stack/pr-bodies/13-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/13-vlad-315-lib-injection-ssi.md          @DataDog/python-guild
scripts/py315-stack/pr-bodies/13-vlad-315-profiling-release-note.md     @DataDog/python-guild
scripts/py315-stack/pr-bodies/14-compare.url                            @DataDog/python-guild
scripts/py315-stack/pr-bodies/14-vlad-315-official-support.md           @DataDog/python-guild
scripts/py315-stack/pr-bodies/14-vlad-315-profiling-release-note.md     @DataDog/python-guild
scripts/py315-stack/pr-bodies/VERSION-AUDIT-DRAFTS.md                   @DataDog/python-guild
scripts/py315-stack/pr-numbers.env                                      @DataDog/python-guild
scripts/py315-stack/rebuild-stack.sh                                    @DataDog/python-guild
scripts/py315-stack/sync-pr-numbers.sh                                  @DataDog/python-guild
scripts/run-profiling-tests                                             @DataDog/python-guild
scripts/verify_profiler_compatibility.py                                @DataDog/python-guild

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 24, 2026

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 3 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-asyncio-monitoring branch from 4b85b1d to 636fd80 Compare August 21, 2026 04:53
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-dev-tooling branch from e703ab0 to e95f62f Compare August 21, 2026 04:53
@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-asyncio-monitoring branch from 636fd80 to ba0a39e Compare August 21, 2026 17:43
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-dev-tooling branch 2 times, most recently from b906b1e to ba0e2bf Compare August 21, 2026 18:43
@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-asyncio-monitoring branch from ba0a39e to 082c37a Compare August 21, 2026 18:43
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 21, 2026

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 240 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 240 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=135)
ddtrace.llmobs._integrations.openai -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)
ddtrace.internal.test_visibility.api -×-> ddtrace.trace  (product:ci_visibility -> product:tracing, score=133)
ddtrace.debugging._signal.tracing -×-> ddtrace.trace  (product:debugging -> product:tracing, score=133)
ddtrace.internal.ci_visibility.filters -×-> ddtrace.trace  (product:ci_visibility -> product:tracing, score=133)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-asyncio-monitoring branch from 082c37a to 6d9945a Compare August 28, 2026 00:45
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-dev-tooling branch from 50bf856 to f29e0ea Compare August 28, 2026 00:45
vlad-scherbich added a commit that referenced this pull request Aug 28, 2026
Drop 6-PR / 14-PR numbering. Live stack is #19267#19269#19270#19272#19273. Leave #17849 and official-support / riot off this chain.
@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-asyncio-monitoring branch from 6d9945a to 5059048 Compare August 28, 2026 15:51
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-dev-tooling branch from f29e0ea to 8b78ec5 Compare August 28, 2026 15:51
vlad-scherbich added a commit that referenced this pull request Aug 28, 2026
Drop 6-PR / 14-PR numbering. Live stack is #19267#19269#19270#19272#19273. Leave #17849 and official-support / riot off this chain.
@vlad-scherbich vlad-scherbich changed the title docs(profiling): py3.15 dev tooling and CPython upgrade runbook docs(profiling-315): py3.15 dev tooling and CPython upgrade runbook Aug 28, 2026
@vlad-scherbich
vlad-scherbich requested a lite review from Copilot August 28, 2026 16:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Python 3.15 profiling bring-up tooling and documentation to support the ongoing py3.15 profiling stack work: a standalone profiler compatibility verifier, a one-command profiling test runner, CPython 3.14→3.15 ABI change notes, and a maintainer runbook for bringing profiling up on new CPython minors.

Changes:

  • Introduces scripts/verify_profiler_compatibility.py (asyncio guard validation + real pprof sample collection + baseline compare) and scripts/run-profiling-tests (runner wiring compatibility checks + Riot profiling suites).
  • Adds py3.15 stack navigation/helpers under scripts/py315-stack/ (tracking, PR body generation, PR-number syncing, historical rebuild notes).
  • Adds/updates documentation for CPython 3.15 profiling bring-up (docs/contributing-profiling-new-cpython.rst, docs/cpython-diffs/analysis_314_to_315.md) and links from contributor docs.

Reviewed changes

Copilot reviewed 57 out of 57 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
scripts/verify_profiler_compatibility.py New Python script to validate asyncio profiling guards + collect/validate profiler samples and compare to stored baselines.
scripts/run-profiling-tests New Bash runner to execute the compatibility script plus Riot profiling suites on a target Python.
scripts/py315-stack/TRACKING.md Post-split tracking notes for remaining py3.15 parity follow-ups.
scripts/py315-stack/sync-pr-numbers.sh Helper to sync PR numbers for stack navigation using gh.
scripts/py315-stack/rebuild-stack.sh Historical script for rebuilding the older 14-PR stack chain.
scripts/py315-stack/PROFILING_STACK.md Defines the live 5-PR profiling stack merge order and validation legs.
scripts/py315-stack/pr-numbers.env Stores PR numbers for stack navigation tooling.
scripts/py315-stack/pr-bodies/VERSION-AUDIT-DRAFTS.md Draft PR-body correction notes for version-claim audit.
scripts/py315-stack/pr-bodies/14-vlad-315-profiling-release-note.md Generated PR body for historical stack layer 14 (release note).
scripts/py315-stack/pr-bodies/14-vlad-315-official-support.md Generated PR body for historical stack layer 14 (official support).
scripts/py315-stack/pr-bodies/14-compare.url Generated GitHub compare URL for historical stack layer 14.
scripts/py315-stack/pr-bodies/13-vlad-315-profiling-release-note.md Generated PR body for historical stack layer 13 (release note).
scripts/py315-stack/pr-bodies/13-vlad-315-lib-injection-ssi.md Generated PR body for historical stack layer 13 (SSI).
scripts/py315-stack/pr-bodies/13-compare.url Generated GitHub compare URL for historical stack layer 13.
scripts/py315-stack/pr-bodies/12-vlad-315-profiling-dev-tooling.md Generated PR body for historical stack layer 12 (dev tooling).
scripts/py315-stack/pr-bodies/12-vlad-315-lib-injection-ssi.md Generated PR body for historical stack layer 12 (SSI).
scripts/py315-stack/pr-bodies/12-compare.url Generated GitHub compare URL for historical stack layer 12.
scripts/py315-stack/pr-bodies/11-vlad-ddtracepy-315-profiling-asyncio-monitoring.md Generated PR body for historical stack layer 11 (asyncio monitoring).
scripts/py315-stack/pr-bodies/11-vlad-315-profiling-dev-tooling.md Generated PR body for historical stack layer 11 (dev tooling).
scripts/py315-stack/pr-bodies/11-compare.url Generated GitHub compare URL for historical stack layer 11.
scripts/py315-stack/pr-bodies/10-vlad-ddtracepy-315-profiling-only.md Generated PR body for historical stack layer 10 (profiling-only).
scripts/py315-stack/pr-bodies/10-vlad-ddtracepy-315-profiling-asyncio-monitoring.md Generated PR body for historical stack layer 10 (asyncio monitoring).
scripts/py315-stack/pr-bodies/10-compare.url Generated GitHub compare URL for historical stack layer 10.
scripts/py315-stack/pr-bodies/09-vlad-ddtracepy-315-profiling-only.md Generated PR body for historical stack layer 9 (profiling-only).
scripts/py315-stack/pr-bodies/09-vlad-ddtracepy-315-profiling-collectors.md Generated PR body for historical stack layer 9 (collectors).
scripts/py315-stack/pr-bodies/09-compare.url Generated GitHub compare URL for historical stack layer 9.
scripts/py315-stack/pr-bodies/08-vlad-ddtracepy-315-profiling-native.md Generated PR body for historical stack layer 8 (native ABI).
scripts/py315-stack/pr-bodies/08-vlad-ddtracepy-315-profiling-collectors.md Generated PR body for historical stack layer 8 (collectors).
scripts/py315-stack/pr-bodies/08-compare.url Generated GitHub compare URL for historical stack layer 8.
scripts/py315-stack/pr-bodies/07-vlad-profiling-native-test-install-subdir.md Generated PR body for historical stack layer 7 (INSTALL_SUBDIR).
scripts/py315-stack/pr-bodies/07-vlad-ddtracepy-315-profiling-native.md Generated PR body for historical stack layer 7 (native ABI).
scripts/py315-stack/pr-bodies/07-compare.url Generated GitHub compare URL for historical stack layer 7.
scripts/py315-stack/pr-bodies/06-vlad-profiling-native-test-install-subdir.md Generated PR body for historical stack layer 6 (INSTALL_SUBDIR).
scripts/py315-stack/pr-bodies/06-vlad-315-peripheral-compat.md Generated PR body for historical stack layer 6 (peripheral compat).
scripts/py315-stack/pr-bodies/06-compare.url Generated GitHub compare URL for historical stack layer 6.
scripts/py315-stack/pr-bodies/05-vlad-315-peripheral-compat.md Generated PR body for historical stack layer 5 (peripheral compat).
scripts/py315-stack/pr-bodies/05-vlad-315-official-support.md Generated PR body for historical stack layer 5 (official support).
scripts/py315-stack/pr-bodies/05-compare.url Generated GitHub compare URL for historical stack layer 5.
scripts/py315-stack/pr-bodies/04-vlad-315-ci-autoregen-lockfiles.md Generated PR body for historical stack layer 4 (lockfile regen).
scripts/py315-stack/pr-bodies/04-compare.url Generated GitHub compare URL for historical stack layer 4.
scripts/py315-stack/pr-bodies/03-vlad-315-ci-matrix.md Generated PR body for historical stack layer 3 (CI matrix).
scripts/py315-stack/pr-bodies/03-compare.url Generated GitHub compare URL for historical stack layer 3.
scripts/py315-stack/pr-bodies/02-compare.url Generated GitHub compare URL for historical stack layer 2.
scripts/py315-stack/pr-bodies/02-chore-315-wrapping-context.md Generated PR body for historical stack layer 2 (wrapping context).
scripts/py315-stack/pr-bodies/01-gab-315-monitoring-multiplexer.md Generated PR body for historical stack layer 1 (monitoring multiplexer).
scripts/py315-stack/pr-bodies/01-compare.url Generated GitHub compare URL for historical stack layer 1.
scripts/py315-stack/open-all-draft-prs.sh Helper to open compare URLs for drafting PRs in historical stack order.
scripts/py315-stack/MANUAL_EXISTING_PRS.md Manual navigation notes for the live 5-PR stack and related PRs.
scripts/py315-stack/generate-pr-urls.sh Generator for compare URLs and PR bodies (historical 14-PR chain).
scripts/py315-stack/DRAFT_PRS.md Generated index of historical 14-PR compare URLs and bodies (archival).
scripts/profiles/compatibility_baselines.json Adds saved compatibility baselines for 3.9–3.15 for the verifier script.
docs/cpython-diffs/analysis_314_to_315.md New CPython 3.14→3.15 header/change analysis notes for Echion/profiling.
docs/contributing.rst Adds link to the new profiling-on-new-CPython guide and updates toctree.
docs/contributing-testing.rst Documents using scripts/run-profiling-tests for profiling validation on new Python versions.
docs/contributing-profiling-new-cpython.rst New maintainer runbook for profiling bring-up on new CPython minors (stack/Echion, gating, CI, validation).
AGENTS.md Documents the type-annotate skill in the project’s AI assistant guidance.
.claude/skills/type-annotate/SKILL.md Adds the new type-annotate skill definition and workflow guidance.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +75 to +81
# Import _asyncio BEFORE asyncio to ensure the ModuleWatchdog callback is
# registered first. The callback fires when asyncio is subsequently imported.
# Importing asyncio triggers the ModuleWatchdog callback, which runs
# _call_init_asyncio() — exercising all the hasattr guards.
import asyncio

import ddtrace.profiling._asyncio as _asyncio_mod
failures.append(f"{suite}: was PASS in baseline, now FAIL — {cur.get('error', '?')}")

# Check sample count regression
if "wall_time_samples" in ref and "wall_time_samples" in cur:
Comment on lines +101 to +102
- Tagged int shift changed: `(i << 2)` instead of `(i << 2)` — same for non-debug,
but `Py_TAGGED_SHIFT = 2` is now the canonical name.
Comment thread docs/contributing.rst
``validate_supported_configurations_v2_local_file`` GitLab CI job to fail, displaying the missing keys
in its output. A Datadog maintainer must add the key to the registry before the PR can merge.

.. toctree::
Profiling bring-up scripts, compatibility baselines, Echion migration runbook,
and py315-stack PR navigation helpers (PR 12/14).
Update pr-numbers, generate-pr-urls, MANUAL_EXISTING_PRS, and rebuild-stack
to reflect linear stack with peripheral/profiling before packaging (#19254).
Replacement PRs #19267#19275 supersede wrongly-merged #19255#19259.
Point the 3.15 runbook row at this stack instead of closed #17294/hasattr
guards, and draft paste-ready PR bodies that do not treat sys.monitoring
as a 3.15-only API.
Drop 6-PR / 14-PR numbering. Live stack is #19267#19269#19270#19272#19273. Leave #17849 and official-support / riot off this chain.
@vlad-scherbich
vlad-scherbich force-pushed the vlad/ddtracepy-315-profiling-asyncio-monitoring branch from 5059048 to 20c0559 Compare August 28, 2026 18:38
@vlad-scherbich
vlad-scherbich force-pushed the vlad/315-profiling-dev-tooling branch from 8b78ec5 to 1c11f4f Compare August 28, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants