Skip to content

refactor(settings): move standalone/APM opt-out logic out of ASMConfig - #19893

Open
avara1986 wants to merge 4 commits into
mainfrom
alberto.vara/APPSEC-69192-standalone-settings-refactor
Open

refactor(settings): move standalone/APM opt-out logic out of ASMConfig#19893
avara1986 wants to merge 4 commits into
mainfrom
alberto.vara/APPSEC-69192-standalone-settings-refactor

Conversation

@avara1986

@avara1986 avara1986 commented Aug 27, 2026

Copy link
Copy Markdown
Member

Description

Jira: APPSEC-69192

Standalone (a.k.a. APM opt-out) is not an ASM concept. It turns on whenever any of AppSec,
IAST, SCA or AI Guard is enabled while APM tracing is off — yet the decision lived on
ASMConfig as _apm_opt_out, which meant:

  • ASMConfig had to lazy-import aiguard_config (with an explanatory comment) purely to avoid an
    import-time dependency on a package it doesn't own;
  • all ~20 call sites across the writer, tracer, pin, span processor and two contrib integrations
    read as if they were toggling an AppSec setting.

This PR introduces ddtrace/internal/settings/standalone.py. StandaloneConfig now owns:

  • DD_APM_TRACING_ENABLED, moved off ASMConfig. Its constant moved from
    APPSEC.APM_TRACING_ENV to ddtrace.internal.constants.APM_TRACING_ENV (it had exactly one
    consumer, and nothing iterates the APPSEC constant class to enumerate env vars).
  • the aggregated apm_opt_out decision, previously ASMConfig._apm_opt_out.

_apm_opt_out was removed from ASMConfig outright rather than left as a delegating property:
delegating would have made asm.py import standalone.py, which imports asm.py. Full migration
also deletes the lazy aiguard import that was the original symptom. propagation/http.py,
_trace/pin.py, _trace/processor/__init__.py and contrib/internal/requests/connection.py no
longer import asm_config at all.

StandaloneConfig is a real DDConfig subclass so DD_APM_TRACING_ENABLED keeps resolving
through fleet/local stable config, not just os.environ.

DD_APM_TRACING_ENABLED is deliberately still readable/writable at runtime via
Tracer.configure(apm_tracing_disabled=...), and apm_opt_out reads every product flag live, so
in-process config mutation keeps working exactly as before.

LLMObs de-duplication

LLMObs read DD_APM_TRACING_ENABLED straight from os.environ in two places
(_processor.py, _llmobs.py). Both now use the shared config, so LLMObs and the tracer can no
longer disagree about whether APM tracing is on. See Risks below.

Configuration telemetry

StandaloneConfig reports itself via report_configuration(). This is not incidental: no
product plugin owns this config, and DD_APM_TRACING_ENABLED previously reached app-started
configuration telemetry by riding along with the appsec product plugin's ASMConfig
(products.py reports any product module's config attribute). Without the explicit call the
key would silently vanish from config telemetry and break the cross-language config-parity
registry. It is reported from standalone.py rather than the explicit list in
telemetry/__init__.py because the latter creates a cycle
(telemetrystandaloneasm_configtelemetry).

Verified:

ASMConfig reports DD_APM_TRACING_ENABLED (before): True
ASMConfig reports DD_APM_TRACING_ENABLED (after):  False
StandaloneConfig reportable items:                 ['DD_APM_TRACING_ENABLED']
reported value with DD_APM_TRACING_ENABLED=false:  False

Risks

Behaviour change (LLMObs). DD_APM_TRACING_ENABLED is now resolved once at ddtrace import
time, like every other DD_ setting. Setting it in the process environment before startup is
unaffected — the case that changes is mutating os.environ["DD_APM_TRACING_ENABLED"] after
import ddtrace but before LLMObs.enable(), which previously took effect and no longer does.
LLMObs was the outlier here (the tracer and ASMConfig already snapshotted at import), and the
old split meant LLMObs could disagree with tracer.enabled about whether APM tracing was on.
Flagging explicitly because it is the one non-mechanical part of this change.

No public API change. _apm_opt_out and _apm_tracing_enabled were both private.

Standalone (APM opt-out) is not an ASM concept: it turns on whenever AppSec,
IAST, SCA or AI Guard is enabled while APM tracing is off. Aggregating that
decision inside ASMConfig forced a lazy aiguard import and made all ~20 call
sites read as if they were toggling an AppSec setting.

Introduce ddtrace/internal/settings/standalone.py, which now owns:
  - DD_APM_TRACING_ENABLED, moved off ASMConfig; its constant moved from
    APPSEC.APM_TRACING_ENV to ddtrace.internal.constants.APM_TRACING_ENV
  - the aggregated apm_opt_out decision, was ASMConfig._apm_opt_out

LLMObs read DD_APM_TRACING_ENABLED straight from os.environ in two places;
both now use the shared config, so LLMObs and the tracer can no longer
disagree about whether APM tracing is on. Behaviour change: the value is
resolved at import time like every other DD_ setting, so mutating
os.environ after importing ddtrace no longer affects LLMObs.

StandaloneConfig reports itself to configuration telemetry: no product
plugin owns it, and DD_APM_TRACING_ENABLED previously reached telemetry by
riding along with the appsec product's ASMConfig.

In tests, override_global_config routes apm_tracing_enabled (renamed from
_apm_tracing_enabled) to standalone_config.

Jira: https://datadoghq.atlassian.net/browse/APPSEC-69192

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@avara1986 avara1986 added the changelog/no-changelog A changelog entry is not required for this PR. label Aug 27, 2026
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 27, 2026

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 3 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 27, 2026

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 240 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 240 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=135)
ddtrace.internal.opentelemetry.span -×-> ddtrace.trace  (product:opentelemetry -> product:tracing, score=133)
ddtrace.appsec._contrib.django -×-> ddtrace.trace  (product:appsec -> product:tracing, score=133)
ddtrace.internal.opentelemetry.trace -×-> ddtrace.trace  (product:opentelemetry -> product:tracing, score=133)
ddtrace.aiguard._api_client -×-> ddtrace.trace  (product:aiguard -> product:tracing, score=133)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

ddtrace/_trace/pin.py                                                   @DataDog/apm-sdk-capabilities-python
ddtrace/_trace/processor/__init__.py                                    @DataDog/apm-sdk-capabilities-python
ddtrace/_trace/tracer.py                                                @DataDog/apm-sdk-capabilities-python
ddtrace/appsec/_api_security/api_manager.py                             @DataDog/asm-python
ddtrace/appsec/_constants.py                                            @DataDog/asm-python
ddtrace/contrib/internal/requests/connection.py                         @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/contrib/internal/trace_utils.py                                 @DataDog/apm-core-python @DataDog/apm-idm-python
ddtrace/internal/constants.py                                           @DataDog/apm-core-python
ddtrace/internal/settings/asm.py                                        @DataDog/asm-python
ddtrace/internal/settings/standalone.py                                 @DataDog/apm-core-python
ddtrace/internal/writer/writer.py                                       @DataDog/apm-core-python
ddtrace/llmobs/_llmobs.py                                               @DataDog/ml-observability
ddtrace/llmobs/_processor.py                                            @DataDog/ml-observability
ddtrace/propagation/http.py                                             @DataDog/apm-sdk-capabilities-python
tests/aiguard/api/test_ai_guard_standalone.py                           @DataDog/asm-python
tests/appsec/appsec/api_security/test_api_security_manager.py           @DataDog/asm-python
tests/appsec/contrib_appsec/utils.py                                    @DataDog/asm-python
tests/contrib/mistralai/conftest.py                                     @DataDog/ml-observability
tests/contrib/requests/test_requests_distributed.py                     @DataDog/apm-core-python @DataDog/apm-idm-python
tests/contrib/urllib3/test_urllib3_appsec.py                            @DataDog/asm-python
tests/llmobs/test_llmobs.py                                             @DataDog/ml-observability
tests/tracer/test_standalone_config.py                                  @DataDog/apm-sdk-capabilities-python
tests/utils.py                                                          @DataDog/python-guild

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f368e38 | Docs | View more details | Give us feedback!

@pr-commenter

pr-commenter Bot commented Aug 27, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-08-28 10:55:02

Comparing candidate commit f368e38 in PR branch alberto.vara/APPSEC-69192-standalone-settings-refactor with baseline commit e6513ea in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 6 performance regressions! Performance is the same for 580 metrics, 10 unstable metrics, 3 known flaky benchmarks, 15 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:httppropagationinject-ids_only

  • 🟥 execution_time [+2.611µs; +2.764µs] or [+15.849%; +16.782%]

scenario:iastaspects-add_aspect

  • 🟥 execution_time [+10.361µs; +12.130µs] or [+10.233%; +11.981%]

scenario:iastaspectsospath-ospathbasename_aspect

  • 🟥 execution_time [+127.657µs; +135.344µs] or [+30.402%; +32.233%]

scenario:iastaspectssplit-rsplit_aspect

  • 🟥 execution_time [+36.171µs; +43.008µs] or [+25.174%; +29.932%]

scenario:telemetryaddmetric-1-count-metric-1-times

  • 🟥 execution_time [+597.326ns; +643.318ns] or [+21.836%; +23.517%]

scenario:tracer-small

  • 🟥 execution_time [+34.496µs; +36.619µs] or [+10.796%; +11.460%]

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:coreapiscenario-context_with_data_listeners

  • unstable execution_time [-773.830ns; +691.710ns] or [-7.033%; +6.287%]

scenario:coreapiscenario-core_dispatch_1_listener

  • unstable execution_time [-35.479ns; +31.482ns] or [-5.812%; +5.157%]

scenario:coreapiscenario-core_dispatch_50_listeners

  • unstable execution_time [-1544.769ns; +1778.907ns] or [-9.097%; +10.476%]

scenario:coreapiscenario-core_dispatch_exception_listeners

  • unstable execution_time [-1294.483ns; +1238.883ns] or [-9.812%; +9.390%]

scenario:coreapiscenario-core_dispatch_listeners

  • unstable execution_time [-328.192ns; +322.987ns] or [-8.921%; +8.779%]

scenario:coreapiscenario-core_dispatch_no_args_listeners

  • unstable execution_time [-255.814ns; +257.823ns] or [-8.717%; +8.786%]

scenario:coreapiscenario-core_dispatch_with_results_1_listener

  • unstable execution_time [-64.672ns; +80.206ns] or [-5.525%; +6.852%]

scenario:coreapiscenario-core_dispatch_with_results_50_listeners

  • unstable execution_time [-3706.532ns; +4267.129ns] or [-9.110%; +10.488%]

scenario:coreapiscenario-core_dispatch_with_results_listeners

  • unstable execution_time [-770.066ns; +777.423ns] or [-9.535%; +9.626%]

scenario:packagesupdateimporteddependencies-import_many_stdlib_cached

  • unstable execution_time [-59.968µs; +57.186µs] or [-9.587%; +9.142%]

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

scenario:iastaspects-lower_aspect

  • 🟥 execution_time [+70.894µs; +76.679µs] or [+29.700%; +32.124%]

scenario:iastaspects-title_noaspect

  • 🟥 execution_time [+24.351µs; +28.331µs] or [+12.369%; +14.390%]

scenario:span-start

  • 🟥 execution_time [+1.490ms; +1.740ms] or [+10.992%; +12.843%]

Known flaky benchmarks without significant changes:

  • scenario:errortrackingflasksqli-baseline
  • scenario:flasksimple-iast-get
  • scenario:iastaspects-casefold_aspect
  • scenario:iastaspects-casefold_noaspect
  • scenario:iastaspects-index_aspect
  • scenario:iastaspects-ljust_noaspect
  • scenario:iastaspects-replace_aspect
  • scenario:iastaspects-swapcase_aspect
  • scenario:iastaspects-translate_aspect
  • scenario:iastaspects-translate_noaspect
  • scenario:iastaspects-upper_noaspect
  • scenario:packagespackageforrootmodulemapping-cache_off
  • scenario:packagespackageforrootmodulemapping-cache_on
  • scenario:sethttpmeta-all-enabled
  • scenario:telemetryaddmetric-record-100-metrics

@avara1986

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: e921371c3b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

The llmobs/mistralai py3.10 + mistralai 2.0.5 job fails with two
mistralai.request spans where one is expected, only on this branch. Rule out
runtime config leakage as the cause: standalone_config.apm_tracing_enabled is
mutable at runtime (Tracer.configure(apm_tracing_disabled=...) and
override_global_config), unlike the os.environ read it replaced, so a value
left behind by an earlier test would now be visible to LLMObs span routing.

Reset it around every test in this suite so each one starts from the
environment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@avara1986
avara1986 force-pushed the alberto.vara/APPSEC-69192-standalone-settings-refactor branch from cc08c1e to 650c313 Compare August 28, 2026 09:27
@avara1986
avara1986 marked this pull request as ready for review August 28, 2026 10:27
@avara1986
avara1986 requested review from a team as code owners August 28, 2026 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/no-changelog A changelog entry is not required for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants