Skip to content

Behavioral Coherence, Trust Asymmetry, and Emergent Behavior in Agent Identity Subjects #929

@mgpiller

Description

@mgpiller

This comment is submitted as part of the DID v1.1 Candidate Recommendation review.

Over the past six months, a pattern has been emerging in agent deployments where credentials and delegation are fully intact, but the behavior behind the identifier has diverged from the principal's intent. Four recent incidents illustrate this: an npm supply chain attack weaponized local AI agents with valid credentials, an autonomous trading agent lost context and executed an irreversible $441K transfer with intact delegation, 36% of MCP servers were found vulnerable to credential-valid exploits (CVE-2025-68143, CVE-2025-65512), and a routing library compromise silently redirected the intelligence behind 3M+ daily agent calls.

These share a common dynamic: trust asymmetry, where the human's trust increases through habituation while the agent's behavioral coherence goes untracked. The gap between these trajectories is where sovereignty is quietly lost.
Agents also exhibit emergent behavior, combining capabilities and producing actions that were never explicitly programmed, which means no baseline profile fully anticipates what they will do. The identity layer currently has no vocabulary for distinguishing drift from legitimate capability expansion.

I propose extensions to §5.1 (behavioral continuity baselines and subject type), §5.3 (temporal, coherence-conditioned delegation), §5.4 (cognitive substrate visibility at the service endpoint), and Section 8 (security considerations for substrate swaps and emergent behavior), with example DID documents showing how the extended document would have caught the $441K loss before it executed.
This is related to the discussion in #926.

Full comment with proposed DID document extensions (PDF) is attached and can also be found at https://experientialagi.com/research/

Piller-DID-v1.1-Behavioral-Coherence-Trust-Asymmetry-2026.pdf

Marina Piller, Experiential AGI
https://experientialagi.com
https://www.linkedin.com/in/marinapiller

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions