FEAT: Realtime streaming session support and server-side barge-in attack

[adrian-gavrila]

Description

Adds persistent streaming session support to OpenAIRealtimeTarget and introduces BargeInAttack, a streaming attack that leverages server-side VAD to detect and exploit barge-in (interruption) behavior. Previously the target only supported single-turn fire-and-forget audio exchanges; this PR adds the transport primitives needed for multi-turn streaming sessions with incremental audio push, event subscription, and mid-session response requests.

When the server detects new user speech while the assistant is still responding, the in-flight response is automatically interrupted and the conversation history is truncated to match what was actually delivered.

Key additions:

• OpenAIRealtimeTarget streaming primitives — connect_async, push_audio_chunk_async, insert_user_audio_async, subscribe_events_async, request_response_async, send_streaming_session_config_async. These expose transport-level operations over a persistent WebSocket connection.
• _RealtimeEventDispatcher — ABC that owns a realtime connection's event stream, routes provider-specific events to the active turn, and fires an on_user_audio_committed callback when server VAD finalizes a turn. Provider-specific routing is isolated to _route_event / _cancel abstract methods.
• BargeInAttack — streaming attack that pushes audio chunks into a persistent session, applies configured converters on each server-committed turn (convert-on-commit), requests responses, and tracks interruptions. Per-turn Message pairs are persisted to CentralMemory with prompt_metadata["interrupted"] = True on interrupted turns.
• ServerVadConfig / RealtimeTargetResult — shared types for configuring server VAD and representing turn results (audio, transcripts, interruption flag).
• PromptNormalizer.convert_audio_async — applies audio converter configurations to raw PCM bytes for streaming attacks that hold audio mid-turn rather than a Message.

The target exposes only transport primitives; all attack logic (buffering, convert-on-commit dance, interruption signaling) lives in BargeInAttack.

Tests and Documentation

• 82 unit tests across 3 test files covering: event dispatch and routing, turn lifecycle, interruption detection, converter application, error paths, multi-turn connection reuse, and the full attack lifecycle.
• Coverage: 98% on realtime_audio.py, 72% on openai_realtime_target.py (uncovered lines are pre-existing code paths, not new additions).
• Notebook: doc/code/executor/attack/barge_in_attack.py (jupytext py:percent format) demonstrates the attack against a live OpenAI Realtime API endpoint with server VAD. Ran successfully against gpt-4o-realtime-preview — outputs cleared for CI (requires live credentials).

[hannahwestra25]

i think should be last_assistant_message and executed_turns be in the context instead so the state only contains variables related to the async functionality

[hannahwestra25]

is it valuable to have this be configurable?

[hannahwestra25]

this is specific to OAI realtime target so we should probably change it to make it configurable

[hannahwestra25]

I'm a little concerned with how this attack deviates from the other attacks in that it doesn't use the send_prompt_async workflow. The attack is doing a lot of plumbing that other attacks delegate to send_async — audio normalization (bypasses the pipeline), the swap + response trigger, message construction, memory persistence. That makes it inconsistent with the standard "build prompt → send_prompt_async → response" workflow and tightly couples it to RealtimeTarget internals.

Could we push the streaming session work into RealtimeTarget.send_async so the attack just does:

Connect + subscribe
Push chunks
On VAD commit → build a SeedPromptGroup from the snapshot and call send_prompt_async (target handles normalize + swap + response + persist)
Cleanup
Streaming-specific behavior still lives somewhere — but it belongs in the target that owns the WebSocket, not leaked into every streaming attack. This would also let AudioStreamNormalizer live inside the target as an implementation detail rather than something the attack reaches across the boundary to use.