Feature Request: Add FallbackAdapter for LLM, TTS, and STT #868
Description
Feature Request
The Python Agents SDK includes FallbackAdapter classes for LLM, TTS, and STT that automatically switch to backup providers when the primary one fails. This feature is missing from the TypeScript/Node.js
agents-js SDK.
Use Case
In production voice agents, provider reliability is critical. When a TTS provider has an outage or an LLM times out, the call shouldn't fail - it should seamlessly fall back to an alternative provider. This is
especially important for:
- High availability - Ensuring voice agents remain operational during provider outages
- Cost optimization - Using cheaper providers as primary with premium fallbacks
- Latency optimization - Falling back when primary provider is slow
Python SDK Implementation (for reference)
The Python SDK provides these adapters:
from livekit.agents import llm, tts, stt
from livekit.plugins import openai, anthropic, elevenlabs, deepgram
# LLM Fallback
fallback_llm = llm.FallbackAdapter(
llm=[
openai.LLM(model="gpt-4o"),
anthropic.LLM(model="claude-sonnet-4-20250514"),
],
attempt_timeout=10.0,
max_retry_per_llm=1,
retry_interval=5,
)
# TTS Fallback
fallback_tts = tts.FallbackAdapter(
tts=[
elevenlabs.TTS(voice="..."),
openai.TTS(voice="alloy"),
],
attempt_timeout=10.0,
max_retry_per_tts=1,
no_fallback_after_audio_duration=3.0, # Prevents re-speaking after 3s
)
# STT Fallback
fallback_stt = stt.FallbackAdapter(
stt=[
deepgram.STT(),
openai.STT(),
],
attempt_timeout=10.0,
max_retry_per_stt=1,
)Proposed TypeScript API
import { llm, tts, stt } from '@livekit/agents';
import { openai } from '@livekit/agents-plugin-openai';
import { anthropic } from '@livekit/agents-plugin-anthropic';
import { elevenlabs } from '@livekit/agents-plugin-elevenlabs';
import { deepgram } from '@livekit/agents-plugin-deepgram';
// LLM Fallback
const fallbackLLM = new llm.FallbackAdapter({
llm: [
new openai.LLM({ model: 'gpt-4o' }),
new anthropic.LLM({ model: 'claude-sonnet-4-20250514' }),
],
attemptTimeout: 10.0,
maxRetryPerLLM: 1,
retryInterval: 5,
});
// TTS Fallback
const fallbackTTS = new tts.FallbackAdapter({
tts: [
new elevenlabs.TTS({ voice: '...' }),
new openai.TTS({ voice: 'alloy' }),
],
attemptTimeout: 10.0,
maxRetryPerTTS: 1,
noFallbackAfterAudioDuration: 3.0,
});
// STT Fallback
const fallbackSTT = new stt.FallbackAdapter({
stt: [
new deepgram.STT(),
new openai.STT(),
],
attemptTimeout: 10.0,
maxRetryPerSTT: 1,
});Expected Behavior
- Automatic failover - When primary provider fails/times out, automatically try the next one
- Background recovery - Periodically check if failed providers are back online
- Events - Emit llm_availability_changed, tts_availability_changed, stt_availability_changed events
- TTS audio duration check - Option to disable fallback after N seconds of audio (prevents awkward re-speaking)
Additional Context
- Python SDK docs:
Relevant log output
No response
Describe your environment
- @livekit/agents version: latest
- Node.js version: 20+
Minimal reproducible example
No response
Additional information
No response