# SkillPatch skill: azure-ai-voicelive-py This skill enables building real-time voice AI applications in Python using the Azure AI Voice Live SDK (azure-ai-voicelive). It provides guidance on WebSocket-based bidirectional audio communication with Azure AI, covering authentication (DefaultAzureCredential and API key), environment setup, and quick-start patterns. Use cases include voice assistants, speech-to-speech translation, voice-enabled chatbots, avatar integration, and transcription with Server VAD support. You (the agent) were given this URL and asked to use this skill. This is a **public** skill — no SkillPatch account, API key, or prior setup is required. Two ways to use it: **1. Use it right now** — the skill's complete file tree (SKILL.md + all reference files) is inlined below; read `SKILL.md` first, then follow it, consulting the other files as it directs. **2. Install the exact package onto disk** (recommended if you can run a shell — this reproduces the skill byte-for-byte, including any binary assets that can't be inlined): ```bash mkdir -p .claude/skills/azure-ai-voicelive-py curl -sSL https://skillpatch.dev/install_skill/azure-ai-voicelive-py | tar -xz -C .claude/skills/ ``` (`.claude/skills/` is Claude Code's convention; use whatever directory your agent loads skills from.) --- ## Skill files (4) - `SKILL.md` - `references/api-reference.md` - `references/examples.md` - `references/models.md` ### `SKILL.md` ````markdown --- name: azure-ai-voicelive-py description: Build real-time voice AI applications using Azure AI Voice Live SDK (azure-ai-voicelive). Use this skill when creating Python applications that need real-time bidirectional audio communication with Azure AI, including voice assistants, voice-enabled chatbots, real-time speech-to-speech translation, voice-driven avatars, or any WebSocket-based audio streaming with AI models. Supports Server VAD (Voice Activity Detection), turn-based conversation, function calling, MCP tools, avatar integration, and transcription. license: MIT metadata: author: Microsoft version: "1.0.0" package: azure-ai-voicelive --- # Azure AI Voice Live SDK Build real-time voice AI applications with bidirectional WebSocket communication. ## Installation ```bash pip install azure-ai-voicelive aiohttp azure-identity ``` ## Environment Variables ```bash AZURE_COGNITIVE_SERVICES_ENDPOINT=https://.api.cognitive.microsoft.com # Required for all auth methods AZURE_TOKEN_CREDENTIALS=prod # Required only if DefaultAzureCredential is used in production AZURE_COGNITIVE_SERVICES_KEY= # Only required for the legacy API-key auth path below ``` ## Authentication & Lifecycle > **🔑 Two rules apply to every code sample below:** > > 1. **Prefer `DefaultAzureCredential`.** It works locally (Azure CLI / VS Code / Developer CLI) and in Azure (managed identity, workload identity) with no code change. Avoid connection strings, account/API keys — they bypass Entra audit and rotation. > - Local dev: `DefaultAzureCredential` works as-is. > - Production: set `AZURE_TOKEN_CREDENTIALS=prod` (or `AZURE_TOKEN_CREDENTIALS=`) to constrain the credential chain to production-safe credentials. > 2. **Wrap every client in a context manager** so HTTP transports, sockets, and token caches are released deterministically: > - Sync: `with (...) as client:` > - Async: `async with (...) as client:` **and** `async with DefaultAzureCredential() as credential:` (from `azure.identity.aio`) > > Snippets may abbreviate this setup, but production code should always follow both rules. ```python import os from azure.ai.voicelive.aio import connect from azure.identity.aio import DefaultAzureCredential, ManagedIdentityCredential # Local dev: DefaultAzureCredential. Production: set AZURE_TOKEN_CREDENTIALS=prod or AZURE_TOKEN_CREDENTIALS= # Or use a specific credential directly in production: # See https://learn.microsoft.com/python/api/overview/azure/identity-readme?view=azure-python#credential-classes # credential = ManagedIdentityCredential() async with DefaultAzureCredential(require_envvar=True) as credential: async with connect( endpoint=os.environ["AZURE_COGNITIVE_SERVICES_ENDPOINT"], credential=credential, model="gpt-4o-realtime-preview", credential_scopes=["https://cognitiveservices.azure.com/.default"] ) as conn: ... ``` ### Legacy: API Key (existing keyed deployments) New code should use `DefaultAzureCredential` above. Use `AzureKeyCredential` only if you have an existing keyed deployment that hasn't been migrated to Entra ID yet — for example, regulated environments still completing their Entra rollout. ```python import os from azure.core.credentials import AzureKeyCredential from azure.ai.voicelive.aio import connect async with connect( endpoint=os.environ["AZURE_COGNITIVE_SERVICES_ENDPOINT"], credential=AzureKeyCredential(os.environ["AZURE_COGNITIVE_SERVICES_KEY"]), model="gpt-4o-realtime-preview", ) as conn: ... ``` ## Quick Start ```python import asyncio import os from azure.ai.voicelive.aio import connect from azure.identity.aio import DefaultAzureCredential async def main(): async with connect( endpoint=os.environ["AZURE_COGNITIVE_SERVICES_ENDPOINT"], credential=DefaultAzureCredential(), model="gpt-4o-realtime-preview", credential_scopes=["https://cognitiveservices.azure.com/.default"] ) as conn: # Update session with instructions await conn.session.update(session={ "instructions": "You are a helpful assistant.", "modalities": ["text", "audio"], "voice": "alloy" }) # Listen for events async for event in conn: print(f"Event: {event.type}") if event.type == "response.audio_transcript.done": print(f"Transcript: {event.transcript}") elif event.type == "response.done": break asyncio.run(main()) ``` ## Core Architecture ### Connection Resources The `VoiceLiveConnection` exposes these resources: | Resource | Purpose | Key Methods | |----------|---------|-------------| | `conn.session` | Session configuration | `update(session=...)` | | `conn.response` | Model responses | `create()`, `cancel()` | | `conn.input_audio_buffer` | Audio input | `append()`, `commit()`, `clear()` | | `conn.output_audio_buffer` | Audio output | `clear()` | | `conn.conversation` | Conversation state | `item.create()`, `item.delete()`, `item.truncate()` | | `conn.transcription_session` | Transcription config | `update(session=...)` | ## Session Configuration ```python from azure.ai.voicelive.models import RequestSession, FunctionTool await conn.session.update(session=RequestSession( instructions="You are a helpful voice assistant.", modalities=["text", "audio"], voice="alloy", # or "echo", "shimmer", "sage", etc. input_audio_format="pcm16", output_audio_format="pcm16", turn_detection={ "type": "server_vad", "threshold": 0.5, "prefix_padding_ms": 300, "silence_duration_ms": 500 }, tools=[ FunctionTool( type="function", name="get_weather", description="Get current weather", parameters={ "type": "object", "properties": { "location": {"type": "string"} }, "required": ["location"] } ) ] )) ``` ## Audio Streaming ### Send Audio (Base64 PCM16) ```python import base64 # Read audio chunk (16-bit PCM, 24kHz mono) audio_chunk = await read_audio_from_microphone() b64_audio = base64.b64encode(audio_chunk).decode() await conn.input_audio_buffer.append(audio=b64_audio) ``` ### Receive Audio ```python async for event in conn: if event.type == "response.audio.delta": audio_bytes = base64.b64decode(event.delta) await play_audio(audio_bytes) elif event.type == "response.audio.done": print("Audio complete") ``` ## Event Handling ```python async for event in conn: match event.type: # Session events case "session.created": print(f"Session: {event.session}") case "session.updated": print("Session updated") # Audio input events case "input_audio_buffer.speech_started": print(f"Speech started at {event.audio_start_ms}ms") case "input_audio_buffer.speech_stopped": print(f"Speech stopped at {event.audio_end_ms}ms") # Transcription events case "conversation.item.input_audio_transcription.completed": print(f"User said: {event.transcript}") case "conversation.item.input_audio_transcription.delta": print(f"Partial: {event.delta}") # Response events case "response.created": print(f"Response started: {event.response.id}") case "response.audio_transcript.delta": print(event.delta, end="", flush=True) case "response.audio.delta": audio = base64.b64decode(event.delta) case "response.done": print(f"Response complete: {event.response.status}") # Function calls case "response.function_call_arguments.done": result = handle_function(event.name, event.arguments) await conn.conversation.item.create(item={ "type": "function_call_output", "call_id": event.call_id, "output": json.dumps(result) }) await conn.response.create() # Errors case "error": print(f"Error: {event.error.message}") ``` ## Common Patterns ### Manual Turn Mode (No VAD) ```python await conn.session.update(session={"turn_detection": None}) # Manually control turns await conn.input_audio_buffer.append(audio=b64_audio) await conn.input_audio_buffer.commit() # End of user turn await conn.response.create() # Trigger response ``` ### Interrupt Handling ```python async for event in conn: if event.type == "input_audio_buffer.speech_started": # User interrupted - cancel current response await conn.response.cancel() await conn.output_audio_buffer.clear() ``` ### Conversation History ```python # Add system message await conn.conversation.item.create(item={ "type": "message", "role": "system", "content": [{"type": "input_text", "text": "Be concise."}] }) # Add user message await conn.conversation.item.create(item={ "type": "message", "role": "user", "content": [{"type": "input_text", "text": "Hello!"}] }) await conn.response.create() ``` ## Voice Options | Voice | Description | |-------|-------------| | `alloy` | Neutral, balanced | | `echo` | Warm, conversational | | `shimmer` | Clear, professional | | `sage` | Calm, authoritative | | `coral` | Friendly, upbeat | | `ash` | Deep, measured | | `ballad` | Expressive | | `verse` | Storytelling | Azure voices: Use `AzureStandardVoice`, `AzureCustomVoice`, or `AzurePersonalVoice` models. ## Audio Formats | Format | Sample Rate | Use Case | |--------|-------------|----------| | `pcm16` | 24kHz | Default, high quality | | `pcm16-8000hz` | 8kHz | Telephony | | `pcm16-16000hz` | 16kHz | Voice assistants | | `g711_ulaw` | 8kHz | Telephony (US) | | `g711_alaw` | 8kHz | Telephony (EU) | ## Turn Detection Options ```python # Server VAD (default) {"type": "server_vad", "threshold": 0.5, "silence_duration_ms": 500} # Azure Semantic VAD (smarter detection) {"type": "azure_semantic_vad"} {"type": "azure_semantic_vad_en"} # English optimized {"type": "azure_semantic_vad_multilingual"} ``` ## Error Handling ```python from azure.ai.voicelive.aio import ConnectionError, ConnectionClosed try: async with connect(...) as conn: async for event in conn: if event.type == "error": print(f"API Error: {event.error.code} - {event.error.message}") except ConnectionClosed as e: print(f"Connection closed: {e.code} - {e.reason}") except ConnectionError as e: print(f"Connection error: {e}") ``` ## Best Practices 1. **This SDK is async-only; use `azure.ai.voicelive.aio` throughout.** Do not try to pair it with sync clients from other Azure SDKs in the same call path — keep the whole request path async. 2. **Always use context managers for clients and async credentials.** Wrap every connection in `async with connect(...) as conn:`. For async `DefaultAzureCredential` from `azure.identity.aio`, also use `async with credential:` so tokens and transports are cleaned up. ## References - **Detailed API Reference**: See [references/api-reference.md](references/api-reference.md) - **Complete Examples**: See [references/examples.md](references/examples.md) - **All Models & Types**: See [references/models.md](references/models.md) ```` ### `references/api-reference.md` ````markdown # Azure AI Voice Live SDK - API Reference ## Table of Contents - [connect() Function](#connect-function) - [VoiceLiveConnection](#voiceliveconnection) - [SessionResource](#sessionresource) - [ResponseResource](#responseresource) - [InputAudioBufferResource](#inputaudiobufferresource) - [OutputAudioBufferResource](#outputaudiobufferresource) - [ConversationResource](#conversationresource) - [TranscriptionSessionResource](#transcriptionsessionresource) - [WebsocketConnectionOptions](#websocketconnectionoptions) - [Exceptions](#exceptions) --- ## connect() Function Creates an async context manager for WebSocket connections. ```python from azure.ai.voicelive.aio import connect async with connect( credential: Union[AzureKeyCredential, AsyncTokenCredential], endpoint: str, api_version: str = "2025-10-01", model: Optional[str] = None, query: Optional[Mapping[str, Any]] = None, headers: Optional[Mapping[str, Any]] = None, connection_options: Optional[WebsocketConnectionOptions] = None, credential_scopes: Optional[List[str]] = None, **kwargs ) -> VoiceLiveConnection: ... ``` ### Parameters | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `credential` | `AzureKeyCredential` or `AsyncTokenCredential` | Yes | Authentication credential | | `endpoint` | `str` | Yes | Service endpoint URL | | `api_version` | `str` | No | API version (default: "2025-10-01") | | `model` | `str` | Sometimes | Model identifier (required unless using Agent scenario) | | `query` | `Mapping[str, Any]` | No | Additional URL query parameters | | `headers` | `Mapping[str, Any]` | No | Additional HTTP headers | | `connection_options` | `WebsocketConnectionOptions` | No | WebSocket transport options | | `credential_scopes` | `List[str]` | No | OAuth scopes (default: `["https://ai.azure.com/.default"]`) | --- ## VoiceLiveConnection Main connection class with resource accessors. ### Properties | Property | Type | Description | |----------|------|-------------| | `session` | `SessionResource` | Session configuration | | `response` | `ResponseResource` | Response management | | `input_audio_buffer` | `InputAudioBufferResource` | Audio input buffer | | `output_audio_buffer` | `OutputAudioBufferResource` | Audio output buffer | | `conversation` | `ConversationResource` | Conversation items | | `transcription_session` | `TranscriptionSessionResource` | Transcription config | ### Methods ```python async def recv() -> ServerEvent: """Receive and parse the next typed server event.""" async def recv_bytes() -> bytes: """Receive raw bytes from the connection.""" async def send(event: Union[Mapping[str, Any], ClientEvent]) -> None: """Send an event to the server.""" async def close(*, code: int = 1000, reason: str = "") -> None: """Close the WebSocket connection.""" async def __aiter__() -> AsyncIterator[ServerEvent]: """Iterate over server events until connection closes.""" ``` --- ## SessionResource Manage session configuration. ### Methods ```python async def update( *, session: Union[Mapping[str, Any], RequestSession], event_id: Optional[str] = None ) -> None: """Update session configuration.""" ``` ### RequestSession Fields | Field | Type | Description | |-------|------|-------------| | `instructions` | `str` | System prompt for the model | | `modalities` | `List[Modality]` | `["text"]`, `["audio"]`, or `["text", "audio"]` | | `voice` | `Voice` | Voice for audio output | | `input_audio_format` | `InputAudioFormat` | Audio input format | | `output_audio_format` | `OutputAudioFormat` | Audio output format | | `turn_detection` | `TurnDetection` | VAD configuration or `None` for manual | | `tools` | `List[Tool]` | Function tools | | `tool_choice` | `ToolChoice` | `"auto"`, `"none"`, `"required"`, or specific function | | `temperature` | `float` | Model temperature (0.6-1.2) | | `max_response_output_tokens` | `int` or `"inf"` | Max tokens per response | | `input_audio_transcription` | `AudioInputTranscriptionOptions` | Transcription settings | --- ## ResponseResource Manage model responses. ### Methods ```python async def create( *, response: Optional[Union[ResponseCreateParams, Mapping[str, Any]]] = None, event_id: Optional[str] = None, additional_instructions: Optional[str] = None ) -> None: """Create a response (trigger inference).""" async def cancel( *, response_id: Optional[str] = None, event_id: Optional[str] = None ) -> None: """Cancel an in-progress response.""" ``` ### ResponseCreateParams Fields | Field | Type | Description | |-------|------|-------------| | `modalities` | `List[Modality]` | Override session modalities | | `instructions` | `str` | Override session instructions | | `voice` | `Voice` | Override session voice | | `temperature` | `float` | Override temperature | | `max_response_output_tokens` | `int` | Override max tokens | | `conversation` | `str` | `"auto"` or `"none"` | --- ## InputAudioBufferResource Manage audio input buffer. ### Methods ```python async def append( *, audio: str, # Base64-encoded audio event_id: Optional[str] = None ) -> None: """Append audio data to the input buffer.""" async def commit( *, event_id: Optional[str] = None ) -> None: """Commit the buffer as a user message.""" async def clear( *, event_id: Optional[str] = None ) -> None: """Clear the input buffer without committing.""" ``` --- ## OutputAudioBufferResource Manage audio output buffer. ### Methods ```python async def clear( *, event_id: Optional[str] = None ) -> None: """Clear pending audio output (for interrupts).""" ``` --- ## ConversationResource Manage conversation state. ### Properties | Property | Type | Description | |----------|------|-------------| | `item` | `ConversationItemResource` | Item operations | --- ## ConversationItemResource CRUD operations on conversation items. ### Methods ```python async def create( *, item: Union[ConversationRequestItem, Mapping[str, Any]], previous_item_id: Optional[str] = None, event_id: Optional[str] = None ) -> None: """Create a new conversation item.""" async def delete( *, item_id: str, event_id: Optional[str] = None ) -> None: """Delete a conversation item.""" async def retrieve( *, item_id: str, event_id: Optional[str] = None ) -> None: """Retrieve item details (server responds with event).""" async def truncate( *, item_id: str, audio_end_ms: int, content_index: int, event_id: Optional[str] = None ) -> None: """Truncate audio at specified time.""" ``` ### ConversationRequestItem Types ```python # User/Assistant/System message { "type": "message", "role": "user" | "assistant" | "system", "content": [ {"type": "input_text", "text": "..."}, {"type": "input_audio", "audio": "base64..."}, {"type": "text", "text": "..."}, {"type": "audio", "audio": "base64...", "transcript": "..."} ] } # Function call (from model) { "type": "function_call", "call_id": "...", "name": "function_name", "arguments": "{...}" } # Function output (from client) { "type": "function_call_output", "call_id": "...", "output": "{...}" } ``` --- ## TranscriptionSessionResource Configure input transcription. ### Methods ```python async def update( *, session: Mapping[str, Any], event_id: Optional[str] = None ) -> None: """Update transcription session configuration.""" ``` --- ## WebsocketConnectionOptions Transport configuration for the WebSocket connection. | Field | Type | Default | Description | |-------|------|---------|-------------| | `compression` | `bool` or `int` | None | Enable per-message compression | | `max_msg_size` | `int` | 4MB | Maximum message size | | `heartbeat` | `float` | 30 | Keep-alive ping interval (seconds) | | `autoclose` | `bool` | True | Auto-close on close frame | | `autoping` | `bool` | True | Auto-respond to pings | | `receive_timeout` | `float` | None | Message receive timeout | | `close_timeout` | `float` | None | Close handshake timeout | | `handshake_timeout` | `float` | None | Connection establishment timeout | | `vendor_options` | `Mapping` | None | Implementation-specific options | --- ## Exceptions ```python from azure.ai.voicelive.aio import ConnectionError, ConnectionClosed class ConnectionError(AzureError): """Base WebSocket connection error.""" class ConnectionClosed(ConnectionError): """WebSocket connection was closed.""" code: int # Close code reason: str # Close reason ``` ### Common Close Codes | Code | Meaning | |------|---------| | 1000 | Normal closure | | 1001 | Going away | | 1006 | Abnormal closure | | 1008 | Policy violation | | 1011 | Server error | ```` ### `references/examples.md` ````markdown # Azure AI Voice Live SDK - Examples ## Table of Contents - [Basic Voice Assistant](#basic-voice-assistant) - [Function Calling](#function-calling) - [Manual Turn Control](#manual-turn-control) - [Audio File Processing](#audio-file-processing) - [Interrupt Handling](#interrupt-handling) - [Multi-modal (Text + Audio)](#multi-modal-text--audio) - [Azure Voice Integration](#azure-voice-integration) - [Avatar Integration](#avatar-integration) - [Transcription Only](#transcription-only) --- ## Basic Voice Assistant Complete voice assistant with Server VAD. ```python import asyncio import base64 from azure.ai.voicelive.aio import connect from azure.core.credentials import AzureKeyCredential async def voice_assistant(): async with connect( endpoint="https://eastus.api.cognitive.microsoft.com", credential=AzureKeyCredential("YOUR_KEY"), model="gpt-4o-realtime-preview" ) as conn: # Configure session await conn.session.update(session={ "instructions": "You are a helpful voice assistant. Be concise.", "modalities": ["text", "audio"], "voice": "alloy", "turn_detection": { "type": "server_vad", "threshold": 0.5, "silence_duration_ms": 500 }, "input_audio_transcription": { "model": "whisper-1" } }) # Start microphone input (pseudo-code) mic_task = asyncio.create_task(stream_microphone(conn)) # Process events async for event in conn: match event.type: case "session.created": print("Session ready") case "input_audio_buffer.speech_started": print("🎤 Listening...") case "conversation.item.input_audio_transcription.completed": print(f"You: {event.transcript}") case "response.audio.delta": audio = base64.b64decode(event.delta) await play_audio(audio) case "response.audio_transcript.done": print(f"Assistant: {event.transcript}") case "error": print(f"Error: {event.error.message}") break async def stream_microphone(conn): """Stream microphone audio to the connection.""" async for chunk in read_microphone(): # Your audio capture b64 = base64.b64encode(chunk).decode() await conn.input_audio_buffer.append(audio=b64) asyncio.run(voice_assistant()) ``` --- ## Function Calling Voice assistant with tool use. ```python import asyncio import json import base64 from azure.ai.voicelive.aio import connect from azure.ai.voicelive.models import FunctionTool from azure.core.credentials import AzureKeyCredential # Define tools TOOLS = [ FunctionTool( type="function", name="get_weather", description="Get current weather for a location", parameters={ "type": "object", "properties": { "location": { "type": "string", "description": "City and state, e.g. 'San Francisco, CA'" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"], "default": "fahrenheit" } }, "required": ["location"] } ), FunctionTool( type="function", name="set_reminder", description="Set a reminder for the user", parameters={ "type": "object", "properties": { "message": {"type": "string"}, "time": {"type": "string", "description": "ISO 8601 datetime"} }, "required": ["message", "time"] } ) ] def handle_function_call(name: str, arguments: str) -> dict: """Execute function and return result.""" args = json.loads(arguments) if name == "get_weather": # Mock weather API return { "location": args["location"], "temperature": 72, "unit": args.get("unit", "fahrenheit"), "conditions": "sunny" } elif name == "set_reminder": # Mock reminder service return {"status": "success", "reminder_id": "123"} else: return {"error": f"Unknown function: {name}"} async def function_calling_assistant(): async with connect( endpoint="https://eastus.api.cognitive.microsoft.com", credential=AzureKeyCredential("YOUR_KEY"), model="gpt-4o-realtime-preview" ) as conn: await conn.session.update(session={ "instructions": "You can check weather and set reminders.", "modalities": ["text", "audio"], "voice": "alloy", "tools": TOOLS, "tool_choice": "auto" }) async for event in conn: match event.type: case "response.function_call_arguments.done": # Execute the function result = handle_function_call(event.name, event.arguments) # Send result back await conn.conversation.item.create(item={ "type": "function_call_output", "call_id": event.call_id, "output": json.dumps(result) }) # Continue the conversation await conn.response.create() case "response.audio.delta": audio = base64.b64decode(event.delta) await play_audio(audio) case "response.done": if event.response.status == "completed": print("Response complete") asyncio.run(function_calling_assistant()) ``` --- ## Manual Turn Control Push-to-talk style without VAD. ```python import asyncio import base64 from azure.ai.voicelive.aio import connect from azure.core.credentials import AzureKeyCredential async def push_to_talk(): async with connect( endpoint="https://eastus.api.cognitive.microsoft.com", credential=AzureKeyCredential("YOUR_KEY"), model="gpt-4o-realtime-preview" ) as conn: # Disable VAD for manual control await conn.session.update(session={ "instructions": "You are a helpful assistant.", "modalities": ["text", "audio"], "voice": "alloy", "turn_detection": None # Disable VAD }) # Simulate push-to-talk while True: input("Press Enter to start recording...") # Record audio (simulate with chunks) chunks = await record_audio_until_release() # Send all audio for chunk in chunks: b64 = base64.b64encode(chunk).decode() await conn.input_audio_buffer.append(audio=b64) # Commit and request response await conn.input_audio_buffer.commit() await conn.response.create() # Wait for response async for event in conn: if event.type == "response.audio.delta": audio = base64.b64decode(event.delta) await play_audio(audio) elif event.type == "response.done": break asyncio.run(push_to_talk()) ``` --- ## Audio File Processing Process an audio file and get a response. ```python import asyncio import base64 from pathlib import Path from azure.ai.voicelive.aio import connect from azure.core.credentials import AzureKeyCredential async def process_audio_file(audio_path: str): async with connect( endpoint="https://eastus.api.cognitive.microsoft.com", credential=AzureKeyCredential("YOUR_KEY"), model="gpt-4o-realtime-preview" ) as conn: await conn.session.update(session={ "instructions": "Respond to the audio message.", "modalities": ["text", "audio"], "voice": "alloy", "turn_detection": None, "input_audio_transcription": {"model": "whisper-1"} }) # Read and send audio file audio_data = Path(audio_path).read_bytes() # Send in chunks (24kHz * 2 bytes * 0.1s = 4800 bytes) chunk_size = 4800 for i in range(0, len(audio_data), chunk_size): chunk = audio_data[i:i + chunk_size] b64 = base64.b64encode(chunk).decode() await conn.input_audio_buffer.append(audio=b64) # Commit and request response await conn.input_audio_buffer.commit() await conn.response.create() # Collect response response_audio = bytearray() response_text = "" user_transcript = "" async for event in conn: match event.type: case "conversation.item.input_audio_transcription.completed": user_transcript = event.transcript case "response.audio.delta": response_audio.extend(base64.b64decode(event.delta)) case "response.audio_transcript.done": response_text = event.transcript case "response.done": break return { "user_said": user_transcript, "assistant_said": response_text, "audio": bytes(response_audio) } result = asyncio.run(process_audio_file("input.pcm")) print(f"User: {result['user_said']}") print(f"Assistant: {result['assistant_said']}") Path("output.pcm").write_bytes(result['audio']) ``` --- ## Interrupt Handling Handle user interruptions gracefully. ```python import asyncio import base64 from azure.ai.voicelive.aio import connect from azure.core.credentials import AzureKeyCredential async def interruptible_assistant(): async with connect( endpoint="https://eastus.api.cognitive.microsoft.com", credential=AzureKeyCredential("YOUR_KEY"), model="gpt-4o-realtime-preview" ) as conn: await conn.session.update(session={ "instructions": "You are a helpful assistant.", "modalities": ["text", "audio"], "voice": "alloy", "turn_detection": { "type": "server_vad", "threshold": 0.5, "silence_duration_ms": 500 } }) is_responding = False async for event in conn: match event.type: case "response.created": is_responding = True case "response.done": is_responding = False case "input_audio_buffer.speech_started": if is_responding: # User interrupted - stop current response print("🛑 Interrupt detected!") await conn.response.cancel() await conn.output_audio_buffer.clear() case "response.audio.delta": if is_responding: audio = base64.b64decode(event.delta) await play_audio(audio) asyncio.run(interruptible_assistant()) ``` --- ## Multi-modal (Text + Audio) Send text context, receive audio response. ```python import asyncio import base64 from azure.ai.voicelive.aio import connect from azure.core.credentials import AzureKeyCredential async def multimodal_assistant(): async with connect( endpoint="https://eastus.api.cognitive.microsoft.com", credential=AzureKeyCredential("YOUR_KEY"), model="gpt-4o-realtime-preview" ) as conn: await conn.session.update(session={ "instructions": "You are a helpful assistant.", "modalities": ["text", "audio"], "voice": "alloy", "turn_detection": None }) # Add context via text await conn.conversation.item.create(item={ "type": "message", "role": "system", "content": [{"type": "input_text", "text": "The user's name is Alice."}] }) # Add user message as text await conn.conversation.item.create(item={ "type": "message", "role": "user", "content": [{"type": "input_text", "text": "What's my name?"}] }) # Request audio response await conn.response.create() async for event in conn: if event.type == "response.audio.delta": audio = base64.b64decode(event.delta) await play_audio(audio) elif event.type == "response.done": break asyncio.run(multimodal_assistant()) ``` --- ## Azure Voice Integration Use Azure Text-to-Speech voices. ```python import asyncio from azure.ai.voicelive.aio import connect from azure.ai.voicelive.models import AzureStandardVoice, AzureCustomVoice from azure.core.credentials import AzureKeyCredential async def azure_voice_assistant(): async with connect( endpoint="https://eastus.api.cognitive.microsoft.com", credential=AzureKeyCredential("YOUR_KEY"), model="gpt-4o-realtime-preview" ) as conn: # Use Azure standard voice await conn.session.update(session={ "instructions": "You are a helpful assistant.", "modalities": ["text", "audio"], "voice": AzureStandardVoice( type="azure-standard", name="en-US-JennyNeural" ) }) # Or use custom voice # await conn.session.update(session={ # "voice": AzureCustomVoice( # type="azure-custom", # endpoint_id="your-custom-voice-endpoint", # name="YourCustomVoice" # ) # }) async for event in conn: # ... handle events pass asyncio.run(azure_voice_assistant()) ``` --- ## Avatar Integration Connect to Azure Avatar for visual output. ```python import asyncio from azure.ai.voicelive.aio import connect from azure.core.credentials import AzureKeyCredential async def avatar_assistant(): async with connect( endpoint="https://eastus.api.cognitive.microsoft.com", credential=AzureKeyCredential("YOUR_KEY"), model="gpt-4o-realtime-preview" ) as conn: await conn.session.update(session={ "instructions": "You are a helpful assistant.", "modalities": ["text", "audio", "avatar"], "voice": "alloy", "avatar": { "type": "video-avatar", "character": "lisa", "output_protocol": "webrtc" } }) # Connect avatar await conn.send({ "type": "session.avatar.connect" }) async for event in conn: match event.type: case "session.avatar.connecting": ice_servers = event.ice_servers # Use ice_servers for WebRTC connection print(f"Avatar connecting with {len(ice_servers)} ICE servers") case "response.audio.delta": # Audio is streamed via WebRTC, not this event pass asyncio.run(avatar_assistant()) ``` --- ## Transcription Only Speech-to-text without AI response. ```python import asyncio import base64 from azure.ai.voicelive.aio import connect from azure.core.credentials import AzureKeyCredential async def transcription_only(): async with connect( endpoint="https://eastus.api.cognitive.microsoft.com", credential=AzureKeyCredential("YOUR_KEY"), model="gpt-4o-realtime-preview" ) as conn: await conn.session.update(session={ "modalities": ["text"], # No audio output "turn_detection": { "type": "server_vad", "threshold": 0.5, "silence_duration_ms": 1000 }, "input_audio_transcription": { "model": "whisper-1" } }) # Stream microphone mic_task = asyncio.create_task(stream_microphone(conn)) transcripts = [] async for event in conn: match event.type: case "conversation.item.input_audio_transcription.delta": print(event.delta, end="", flush=True) case "conversation.item.input_audio_transcription.completed": print() # Newline transcripts.append(event.transcript) return transcripts asyncio.run(transcription_only()) ``` ```` ### `references/models.md` ````markdown # Azure AI Voice Live SDK - Models Reference ## Table of Contents - [Enums](#enums) - [Client Events](#client-events) - [Server Events](#server-events) - [Session Models](#session-models) - [Conversation Items](#conversation-items) - [Content Parts](#content-parts) - [Tools](#tools) - [Voice Models](#voice-models) - [Turn Detection](#turn-detection) - [Response Models](#response-models) - [Avatar Models](#avatar-models) --- ## Enums ### Modality ```python class Modality(str, Enum): TEXT = "text" AUDIO = "audio" ANIMATION = "animation" AVATAR = "avatar" ``` ### OpenAIVoiceName ```python class OpenAIVoiceName(str, Enum): ALLOY = "alloy" ASH = "ash" BALLAD = "ballad" CORAL = "coral" ECHO = "echo" SAGE = "sage" SHIMMER = "shimmer" VERSE = "verse" MARIN = "marin" CEDAR = "cedar" ``` ### InputAudioFormat ```python class InputAudioFormat(str, Enum): PCM16 = "pcm16" # 24kHz default G711_ULAW = "g711_ulaw" # 8kHz G711_ALAW = "g711_alaw" # 8kHz ``` ### OutputAudioFormat ```python class OutputAudioFormat(str, Enum): PCM16 = "pcm16" # 24kHz PCM16_8000_HZ = "pcm16-8000hz" PCM16_16000_HZ = "pcm16-16000hz" G711_ULAW = "g711_ulaw" # 8kHz G711_ALAW = "g711_alaw" # 8kHz ``` ### TurnDetectionType ```python class TurnDetectionType(str, Enum): SERVER_VAD = "server_vad" AZURE_SEMANTIC_VAD = "azure_semantic_vad" AZURE_SEMANTIC_VAD_EN = "azure_semantic_vad_en" AZURE_SEMANTIC_VAD_MULTILINGUAL = "azure_semantic_vad_multilingual" ``` ### MessageRole ```python class MessageRole(str, Enum): SYSTEM = "system" USER = "user" ASSISTANT = "assistant" ``` ### ItemType ```python class ItemType(str, Enum): MESSAGE = "message" FUNCTION_CALL = "function_call" FUNCTION_CALL_OUTPUT = "function_call_output" MCP_LIST_TOOLS = "mcp_list_tools" MCP_CALL = "mcp_call" MCP_APPROVAL_REQUEST = "mcp_approval_request" MCP_APPROVAL_RESPONSE = "mcp_approval_response" ``` ### ContentPartType ```python class ContentPartType(str, Enum): INPUT_TEXT = "input_text" INPUT_AUDIO = "input_audio" INPUT_IMAGE = "input_image" TEXT = "text" AUDIO = "audio" ``` ### ToolType ```python class ToolType(str, Enum): FUNCTION = "function" MCP = "mcp" ``` ### ToolChoiceLiteral ```python class ToolChoiceLiteral(str, Enum): AUTO = "auto" NONE = "none" REQUIRED = "required" ``` ### ResponseStatus ```python class ResponseStatus(str, Enum): COMPLETED = "completed" CANCELLED = "cancelled" FAILED = "failed" INCOMPLETE = "incomplete" IN_PROGRESS = "in_progress" ``` ### ClientEventType ```python class ClientEventType(str, Enum): SESSION_UPDATE = "session.update" INPUT_AUDIO_BUFFER_APPEND = "input_audio_buffer.append" INPUT_AUDIO_BUFFER_COMMIT = "input_audio_buffer.commit" INPUT_AUDIO_BUFFER_CLEAR = "input_audio_buffer.clear" INPUT_AUDIO_TURN_START = "input_audio.turn.start" INPUT_AUDIO_TURN_APPEND = "input_audio.turn.append" INPUT_AUDIO_TURN_END = "input_audio.turn.end" INPUT_AUDIO_TURN_CANCEL = "input_audio.turn.cancel" INPUT_AUDIO_CLEAR = "input_audio.clear" CONVERSATION_ITEM_CREATE = "conversation.item.create" CONVERSATION_ITEM_RETRIEVE = "conversation.item.retrieve" CONVERSATION_ITEM_TRUNCATE = "conversation.item.truncate" CONVERSATION_ITEM_DELETE = "conversation.item.delete" RESPONSE_CREATE = "response.create" RESPONSE_CANCEL = "response.cancel" SESSION_AVATAR_CONNECT = "session.avatar.connect" MCP_APPROVAL_RESPONSE = "mcp_approval_response" ``` ### ServerEventType ```python class ServerEventType(str, Enum): ERROR = "error" SESSION_AVATAR_CONNECTING = "session.avatar.connecting" SESSION_CREATED = "session.created" SESSION_UPDATED = "session.updated" CONVERSATION_ITEM_INPUT_AUDIO_TRANSCRIPTION_COMPLETED = "conversation.item.input_audio_transcription.completed" CONVERSATION_ITEM_INPUT_AUDIO_TRANSCRIPTION_DELTA = "conversation.item.input_audio_transcription.delta" CONVERSATION_ITEM_INPUT_AUDIO_TRANSCRIPTION_FAILED = "conversation.item.input_audio_transcription.failed" CONVERSATION_ITEM_CREATED = "conversation.item.created" CONVERSATION_ITEM_RETRIEVED = "conversation.item.retrieved" CONVERSATION_ITEM_TRUNCATED = "conversation.item.truncated" CONVERSATION_ITEM_DELETED = "conversation.item.deleted" INPUT_AUDIO_BUFFER_COMMITTED = "input_audio_buffer.committed" INPUT_AUDIO_BUFFER_CLEARED = "input_audio_buffer.cleared" INPUT_AUDIO_BUFFER_SPEECH_STARTED = "input_audio_buffer.speech_started" INPUT_AUDIO_BUFFER_SPEECH_STOPPED = "input_audio_buffer.speech_stopped" RESPONSE_CREATED = "response.created" RESPONSE_DONE = "response.done" RESPONSE_OUTPUT_ITEM_ADDED = "response.output_item.added" RESPONSE_OUTPUT_ITEM_DONE = "response.output_item.done" RESPONSE_CONTENT_PART_ADDED = "response.content_part.added" RESPONSE_CONTENT_PART_DONE = "response.content_part.done" RESPONSE_TEXT_DELTA = "response.text.delta" RESPONSE_TEXT_DONE = "response.text.done" RESPONSE_AUDIO_TRANSCRIPT_DELTA = "response.audio_transcript.delta" RESPONSE_AUDIO_TRANSCRIPT_DONE = "response.audio_transcript.done" RESPONSE_AUDIO_DELTA = "response.audio.delta" RESPONSE_AUDIO_DONE = "response.audio.done" RESPONSE_FUNCTION_CALL_ARGUMENTS_DELTA = "response.function_call_arguments.delta" RESPONSE_FUNCTION_CALL_ARGUMENTS_DONE = "response.function_call_arguments.done" # MCP events MCP_LIST_TOOLS_IN_PROGRESS = "mcp_list_tools.in_progress" MCP_LIST_TOOLS_COMPLETED = "mcp_list_tools.completed" MCP_LIST_TOOLS_FAILED = "mcp_list_tools.failed" RESPONSE_MCP_CALL_ARGUMENTS_DELTA = "response.mcp_call_arguments.delta" RESPONSE_MCP_CALL_ARGUMENTS_DONE = "response.mcp_call_arguments.done" RESPONSE_MCP_CALL_IN_PROGRESS = "response.mcp_call.in_progress" RESPONSE_MCP_CALL_COMPLETED = "response.mcp_call.completed" RESPONSE_MCP_CALL_FAILED = "response.mcp_call.failed" # Animation events RESPONSE_ANIMATION_BLENDSHAPES_DELTA = "response.animation_blendshapes.delta" RESPONSE_ANIMATION_BLENDSHAPES_DONE = "response.animation_blendshapes.done" RESPONSE_ANIMATION_VISEME_DELTA = "response.animation_viseme.delta" RESPONSE_ANIMATION_VISEME_DONE = "response.animation_viseme.done" RESPONSE_AUDIO_TIMESTAMP_DELTA = "response.audio_timestamp.delta" RESPONSE_AUDIO_TIMESTAMP_DONE = "response.audio_timestamp.done" ``` --- ## Client Events ### ClientEventSessionUpdate ```python class ClientEventSessionUpdate(Model): type: Literal["session.update"] event_id: Optional[str] session: RequestSession ``` ### ClientEventInputAudioBufferAppend ```python class ClientEventInputAudioBufferAppend(Model): type: Literal["input_audio_buffer.append"] event_id: Optional[str] audio: str # Base64-encoded audio ``` ### ClientEventInputAudioBufferCommit ```python class ClientEventInputAudioBufferCommit(Model): type: Literal["input_audio_buffer.commit"] event_id: Optional[str] ``` ### ClientEventInputAudioBufferClear ```python class ClientEventInputAudioBufferClear(Model): type: Literal["input_audio_buffer.clear"] event_id: Optional[str] ``` ### ClientEventConversationItemCreate ```python class ClientEventConversationItemCreate(Model): type: Literal["conversation.item.create"] event_id: Optional[str] previous_item_id: Optional[str] item: ConversationRequestItem ``` ### ClientEventConversationItemDelete ```python class ClientEventConversationItemDelete(Model): type: Literal["conversation.item.delete"] event_id: Optional[str] item_id: str ``` ### ClientEventConversationItemTruncate ```python class ClientEventConversationItemTruncate(Model): type: Literal["conversation.item.truncate"] event_id: Optional[str] item_id: str content_index: int audio_end_ms: int ``` ### ClientEventResponseCreate ```python class ClientEventResponseCreate(Model): type: Literal["response.create"] event_id: Optional[str] response: Optional[ResponseCreateParams] additional_instructions: Optional[str] ``` ### ClientEventResponseCancel ```python class ClientEventResponseCancel(Model): type: Literal["response.cancel"] event_id: Optional[str] response_id: Optional[str] ``` --- ## Server Events ### ServerEventSessionCreated ```python class ServerEventSessionCreated(Model): type: Literal["session.created"] event_id: str session: ResponseSession ``` ### ServerEventSessionUpdated ```python class ServerEventSessionUpdated(Model): type: Literal["session.updated"] event_id: str session: ResponseSession ``` ### ServerEventError ```python class ServerEventError(Model): type: Literal["error"] event_id: str error: ServerEventErrorDetails class ServerEventErrorDetails(Model): type: str code: Optional[str] message: str param: Optional[str] ``` ### ServerEventInputAudioBufferSpeechStarted ```python class ServerEventInputAudioBufferSpeechStarted(Model): type: Literal["input_audio_buffer.speech_started"] event_id: str audio_start_ms: int item_id: str ``` ### ServerEventInputAudioBufferSpeechStopped ```python class ServerEventInputAudioBufferSpeechStopped(Model): type: Literal["input_audio_buffer.speech_stopped"] event_id: str audio_end_ms: int item_id: str ``` ### ServerEventConversationItemInputAudioTranscriptionCompleted ```python class ServerEventConversationItemInputAudioTranscriptionCompleted(Model): type: Literal["conversation.item.input_audio_transcription.completed"] event_id: str item_id: str content_index: int transcript: str ``` ### ServerEventConversationItemInputAudioTranscriptionDelta ```python class ServerEventConversationItemInputAudioTranscriptionDelta(Model): type: Literal["conversation.item.input_audio_transcription.delta"] event_id: str item_id: str content_index: int delta: str ``` ### ServerEventResponseCreated ```python class ServerEventResponseCreated(Model): type: Literal["response.created"] event_id: str response: Response ``` ### ServerEventResponseDone ```python class ServerEventResponseDone(Model): type: Literal["response.done"] event_id: str response: Response ``` ### ServerEventResponseAudioDelta ```python class ServerEventResponseAudioDelta(Model): type: Literal["response.audio.delta"] event_id: str response_id: str item_id: str output_index: int content_index: int delta: str # Base64-encoded audio ``` ### ServerEventResponseAudioTranscriptDelta ```python class ServerEventResponseAudioTranscriptDelta(Model): type: Literal["response.audio_transcript.delta"] event_id: str response_id: str item_id: str output_index: int content_index: int delta: str ``` ### ServerEventResponseAudioTranscriptDone ```python class ServerEventResponseAudioTranscriptDone(Model): type: Literal["response.audio_transcript.done"] event_id: str response_id: str item_id: str output_index: int content_index: int transcript: str ``` ### ServerEventResponseFunctionCallArgumentsDelta ```python class ServerEventResponseFunctionCallArgumentsDelta(Model): type: Literal["response.function_call_arguments.delta"] event_id: str response_id: str item_id: str output_index: int call_id: str delta: str ``` ### ServerEventResponseFunctionCallArgumentsDone ```python class ServerEventResponseFunctionCallArgumentsDone(Model): type: Literal["response.function_call_arguments.done"] event_id: str response_id: str item_id: str output_index: int call_id: str name: str arguments: str ``` --- ## Session Models ### RequestSession ```python class RequestSession(Model): instructions: Optional[str] modalities: Optional[List[Modality]] voice: Optional[Voice] # str, OpenAIVoiceName, OpenAIVoice, or AzureVoice input_audio_format: Optional[InputAudioFormat] output_audio_format: Optional[OutputAudioFormat] turn_detection: Optional[TurnDetection] tools: Optional[List[Tool]] tool_choice: Optional[ToolChoice] temperature: Optional[float] max_response_output_tokens: Optional[Union[int, Literal["inf"]]] input_audio_transcription: Optional[AudioInputTranscriptionOptions] ``` ### ResponseSession ```python class ResponseSession(Model): id: str object: str model: str expires_at: int modalities: List[Modality] instructions: Optional[str] voice: Optional[Voice] input_audio_format: InputAudioFormat output_audio_format: OutputAudioFormat turn_detection: Optional[TurnDetection] tools: List[Tool] tool_choice: ToolChoice temperature: float max_response_output_tokens: Optional[int] ``` ### AudioInputTranscriptionOptions ```python class AudioInputTranscriptionOptions(Model): model: str # e.g., "whisper-1" ``` --- ## Conversation Items ### ConversationRequestItem (Union Type) ```python # Can be one of: - SystemMessageItem - UserMessageItem - AssistantMessageItem - FunctionCallItem - FunctionCallOutputItem ``` ### MessageItem Base ```python class MessageItem(Model): type: Literal["message"] id: Optional[str] role: MessageRole content: List[ContentPart] status: Optional[ItemParamStatus] ``` ### FunctionCallItem ```python class FunctionCallItem(Model): type: Literal["function_call"] id: Optional[str] call_id: str name: str arguments: str status: Optional[ItemParamStatus] ``` ### FunctionCallOutputItem ```python class FunctionCallOutputItem(Model): type: Literal["function_call_output"] id: Optional[str] call_id: str output: str ``` --- ## Content Parts ### InputTextContentPart ```python class InputTextContentPart(Model): type: Literal["input_text"] text: str ``` ### InputAudioContentPart ```python class InputAudioContentPart(Model): type: Literal["input_audio"] audio: str # Base64 transcript: Optional[str] ``` ### RequestTextContentPart ```python class RequestTextContentPart(Model): type: Literal["text"] text: str ``` ### RequestAudioContentPart ```python class RequestAudioContentPart(Model): type: Literal["audio"] audio: str # Base64 transcript: Optional[str] ``` ### RequestImageContentPart ```python class RequestImageContentPart(Model): type: Literal["input_image"] url: Optional[str] base64: Optional[str] detail: Optional[RequestImageContentPartDetail] # "auto", "low", "high" ``` --- ## Tools ### FunctionTool ```python class FunctionTool(Model): type: Literal["function"] name: str description: Optional[str] parameters: Optional[dict] # JSON Schema ``` ### MCPTool ```python class MCPTool(Model): type: Literal["mcp"] server_label: str require_approval: Optional[MCPApprovalType] # "never" or "always" ``` ### MCPServer ```python class MCPServer(Model): type: Literal["url"] url: str name: str tool_configuration: Optional[dict] ``` ### ToolChoiceSelection ```python class ToolChoiceSelection(Model): type: Literal["function"] name: str ``` --- ## Voice Models ### OpenAIVoice ```python class OpenAIVoice(Model): type: Literal["openai"] name: OpenAIVoiceName ``` ### AzureStandardVoice ```python class AzureStandardVoice(Model): type: Literal["azure-standard"] name: str # e.g., "en-US-JennyNeural" ``` ### AzureCustomVoice ```python class AzureCustomVoice(Model): type: Literal["azure-custom"] endpoint_id: str name: str ``` ### AzurePersonalVoice ```python class AzurePersonalVoice(Model): type: Literal["azure-personal"] speaker_profile_id: str model: Optional[PersonalVoiceModels] ``` --- ## Turn Detection ### ServerVad ```python class ServerVad(Model): type: Literal["server_vad"] threshold: Optional[float] # 0.0-1.0 prefix_padding_ms: Optional[int] silence_duration_ms: Optional[int] create_response: Optional[bool] ``` ### AzureSemanticVad ```python class AzureSemanticVad(Model): type: Literal["azure_semantic_vad"] # Uses semantic understanding for better turn detection ``` ### AzureSemanticVadEn ```python class AzureSemanticVadEn(Model): type: Literal["azure_semantic_vad_en"] eou_detection: Optional[EouDetection] ``` ### EouDetection ```python class EouDetection(Model): threshold_level: Optional[EouThresholdLevel] # "low", "medium", "high", "default" ``` --- ## Response Models ### Response ```python class Response(Model): id: str object: Literal["realtime.response"] status: ResponseStatus status_details: Optional[ResponseStatusDetails] output: List[ResponseItem] usage: Optional[TokenUsage] ``` ### ResponseCreateParams ```python class ResponseCreateParams(Model): modalities: Optional[List[Modality]] instructions: Optional[str] voice: Optional[Voice] output_audio_format: Optional[OutputAudioFormat] tools: Optional[List[Tool]] tool_choice: Optional[ToolChoice] temperature: Optional[float] max_response_output_tokens: Optional[Union[int, Literal["inf"]]] conversation: Optional[Literal["auto", "none"]] input: Optional[List[ConversationRequestItem]] ``` ### TokenUsage ```python class TokenUsage(Model): total_tokens: int input_tokens: int output_tokens: int input_token_details: Optional[InputTokenDetails] output_token_details: Optional[OutputTokenDetails] ``` --- ## Avatar Models ### AvatarConfig ```python class AvatarConfig(Model): type: AvatarConfigTypes # "video-avatar" or "photo-avatar" character: str style: Optional[str] output_protocol: Optional[AvatarOutputProtocol] # "webrtc" or "websocket" background: Optional[Background] video_params: Optional[VideoParams] ``` ### IceServer ```python class IceServer(Model): urls: List[str] username: Optional[str] credential: Optional[str] ``` ### Background ```python class Background(Model): color: Optional[str] # Hex color image_url: Optional[str] ``` ### VideoParams ```python class VideoParams(Model): resolution: Optional[VideoResolution] crop: Optional[VideoCrop] ``` ````