01 / Real-time
Know who is speaking, and when to answer.
Send raw audio over a single WebSocket. Get back structured JSON events — turns, identity, interruptions, noise — in under a second, without a transcript.
01
One connection
Open /v1/stream, push audio frames, read events. There is no model to host, no GPU to rent, and no transcript in the loop — decisions arrive as JSON while the person is still in the room.
from vsip import VSIPClient
client = VSIPClient(api_key="vsip_...", mode="full")
async with client:
async for event in client.events():
if event.type == "speaker_verification":
print(event.should_respond) # authoritative response gate
elif event.type == "barge_in":
await my_tts.stop() # they cut in — stop talking02
Identity-gated barge-in
Most stacks either ignore interruptions or stop for every sound. VSIP locks onto the speaker's voice in their first sentence, then only lets that voice interrupt. A television in the background, a colleague across the desk, or the agent's own TTS echo will not take the floor.
When speaker_verification fails, should_respond is false — your agent stays quiet instead of answering a stranger.
03
The events
turn_start / turn_end
Semantic endpointing decides when a person is actually finished — not when they merely paused. Mid-thought silences no longer trigger your agent.
barge_in
Fires in ~250ms when the verified speaker interrupts. Background voices and TV audio do not. The interrupting utterance is a first-class turn.
speaker_verification
A verdict on whether the current voice matches the locked identity, from the first sentence — with a confidence score you can threshold.
speaker_change
A different person took the floor. Useful for multi-party rooms and for dropping responses to whoever is not the customer.
noise_event
Classified non-speech audio — TV, music, keyboard, dog. Never reaches your LLM as a query.
lock_status
Whether the session has locked onto an identity yet, and to whom. Gate your responses on it.
04
Wire format & limits
05
Works with your stack
Adapters ship for Pipecat, LiveKit and Twilio Media Streams, and the sidecar mode runs alongside speech-to-speech stacks like OpenAI Realtime that own their own audio loop. Sidecar emits the guaranteed identity and separation events without controlling TTS or barge-in.