A voice agent framework can make a demo easy while leaving your team responsible for media transport, turn-taking, deployment, and every failure between providers. The right choice depends on which of those layers you need to control and which you are prepared to operate. Here is a practical way to compare Dasha, LiveKit Agents, Pipecat, and the OpenAI Agents SDK against a real production workload.
What is a voice agent framework?
A voice agent framework coordinates live audio, turn-taking, reasoning, tools, speech output, cancellation, and conversation state. A production-ready framework also supplies or connects the deployment and diagnostic systems needed to operate that loop.
The familiar speech-to-text (STT), large language model (LLM), and text-to-speech (TTS) pipeline describes three components. It leaves out the work that makes a conversation function:
- moving audio between a phone, browser, or app and the runtime;
- deciding when the user has started and finished a turn;
- stopping queued speech and generation after an interruption;
- keeping tool calls, spoken words, and conversation history consistent;
- recovering from model, network, carrier, and business-system failures; and
- correlating audio, transcripts, tool activity, latency, cost, and outcomes.
This broader definition prevents a common category error. LangGraph, CrewAI, Google ADK, Mastra, and similar general agent frameworks can organize reasoning, memory, and tools. They do not automatically provide real-time audio transport, voice activity detection, endpointing, barge-in, playback cancellation, or phone-call lifecycle management. They can sit behind a voice runtime. They do not replace it.
Our voice AI stack model expands these responsibilities into seven layers. For selecting a framework, the useful question is simpler: which layers become product dependencies, and who operates them at 3 a.m.?
Choose an architecture and operating model first
Framework comparisons often mix three different acquisition models. Separate them before comparing features.
| Operating model | What you receive | What your team owns |
|---|---|---|
| Managed production platform | Hosted voice runtime, agent configuration, telephony connections, deployment, and operational surfaces | Product logic, business systems, policy, carrier configuration, and acceptance criteria |
| Open-source framework plus cloud | Source framework with an optional managed media or deployment layer | Agent code and chosen self-hosted layers; cloud responsibility depends on the services used |
| Open-source SDK plus model API | Agent abstractions around a managed model service | Media integration, deployment, telephony, application state, and production operations outside the API |
The audio architecture creates a second decision.
- Cascaded STT, LLM, and TTS gives you explicit transcripts and component choice. It also creates more provider boundaries, cancellation paths, and failure modes.
- Speech-to-speech sends audio to a real-time model and receives audio back. It reduces visible stages, while component-level inspection and portability follow the model API.
- Hybrid routes conversational turns through speech-to-speech and precise transactions through structured state or a cascaded path. It offers flexibility at the cost of more state reconciliation and more regression cases.
No architecture removes the need to handle interruptions, tool timeouts, partial actions, and caller-side latency. It only moves those responsibilities.
Four voice agent framework options
| Option | Model | Strong fit | Main responsibility retained by your team |
|---|---|---|---|
| Dasha, our recommended managed option | Managed production platform | Technical teams shipping phone or web voice products without owning the real-time runtime | Business logic, tools, carrier account, policy, and outcome measurement |
| LiveKit Agents | Apache-2.0 framework plus optional cloud | Teams building voice, video, or multimodal participants around WebRTC rooms | Agent application and every layer not assigned to LiveKit Cloud |
| Pipecat | BSD-2-Clause Python framework plus optional cloud | Teams that want frame-level pipeline composition and broad provider choice | Pipeline behavior, deployment choice, adapter lifecycle, and application operations |
| OpenAI Agents SDK | MIT SDK around managed model APIs | OpenAI-centered apps that need real-time agents, tools, handoffs, and guardrails | Application hosting, channel integration, and operational systems around the API |
1. Dasha: managed runtime plus production operations
We recommend Dasha for technical teams that want to build a differentiated conversational product without becoming the maintainer of the real-time voice stack.
Dasha provides a managed runtime, REST APIs, and a web application. The current platform covers inbound and outbound calls, SIP and Twilio connections, browser voice and chat, LLM and voice configuration, tools, webhooks, Model Context Protocol (MCP) connections, knowledge bases, call history, activity logs, and completed-call inspection. Call Inspector brings the transcript, recording when enabled, model activity, tool executions, event timeline, and latency breakdown into one diagnostic view.
Your team still owns the agent's business rules, tool endpoints, downstream data, carrier account and routing, compliance decisions, and release criteria. Dasha is not an open-source media framework, and it is not the right choice when modifying buffers, codecs, or transport internals is part of your product. It fits teams that want those concerns behind a managed contract while retaining control of the customer experience and business workflow.
2. LiveKit Agents: real-time participants on a media platform
LiveKit Agents is an open-source Python and Node.js framework for server-side agents that participate in LiveKit rooms. Its framework combines agent sessions, model plugins, tools, handoffs, turn detection, testing primitives, and dispatch. The wider LiveKit stack supplies WebRTC clients and SIP telephony. Teams can self-host the open-source server and agent components or use LiveKit Cloud for inference, deployment, and observability.
This model suits products where rooms, web or mobile clients, video, or other real-time participants are first-class requirements. It also gives teams a path between self-hosting and managed cloud services.
The flexibility creates an explicit boundary. Your application still owns its domain state, tools, release policy, and whichever media, model, deployment, and observability layers are kept outside LiveKit Cloud. The Agents framework uses Apache 2.0; LiveKit's turn-detection models have a separate model license.
3. Pipecat: composable Python pipelines
Pipecat is a BSD-2-Clause Python framework maintained by Daily and its developer community. It moves typed frames through a real-time pipeline and offers integrations for transports, STT, LLMs, real-time models, TTS, tool calls, metrics, and tracing. Pipecat Flows adds structured conversation paths. Deployment can run on infrastructure your team controls or through Pipecat Cloud.
Pipecat is a strong fit when the order and behavior of the pipeline are product concerns. Engineers can replace services, intercept frames, create processors, and control how transport and model events move through the system.
That control also exposes difficult state problems. Interruption events can race with transcript, TTS, and tool frames. Reconnection can leave unplayed speech or incomplete tool state. Provider adapters change as upstream APIs evolve. Pipecat supplies the framework abstractions, while your architecture and operating model decide who owns those seams in production.
4. OpenAI Agents SDK: OpenAI-centered real-time agents
The OpenAI Agents SDK has separate Python and TypeScript packages. Its TypeScript RealtimeAgent and RealtimeSession support spoken interactions with tools, guardrails, handoffs, and conversation history over the Realtime API. The Python SDK also provides a cascaded VoicePipeline that runs speech recognition, an agent workflow, and speech synthesis.
This is the smallest conceptual step for a team already committed to OpenAI's model APIs and agent abstractions. The SDK is MIT-licensed, but the working voice system still depends on managed model services. Telephony, browser or app transport, deployment, state beyond the session, business tools, and production monitoring remain part of the application architecture.
Choose this path when OpenAI alignment is intentional and your team is prepared to build the surrounding voice product. A provider-neutral media layer or managed runtime fits better when model portability, telephony operations, or one correlated operational surface is a primary requirement.
Evaluate the framework at its failure boundaries
A polished greeting proves very little. A useful evaluation follows one real outcome through the deployed channel and forces the seams to fail.
1. Draw the actual media path
Name every hop from the caller to the agent and back: carrier, SIP trunk, media gateway, region, codec, runtime, model providers, and playback buffer. Browser audio and telephone audio are separate acceptance paths. A clean laptop microphone cannot represent packet loss, narrowband audio, mobile jitter, dual-tone multi-frequency input, or a transfer.
2. Exercise turn-taking and cancellation together
Voice activity detection (VAD) only estimates whether speech is present. Endpointing decides whether the user has yielded the turn. Barge-in decides when user speech cancels agent playback. Our VAD guide explains why these controls need separate evidence.
An interruption case should show:
- when caller speech began;
- when outbound audio stopped;
- which generated words were actually played;
- which model and TTS work was cancelled; and
- what conversation state entered the next turn.
Without those events, the agent may repeat itself or treat unheard speech as delivered context.
3. Make tools fail after work has started
Use at least one read and one consequential write. Include a timeout, malformed result, duplicate delivery, lost connection, and partial success. The framework must preserve idempotency, user confirmation, retry policy, and the real downstream state. A spoken success message is not proof that a booking, payment, or account change occurred.
4. Measure the complete turn and its tail
Component latency can locate a bottleneck. Caller-perceived latency measures the experience. Record user speech end, endpoint commit, stable transcript, first model output, tool time, first generated audio, and first audible caller-side audio. Keep p50, p95, and failed or no-audio turns together. Our voice AI latency guide provides the full measurement model.
5. Require correlated evidence
One call identifier should connect the audio, transcript, agent version, model events, tools, downstream outcome, transfer, and cost. OpenTelemetry's trace model provides spans, events, attributes, and links for distributed work. OpenTelemetry is one implementation path. Equivalent evidence that an operator can query is the requirement.
6. Compare outcome quality separately from voice experience
A voice agent can sound fluid and make the wrong change. It can complete the task and still interrupt the caller. ServiceNow's end-to-end EVA research found that evaluated systems occupied different accuracy and experience regions, and that critical-entity transcription was strongly associated with task completion. A single aggregate quality score hides that distinction.
Use separate gates for task success, critical-entity accuracy, tool correctness, policy violations, response latency, interruption cutoff, transfer completion, and technical failures. The voice agent evaluation framework shows how to turn those measures into versioned release gates.
A production pilot that can settle the decision
Run the same bounded pilot on each serious option:
- One real call path: use the intended phone or web channel, production-like routing, and one target caller region.
- One complete business outcome: begin from known data and end with a deterministic system-of-record assertion.
- One interruption-heavy scenario: include a correction, a hesitation, a backchannel, and barge-in during speech.
- One dependency failure set: delay the model or tool, return malformed data, duplicate an event, and fail a transfer.
- One concurrency step: reproduce the expected call and turn mix, including downstream rate limits and cold starts.
- One release change: alter the prompt, model, tool schema, or turn policy, then compare the candidate with the approved version and exercise the disable or rollback path.
The pilot result should include caller-side audio, a correlated trace, final backend state, p50 and tail latency, failures, engineering time, and the monthly operating responsibilities created by the design. Per-minute API cost alone cannot represent the cost of a framework that adds deployment, observability, on-call, and provider-upgrade work.
The NIST AI Risk Management Framework calls for evaluation under conditions similar to deployment, ongoing monitoring, and mechanisms to disengage systems that operate outside their intended use. Those are practical framework requirements, rather than paperwork added after launch.
Frequently asked questions
Is LangChain or LangGraph a voice agent framework?
No. They are general agent and workflow frameworks. They can provide reasoning, graph state, memory, or tool orchestration behind a voice agent. A separate layer still has to manage live audio, endpointing, interruptions, playback, channel lifecycle, and voice-specific operational evidence.
Pipecat or LiveKit: which model fits better?
Pipecat centers on a composable Python frame pipeline with broad transport and model integrations. LiveKit centers on real-time participants inside a WebRTC room ecosystem and supports Python and Node.js agents. Pipecat fits teams that want to shape the pipeline directly. LiveKit fits teams whose product is already organized around rooms, web or mobile clients, SIP, video, or the LiveKit deployment stack.
Do I need an open-source voice agent framework?
Open source is valuable when source access, self-hosting, low-level media changes, or independent infrastructure ownership are hard requirements. It also transfers deployment, upgrades, incident response, and integration behavior to your team unless a managed cloud covers those layers. A managed runtime fits when product control matters more than owning the real-time internals.
What should I build outside the framework?
Keep domain policy, business data, authorization, idempotency, compliance logic, and outcome measurement in systems your product controls. Framework-specific code should translate those stable contracts into sessions, tools, media, and events. This boundary makes provider changes and incident recovery less disruptive.
If a managed runtime matches that ownership boundary, start building with Dasha using one real call path, one business tool, and one measurable outcome.