A conversational AI demo can look finished after one good exchange. Production exposes the harder work: ambiguous requests, stale knowledge, failed tools, interruptions, permissions, and conversations that wander off the happy path. A useful development process designs for those conditions from the start. Here is a practical architecture, build sequence, and test plan for taking a text or voice agent from idea to controlled release.
What conversational AI development includes
Conversational AI development is the design, implementation, testing, and operation of software that understands natural-language input and responds through text or voice. Modern systems can answer questions, collect structured information, retrieve private data, and take actions in other applications.
The model is one component. A production system also needs explicit state, approved knowledge, constrained tools, application-level policy, observability, and a human path for cases the software should not resolve.
For technical teams building real-time voice products, Dasha provides the managed runtime and operations layer through REST APIs and a web application. We handle telephony, integrations, testing, monitoring, and call execution. Your team still owns the workflow, permissions, data policy, and definition of success.
A production conversational AI architecture
Separate the system into layers with clear contracts. This makes a failure traceable. It also lets you change a model, speech provider, or business integration without rewriting the whole application.
| Layer | Responsibility | Failure to design for |
|---|---|---|
| Channel and transport | Accept web, app, messaging, or phone input and return the response | Reconnects, duplicate events, poor audio, channel limits |
| Turn and session manager | Decide when a turn starts and ends; associate it with the right session | Talk-over, lost context, abandoned sessions |
| Conversation state | Store the task, verified fields, consent, permissions, and progress | The agent asks twice or acts on an old value |
| Orchestrator | Choose whether to respond, retrieve, call a tool, clarify, refuse, or escalate | Loops, conflicting actions, uncontrolled autonomy |
| Language model | Interpret varied language and compose a response | Unsupported claims, malformed tool arguments |
| Knowledge retrieval | Supply approved, relevant passages at request time | Stale or irrelevant grounding, empty retrieval |
| Tool gateway | Read or change data through narrow, typed operations | Timeouts, duplicate writes, unauthorized access |
| Policy and safety | Enforce hard rules outside the model | Prompt injection, data exposure, unsafe actions |
| Telemetry and evaluation | Record traces, measure outcomes, and catch regressions | Failures that cannot be reproduced or compared |
The contracts between layers matter more than the number of services. A small system can keep several layers in one process. It should still distinguish transcript context from authoritative state and a model suggestion from an approved action.
Give each decision to the right owner
Language models are good at interpreting phrasing and producing language. They should not become the authority for permissions, pricing rules, account balances, or irreversible changes.
| Decision | Owner | Example |
|---|---|---|
| What the user appears to want | Model, constrained to allowed routes | Map “move my visit to Friday” to rescheduling |
| Which current policy passage applies | Retriever plus model | Find the cancellation window and explain it |
| Whether the user may access a record | Application code | Check that an appointment belongs to the signed-in user |
| Whether an action is allowed | Business service | Reject a change after the cutoff |
| Whether a write can be retried | Tool implementation | Use an idempotency key for a reschedule request |
| Whether to approve an exception | Human | Waive a fee outside standard policy |
This division keeps the conversation flexible while making consequential behavior predictable.
How to build conversational AI in eight steps
1. Define one bounded job
Start with an outcome that has a clear beginning and end. “Answer product questions” is vague. “Answer questions from the current installation guide and create a support ticket when no supported answer exists” is testable.
Write three lists before selecting a model:
- tasks the agent may complete;
- requests it must refuse or hand off; and
- the source of truth for every fact and action.
Choose outcome metrics at the same time. Task completion, correct escalation, unsupported-answer rate, tool success, and response time are more useful than counting messages.
Exit criterion: a reviewer can label any representative request as allowed, disallowed, or escalation-required.
2. Map the conversation and its recovery paths
Describe the happy path, then spend more time on recovery. Users provide incomplete details, correct themselves, switch topics, stay silent, and ask for a person.
For each step, record:
- required information and where it comes from;
- which values need explicit confirmation;
- the next valid routes;
- timeout and retry behavior; and
- what context a human receives on transfer.
Avoid one giant conversation tree. Model the task as reusable capabilities such as authenticate, look up availability, confirm a slot, and transfer. The orchestrator can compose them while application code preserves their rules.
Exit criterion: every external dependency and every write action has a defined failure path.
3. Store authoritative state outside the transcript
A transcript is useful context. It is a poor database. Store critical fields in typed application state, including user identity, current task, verified entities, tool results, confirmation status, and handoff status.
Summaries can reduce model context, but a summary should never replace the values used to authorize or execute an action. Expire temporary state deliberately and keep retention aligned with the data the workflow actually needs.
Exit criterion: the application can resume or inspect a task without asking the model to reconstruct the truth from prose.
4. Add knowledge and tools for different kinds of truth
Use retrieval for explanatory content such as policies, manuals, and support procedures. The original RAG research describes combining model knowledge with an external index that can be queried at generation time. In an application, this lets owned content change without retraining the model.
Use tools for live, private, or transactional data. An appointment policy belongs in retrieval. Available appointment slots belong behind an authenticated tool.
Tool contracts should be narrow:
- Accept a typed request with allowed values.
- Authenticate and authorize in application code.
- Apply business validation.
- Use timeouts and an idempotency key where retries could duplicate a write.
- Return a structured success or named error.
- Record the request, result, and correlation ID.
Test retrieval separately from generation. If the wrong passage was selected, rewriting the response prompt only hides the real defect.
Exit criterion: every factual response identifies a source, and every action has a validated tool path.
5. Build policy around the model
Prompts express desired behavior. Enforcement belongs in code and infrastructure. Treat user input, retrieved documents, model output, and tool arguments as untrusted until the appropriate layer validates them.
This is especially important when the system can act. OWASP's prompt-injection guidance covers direct instructions from users and indirect instructions hidden in files or webpages. Retrieval and fine-tuning do not remove that risk.
Apply least-privilege credentials, allowlisted tools, input and output validation, explicit confirmation for costly or destructive operations, and a human approval step for exceptions. Minimize sensitive data in prompts and logs. The NIST GenAI Profile is a useful lifecycle framework for connecting these controls to broader risk management.
Exit criterion: a model response alone cannot authorize access or execute a privileged operation.
6. Design channel behavior
Keep business capability shared across channels, but adapt the interaction.
| Text | Voice |
|---|---|
| Users can scan longer answers | Keep each spoken turn short |
| Buttons and links can constrain choices | Confirm choices in language |
| Pauses are often harmless | Silence changes turn-taking behavior |
| The full message usually arrives at once | Partial transcripts may change while the user speaks |
| Users can reread prior context | Repeat only the detail needed for the next decision |
For voice, do not trigger an irreversible action from a partial transcript. Decide how the agent responds to interruption, silence, background speech, speech-recognition corrections, and failed audio delivery before release.
Exit criterion: the same capability works through each intended channel without copying business policy into channel code.
7. Evaluate complete conversations
Unit tests still matter for schemas, permissions, and integrations. Agent quality needs multi-turn tests that inspect the full trace.
Create a versioned evaluation set from real task shapes. Include normal cases, ambiguous requests, changed intent, unsupported questions, tool timeouts, permission failures, prompt injection, long conversations, and human handoffs. For voice, add interruptions, long pauses, digits, names, accents represented in the intended audience, and realistic noise.
Score observable behavior:
- outcome: was the task completed correctly?
- grounding: did the answer stay within approved evidence?
- tool integrity: was the right tool called with valid arguments?
- policy: did the system protect data and respect action boundaries?
- recovery: did it exit loops and provide the next valid step?
- handoff: did the person receive verified context?
- performance: where did time accrue across input, model, tools, and output?
Run the same set after changes to models, prompts, retrieval, tools, voices, or orchestration. A general model benchmark cannot tell you whether your rescheduling workflow still works.
Exit criterion: every release has a repeatable comparison against the last accepted version.
8. Release gradually and operate the system
Start with one workflow and a limited traffic slice. Trace each turn across retrieval, model calls, tools, state changes, and channel output. Measure latency at the component and end-to-end levels, using percentiles rather than a single average.
Review complete failed conversations. Classify the responsible layer, fix it there, and add the case to the regression set. Monitor business outcomes alongside technical health so a faster response does not mask a lower completion rate.
Exit criterion: the team can find a failed session, explain what happened, reproduce it, and confirm the fix before wider rollout.
Voice development adds a real-time control problem
Text and voice can share knowledge, tools, and business logic. Voice adds a streaming system where several components operate at once. Speech recognition produces partial input, the model may stream tokens, speech synthesis starts before the full response is ready, and the caller may interrupt at any point.
Four details usually decide whether the result feels usable:
- Endpointing: detect the end of a turn without cutting off a pause or waiting so long that the conversation stalls.
- Barge-in: stop queued speech when the caller interrupts, preserve the right state, and process the new request once.
- Latency budget: trace speech input, model work, tools, speech output, and network time separately. There is no useful universal target for every workflow. A lookup with a slow system of record has a different budget from a greeting.
- Speech accuracy at business boundaries: confirm names, dates, addresses, amounts, and identifiers before using them in a write operation.
Our guides to the voice AI stack, voice latency, and voice agent testing go deeper into these runtime concerns.
Choose a stack by the operations you want to own
The framework decision is also an operating-model decision. Compare who will own the runtime after the prototype works.
| Approach | Best fit | Your team owns | Main tradeoff |
|---|---|---|---|
| Dasha managed runtime and operations | Technical teams shipping a real-time voice product | Product workflow, integrations, permissions, data policy, evaluation | Dependence on a managed platform |
| Hosted component APIs | Teams differentiating through custom orchestration | Transport, state, provider coordination, tracing, failure recovery | More vendor seams and on-call surface |
| Open-source framework | Teams that need code-level control and can run the stack | Deployment, scaling, upgrades, security, observability, integrations | Higher infrastructure and maintenance load |
| Fully custom runtime | Teams whose runtime is core intellectual property or has unusual deployment constraints | Every layer and every production failure | Longest path to a reliable release |
Select with a real vertical slice: one channel, one workflow, one knowledge source, one read tool, one write tool, and one handoff. Measure the team effort to debug and operate it, not just the time needed to get the first response.
A minimum production test plan
| Test | Scenario | Passing behavior |
|---|---|---|
| Scope | User requests an unsupported action | Declines clearly and offers a valid route |
| Grounding | Knowledge has no strong match | Does not invent an answer; clarifies or escalates |
| Authorization | User asks for another person's record | Tool gateway denies access without leaking data |
| Retry safety | Write tool times out after execution | Retry does not create a duplicate action |
| Correction | User changes a confirmed date | State updates once and the final confirmation is correct |
| Topic switch | User interrupts one task with another | Pauses, resolves, resumes, or escalates by policy |
| Prompt injection | Retrieved content contains instructions | Treats the content as data and preserves system boundaries |
| Dependency failure | Model, retrieval, or business API is unavailable | Stops the affected path and gives the next valid step |
| Handoff | User asks for a person | Transfers verified fields, relevant trace, and conversation context |
| Voice interruption | User speaks over generated audio | Stops playback and handles the new turn without duplication |
Add workflow-specific cases before release. A payments agent, a scheduling agent, and an internal knowledge assistant need different acceptance thresholds and approval paths.
Frequently asked questions
What is the difference between a chatbot and conversational AI?
A chatbot is a user interface for exchanging messages. Conversational AI is the wider system that interprets language, maintains state, retrieves knowledge, calls tools, applies policy, and produces the response. A chatbot can use simple rules or a full conversational AI stack.
Does every conversational AI system need RAG?
No. Retrieval-augmented generation is useful when the agent answers from a meaningful body of changing documents. A narrow transactional agent may rely mainly on structured state and authenticated tools. Adding retrieval without a clear knowledge need creates another failure mode to test.
Should you fine-tune a model for conversational AI?
Start with a capable base model, explicit instructions, good context, retrieval, and tool contracts. Fine-tuning is useful when a stable, well-measured behavior gap remains and you have representative data. It does not replace permissions, current knowledge, tool validation, or system-level evaluation.
Which programming language is best for conversational AI development?
Use the language your team can operate reliably and that fits the platform or framework you choose. The more important requirements are typed tool contracts, asynchronous I/O for streaming and external calls, reliable state storage, and mature telemetry. Architecture and tests matter more than language preference.
Build the first production slice
The shortest route to production is a narrow workflow with explicit state, constrained actions, a real evaluation set, and a trace your team can debug. Expand only after that slice survives failures and real user variation.
If you are building a real-time voice product and want the runtime plus operations managed, start a Dasha evaluation with one real workflow, one integration, and the test plan above.