Voice AI agents that listen, answer and hand over gracefully
Phone and in-app voice agents built on a streaming speech-to-text, LLM and text-to-speech pipeline with an explicit latency budget, interruption handling and warm transfer to people.
A voice turn has three stages: hear, think, speak. Run them one after another and the silence is long enough that callers assume the line is dead. Run them as overlapping streams and the first words come back while the model is still thinking.
The same turn, built two ways. Bar lengths are illustrative; the point is the overlap. The one-second figure is a design goal we engineer towards and then measure per deployment.
Use cases
Where voice agents fit, and what each needs
Use case
Fit
Main risk
Integration needed
Inbound support line
Strong for status, FAQs, simple changes
Over-confident answers on account matters; must verify identity first
Telephony, CRM or helpdesk, knowledge base, warm transfer
Outbound reminders and confirmations
Strong; short, scripted, verifiable
Calling at the wrong time or the wrong person; consent rules
Scheduling or booking system, calendar, SMS fallback
In-app voice assistant
Strong where hands-free matters
Noisy environments; privacy of on-device audio
Mobile app, app back-end, optional on-device speech; see AI for mobile apps
IVR replacement
Good; natural language beats "press 2"
Routing errors; callers who want a person being delayed
Telephony routing, queue system, agent desktop
Clinical or financial advice
Only with strict scope and human review
Regulatory exposure; hallucinated confirmations
Compliance sign-off, recording and audit, human escalation by default
Sector specifics differ. See healthcare for consent and recording constraints and travel and hospitality for booking-heavy voice flows.
Who this is for
Teams with a phone line that is too busy, or a product that needs a voice
Voice agents pay off where call volume is high and most calls are predictable: order status, appointment changes, balance checks, opening hours, simple bookings. They also fit products used hands-free, in a vehicle, on a site or in a kitchen. They are a poor fit where every call is unique or emotionally charged; those calls should reach a person quickly, and the agent's job is to route them well.
Every deployment reports the same measures, so you can judge it on evidence rather than demos:
Latency per stage and for the whole turn, as percentiles, per call
Interruption and talk-over rate
Transfer rate and the reason for each transfer
Task completion without a human, verified against the back-end system
Cost per minute and per completed task
What we deliver
The whole pipeline, measured
Hearing
Streaming speech recognition with turn detection tuned to your callers, custom vocabulary for product names, and noise handling. Built on our speech-to-text work.
Thinking
A grounded, tool-using conversation core shared with our chatbot builds, with short spoken-style answers, confirmation before actions and clear transfer triggers.
Speaking and measuring
Streaming text-to-speech in your chosen voice, barge-in handling, and per-call metrics for latency, transfer rate, task completion and cost.
Process
Six to ten weeks, with real calls from week three
Design the conversation
Call flows, verification rules, transfer triggers, consent wording, and the eval set of recorded or scripted calls.
Build the pipeline
Telephony, streaming STT, conversation core, TTS, and the latency instrumentation for every stage.
Test with people
Internal callers, then friendly customers. Accents, noise, interruptions, silence and angry callers are all in the test plan.
Launch and tune
A share of traffic first, with humans listening. Thresholds and prompts tuned on real calls, cost and latency reported weekly.
What goes wrong
Four failures account for most bad voice agents: talking over the caller because turn detection is naive, confirming an action that was never performed, refusing to transfer when asked, and recording calls without the right consent. All four are design decisions, which is why they are in our week-one checklist rather than our incident log.
Fast enough that the caller does not start talking again. Our design goal is a first audible reply within roughly a second of the caller finishing, achieved by streaming every stage rather than waiting for each to complete. We publish the measured figures for your deployment.
Can the voice agent hand over to a person?
Yes, and it must. We build warm transfer: the agent summarises the call aloud to the human and attaches the transcript, so the caller never repeats themselves. Transfer triggers include low confidence, sensitive topics and a direct request.
Does it work with our existing phone system?
Usually. We connect through SIP or WebRTC to most modern telephony platforms and contact-centre systems, and can run entirely in-app for mobile. We confirm the integration path during scoping.
What about recording consent and data protection?
Consent announcements, recording policies, retention rules and redaction of card numbers or health details are designed in from the start with our AI data privacy practice. Requirements differ by country and sector; we map yours before build.
How long does a voice agent take to build?
A single-purpose agent such as appointment confirmations or an order-status line typically takes six to ten weeks to production. Inbound support agents with account actions and transfers run longer, mostly on integration and testing.
Talk to us about your AI project
Tell us what you are trying to do, which systems are involved and what is in the way. We reply within one business day with an opinion and a suggested next step.