AI Voice Agent Development Services in India
We build phone agents that answer, listen, interrupt politely, and hand off to a person when they should. Speech recognition, telephony, sub-second turn latency and the compliance work around call recording, delivered by an engineering team in India for companies in the US, UK, Canada, Australia and New Zealand.
What Is an AI Voice Agent, and Where Does It Actually Work?
An AI voice agent is software that holds a spoken conversation over a phone line or a browser session. It transcribes the caller as they speak, works out what they want, does something useful about it, and answers in a synthetic voice. AI voice agent development services in India exist because the engineering behind that sentence is unglamorous, wide, and needs a team rather than one clever prompt: real-time audio transport, a recogniser tuned to your callers, a model that stays inside your business rules, a synthesiser that starts speaking fast, and about forty edge cases that only appear on a real phone network.
The demos are easy. Anyone can wire an off-the-shelf platform to a phone number in an afternoon and have a pleasant conversation with it in a quiet room. That is not the job. The job is the caller on a motorway with the window down, the one who says "yeah no sorry, the other one", the one who gives a postcode as "double-oh eight", the one who starts talking two words into your greeting, and the one whose call must reach a human in under thirty seconds because the reason they rang is an emergency.
Where voice agents earn their keep today is the narrow, high-volume, structured call. Booking and rescheduling appointments. Order and delivery status. Qualifying an inbound enquiry before it reaches a salesperson. Taking a first-notification-of-loss. Confirming attendance. Chasing an overdue form. Answering the twenty questions that make up most of your after-hours volume. Where they still struggle is open-ended advice, distressed callers, anything with a long tail of exceptions, and any conversation where being wrong is expensive and hard to detect.
We say that plainly at the start of every engagement, because the fastest way to waste six months is to point a voice agent at the hardest call type in the business. Pick the call you take four hundred times a week that has three outcomes, and automate that properly. The rest can follow once you trust the instrumentation.
What Breaks on Your Phone Line Today
Most companies who come to us for voice work do not open with a technology problem. They open with a number off their own dashboard. The share of calls abandoned before anyone picked up last quarter. A booking line that goes to voicemail from six in the evening, in a business where the customer books whoever answers first. A support queue where the first ninety seconds of every call is a person reading out an order number that the system already has.
The old fix was an IVR tree, and everyone hates it for a specific reason: it makes the caller translate their problem into your org chart. "Press four for accounts" only works if the caller knows they have an accounts problem. Deeper menus reduce the number of misrouted calls slightly and increase abandonment considerably, which is why so many trees end with a shortcut everyone in the office privately uses to skip them.
The second failed fix was hiring more people for the peak. Peaks are peaks. You staff for the Monday morning after a bank holiday and pay for that headcount on a wet Thursday in February. Contact centre attrition is brutal, and the roles you are staffing are exactly the repetitive ones that people leave first, so you are re-running induction training on the same twenty scripted calls forever.
The third failed fix is the one we spend most of our time rescuing. A team buys a no-code voice platform, builds a flow, launches it, and discovers a month later that containment is nowhere near the demo, callers are asking for a human on the second turn, and nobody can say why because the platform reports minutes used and not much else. The failure is almost never the model. It is a 900 millisecond gap after the caller stops speaking, an inability to interrupt the agent mid-sentence, a recogniser that turns "Featherstonehaugh" into three different surnames, or a transfer that dumps the caller into the queue they were trying to avoid.
What that costs you is measurable and worth measuring before you build anything. Abandoned calls that were going to buy. Callers who hang up and go to the competitor whose line was answered. Staff time spent on calls that never needed a person. Out-of-hours revenue you simply do not capture. Get those four numbers on paper first, because they decide whether the project is worth doing and they become the scoreboard afterwards.
What an AI Voice Agent Build Includes
Scope on voice projects goes wrong when it is written as "build a voice bot". Here is what we actually deliver, and what we consider the build finished.
Call flow design and slot definition
Before any code, we write out the conversation as a set of goals and the pieces of information each goal needs. A rescheduling call needs an identified booking, a proposed new time, and a confirmation. Each of those is a slot with a type, a validation rule, a confirmation policy, and a defined behaviour when the caller will not or cannot supply it. This document is short and it is the thing that gets argued over, which is the point. Most disputes about voice agents are really disputes about business rules that were never written down.
The audio path
Phone number provisioning or SIP trunk connection into your existing carrier, media streaming into the application, echo handling, and the codec decisions that follow from the caller's network. We build this so it can be pointed at a test number and a production number without a code change, because you will want to run both in parallel for weeks.
Recognition tuning
Engine selection benchmarked on your recordings, custom vocabulary for product names, drug names, part numbers, place names and anything else the general model has never seen, and a formatting layer that turns spoken digits into structured values correctly. "Oh" is zero, "double four" is two fours, and a British caller saying a sort code does not pause where an API expects a delimiter.
Reasoning, tools and guardrails
The model layer that decides what to say next, connected to your systems through explicit tool calls rather than being asked to imagine an answer. Booking lookups, availability checks, order status, CRM writes. Plus the guardrails: what the agent may never say, what it must never promise, which topics end the call, and which trigger an immediate transfer.
Voice and speech synthesis
Voice selection, pronunciation dictionaries for the words your business says all day, and text normalisation so that "£1,240.50" and "2026-03-04" and "Dr Ngo" come out of the speaker as a human would say them. This sounds trivial and it is the single most common reason a pilot sounds cheap.
Turn-taking behaviour
Barge-in, endpointing, backchannels, filler handling and interruption recovery, tuned against real calls rather than left at vendor defaults. This is where most of the perceived quality of a voice agent lives, and it gets its own section further down because it deserves one.
Escalation and human handoff
The rules that trigger a transfer, the warm transfer mechanism itself, the context package the human receives, and the fallback if no human is available. A voice agent without a good exit is worse than no voice agent.
Observability
Per-call records with the audio, the transcript, the tool calls, the latency of each stage, the recogniser confidence per slot, and the outcome. A dashboard your operations team can read without us. Alerting on the things that matter: transfer rate climbing, endpoint latency drifting, recogniser confidence falling after a vendor model update.
Compliance implementation
Disclosure, consent capture, recording controls including pause and resume around card data, retention and deletion, redaction in transcripts and logs, and the access model for who in your business can listen to what.
Definition of done
We consider a voice agent finished when it meets agreed thresholds on a held-out set of real calls: task completion rate, transfer rate, median and 95th percentile response latency, barge-in responsiveness, and zero occurrences of a defined list of unacceptable behaviours. Plus a runbook your on-call engineer can follow at three in the morning, and a documented rollback to the previous prompt and model versions. Not "it works in the demo".
The Voice Stack: ASR, TTS, Reasoning and Telephony
There is no single right stack. There are four or five decisions, each with a real trade-off, and the correct combination depends on your callers, your regulator and your latency budget. Here is how we think about each one, including when we would not choose a given option.
Speech recognition
Deepgram is our usual starting point for streaming English. It was built for real-time from the outset, its interim results arrive fast enough to drive endpointing decisions, and its endpointing and keyword boosting are exposed as parameters rather than hidden. It is strong on North American and British English and less impressive on some Indian, Nigerian and Filipino accents, which matters if your callers or your own offshore staff are on the line.
AssemblyAI competes closely on streaming accuracy and has invested heavily in formatting and entity detection, which saves you post-processing work on names, addresses and amounts. Worth benchmarking head to head with Deepgram on your own audio rather than trusting either vendor's published numbers, which are always measured on clean data you will never see.
Azure Speech is the one we reach for when procurement is involved. Custom Speech lets you fine-tune on your own audio and text, phrase lists handle domain vocabulary, and the regional deployment and BAA options are what health and financial customers ask about first. The trade-off is more configuration surface and slightly heavier integration than the developer-first vendors.
Google Cloud Speech-to-Text with the Chirp family is strong multilingually and a natural fit if the rest of your estate is on Google Cloud. Latency is competitive though we have found tuning endpointing behaviour less direct than with Deepgram.
Whisper deserves a careful answer because it gets proposed constantly. It is excellent, it is open, it handles many languages, and it can run in your own environment which solves several data problems at once. It is also not a streaming model. Running it live means slicing audio with a voice activity detector, transcribing chunks, and stitching the result, which adds latency and creates seams exactly where a caller pauses mid-sentence. It also hallucinates on silence and on music, producing confident transcriptions of nothing, which is a genuinely dangerous failure mode on a phone line. Where we use it, it is usually faster-whisper or whisper.cpp on a GPU we control, for a use case with a real data-residency constraint, and we accept the latency cost knowingly.
The measurement that settles these arguments is word error rate on your own recordings, segmented by accent, by handset type, and by whether the caller is in a car. We run that benchmark in week one. It has changed the vendor choice on more than half the voice projects we have scoped.
Speech synthesis
ElevenLabs produces the most natural output most of the time, and its low-latency models are fast enough for conversational use. It is the expensive option, and if your agent handles high call volume the synthesis line on your bill will be visible. Voice cloning is available and is a legal question as much as a technical one, particularly if you are cloning a real employee.
Cartesia is built around time-to-first-byte, and on a stack where you are fighting for every hundred milliseconds that focus shows. When the latency budget is the binding constraint, it is usually on the shortlist.
Azure Neural TTS and Amazon Polly are cheaper, entirely adequate for informational content, and noticeably flatter on anything emotive. For an order-status line nobody minds. For a bereavement services line it is the wrong choice and we would say so.
Whichever engine you pick, the work that decides how it sounds is not the engine. It is streaming synthesis so the first sentence starts playing while the rest is still generating, sentence-level chunking so a long answer does not block, a pronunciation lexicon for your brand and product names, and text normalisation for numbers, dates, currency and abbreviations. We have heard a perfectly good voice model read a UK phone number as a single enormous integer because nobody wrote the normaliser.
The reasoning layer, and the speech-to-speech question
The classic pipeline is recognise, reason, synthesise. Each stage is separately observable, separately swappable, and separately cheap to fix. You can log exactly what was heard, exactly what was decided and exactly what was said, which is what makes a voice agent debuggable and auditable.
Native speech-to-speech models, including the realtime APIs from OpenAI and Google, collapse those stages. They are faster, they preserve tone and prosody, and they can react to how something was said rather than only to the words. They also give you a much weaker seam for instrumentation, tighter vendor coupling, and less deterministic control over exactly what leaves the speaker. For a regulated call where a compliance team needs the model's decision recorded, the pipeline still wins. For a low-stakes, latency-critical conversational experience, speech-to-speech is genuinely better and we will recommend it. We have built both and we do not think either is the future to the exclusion of the other.
Model choice inside the pipeline is a smaller decision than people expect. A mid-sized model with tight prompting, good tools and strict guardrails beats a frontier model with a vague prompt, and it responds faster. Time to first token matters more than benchmark scores, because the caller is listening to silence while you wait for it.
Telephony and transport
Twilio is the default for a reason: numbers in most countries, Media Streams for bidirectional audio over a WebSocket, elastic SIP trunking when you want to keep your existing carrier, and documentation that does not lie to you. The audio you get on the PSTN side is 8 kHz G.711, which is narrowband, and no recogniser will ever recover the high frequencies your microphone never captured.
Vonage, Plivo and Telnyx are viable alternatives with different pricing structures and regional strengths. LiveKit is the one we use when the same agent has to work over both a phone line and a browser or mobile app, because it handles WebRTC properly and its agent framework was designed for this problem rather than adapted to it.
If you have an existing contact centre platform, connecting by SIP to what you already own is usually better than replacing it. Your carrier contracts, your numbers, your routing rules and your reporting all stay where they are, and the agent becomes one more destination in the routing plan rather than a parallel universe.
Orchestration: build or buy
Managed platforms such as Vapi, Retell and Bland will get you to a working agent in days. That is real value and we will tell you to use one when the requirement is simple, the volume is modest, and nobody needs to inspect the internals. What you give up is control of the turn-taking logic, portability between vendors, depth of observability, and the ability to fix a specific latency problem in a specific stage.
Pipecat and LiveKit Agents are the open frameworks we build on when the requirement is not simple. Both give you a pipeline you own, per-stage metrics, and freedom to swap a recogniser without rewriting the agent. The cost is that you are now responsible for the operational side, which is exactly the part that a managed platform hides and exactly the part that determines whether the thing works at two in the morning.
Our honest rule of thumb: if the agent handles fewer than a few thousand calls a month, has no regulatory exposure and no deep system integration, buy. If it touches money, health, identity or a contact centre you already run, build on an open framework and keep the seams visible.
Why Is Latency the Whole Game in Voice?
In text chat, a two second wait is fine. The user watches a typing indicator and knows something is happening. On a phone call, two seconds of silence is a broken line. People say "hello?", they repeat themselves, they start a new sentence, and now your agent is answering a question the caller has already abandoned.
The target we design to is under one second from the caller's last syllable to the first audible syllable of the reply, with a stretch goal closer to 700 milliseconds on simple turns. That is not one component being fast. It is every component being fast at once, and the budget divides up roughly like this.
Where the milliseconds go
Endpointing wait is usually the largest single item and the one teams forget. If you wait 700 milliseconds of silence before deciding the caller has finished, you have spent 700 milliseconds before any processing starts. Drop it to 300 and you will cut people off mid-thought when they pause to remember a house number.
Recognition finalisation adds tens to low hundreds of milliseconds after the endpoint, depending on engine and whether you can act on interim results. Model time to first token is typically the second largest item and varies enormously with prompt length, which is why we keep system prompts tight and push knowledge into retrieval and tools rather than into the context window.
Speech synthesis time to first byte is where a slow vendor hurts most, because it sits directly between the decision and the sound. Network transit adds the rest, and it is not small: audio crossing an ocean twice on every turn is a self-inflicted wound.
The tricks that actually work
Stream everything. Interim transcripts feed the model's speculative work, the model streams tokens, synthesis starts on the first complete clause, and audio starts playing before the sentence is finished. A pipeline that waits for each stage to complete before starting the next is roughly twice as slow as one that does not, and the difference is audible.
Speculate on likely turns. When the caller is midway through an answer you can often predict the tool call and issue it early, discarding the result if you were wrong. Availability lookups and account fetches are cheap to run speculatively and expensive to wait for.
Use a short acknowledgement to buy time honestly. A quick "let me check that" while a slow lookup runs is what a human does and it is not a trick. What is a trick, and what we avoid, is a filler phrase on every single turn, because callers notice the pattern within four exchanges.
Colocate. The media server, the recogniser, the model endpoint and the synthesiser should sit in the same cloud region, and that region should be near your callers. This is the single most common architectural mistake we find in half-built voice projects, and it is the reason our own delivery model separates where the team sits from where the system runs.
Measure the tail, not the average
Median latency flatters every voice agent. The calls that go wrong are in the 95th percentile, and they are usually caused by one slow tool call, one cold start, or one retry against a rate-limited API. We instrument every stage per turn and alert on the tail. A p50 of 800 milliseconds with a p95 of four seconds is a bad agent that looks good on a slide.
Turn-Taking, Barge-In and End-of-Speech Detection
Ask ten people why a voice agent felt robotic and none of them will say "the endpointing threshold was too aggressive". They will say it kept interrupting, or it took ages, or it talked over them. All three are turn-taking, and it is the part of the build that separates a system people tolerate from one they do not think about.
Voice activity detection is not enough
The naive approach watches audio energy and declares the turn over after a fixed silence. Silero VAD and the WebRTC VAD both do this job well at the acoustic level, and both are blind to meaning. A caller saying "my account number is four, seven, two..." pauses exactly like a caller who has finished. Set the threshold short and you cut them off. Set it long and every turn drags.
Semantic endpointing
The better approach adds a second signal: does the transcript so far look like a finished thought? A small classifier over the interim transcript can distinguish "I want to book an appointment" from "I want to book an appointment for", and adjust the silence threshold accordingly. LiveKit ships a turn detection model for exactly this, and it is straightforward to train your own on your call transcripts once you have a few thousand.
Context helps further. When the agent has just asked for a sixteen digit card number, the endpointing threshold should be generous and the agent should also count digits. When it has asked a yes or no question, it should be aggressive. Slot-aware endpointing is one of the cheapest quality wins available and almost nobody implements it.
Barge-in
The caller must be able to interrupt. If they cannot, the agent is an IVR with a nicer voice. Barge-in means detecting speech while the agent is talking, stopping playback within about 200 to 300 milliseconds, flushing whatever audio is already buffered downstream, and cancelling the synthesis stream so you are not paying to generate a sentence nobody will hear.
The hard part is the false positive. The agent's own voice echoing back through a cheap speakerphone will trigger barge-in unless echo cancellation is right. A television in the background will do the same. So will a caller saying "mm-hmm" to show they are listening, which is a backchannel and not an interruption. We handle backchannels with a short list of acknowledgement tokens that do not stop playback, and we tune the false-interrupt rate as an explicit metric.
Recovery matters as much as detection. When the agent is interrupted it must know how much of its message actually reached the caller, because "your appointment is confirmed for Tuesday the" followed by silence is a support ticket. We track playback position and resume or restate accordingly.
The awkward middle cases
Double-talk, where both parties speak at once and neither yields, needs an explicit rule about who backs off, and it should be the agent. Long thinking pauses need an acknowledgement rather than dead air. Callers who go quiet entirely need a gentle prompt after a defined interval and a defined number of prompts before the agent hangs up, otherwise you will pay for hours of open line to an empty room. And a caller who says nothing but breathes into the handset needs to be distinguished from silence, which is a VAD sensitivity question and a genuinely annoying one.
Accents, Noise and the Audio You Will Actually Get
Every speech vendor publishes accuracy figures. Every one of those figures was measured on clean, close-microphone, mostly American read speech. Your callers are on a phone.
Start from what the network gives you. A PSTN call arrives as 8 kHz narrowband audio, usually G.711, sometimes worse if it has crossed a poor mobile leg. Everything above about 3.4 kHz is gone before your recogniser ever sees it, which is precisely where the acoustic difference between "s" and "f", or between the letters B, P, V, D, E and T, lives. A WebRTC call in a browser gives you wideband Opus and a materially easier problem. If you are building for both, measure both, because a model tuned on your app calls will underperform on the phone line.
Then there is who is speaking. Recognition accuracy varies substantially across accents and dialects, and the variation is not symmetric: models trained predominantly on American and British English do worse on Scottish, Irish, South Asian, West African and Caribbean English, and worse again on speakers with a heavy regional accent speaking quickly. If your customer base includes those speakers, and in the UK, Canada and Australia it certainly does, this is not an academic concern. It is the difference between a system that serves your customers and one that quietly serves the easiest half of them.
What we do about it
We benchmark on your recordings first, segmented by whatever accent groups your call data actually contains, and we report the per-segment word error rate rather than one headline number. Where an engine is weak on a group that matters to you, that engine is out, however good it looks elsewhere.
We then reduce the guessing. Custom vocabulary and phrase lists for product names, clinic names, street names in your service area and any jargon your callers use. Constrained expectations per slot, so when the agent has asked for a UK postcode it is validating against a postcode format and an address lookup rather than accepting free text. Readback confirmation on anything that would be expensive to get wrong, which is not every field, because confirming everything is exhausting and callers start saying yes reflexively.
And we build the escape hatch. Two consecutive low-confidence recognitions on the same slot should switch modality: keypad entry for numbers, an SMS with a link for addresses and email, or a transfer. Confidence scores are imperfect but they are far better than the alternative, which is an agent that cheerfully books the wrong appointment.
Noise deserves its own treatment. Cars, wind, building sites, open-plan offices, hold music from a system the caller has just escaped, and other people talking nearby. Noise suppression on the inbound stream helps and can also strip speech if it is too aggressive. The pragmatic answer is to detect a persistently poor signal and say something about it, because a human agent would.
How Does Voice Fail Differently From Text Chat?
If you have already shipped a web chatbot, you have useful instincts and several misleading ones. These are the differences that catch teams out, and they are the reason we treat voice as its own discipline rather than chat with a microphone attached.
There is no scrollback
A chat user can re-read the last four messages. A caller cannot. Anything the agent says must be understood the first time, in order, at speaking pace. That rules out long lists, nested options, and any answer that requires holding three facts in your head. Five options is too many; three is fine. If you need to convey a list, send it by SMS or email during the call and say you have done so.
The user cannot correct a typo
In chat, a misunderstanding is visible in the transcript and the user fixes it. In voice, a recognition error is silent. The agent heard "sixteen" instead of "sixty", believed it, and acted on it. The caller has no idea until something goes wrong days later. This is the single most important structural difference, and it is why confirmation strategy, confidence thresholds and readback design get more attention on a voice project than anything else.
Silence is a failure state
Chat tolerates thinking time. Voice does not. Every slow tool call, every retry, every cold start is dead air that the caller interprets as a dropped line. Your timeout strategy on downstream systems has to be built around what the caller will tolerate, not around what the API usually does.
Users leave without complaining
A frustrated chat user rage-types and you see it. A frustrated caller hangs up, and unless you are instrumenting for it, that call looks like any other short call. Hang-up point analysis, per-turn, is the first dashboard we build. Clusters of hang-ups at the same turn tell you exactly which sentence is broken.
The clock is always running
Chat costs you per message and mostly per token. Voice costs you per second of recognition, per character of synthesis, per token of inference and per minute of telephony, all at once, on every call, including the sixty seconds a caller spends on hold while your CRM thinks. Slow code has a direct line item. That focuses the mind pleasantly.
Regulatory exposure is higher
A web chat sits behind a privacy policy the user clicked past. A phone call is a recording of a person's voice, which is biometric-adjacent data in several jurisdictions, often subject to specific consent law, and increasingly subject to disclosure rules about the fact that the other party is a machine. Voice raises the compliance floor and it does so before launch, not after.
The failure is public and it is audible
A broken chatbot produces a screenshot. A broken voice agent produces a recording of your company sounding foolish, and those get shared. We build a hard kill switch that routes all traffic back to your existing queue in one action, and we test it before launch, because the day you need it you will not want to be reading documentation.
Handing the Call to a Human Without Losing the Caller
The transfer is the most important thing your voice agent does, because it is the moment a caller decides whether the automation helped or wasted their time. Get it wrong and every good turn before it is cancelled out.
When to transfer
We define triggers explicitly rather than leaving it to the model. An explicit request from the caller, always, on the first ask, with no attempt to talk them out of it. Detected distress or anger. Any topic on the excluded list, which for most clients includes complaints, legal threats, safeguarding, bereavement and anything involving a vulnerable customer. Repeated recognition failure on the same slot. A tool call that fails in a way the agent cannot work around. And an explicit confidence rule: if the agent cannot determine intent within a set number of turns, it stops trying.
Cold transfer
A SIP REFER hands the call to another destination and the agent disappears. It is simple, cheap and appropriate when the caller has given nothing away yet, for instance an immediate "speak to sales" at the greeting. It is the wrong answer once the caller has spent ninety seconds explaining themselves, because they will explain themselves again and be annoyed about it.
Warm transfer
The agent brings a human into a conference, briefs them privately with a short summary while the caller is on hold, then joins the caller and drops out. The human arrives knowing who is on the line, what they want and what has already been checked. Twilio's conference primitives support this directly, and it is what we build by default for anything past the first turn.
The technical piece people underestimate is the context handoff to the desktop the human is actually looking at. A screen pop carrying the transcript, the captured slots, the recording link and the reason for transfer, delivered into Salesforce, HubSpot, Zendesk, ServiceNow or whatever your team lives in. Without that, a warm transfer is just a slower cold transfer. That integration work is why voice projects and enterprise integration work tend to arrive together.
When there is no human available
It is three in the morning and the queue is closed. This case must be designed, not discovered. Options in order of preference: book a callback into a real calendar slot the caller can hear and agree to, take a structured message with a promised response window that your team can actually meet, or send an SMS with a self-service link. What must not happen is the agent apologising in a loop or dropping the call.
Measuring it
Transfer rate on its own is a bad metric because zero transfers is not the goal. We track transfer rate alongside the reason code, the turn at which it happened, and whether the caller's issue was resolved afterwards. A rising transfer rate at turn one means the greeting is wrong. A rising rate at turn six means a tool is failing.
Recording, Consent and Compliance
This section is deliberately cautious, and none of it is legal advice. It is the checklist we work through with your counsel, because voice projects get stopped at legal review more often than at technical review and it is entirely avoidable by starting the conversation in week one.
Call recording consent
Recording law is not uniform. In the United States, federal law and most states operate on one-party consent, while roughly a dozen states, including California, Florida, Illinois, Massachusetts, Pennsylvania and Washington, require all parties to consent. Because a caller's location is not reliably knowable from a phone number any more, the practical approach used by most national operations is to announce recording to everyone at the start of every call. In the UK, recording and retaining call audio is processing of personal data under UK GDPR with the ICO expecting a lawful basis and clear notification. Canada's PIPEDA requires you to tell the individual the purpose. Australia layers state and territory surveillance devices legislation on top of federal law, so the safe default there is also to announce.
Our engineering position is simple: build the announcement, the consent state and the recording switch as first-class parts of the call flow, so that a policy change is a configuration change and not a rebuild. Also build the honouring of a refusal, because a caller who says "do not record me" needs a defined path that is not a dead end.
Disclosing that the caller is talking to a machine
The direction of travel is clear. The FCC's February 2024 declaratory ruling confirmed that AI-generated voices count as artificial voices under the TCPA in the United States, which puts outbound AI calling squarely inside existing consent rules. California's bot disclosure law requires disclosure in certain commercial contexts. The EU AI Act imposes transparency obligations on systems that interact directly with people. Several US states have introduced their own requirements and more are coming.
We disclose by default in the opening line, in plain language, and we do not use a cloned human voice to imply otherwise. Beyond compliance, callers who are told upfront behave better: they speak more clearly, they use shorter sentences, and they ask for a human earlier when they need one, which improves your numbers rather than harming them.
Outbound calling rules
If you are dialling out, the TCPA in the US, the CRTC rules in Canada, Ofcom and PECR in the UK and the Do Not Call Register regime in Australia all have opinions about consent, calling hours, identification and suppression lists. Answering machine detection interacts with this: leaving an artificial-voice message has its own constraints, and a badly tuned detector that starts talking over the beep leaves half a message and an unhappy recipient. Build the suppression list check into the dialler, not into a spreadsheet somebody maintains.
Payment card data
If a caller reads out a card number, that audio is in scope for PCI DSS and so is your transcript, your logs and your model provider's infrastructure. The standard answers are to pause recording around the card capture and resume afterwards, or to use DTMF masking so the caller keys the number and the tones never reach the agent or the recording. We build the second where we can, because it keeps card data out of the AI pipeline entirely rather than trusting a redaction step.
Health, identity and sensitive categories
Where HIPAA applies, every vendor in the chain that touches the audio needs a business associate agreement, and not all of them will sign one, which constrains your stack before you write a line of code. Special category data under GDPR, including health and biometric information, needs an identified lawful basis and a documented retention period. Voiceprints used for authentication are biometric data in their own right and carry additional obligations of their own, particularly under Illinois' BIPA, which has generated real litigation.
What we build in
Redaction of card numbers, national identifiers and other configured patterns from transcripts and logs at write time rather than on a nightly job. Retention policies applied per data type with automated deletion. Role-based access to recordings with an audit trail of who listened to what. Regional data residency where your regulator requires it, including the ability to run recognition inside your own tenancy. Deletion on request that actually reaches the audio store, the transcript store, the analytics warehouse and the backups. Data protection work on the security side is not a separate project bolted on afterwards; it is part of the same delivery.
How We Build It, Week by Week
Weeks 1 to 2: listen to the calls
We take a sample of your real recordings and transcripts, sort the calls by type and volume, and pick the one to automate. In parallel we run the recogniser benchmark on that same audio, so vendor choice is decided by data rather than by preference. Output is a call flow document, a slot specification and a written definition of done with thresholds.
Weeks 2 to 3: the thin path
One call type, end to end, on a test number, with real telephony, real recognition and a real tool call into a staging system. Ugly but complete. This exists to prove the latency budget and the audio path early, because those are the things that kill projects, not the conversation design.
Weeks 3 to 5: turn-taking and recognition tuning
Endpointing thresholds, barge-in behaviour, backchannel handling, confidence thresholds, custom vocabulary and readback design, all tuned against recorded calls replayed through the pipeline. This is the least glamorous fortnight and it is the one that decides whether callers like the thing.
Weeks 4 to 6: integration and escalation
Tool calls into your booking system, CRM or order platform, with timeouts and fallbacks defined per call. Warm transfer built and tested against your actual contact centre, including the screen pop. Compliance controls implemented: disclosure, consent, recording switch, redaction, retention.
Weeks 6 to 8: shadow and pilot
The agent runs against a small share of live traffic, usually starting with out-of-hours calls where the current alternative is voicemail. Every call is reviewed for the first week. We fix what the calls tell us to fix, not what we assumed in week one.
Week 8 onward: scale and hand over
Traffic share increases against agreed thresholds, with the kill switch rehearsed. Handover covers the runbook, the dashboards, the prompt and model version history, the rollback procedure, and training for whoever in your team will own the call flow. Voice agents need ongoing tuning, and we would rather your team could do most of it.
Those durations describe a single well-scoped call type against systems that have usable APIs. A programme covering five call types across a legacy contact centre with no API on the booking system runs longer, and the discovery phase is where the honest number comes from. We would rather revise the estimate after week two than defend a guess made before we heard a single call.
Three Situations We See Repeatedly
The after-hours booking line that loses the job to whoever answers first
A home services company with a regional service area takes bookings by phone. Roughly a third of the calls arrive after six in the evening, they go to voicemail, and by the time somebody rings back the next morning the customer has booked a competitor. The obvious fix is a voice agent that takes the booking directly against the scheduling system.
What goes wrong: addresses. Callers give a postcode or ZIP quickly and indistinctly, the recogniser confuses the letters in the E-set, and the agent books a job forty miles outside the service area. The engineer finds out on the day.
What the fix looks like: the address slot is never free text. The agent captures the postcode, validates it against an address lookup service, reads back the matched address for confirmation, and checks it against the service area polygon before offering any slot at all. Two low-confidence attempts switch to keypad entry of the house number and postcode. The scheduling tool call is made with an idempotency key so a retry cannot double-book, and any booking outside the service area is a hard stop with a transfer, not a warning the agent can talk past.
The outbound reminder campaign that half-fills voicemails
A clinic or a logistics operator wants to confirm appointments or delivery windows the day before. Volume is high, the conversation is short, and the business case is straightforward because every unconfirmed slot is a wasted van or a wasted room.
What goes wrong: answering machines. A large share of the calls never reach a person at all. A detector that is too eager starts speaking to a human before they have finished saying hello. A detector that is too slow starts speaking over a voicemail greeting and leaves a message that begins mid-sentence. Meanwhile compliance is asking whether an artificial voice leaving a message is covered by consent that was gathered for something else, and nobody has an answer.
What the fix looks like: answering machine detection with beep detection rather than a fixed timer, and a separate, shorter script for the voicemail case that starts only after the beep. Calling hours enforced per recipient timezone rather than per server timezone. A suppression check against the do-not-call list at dial time. Consent basis reviewed and recorded per campaign before the first call goes out. Retry policy capped and logged, because repeated automated calls are the fastest route to a complaint.
The tier-one queue where the caller repeats themselves to a human
A subscription business runs a support line where most calls are five known issues. They want the agent to identify the caller, work out which issue it is, resolve the easy ones, and pass the rest to a person with context. The goal is not deflection. It is making the human's first thirty seconds useful.
What goes wrong: the transfer. The first build used a cold SIP transfer into the existing queue, so the caller heard hold music, then a human who asked for their account number again. Satisfaction on those calls lands below where it was with no automation at all, and the operations team starts asking to switch the whole thing off. That is the usual sequence.
What the fix looks like: warm transfer through a conference, a whisper to the human summarising the caller and the issue, and a screen pop into the support desktop carrying the transcript, the verified account and the diagnostic steps already attempted. Transfer reason codes recorded per call so the pattern is visible. And a rule that the agent never asks for information the CRM already holds against the calling number, which on its own takes fifteen or twenty seconds off the front of the call.
How Offshore Delivery Actually Runs From India
Voice work has a specific offshore problem that most software projects do not, and we would rather set it out honestly than let you find it in month two. Your voice agent is a live phone system. Its busiest hours are your customers' business hours, and for a US or UK client those hours are the Indian night.
The overlap window, without spin
Our standard working day is 09:30 to 18:30 IST. Against the UK that gives you real overlap: roughly 05:00 to 14:00 UK time, so from the moment your team starts until early afternoon. Against Australian eastern time it is roughly 14:00 to 23:00 AEST, giving you an afternoon overlap of about three hours. Against New Zealand it is thinner, an hour or so at the end of your day. Against US Eastern time, that standard day gives you almost nothing, because 18:30 IST is eight or nine in the morning in New York, depending on the time of year.
So for North American clients we run a shifted team rather than pretending. A 13:30 to 22:30 IST shift produces roughly 09:00 to 13:00 Eastern overlap. A 16:00 to 01:00 IST shift reaches the US Pacific morning. Both are real and both cost something: shift work is harder to staff, harder to retain, and adds a coordination burden because part of your team is asleep when another part is working. We plan the shift explicitly, we tell you who is on it, and we do not rotate people through it without telling you.
What we will not tell you is that you get 24/7 coverage for free. Genuine round-the-clock coverage means three shifts or a follow-the-sun arrangement with a second location, and it costs roughly what three shifts cost. For most clients the right answer is one shift aligned to your mornings plus a defined on-call rota for production incidents, which is much cheaper and covers the cases that actually matter.
On-call for a system that runs while India sleeps
This is the part specific to voice. If your booking line breaks at 10am in Chicago, that is 8:30pm in Mumbai. We handle this with a paging rota with named engineers and a response target, alerting tuned to the things that break a phone system rather than to CPU graphs, and a kill switch your own operations team can pull without us. Your team should never be waiting for India to wake up while calls fail. That is a design requirement, not a service level to negotiate afterwards.
Where the system runs
The engineering team is in India. The system is not. A voice agent serving US callers runs its media servers, its recogniser and its model endpoints in a US region, because a round trip between Mumbai and Northern Virginia typically measures in the low hundreds of milliseconds and the whole conversational budget is around one second. We measure the actual path early and place the components accordingly. Where data residency rules apply, that placement is also the compliance answer.
Working with a team you never see
Written first. Decisions land in the ticket or the document, not only in the call, so the person picking it up eight hours later has what they need. A daily written standup posted in your Slack or Teams before your morning, covering what moved, what is blocked and what is planned. One weekly call at a time that suits you, with an agenda circulated beforehand.
You get named engineers, not a pool. You interview them before they start and you can decline. Whoever leads your work is the person you speak to, not an account manager relaying messages. On voice projects that matters more than usual, because listening to calls together is how the tuning conversations actually happen and a relay in the middle makes that impossible.
Every change goes through pull request review by a second engineer, with tests, and nothing reaches production without it. Voice adds a specific gate: a regression suite of recorded calls replayed through the pipeline on every change, checking transcription accuracy on the known-hard audio, latency per stage, and that the agent still refuses to do the things it must never do. Prompt and model version changes are treated as deployments with the same review and the same rollback, because a prompt edit can change behaviour more than a code change.
English and communication
English is the working language of Indian engineering and has been for a long time, but written fluency and being clear on a call about a subtle turn-taking bug are different skills. We assess the second one in interview by asking candidates to explain a technical decision to a non-specialist. On voice projects specifically, we favour engineers who have worked with UK, US or Australian clients before, because the ear for what sounds natural to your customers is not something you can pick up from documentation.
What Goes Wrong, and How We Handle It
Quality control when you cannot watch the work
You should not have to take our word for it. Every engineer on the team has a public commit history in your repository, reviewed by a second person. Every sprint ends with a working demo you can dial, on a real number, rather than a status report. Call recordings from the test line are shared weekly with the metrics attached. If the numbers move the wrong way, you see it in the same week, not at the end of the phase.
Vendor and model drift
Speech and language vendors update models without asking you. A recogniser that scored well in March can behave differently in June, and a voice model can change its prosody in a way that breaks your pronunciation lexicon. Our answer is the regression suite: recorded calls, replayed on a schedule and on every deploy, with the accuracy and latency numbers tracked over time. Pinned versions where the vendor supports pinning. And an abstraction thin enough that swapping the recogniser is a week, not a rewrite.
Who owns the code and the data
You do, from the first commit, assigned in the contract rather than promised in a call. Work happens in your repository, your cloud accounts and your vendor accounts wherever possible, so there is no moment where handover means moving anything. The MSA covers assignment of everything you'd expect: the code, the tuning history, the call recordings, agreed with you before the first commit rather than negotiated after the fact. Where GDPR applies we sign a data processing agreement and name the sub-processors, which for a voice project means naming the speech vendors explicitly. Call recordings are your data and they stay in your storage.
Access and device security
Least privilege by default, access through your identity provider so you can revoke it in one action, and no production credentials on developer machines. Company-managed laptops with disk encryption and endpoint protection. Access to production call recordings restricted to named individuals with an audit trail, and separated from access to the code. Where your compliance regime requires background checks, dedicated network segments or specific certifications from us, tell us during scoping rather than after, because some of those are a lead time rather than a cost.
If it does not work out
How an engineer transitions off the project, and what happens if you end the engagement, is written into the MSA rather than worked out under pressure. If an engineer is not right for the work, tell us and we will address it; how the replacement and its ramp-up are handled is one of the things that agreement should cover. If you want to end the engagement, the exit is defined at the start: documentation current, credentials transferred, a handover period with the outgoing engineer available, and a recorded walkthrough of the system. On voice specifically that includes the tuning history, which is the part that took the longest to build and is easiest to lose.
The hidden costs nobody quotes
Ramp-up is real. A new engineer on a voice codebase is genuinely productive somewhere in week three, and anyone who tells you week one has not done it. Expect to spend your own team's time in the first month: listening to calls with us, deciding business rules, and answering questions that only you can answer. Voice agents also need ongoing tuning after launch rather than being finished, because your callers, your products and your vendors all change. Budget for a maintenance rhythm rather than a project that ends. And the running cost of the system itself is per second of conversation, so a change that makes the agent more talkative shows up on the bill.
Engagement Models
Dedicated voice team
Named engineers working only on your product, embedded in your standups and your board. Right when voice is a core part of what you sell, when the roadmap runs past a single call type, or when you want the capability to live inside your team rather than at a vendor. Typically a voice engineer, a backend engineer and a part-time QA specialist, scaling as the call types multiply.
Fixed-scope build
One call type, defined thresholds, agreed price against a written scope produced after the discovery phase rather than before it. Right when you want to prove the case on a single line before committing further. Ends with the agent live, the handover complete and your team able to change the flow.
Tuning and support retainer
A monthly block for the work that follows launch: reviewing calls, adjusting prompts and thresholds, adding vocabulary, responding to vendor model changes, and holding the on-call rota. Right when your team owns the product but does not want to own the speech pipeline. Often runs alongside a fixed-scope build once the first agent is live.
If the work is really about connecting the agent to systems that do not want to be connected, the same team covers enterprise integration. If it is about the model layer rather than the phone line, LLM integration is the closer fit. And where you want the capability in your own team permanently, we can staff it directly: AI engineers for the model and pipeline side, Python developers in India for the orchestration and tooling work that sits underneath.
Frequently Asked Questions
What is an AI voice agent?
A program that answers or places phone calls, listens to the caller in real time, decides what to do, and speaks back. Under the hood it is four parts: speech recognition, a language model, speech synthesis, and a telephony layer that carries the audio. What separates a working one from a demo is how it handles interruption, silence, mishearing, and the moment it has to hand the call to a person.
How fast does a voice agent have to respond?
Aim for under one second from the moment the caller stops speaking to the first sound of the reply, and under 300 milliseconds to stop talking when the caller interrupts. Past roughly 1.2 seconds of silence people repeat themselves or say hello, which corrupts the turn. That budget is spent across endpointing, recognition, model inference and synthesis, so no single component gets much of it.
Which speech recognition engine do you use?
It depends on the call. Deepgram and AssemblyAI are strong for streaming English at low latency. Azure Speech earns its place when you need custom vocabulary, a signed BAA or a specific regional deployment. Whisper is excellent offline and multilingual but is not a native streaming model, so running it live means chunking with a voice activity detector and accepting the added delay. We benchmark two or three against your own call recordings before choosing.
Can the agent transfer a call to a human?
Yes, and how it does so matters. A cold transfer over SIP REFER is cheap and drops the caller into a queue with no context. A warm transfer keeps the agent in a conference, briefs the human, and passes the transcript and captured fields to the agent desktop before the caller is joined. We build warm transfer as the default for anything where the caller has already spent a minute giving information.
Do we have to tell callers they are speaking to an AI?
Increasingly yes, and it depends on jurisdiction. The FCC's February 2024 declaratory ruling brought AI-generated voices under the TCPA restrictions on artificial voice calls in the United States. California's bot disclosure law and the EU AI Act transparency requirements point the same way. Our default is to disclose in the opening line, because it also reduces the number of callers who feel tricked when they work it out.
How do you handle strong accents and noisy calls?
By measuring rather than guessing. We take a sample of your real recordings, including the bad ones, and score word error rate per engine and per accent group before committing to a vendor. Then we reduce what the recogniser has to guess: phrase lists and custom vocabulary for your product names, digit and postcode readback, and a fallback to keypad entry when confidence drops twice in a row.
What happens when the agent does not understand?
It should degrade in a defined order rather than loop. The first failure triggers a narrower reprompt. The second switches modality, usually to keypad entry or an SMS link. The third transfers to a human with everything captured so far. Silent retry loops are the most common reason callers hang up on a voice agent, and a hard cap on retries per slot prevents them.
Where does the infrastructure run if the team is in India?
In the region closest to your callers. Our engineers work from India, but a voice agent for US callers runs its media path and its inference in a US region, because a round trip across the Indian Ocean eats most of the latency budget on its own. Deployment region is a design decision driven by where the phone numbers ring, not by where the developers sit.