AI Voice Agents vs IVR: How They Work and Where Each Fits
An AI voice agent is software that answers or places phone calls and holds a spoken conversation, using speech recognition, a language model and speech synthesis. A traditional IVR plays recorded prompts and routes the call by keypad presses or a few set words. A voice agent can handle requests that a menu cannot. In exchange it needs more engineering, more testing and a legal review of consent and disclosure.
What are AI voice agents?
AI voice agents are programs that talk with callers in ordinary speech and take actions during the call. Typical jobs are answering a business line, booking appointments, answering questions from a knowledge base, qualifying leads, taking messages and transferring the caller to a person.
The same product is sold as an AI phone agent, a voice AI agent or an AI receptionist. IVR stands for interactive voice response, the "press 1 for sales" menu.
How does an AI phone agent work, step by step?
An AI phone agent repeats a loop of four parts for the whole call: telephony, speech-to-text, a language model and text-to-speech. One turn of the conversation runs as follows.
- Telephony carries the call. A carrier or a SIP connection delivers the call and streams the audio to the software. Phone audio carries less detail than app audio. Twilio's Media Streams documentation, read on October 3, 2026, specifies the format as
audio/x-mulawwith a sample rate of 8000. - Speech-to-text transcribes the caller. The system also has to decide when the caller has finished. Deepgram's documentation, read on October 3, 2026, calls this endpointing and describes a voice activity detector that "monitors the incoming audio and triggers when a sufficiently long pause is detected."
- The language model decides the reply. It reads the transcript, its instructions and any knowledge-base content. It returns either words to say or a tool call: check a calendar, create a booking, send an SMS or transfer the call.
- Text-to-speech speaks the reply. The text is synthesized into audio and streamed back down the line.
- The loop manages turns. If the caller starts talking, playback stops and the agent listens.
- The call becomes a record. The system stores a recording, a transcript and a summary.
A production system usually takes each part from a separate provider, chosen for language coverage, response time and price. Some systems replace the three middle stages with one speech-to-speech model. OpenAI's Realtime API guide, read on October 3, 2026, describes that design: "The model works directly with audio, maintains conversation state, and can call tools."
What makes an AI voice agent hard to build?
Four problems decide whether callers accept a voice agent: response delay, interruptions, call transfer, and accents and noise.
Response delay
People expect a reply almost at once. Levinson and Torreira, in a review published June 12, 2015, report that in human conversation "the gaps between turns are short (of the order of 200 ms)", even though "the latencies involved in language production are much longer (over 600 ms)".
The phone network uses part of that time before any AI is involved. ITU-T Recommendation G.114, approved in May 2003, says that if mouth-to-ear delays "can be kept below" 150 ms, "most applications, both speech and non-speech, will experience essentially transparent interactivity", and recommends "to not exceed a one-way delay of 400 ms for general network planning".
A voice agent then adds the pause it waits to be sure the caller has finished, the model's processing time and the start of speech synthesis. Engineers reduce the total by streaming every stage. We give no target figure, because the result depends on the providers, the prompt and the route the call takes.
Interruptions
Callers talk over an agent as they talk over each other. The agent has to stop speaking and listen, without treating a cough or background noise as an interruption. Voice platforms expose this as a setting. Twilio's ConversationRelay documentation, read on October 3, 2026, has an interruptible attribute that "Specifies if caller speech can interrupt TTS playback."
Call transfer
A transfer needs three decisions: when to hand over, where the call goes and what information goes with it. Twilio's documentation shows handoff data carrying a reason such as "The caller requested to talk to a real person". The design must also cover an unanswered transfer.
The cost of a missing exit is documented for text chatbots. The Consumer Financial Protection Bureau reported on June 6, 2023: "Chatbots may be useful for resolving basic inquiries, but their effectiveness wanes as problems become more complex."
Accents and noise
Speech recognition makes more errors on some voices than others. Koenecke et al., published in PNAS on March 23, 2020, tested five commercial systems, from Amazon, Apple, Google, IBM and Microsoft, and found an "average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers". Those systems have since been replaced, so the figures do not describe current products. The practical step is unchanged: test recognition on recordings of your own callers, including names, addresses and numbers.
How does an AI voice agent compare with a traditional IVR system?
The two differ in how the caller speaks, how the call flows and how they fail. An IVR is predictable and narrow. A voice agent is flexible and needs supervision.
| Touch-tone or menu IVR | AI voice agent | |
|---|---|---|
| How the caller responds | Presses a key or says one of a few set words | Speaks in ordinary sentences |
| Call flow | A menu tree fixed in advance | Generated turn by turn, within written instructions and rules |
| What it completes | Routing, status lookups, keypad entry | Bookings, answers from a knowledge base, lead qualification, messages |
| Requests outside the script | Repeats the menu or queues the caller | Asks a follow-up, answers, or transfers with a summary |
| Typical failure | The caller cannot find the right option | Misheard words, a wrong answer, awkward timing |
| Response timing | The prompt plays at once | Each reply waits on recognition, the model and synthesis |
| Predictability | The same path every time | Wording varies, so it needs test calls, limits and monitoring |
| Outbound calls in the US | Prerecorded-voice rules apply | The FCC treats AI-generated voices as "artificial" under the same rules |
Where does each fit?
An IVR fits calls with a few fixed choices, and a voice agent fits calls where people explain a need in their own words.
A menu IVR fits when:
- The call is routing: sales, support, billing.
- The caller enters digits, such as an account number.
- The script must be identical every time.
An AI voice agent fits when:
- Callers ask open questions or describe a problem.
- The call needs back-and-forth, such as finding an appointment time.
- Calls arrive after hours or in bursts that staff cannot cover.
The two can be combined. A voice agent can accept keypad input as a fallback and transfer to a person when its rules say so.
How does an AI receptionist compare with a human receptionist?
An AI receptionist covers one part of a receptionist's job, the phone, and covers it at all hours. The US Bureau of Labor Statistics, on a page last modified August 27, 2026, describes the whole job: "Receptionists do administrative tasks, such as answering phones, greeting visitors, and providing general information about their organization."
The same page counts 947,500 receptionist jobs in the United States in 2025, at median pay of $18.27 per hour, and says employment "is projected to decline 2 percent from 2025 to 2035."
| AI receptionist | Human receptionist | |
|---|---|---|
| Hours | Any hour, several calls at once | Working hours, one call at a time |
| Scope | Phone calls and the tasks connected to them | Phone, visitors, and office tasks |
| Unusual or upset callers | Should transfer | Uses judgment |
| Cost basis | Usage-based fees set by the vendor | Wages and benefits |
We give no cost figure for the AI side, because vendor pricing varies. A sensible arrangement gives the agent routine, overflow and after-hours calls, and sends transfers to people.
What legal points must a business check?
A business must check at least four things: consent for outbound calls, identification and AI disclosure, call-recording consent, and the rules of each country it calls. Verify each point below with counsel.
| Topic | Regulator's source | What it says |
|---|---|---|
| Outbound calls with an AI voice (US) | FCC Declaratory Ruling FCC 24-17, released February 8, 2024 | "we confirm that the TCPA's restrictions on the use of 'artificial or prerecorded voice' encompass current AI technologies that generate human voices." Callers "must obtain prior express consent from the called party". For telemarketing, the FCC's rules require "prior express written consent". |
| Identification on those calls (US) | Same ruling, quoting 47 CFR § 64.1200(b)(1) | An artificial or prerecorded voice message must "At the beginning of the message, state clearly the identity of the business, individual, or other entity that is responsible for initiating the call." Telemarketing messages must also offer an opt-out. |
| Call recording (US federal) | 18 U.S.C. § 2511(2)(d) | Interception is not unlawful under that chapter "where such person is a party to the communication or where one of the parties to the communication has given prior consent". |
| Call recording (US states) | FCC consumer guide, updated December 30, 2019; Washington RCW 9.73.030 | The FCC: "The FCC has no rules regarding recording of telephone conversations by individuals, but some state laws prohibit this practice." Washington requires "the consent of all the participants in the communication" and treats an announcement that the call is about to be recorded as consent. |
| Automated marketing calls (UK) | ICO guidance on telephone marketing, read October 3, 2026 | "You must not make an automated marketing call – that is, a call made by an automated dialling system that plays a recorded message – unless the person has specifically consented to receive this type of call from you." |
| AI disclosure (EU) | AI Act, Article 50(1) | Providers must ensure "the natural persons concerned are informed that they are interacting with an AI system", unless that is obvious. |
Three notes on the table. First, the FCC ruling is about outbound calls. Its closing paragraph says the requirements apply "to any AI technology that initiates any outbound telephone call using an artificial or prerecorded voice to consumers." Inbound answering raises the recording and disclosure questions instead.
Second, on the EU rule, the European Commission's AI Act page, updated August 3, 2026, states: "The transparency rules of the AI Act will come into effect in August 2026."
Third, voice platforms place these duties on the business that deploys the agent. Calldone, a voice AI platform built by Easital for a client in the United States, is one example. Its terms, last updated August 6, 2026, say: "Where required, you must disclose to callers that they are interacting with an AI agent and that the call may be recorded".
Key takeaways
- An AI voice agent runs a loop of telephony, speech-to-text, a language model and text-to-speech. Some systems use a single speech-to-speech model.
- The hard problems are response delay, interruptions, call transfer, and accents and noise.
- A menu IVR fits fixed choices and keypad entry. A voice agent fits open requests and needs testing and monitoring.
- In the US, an FCC ruling released February 8, 2024 treats AI-generated voices as "artificial" under the TCPA, so outbound AI calls need prior express consent.
- Recording consent varies by state, and the UK and EU have their own rules. Have counsel review before launch.
Frequently asked questions
What is an AI voice agent?
An AI voice agent is software that holds a spoken phone conversation. It converts the caller's speech to text, uses a language model to decide what to say or do, and converts the reply to speech.
How is an AI voice agent different from a traditional IVR system?
An IVR follows a fixed menu and accepts keypresses or a few set words. An AI voice agent accepts ordinary sentences and generates each reply within its instructions. The IVR is more predictable, and the voice agent covers more kinds of request.
Can an AI receptionist replace a human receptionist?
It can take over routine phone work: answering, booking, basic questions and messages, at any hour. It does not greet visitors or handle office tasks, which the Bureau of Labor Statistics lists as part of the job. Unusual or upset callers should be transferred to a person.
Is it legal to make outbound calls with an AI voice agent in the US?
Outbound calls with an AI-generated voice fall under the TCPA. The FCC's ruling released February 8, 2024 says callers "must obtain prior express consent from the called party", and telemarketing calls need prior express written consent. State laws add their own rules.
Do you have to tell callers they are talking to an AI?
It depends on the jurisdiction. Article 50 of the EU AI Act requires that people are informed they are interacting with an AI system unless that is obvious. FCC rules require artificial-voice messages to identify the business responsible for the call. Disclosing in every call is the simpler policy.
Easital Technologies Ltd. designs and builds AI voice agents and the platforms that run them. Calldone, mentioned above, was built by Easital for a client in the United States. See voice AI agent development for the service, the Calldone case study for the build, and AI agent development for agents outside the phone channel. Easital does not give legal advice.
Sources
All sources were opened and checked on October 3, 2026.
- Twilio, "Media Streams: WebSocket messages", documentation, undated. https://www.twilio.com/docs/voice/media-streams/websocket-messages
- Twilio, "TwiML Voice: ConversationRelay", documentation, undated. https://www.twilio.com/docs/voice/twiml/connect/conversationrelay
- Deepgram, "Endpointing", documentation, undated. https://developers.deepgram.com/docs/endpointing
- OpenAI, "Realtime API" guide, documentation, undated. https://developers.openai.com/api/docs/guides/realtime
- Calldone, Terms of Service, last updated August 6, 2026. https://calldone.ai/terms
- Levinson and Torreira, "Timing in turn-taking and its implications for processing models of language", Frontiers in Psychology, June 12, 2015. https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2015.00731/full
- International Telecommunication Union, ITU-T Recommendation G.114, "One-way transmission time", approved May 2003. https://www.itu.int/rec/T-REC-G.114-200305-I/en
- Consumer Financial Protection Bureau, "Chatbots in consumer finance", June 6, 2023. https://www.consumerfinance.gov/data-research/research-reports/chatbots-in-consumer-finance/chatbots-in-consumer-finance/
- Koenecke, Nam, Lake, Nudell, Quartey, Mengesha, Toups, Rickford, Jurafsky, Goel, "Racial disparities in automated speech recognition", PNAS, March 23, 2020. https://pmc.ncbi.nlm.nih.gov/articles/PMC7149386/
- US Bureau of Labor Statistics, Occupational Outlook Handbook, "Receptionists", last modified August 27, 2026. https://www.bls.gov/ooh/office-and-administrative-support/receptionists.htm
- Federal Communications Commission, Declaratory Ruling FCC 24-17, CG Docket No. 23-362, adopted February 2, 2024, released February 8, 2024. https://docs.fcc.gov/public/attachments/FCC-24-17A1.pdf
- Federal Communications Commission, consumer guide "Recording Telephone Conversations", last updated December 30, 2019 (read through a reader proxy). https://www.fcc.gov/consumers/guides/recording-telephone-conversations
- United States Code, 18 U.S.C. § 2511, as published by the Legal Information Institute, Cornell Law School. https://www.law.cornell.edu/uscode/text/18/2511
- Washington State Legislature, RCW 9.73.030. https://app.leg.wa.gov/RCW/default.aspx?cite=9.73.030
- Information Commissioner's Office, "Telephone marketing", Guide to PECR, undated. https://ico.org.uk/for-organisations/direct-marketing-and-privacy-and-electronic-communications/guide-to-pecr/electronic-and-telephone-marketing/telephone-marketing/
- European Commission, AI Act Service Desk, "Article 50: Transparency obligations for providers and deployers of certain AI systems". https://ai-act-service-desk.ec.europa.eu/en/ai-act/article-50
- European Commission, "AI Act" policy page, updated August 3, 2026. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai