Real-time voice translation on iPhone: how it works

You finish a sentence and nothing happens. A second passes, maybe two, then the phone speaks. That pause is not network lag, and it is not the translation model being slow. In most apps it is a deliberate design decision, and knowing which decision an app made tells you more about how it will feel than any accuracy score.

Every voice translator on iPhone runs the same four stages. What separates them is whether those stages are allowed to overlap.

An app that runs stages 2 to 4 strictly in sequence, after you go quiet, cannot be fast no matter how good its models are. An app that overlaps them starts translating while you are mid-sentence, and pays for it in a different currency: revisions.

Disclosure. We make OmniTalk, a voice translation app for iPhone. It sits in the second group. This article is about the mechanism, and the section titled "What the streaming design costs" is about where our approach is the wrong one.

Stage 1: capture, and why the microphone matters more than you think

The recogniser does not receive "your voice". It receives a stream of audio buffers at a fixed sample rate, and the quality of that stream sets a hard ceiling on everything downstream.

On the iPhone's own microphone this is uninteresting: you get wideband audio and the ceiling is high. It becomes interesting the moment Bluetooth is involved. When an app opens the microphone on a Bluetooth headset, iOS switches the whole connection from A2DP, the high quality one-way profile, to HFP, the bidirectional call profile. Both directions drop to call quality at the same time. There is no combination of settings that gives an app a good microphone and good playback simultaneously on the same headset.

That constraint shapes every earbud translation feature on the platform, and we covered it in detail, with Apple's own statement of it, in the AirPods comparison.

Practical consequence: a voice translator tested with the phone's microphone and a voice translator tested through earbuds are not the same test. Recognition error rates rise measurably on narrowband audio, and no amount of translation quality recovers a word the recogniser never heard.

Stage 2: recognition, and the endpoint problem

This is where the pause comes from.

Speech recognition has to decide when you have finished speaking. That decision is called endpoint detection, and the common implementation is a silence timeout: after roughly 700 to 1,000 milliseconds of no speech, the recogniser declares the utterance complete and emits a final transcript. Only then does a turn-based app move to stage 3.

Google Translate's conversation mode is explicit about working this way. Google's own instructions are "Take turns speaking", and the microphone "automatically detects when one language stops and the other language starts" (Google Translate Help, retrieved 12 August 2026). That is a well-built turn detector. It is still a turn detector, and turn detection means waiting for silence by definition.

The alternative is to use the partial results the recogniser is already producing. Apple's Speech framework exposes this directly: results arrive as an AsyncSequence rather than a single callback at the end, and the API distinguishes between volatile results, which the recogniser may revise, and finalised ones (SpeechAnalyzer, retrieved 12 August 2026). The framework even offers a preheat step, prepareToAnalyze, which Apple says "may improve how quickly the modules return their first results".

An app built on volatile results can push text into stage 3 roughly 200 to 400 milliseconds after you say a word, rather than a second after you stop. OmniTalk's App Store listing describes the intended effect: "Translations appear the moment you speak, built for real conversations, not sentence-by-sentence waiting."

Why it costs time: the silence timeout is not a bug you can tune away. Shorten it and the recogniser cuts people off mid-thought, because natural speech contains pauses longer than 500 milliseconds. Lengthen it and every exchange gains a second. Turn-based apps sit at the least-bad point on that curve.

Simultaneous versus consecutive, borrowed from human interpreters

The distinction has a hundred years of professional vocabulary behind it, and it maps onto apps precisely.

Consecutive interpreting. The speaker talks, stops, and the interpreter renders what was said. Total time is roughly double the original speech, because nothing overlaps. This is what a turn-based app does.

Simultaneous interpreting. The interpreter speaks continuously, running two to four seconds behind the speaker, and never asks anyone to pause. Conference interpreters work this way, in pairs, in booths, in 30-minute shifts, because the cognitive load is severe. This is what a streaming app approximates.

The reason human simultaneous interpreters are exhausting to be and expensive to hire is exactly the reason it is hard to build: you have to start committing to an output before the input is complete, and sometimes the end of the sentence proves you wrong.

Stage 3: translation, and why partial sentences are a harder problem

Translating a complete sentence is a solved-enough problem. Translating the first half of one is not, and the difficulty is grammatical rather than computational.

German puts the verb of a subordinate clause at the end. Japanese is subject-object-verb and marks negation on the verb. Turkish stacks meaning onto suffixes. In all three, the last word can invert the sentence. A system that starts translating early is repeatedly in a position where the correct output depends on a word that has not been spoken yet.

There are only three responses, and every streaming design picks one:

1. Wait for enough context. Hold output until a syntactic boundary looks safe. Accurate, and reintroduces some of the delay you were trying to remove.
2. Commit and revise. Emit a best guess and rewrite it on screen when later words contradict it. Fast, and produces visible text churn.
3. Commit and do not revise. Fastest, and wrong often enough to be dangerous on negation.

Read that list against what you see on screen and you can identify any app's strategy in one conversation. Text that appears and then silently changes is strategy 2. Text that appears in complete clauses, slightly late, is strategy 1.

The measurable consequence: revision behaviour is why two apps with identical benchmark accuracy feel different. Benchmarks score the final string. Users experience the intermediate ones.

Stage 4: speech output

Text-to-speech adds latency in two places that get confused with each other.

The first is time to first audio: how long after receiving text the voice starts speaking. Small on-device voices start almost immediately. Higher quality neural voices are usually synthesised server-side, which adds a round trip, and some implementations synthesise the entire utterance before playing any of it, so a long sentence is slower to start than a short one.

The second is the duration of the speech itself. A translated sentence takes as long to say as a sentence takes to say. No architecture removes this, and it is why hands-free earbud translation of a fast talker falls progressively behind: the audio queue grows faster than it drains.

Detail worth knowing: speaking the same sentence twice usually costs nothing the second time, because synthesised audio is cached. Apps that meter premium voices by audio length generally charge only for the first playback. Replays are free.

There is a second-order effect here that shows up in noisy places: if the other person cannot hear the phone, the fix is to replay rather than repeat yourself, because a replay costs no recognition, no translation and no metered voice time.

On-device or cloud: what each actually buys

The line is usually drawn as a privacy question. It is also a latency and coverage question, and the trade is sharper than the marketing on either side suggests.

Apple's Translation framework is unambiguous about where the work happens: "All translations using the TranslationSession class are processed on the user's device. Apple may collect API usage and performance metrics including the app bundle ID and the original and translated language, but this data does not include the original or translated content" (TranslationSession, retrieved 12 August 2026).

The catch is availability, and Apple's speech API models it explicitly with two separate properties. supportedLocales is "the locales that the transcriber can transcribe into, including locales that may not be installed but are downloadable". installedLocales counts "only locales that are installed on the device" (SpeechTranscriber, retrieved 12 August 2026).

Those two numbers are rarely equal, which is the source of a specific and very common failure: an app lists a language, you board a plane, and it stops working. The language was supported but never downloaded. Recognition packs and translation packs are also separate downloads, so a pair can be half-installed and fail in one direction only.

How to check your own phone before you travel: download the pair, enable Airplane Mode, and run a full exchange in both directions. It is the only test that distinguishes "supported" from "installed", and it takes about a minute.

Where the time actually goes

Rough figures for one short exchange on a good connection, to show which stages are worth optimising.

The endpoint timeout is the single largest fixed cost, and it is the only one that a different architecture removes outright rather than shortens. Everything else is a matter of hundreds of milliseconds. That is why the perceived gap between turn-based and streaming apps is so much larger than their difference in model quality.

How to measure any translator yourself

Four measurements, a stopwatch, and about ten minutes. Do it with the app you already have before you install another one.

1. Time to first output. Say a sentence of about eight words. Start timing at your last syllable. Stop when the first translated text appears. This isolates the endpoint timeout.
2. Time to first audio. Same sentence, stop timing when the voice starts. The difference between this and measurement 1 is your text-to-speech cost.
3. The interruption test. Say a sentence with a deliberate 1.5-second pause in the middle: "I need a doctor ... not a pharmacy." A turn-based app will usually treat this as two utterances and may translate the second as a standalone fragment. This is the failure mode with real consequences.
4. The revision test. Watch the screen, not the clock, while you say a sentence whose meaning turns on the final word. Note whether displayed text changes after it appears.

Run all four in Airplane Mode as well. Any app that behaves differently is using the network for something it did not tell you about.

What breaks, in every app tested

Negation. The most consequential failure class, and worse in streaming designs for the structural reason above. "She is not allergic to penicillin, but I am" is a sentence where a dropped negation and a swapped subject are both plausible errors and both dangerous. Our rule for anything medical: say it in two short sentences, and read the screen rather than trusting the audio.

Numbers spoken naturally. "Half four" and "quarter to six" survive far less often than "16:30". Times, prices and dosages are worth confirming on screen rather than by ear.

Code-switching mid-sentence. A proper noun in another language inside a sentence tends to be either translated when it should not be, or transcribed phonetically into nonsense. Names of people and places are the common casualty.

Self-correction. "Sorry, I meant tomorrow, not today" frequently produces two contradictory translated sentences rather than one corrected one, because the recogniser finalised the first clause before the correction arrived.

What the streaming design costs

Committing early has a real price, and OmniTalk pays it in three places.

Visible revision. Text that updates as you speak is unsettling the first time, and in verb-final languages it updates more. An app that waits looks calmer, because it hides the uncertainty rather than showing it. We show it, which is honest and occasionally annoying.

No advantage on short utterances. For "how much is this?" the endpoint timeout barely matters. Turn-based apps are perfectly good at single-phrase transactional exchanges, which is a large share of what travellers actually do.

Nothing is free. Apple's Live Translation is built into the system on qualifying hardware at no cost, and if you have an eligible iPhone with Apple Intelligence enabled and compatible AirPods, that is the first thing to try. We said so in the AirPods comparison and it is still true.

FAQ

Is real-time translation actually simultaneous? No. Even the fastest streaming design runs behind the speaker, in the same way a human simultaneous interpreter does. The realistic target is two to four seconds behind continuous speech, not zero.

Why does my translator cut me off mid-sentence? Its endpoint timeout expired during a natural pause. Speaking in shorter complete sentences works around it. So does an app that does not depend on detecting silence.

Does on-device translation mean it works with no internet at all? Only after the language has been downloaded, and recognition and translation are usually separate downloads. Verify in Airplane Mode before you need it.

Does a bigger app mean better translation? No. Bundled offline language models dominate app size. Google Translate is 290.2 MiB and Microsoft Translator is 25.6 MiB, and that difference is packaging, not quality (App Store records, retrieved 12 August 2026).

Why does it get worse when I use earbuds? Opening the Bluetooth microphone forces the whole link to call-quality audio in both directions. Recognition accuracy drops with it, and no app can opt out.

Which approach should I choose? Turn-based is fine for short transactional exchanges and is calmer to read. Streaming is worth it for actual conversation, where waiting a second after every sentence compounds into something nobody wants to sit through.

Figures retrieved 12 August 2026. App Store sizes are MiB, as the store displays them. Latency ranges are order-of-magnitude figures for illustration, not measurements from a controlled test; the method above lets you produce your own.

👉 Get OmniTalk on the App Store · OmniTalk features · Support FAQ