You are standing at a menu, a ticket machine, or a customs form. You cannot read the text. Point your phone camera at it. Hold still. Wait for the translated overlay to settle over the original. Read the translation. If the overlay flickers, refuses to appear, or gives nonsense, take one clean photo and switch to text translation. That works on stills when the live view fails.
Disclosure. We make OmniTalk, one of the apps that does this. This article is about how camera translation works in general, and it is blunt about where the technique fails regardless of whose app you point at the menu.
Camera translation is for gist and orientation, not certainty. It will read a menu fast. It will get a handwritten special wrong occasionally. It is not the right tool for a drug dose or a legal form. But for most travel moments, it is the quickest way to know what you are looking at.
Every phone-camera translator works the same four stages, and each stage fails in its own way. Knowing which one broke tells you what to do about it: move closer, change the light, take a still, or give up and ask a person.
The four stages of camera translation
Every camera translator works through four stages. Each one can fail on its own.
Detection fails when there is no contrast: pale text on a pale menu, a glossy sticker with reflections, a tilted sign at night. If the camera cannot find the text, nothing else happens. You usually see the failure as a box that never appears, or a box that flickers around the screen.
Recognition is optical character recognition, OCR: it turns pixels into characters. A clean printed menu is easy. Three things are not: a handwritten chalkboard, a curly display typeface, and low contrast. This stage is also where diacritics, tightly kerned scripts, and distorted perspective cause trouble. The camera may recognise individual letters correctly but assemble them into a word that does not exist.
Grouping decides what belongs together, and a two-column menu is the classic failure. The OCR reads the left column, then the right column, left-to-right. The result is "Spaghetti carbonara pork belly with rocket prices euros" as one item. Correct grouping requires understanding layout, not just characters. It has to know that a column means a separate list, not a continuation of the same sentence.
How to hold the phone for a clean read
A few seconds of positioning change the success rate.
Hold the phone parallel to the text, not at an angle. A straight-on photo removes perspective distortion, which hurts OCR. Tilted text means the system has to estimate a rotated box before it can translate, and that estimate is a common source of errors.
Tap the screen to focus, then let it settle. Do not move while the overlay draws. Live video translation works frame by frame at roughly 30 frames a second: movement hands it thirty blurry frames instead of one sharp one.
Move closer instead of zooming. Digital zoom throws away the detail OCR needs: a 2x digital crop gives the recogniser a quarter of the pixels it had.
Watch for shadow and glare: move your body, or move the object, until the light is even. A reflective machine screen at a train station is worse at noon than at night because overhead lights create hotspots.
If the text is on a curved surface, flatten it or photograph the flattest part: a bottle label, a tube, a coffee cup. Curved text is the hardest detection problem a camera translator has to solve.
Why the redraw stage is the hard part
The first three stages are hard. Stage 4 is where the visible result lives or dies. A camera overlay has to solve four problems at once.
First, translated text is usually longer than the source, and the shorter the original the worse it gets. IBM's globalisation guidelines, quoted by the W3C, put average expansion from English into European languages at 200–300% for strings up to 10 characters, 180–200% at 11–20 characters, and 130% once you pass 70 (W3C, retrieved 20 August 2026).
Read that against a sign. A one-word button label can come back three times as wide. A paragraph of safety instructions grows by a third. So the overlay's hardest job is the smallest text: the word on the ticket-machine button, not the notice underneath it. If you shrink the text to fit the original box, it becomes unreadable. The overlay has to condense the text horizontally — tighten spacing, reflow lines, or widen the panel — without turning it into four-point type. Condensing, not shrinking, is the difference between readable and decorative.
Second, the panel behind the text has to be coloured per paragraph. Take a single average colour from the whole image and you get a grey mush: white page, dark table, average becomes mid-grey. The overlay must sample the region directly behind the text and paint the panel to match that local background. One wrong average and the overlay looks like a sticker from a different photo pasted over the sign.
Third, masks and text must render in two separate passes. If they interleave, a later line's background panel paints over an earlier line's translated text. You end up with a clean background and no words. The pipeline is: paint all the background masks first, then draw all the text on top. Any other order erases translations. This is not a subtle bug; it makes the feature useless.
Fourth, a tilted photo needs the text's oriented corner quad, not an axis-aligned rectangle. Text on a diagonal sign lives in a rotated box. If the system draws an axis-aligned rectangle around it, you get a huge box overlapping neighbouring text, and horizontal translation over a diagonal sign. The system has to estimate the four corners of the rotated text block and fit the translated text into that quad.
None of this is specific to one app. We hit all four building ours, and so has everyone else who ships an overlay: they are the general problems the technique creates. When you see a clean overlay on a menu, all four are working. When you see a jumbled mess, one of them has failed.
Seven places camera translation fails, and what to do
The moments that matter are not the big street signs. They are the small, urgent ones. A street sign is designed to be read. A ticket machine is designed to be operated. A customs form is designed to be completed. None are designed for translation. Each breaks the technology in a different way.
| Moment | Why it breaks | What to do when it fails |
|---|---|---|
| Transit ticket machine | Small screen, button labels, abbreviations, glare from overhead lights. The camera reads fragments, not the screen as a whole. | Step back, tap the screen to wake it, photograph one part at a time. Use text translation on the photo if the overlay jumps. |
| Arrival or customs form | Small print, boxed rows, legal wording. The camera can translate fields but loses the order of the rows. | Photograph the form flat. Translate line by line. If the instructions matter, ask a staff member. |
| Pharmacy label | Tiny text, medical terms, critical meaning. A wrong translation is dangerous. | Do not rely on an overlay. Ask a pharmacist. Use the camera only to identify the product, not the dose. |
| Grocery label | Multilingual stacked text, small fonts, barcode interference. The camera may translate the wrong language. | Point at the ingredient list, not the front panel. Use text translation if the overlay mixes languages. |
| Handwritten specials board | Chalk, uneven spacing, curly letters. OCR fails on handwriting. | Try once. If it fails, take a photo and ask. A phrasebook is the fallback when the camera gives up. |
| Taxi receipt | Faint thermal print, creases, low contrast. OCR reads fragments. | Flatten the receipt, photograph under direct light. Increase contrast in the photo, then use text translation. |
| Rental-car warning sticker | Symbol plus text, diagonal placement, glossy reflection. Detection may find the icon but miss the text. | Photograph from straight on. If the warning matters at all (brakes, lights, fuel type), ask the rental counter. |
When the camera fails, paper does not. We publish 30 free printable travel phrasebooks, 36 phrases each, no signup. They cover the same urgent moments and do not need a signal or a clean photo. Print one before the trip and it works when the phone is dead, the text is handwritten, or the lighting is impossible.
Text translation is the fallback when the overlay fails
Live overlay is impressive when it works. It also fails often. The phone is moving, the lighting is poor, the text is at an angle. A still photo gives the camera translation system more time and a stable frame.
When the overlay refuses to settle, take one photo. Then run it through text translation. You get the same recognition and translation, but without the live redraw pressure. The result is usually more accurate because the OCR sees one sharp frame instead of twenty blurry ones.
Text translation and saved transcripts both help here. Photograph a menu, translate the text, keep the result. That matters when you need the translation later — an allergy warning you want to double-check, a receipt you need for expenses, a form you must fill in again.
When camera translation is the wrong tool
Do not use a camera overlay for anything you must be certain of. The list is short and it is absolute:
A drug dosage. A legal form. An allergy warning. A contract clause. A safety instruction on a gas appliance. If a mistake has a real cost, the overlay is not enough. Camera translation gives you gist and orientation. It can tell you a label says "take with food". It must not be the thing that decides a dose.
Ask a person: a pharmacist, a doctor, a lawyer, a hotel receptionist, a police officer. We would rather you did that than trusted our overlay. They can read the original, understand the context, and tell you what it actually means. A translation app cannot be that person.
This is not a soft warning. If you have a child with a nut allergy and a foreign-language food label, do not point the camera and trust the result. Ask someone.
Offline: download the language pack before you fly
Camera translation works without a signal — if you have the language pack on the phone.
Do not wait until the airport. Airport Wi-Fi is slow, metered, or broken. The moment you need the pack is the moment you have just landed, with no data, standing at a ticket machine. Download the pack before you leave home.
Ours works offline once a language pack is downloaded, camera translation included; the free tier covers one hour of live conversation a month, and camera and text translation are not metered at all. The phone does the recognition and translation on-device, so no signal is needed. Download the languages you will need before the flight, not at the gate.
The same rule applies to any app that claims offline translation. If it needs a one-time download, do it early. The download is not a convenience. It is the difference between reading the exit sign and not reading it.