Translating text on a web page is easy. The browser already knows where the text is, what each sentence says, and how it should be displayed.
Images are different.
A screenshot, restaurant menu, product label, poster, or scanned document is just a collection of pixels. Before we can translate anything, we first need to figure out where the text is, recognize what it says, translate it, and somehow put the translated text back into the image.
That makes image translation an interesting combination of computer vision, OCR, machine translation, and image generation.
Step 1: Finding Text Inside the Image
The first problem is text detection.
An image may contain multiple text blocks with different fonts, sizes, colors, and orientations. Text can also appear on complex backgrounds, signs, packaging, screenshots, or photographs.
A typical OCR pipeline first detects regions that are likely to contain text.
Instead of seeing:
Welcome to Tokyo
Platform 3
Exit A
the system initially sees something closer to:
[x1, y1, x2, y2]
[x1, y1, x2, y2]
[x1, y1, x2, y2]
Each bounding box represents an area where text probably exists.
Step 2: Recognizing the Text
Once the text regions are detected, OCR converts the pixels inside those regions into characters.
For clean screenshots, this can work extremely well.
Real-world photos are harder.
Some common problems include:
blurry images
low contrast
decorative fonts
rotated text
shadows and reflections
curved product packaging
mixed languages
This is why image quality still matters even with modern AI models.
Step 3: Translating the Content
After OCR, the recognized text can be sent to a translation model.
At this stage, context becomes important.
Imagine translating a menu containing:
Cold Brew
House Special
Market Price
Translating each phrase independently may produce technically correct results, but the wording can feel unnatural.
Modern translation systems can use surrounding text and context to produce better results.
Language detection is also useful here. Instead of asking the user to manually select both the source and target language, the application can detect the source language automatically and only ask where the text should be translated.
Step 4: Putting the Translation Back Into the Image
This is probably the most interesting part.
Returning translated text as plain text is easy.
Producing a translated image is much harder.
The system needs to remove or cover the original text and place the translated version in approximately the same location while considering:
text size
line wrapping
alignment
background
spacing
surrounding visual elements
Different languages can also have dramatically different text lengths.
A short English phrase may become a much longer sentence in German, while several Chinese characters may translate into multiple English words.
The goal is therefore not pixel-perfect reconstruction. It is preserving enough of the original visual context that the translated image remains easy to understand.
Why This Is Useful
There are many situations where keeping the translation inside the image is more useful than extracting plain text.
For example:
translating restaurant menus while traveling
reading screenshots from international communities
understanding product labels
translating posters and notices
reading foreign-language study materials
translating UI screenshots
sharing translated images with other people
You can immediately see which translated sentence corresponds to which part of the original image.
I Built a Small Tool Around This Idea
I recently experimented with this workflow and built PictureTranslator.com.
The idea is intentionally focused: upload an image, choose a target language, and receive a translated version of the image.
It currently works with common formats such as JPG, PNG, and WebP, automatically detects the source language, and lets you compare the translated result with the original.
I wanted the interaction to stay lightweight because image translation is often something you need for a single screenshot, sign, menu, or document and then move on.
It has also been a fun example of how AI can turn several traditionally separate tasks—OCR, translation, layout understanding, and image editing—into one small web workflow.
If you're experimenting with OCR, multimodal AI, or translation tools, image translation is a surprisingly interesting problem to explore.
Top comments (0)