CamCyclopedia
Now read foreign languages in images 'exactly as they appear': The World of Overlay Translation Technology
Have you ever felt inconvenienced by menus or manuals in foreign languages while traveling abroad or shopping for an imported product?
We are always hard at work for those of you who reach for your camera and turn it on in front of a menu board at a restaurant in an unfamiliar land or a complicated manual for an imported product.
You no longer need to manually type it out to translate the text within an image.
Beyond simply displaying text, the Galaxy Overlay Translation technology will serve as your eyes, displaying the translation naturally 'as if it were the original text in the image'.
1. Introduction to Overlay Translation
“Provides easier understanding by breaking through language barriers”
Galaxy's overlay translation is an innovative technology that goes beyond simple text conversion, changing the language in a natural way while preserving the visual elements of the image as they are.
By accurately recognizing and translating the foreign language in the image and perfectly placing the translated text in the original text's location, users can intuitively understand the content while maintaining the original image's layout and design.
Check out the difference between before and after!
- Before: The confusing experience of taking a photo of a Japanese menu only to have the text translated separately resulting in a mismatch between the menu items and their corresponding photos.
- After: The Japanese menu in the photo is instantly translated into English while retaining its original layout, creating a seamless experience where the menu image and translated text match perfectly, making it easy to place your order.
Experience a more convenient and smarter daily life with Galaxy Overlay Translation!
2. Core Process of the Overlay Translation Technology
"An extraordinary translation experience powered by AI"
Beneath the surface of the seemingly simple overlay translation feature, a sophisticated three-stage process takes place powered by Samsung's advanced AI technology.
① Text Recognition within Images (OCR: Optical Character Recognition): Accurately recognizes the location and content of text even in various fonts, tilted angles, and complex backgrounds within images.
② Intelligent Translation (NMT: Neural Machine Translation): Analyzes the context of the recognized text and translates it into the most natural English (or the selected language). It provides natural translations appropriate to the situation, rather than mere word substitution.
③ Image Reconstruction (Renderer: In-painting & Overlay): This is the most essential and challenging technique. After removing the original text, the surrounding background is analyzed to naturally fill the resulting empty space (In-painting). The translated text is then placed in a font and color similar to the original, ensuring seamless integration without any visual discordance.
This entire process happens in an instant, providing users with a natural experience as if the content were originally written in that language. Experience the true 'magic of translation' created by Samsung's AI technology.
The In-painting technology that supports Galaxy's overlay translation feature proceeds in the following three steps.
Text Area Recognition and Masking
The system automatically and accurately identifies text areas within an image that require translation. The recognized text areas are masked and prepared for the next stage.
Restoration of Areas Using AI Eraser
After the masked text area is removed, Samsung's AI technology analyzes the surrounding background. Based on the analyzed data, the removed area is restored to blend in with the surroundings naturally.
Apply Translation Text Overlay
The translated text is positioned precisely over the restored image, replicating the style of the original. This ensures that the final translated output appears natural and harmonious.
This sophisticated in-painting technique provides users with a natural experience as if the translated text had been in the original image from the very beginning.
※ Due to processing speed limitations, the in-painting technology is currently not supported for live camera translation and is only used in still image-based translation services, such as Gallery photo translation feature.
3. Try using Galaxy Overlay Translation like this!
If you are a Galaxy smartphone user, you can conveniently utilize the overlay translation feature anytime, anywhere, without the need to install a separate app.
Before using overlay translation, the corresponding language pack must be installed on your device. The method to install the language pack is as follows.
① Settings > Select General Management.
② Select 'Language packs'.
③ Download the desired language pack from the list of supported languages.
Once the language pack installation is complete, you can immediately use the overlay translation feature. We recommend using Wi-Fi to download and install as the file size may be large.
You can utilize the overlay translation feature in various apps such as Gallery, Internet, and more. The simple usage methods in each app are as follows.
Overlay Translation in the Gallery App
① Select the image that needs to be translated.
② Once text is detected within the image, a 'T' button will automatically appear.
③ Tap the 'Translate' icon to initiate the translation.
※ The translation option appears only when a language different from the device's system language is detected.
④ You can change the translation language using the icon on the right.
Overlay Translation in Internet app
① Open the webpage you wish to translate.
② Select the star-shaped icon located at the bottom center of the screen.
③ Select the "Translate" option to automatically translate the entire webpage, including any text embedded within images.
④ If necessary, you can change the translation language using the language icon.
Live Translation through the Gallery app
① Point your camera at the text you wish to translate.
② Once text is detected on the preview screen, a 'T' button will appear; select this button.
③ Select the 'Live Translation' capsule.
④ After a brief moment, the translation results will appear as an overlay; as you move the camera, the translated text will move along with it.
⑤ You can change the translation language using the language change icon.
Enjoy a more convenient smartphone experience with Galaxy's overlay translation feature!
※ The live translation feature is supported starting with the Galaxy S26 models and will also be available on new products released in the future.
A Word from the Developers:
Beyond mere text conversion, we aim to offer an experience that broadens your horizons. We endeavored to transform those moments of frustration when text felt like images into "joy of understanding," made possible through Galaxy. Moving forward, we will continue to do our utmost to deliver the best possible user experience through even more sophisticated background restoration and natural translation technology. If you have any valuable feedback while using the service such as areas you feel could be improved, please feel free to leave a comment at any time!
Your feedback is the driving force behind creating a better service.
— Galaxy Overlay Translation Development Team
[Overlay Translation]
Supported models: Galaxy S22/S23/S24/S25/S26 series, Galaxy S23/S24/S25 FE, Galaxy Z Fold4/5/6/7, Galaxy Z Flip4/5/6/7, Galaxy Z Fold Special Edition, Galaxy Z TriFold, Galaxy Z Flip7 FE, Galaxy Tab S8/S9/S10/S11 series
[Live Translation]
Supported models: Galaxy S26 series
Supported Languages
Gujarati (India)
Dutch
German
Russian
Romanian
Vietnamese
Swedish
Spanish (Mexico), Spanish (United States), Spanish (Spain)
Arabic
English (United States), English (United Kingdom), English (Australia), English (India)
Italian
Indonesian
Japanese
Chinese (Taiwan), Chinese (Mainland China), Chinese (Hong Kong)
Thai
Turkish
Portuguese (Brazil), Portuguese (Portugal)
Polish
French (Canada), French (France), French (Philippines)
Korean
Hindi
※ Supported languages may be expanded, and details may change later according to company policies, etc.
