How to use Machine Learning to quickly develop a photo translation app

irene83018774 profile image Irene ・5 min read

Photo translation app is quite useful when traveling abroad and this article will help the developers build this app in short time

We use HUAWEi Mlkit help to build this app and this will largely accelerate the whole development process.


There must be a lot of friends who like to travel. Sometimes it's better to go abroad for a tour. Before the tour, we will make all kinds of strategies for eating, wearing, living, traveling and playing routes.

Imaginary tourism:

Before departure, the imagined tourist destination may have beautiful buildings:
Alt Text
delicious food:
Alt Text
beautiful women:
Alt Text

Actual tourism:

But in reality, if you go to a place where the language is different from ur mother tongue, you may encounter the following problems:
A confusing map
Alt Text
Unreadable menu
Alt Text
Street sign
Alt Text
Various goods:
Alt Text
That's too hard to travel abroad without any translation tool !!!

Photo translator will help you

With text recognition and translation services, none of the above is a problem. There are only two steps to complete the development of photo translation small application:

Text recognition

First take a photo and then send the image to Huawei HMS ml kit text recognition service for text recognition

Huawei's text recognition service provides offline SDK (end side) and cloud side at the same time. The end side is free and can be detected in real time, and the cloud side recognition type and accuracy are higher. In this actual battle, we use the capabilities provided by cloud side.

Photo translation app development

1 Development preparation

Due to the use of cloud services, it is necessary to register the developer account with Huawei's developer alliance and open these services in the cloud. Here we will not go into details, just follow the operation steps of the official appgallery connect configuration and service opening:
Registered developer, open service reference please go to:

1.1 add Maven in project level gradle

Open the Android studio project level build.gradle file.
Alt Text
Add the maven address

buildscript {
    repositories {        
        maven {url 'http://developer.huawei.com/repo/'}
    }    }allprojects {
    repositories {       
        maven { url 'http://developer.huawei.com/repo/'}

1.2 add SDK dependency in application level build.gradle

Alt Text

    implementation 'com.huawei.hms:ml-computer-vision:'
    implementation 'com.huawei.hms:ml-computer-translate:'}

1.3 apply for camera and storage permission in Android manifest.xml file

<uses-permission android:name="android.permission.CAMERA" /><uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /><uses-feature android:name="android.hardware.camera" /><uses-feature android:name="android.hardware.camera.autofocus" />

Two key steps of code development

2.1 dynamic authority application

private static final int CAMERA_PERMISSION_CODE = 1; @Override 
public void onCreate(Bundle savedInstanceState) { 
    // Checking camera permission   
   if (!allPermissionsGranted()) {

2.2 create a cloud text analyzer. You can create a text analyzer from the text detection configurator "mlremotetextsetting".

MLRemoteTextSetting setting = (new MLRemoteTextSetting.Factory()).
         setTextDensityScene(MLRemoteTextSetting.OCR_LOOSE_SCENE).create();this.textAnalyzer = MLAnalyzerFactory.getInstance().getRemoteTextAnalyzer(setting);

2.3 create "mlframe" object through android.graphics.bitmap for analyzer to detect pictures.

MLFrame mlFrame = new MLFrame.Creator().setBitmap(this.originBitmap).create();

2.4 call "asyncanalyseframe" method for text detection.

Task<MLText> task = this.textAnalyzer.asyncAnalyseFrame(mlFrame);
            task.addOnSuccessListener(new OnSuccessListener<MLText>() {
                @Override                public void onSuccess(MLText mlText) {
                    // Transacting logic for segment success.
                    if (mlText != null) {
                    } else {
            }).addOnFailureListener(new OnFailureListener() {
                @Override                public void onFailure(Exception e) {
                    // Transacting logic for segment failure.

2.5 create a text translator. You can create a translator through class "mlremotetranslatesetting".

MLRemoteTranslateSetting.Factory factory = new MLRemoteTranslateSetting
                // Set the target language code. The ISO 639-1 standard is used.
        if (!this.srcLanguage.equals("AUTO")) {
            // Set the source language code. The ISO 639-1 standard is used.
        this.translator = MLTranslatorFactory.getInstance().getRemoteTranslator(factory.create());

2.6 call "asyncanalyseframe" method to translate the content obtained by text recognition.

final Task<String> task = translator.asyncTranslate(this.sourceText);
        task.addOnSuccessListener(new OnSuccessListener<String>() {
            @Override            public void onSuccess(String text) {
                if (text != null) {
                } else {
        }).addOnFailureListener(new OnFailureListener() {
            @Override            public void onFailure(Exception e) {

2.7 release resources after translation.

if (this.textAnalyzer != null) {
            try {
            } catch (IOException e) {
                SmartLog.e(RemoteTranslateActivity.TAG, "Stop analyzer failed: " + e.getMessage());
        if (this.translator != null) {

3 source code

The demo source code has been uploaded to GitHub(the project directory is: Photo translate). You can do scene based optimization for reference.

4 demo

Alt Text

5 Brainstorming

The app development demonstrats how to use the two cloud side capabilities of Huawei HMS ml kit, text recognition and translation. Huawei's text recognition and translation can also help developers to do many other interesting and powerful functions, such as:

[general text recognition]

  1. text recognition of bus license plate
  2. Text recognition in document reading [card recognition]
  3. The card number of the bank card can be identified through text recognition, which is used in the scenarios such as bank card binding, etc
  4. Of course, in addition to identifying bank cards, you can also identify various card numbers in your life, such as membership cards and preferential cards
  5. In addition, it can also realize the identification of ID card, Hong Kong and Macao pass and other certificate numbers


  1. Signpost and signboard translation
  2. Document translation
  3. Web page translation, such as identifying the language type of the comment area of the website and translating it into the language of the corresponding country;
  4. Introduction and translation of overseas products
  5. Translation of restaurant order menu


Previous link:
NO. 1:One article to understand Huawei HMS ML Kit text recognition, bank card recognition, general card identification

NO.2:Integrating MLkit and publishing ur app on Huawei AppGallery

NO.3.:Comparison Between Zxing and Huawei HMS Scan Kit

Posted on by:


markdown guide