How to Recognize 17,000+ Landmarks With Machine Learning

Ever seen a breathtaking landmark or scenery while flipping through a book or magazine, and been frustrated because you don't know what it's called or where it is? Wouldn't it be great if there was an app that could tell you what you're seeing! Fortunately, there's our ML Kit. It comes with a landmark recognition service and makes it remarkably easy to develop such an app. So let's take a look at how to use this service!

Introduction to Landmark Recognition​

The landmark recognition service enables you to obtain the landmark name, landmark longitude and latitude, and even a confidence value of the input image. When you input an image for recognition, a confidence value will be provided whereby a higher confidence value indicates that the landmark in the input image is more likely to be recognized. You can then use this information to create a highly-personalized experience for your users. Currently, the service is capable of recognizing more than 17,000 landmarks around the world.

When using landmark recognition, the device calls the on-cloud API for detection, and the detection algorithm model runs on the cloud. You'll need to ensure that the device is connected to the Internet while using this service.

Preparations​

Configuring the development environment

1. Create an app in AppGallery Connect.
   
   For details, see Getting Started with Android.

2. Enable ML Kit.
   
   Click here for more details.

3. Download the agconnect-services.json file, which is automatically generated after the app is created. Copy it to the app directory of the Android Studio project.
   
   

4. Configure the Maven repository address for the HMS Core SDK.

5. Integrate the landmark recognition SDK.
   Configure the SDK in the build.gradle file in the app directory.
   

// Import the landmark recognition SDK.
implementation 'com.huawei.hms:ml-computer-vision-cloud:2.0.5.304'

Add the AppGallery Connect plugin configuration as needed through either of the following methods:
Method 1: Add the following information under the declaration in the file header:

apply plugin: 'com.android.application'
apply plugin: 'com.huawei.agconnect'

Method 2: Add the plugin configuration in the plugins block:

plugins {
    id 'com.android.application'
    id 'com.huawei.agconnect'
}

Code Development

1. Obtain the camera permission to use the camera. (Mandatory) Set the static permission.

<uses-permission android:name="android.permission.CAMERA" />

(Mandatory) Obtain the permission dynamically.

ActivityCompat.requestPermissions(
        this, new String[]{Manifest.permission. CAMERA

}, 1);

1. Set the API key. This service runs on the cloud, meaning an API key is required to set the cloud authentication information for the app. This step is mandatory, and failure to complete it will result in an error being reported when the app is running.

// Set the API key to access the on-cloud services.
private void setApiKey() {

// Parse the agconnect-services.json file to obtain its information.
AGConnectServicesConfig config = AGConnectServicesConfig.fromContext(getApplication());
// Sets the API key.
MLApplication.getInstance().setApiKey(config.getString("client/api_key"));


    }

1. Create a landmark analyzer through either of the following methods:

// Method 1: Use default parameter settings.

MLRemoteLandmarkAnalyzer analyzer = MLAnalyzerFactory.getInstance().getRemoteLandmarkAnalyzer();

// Method 2: Use customized parameter settings through the MLRemoteLandmarkAnalyzerSetting class.



/**
 * Use custom parameter settings.
 * setLargestNumOfReturns indicates the maximum number of recognition results.
 * setPatternType indicates the analyzer mode.
 * MLRemoteLandmarkAnalyzerSetting.STEADY_PATTERN: The value 1 indicates the stable mode.
 * MLRemoteLandmarkAnalyzerSetting.NEWEST_PATTERN: The value 2 indicates the latest mode.
 */
private void initLandMarkAnalyzer() {
    settings = new MLRemoteLandmarkAnalyzerSetting.Factory()
            .setLargestNumOfReturns(1)
            .setPatternType(MLRemoteLandmarkAnalyzerSetting.STEADY_PATTERN)
            .create();
    analyzer = MLAnalyzerFactory.getInstance().getRemoteLandmarkAnalyzer(settings);
}

1. Convert the image collected from the camera or album to a bitmap. This is not provided by the landmark recognition SDK, so you'll need to implement it on your own.

// Select an image.
private void selectLocalImage() {
    Intent intent = new Intent(Intent.ACTION_PICK, null);
    intent.setDataAndType(MediaStore.Images.Media.EXTERNAL_CONTENT_URI, "image/*");
    startActivityForResult(intent, REQUEST_SELECT_IMAGE);
}

Enable the landmark recognition service in the callback.

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    super.onActivityResult(requestCode, resultCode, data);
    // Image selection succeeded.
    if (requestCode == REQUEST_SELECT_IMAGE && resultCode == RESULT_OK) {
        if (data != null) {
            // Obtain the image URI through getData().              imageUri = data.getData();

// Implement the BitmapUtils class by yourself. Obtain the bitmap of the image with its URI.

bitmap = BitmapUtils.loadFromPath(this, imageUri, getMaxWidthOfImage(), getMaxHeightOfImage());
        }
        // Start landmark recognition.
        startAnalyzerImg(bitmap);
    }
}

1. Start landmark recognition after obtaining the bitmap of the image. As this service runs on the cloud, a poor network connection may slow down data transmission. Therefore, it's recommended that you add a mask to the bitmap prior to landmark recognition.

// Start landmark recognition.
private void startAnalyzerImg(Bitmap bitmap) {
    if (imageUri == null) {
        return;
    }
    // Add a mask.
    progressBar.setVisibility(View.VISIBLE);
    img_analyzer_landmark.setImageBitmap(bitmap);

    // Create an MLFrame object using android.graphics.Bitmap. JPG, JPEG, PNG, and BMP images are supported. It is recommended that the image size be greater than or equal to 640 x 640 px.
    MLFrame mlFrame = new MLFrame.Creator().setBitmap(bitmap).create();
    Task<List<MLRemoteLandmark>> task = analyzer.asyncAnalyseFrame(mlFrame);
    task.addOnSuccessListener(new OnSuccessListener<List<MLRemoteLandmark>>() {
        public void onSuccess(List<MLRemoteLandmark> landmarkResults) {
            progressBar.setVisibility(View.GONE);
            // Called upon recognition success.
            Log.d("BitMapUtils", landmarkResults.get(0).getLandmark());
        }
    }).addOnFailureListener(new OnFailureListener() {
        public void onFailure(Exception e) {
            progressBar.setVisibility(View.GONE);
            // Called upon recognition failure.
            // Recognition failure.
            try {
                MLException mlException = (MLException) e;
                // Obtain the result code. You can process the result code and set a different prompt for users for each result code.
                int errorCode = mlException.getErrCode();
                // Obtain the error message. You can quickly locate the fault based on the result code.
                String errorMessage = mlException.getMessage();
                // Record the code and message of the error in the log.
                Log.d("BitMapUtils", "errorCode: " + errorCode + "; errorMessage: " + errorMessage);
            } catch (Exception error) {
                // Handle the conversion error.
            }
        }
    });
}

Testing the App​

The following illustrates how the service works, using the Oriental Pearl Tower in Shanghai and Pyramid of Menkaure as examples:

More Information​

1. Before performing landmark recognition, set the API key to set the cloud authentication information for the app. Otherwise, an error will be reported while the app is running.
2. Landmark recognition runs on the cloud, so completion may be slow. It is recommended that you add a mask before performing landmark recognition.

References​

For more details, you can go to:
ML Kit official website
ML Kit Development Documentation page, to find the documents you need
Reddit to join our developer discussion
GitHub to download ML Kit sample codes
Stack Overflow to solve any integration problems