DEV Community

Cover image for Integration of Huawei ML Kit in KnowMyBoard App using Kotlin Part 2
HMS Community
HMS Community

Posted on

Integration of Huawei ML Kit in KnowMyBoard App using Kotlin Part 2

Introduction

In this article, we will learn how to integrate the Huawei ML kit and Location kit into the Android application KnowMyBoard.

If you are new to this series follow the below article.

Intermediate: Integration of Huawei Account Kit and Analytics Kit in Android App KnowMyBoard Part -1

ML Kit provides an app to easily leverage Huawei's long-term proven expertise in machine learning to support diverse artificial intelligence (AI) applications throughout a wide range of industries.

ML kit provides various services, in this application, we will be integrating its text-related service like text recognition, text detection, and text translation services, which helps in achieving the goal of the application.

Location kit SDK for Android offers location-related APIs for Android apps. These APIs mainly relate to 6 functions fused location, activity identification, geofence, high-precision location, indoor location, and geocoding. This mode is applicable to mobile phones and Huawei tablets. We are using the Location kit to get the location of the user.

Hardware Requirements

A computer (desktop or laptop) running Windows 10.
Android phone (with the USB cable), which is used for debugging.
Java JDK 1.8 or later.
Android Studio software or Visual Studio or Code installed.
HMS Core (APK) 4.X or later
Integration steps

Step 1. Huawei developer account and complete identity verification in Huawei developer website, refer to register Huawei ID.

Step 2. Create a project in AppGallery Connect

Step 3. Adding HMS Core SDK

Let's start coding

MainActivity.kt

package com.huawei.hms.knowmyboard.dtse.activity.view

import com.huawei.hms.knowmyboard.dtse.activity.app.MyApplication.Companion.activity
import androidx.appcompat.app.AppCompatActivity
import com.huawei.hms.knowmyboard.dtse.activity.viewmodel.LoginViewModel
import com.huawei.hms.mlsdk.text.MLTextAnalyzer
import android.graphics.Bitmap
import com.huawei.hms.mlsdk.langdetect.local.MLLocalLangDetector
import com.huawei.hms.mlsdk.translate.local.MLLocalTranslator
import android.app.ProgressDialog
import android.os.Bundle
import androidx.lifecycle.ViewModelProvider
import android.content.Intent
import com.huawei.hms.support.account.AccountAuthManager
import com.google.gson.Gson
import com.huawei.hms.common.ApiException
import android.provider.MediaStore
import com.huawei.hms.mlsdk.common.MLFrame
import com.huawei.hms.mlsdk.text.MLText
import com.huawei.hmf.tasks.OnSuccessListener
import com.huawei.hms.mlsdk.langdetect.MLLangDetectorFactory
import com.huawei.hms.mlsdk.langdetect.local.MLLocalLangDetectorSetting
import com.huawei.hmf.tasks.OnFailureListener
import android.content.DialogInterface
import android.net.Uri
import android.util.Log
import androidx.appcompat.app.AlertDialog
import com.huawei.hms.knowmyboard.dtse.R
import com.huawei.hms.knowmyboard.dtse.activity.model.UserData
import com.huawei.hms.mlsdk.common.MLApplication
import com.huawei.hms.mlsdk.translate.local.MLLocalTranslateSetting
import com.huawei.hms.mlsdk.translate.MLTranslatorFactory
import com.huawei.hms.mlsdk.model.download.MLModelDownloadStrategy
import com.huawei.hms.mlsdk.model.download.MLModelDownloadListener
import com.huawei.hms.mlsdk.text.MLLocalTextSetting
import com.huawei.hms.mlsdk.MLAnalyzerFactory
import java.io.IOException
import java.lang.Exception
import java.util.ArrayList

class MainActivity() : AppCompatActivity() {
    var loginViewModel: LoginViewModel? = null
    private var mTextAnalyzer: MLTextAnalyzer? = null
    var imagePath: Uri? = null
    var bitmap: Bitmap? = null
    var result = ArrayList<String>()
    var myLocalLangDetector: MLLocalLangDetector? = null
    var myLocalTranslator: MLLocalTranslator? = null
    var textRecognized: String? = null
    var progressDialog: ProgressDialog? = null
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        loginViewModel = ViewModelProvider(this).get(LoginViewModel::class.java)
        activity = this
        progressDialog = ProgressDialog(this)
        progressDialog!!.setCancelable(false)
    }

    override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
        // Process the authorization result to obtain the authorization code from AuthAccount.
        super.onActivityResult(requestCode, resultCode, data)
        if (requestCode == 8888) {
            val authAccountTask = AccountAuthManager.parseAuthResultFromIntent(data)
            if (authAccountTask.isSuccessful) {
                // The sign-in is successful, and the user's ID information and authorization code are obtained.
                val authAccount = authAccountTask.result
                val userData = UserData()
                userData.accessToken = authAccount.accessToken
                userData.countryCode = authAccount.countryCode
                userData.displayName = authAccount.displayName
                userData.email = authAccount.email
                userData.familyName = authAccount.familyName
                userData.givenName = authAccount.givenName
                userData.idToken = authAccount.idToken
                userData.openId = authAccount.openId
                userData.uid = authAccount.uid
                userData.photoUriString = authAccount.avatarUri.toString()
                userData.unionId = authAccount.unionId
                val gson = Gson()
                Log.e("TAG", "sign in success : " + gson.toJson(authAccount))
                loginViewModel = ViewModelProvider(this@MainActivity).get(
                    LoginViewModel::class.java
                )
                loginViewModel!!.sendData(authAccount.displayName)
                progressDialog!!.dismiss()
            } else {
                // The sign-in failed.
                Log.e(
                    "TAG",
                    "sign in failed:" + (authAccountTask.exception as ApiException).statusCode
                )
                progressDialog!!.dismiss()
            }
        }
        if ((requestCode == 2323) && (resultCode == RESULT_OK) && (data != null)) {
            progressDialog!!.setMessage("Initializing text detection..")
            progressDialog!!.show()
            imagePath = data.data
            try {
                bitmap = MediaStore.Images.Media.getBitmap(this.contentResolver, imagePath)
            } catch (e: IOException) {
                e.printStackTrace()
                Log.e("TAG", " BITMAP ERROR")
            }
            Log.d("TAG", "Path " + imagePath!!.path)
            try {
                val selectedBitmap = MediaStore.Images.Media.getBitmap(contentResolver, imagePath)
                asyncAnalyzeText(selectedBitmap)
            } catch (e: IOException) {
                e.printStackTrace()
                progressDialog!!.dismiss()
            }
        }
    }

    private fun asyncAnalyzeText(bitmap: Bitmap) {
        if (mTextAnalyzer == null) {
            createMLTextAnalyzer()
        }
        val frame = MLFrame.fromBitmap(bitmap)
        val task = mTextAnalyzer!!.asyncAnalyseFrame(frame)
        task.addOnSuccessListener(object : OnSuccessListener<MLText> {
            override fun onSuccess(text: MLText) {
                progressDialog!!.setMessage("Initializing language detection..")
                Log.d("TAG", "#==>" + text.stringValue)
                textRecognized = text.stringValue.trim { it <= ' ' }
                if (!textRecognized!!.isEmpty()) {
                    // Create a local language detector.
                    val factory = MLLangDetectorFactory.getInstance()
                    val setting =
                        MLLocalLangDetectorSetting.Factory() // Set the minimum confidence threshold for language detection.
                            .setTrustedThreshold(0.01f)
                            .create()
                    myLocalLangDetector = factory.getLocalLangDetector(setting)
                    val firstBestDetectTask = myLocalLangDetector?.firstBestDetect(textRecognized)
                    firstBestDetectTask?.addOnSuccessListener(OnSuccessListener { languageDetected ->
                        progressDialog!!.setMessage("Initializing text translation..")
                        // Processing logic for detection success.
                        Log.d("TAG", "Lang detect :$languageDetected")
                        textTranslate(languageDetected, textRecognized!!, bitmap)
                    })?.addOnFailureListener(object : OnFailureListener {
                        override fun onFailure(e: Exception) {
                            // Processing logic for detection failure.
                            Log.e("TAG", "Lang detect error:" + e.message)
                        }
                    })
                } else {
                    progressDialog!!.dismiss()
                    showErrorDialog("Failed to recognize text.")
                }
            }
        }).addOnFailureListener(object : OnFailureListener {
            override fun onFailure(e: Exception) {
                Log.e("TAG", "#==>" + e.message)
            }
        })
    }

    private fun showErrorDialog(msg: String) {
        val alertDialog = AlertDialog.Builder(this).create()
        alertDialog.setTitle("Error")
        alertDialog.setMessage(msg)
        alertDialog.setButton(
            AlertDialog.BUTTON_POSITIVE,
            "OK",
            object : DialogInterface.OnClickListener {
                override fun onClick(dialog: DialogInterface, which: Int) {
                    dialog.dismiss()
                }
            })
        alertDialog.show()
    }

    private fun textTranslate(languageDetected: String, textRecognized: String, uri: Bitmap) {
        MLApplication.initialize(application)
        MLApplication.getInstance().apiKey =
            "DAEDAF48ZIMI4ettQdTfCKlXgaln/E+TO/PrsX+LpP2BubkmED/iC0iVEps5vfx1ol27rHvuwiq64YphpPkGYWbf9La8XjnvC9qhwQ=="

        // Create an offline translator.
        val setting =
            MLLocalTranslateSetting.Factory() // Set the source language code. The ISO 639-1 standard is used. This parameter is mandatory. If this parameter is not set, an error may occur.
                .setSourceLangCode(languageDetected) // Set the target language code. The ISO 639-1 standard is used. This parameter is mandatory. If this parameter is not set, an error may occur.
                .setTargetLangCode("en")
                .create()
        myLocalTranslator = MLTranslatorFactory.getInstance().getLocalTranslator(setting)
        // Set the model download policy.
        val downloadStrategy = MLModelDownloadStrategy.Factory()
            .needWifi() // It is recommended that you download the package in a Wi-Fi environment.
            .create()
        // Create a download progress listener.
        val modelDownloadListener: MLModelDownloadListener = object : MLModelDownloadListener {
            override fun onProcess(alreadyDownLength: Long, totalLength: Long) {
                runOnUiThread(object : Runnable {
                    override fun run() {
                        // Display the download progress or perform other operations.
                    }
                })
            }
        }
        myLocalTranslator?.preparedModel(downloadStrategy, modelDownloadListener)
            ?.addOnSuccessListener(object : OnSuccessListener<Void?> {
                override fun onSuccess(aVoid: Void?) {
                    // Called when the model package is successfully downloaded.
                    // input is a string of less than 5000 characters.
                    val task = myLocalTranslator?.asyncTranslate(textRecognized)
                    // Before translation, ensure that the models have been successfully downloaded.
                    task?.addOnSuccessListener(object : OnSuccessListener<String> {
                        override fun onSuccess(translated: String) {
                            // Processing logic for detection success.
                            result.clear()
                            result.add(languageDetected.trim { it <= ' ' })
                            result.add(textRecognized.trim { it <= ' ' })
                            result.add(translated.trim { it <= ' ' })
                            loginViewModel!!.setImage(uri)
                            loginViewModel!!.setTextRecognized(result)
                            progressDialog!!.dismiss()
                        }
                    })?.addOnFailureListener(object : OnFailureListener {
                        override fun onFailure(e: Exception) {
                            // Processing logic for detection failure.
                            progressDialog!!.dismiss()
                        }
                    })
                }
            })?.addOnFailureListener(object : OnFailureListener {
            override fun onFailure(e: Exception) {
                // Called when the model package fails to be downloaded.
                progressDialog!!.dismiss()
            }
        })
    }

    private fun createMLTextAnalyzer() {
        val setting = MLLocalTextSetting.Factory()
            .setOCRMode(MLLocalTextSetting.OCR_DETECT_MODE)
            .create()
        mTextAnalyzer = MLAnalyzerFactory.getInstance().getLocalTextAnalyzer(setting)
    }

    override fun onStop() {
        if (myLocalLangDetector != null) {
            myLocalLangDetector!!.stop()
        }
        if (myLocalTranslator != null) {
            myLocalTranslator!!.stop()
        }
        if (progressDialog != null) {
            progressDialog!!.dismiss()
        }
        super.onStop()
    }
}
Enter fullscreen mode Exit fullscreen mode

LoginViewModel.kt

package com.huawei.hms.knowmyboard.dtse.activity.viewmodel

import android.app.Application
import com.huawei.hms.knowmyboard.dtse.activity.app.MyApplication.Companion.activity
import androidx.lifecycle.AndroidViewModel
import com.huawei.hms.support.account.service.AccountAuthService
import androidx.lifecycle.MutableLiveData
import android.graphics.Bitmap
import android.util.Log
import android.view.View
import com.huawei.hms.location.LocationResult
import androidx.lifecycle.LiveData
import android.widget.Toast
import com.huawei.hms.support.account.request.AccountAuthParams
import com.huawei.hms.support.account.request.AccountAuthParamsHelper
import com.huawei.hms.support.account.AccountAuthManager
import com.huawei.hms.common.ApiException
import java.util.ArrayList

class LoginViewModel(application: Application) : AndroidViewModel(application) {
    var service: AccountAuthService? = null
    private val message = MutableLiveData<String>()
    val textRecongnized = MutableLiveData<ArrayList<String>>()
    val imagePath = MutableLiveData<Bitmap>()
    val locationResult = MutableLiveData<LocationResult>()
    fun sendData(msg: String) {
        message.value = msg
    }

    fun getMessage(): LiveData<String> {
        return message
    }

    fun setImage(imagePath: Bitmap) {
        this.imagePath.value = imagePath
    }

    fun setLocationResult(locationResult: LocationResult) {
        this.locationResult.value = locationResult
    }

    fun setTextRecognized(textRecognized: ArrayList<String>) {
        this.textRecongnized.value = textRecognized
    }

    fun logoutHuaweiID() {
        if (service != null) {
            service!!.signOut()
            sendData("KnowMyBoard")
            Toast.makeText(getApplication(), "You are logged out from Huawei ID", Toast.LENGTH_LONG)
                .show()
        }
    }

    fun loginClicked(view: View?) {
        val authParams =
            AccountAuthParamsHelper(AccountAuthParams.DEFAULT_AUTH_REQUEST_PARAM).setAuthorizationCode()
                .createParams()
        service = AccountAuthManager.getService(activity, authParams)

        activity!!.startActivityForResult(service?.signInIntent, 8888)
    }

    fun cancelAuthorization() {
        if (service != null) {
            // service indicates the AccountAuthService instance generated using the getService method during the sign-in authorization.
            service!!.cancelAuthorization().addOnCompleteListener { task ->
                if (task.isSuccessful) {
                    // Processing after a successful authorization cancellation.
                    Log.i("TAG", "onSuccess: ")
                    sendData("KnowMyBoard")
                    Toast.makeText(getApplication(), "Cancelled authorization", Toast.LENGTH_LONG)
                        .show()
                } else {
                    // Handle the exception.
                    val exception = task.exception
                    if (exception is ApiException) {
                        val statusCode = exception.statusCode
                        Log.i("TAG", "onFailure: $statusCode")
                        Toast.makeText(
                            getApplication(),
                            "Failed to cancel authorization",
                            Toast.LENGTH_LONG
                        ).show()
                    }
                }
            }
        } else {
            Toast.makeText(getApplication(), "Login required", Toast.LENGTH_LONG).show()
        }
    }

    fun onClickScan() {
        Log.d("TAG", "...Scan...")
    }
}
Enter fullscreen mode Exit fullscreen mode

RequestLocationData.kt

package com.huawei.hms.knowmyboard.dtse.activity.util

import android.Manifest
import androidx.fragment.app.FragmentActivity
import com.huawei.hms.knowmyboard.dtse.activity.viewmodel.LoginViewModel
import android.app.Activity
import android.content.Context
import com.huawei.hmf.tasks.OnSuccessListener
import com.huawei.hms.knowmyboard.dtse.activity.util.RequestLocationData
import com.huawei.hmf.tasks.OnFailureListener
import android.os.Build
import androidx.core.app.ActivityCompat
import android.content.pm.PackageManager
import com.google.gson.Gson
import android.os.Looper
import android.location.Geocoder
import android.util.Log
import com.huawei.hms.location.*
import java.io.IOException
import java.lang.StringBuilder
import java.util.*

class RequestLocationData(
    context: Context?,
    activity: FragmentActivity?,
    loginViewModel: LoginViewModel?
) {
    var settingsClient: SettingsClient? = null
    private var isLocationSettingSuccess = 0
    private var myLocationRequest: LocationRequest? = null

    // Define a fusedLocationProviderClient object.
    private var fusedLocationProviderClient: FusedLocationProviderClient? = null
    var myLocationCallback: LocationCallback? = null
    var context: Context? = null
    var activity: Activity? = null
    var locationResult: LocationResult? = null
    var loginViewModel: LoginViewModel? = null
    fun initFusionLocationProviderClint() {
        // Instantiate the fusedLocationProviderClient object.
        fusedLocationProviderClient = LocationServices.getFusedLocationProviderClient(activity)
        settingsClient = LocationServices.getSettingsClient(activity)
    }

    fun checkDeviceLocationSettings() {
        val builder = LocationSettingsRequest.Builder()
        myLocationRequest = LocationRequest()
        builder.addLocationRequest(myLocationRequest)
        val locationSettingsRequest = builder.build()
        // Check the device location settings.
        settingsClient!!.checkLocationSettings(locationSettingsRequest) // Define the listener for success in calling the API for checking device location settings.
            .addOnSuccessListener { locationSettingsResponse: LocationSettingsResponse ->
                val locationSettingsStates = locationSettingsResponse.locationSettingsStates
                val stringBuilder = StringBuilder()
                // Check whether the location function is enabled.
                stringBuilder.append(",\nisLocationUsable=")
                    .append(locationSettingsStates.isLocationUsable)
                // Check whether HMS Core (APK) is available.
                stringBuilder.append(",\nisHMSLocationUsable=")
                    .append(locationSettingsStates.isHMSLocationUsable)
                Log.i(TAG, "checkLocationSetting onComplete:$stringBuilder")
                // Set the location type.
                myLocationRequest!!.priority = LocationRequest.PRIORITY_HIGH_ACCURACY
                // Set the number of location updates to 1.
                myLocationRequest!!.numUpdates = 1
                isLocationSettingSuccess = 1
            } // Define callback for failure in checking the device location settings.
            .addOnFailureListener { e -> Log.i(TAG, "checkLocationSetting onFailure:" + e.message) }
    }

    fun checkPermission() {
        // Dynamically apply for required permissions if the API level is 28 or lower.
        if (Build.VERSION.SDK_INT <= Build.VERSION_CODES.P) {
            Log.i(TAG, "android sdk <= 28 Q")
            if (ActivityCompat.checkSelfPermission(
                    context!!,
                    Manifest.permission.ACCESS_FINE_LOCATION
                ) != PackageManager.PERMISSION_GRANTED
                && ActivityCompat.checkSelfPermission(
                    context!!,
                    Manifest.permission.ACCESS_COARSE_LOCATION
                ) != PackageManager.PERMISSION_GRANTED
            ) {
                val strings = arrayOf(
                    Manifest.permission.READ_EXTERNAL_STORAGE,
                    Manifest.permission.MANAGE_MEDIA,
                    Manifest.permission.MEDIA_CONTENT_CONTROL,
                    Manifest.permission.ACCESS_FINE_LOCATION,
                    Manifest.permission.ACCESS_COARSE_LOCATION
                )
                ActivityCompat.requestPermissions(activity!!, strings, 1)
            }
        } else {
            // Dynamically apply for the android.permission.ACCESS_BACKGROUND_LOCATION permission in addition to the preceding permissions if the API level is higher than 28.
            if (ActivityCompat.checkSelfPermission(
                    activity!!,
                    Manifest.permission.ACCESS_FINE_LOCATION
                ) != PackageManager.PERMISSION_GRANTED && ActivityCompat.checkSelfPermission(
                    context!!,
                    Manifest.permission.ACCESS_COARSE_LOCATION
                ) != PackageManager.PERMISSION_GRANTED && ActivityCompat.checkSelfPermission(
                    context!!,
                    "android.permission.ACCESS_BACKGROUND_LOCATION"
                ) != PackageManager.PERMISSION_GRANTED
            ) {
                val strings = arrayOf(
                    Manifest.permission.ACCESS_FINE_LOCATION,
                    Manifest.permission.ACCESS_COARSE_LOCATION,
                    Manifest.permission.MEDIA_CONTENT_CONTROL,
                    Manifest.permission.MANAGE_MEDIA,
                    "android.permission.ACCESS_BACKGROUND_LOCATION"
                )
                ActivityCompat.requestPermissions(activity!!, strings, 2)
            }
        }
    }

    fun refreshLocation(): LocationResult? {
        Log.d(TAG, "Refreshing location")
        if (isLocationSettingSuccess == 1) {
            myLocationCallback = object : LocationCallback() {
                override fun onLocationResult(locationResult: LocationResult) {
                    if (locationResult != null) {
                        val gson = Gson()
                        Log.d(
                            TAG,
                            " Location data :" + locationResult.lastLocation.latitude + " : " + locationResult.lastLocation.longitude
                        )
                        Log.d(TAG, " Location data :" + gson.toJson(locationResult.lastHWLocation))
                        Log.d(TAG, " Location data :" + locationResult.lastHWLocation.countryName)
                        Log.d(TAG, " Location data :" + locationResult.lastHWLocation.latitude)
                        Log.d(TAG, " Location data :" + locationResult.lastHWLocation.longitude)
                        // binding.textDetected.setText("Latitude " + locationResult.getLastHWLocation().getLatitude() + " Longitude " + locationResult.getLastHWLocation().getLongitude());
                        getGeoCoderValues(
                            locationResult.lastHWLocation.latitude,
                            locationResult.lastHWLocation.longitude
                        )
                        //locationResult = locationResult1;
                        loginViewModel!!.setLocationResult(locationResult)
                    }
                }
            }
            fusedLocationProviderClient!!.requestLocationUpdates(
                myLocationRequest,
                myLocationCallback,
                Looper.getMainLooper()
            )
        } else {
            Log.d(TAG, "Failed to get location settings")
        }
        return locationResult
    }

    fun disableLocationData() {
        fusedLocationProviderClient!!.disableBackgroundLocation()
        fusedLocationProviderClient!!.removeLocationUpdates(myLocationCallback)
    }

    private fun getGeoCoderValues(latitude: Double, longitude: Double) {
        getAddress(context, latitude, longitude)
        /*  Geocoder geocoder;
        List<Address> addresses;
        Locale locale = new Locale("en", "IN");
        geocoder = new Geocoder(getContext(), locale);

        try {
            addresses = geocoder.getFromLocation(latitude, longitude, 1); // Here 1 represent max location result to returned, by documents it recommended 1 to 5
            Gson gson=new Gson();
            Log.d(TAG,"Geo coder :"+gson.toJson(addresses));
        String address = addresses.get(0).getAddressLine(0); // If any additional address line present than only, check with max available address lines by getMaxAddressLineIndex()
        String city = addresses.get(0).getLocality();
        String state = addresses.get(0).getAdminArea();
        String country = addresses.get(0).getCountryName();
        String postalCode = addresses.get(0).getPostalCode();
        String knownName = addresses.get(0).getFeatureName();
        } catch (IOException e) {
            e.printStackTrace();
            Log.e(TAG,"Error while fetching Geo coder :"+e.getMessage());
        }*/
        /* Locale locale = new Locale("en", "IN");
        GeocoderService geocoderService =
                LocationServices.getGeocoderService(getActivity().getBaseContext(), locale);
        // Request reverse geocoding.
        GetFromLocationRequest getFromLocationRequest = new GetFromLocationRequest(latitude, longitude, 5);
        // Initiate reverse geocoding.
        geocoderService.getFromLocation(getFromLocationRequest)
                .addOnSuccessListener(hwLocation -> {
                    Gson gson=new Gson();
                    Log.d(TAG,"Geo coder :"+gson.toJson(hwLocation));

                })
                .addOnFailureListener(e -> {

                    Log.e(TAG,"Error while fetching Geo coder :"+e.getMessage());
                });*/
    }

    companion object {
        var TAG = "TAG"
        fun getAddress(context: Context?, LATITUDE: Double, LONGITUDE: Double) {
            //Set Address
            try {
                val geocoder = Geocoder(context, Locale.getDefault())
                val addresses = geocoder.getFromLocation(LATITUDE, LONGITUDE, 1)
                if (addresses != null && addresses.size > 0) {
                    val address =
                        addresses[0].getAddressLine(0) // If any additional address line present than only, check with max available address lines by getMaxAddressLineIndex()
                    val city = addresses[0].locality
                    val state = addresses[0].adminArea
                    val country = addresses[0].countryName
                    val postalCode = addresses[0].postalCode
                    val knownName = addresses[0].featureName // Only if available else return NULL
                    Log.d(TAG, "getAddress:  address$address")
                    Log.d(TAG, "getAddress:  city$city")
                    Log.d(TAG, "getAddress:  state$state")
                    Log.d(TAG, "getAddress:  postalCode$postalCode")
                    Log.d(TAG, "getAddress:  knownName$knownName")
                }
            } catch (e: IOException) {
                e.printStackTrace()
                Log.e(TAG, "Error while fetching Geo coder :" + e.message)
            }
        }
    }

    init {
        this.context = context
        this.activity = activity
        this@RequestLocationData.loginViewModel = loginViewModel
    }
}
Enter fullscreen mode Exit fullscreen mode

LoginFragment.kt

package com.huawei.hms.knowmyboard.dtse.activity.fragments
import com.huawei.hms.knowmyboard.dtse.activity.viewmodel.LoginViewModel
import androidx.navigation.Navigation.findNavController
import android.content.SharedPreferences
import androidx.navigation.NavController
import android.os.Bundle
import androidx.databinding.DataBindingUtil
import androidx.lifecycle.ViewModelProvider
import android.annotation.SuppressLint
import android.content.Context
import android.util.Log
import android.view.*
import androidx.fragment.app.Fragment
import com.huawei.hms.knowmyboard.dtse.R
import java.lang.Exception

class LoginFragment : Fragment() {
    var loginBinding: FragmentLoginBinding? = null
    var loginViewModel: LoginViewModel? = null
    var menu: Menu? = null
    var prefs: SharedPreferences? = null
    var editor: SharedPreferences.Editor? = null
    var navController: NavController? = null
    private val MY_PREF_NAME = "my_pref_name"
    private val TAG = "TAG"
    override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
        super.onViewCreated(view, savedInstanceState)
        navController = findNavController(view)
    }

    override fun onCreateView(
        inflater: LayoutInflater, container: ViewGroup?,
        savedInstanceState: Bundle?
    ): View? {
        loginBinding = DataBindingUtil.inflate(inflater, R.layout.fragment_login, container, false)
        loginViewModel = ViewModelProvider(requireActivity()).get(LoginViewModel::class.java)
        loginBinding?.loginViewModel = loginViewModel
        Log.d(TAG, " Pref $preferenceValue")
        if (preferenceValue == "user_name") {
            loginBinding?.btnHuaweiIdAuth?.visibility = View.VISIBLE
        } else {
            enableMenu(menu)
            requireActivity().title = preferenceValue
            loginBinding?.btnHuaweiIdAuth?.visibility = View.GONE
        }
        loginBinding?.imageLogo?.setOnClickListener { v: View? -> navController!!.navigate(R.id.action_loginFragment_to_mainFragment) }
        loginViewModel!!.getMessage().observeForever { message ->
            updateMessage(message)
            if (message != resources.getString(R.string.app_name)) {
                preferenceValue = message
                enableMenu(menu)
                loginBinding?.btnHuaweiIdAuth?.visibility = View.GONE
            } else {
                disableMenu(menu)
                loginBinding?.btnHuaweiIdAuth?.visibility = View.VISIBLE
                preferenceValue = "user_name"
            }
        }
        return loginBinding?.root
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setHasOptionsMenu(true)
    }

    override fun onCreateOptionsMenu(menu: Menu, inflater: MenuInflater) {
        menu.clear()
        super.onCreateOptionsMenu(menu, inflater)
        inflater.inflate(R.menu.main, menu)
        this.menu = menu
        disableMenu(menu)
    }

    private fun disableMenu(menu: Menu?) {
        try {
            if (menu != null) {
                if (preferenceValue == "user_name") {
                    menu.findItem(R.id.menu_login_logout).isVisible = false
                    menu.findItem(R.id.menu_cancel_auth).isVisible = false
                    requireActivity().title = resources.getString(R.string.app_name)
                } else {
                    menu.findItem(R.id.menu_login_logout).isVisible = true
                    menu.findItem(R.id.menu_cancel_auth).isVisible = true
                }
            }
        } catch (e: Exception) {
            e.printStackTrace()
        }
    }

    private fun enableMenu(menu: Menu?) {
        try {
            menu!!.findItem(R.id.menu_login_logout).isVisible = true
            menu.findItem(R.id.menu_cancel_auth).isVisible = true
        } catch (e: Exception) {
            e.printStackTrace()
        }
    }

    @SuppressLint("NonConstantResourceId")
    override fun onOptionsItemSelected(item: MenuItem): Boolean {
        when (item.itemId) {
            R.id.menu_cancel_auth -> {
                preferenceValue = "user_name"
                loginViewModel!!.cancelAuthorization()
                loginBinding!!.btnHuaweiIdAuth.visibility = View.VISIBLE
                disableMenu(menu)
                return true
            }
            R.id.menu_login_logout -> {
                preferenceValue = "user_name"
                loginViewModel!!.logoutHuaweiID()
                loginBinding!!.btnHuaweiIdAuth.visibility = View.VISIBLE
                disableMenu(menu)
                return true
            }
            else -> {}
        }
        return super.onOptionsItemSelected(item)
    }

    private fun updateMessage(msg: String?) {
        //loginBinding.txtMessage.setText(msg);
        requireActivity().title = msg
    }

    var preferenceValue: String?
        get() {
            prefs = requireActivity().getSharedPreferences(MY_PREF_NAME, Context.MODE_PRIVATE)
            return prefs?.getString("user_name", "user_name")
        }
        set(userName) {
            editor = requireActivity().getSharedPreferences(MY_PREF_NAME, Context.MODE_PRIVATE).edit()
            editor?.putString("user_name", userName)
            editor?.apply()
        }
}
Enter fullscreen mode Exit fullscreen mode

MainFragment.kt

package com.huawei.hms.knowmyboard.dtse.activity.fragments

import com.huawei.hms.knowmyboard.dtse.activity.viewmodel.LoginViewModel
import com.huawei.hms.knowmyboard.dtse.activity.util.RequestLocationData
import android.os.Bundle
import androidx.databinding.DataBindingUtil
import androidx.lifecycle.ViewModelProvider
import android.content.Intent
import android.annotation.SuppressLint
import android.util.Log
import android.view.*
import androidx.fragment.app.Fragment
import com.huawei.hms.knowmyboard.dtse.R
import com.huawei.hms.knowmyboard.dtse.databinding.FragmentMainFragmentBinding
import java.lang.Exception

class MainFragment : Fragment() {
    var binding: FragmentMainFragmentBinding? = null
    var loginViewModel: LoginViewModel? = null
    var locationData: RequestLocationData? = null
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setHasOptionsMenu(true)
    }

    override fun onCreateView(
        inflater: LayoutInflater, container: ViewGroup?,
        savedInstanceState: Bundle?
    ): View? {
        // Inflate the layout for this fragment
        binding =
            DataBindingUtil.inflate(inflater, R.layout.fragment_main_fragment, container, false)
        // settingsClient = LocationServices.getSettingsClient(getActivity());
        loginViewModel = ViewModelProvider(requireActivity()).get(LoginViewModel::class.java)
        binding?.loginViewModel = loginViewModel
        locationData = RequestLocationData(context, activity, loginViewModel)
        locationData!!.initFusionLocationProviderClint()
        locationData!!.checkPermission()
        locationData!!.checkDeviceLocationSettings()
        // checkDeviceLocationSettings();
        // Instantiate the fusedLocationProviderClient object.
        // fusedLocationProviderClient = LocationServices.getFusedLocationProviderClient(getActivity());
        binding?.buttonScan?.setOnClickListener {
            Log.d(TAG, "*******")
            //loginViewModel.setTextRecongnized("");
            scan()
        }
        loginViewModel!!.imagePath.observeForever { bitmap ->
            try {
                binding?.imageView?.setImageBitmap(bitmap)
            } catch (e: Exception) {
                e.printStackTrace()
                Log.e("TAG", "Error : " + e.message)
            }
        }
        loginViewModel!!.textRecongnized.observeForever { res ->
            Log.i("TAG", "OBSERVER : " + "Language : " + getStringResourceByName(
                res[0].trim { it <= ' ' }) +
                    " Detected text : " + res[1].trim { it <= ' ' } +
                    " Translated text : " + res[2].trim { it <= ' ' })
            binding?.textLanguage?.text = "Language : " + getStringResourceByName(res[0])
            binding?.textDetected?.text = "Detected text : " + res[1]
            binding?.textTranslated?.text = "Translated text : " + res[2]
        }
        loginViewModel!!.locationResult.observeForever { locationResult ->
            binding?.textDetected?.text =
                "Latitude " + locationResult.lastHWLocation.latitude + " Longitude " + locationResult.lastHWLocation.longitude
        }
        return binding?.root
    }

    private fun getStringResourceByName(aString: String): String {
        val packageName = requireActivity().packageName
        val resId = resources
            .getIdentifier(aString, "string", packageName)
        return if (resId == 0) {
            aString
        } else {
            getString(resId)
        }
    }

    private fun scan() {
        /* MLTextAnalyzer analyzer = new MLTextAnalyzer.Factory(getContext()).create();
        analyzer.setTransactor(new OcrDetectorProcessor());
        LensEngine lensEngine = new LensEngine.Creator(getContext(), analyzer)
                .setLensType(LensEngine.BACK_LENS)
                .applyDisplayDimension(1440, 1080)
                .applyFps(30.0f)
                .enableAutomaticFocus(true)
                .create();
        SurfaceView mSurfaceView = new SurfaceView(getContext());
        try {
            lensEngine.run(mSurfaceView.getHolder());
        } catch (IOException e) {
            // Exception handling logic.
            Log.e(TAG,e.getMessage());
        }*/
        val intent = Intent(Intent.ACTION_GET_CONTENT)
        intent.type = "image/*"
        requireActivity().startActivityForResult(intent, 2323)
    }

    override fun onCreateOptionsMenu(menu: Menu, inflater: MenuInflater) {
        menu.clear()
        super.onCreateOptionsMenu(menu, inflater)
        inflater.inflate(R.menu.main_fragment_menu, menu)
    }

    @SuppressLint("NonConstantResourceId")
    override fun onOptionsItemSelected(item: MenuItem): Boolean {
        when (item.itemId) {
            R.id.option_refresh_location -> {
                //refreshLocation();
                locationData!!.refreshLocation()
                return true
            }
        }
        return super.onOptionsItemSelected(item)
    }

    /* private void checkDeviceLocationSettings() {
        LocationSettingsRequest.Builder builder = new LocationSettingsRequest.Builder();
        myLocationRequest = new LocationRequest();
        builder.addLocationRequest(myLocationRequest);
        LocationSettingsRequest locationSettingsRequest = builder.build();
        // Check the device location settings.
        settingsClient.checkLocationSettings(locationSettingsRequest)
                // Define the listener for success in calling the API for checking device location settings.
                .addOnSuccessListener(locationSettingsResponse -> {
                    LocationSettingsStates locationSettingsStates =
                            locationSettingsResponse.getLocationSettingsStates();
                    StringBuilder stringBuilder = new StringBuilder();
                    // Check whether the location function is enabled.
                    stringBuilder.append(",\nisLocationUsable=")
                            .append(locationSettingsStates.isLocationUsable());
                    // Check whether HMS Core (APK) is available.
                    stringBuilder.append(",\nisHMSLocationUsable=")
                            .append(locationSettingsStates.isHMSLocationUsable());
                    Log.i(TAG, "checkLocationSetting onComplete:" + stringBuilder.toString());
                    // Set the location type.
                    myLocationRequest.setPriority(LocationRequest.PRIORITY_HIGH_ACCURACY);
                    // Set the number of location updates to 1.
                    myLocationRequest.setNumUpdates(1);
                    isLocationSettingSuccess = 1;


                })
                // Define callback for failure in checking the device location settings.
                .addOnFailureListener(new OnFailureListener() {
                    @Override
                    public void onFailure(Exception e) {
                        isLocationSettingSuccess = 0;
                        Log.i(TAG, "checkLocationSetting onFailure:" + e.getMessage());
                    }
                });

    }*/
    /*  private void refreshLocation() {
        Log.d(TAG, "Refreshing location");
        if (isLocationSettingSuccess == 1) {
            myLocationCallback = new LocationCallback() {
                @Override
                public void onLocationResult(LocationResult locationResult) {
                    if (locationResult != null) {
                        Gson gson = new Gson();
                        Log.d(TAG, " Location data :" + locationResult.getLastLocation().getLatitude() + " : " + locationResult.getLastLocation().getLongitude());
                        Log.d(TAG, " Location data :" + gson.toJson(locationResult.getLastHWLocation()));
                        Log.d(TAG, " Location data :" + locationResult.getLastHWLocation().getCountryName());
                        Log.d(TAG, " Location data :" + locationResult.getLastHWLocation().getLatitude());
                        Log.d(TAG, " Location data :" + locationResult.getLastHWLocation().getLongitude());
                        binding.textDetected.setText("Latitude " + locationResult.getLastHWLocation().getLatitude() + " Longitude " + locationResult.getLastHWLocation().getLongitude());
                        getGeoCoderValues(locationResult.getLastHWLocation().getLatitude(),locationResult.getLastHWLocation().getLongitude());
                    }
                }
            };
            fusedLocationProviderClient.requestLocationUpdates(myLocationRequest, myLocationCallback, Looper.getMainLooper());

        } else {
            Log.d(TAG, "Failed to get location settings");
        }

    }*/
    /*private void getGeoCoderValues(double latitude, double longitude) {
        Locale locale = new Locale("en", "in");

        GeocoderService geocoderService =
                LocationServices.getGeocoderService(getActivity().getBaseContext(), locale);
        // Request reverse geocoding.
        GetFromLocationRequest getFromLocationRequest = new GetFromLocationRequest(latitude, longitude, 5);
        // Initiate reverse geocoding.
        geocoderService.getFromLocation(getFromLocationRequest)
                .addOnSuccessListener(hwLocation -> {
                    Gson gson=new Gson();
                    Log.d(TAG,"Geo coder :"+gson.toJson(hwLocation));

                })
                .addOnFailureListener(e -> {
                    // TODO: Processing when the API call fails.
                    Log.e(TAG,"Error while fetching Geo coder :"+e.getMessage());
                });
    }*/
    /*   void checkPermission() {
        // Dynamically apply for required permissions if the API level is 28 or lower.
        if (Build.VERSION.SDK_INT <= Build.VERSION_CODES.P) {
            Log.i(TAG, "android sdk <= 28 Q");
            if (ActivityCompat.checkSelfPermission(getContext(),
                    Manifest.permission.ACCESS_FINE_LOCATION) != PackageManager.PERMISSION_GRANTED
                    && ActivityCompat.checkSelfPermission(getContext(),
                    Manifest.permission.ACCESS_COARSE_LOCATION) != PackageManager.PERMISSION_GRANTED) {
                String[] strings =
                        {Manifest.permission.READ_EXTERNAL_STORAGE,Manifest.permission.MANAGE_MEDIA,Manifest.permission.MEDIA_CONTENT_CONTROL,Manifest.permission.ACCESS_FINE_LOCATION, Manifest.permission.ACCESS_COARSE_LOCATION};
                ActivityCompat.requestPermissions(getActivity(), strings, 1);
            }
        } else {
            // Dynamically apply for the android.permission.ACCESS_BACKGROUND_LOCATION permission in addition to the preceding permissions if the API level is higher than 28.
            if (ActivityCompat.checkSelfPermission(getActivity(),
                    Manifest.permission.ACCESS_FINE_LOCATION) != PackageManager.PERMISSION_GRANTED
                    && ActivityCompat.checkSelfPermission(getContext(),
                    Manifest.permission.ACCESS_COARSE_LOCATION) != PackageManager.PERMISSION_GRANTED
                    && ActivityCompat.checkSelfPermission(getContext(),
                    "android.permission.ACCESS_BACKGROUND_LOCATION") != PackageManager.PERMISSION_GRANTED) {
                String[] strings = {android.Manifest.permission.ACCESS_FINE_LOCATION,
                        android.Manifest.permission.ACCESS_COARSE_LOCATION,Manifest.permission.MEDIA_CONTENT_CONTROL,Manifest.permission.MANAGE_MEDIA,
                        "android.permission.ACCESS_BACKGROUND_LOCATION"};
                ActivityCompat.requestPermissions(getActivity(), strings, 2);
            }
        }
    }*/
    override fun onStop() {
        super.onStop()
        locationData!!.disableLocationData()
    }

    companion object {
        var TAG = "TAG"
    }
}
Enter fullscreen mode Exit fullscreen mode

Result

Image description

Tricks and Tips

Makes sure that agconnect-services.json file added.
Make sure required dependencies are added
Make sure that ML service is enabled in AGC
Images has clear visibility of text

Conclusion

In this article, we have learnt how to integrate the Huawei Location kit and ML kit into the Android application KnowMyBoard. And also we have learnt how to convert the image to text Using HMS ML service, In my previous article I have written an article about account kit please go through the previous article which is available in the introduction section. barrier.

Reference

ML Kit – Training video

Location Kit – Training video

Top comments (0)