Overview
In this article, I will create a Doctor Consult Demo App along with the integration of ML Product Visual Search Api. Which provides an easy interface to consult with doctor. Users can scan their prescriptions using Product Visual Search Api.
Previous Articles Link:
https://forums.developer.huawei.com/forumPortal/en/topic/0201829733289720014?fid=0101187876626530001
https://forums.developer.huawei.com/forumPortal/en/topic/0201817617825540005?fid=0101187876626530001
https://forums.developer.huawei.com/forumPortal/en/topic/0201811543541800017?fid=0101187876626530001
HMS Core Map Service Introduction
ML Kit allows your apps to easily leverage Huawei's long-term proven expertise in machine learning to support diverse artificial intelligence (AI) applications throughout a wide range of industries. Thanks to Huawei's technology accumulation, ML Kit provides diversified leading machine learning capabilities that are easy to use, helping you to develop various AI apps.
Product Visual Search: This service searches for the same or similar products in the pre-established product image library based on a product photo taken by a user, and returns the IDs of those products and related information. In addition, to better manage products in real time, this service supports offline product import, online product adding, deletion, modification, and query, and product distribution.
Prerequisite
1.Huawei Phone EMUI 3.0 or later.
2.Non-Huawei phones Android 4.4 or later (API level 19 or higher).
3.Android Studio.
4.AppGallery Account.
App Gallery Integration process
1.Sign In and Create or Choose a project on AppGallery Connect portal.
2.Navigate to Project settings and download the configuration file.
3.Navigate to General Information, and then provide Data Storage location.
App Development
1.Create A New Project.
2.Configure Project Gradle.
buildscript {
repositories {
google()
jcenter()
maven {url 'https://developer.huawei.com/repo/'}
}
dependencies {
classpath "com.android.tools.build:gradle:4.0.1"
classpath 'com.huawei.agconnect:agcp:1.4.2.300'
// NOTE: Do not place your application dependencies here; they belong
// in the individual module build.gradle files
}
}
allprojects {
repositories {
google()
jcenter()
maven {url 'https://developer.huawei.com/repo/'}
}
}
task clean(type: Delete) {
delete rootProject.buildDir
}
3.Configure App Gradle.
implementation 'com.github.bumptech.glide:glide:4.11.0'
implementation 'com.huawei.hms:ml-computer-vision-cloud:2.0.1.300'
implementation 'com.huawei.hms:ml-computer-vision-object-detection-model:2.0.1.300'
4.Configure AndroidManifest.xml.
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
5.Create Activity class with XML UI.
ScanDataActivity:
package com.hms.doctorconsultdemo.ml;
import android.content.Context;
import android.content.Intent;
import android.os.Bundle;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import androidx.appcompat.app.AppCompatActivity;
import androidx.appcompat.widget.Toolbar;
import androidx.databinding.DataBindingUtil;
import com.hms.doctorconsultdemo.R;
import com.hms.doctorconsultdemo.databinding.ActivityScanBinding;
import com.huawei.hms.mlsdk.productvisionsearch.MLVisionSearchProductImage;
import java.util.ArrayList;
import java.util.List;
public class ScanDataActivity extends BaseActivity {
private static List<Item> items;
private ActivityScanBinding mActivityScanBinding;
private DoctorAdapter mDoctorAdapter;
public static void start(@NonNull Context context, List<MLVisionSearchProductImage> products) {
if (context instanceof AppCompatActivity) {
Intent intent = new Intent(context, ScanMainActivity.class);
context.startActivity(intent);
startActivityAnimation(context);
items = new ArrayList<>();
for (MLVisionSearchProductImage product : products) {
Item item = new Item();
item.setFruitName(product.getProductId());
item.setImageUrl(product.getImageId());
item.setPrice(8500);
items.add(item);
}
}
}
@Override
protected void onCreate(@Nullable Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mActivityScanBinding = DataBindingUtil.setContentView(this, R.layout.activity_scan);
init();
}
private void init() {
Toolbar toolbar = findViewById(R.id.toolbar);
toolbar.setTitle("Scan Prescription");
setSupportActionBar(toolbar);
mDoctorAdapter = new DoctorAdapter(this, items, item4 -> {
});
mActivityScanBinding.rvStore.setAdapter(mDoctorAdapter);
}
}
ScanMainActivity:
package com.hms.doctorconsultdemo.ml;
import android.Manifest;
import android.content.Intent;
import android.content.pm.PackageManager;
import android.graphics.Bitmap;
import android.os.Bundle;
import android.provider.MediaStore;
import android.util.Log;
import androidx.annotation.Nullable;
import androidx.core.app.ActivityCompat;
import com.huawei.hmf.tasks.Task;
import com.huawei.hms.mlsdk.MLAnalyzerFactory;
import com.huawei.hms.mlsdk.common.MLException;
import com.huawei.hms.mlsdk.common.MLFrame;
import com.huawei.hms.mlsdk.productvisionsearch.MLProductVisionSearch;
import com.huawei.hms.mlsdk.productvisionsearch.MLVisionSearchProduct;
import com.huawei.hms.mlsdk.productvisionsearch.MLVisionSearchProductImage;
import com.huawei.hms.mlsdk.productvisionsearch.cloud.MLRemoteProductVisionSearchAnalyzer;
import com.huawei.hms.mlsdk.productvisionsearch.cloud.MLRemoteProductVisionSearchAnalyzerSetting;
import java.util.ArrayList;
import java.util.List;
public class ScanMainActivity extends BaseActivity {
private static final String TAG = ScanMainActivity.class.getName();
private static final int CAMERA_PERMISSION_CODE = 100;
MLRemoteProductVisionSearchAnalyzer analyzer;
@Override
protected void onCreate(@Nullable Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
init();
initializeProductVisionSearch();
}
private void init() {
if (!(ActivityCompat.checkSelfPermission(this, Manifest.permission.CAMERA)
== PackageManager.PERMISSION_GRANTED)) {
this.requestCameraPermission();
}
initializeProductVisionSearch();
Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
startActivityForResult(intent, 101);
}
private void requestCameraPermission() {
final String[] permissions = new String[]{Manifest.permission.CAMERA};
if (!ActivityCompat.shouldShowRequestPermissionRationale(this, Manifest.permission.CAMERA)) {
ActivityCompat.requestPermissions(this, permissions, this.CAMERA_PERMISSION_CODE);
return;
}
}
@Override
protected void onActivityResult(int requestCode, int resultCode, @Nullable Intent data) {
super.onActivityResult(requestCode, resultCode, data);
if (requestCode == 101) {
if (resultCode == RESULT_OK) {
Bitmap bitmap = (Bitmap) data.getExtras().get("data");
if (bitmap != null) {
MLFrame mlFrame = new MLFrame.Creator().setBitmap(bitmap).create();
mlImageDetection(mlFrame);
}
}
}
}
private void mlImageDetection(MLFrame mlFrame) {
Task<List<MLProductVisionSearch>> task = analyzer.asyncAnalyseFrame(mlFrame);
task.addOnSuccessListener(products -> {
Log.d(TAG, "success");
displaySuccess(products);
})
.addOnFailureListener(e -> {
try {
MLException mlException = (MLException) e;
int errorCode = mlException.getErrCode();
String errorMessage = mlException.getMessage();
} catch (Exception error) {
// Handle the conversion error.
}
});
}
private void initializeProductVisionSearch() {
MLRemoteProductVisionSearchAnalyzerSetting settings = new MLRemoteProductVisionSearchAnalyzerSetting.Factory()
.setLargestNumOfReturns(16)
.setRegion(MLRemoteProductVisionSearchAnalyzerSetting.REGION_DR_CHINA)
.create();
analyzer
= MLAnalyzerFactory.getInstance().getRemoteProductVisionSearchAnalyzer(settings);
}
private void displaySuccess(List<MLProductVisionSearch> productVisionSearchList) {
List<MLVisionSearchProductImage> productImageList = new ArrayList<>();
String prodcutType = "";
for (MLProductVisionSearch productVisionSearch : productVisionSearchList) {
Log.d(TAG, "type: " + productVisionSearch.getType());
prodcutType = productVisionSearch.getType();
for (MLVisionSearchProduct product : productVisionSearch.getProductList()) {
productImageList.addAll(product.getImageList());
Log.d(TAG, "custom content: " + product.getCustomContent());
}
}
StringBuffer buffer = new StringBuffer();
for (MLVisionSearchProductImage productImage : productImageList) {
String str = "ProductID: " + productImage.getProductId() + "\nImageID: " + productImage.getImageId() + "\nPossibility: " + productImage.getPossibility();
buffer.append(str);
buffer.append("\n");
}
Log.d(TAG, "display success: " + buffer.toString());
ScanDataActivity.start(this, productImageList);
}
}
App Build Result
Tips and Tricks
Images in PNG, JPG, JPEG, and BMP formats are supported. GIF images are not supported.
ML Kit complies with GDPR requirements for data processing.
Face detection requires Android phones with the Arm architecture.
Conclusion
In this article, we have learned how to integrate HMS ML Kit using Product Visual Search Api in Android application. After completely read this article user can easily implement HMS ML Kit using Product Visual Search Api. So that users can scan their prescriptions using Product Visual Search Api.
Thanks for reading this article. Be sure to like and comment to this article, if you found it helpful. It means a lot to me.
References
HMS ML Docs:
https://developer.huawei.com/consumer/en/doc/development/hiai-Guides/service-introduction-0000001050040017
HMS Training Videos:
https://developer.huawei.com/consumer/en/training/
Top comments (0)