DEV Community

John Sheehy
John Sheehy

Posted on

Flutter + ML Kit: the missing dependency that reads a Japanese menu as romaji

#flutter #showdev #android #ios

SofiLingo is the offline travel translator I make in Flutter, for iPhone and Android. Its camera mode reads menus, signs and tickets with Google's ML Kit text recognition, then translates them on the device.

One part of that setup fails without a single error: on iOS, on Android, in flutter analyze, in a Dart test or in a build log. If you use google_mlkit_text_recognition with anything other than Latin script, it is worth five minutes.

ML Kit's recognizer is per script

ML Kit ships five on-device text recognizers: Latin, Chinese, Devanagari, Japanese and Korean. You pick one when you create the recognizer:

final recognizer = TextRecognizer(script: TextRecognitionScript.japanese);
Enter fullscreen mode Exit fullscreen mode

The wrong one doesn't fail. It returns confident nonsense. Point the Latin model at a Japanese menu and you get stray romaji fragments, which look like a bad photo rather than a bug. So the app picks the recognizer from the source language:

static const _scripts = <TranslateLanguage, TextRecognitionScript>{
  TranslateLanguage.chinese: TextRecognitionScript.chinese,
  TranslateLanguage.japanese: TextRecognitionScript.japanese,
  TranslateLanguage.korean: TextRecognitionScript.korean,
  TranslateLanguage.hindi: TextRecognitionScript.devanagiri,
  TranslateLanguage.marathi: TextRecognitionScript.devanagiri,
};
Enter fullscreen mode Exit fullscreen mode

That part is in the docs. The trap is underneath it.

The plugin ships Latin only

The Flutter plugin's text recognition pod contains the Latin model and nothing else. Chinese, Japanese, Korean and Devanagari are separate native dependencies, and the plugin does not pull them in for you.

Leave them out and here is what happens.

iOS: nothing visible. The plugin detects which scripts were compiled in with __has_include, and its switch has a bare default: that returns the Latin recognizer for any script it wasn't built with. Your code asks for Japanese, gets Latin, and the Japanese menu comes back as romaji. No exception, no log line.

Android: the plugin declares those libraries compileOnly, so they are on its compile path but absent from your APK. The first frame throws NoClassDefFoundError. In a camera preview that processes frames in a loop, that throw usually lands in a per-frame catch (_), and you get a preview that never finds any text.

Two platforms, two different failures, neither of them loud.

The fix: declare the scripts yourself

iOS, in ios/Podfile inside the Runner target (about 1.2 to 1.6 MB each):

pod 'GoogleMLKit/TextRecognitionChinese', '~> 9.0.0'
pod 'GoogleMLKit/TextRecognitionDevanagari', '~> 9.0.0'
pod 'GoogleMLKit/TextRecognitionJapanese', '~> 9.0.0'
pod 'GoogleMLKit/TextRecognitionKorean', '~> 9.0.0'
Enter fullscreen mode Exit fullscreen mode

Android, in android/app/build.gradle.kts:

dependencies {
    implementation("com.google.mlkit:text-recognition-chinese:16.0.1")
    implementation("com.google.mlkit:text-recognition-devanagari:16.0.1")
    implementation("com.google.mlkit:text-recognition-japanese:16.0.1")
    implementation("com.google.mlkit:text-recognition-korean:16.0.1")
}
Enter fullscreen mode Exit fullscreen mode

Match the versions to the ML Kit version your plugin release uses.

Guard it with a test

Nothing in the toolchain will notice if someone tidies those lines away later, so a plain Dart test reads the two build files and fails if a script goes missing:

const scripts = ['Chinese', 'Devanagari', 'Japanese', 'Korean'];

test('the iOS pods are declared', () {
  final podfile = File('ios/Podfile').readAsStringSync();
  for (final script in scripts) {
    expect(podfile, contains('GoogleMLKit/TextRecognition$script'),
        reason: '$script has no pod, so it silently reads as Latin');
  }
});

test('the Android dependencies are declared', () {
  final gradle = File('android/app/build.gradle.kts').readAsStringSync();
  for (final script in scripts) {
    expect(gradle, contains('text-recognition-${script.toLowerCase()}'),
        reason: '$script is not in the APK, so it throws at the first frame');
  }
});
Enter fullscreen mode Exit fullscreen mode

It is a blunt test, but it is the only place this failure can be caught before a user points their phone at a menu.

One more: list your Latin languages, don't default to them

A related trap: if the language-to-script lookup falls back to Latin for anything it doesn't know, every new language silently claims the camera works for it. Russian through the Latin recognizer returns nothing at all, again looking like a bad photograph. SofiLingo lists its Latin-script languages explicitly, and any language in neither list has no on-device path, so it goes to cloud OCR instead (Cyrillic, Greek, Hebrew, Arabic script, Thai and every Indic script except Devanagari).

SofiLingo translates menus, signs, speech and text in 58 languages, with typing that works offline, no account, and translations that stay on your phone. It's free with ads, and a one-off Holiday pass removes them for 14 days. sofilingo.com

Top comments (0)