Japanese OCR on iPhone with the Vision framework

#ios #swift #ocr #vision

Easy on-device text recognition

If you can recognize text on iPhone, you can build handy things like transcribing whiteboards or reading signs.

A 2022 update made Japanese available

Since iOS 16 (2022), Japanese text recognition is possible — using only the built-in framework. The accuracy is quite good; personally I think it's usable in production for many apps.

How to use it

Use Vision's VNRecognizeTextRequest. Set recognitionLanguages to "ja". Requires macOS 13, Xcode 14, iOS 16 or later.

let request = VNRecognizeTextRequest()
request.recognitionLanguages = ["ja"] // specify Japanese
let handler = VNImageRequestHandler(cvPixelBuffer: pixelBuffer)
do {
    try handler.perform([request])
} catch let error {
    print(error)
}
guard let observations = request.results as? [VNTextObservation] else { return }
for observation in observations {
    let box = observation.boundingBox // bounding box of the position
    let topCandidate = observation.topCandidates(1)
    if let recognizedText = topCandidate.first?.string { // recognized text
        print(recognizedText)
    }
}