DEV Community: Code on the Rocks

Flutter Inception: A Free Shorebird Alternative

Code on the Rocks — Thu, 10 Aug 2023 15:44:12 +0000

There is an intimacy between web developers and their users that native app developers are jealous of. Within minutes of finding and diagnosing a software bug, web developers can publish a fix that all of their users will see. There is no laborious back-and-forth with a third-party app store or a 4-day wait for an simple app review. The web developer builds their new application and sends it to the masses.

Native app developers are not so lucky. App review times on the Google Play and Apple App stores are infamously unpredictable and long, delaying what could have otherwise been immediate fixes for an indeterminate amount of time. This slow but necessary step can ultimately lead to poor reviews, lost users, and developer frustration.

Flutter developers in particular are negatively impacted by these review process because they typically aim to publish their apps on all platforms. Every time they are ready to release an update, they must ask themselves, "Am I ready for two app reviews?". Many other cross-platform developers face the same soul-crushing processes but in this article, I want to present a potential solution for the Flutter crowd.

On the Web and in your Pocket

Flutter is special among cross-platform frameworks because it works on Android, iOS, and the web with very little modification. In fact, every Flutter web application can be downloaded as a progressive web app (PWA) that performs similarly to a native application. In my opinion, PWAs are a valid solution to the app review problem because they
allows developers to publish updates at their own pace without sacrificing (too much of) the native experience. The issue with this approach, of course, is that a majority of people don't know how to download a PWA and among those that do, the willingness to do so is low.

Forget the PWA experience, then. Even if users don't download your Flutter app as a PWA, they can still use it in their browser which allows them the benefit of real-time updates. While good, this solution is not perfect either because native apps are still more performant and the app stores
provide a level of trust that the wild web does not.

If not a PWA or web app, then what? Well, what if we could have both? What if we could have the native app experience 95% of the time and the web experience the other 5%? What if we could have our cake and eat it too? That would be nice.

The final solution I propose is to ship your native apps with a webview fallback. When the latest version of your app has passed app store review, users will get the native experience. When the latest version of your app is still in review, users will get the web experience. From a user perspective, the difference should be subtle to the point of being unnoticeable which is leaps and bounds better than them seeing a bug. Whether or not the webview is used can be
determined by a simple API call to your server, a Firebase Remote Config value, or an automated process that compares the current native version to the latest web version. The rest of this article will discuss the setup in more details.

Deployment Approach

My personal suggestion is to take a web-first approach to app development. Web applications are faster to update, globally accessible, and many browsers include a mobile device emulator that makes testing the responsiveness of your app as easy as resizing your browser window. By making your application welcoming to web travelers, you will make it welcoming to all users, regardless of the color of their SMS messages.

A specific suggestion would be to develop and test your application on a chrome browser using the built in dev tools to emulate how it would look on a mobile device. When everything looks good to go, ship it and start testing on native platforms. Taking this approach will ultimately lead to a drastically higher number of web deployments compared to Android or iOS deployments but that's fine. If you're marketing campaigns are web-focused, most of your traffic will be browser-bound anyway.

Code

Web Application

The first step is to create a web application that will be used as the fallback for your native app. This application should be hosted on a server that you control and should be accessible via a URL. The URL can be stored in your app as a global variable or passed to your application using dart-define. I recommend using a Firebase Hosting since its super easy to setup. Develop this application as you would any other Flutter application and publish it.

In App Webview

To effectively implement this solution, you'll need a trusty webview package that works on both Android and iOS platforms. I prefer to use the flutter_inappwebview package which you can add to your pubspec.yaml using the following line:

flutter_inappwebview: ^5.7.2+3

There is a beta version of this plugin that also works on the web

Once you've added the dependency, the next step is to create an app_webview widget with the sole purpose of displaying your web application in a webview. A small visual indicator, like a different colored status bar when the app is using the
webview, can help you avoid being confused during development. The bare bones code looks like this:

class AppWebview extends StatefulWidget {
  const AppWebview({Key? key}) : super(key: key);

  @override
  State<AppWebview> createState() => _AppWebviewState();
}

class _AppWebviewState extends State<AppWebview> {
  final GlobalKey webViewKey = GlobalKey();

  InAppWebViewController? webViewController;

  @override
  Widget build(BuildContext context) {
    return 
      Scaffold(
        appBar: AppBar(
          toolbarHeight: 0,
          backgroundColor: Colors.black,
        ),
        body: InAppWebView(
          key: webViewKey,
          pullToRefreshController: pullToRefreshController,
          initialUrlRequest: URLRequest(url: WebUri(const String.fromEnvironment('APP_WEBVIEW_BASE_URL'))),
          onWebViewCreated: (controller) {
            webViewController = controller;
          },
        ),
    );
  }
}

The webview variant of your native app doesn't need to use the router your app normally uses (ex.
auto_route, go_router, etc) so you can add it directly to a MaterialApp widget. The app inside
the webview will use the router as designed.

The biggest problem with this code is that your native application will think it only has a single route. In other words, the native Android and iOS apps can't see the router in the webview. One tap of the Android back button or a single right swipe on iOS will close the application completely. We don't want that.

To fix this, we need to wrap the webview widget in a WillPopScope widget that will detect back
button presses and communicate them to the webview controller. If the webview can navigate backwards in its history, a back button press should do that instead of closing the app:

return WillPopScope(
      onWillPop: () async {
        final controller = webViewController;
        if (controller != null) {
          if (await controller.canGoBack()) {
            controller.goBack();
            return false;
          }
        }
        return false;
      },
      child: Scaffold(
        appBar: AppBar(
          toolbarHeight: 0,
          backgroundColor: Colors.black,
        ),
        body: // ...webview code
      ),
    );

When to Webview

Now that we have a webview set up, we need to decide when to use that instead of the normal app. This part is up to you but I'll provide a few ideas here. Since the app stores operate at different speeds, its a good idea to create separate flags for Android and iOS.

Firebase Remote Config

Firebase Remote Config lets you create and update feature flags easily through your project's
Firebase console. On the Remote Config tab, add two new boolean parameters (android_webview and
ios_webview), set them to false, and publish the changes. When a platform should use the webview,
change its corresponding flag to true and publish.

In the app, you can read these values using the firebase_remote_config package. The following code sets up the remote config and
reads the values for the android_webview and ios_webview flags:

final remoteConfig = FirebaseRemoteConfig.instance;

Future<void> setup() async {
  await remoteConfig.setConfigSettings(RemoteConfigSettings(
    fetchTimeout: const Duration(minutes: 1),
    minimumFetchInterval: const Duration(hours: 1),
  ));

  await remoteConfig.setDefaults({
    'android_webview': false,
    'ios_webview': false,
  });

  await remoteConfig.fetchAndActivate();
}

bool get androidWebview => remoteConfig.getBool('android_webview');

bool get iosWebview => remoteConfig.getBool('ios_webview');

Pros:

✅ Fast to change

✅ Fast to deploy

Cons:

❌ Requires Firebase project and packages

❌ Not always immediate

Dart Endpoint

If you're a full stack Dartists, creating a simple endpoint using Dart Frog can be a fast way to
add remote control to your app. Create a new endpoint that returns a boolean value for each
platform. When the endpoint is hit, the app can read the value and decide if it should use the
webview.

Pros:

✅ Fully customizable

✅ Lightweight

Cons:

❌ Requires a new server deploy for each change

❌ More setup

If you'd rather your app automatically detect when it should use the webview, you can tweak this implementation slightly and store the latest version in Remote Config or on your server. Then, when your native apps start up, they can check their own version against this server version and decide if the webview is necessary.

You can get the current version of your app using the package_info_plus package. Add it to your pubspec.yaml:

package_info_plus: ^1.3.0

Then, in your main.dart file, you can get the current version of your app:

final packageInfo = await PackageInfo.fromPlatform();
final currentVersion = packageInfo.version;

Regardless of how you decide to set these flags, the implementation on the frontend will look similar. If the app is running on the web, it should never use the webview. If its running on native Android or iOS, it should check the relevant flag and decide what to do:

class MyApp extends StatelessWidget {
  MyApp({super.key});

  final MaterialApp baseApp = MaterialApp.router(
    routerConfig: router.config(),
    theme: ThemeData(
      colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
      useMaterial3: true,
    ),
  );

  @override
  Widget build(BuildContext context) {
    return Builder(builder: (context) {
      if (kIsWeb) {
        return baseApp;
      } else {
        bool androidWebview = Platform.isAndroid && remoteConfigService.androidWebview;
        bool iosWebview = Platform.isIOS && remoteConfigService.iosWebview;
        bool showWebView = androidWebview || iosWebview;

        if (!kIsWeb && showWebView) {
          return const MaterialApp(home: AppWebview());
        } else {
          return baseApp;
        }
      }
    });
  }
}

What about the Guidelines?

According to section 2.5.2 of the Apple Developer guidelines:

Apps should be self-contained in their bundles, and may not read or write data outside the
designated container area, nor may they download, install, or execute code which introduces or changes features or functionality of the app, including other apps. Educational apps designed to teach, develop, or allow students to test executable code may, in limited circumstances, download code provided that such code is not used for other purposes. Such apps must make the source code provided by the app completely viewable and editable by the user.

If we're comparing the webview fallback method to Shorebird.dev, I'd argue that the webview approach is more acceptable since the contents of the users app does not change until they want it to (by updating the app through the App Store).

Conclusion

If you've ever accidentally shipped a bug to production, you know what true helplessness feels like. Regardless of how simple the fix is, you can be stuck watching users encounter the bug for hours or even days as stakeholders question you about when it will be squashed. Its a stressful time and stress is bad for your health. With the webview fallback solution I've described here, you might just get days of your life back. Most of the time your users will see the native
experience they're used to. For the other handful of times, when a bug has crept into prod or a much needed update is standing in the TSA line, you can flip a switch and sleep easy. How can you beat that?

Butler Labs OCR in Flutter

Code on the Rocks — Mon, 10 Jul 2023 16:05:51 +0000

Butler Labs is an AI-powered optical character recognition (OCR) platform with models fine tuned to extract important data from a variety of commonly encountered documents:

Driver's Licenses
Passports
Health Insurance Cards
Paystubs
Invoices
Receipts
W9s
Mortgage Statements

What makes this platform even more attractive from a developer standpoint is that they provide a generous free tier of 500 scans per month and a comprehensive REST API. This means you can basically add an OCR upgrade to your app today.

Setup

API Key

To start using the Butler Labs OCR product, you'll need two values:

An API key
A model-specific queue ID

After creating an account, you'll find your API key at the top of the "Settings" page. The API key is a sensitive value that enables access to your account. For that reason, you should avoid adding it to version control and instead pass it to your app using dart-define:

flutter run --dart-define BUTLER_API_KEY=1234567890

Or my preferred method, dart-define-from-file. First create a config.json file in your app's asset folder:

{
  "BUTLER_API_KEY": "YOUR_KEY",
}

Then run your app using the following command:

flutter run --dart-define-from-file=assets/config.json

Inside the app, you can retrieve dart-define values using the String.fromEnvironment() constructor:

String apiKey = const String.fromEnvironment("BUTLER_API_KEY");

Queue ID

In this post, we'll be using the US Driver's License model to extract a ton of information from images of driver's licenses. Before we can do this though, we'll need a queue ID (aka API ID) for our instance of the model. On the "Explore Models" page, locate the US Driver's License model.

The process for adding any model is the same:

Click on it to add it to your "My Models" page
On the "My Models" page, select the new model
Select the "APIs" tab
Copy the queue ID at the top of the page

Image Picker

Users will need to either capture or upload images to your app so they can be analyzed and by far the easiest way to accomplish this is with the image_picker package. It works seamlessly on Android, iOS, and the mobile and desktop variants of web. Add the dependency to your pubspec.yaml:

dependencies:
  image_picker: ^1.0.0

And then create a simple button that users can tap to take an image.

ElevatedButton(
  onPressed: () async {
    ImagePicker picker = ImagePicker();
    XFile? pickedImage = await picker.pickImage(source: ImageSource.gallery);
    if (pickedImage == null) return;
    // todo analyze image
  },
  child: const Text('Upload Image'),
),

In the next step we'll send the user's image to Butler Labs and parse the result.

A Driver's License Example

In this post, we'll running our tests on the sample licenses from the Driver's licenses in the United States Wikipedia page.

Click on each one separately on the wiki and download it. Once you have an image on your computer, you can drag it onto your Android emulator to add it to the list of files.

Now onto the fun part!

Making the Request

After a user selects an image, we want to send it to the extract endpoint where Butler Labs will analyze it and return a JSON object with the parsed results. An example response looks like this:

{
  "documentId": "1540d070-c500-4994-a0bc-4196499c41de",
  "documentStatus": "Completed",
  "fileName": "alabama.jpeg",
  "mimeType": "image/jpeg",
  "documentType": "US Driver's License",
  "confidenceScore": "High",
  "formFields": [
    {
      "fieldName": "Document Number",
      "value": "1234567",
      "confidenceScore": "High"
    },
    {
      "fieldName": "First Name",
      "value": "CONNOR",
      "confidenceScore": "Low"
    },
    {
      "fieldName": "Last Name",
      "value": "SAMPLE",
      "confidenceScore": "High"
    },
    {
      "fieldName": "Birth Date",
      "value": "01-05-1948",
      "confidenceScore": "High"
    },
    {
      "fieldName": "Expiration Date",
      "value": "01-05-2014",
      "confidenceScore": "High"
    },
    {
      "fieldName": "Address",
      "value": "1 WONDERFUL DRIVE MONTGOMERY AL 36104-1234",
      "confidenceScore": "High"
    },
    {
      "fieldName": "Sex",
      "value": "SEXM",
      "confidenceScore": "High"
    },
    {
      "fieldName": "State",
      "value": "Alabama",
      "confidenceScore": "High"
    }
  ],
  "tables": []
}

The image files need to be sent to the API in a Multipart request. Luckily the http package comes with out-of-the-box support for this so the first step is to import that and a few others:

import 'dart:io'; // For the HttpHeaders
import 'dart:convert'; // To decode the response from Butler Labs
import 'package:http/http.dart'; // For the MultipartRequest class
import 'package:http_parser/http_parser.dart'; // For the MediaType class

Then, you need to create the MultipartRequest using the extract endpoint. This is also where you'll add your API key to the request header:

String butlerApiKey = const String.fromEnvironment('BUTLER_API_KEY');

    MultipartRequest request = MultipartRequest('POST', Uri.parse('https://app.butlerlabs.ai/api/queues/$queueId/documents'));

    request.headers.addAll({
      HttpHeaders.acceptHeader: '*/*',
      HttpHeaders.authorizationHeader: 'Bearer $butlerApiKey',
      HttpHeaders.contentTypeHeader: 'multipart/form-data',
    });

Finally, we need to add the image to the request. This can be done using the image's file path or its raw bytes and the option you choose will depend on the platform you're using. Files returned by ImagePicker on Flutter web will not have a path so on the web you will need to use the bytes method. I've included the full code snippets for both methods below.

Using the Image Path

ElevatedButton(
  onPressed: () async {
    ImagePicker picker = ImagePicker();
    XFile? pickedImage = await picker.pickImage(source: ImageSource.gallery);
    if (pickedImage == null) return;

    String butlerApiKey = const String.fromEnvironment('BUTLER_API_KEY');
    String queueId = 'fdf1f80a-03f5-40e5-83f0-a33694318532'; // Replace with your own queueId

    MultipartRequest request = MultipartRequest('POST', Uri.parse('https://app.butlerlabs.ai/api/queues/$queueId/documents'));

    request.headers.addAll({
      HttpHeaders.acceptHeader: '*/*',
      HttpHeaders.authorizationHeader: 'Bearer $butlerApiKey',
      HttpHeaders.contentTypeHeader: 'multipart/form-data',
    });

    if (defaultTargetPlatform == TargetPlatform.android || defaultTargetPlatform == TargetPlatform.iOS) {
      request.files.add(
        await MultipartFile.fromPath(
          'file',
           pickedImage.path,
          contentType: MediaType('image', 'jpeg'),
        ),
      );
    } else {
      throw Exception('Platform not supported');
    }

    StreamedResponse response = await request.send();

    ButlerResult? result;
    String value = await response.stream.transform(utf8.decoder).join();
    result = ButlerResult.fromJson(jsonDecode(value));
    debugPrint('Response stream: $value');
  },
  child: const Text('Upload Image'),
),

Using the Image Bytes

ElevatedButton(
  onPressed: () async {
    ImagePicker picker = ImagePicker();
    XFile? pickedImage = await picker.pickImage(source: ImageSource.gallery);
    if (pickedImage == null) return;

    String butlerApiKey = const String.fromEnvironment('BUTLER_API_KEY');
    String queueId = 'fdf1f80a-03f5-40e5-83f0-a33694318532'; // Replace with your own queueId

    MultipartRequest request = MultipartRequest('POST', Uri.parse('https://app.butlerlabs.ai/api/queues/$queueId/documents'));

    request.headers.addAll({
      HttpHeaders.acceptHeader: '*/*',
      HttpHeaders.authorizationHeader: 'Bearer $butlerApiKey',
      HttpHeaders.contentTypeHeader: 'multipart/form-data',
    });

    Uint8List imageBytes = await pickedImage.readAsBytes();

    MultipartFile file = MultipartFile(
      'file',
      ByteStream.fromBytes(imageBytes),
      imageBytes.lengthInBytes,
      filename: 'temp.jpg',
      contentType: MediaType('image', 'jpeg'),
    );

    request.files.add(file);

    StreamedResponse response = await request.send();

    ButlerResult? result;
    String value = await response.stream.transform(utf8.decoder).join();
    result = ButlerResult.fromJson(jsonDecode(value));
    log('Response stream: $value');
  },
  child: const Text('Upload Image'),
),

The bytes method will work on all platforms so I suggest using that

Deciphering the Results

The request may take a few seconds but soon you should have a nice, juicy JSON blob to sink your teeth into. The shape of the result is fairly consistent from model to model too, meaning you won't have to build custom deserializers if you add more models later.

Each result will contain the following top-level fields:

Document ID: Unique ID for the document you uploaded
Upload ID: An ID you can use to delete the image that was uploaded
Document Status: A value indicating the status of the document according to Butler Labs
File Name: The name of the file that was uploaded
Document Type: The type of the document, typically matching the model you used (ex. Health Insurance Card)
Confidence Score: A Low/Medium/High value indicating the model's overall confidence in the parsed results. You can see the breakdown of all your organization's confidence scores on the Butler Labs dashboard

Beneath these top-level fields you will also find a list of "Form Fields" corresponding to the fields identified by your model (see the model's page in the Butler Docs). Each form field will contain the following values:

Field Name
Value: The value detected by the model for this field
Confidence Score: A Low/Medium/High value indicating the model's confidence in this specific field
Confidence Value: A value between 0 and 1 indicating the likelihood the result is correct
OCR Confidence Value: A value between 0 and 1 indicating the likelihood the model found the correct field

In the sample app linked below, I used the confidence scores and values to create a simple UI illustrating the model's performance:

Butler Labs Package

While the code required for this OCR product is fairly unsophisticated, you can avoid the hassle of piecing everything together yourself and use the butler_labs package on pub.dev. This package currently only supports the Extract document endpoint. It includes a generic ButlerResult class and models specific to each model on the Butler Labs Website so you can convert the generic result from the API into something that's more usable.

Conclusion

OCR is a solved problem and Butler Labs has a product that is more than satisfactory for many common use cases. If the built-in models don't meet your needs, their platform also allows you to create your own models. You can read more about that here. Happy coding!