DEV Community: João Pimenta

The Complete Beginner’s Guide to Flutter’s Automated Testing

João Pimenta — Tue, 22 Jul 2025 17:37:48 +0000

Introduction

Testing is a critical, yet overlooked, part of modern app development as it helps prevent bugs from slipping through the cracks and affecting users. It should be an integral part of a healthy code base and must be maintained like any other code written in the repository.

Flutter provides a robust framework and excellent tools for writing and running tests at different levels of your application. Whether you're building a small app or scaling a large codebase, testing ensures that your code is reliable, maintainable, and free from regressions.

In this article, we'll explore the general idea of automated testing and the three main types of testing in Flutter: Unit Testing, Integration Testing, and Widget Testing. Each serves a specific purpose in the development of a lifecycle and offers tools tailored for different testing scopes.

Testing Fundamentals

Testing Pyramid

First introduced by Mike Cohn in his book Succeeding with Agile: Software Development Using Scrum, the testing pyramid is a conceptual framework that describes the recommended structure and proportion of automated tests in a software project. The idea is:

A large base of fast, isolated tests.
A smaller middle layer of more integrated tests.
A very small top layer of slow, comprehensive tests

Testing Essentials

1) What makes a test a good test?

It should be easy to understand, implement and maintain
Readable – clear title, subtitle, and description
Reliable – produces consistent results every time it runs
Independent - does not depend on other tests or external states
Follows DRY – Do Not Repeat Yourself (Avoid redundant code)
Follow DAMP – Descriptive And Meaningful Phrases

2) Configuring Automated Tests in Flutter

By convention, all test files should be placed inside the test/ folder at the root of your Flutter project. It’s also good practice to name your test files with the suffix _test.dart, so the Flutter test runner can detect and execute them automatically.

lib/
    my_app.dart
test/
    widgets/
        my_widget_test.dart
    integration_test/
        my_integration_test.dart
    unit_test/
        my_unit-test.dart

Each test file in Flutter must define its own main() as the flutter test command executes each test file in isolation, meaning each file gets its own separate memory and process. This strict isolation guarantees that one test can never interfere with another. It ensures a clean slate and predictable results for every test run and without a main(), there’s nothing for the test runner to execute, and the file will be ignored.

import 'package:flutter_test/flutter_test.dart';

void main() {
  test('example test', () {
    expect(2 + 2, equals(4));
  });
}

3) Arrange, Act, Assert

It is a common testing pattern that helps structure your test cases clearly and consistently by dividing each test into three distinct phases.

Arrange: dedicated to setting up the necessary preconditions for your test. This includes initializing objects, configuring mock dependencies, or preparing any specific state required for the scenario you're testing.

Act: execute the specific action or behavior that you intend to test. This could involve calling a method, triggering a user interaction on a widget, or invoking a specific function.

Assert: verify the outcome of the action performed in the "Act" stage. This involves making assertions to confirm that the resulting state, return value, or side effects are precisely what you expect.

Flutter's Automated Testing

Unit Tests: The Base of the Pyramid

Unit testing focuses on testing individual components of your code in isolation. These components are typically small, self-contained units like functions, methods, or classes. The primary goal of unit testing is to ensure that each isolated piece of code behaves as expected. It's ideal for catching logic errors and verifying that your core algorithms and business logic work correctly. External dependencies (e.g., APIs, databases, services) are often replaced with mock objects, fakes, or stubs to isolate the unit being tested.

To perform a unit test in Flutter, we will be using the test() function. It takes two arguments: a String description (red) and a callback function containing the test logic (green).

Let’s say I want o test the following function – getAllUserDecks(), which gets all the decks of the user from the SharedPreference.

Arrange Phase: Setting the Stage

1) Prepare Mock Data (fakeDecks): First, a List named fakeDecks is initialized. This list will mock the data we expect to be both stored in and retrieved from SharedPreferences.

2) SharedPreferences.setMockInitialValues() is used. This method, provided by the shared_preferences package, allows us to pre-populate SharedPreferences with our fakeDecks data specifically for the test's execution. This way, when the controller attempts to read from SharedPreferences, it receives our predefined mock data.

3) Initialize the Controller: Finally, an instance of the DeckPageController (the class containing the function under test) is created. This prepares the object on which we will invoke the method being tested.

Act Phase: Executing the Action

4) Call the Function Under Test: In the "Act" phase, the getAllUserDecks() method is called on the initialized controller. This simulates the real application flow where the controller would attempt to load data from SharedPreferences.

Assert Phase: Verifying the Outcome

5) Validate Expected Results with expect() from the flutter_test package to verify the results. expect() receives the actual result (of dynamic type) and compares it against the expected value (also of dynamic type). In this particular test, expect() is used multiple times to:

Confirm that controller.userDeck is not null, indicating data was loaded.
Verify the correct number of decks were loaded.
Check specific properties of the loaded decks (e.g., deckName, identity) to ensure accurate deserialization and data integrity.

Widget Test: The Middle Layer

Widget tests targets the user interface of your Flutter app. It allows you to interact with widgets, and assert their visual output. The main goal is to verify that widgets render correctly. It's a middle ground between unit and integration testing — more comprehensive than the former, but faster and more isolated than the latter. Widget tests use real Flutter widgets within a simulated environment.

Instead of the test() funcion used, which is used for unit tests, we are going to use testWidgets() for the widget tests.

The testWidgets() function is the entry point for defining a widget test in Flutter, and it is provided by the flutter_test package.

It takes two arguments:

a String description of the test, and
a WidgetTesterCallback, which contains the test logic.

This callback is executed inside the Flutter test environment, and it receives a WidgetTester instance as its argument. For each testWidgets() call, Flutter creates a new, isolated WidgetTester instance, ensuring that tests don’t interfere with each other’s state.

pumpWidget, pump and pumpAndSettle

In order to render a widget tree in the test environment, Flutter provides the pumpWidget() method. It Calls runApp with the given widget, triggers a frame, and flushes microtasks (by calling pump internally). It builds and renders the provided widget in the test environment.

However, in flutter’s automated tests, the test environment runs in a FakeAsync zone, which lets you control time and frames explicitly. For example, when a widget triggers a setState() (e.g., when a button is pressed), the widget tree doesn’t automatically rebuild. You need to call pump() to simulate a frame and rebuild the widget tree.

Similarly, you can use pumpAndSettle() to keep pumping frames until all animations and microtasks are complete.

Finding the Widgets We Want to Test:

There are more than 10 built-in ways to find the widgets. On this article Only the most reliable and used will be shown, but, if you want to see all of them, check them here.

1) find.byKey
Finds widgets that have a specific Key assigned to them.

Benefits:

Most reliable: Keys are unique identifiers within a widget tree, making this the most precise and stable way to find a widget, especially if multiple widgets of the same type exist or their text/icon content might change.
Ideal for critical interactions: Use keys for widgets you need to interact with directly (e.g., buttons, text fields) or whose presence is crucial to the test.

Drawbacks:

Requires explicit key assignment: You must explicitly add Key objects to your widgets in your application code, which can add verbosity if not already done.

Note: ValueKey is commonly used to differentiate keys with meaningful values.

2) find.byType
Finds widgets of a specific Type (e.g., Text, ElevatedButton, TextField).

Benefits:

Easy to use for common widget types: Great for finding all instances of a particular widget type on the screen.
No code changes required: You don't need to modify your application code to add keys or other identifiers.

Drawbacks:

Can be ambiguous: If there are multiple widgets of the same type on the screen, this method will return all of them, which might not be what you want. You'll then need to refine your finder (e.g., find.byType(TextField).at(0) for the first one).
Doesn't distinguish instances: It finds based on the class, not a specific instance of that class.

3) find.byText
Finds widgets that display the exact given text. This typically works for Text widgets and their descendants.

Benefits:

Very readable: Tests often become self-documenting (e.g., find.text('Submit')).
Convenient for UI verification: Useful for asserting that certain text content is displayed on the screen.

Drawbacks:

Fragile if text changes: If the displayed text changes (e.g., due to internationalization, dynamic content), the test will break.
Case-sensitive: Matches exactly, so "Submit" is different from "submit".
Can be ambiguous: If the same text appears in multiple places, it will find all of them.

4) find.byIcon
Finds Icon widgets displaying a specific IconData (e.g., Icons.add).

Benefits:

Useful for icon-based interactions: Good for verifying the presence of specific icons or interacting with buttons that primarily use an icon.

Drawbacks:

Limited to Icon widgets: Only works for Icon widgets.
Fragile if icons change: If the icon is replaced, the test will break.

5) find.bySemanticsLabel
Finds widgets based on their semantic label, which is used for accessibility purposes. Can match by exact string or RegExp.

Benefits:

Robust for accessibility: Good for ensuring that your widgets are accessible and for testing interactions with screen readers.
Less fragile than text: Semantic labels are less likely to change frequently than visible text, especially for static elements.

Drawbacks:

Requires Semantics widget or explicit label: You need to set a semanticsLabel property on your widget or wrap it in a Semantics widget.
Not always present: Many widgets might not have a semanticsLabel if it's not explicitly provided.

6) find.byWidgetPredicate
A highly flexible method that takes a WidgetPredicate function. This function receives a Widget and returns true if the widget matches your criteria.

Benefits:

Most powerful and flexible: Allows for complex and custom matching logic. You can check any property of the Widget (e.g., its color, specific controller properties, etc.).

Drawbacks:

More verbose: Requires writing a custom function, which can be more complex than simple direct finders.
Can be less performant: If the predicate is complex, it might take longer to traverse the widget tree.
Can be prone to errors: Requires careful implementation of the predicate to avoid unintended matches or missed widgets.

7) find.descendant
Finds widgets that are descendants of a widget found by of and that also match the matching finder.

Benefits:

Scoping searches: Useful when you have multiple similar widgets but want to find one within a specific parent (e.g., a "Submit" button within a "Login Form").
Reduces ambiguity: Helps narrow down the search space.

Drawbacks:

More complex syntax: Requires two finders.
Can be slow: If the of finder matches a large subtree, the search for matching can be extensive.

8) find.ancestor
Finds widgets that are ancestors of a widget found by of and that also match the matching finder.

Benefits:

Finding parent widgets: Useful for asserting properties of a parent based on a child, or for interacting with a parent widget when you only have a reference to a child.

Drawbacks:

Less common for direct interaction: You typically interact with descendants more than ancestors in testing.
More complex syntax: Similar to descendant, it requires two finders.

Let's use the following code to show an example of widget test

Integration Test: The Top Layer

Integration tests verifies that different parts of your application work together cohesively by simulating “real” user interactions and flows across multiple widgets, screens, and external dependencies such as APIs or databases.

On Flutter, Integration tests work a bit different than the widget and unit test and it requires some set up and configuration.
First, is to add the integration_test package to your project (it is available by using the following command:

flutter pub add 'dev:integration_test:{"sdk":"flutter"}'

or by adding the following to the pubspec.yaml, under dev_dependencies.

The integration test should reside in a separate directory inside your flutter project. You can do this by:

1) Creating a new directory named integration_test
2) Add an empty file named app_test.dart in that directory.

Now we are ready to begin writing our automated tests.

There are one important new thing on this example:

IntegrationTestWidgetsFlutterBinding.ensureInitialized()

In Flutter, integration tests use a different testing binding than regular flutter_test widget tests.

Regular widget tests run in a simulated (headless) environment with a fake WidgetsBinding (TestWidgetsFlutterBinding), which is fast but limited — no real device features.
Integration tests, however, run on a real device or emulator, so they need to hook into the real system properly.

Therefore, IntegrationTestWidgetsFlutterBinding.ensureInitialized() its going to set up a testing environment to communicate test results back to the host via the Flutter Driver extension, allowing interaction with real platform channels, plugins and async events as they happen on a real device.

Since every testWidgets() instantiate a WidgetTester object, we can use it to simulate user interactions like tapping, dragging, creating gestures and much more.

Tap
It dispatches a pointer down / pointer up sequence at the center of the given widget, assuming it is exposed.

enterText
Give the text input widget specified by finder the focus and replace its content with text, as if it had been provided by the onscreen keyboard.

Drag
Attempts to drag the given widget by the given offset, by starting a drag in the middle of the widget.

longPress
Dispatch a pointer down / pointer up sequence (with a delay of kLongPressTimeout + kPressTimeout between the two events) at the center of the given widget, assuming it is exposed.

Fling
Starts a drag gesture with a specific velocity & offset (useful for testing scrolls with momentum).

scrollUntilVisible
Scrolls a scrollable until a widget matching the finder becomes visible.

All the methods of the WidgetController are available here.

Extras

group, setUp and tearDown

Group()
It is used when we want to run a series of related tests. It categorize the tests and once tests are put into a group, all of them are going to be ran by a single command.

setup()
You should use a setUp() function when you have common initialization code that needs to run before each test in a group. This is especially useful when multiple tests need to create and configure the same objects.
The setUp() function registers a callback that is executed before each test. The body of the callback can be synchronous or asynchronous — if it’s asynchronous, it must return a Future.
If you call setUp() inside a test group (group()), it only applies to the tests within that group. The setup code will run after any setUp() callbacks defined in parent groups or at the top level.
When there are multiple setUp() callbacks at the same level (either top-level or within a group), they are executed in the order they were declared before each test runs.

teardown()
You should use a tearDown() function when you need to clean up resources after each test in a group. This is useful if your tests create objects, connections, or state that should be reset or disposed of after each test runs.
The tearDown() function registers a callback that is executed after each test. Like setUp(), its body can be synchronous or asynchronous — if asynchronous, it must return a Future.
If you call tearDown() inside a test group (group()), it applies only to the tests within that group. The teardown code runs before any tearDown() callbacks in parent groups or at the top level, which is the opposite order of setUp().
When multiple tearDown() callbacks are declared at the same level, they execute in the order they were declared after each test completes.

Mocking API Responses

When your app or service fetches data from an API, you don’t want your tests to actually call the real server.
Instead, you mock the API response so that your tests are fast, reliable, and don’t depend on the network.
In Dart/Flutter tests, you can use the http package together with mockito or just a custom fake client to mock HTTP calls. In this example, we’ll use the latter: a custom fake client.

Suppose we have a service that fetches data from an API.

The ApiService class is responsible for fetching post data from the endpoint https://example.com/posts.
It takes an http.Client in its constructor. This allows you to inject either a real client or a mock client.
The method fetchPostTitles() Sends a GET request to the API and If the response status code is 200 OK, it decodes the JSON response body into a list and then extracts the title field from each post and returns a list of titles. Otherwise, if the response has an error status, it throws an exception.

This test checks that fetchPostTitles() works as expected when the API responds successfully.
A MockClient is created, which simulates the HTTP client. The mock client is configured to always return a “200 OK” response with a JSON array.
Then, An instance of ApiService is created with this mock client and the fetchPostTitles() method is called. If the result matches, the test passes; otherwise, it fails.

Conclusion

By applying unit, widget, and integration tests appropriately, you can catch bugs early, prevent regressions, and build confidence in your codebase as it grows. Flutter provides powerful tools and a flexible testing framework to help you write clear, reliable, and efficient tests at every layer of the testing pyramid. Embracing automated testing with Flutter means fostering a development environment where confidence in code changes is high, regressions are minimized, and the user experience remains consistently excellent. By integrating these practices into your workflow, you'll not only deliver higher-quality applications but also cultivate a more efficient and enjoyable development process.

Sources

Others

Flutter Engineering by Majid Hajian

https://www.leapwork.com/blog/testing-pyramid

Flutter's Documentation

https://docs.flutter.dev/cookbook/testing/widget/introduction

https://api.flutter.dev/flutter/flutter_test/Finder-class.html

https://api.flutter.dev/flutter/flutter_test/CommonFinders-class.html

https://docs.flutter.dev/cookbook/testing/widget/introduction

https://api.flutter.dev/flutter/flutter_test/test.html#:~:text=Creates%20a%20new%20test%20case,be%20run%20on%20matching%20platforms

https://docs.flutter.dev/testing/overview

https://api.flutter.dev/flutter/flutter_test/test.html#:~:text=Creates%20a%20new%20test%20case,be%20run%20on%20matching%20platforms

https://api.flutter.dev/flutter/flutter_test/expect.html

https://api.flutter.dev/flutter/flutter_test/testWidgets.html

https://api.flutter.dev/flutter/flutter_test/WidgetTester-class.html

https://api.flutter.dev/flutter/flutter_test/WidgetTester/pumpWidget.html

https://api.flutter.dev/flutter/package-fake_async_fake_async/FakeAsync-class.html

https://api.flutter.dev/flutter/flutter_test/WidgetTester/pump.html

https://api.flutter.dev/flutter/flutter_test/WidgetTester/pumpAndSettle.html

https://github.com/flutter/flutter/blob/master/packages/flutter_test/lib/src/finders.dart

https://github.com/flutter/flutter/tree/main/packages/integration_test#integration_test

https://api.flutter.dev/flutter/flutter_driver_extension/

https://api.flutter.dev/flutter/flutter_test/WidgetController/tap.html

https://api.flutter.dev/flutter/flutter_test/WidgetTester/enterText.html

https://api.flutter.dev/flutter/flutter_test/WidgetController/drag.html

https://api.flutter.dev/flutter/flutter_test/WidgetController/longPress.html
https://api.flutter.dev/flutter/flutter_test/WidgetController/fling.html

https://api.flutter.dev/flutter/flutter_test/WidgetController/scrollUntilVisible.html

https://api.flutter.dev/flutter/flutter_test/WidgetController-class.html

https://api.flutter.dev/flutter/flutter_test/group.html

https://api.flutter.dev/flutter/flutter_test/setUp.html

https://api.flutter.dev/flutter/flutter_test/tearDown.html

A Simple Automatic Document Cropping in Flutter Using the Canny Edge Detector

João Pimenta — Mon, 09 Jun 2025 01:44:27 +0000

Introduction

In the age of mobile solutions, the ability to scan and manage documents has evolved from a niche feature to a common expectation in modern applications. Whether archiving important documents or interacting with e-government platforms that require uploading forms and receipts, a clear, well-cropped scan can make all the difference.
However, manual cropping is tedious, error-prone, and often leads to inconsistent results. Users don’t want to pinch, zoom, and drag - they expect the app to do the hard work: detect the document automatically, crop it accurately, and deliver a clean, scan-ready image with minimal effort. This is where edge detection algorithms like Canny Edge come in handy.

The Canny Edge detector is a multi-stage algorithm that detects a wide range of image edges. Developed by John F. Canny in 1986, it's a renowned and widely used algorithm in computer vision for its effectiveness and precision. It proceeds through the following phases:

Grayscale Conversion: The initial step converts the input image into a single grayscale channel. This simplifies subsequent intensity analysis by removing color information, focusing solely on luminosity variations.
Gaussian Blur: Smooths the image to remove noise (random variations in pixel intensity) before edge detection.
Gradient Calculation (Sobel Operators): This phase detects the intensity changes (gradients) in both horizontal (Gx) and vertical (Gy) directions using Sobel operators. These gradients highlight regions where pixel intensity changes rapidly, indicating potential edges. The magnitude of the gradient (G) and the direction (θ) are calculated as:

Non-Maximum Suppression: After calculating gradients, many pixels might indicate an edge. Non-Maximum Suppression thins these edges by retaining only the local maxima of the gradient magnitude along the gradient direction. This results in sharper, single-pixel-wide edges.

Double Thresholding: Classifies potential edge pixels into three categories:

Strong edges: Pixels with a gradient magnitude above a high threshold. These are definite edges.
Weak edges: Pixels with a gradient magnitude between a low and a high threshold. These might be edges but need further verification.
Irrelevant pixels: Pixels below the low threshold, discarded as non-edges. Otsu's method can be optionally employed to automatically determine optimal high and low thresholds, adapting to different image lighting conditions and content.
Hysteresis: Connects weak edges to strong ones. This process ensures that only continuous and reliable edges are preserved, effectively filtering out isolated noise.

The output of the Canny Edge Detector is a binary image where the detected borders are prominently highlighted in white.

Our Strategy

Our approach to automatically crop documents leverages the Canny output to precisely identify the document's boundaries is broken down into three steps:

Apply the Canny Algorithm to the original image - One document at a time
Find the outermost borders of the document
Create a crop area based on the previous step, and Crop the image

A detailed full implementation of the Canny Edge Algorithm is beyond the scope of this article (as it would be quite extensive) and won't be provided on this article.

The Code

Apply the Canny Algorithm to the original image

On the code repository, we created a service for the crop, called CropService, so that we can call the crop function anywhere in the codebase without code repetition. Therefore, the code below, we are calling the crop function using a simple button, passing the original image to it.

ElevatedButton.icon(
  style: ElevatedButton.styleFrom(
  backgroundColor: Colors.black,
  padding: const EdgeInsets.symmetric(horizontal: 20, vertical: 12)),
  onPressed: () {
    cropService.cropDocument('assets/img_sample2.JPG');
  },
  icon: const Icon(
    Icons.crop,
    color: Colors.white,
    size: 32,
  ),
  label: const Text('Crop Image',
    style: TextStyle(color: Colors.white, fontSize: 32),
  ),
),

Our crop function is defined as follows

Future<void> processAndTrimImage(String sourceAssetPath) async {
  final rawImageData = File(sourceAssetPath).readAsBytesSync();
  final baseVisualRepresentation = decodeImage(rawImageData);
  if (baseVisualRepresentation == null) {
    print('Failed to decode initial image data from: $sourceAssetPath');
      return;
  }
  await _cannyAlgorithm(baseVisualRepresentation);
  final temporaryWorkArea = await getTemporaryDirectory();
  final edgeFeatureMapPath ='${temporaryPath.path}/map_visual.png';
  File(edgeFeatureMapPath)
    .writeAsBytesSync(encodePng(baseVisualRepresentation));
  print('Edge feature map saved to: $edgeFeatureMapPath');
  await _extractDocument(edgeFeatureMapPath, sourceAssetPath);
}

After this step, we will have the following output and can proceed to the next phase.

Find the outermost borders of the document

The Canny Edge Detector is incredibly effective at identifying all edges within an image, but for our document cropping purpose, this presents a challenge. It will highlight not only the document's perimeter but also internal features like fingerprints, facial outlines, and even individual letters. Therefore, we need a way to isolate the document boundaries specifically.

To do this, we make the assumption that the document is roughly centered in the image. Based on this assumption, we perform the following:

Left Border: We start scanning from the leftmost column, moving inwards towards the center. The moment we hit a white pixel, we record its horizontal coordinate. This marks our minX.
Right Border: Similarly, we scan from the rightmost column, moving inwards. The first white pixel we find gives us the maxX coordinate.
Top Border: We begin scanning from the topmost row, moving downwards. The first white pixel encountered defines our minY.
Bottom Border: Finally, we scan from the bottommost row, moving upwards. The first white pixel here gives us our maxY.

These four coordinates (minX, minY, maxX, maxY) precisely define the rectangular bounding box (assuming the document is not tilted) of your document within the image, setting the stage for the final cropping step

Future<void> _extractDocument(
  String featureMapPath, String originalAssetPath) async {
  final featureMapFile = File(featureMapPath);
  final featureMapRawBytes = await featureMapFile.readAsBytes();
  final featureMapImage = decodeImage(featureMapRawBytes);
  final originalFile = File(originalAssetPath);
  final originalRawBytes = await originalFile.readAsBytes();
  final originalVisual = decodeImage(originalRawBytes);

  if (featureMapImage == null || originalVisual == null) {
    print('Failed to interpret required image data for trimming.');
    return;
  }

  final imageWidth = featureMapImage.width;
  final imageHeight = featureMapImage.height;
  final centralHorizontalPoint = imageWidth ~/ 2;
  final centralVerticalPoint = imageHeight ~/ 2;
  int? boundaryX1, boundaryX2, boundaryY1, boundaryY2;
  const int trimOverlapPixels = 3;

  bool isProminentEdge(Pixel px) => 
    px.r > 12 && px.g > 12 && px.b > 12 && px.a > 12;

  final horizontalScanBuffer = (imageWidth * 0.025).toInt(); 
  final verticalScanBuffer = (imageHeight * 0.025).toInt(); 

  for (int x = horizontalBuffer; x <= centralPoint; x++) {
    if (isProminentEdge(Image.getPixel(x,centralPoint))){ 
      boundaryX1 = x;
      break;
     }
   }

//Repeat for Top->Center, Right->Center, Bottom->Center

Create a cropping area and crop

If all four coordinates are found, we use Flutter's Image package to create a cropping area using the copyCrop() function

if ([boundaryX1, boundaryX2, boundaryY1, boundaryY2].any((b) => 
  b == null)) {
  print('Fail');
  return;
}

final finalCropLeft = (boundaryX1! - trimOverlapPixels).clamp(0, imageWidth);
final finalCropTop = (boundaryY1! - trimOverlapPixels).clamp(0, imageHeight);
final finalCropRight = (boundaryX2! + trimOverlapPixels).clamp(0, imageWidth);
final finalCropBottom = (boundaryY2! + trimOverlapPixels).clamp(0, imageHeight);

final croppedImage = copyCrop(
  originalVisual,
  x: finalCropLeft,
  y: finalCropTop,
  width: finalCropRight - finalCropLeft,
  height: finalCropBottom - finalCropTop,
);

Finally, it just needs to be saved.

final finalProcessedAssetPath = originalAssetPath.replaceFirst(
  RegExp(r'\.(\w+)$'), '_refined_doc.JPG');

await File(finalProcessedAssetPath) .writeAsBytes(encodePng(croppedImage ));
print('Document cropping successful! Output saved to: $finalProcessedAssetPath');

Conclusion

Automatic document cropping is an essential feature in modern applications, transforming the manual process into an efficient experience. By leveraging the power of the Canny Edge Detector and a strategic approach to identify document boundaries, we can achieve highly accurate and consistent results. The multi-stage Canny Algorithm effectively pinpoints edges, and our simple strategy further refines it by locating the outermost borders. This method not only enhances user satisfaction but also ensures the production of clean, scan-ready documents, demonstrating the practical and powerful application of computer vision in everyday mobile solutions.

Comment down below if you have any improvements for this algorithm!

References

https://docs.opencv.org/4.x/da/d22/tutorial_py_canny.html

Demystifying Flutter Layout System: Constraints, Sizing, and Overflow Management

João Pimenta — Sat, 08 Mar 2025 19:31:02 +0000

Introduction

Flutter's layout system is built around a fundamental principle: constraints flow downward, sizes expand upward, and the parent determines the position. This approach provides a structured framework for widget sizing and placement within an application. Grasping this system is essential for developing smooth, flexible apps. In this post, we will explore the inner workings of Flutter's layout mechanics in detail.

Constraints Go Down

Constraints originate from the parent widget and are passed down to its child widgets. These constraints define the minimum and maximum dimensions a child widget can occupy. Every widget must conform to the constraints imposed by its parent while determining its size.

Sizes Go Up

Once a widget receives constraints, it decides its size within those limits. The chosen size is then reported back up to the parent. A widget's size may depend on various factors, including its content, the constraints received, and its internal layout logic.

Parent Sets Position

After determining the sizes of its children, the parent widget decides where to place them within its own layout space. The parent is responsible for setting the exact position of each child, taking into account the reported sizes and any additional layout rules it enforces.

You might be asking: what about the "first" Widget in the Widget Tree? Who decides its constraints? Well, its size is determined by the constraints of the screen itself. Example: A container with no specified size, placed directly as the body of a Scaffold, will expand to fill the entire screen because the screen's dimensions provide the constraints that define its size.

@override
Widget build(BuildContext context) {
  return Scaffold(
    body: Container(color: Colors.amber));
}

If we give this exact container a size, it will adhere to that size, as each widget determines its size after receiving constraints from the parent. Therefore, the Container will be exactly as specified within the given constraints.

@override
Widget build(BuildContext context) {
  return Scaffold(
    body: Container(
      color: Colors.amber,
      height: 100,
      width: 100
    ),
  );
}

Let's delve deeper into our example by adding a Text Widget inside the container.

@override
Widget build(BuildContext context) {
  return Scaffold(
    body: Container(
      color: Colors.amber,
      height: 100,
      width: 100,
      child: Text("Hello, Dev.to!")
    ),
  );
}

Constraints go down: The Container imposes constraints on the Text widget. Here, the constraints might say that the Text widget can be up to 100 pixels wide and 100 pixels tall (the size of the Container).
Sizes go up: The Text widget measures itself to determine how much space it needs to display the text "Hello, Dev.to!". It decides its size within the constraints given by the Container. Suppose the text needs 80 pixels width and 20 pixels height. This size is then reported back up to the Container.
Parent sets position: The Container, knowing the size of the Text widget (80x20), decides where to place it within its own 100x100 space.

ConstrainedBox

Flutter allows developers to specify or force a widget's constraints using the ConstraintBox.

@override
  Widget build(BuildContext context) {
    return Scaffold(
      body: ConstrainedBox(
        constraints: const BoxConstraints(
          minWidth: 30,
          maxWidth: 40,
          minHeight: 30,
          maxHeight: 40,
        ),
        child: Container(color: Colors.amber, height: 100, width: 100),
      ),
    );
  }

Although the container's size is 100, the specified constraints are much smaller, so the container must adjust its size to fit within those constraints.

Tight vs Loose Constraints

In Flutter, tight and loose constraints affect how widgets behave and size themselves. Tight constraints are precise and non-flexible, offering a fixed size for a widget to adopt. This occurs when the minimum and maximum dimensions are the same, meaning the widget must take the exact size specified by the constraints. For example, when the screen dimension constrains the size of a top-level widget, the widget must adopt that exact size. While tight constraints are predictable, they offer less flexibility.
On the other hand, loose constraints provide a range within which a widget can decide its size. These constraints specify both a minimum and maximum size, allowing the widget to choose any size within that range. An example of this is a Column with an unconstrained height, where children can choose their heights within the specified range. Loose constraints allow for more dynamic layouts but require careful management.

Overflowing

Overflow occurs when a widget tries to occupy more space than is available in its parent container or layout. This often happens when a widget's size exceeds the space allocated for it, causing it to "spill over" outside of its designated area. Overflow can happen for various reasons, such as when a widget doesn't have proper constraints or when the constraints allow it to expand indefinitely.
An example of overflow occurs when a ListView, which prefers to expand as much as possible, is placed inside a Column without being wrapped in an Expanded widget. The Column provides an unbounded height to its children, so the ListView attempts to expand infinitely, leading to a FlutterError due to the unbounded vertical viewport.

Exception has occurred.
FlutterError (Vertical viewport was given unbounded height).

Wrapping the ListView in an Expanded widget fixes this issue by giving it a constraint and ensuring it takes only the available space within the Column. However, some overflows won’t throw an exception at runtime. For example, placing a Column inside a SingleChildScrollView.

@override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Column(
        children: [
          SingleChildScrollView(
            child: Column(
              children: List.generate(
                50,
                (index) => ListTile(title: Text("Item $index")),
              ),
            ),
          ),
        ],
      ),
    );
  }

Column does not impose a height constraint on its children, so it can grow indefinitely. When placed inside a SingleChildScrollView, which also does not impose constraints, the Column can grow beyond the screen's boundaries, causing a visual overflow. However, ListView is inherently scrollable, so when properly constrained (e.g., with Expanded), it avoids overflow issues by allowing scrolling.

Conclusion

Flutter's layout system is designed around a clear structure where constraints are passed down from parent to child, the child determines its size within those constraints, and the parent widget positions the child accordingly. Understanding this flow is crucial for managing widget sizes, positioning, and layout behavior effectively.

By carefully applying constraints —tight or loose— developers can create flexible, responsive layouts. Additionally, it's important to be mindful of potential overflow issues, which can occur when widgets are not properly constrained. These can often be prevented with the right use of layout widgets, such as Expanded and SingleChildScrollView. With a solid grasp of these principles, developers can build layouts that are both dynamic and visually consistent across different screen sizes and orientations.

Sources

Flutter Engineering by Majid Hajian
https://docs.flutter.dev/ui/layout/constraints
https://api.flutter.dev/flutter/widgets/ConstrainedBox-class.html

Mastering Flutter Lifecycles: From Application to Render Objects

João Pimenta — Sat, 01 Mar 2025 18:20:24 +0000

Introduction

Continuing our discussion on Flutter Architecture, where we explored the Widget, Element, and Render Object trees (Flutter Architecture Made it Easy) and how a widget is rendered on the screen (Understanding Flutter's Rendering Pipeline: From Widgets to Pixels), we now focus on Flutter’s lifecycle.
We will examine lifecycles from the broadest to the most specific, starting with the Application Lifecycle and progressing down to the RenderObject Lifecycle.
When we talk about lifecycles in Flutter, we refer to a broader concept that applies to Elements, State Objects, and Render Objects. To clarify, the entities that have lifecycles in Flutter are:

The State Object (associated with StatefulWidget)
The Element
The RenderObject
The Application

Surprisingly, Widgets do not have a lifecycle as their immutable nature prevents them from being modified after creation.

Let’s begin with the Application Lifecycle.

Application Lifecycle

Flutter provides two primary mechanisms for managing application lifecycle events: WidgetsBindingObserver and AppLifecycleListener. We'll focus on AppLifecycleListener due to its more organized and thorough approach. It is a listener that detects changes in the application's lifecycle and utilizes AppLifecycleState, an enum that defines the different phases of an application’s lifecycle. The phases are:

Detached

The app is hosted by the Flutter engine but is completely disconnected from the framework. This is the default state before the app starts running.
Triggered when onDetach is called.

Resumed

The app is in the foreground, active, and ready for user interaction.
Called by the onResume. In the past, onStart was also used to trigger this phase, but it seems that it is no longer used.

Inactive

The app is running but transitioning between foreground and background and is not currently interactive.

Non-Web Desktop: The app is not in the foreground but still has visible windows.
Web: The app runs in a window or tab without input focus.
iOS/macOS: The app enters this state during a phone call or while responding to a Touch ID request.
Android: The app may be partially obscured, in split-screen mode, or interrupted by a system dialog, notification shade, or another view.

onInactive and onShow are responsible for triggering this phase.

Hidden

All views of the app are hidden. This occurs when:

The app is about to be paused (iOS/Android).
The app is minimized or placed on an inactive desktop (non-web desktop).
The app runs in a web browser tab that is no longer visible.

onHide and onRestart are the functions used to trigger this phase

Paused

The app is not visible to the user and is not responding to user input.
onPaused triggers the Paused phase.

Source: https://api.flutter.dev/flutter/widgets/AppLifecycleListener-class.html

State Object Lifecycle

State Object is commonly seen and used by the developer in the StatefulWidget, which is an immutable widget that defines the structure of a stateful component. It includes the createState method, responsible for creating an instance of the associated State object and createElement, in charge of creating an element and mounting it to the widget tree. The StatefulWidget itself does not have a state or a lifecycle, instead, it delegates this responsibility to the State Object.
With that in mind, the lifecycle of a State Object can be described below

createState

This method is triggered when a StatefulWidget is created. It generates the State object associated with the widget.
It can be called multiple times throughout a StatefulWidget's lifecycle. For instance, if the widget is inserted into the tree at multiple locations, Flutter creates a separate State object for each location.

initState()

Called once when the State object is first added to the widget tree. Flutter invokes this method exactly one time for each State object it creates.

didChangeDependencies

Executed immediately after initState and whenever the widget's dependencies change. For example, if a previously referenced InheritedWidget changes, Flutter calls this method to notify the State object of the update.

didUpdateWidget

Triggered when the widget’s configuration changes. If the parent widget rebuilds and provides a new widget with the same runtimeType and Widget.key, the framework updates the widget property of the State object to reference the new widget. This method receives the previous widget as an argument.

Build()

This method is responsible for constructing the widget's UI. It is called following initState and didChangeDependencies or whenever there is a need to update the UI.
It is called in several scenarios, including:

After initState
After didUpdateWidget
After receiving a call to setState
After a dependency of this State object changes
After calling deactivate and then reinserting the State object into the tree at another location.

Because it has the potential to be called on every frame, build() should remain efficient. Flutter replaces the subtree beneath this widget with the widget returned by build(), either by updating the existing subtree or by recreating it entirely.
The BuildContext passed as an argument, provides information about the widget’s location within the tree.

setState

Although not strictly part of the lifecycle, setState() plays a crucial role in triggering UI updates. It marks the widget for rebuilding whenever there is a state change.

Element Lifecycle

In Flutter, an element is created for each widget in the widget tree. Elements store information about the widget’s parent, children, and size. If you want to learn more about elements, check out this post.
As for their lifecycle, it is managed by the _ElementLifecycle enum, which defines the different phases an element goes through.

Initial

Element has just been created and still needs to be fully initialized

Active

An element transitions to this state when it is fully initialized and attached to the widget tree. The element spends most of its lifecycle in this state.

Inactive

When it is removed from the widget tree but remains in memory. It still has the potential to be put back on the widget tree. Example: when a parent widget stops incorporating the child in its build method.

Defunct

Element is no longer part of the widget tree and will be removed from memory by the garbage collection.

The Lifecycle

Initial Phase

The framework creates an element by calling createElement().

Active Phase

The element is added to the widget tree when the framework calls mount. Depending on the scenario, it either attaches an existing render object using attachRenderObject , or creates a new one using createRenderObject. At this stage, the element is considered active.
If the parent widget updates, the framework calls update to reconfigure the element with the new widget.

Inactive Phase

At some point, an ancestor might decide to remove this element (or an intermediate ancestor) from the tree, which the ancestor does by calling deactivateChild, removing that element's render object from the tree.

Defunct Phase

If the element does not get reincorporated into the tree by the end of the current animation frame, the framework will call unmount, transitioning to the defunct phase.

RenderObject

Finally, RenderObjects are mutable and manage the size, position, and rendering of their associated elements. Again, If you want to learn more about RenderObjects, refer to this post.

Initialization

Its lifecycle begins in the initialization phase, Usually, who creates a renderObject is a RenderObjectElement. During this phase, the object’s initial state, properties, and relationships within the tree are established.

Layout

This phase involves calculating the size and position of the object. It will determine how the object will fit within the UI.

Painting

The object will translate its information into a visual representation.

Hit Testing

This process identifies which render objects exist at a given position. It is performed using the hitTest.

Disposal

Dispose of the resources of the object.

Conclusion

The Application Lifecycle governs how an app responds to system events, the State Object Lifecycle dictates how stateful widgets manage their state, the Element Lifecycle determines how widgets are mounted and updated within the widget tree, and the RenderObject Lifecycle handles layout, painting, and hit detection.
A deep understanding of these lifecycles empowers developers to optimize widget rebuilding, manage resources effectively, and write more performant Flutter applications. Whether you're designing a dynamic user interface or troubleshooting unexpected behavior, keeping lifecycle management in mind will help you make better architectural decisions.

Sources

Understanding Flutter's Rendering Pipeline: From Widgets to Pixels

João Pimenta — Sat, 15 Feb 2025 20:22:50 +0000

Introduction

Building on my previous post about Flutter’s architecture (You can read it here), we can now address a key question: How does Flutter render everything on the screen? The answer lies in the renderer, the system that converts Flutter’s code—widgets and their associated RenderObjects—into actual pixels on the screen. This multi-stage process begins with the widget tree, where each widget is associated with a corresponding RenderObject. These RenderObjects define both layout (how elements are positioned) and painting (how they appear). Once the layout is determined, the RenderObjects send instructions to the Flutter Engine, which compiles them into an ordered set of commands known as a display list. However, the engine doesn’t handle rendering alone—it leverages the GPU by setting up optimized render pipelines to process the display list efficiently and ensure smooth performance.

Image 1: The phases of the rendering pipeline. Source: https://docs.flutter.dev/resources/architectural-overview

Let's understand deeply each phase of the process.

The Phases

The User Input Phase

Everything begins with the user interacting with the application, such as tapping, dragging, or typing. These interactions trigger event listeners, which in turn modify the application’s state. Any state changes resulting from user input can lead to UI updates, kicking off the rendering pipeline.

The Animation Phase

If the UI includes animations, this phase ensures seamless transitions and dynamic effects. Flutter’s animation system continuously updates animated properties—such as position, opacity, and size—based on time. It takes into account factors like animation duration, easing curves (e.g., linear, ease-in-out), and the current frame rate to deliver smooth and fluid motion.

The build phase

The build phase involves calling the build() method, after which the framework efficiently compares the resulting widget tree to the previous one, updating only the necessary parts. This method is defined in two key places—within StatelessWidget and State. However, before exploring both, it’s important to first understand what state is in Flutter, as it plays a crucial role in how build() functions work and the differences between them.

The State object holds data that can be accessed synchronously during the build process and may change throughout the widget’s lifetime. Essentially, a widget’s state consists of the values stored in its properties at the time of creation, which can be modified in response to user interactions. For instance, when a user taps a checkbox, its state updates, prompting a rebuild to visually reflect the change.

In a StatelessWidget, the build() method defines the user interface purely based on the widget’s current configuration. The Flutter framework calls build() when the widget is first inserted into the widget tree or when its dependencies change, such as when an InheritedWidget it depends on changes. During this process, the framework replaces the widget’s subtree with the one returned by build(), either updating the existing subtree or recreating it from scratch, depending on the behavior of the canUpdate() method. Typically, StatelessWidget implementations return a composition of widgets, structured using the widget’s constructor and the provided BuildContext.
To optimize performance and minimize unnecessary rebuilds, follow these best practices:

Reduce widget creation in build() by avoiding excessive nesting of Row, Column, Padding, and SizedBox in order to position a child in a fancy way. Instead, use alternatives like Align or CustomSingleChildLayout. Similarly, replace multiple nested Container layers with a single CustomPaint when feasible.
Use const widgets: Declare widgets as const whenever possible and provide const constructors for custom widgets. This allows Flutter to reuse widget instances and optimize rendering.
Favor widgets over helper functions: When building reusable UI components, define them as widgets rather than functions. Widgets offer better rebuild optimization, and marking them as const further enhances performance by preventing unnecessary rebuilds.

The State class’s build() method describes the widget’s user interface and is called at various points in the widget’s lifecycle. These include after initState, didUpdateWidget, setState, when a dependency changes, or when a widget is deactivated and later reinserted into the tree. The build() method returns a new composition of widgets, constructed using the widget’s constructor, the given BuildContext, and the internal state maintained by the State object.

The Layout phase

With the widget tree built (or updated), Flutter moves to the layout phase, where it determines the size and position of each widget.
Flutter follows a constraint-based layout system, traversing the render tree in a depth-first manner. During this process, parent widgets pass constraints (such as minimum and maximum width/height) down to their children. In response, each child calculates its size within those constraints and sends it back to its parent.
This phase is also when layout-specific widgets like Row, Column, and Stack position their children accordingly. By the end of this single tree traversal, every object has a defined size within its parent’s constraints and is ready for the next step—rendering—by calling the paint() method.

The Paint Phase

With the size and position of each widget known, the paint phase takes over. Each RenderObject's paint() method will be invoked, defining the visual appearance of the widgets. The paint() takes a PaintingContext and an Offset, where the first provides methods for drawing while the latter determines where the object should be placed on the screen. During this phase, the RenderObjects generate painting instructions which are sent to the Flutter Engine. The Engine then compiles these instructions into a display list, which is used for rendering.

The Compositing Phase

In this phase, Flutter will organize the painted widget into layers, where different parts of the UI are drawn on separate layers. This is essential for performance, especially when dealing with complex animations or overlapping elements. By compositing these layers efficiently, Flutter can optimize the rendering process and avoid redrawing the entire screen for every small change. For example, if you have an animation that moves a widget over a static background, only the layer containing the moving widget needs to be redrawn.

The Rasterization Phase

In this final stage, the composited layers are converted into pixels that can be displayed on the screen. This process is hardware-accelerated by the GPU which takes the layered rendering information and transforms it into the final image that you see on your device.

Conclusion

Flutter's rendering pipeline is a sophisticated, multi-stage, and highly optimized process that transforms code into the visual experiences users interact with. Each step plays a crucial role. By understanding this pipeline, developers can write more efficient Flutter applications, optimize widget usage, minimize rebuilds, and leverage the framework's capabilities to create smooth and engaging user experiences. This intricate process, working behind the scenes, is what empowers Flutter to deliver its cross-platform, high-fidelity visuals.

Sources

Flutter Architecture Made it Easy

João Pimenta — Thu, 06 Feb 2025 23:30:27 +0000

Flutter is a cross-platform UI toolkit that enables developers to write the code once and deploy it across multiple platforms, such as Android, iOS, Web, Windows, and Linux. Think of it as a universal screwdriver that can tackle any screw - on any surface. It aims to be a versatile tool whose goal is to build high-performance, platform-native feeling applications, embracing platform differences while maximizing code reuse.

The development process leverages a VM to enable stateful hot reloads that instantly reflect code changes without recompilation. For release builds, however, the apps are compiled directly to platform-specific machine code, including Intel x64, ARM instructions, or JavaScript. For Android, Flutter's Embedder is written in Java and C++; Swift and Objective-C on iOS; and C++ on Windows and Linux.

In this article, I’ll simplify the abstract concept of Flutter’s architecture, making it easy for everyone to understand. So, grab your cup of coffee (or tea) and some snacks, and join me on this journey!

Architectural Layers

Flutter's architecture is designed as a layered system, where independent libraries build upon each other, creating a flexible and extensible toolkit.

Image 1: Architectural layers of Flutter. Source: https://docs.flutter.dev/resources/architectural-overview

Layer 1: The Embedder – Platform-Specific

At its lowest level, this layer is responsible for how Flutter applications will interact with the operating system (OS). It handles the crucial task of translating Flutter's instructions into a language the OS understands, making it look native to the eyes of the OS. It provides the entry point, coordinating with the OS for access to essential services like rendering, accessibility, and input, allowing Flutter code to be integrated as a module within existing native applications.

Layer 2: The Engine – C/C++

Essentially, the Engine translates your code into the pixels you see on screen. It handles the rasterization of composited scenes for every new frame, providing low-level implementations of Flutter's core APIs, encompassing graphics, text layout, file access, and networking. It is accessible through the dart:ui library, which wraps the C++ code in Dart classes, exposing low-level primitives for graphics and text rendering.

Layer 3: Framework – Dart

The Framework, which is the layer we as developers interact with, comprises four inner layers, and it's a lot like playing with Legos. You start with the basic pieces and build them up step by step, each layer adding more complexity and functionality to your app. It is made of 4 inner-layers:

Foundational Layer: This is like the Lego pieces you start with, simple and essential tools for creating everything. It gives you the building blocks for things like handling touches, drawing images, and adding animations. It provides low-level utilities like gestures, painting, and animation, like GestureDetector and CustomPainter.
Rendering Layer: Once you have your pieces, this layer helps you figure out where to put them and how they should be arranged. It’s like designing the structure of your Lego creation, ensuring everything fits and looks the way you want. It allows developers to build a tree of render objects (more on this later), which defines how elements are drawn on the screen such as the RenderBox.
Widget Layer: This is where you add the details to your Lego house, like windows and doors. Every piece in the rendering layer has a corresponding widget here, which defines the UI elements and how they should be put together. The example would be the Container – which is made of smaller widget such as LimitedBox, ContrainedBox, Allign, Padding, DecoratedBox, and Transform.
Material Layer: Finally, this layer gives you pre-built Lego decorations and furniture. It provides ready-to-use UI components, like buttons and app bars, that you can easily add to your app without having to build them from scratch. These are high-level libraries that provide ready-to-use UI components, like MaterialApp and Scaffold.

Image 2: Think of Flutter's layers like this: the Embedder is the translator, converting Flutter's instructions into the operating system's language. The Engine is like the artist painting on a canvas, rendering the visuals. The Framework is the architect, providing the building blocks for the application, which is like playing with Legos.

Widgets and The Trees

In the previous section, we explored Flutter’s Framework layer and how developers interact with it using building blocks similar to Lego pieces. These blocks, known as Widgets, are the core of Flutter development—hence the motto: Everything is a widget. Widgets serve as the fundamental UI components, nested together to build complex interfaces. From basic elements like text and buttons to advanced components like grids and sliders, everything in Flutter is a widget. This approach enhances flexibility and reusability, as functionalities are implemented as widgets rather than mere properties.

Widgets, however, only serve as a blueprint or description of what the UI should look like, but don't do anything themselves. They are immutable. The actual work of creating and managing the UI is handled by other components. Flutter takes these widget configurations and creates corresponding Elements, which is an instantiation of a widget that exists in Flutter’s UI hierarchy. Each Element holds a reference to its associated widget and is responsible for creating and managing the underlying RenderObject, which in turn, is responsible for the actual layout, painting, and hit-testing of the UI. They dictate how visual elements are drawn and positioned on the screen.

Together, these components create Flutter’s powerful tree-based architecture, which consists of three interconnected trees: the Widget Tree, which defines the UI structure; the Element Tree, which manages widget instances; and the Render Tree, which controls the actual rendering and layout. Understanding how these trees interact is key to grasping Flutter’s reactive framework and efficient UI updates.

Image 3: The trees of Flutter: Widget Tree, Element Tree and RenderObjectTree

Widget Tree

It is a hierarchical arrangement of widgets that defines the structure and properties of the user interface. Since widgets are immutable, any changes to their parameters or state result in them being rebuilt with updated values. This structure follows a tree-like format, where each node represents a widget, establishing a parent-child relationship.

Element Tree

For every widget in the widget tree, Flutter creates a corresponding element, resulting in the element tree. These elements store information about the widget's parent, children, size, and its associated RenderObject. The element tree plays a critical role in maintaining the stability of the widget tree, managing the lifecycle of widgets and their corresponding RenderObjects. A key difference between widgets and elements is that elements are mutable and they are responsible for handling reconciliation, which involves comparing the widget's previous state with its updated state and propagating updates to its child.

RenderObject Tree

The RenderObject tree handles the actual layout and rendering of the UI. Flutter uses this tree to perform layout calculations and paint the screen. RenderObjects are mutable and manage the size, position, and rendering of their associated elements. Flutter performs layout by traversing this tree in a depth-first manner, passing size constraints from parent to child. Each child must adhere to the constraints set by its parent when determining its size. In response, the child reports its final size back to the parent while staying within the provided constraints. At the end of this single walk through the tree, every object has a defined size within its parent's constraints and is ready to be painted by calling the paint() method. The RenderObject Tree utilizes property caching, a mechanism that stores certain computed properties of a RenderObject, preventing the need for recalculating them each time the object is repainted.

Image 4: Representation of the three trees with their respective nodes. Source: https://docs.flutter.dev/resources/architectural-overview

This raises the question: why does Flutter have three separate trees? The answer lies in efficiency. When a layout change occurs, it's most effective to update only the necessary parts of the layout tree, specifically the RenderObject tree. If all trees were combined, updating the layout would require traversing many unnecessary nodes, slowing down the process. By maintaining separate trees, the Widget tree, Element tree, and RenderObject tree, Flutter effectively separates concerns within the UI rendering process - which enables efficient updates and optimal performance by reusing elements rather than rebuilding the entire interface. This separation is a fundamental aspect of Flutter’s high-performance rendering capabilities.

The separation of trees offers further benefits by creating a clearer division of responsibilities. Widgets can remain declarative and focused on configuration, while RenderObjects handle the complexities of rendering. This separation reduces complexity, lowers the risk of bugs, and simplifies testing. It also enhances layout safety: The RenderObject tree can enforce type correctness at runtime, ensuring, for example, that a RenderBox only receives child RenderObjects using box coordinates. Combining the RenderObject and Element trees would necessitate additional checks and complicate widget design, requiring each widget to be aware of its children's specific layout constraints and coordinate systems.

A Tree Example

Let’s say we are developing a simple Flutter application consisting of a Padding widget with a Text child. How would Flutter represent this in its internal tree structures? And what happens when the state changes?

Flutter starts by building the Widget Tree, by adding the Padding Widget

Next, Flutter creates an Element Tree, which acts as a bridge between widgets and their underlying rendering objects.

The SingleChildRenderObjectElement will then ask Flutter to create the RenderPadding, an object responsible for positioning and sizing its child with padding around it.

At this point, the Padding Widget will require its child and the cycle of creating element and renderObject restarts, but this time with the Text Widget.

At this point, our trees are complete. But what happens if the state changes to update the text from “Hello World” to “Hello Flutter”?

Flutter will examine the Element Tree and RenderObject Tree to determine the most efficient way to update the screen. Both the element and render object will assess which parts can be reused and which need to be replaced. Specifically, the TextElement will invoke the canUpdate() function (https://api.flutter.dev/flutter/widgets/Widget/canUpdate.html), which checks whether the old and new widgets share the same runtimeType, ensuring that only necessary updates are made.

static bool canUpdate(Widget oldWidget, Widget newWidget) {
  return oldWidget.runtimeType == newWidget.runtimeType
      && oldWidget.key == newWidget.key;
}

If canUpdate() returns true

The existing element remains in place.
The old Widget is replaced, and the element updates its widget reference.
If the widget contains properties that affect rendering (e.g., updated text, color, or padding), Flutter applies those changes to the existing RenderObject instead of recreating it.

Since the structure remains the same, only necessary parts of the UI are repainted, improving performance.

If canUpdate() returns false:

Dispose of the old widget’s element and the element is removed from the tree.
A new element and renderObject are created for the new widget.

The process involves disposal and recreation; it can be more performance-intensive than updating an existing element.

Conclusion

Flutter’s architecture, with its layered approach and tree-based structure, is designed to optimize performance, maintainability, and flexibility. By separating concerns into the Embedder, Engine, and Framework layers, Flutter ensures efficient rendering while embracing platform differences. The Widget, Element, and RenderObject trees further enhance this efficiency by enabling precise UI updates without unnecessary re-renders. Understanding these core concepts is key to mastering Flutter development, allowing developers to build high-performance, visually rich applications across multiple platforms. As we dive deeper into Flutter’s inner workings, it becomes clear why it has become a preferred choice for modern app development—offering both power and simplicity in a single framework.

Sources

Flutter Engineering by Majid Hajian
https://docs.flutter.dev/resources/architectural-overview
https://api.flutter.dev/flutter/widgets/Widget/canUpdate.html

Accessibility For The Elderly in Mobile Applications: an analysis for optimal UX Design

João Pimenta — Sun, 02 Feb 2025 03:48:42 +0000

Introduction

In my last semester of college, I took a class on Human-Computer Interaction where we had to conduct a study as the final assignment, and I was immediately drawn to a topic about accessibility. I began thinking about my mom and grandparents and how they struggle with their smartphones – the tiny buttons, the confusing menus, the small text. It turns out, they're not alone. It is estimated that 5.31 billion people have at least one mobile device, which is 62.5 percent of the world’s population [1][2].

It's easy to forget that not everyone experiences the digital world similarly. The World Health Organization estimates that a billion people with disabilities struggle to use websites and mobile apps. This includes people with visual, auditory, cognitive, and motor impairments, with older adults being a rapidly growing segment of this population. Unfortunately, many apps aren't designed with these users in mind. Studies have shown that common features like small buttons, complex menus, and tiny text create significant barriers for older adults. In fact, research by Ballantyne and Acosta-Vargas found that 90% of the most popular apps in the Google Store violate accessibility guidelines![2]

To see this in action, I conducted a study to explore how these accessibility issues affect older adults. I wanted to understand if simple design changes could truly make a difference. So, I created two versions of the same app: one with accessibility features turned ON (larger text, simplified menus, etc.), and the other with a standard, "OFF" layout. Imagine two versions of the same app: one designed with accessibility in mind, the other not. Which one would your grandparents find easier to use? I observed two groups of elderly users interacting with an app. One group used a version with key accessibility features like larger text and simplified menus (the "ON" version), while the other used a standard layout (the "OFF" version). I then asked them to complete common tasks like finding a product, adding it to the cart, and etc. The results were eye-opening as it quickly became clear that even small design changes could make a huge difference in how easily elderly users could navigate and use the app. Here are my findings that may help you make your application more accessible to old people.

Figure 1: Prototype of the application. It can be seen the design differences for each group and the numbered features that were investigated 1) Navigation tools 2) Color contrast 3) Space between features 4) Font size 5) Diagonal size of features 6) Icons.

Results

Completion Time, Clicks, and Error Rate:

Ideally, when designing a mobile application, software engineers and developers want the user to perform a task easily. The application should be intuitive to allow everyone to use it. Such tasks can be searching the internet, liking a friend's photo, or buying food online. If the application is not intuitive or has a bad design, the user can take long periods to complete a task, risking losing user engagement.

The completion time of the tasks, the total number of clicks, and the error rate of each participant were recorded, and are shown in Table 2. The error rate was calculated by dividing the exceeding number of clicks of each participant by the total number of clicks necessary to complete the experiment, which is 36.

Table 1: Completion time, number of clicks and error rate of each participant.

Analyzing Table 1, it can be seen that the average completion time was different between the two groups. The participants in the Accessibility On group were approximately three minutes faster than those in the Accessibility Off group, with the minimum and maximum being 2 and 10 minutes for the ON group and 5 and 14 minutes for the OFF group.

Interestingly, the total number of clicks and error rate also followed the same pattern as the time completion, in terms of the Accessibility On group having better results. An average of 38 clicks and a 5.5% error rate for the ON group versus 42 clicks and a 16.17% error rate for the OFF group. It can be suggested that the UX design differences between the two groups impacted all three measurements, with the error rate being the most affected. This indicates that the UX design has a crucial role in how the users interact with the app, affecting the completion time and how much they will commit mistakes.

Overall Experience

In addition to the parametric test, the participants for this study also completed a questionnaire at the end of the trial. The first question of the questionnaire was about the overall user experience and how easy to use the app was for the participant. The accessibility on group had overwhelmingly better results compared to the OFF group: the ON group had a mean of 8.9 rating against 3.46 for OFF. The second question can be used to better understand why such a big difference in rating between the groups: when participants were questioned about "What Features of the app did you have most trouble using?", most of the responses indicated that the issues were coming from the OFF group.

From the accessibility OFF group, 11 participants selected "Icons" as the hardest feature to understand, followed by 8 for "Text", 6 for "Buttons", 2 for "Colors" and "Navigation", 1 for "All of them" and 1 for "None of them". On the other hand, from the ON group, 9 participants answered "None of them", 1 for "Buttons", 1 for "Icons" and 2 for "Navigation".

It is visible that the UX design for the ON group positively impacted the user experience as 69% of the participants in this group had no issues at all ("None of them" option selected) with the features being studied, against the 7% in the OFF group (Only one participant in the OFF group did not have issues).

Icons

"Icon" was the feature with which participants had the most trouble. 69.2% of the participants had an issue understanding what each icon meant. Interestingly, even participants in the ON group said it would be easier if text was used with the icons, even though the accessibility-on group design had text with the icons. This corroborates with Franz and Wobbrock, who state that older people have extreme difficulty understanding and dealing with icons [3].

Regarding the buttons rating, the accessibility On group had an average of 4.69 rating while OFF had an average of 2.3 rating. In the short answer question, 6 participants had very similar responses when questioned about "How would you make the buttons better to use?". Participants 5 and 14 answered: "Larger", Participants 6, 8, and 10 wrote: "Bigger" and Participant 9, "Wider". This corroborates with Khan, Prasara, and Bogdan who described that elders find the buttons to be too small in the majority of the mobile applications [4][5][6].

Figure 2: Moving the cart icon from the app bar to the navigation allows us to use text with it. It can be a way of solving the issue of participants not knowing what the icon means.

Text

An interesting finding was for the Font size feature. Exactly 50% of the participants responded that it would be helpful if the font text were bigger. Even more fascinating was the fact that every one of the ON group answered "No," while every one of the OFF group answered "Yes" when questioned: "Would it be helpful if the font text were bigger?", suggesting that the font size is a significant problem for older people.

This study used a font size of 30 for the accessibility-on group and 15 for the OFF group, which contradicts the findings of Bogdan, who suggested a minimum of 14 points for more elderly-friendly designs [6]. With these findings, it can be suggested that, instead of 14 points, a minimum of 30 points for the font size is more appropriate.

It seems that the larger font size in the accessibility-on version made a significant difference in readability for the participants. This highlights the importance of considering larger font sizes than previously recommended in designs for older adults.

Navigation, space between features, and color contrast

One intriguing result from the questionnaire is that no participant, either in group ON or OFF, responded that they had an issue with the spacing between elements. However, it was observed in the OFF group that participants 14, 19, 21, and 25 had issues identifying the product cards on multiple pages. The cards contained information about each product, such as the name, price, and picture, and they were selecting the wrong product, thinking they were selecting the correct one. A question to be asked is if this problem was due to the color contrast (since all the cards and the background were white), the spacing between the cards (which was set to be minimum), or a mix of both.

In addition, in the OFF group, many participants did not know which of the "Add to cart" buttons, "Increment," and "Decrement" buttons were relative to a certain product (Figure 3). The cards were so close together, and there was no color contrast that they could not differentiate if the buttons were in the card of the product they were interested in or in the previous product card. Figure 4 shows how this problem became more evident due to the fact that they could not read the text inside of cards (the name of the product) and used the images as a reference to search for products. Most likely, the lack of spacing, color contrast, and the small size of the product title were responsible for the confusion of many participants in the OFF group. It can be suggested that the lack of color contrast and spacing between the cards, allied with the small font size, resulted in difficulty identifying the correct button for the product, which resulted in a higher error rate for the OFF group.

Figure 3: Participants of the OFF group could not read the description of the products and were using the image as a reference. However, they could not identify if the top or bottom "Add to cart" button was relative to the product they were looking for.

Most participants, either from the ON or OFF group, had no issues with the navigation features. Out of the 26 responses to the question, "Did you have a hard time navigating through the app?", 84.6% answered "No." Even though all the "Yes" responses were from participants from the OFF group, it may be concluded that navigation is not a very serious issue for the elderly.

Conclusion

This study used two different UX designs—one following the guidelines of elderly accessibility and the other using the most common feature design found in many applications available in the Google App Store—to test if elder performance would be affected. The results have shown significant differences in performance between the two groups, with the difference in completion time average being almost three full minutes and an error rate three times higher when not using accessibility features. In addition, with the questionnaire responses, it was possible to draw some interesting conclusions regarding how best to use icons, font size, and color contrast to make the application more elder-friendly and universal for everyone.

References

[1] Datareportal. [n. d.] Digital around the world - datareportal – global digital insights. (). https://datareportal.com/global-digital-overview.
[2] Patricia Acosta-Vargas, Belén Salvador-Acosta, Rasa Zalakeviciute, Katiuska Alexandrino, Jorge-Luis Pérez-Medina, Yves Rybarczyk, and Mario Gonzalez.2020. Accessibility assessment of mobile meteorological applications for users with low vision. In International Conference on Applied Human Factors and Ergonomics. Springer, 199–205.
[3] Rachel L Franz, Jacob O Wobbrock, Yi Cheng, and Leah Findlater. 2019. Perception and adoption of mobile accessibility features by older adults experiencing ability changes. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility, 267–278.
[4] Khan Kalimullah and Donthula Sushmitha. 2017. Influence of design elements in mobile applications on user experience of elderly people. Procedia computer science, 113, 352–359.
[5] Prasara Jakkaew and Tew Hongthong. 2017. Requirements elicitation to develop mobile application for elderly. In 2017 International Conference on Digital Arts, Media and Technology (ICDAMT). IEEE, 464–467
[6] Ioana Iancu and Bogdan Iancu. 2020. Designing mobile technology for elderly. a theoretical overview. Technological Forecasting and Social Change, 155, 119977.