DEV Community: Olivier Revial

Flutter Vikings - Day 2

Olivier Revial — Fri, 02 Sep 2022 09:46:00 +0000

After a great first day of Flutter Vikings conference in Oslo, let's dive into the second day with the conferences I have seen !

The cookbook of Flutter testing

Teresa Wu started this talk by stating that the might important thing about test might not be a very high code coverage (or at least that aiming for it was not absolutely necessary) but that instead we should aim at doing proper testing.

She then described the different kinds of tests that we can think about in an app :

Behaviour Driven Testing (BDD) with TDD
Unit tests (for data and domain)
Acceptance testing (for domain)
End-to-end and snapshot testing

She particularly insisted on BDD testing in this talk as it might be the less well-known and yet a very important one. Different steps are BDD were described as follows :

Discovery story
Backlog / User acceptance
Example mapping (small meetings to ask all the questions, trying to think about the edge cases, particular scenarios such as failing cases, ...)
Gherkin scenario, using a gherkin package in Flutter to actually write the tests in a spoken language

Here is a small example of example mapping :

To end this talk Teresa talked about widget testing, integration testing and also contract testing to make sure APIs clients behave as expected.

Parallel Beauty with Isolates into the Mandelbrot set

Despite a lot of technical difficulties during the talk, Thomas Burkhart still did a great presentation. He first introduced how Dart is multi-threaded even though we tend to think of Dart as a single-threaded language because we usually only use the main thread in an app.
He also explained the differences between single-threading, multi-threading and parallel processing.

After this useful introduction it was time to dive in the code to learn more about Dart Isolates. Although compute() is great to create small isolates, Thomas showed us in great details how you can manually handle isolates : message streams with send and receive ports, how global variables, static variables and parameters are always passed as copies, how to handle a pool of isolates... Very technical and very instructive too, we definitely learnt a lot!

Writing Flutter apps in Lego style

This title was definitely catchy, I had to see what it was really about!

In this recording talk by Anna and Oleksander Leushchenko (unfortunaly Oleksander could not join in person because of the situation going on in Ukraine 🇺🇦 😢), they started by explaining how Lego is so powerful and why you can still today arrange 1970 bricks with 2022 bricks : extreme modularity.

Although code might not be able to reach such a level of modularity, modular programming exists and we might try to use it.

Basically modularity in Dart resides in making modules (duh! ) aka packages (at least to keep things simple), and all the code needs to rely on interfaces.

Anna and Oleksander then started to explain how we can use (de)composition to achieve greater modularity in our code. The first step is to separate our code in three main parts :

App
Features
Core

Each feature will basically be in its own package, core will be for common things such as design system and app will be here to connect all the features together, handle external navigation and such.

They then had a dive at the feature part which is probably where most of the code will be written. Features can be further decomposed into:

Domain, with models and services
Dependency Injection
Configuration, with either local feature flags, remote feature flags or flavors
Localization, that can either be set through common i18n or through feature-local i18n
UI and state management for only feature-specific parts. As stated previously shared widgets and design will come from the core package
Navigation, for the feature-internal part. External navigation will be handled later by the app package
Tests. As features are very isolated from each other, they are easy to test and should probably tested with BDD (using Gherkin), testing only the entrypoint of the feature as the rest will be called by the feature and is implementation details. Also greater isolation allows for easier testing because the feature can be tested almost without mocks, just adding HTTP interceptors to mock API / backend.

Finally Anna and Oleksander presented various tools to help handling the number of packages, such as Mason to write templates for easier feature package generation and Melos for the "orchestration" part (compiling in a certain order and such).

Building better Bingo backends

In this very fun talk, Frank Van Puffelen and Eric Windmill from the Firebase team took Bingo as an example app for setting a Firestore database (and only that) to create a multi-player app with both clients and host.

They went through the basics of Firestore : how it is document-structured, how it is real-time and you can get snapshots updates as soon as there are changes in the database, how you can use security rules to secure database access (because obvisouly you don't want your clients to be able to touch to the host part) and more features.

Very fun talk while instructive on Firestore basics!

Game Development in Flutter

The last talk I saw on Day 2 was from Filip Hracek where he gave a very honest feedback on Game development when you try to make your own as a personal project (i.e. not working for a big game studio).

In the first part he asked why we, as developers, initially want to develop a game. He took an interesting parallel with movies using a Scorsese quote. Basically, we want to create game because we can do what we want with it, potentially passing a message to players, by taking much more risks than a big studio.

He then explained the various reasons why games fail, often because life happens, or it's just too much work and we lack discipline, or we were stuck on a specific aspect of game design, architectural issue or perf issues. The answer to this was basically to know from the beginning that developing a game is work, it's not just fun all the way.

In the final part he gave some advices to avoid problems stated before. It's okay if it's not beautiful from the very beginning, it's okay if it feels hacky (after all the players don't care!), you can start small and don't need everything done at first. Also, don't reinvent the wheel and finally : try to be your own Product Manager, you have to set boundaries to yourself!

After Filip talk we were close to the end of the Day, but there was a great Q&A session with Dart, Flutter and Firebase teams that answers attendees questions. A great way to get honest answers for the future of Flutter, Dart and Fireb

I must say that it was a great way to end this awesome Flutter Vikings conference.

The end.

...until next year ? 🤞

Flutter Vikings - Day 1

Olivier Revial — Wed, 31 Aug 2022 16:32:49 +0000

This year I have the priviledge of attending Flutter Vikings, the big european conference about Dart and Flutter.

Before I start talking about the great talks I have seen on Day 1, let me say that Flutter team teased a new Flutter version the day before the event : Flutter 3.3. Here is the announcement and here is the changelog. This newest version aims at stabilizing Flutter 3 with lot of useful fixes and a few small features too. Oh and as always, a major Flutter update comes with a Dart update too, today we get Dart 2.18, with no outstanding news thought.

Now let's jump to the subject : Flutter Vikings Day 1 !

Deep dive into Flutter theming

In this very first talk, Mike Rydstron presented various theme related concepts.

He started by presenting how to use ThemeData the right way as there are like 8 different ways to declare colors... ThemeData(), ThemeData.from(), ColorScheme(), ColorScheme.fromSeed(),... Curiously, they don't all work in the same way and some of them might miss a few colors that you then have to fix manually.
The talk continued with the current state of Material 3 and where it stands compared to Material 2.

In the second half of the talk, Mike showed how to create beautiful color schemes and presented a few packages and methods he developed, for example flex_color_scheme to add more flexibility to how color schemes currently work in Flutter Material 3 implementation.

The talk was ended with a demonstration of Theme Playground, a very nice application to play and generate theme related code!

Fun with code generation

Anna Leucshenko gave a great talk on how to use code generation with build_runner package to write less boilerplate code. In this talk she used two applications to compare how code generation can simplify an app code. She went through quite a lot of packages which are :

freezed to generate models with constructors, equals, hash code, copyWith and other awesome features such as pattern matching
json_serializable to generate toJson() and fromJson() methods
retrofit for REST APIs
auto_route for Flutter navigation
flutter_gen for assets linking (making images accessible through constants instead of specifying a path)
injectable to use effective dependency injection with get_it
i69n for localizations
bdd_widget_test to write behavior driven tests using Gherkin

You should definitely check this talk and the packages if you want to learn more about code generation

Flutter At Scale - Experience From 25+ Flutter Devs Working Together On A Mobile Banking App

This talk was a very interesting feedback of two developers working on a large team of 25 Flutter devs and an overall 200 people team. In such project you need to be pretty organized in order for your codebase not to become a real mess.

The speakers led us to various ways of dealing with big projects. Code ownership was the first subject as you need to have people responsible for every piece of code and you cannot just say that the team is responsible for everything.
The following points were focusing on the project structure and architecture : organizing the files by feature, having a robust design system, using atomic design to separate the widgets.

At the end they presented their storybook, a very nice way of showing every component in their different states (e.g. a button that might be enable / disable, hover, etc)

Responsive UI for auth flows

In this talk Salih Guler from AWS Amplify introduced concepts that revolve around adaptative and responsive layouts. The first part of the talk was to explain the difference between those two terms : adaptive is more about adapting the layout, e.g. the small burger menu becoming a left pane on bigger devices, whereas responsive is more about making everything fit within the available space, depending on screen size.

The second and biggest part of the talk was a very nice live coding where he demonstrated a lot of useful widgets :

LayoutBuilder
Media Query.of()
FractionnallySizedBox
OrientationBuilder
FittedBox
AspectRatio
CustomMultiChildLayout

And of course the more obvious and known ones such as Row, Column, Flexible, Expanded and so on.

Understanding Flutter's handling of gestures

Guillaume Diallo-Mulliez explained in plain details how the gesture detection and handling work in Flutter, and how we should do if we ever wanted to implement our own custom gesture detection or gesture listening.

This talk is a bit hard to transcribe because it was really technical but Guillaume talked about GestureDetector, RawGestureDetectore, pointers and PointerEvent, gesture Listener, hit tests and so on.

If you are interested I can only encourage you to watch the replay as it will be much more valuable than a text transcription!

Riverpod 2.0

Last but not least, Rémi Rousselet - famous author of freezed, provider and riverpod packages - gave us a few hints about what Riverpod 2.0 (one of the most-used state management library out there) could look like in, hopefully, the near future! New state inspector in DevTools for Riverpod, custom linter, new documentation generator with a sweet dependency graph,... Many awesome features!

But then we had the main and best part of the talk where Rémi did an awesome live coding session showing a potential future step-by-step tutorial for Riverpod using new annotations to infer providers type and simplify overall use of Riverpod for beginners.

Hopefully it will be available on pub.dev soon because it looked great 🔥

Anyway, that sums up this awesome first day of Flutter Vikings 2022, see you tomorrow for part 2!

Flutter : a journey to higher-quality apps

Olivier Revial — Tue, 26 Jul 2022 11:21:00 +0000

Ever been scared of pressing that CI "release" button ? Do the words "bugs" and "regressions" ring any bell ?

It's time to leave the fear behind and start delivering higher-quality apps !

In this article I will try to show you why code quality is important and how to improve it in your Flutter app through various tools and techniques. Hopefully by the end of this article you will have the keys to deliver your apps with more confidence 🚀

You said code quality ? 🤔

Before jumping right to the solutions, I think it's important to do a small reminder of what code quality is, and why it is (very very very) important.

Code quality is a metric that usually tell whether a given code is good or bad. Obviously this is very subjective and differ from one project to another but it doesn't really matter because the goals behind code quality are almost always the same :

Increasing confidence & reputation 👍

Delivering a software with confidence is extremely important because that way you do not fear new releases and as you deliver with more confidence, you can deliver more often, making your delivery cycle a virtuous circle. And as developers we are always happy not having to work on the week-end because the latest release was full of bugs. 🦟 🪲

Less risk, more fun.

Obviously, your company reputation is also at stake here : the more perceived software quality (fast, secure, reliable, etc) your users get, the more confidence they will have to trust you and keep using your app (and even recommend it to others).

Increasing code reuse ⌨️

Unless you are developing a one-time POC (and even then, beware the "POC to production" effect !), you will spend a lot of time developing and maintaining your app : it could be weeks, months or even years ! The more quality you put at first, the less cost you will have to add further developments in the future.

In the other hand, low quality code would not only make you pay more in the long term but could really damage your company business as any change to the app could get really tricky and will potentially freeze your project.

What metrics make for a good code quality app ? 📊

I won't go into details here because other articles already describe this, but basically we can find :

Reliability. How probable our app will run without unexpected failures.
Maintainability, extensibility and reusability. How easy it is to add new code around existing code or change existing code.
Testability. How easy it is to test the app.
Portability. How well your app will behave in different environments. For a Flutter mobile app you should make sure that your app is running as well on Android and on iOS. For a web or desktop app, you also might want to test multiple browsers or OS platforms.

Code quality in Flutter 📱

Code quality in Flutter is no different from code quality in any other language, framework or anything else in general. Linters, tests, CI, reviews, refactoring, coding conventions, code quality measurement tools...

Fortunately for us, Flutter has them all !

I will now show a few tools that can be used to improve, guarantee and measure our code quality in Flutter. Note that my list might not be exhaustive (please add your tools in the comments !), and is in no particular order. Finally, it is important to understand a few things when talking about code quality :

Good/experimented developers tend to write better quality code. Do not rely solely on tools below to claim that your code is good, rather think about the code you are writing and try to write the best code you can. Tools are only here to help you, but they won't think for you. This is particularly true for business matters where tools can hardly verify what's actually expected.
You should not pick a single tool and think that it will be enough to have a good code quality. Many tools should be used together to incrementally improve code quality and measure it. And in the end, do not forget that tools can only do what you want them to do and their usefulness depends on what you do with them. As an example, it's good to have tests but if no one in the team ever look at whether they are succeeding or failing, they are pretty much useless.

Linters ℹ️

Linters are simple code analysis programs that are able to detect common programming errors and syntax issues. The way linters work is by defining a set of rules, that can usually be configured, and those rules will be checked by a command line or by the IDE every time a new code is added/updated/removed.

The good news is that since Flutter 2.10, the package flutter_lints is added by default to your pubspec.yaml when you create a new app with Flutter command line flutter create :

dev_dependencies:
  flutter_test:
    sdk: flutter
  flutter_lints: ^2.0.1

With that, you also get a file called analysis_options.yaml at the root of your project that will include the source package :

include: package:flutter_lints/flutter.yaml

linter:
  rules:

So what do we have here ? Pretty easy, we are just saying that we want to use pre-configured set of rules from flutter_lints package. Joy of open source, you can jump to the actual repository to check what rules are defined.

Now you have to keep in mind that this set of rules has been defined by the Flutter team as the minimal set of recommended rules. Though we can trust the team that developed the framework to give us good rules, you can always customize this and deactivate or activate individual rules.

Say I want to enforce the declaration of return types and trailing commas, I just have to update my analysis_options.yaml file :

include: package:flutter_lints/flutter.yaml

linter:
  rules:
    always_declare_return_types: true
    required_trailing_commas: true

And if you're curious what rules exists, the complete list is here ! In my opinion it's always good to take a look at the existing rules to understand how to better code in Dart and Flutter, and understand what to avoid.

In my case I usually customize a lot of rules to define code conventions that work for my team. I like to have linter rules by default in new apps but the default linter sets about 44% of rules which is quite lax, I rather have stronger code conventions so everyone in the team uses pretty much the same code, and is able to read others' code. I usually activate around 80% of rules.

If you want to learn more about Flutter linting including a comparison between various popular packages, you should definitely read this Mike Rydstorm article on Flutter Linting, and he even created a Google Sheet table to compare the linter packages rule by rule !

Finally, to take linters to the next level, have a look at Dart Code Metrics. This package takes linter rules a little bit further by checking additional rules, more complex anti-patterns or code metrics, and also by checking for unused files and l10n strings.

Oh and before I forget, if you are new to Dart language or feel like you need a refresher, you absolutely must have a look at Effective Dart guidelines.

Refactoring & IDE features 🧑‍💻

I usually spot a junior developer quickly on the first pair-programming or pair-review session with the way a developer uses its tools. I don't mean to offend anyone, I'm just saying modern IDEs have a lot of tools and features that allow quick and strong refactoring, so we should use and abuse them, that's why there are here for.

VSCode and AndroidStudio (probably used by 99.99% of Flutter developers nowadays) both include powerful tools to refactor. Whether you want to rename a field or a method, extract a code block to a method or inline a method into another, you probably can delegate this cumbersome task to your IDE:

And if you do this often, learn the shortcut and become more efficient in your everyday life !

But your favorite IDE has many other (not-so) hidden gems.

A good example is quickfix that will apply fixes to common mistakes / warnings :

And you know what ? The great thing is that your IDE is able to read your set of defined rules in your analyis_options file and will then show you the errors / warnings / hints directly in concerned files. How great is that ?

Another feature that saved my Flutter-developer life when I started is the auto-completion feature. It is just crazy how fast you can create widgets, wrap widgets into others and do a lot of cool things. If you are currently adding widgets and counting how many closing parenthesis you have, just stop and use auto-completion :

You got it, IDEs are powerful and love to work for you, so help yourself and use them. I'll leave you on the IDE part with the auto-format feature, particularly used in Flutter if you don't want to go crazy :

Tests ✅ ❌

Yes, testing is essential to keep code quality high. It is useful in the design phase because for a component to be testable means that you need to design it correctly (in isolation, should not do too much, etc.). It is also very useful because it is, if done correctly, a repeatable step that can be automated to check for non-regressions as you add features and fix bugs.

I won't go into too much details on testing because the official documentation already contains a comprehensive chapter on testing.

The documentation differentiate three types of tests :

Unit tests test a single function, method, or class.
Widget tests (in other UI frameworks referred to as component test) test a single widget.
Integration tests test a complete app or a large part of an app.

If you don't know the difference between those tests, please go read the doc, it is very important to know which does what.

However I feel that there is a fourth type of test that is often unknown to developers and yet very powerful in my opinion :

🏆 Golden tests 🏆

A golden test is just a specific kind of widget test except that we use Flutter test framework to actually check that our widget rendering is conform to what we except it to be. The widget rendering must match a pre-generated rendered image called a "golden file", hence the name of the test.

I won't go into details here because other articles already do, but basically what you need to remember is that golden tests are widget tests, which means they execute almost as fast as widgets tests, and much (much, much) faster than integration tests, and allow for quickly testing visual rendering of our widgets.

Here is an example where I generated a golden file for my tiles widget, on both light and dark themes (with a single golden test) :

In case anything goes wrong, failures images get generated and come in multiple variants, here in "maskedDiff" mode where images are superposed and only changes are color-highlighted (in pink below) :

Or an isolated diff where only the diff is shown :

I just wanted to show you a quick overview of golden tests, I personally love them and use them to quickly generate image variants for various parameters :

Light and Dark themes
Different orientations (e.g. portrait / landscape)
Different devices sizes and pixel aspect ratio (e.g. iPhone 5 to iPhone 13)

ℹ️ Just a warning though, testing a lot of variants in each test will quickly increase the number of images being generated and the total execution time. Example :

4 devices sizes x 2 themes x 2 orientations = 16 tests (for a single test)

You're the master, you decide what you need to do.

Final advice on golden tests : have a look at Alchemist package to make your life golden ☀️. An alternative for goldens is the more known golden_toolkit package.

And just to sum up, here's a quick overview of all the tests types in Flutter :

Code coverage

First of all a disclaimer : I don't want to start a war on whether code coverage should be used as a measure (or the only measure) of code quality. If you rely solely on code coverage to measure code quality, you have probably a long way to go... code coverage in itself does not mean much as a 100% coverage of a piece of code does not the mean that the tests are good and actually test something.

That said, there are cases when code coverage can be interesting as part of a global code quality measurement, for example you might want to make sure that your code has a decent code coverage and does not get lower and lower as time passes (and code is added).

Good news, Flutter allows you to just do that with the simple command line argument --coverage. However, to actually generate a coverage report from the raw result we need to use a combination of junit and lcov librairies. I usually use the following script in my projects:

#!/bin/bash

## Prepare coverage folder and JUnit report lib
mkdir -p coverage/
flutter pub global activate junitreport

## Run tests with coverage and JUnit report
flutter test --coverage coverage/ --machine | tojunit --output test_results.xml

## Generate coverage report
lcov --remove coverage/lcov.info \
  "lib/main.dart" \
  "lib/other_file_to_remove_from_coverage.dart"
-o coverage/lcov_cleaned.info

genhtml -o coverage/ coverage/lcov_cleaned.info

Once you execute this script, you should get a report in coverage/ folder that looks like this :

Here you go !

CI & CD (Continuous Integration & Continuous Deployment)

I have only one thing to say :

You must use a CI.
You must use a CI.
You must use a CI.

A good CI & CD platform(s) is essential to a successful project for many many reasons, here are a non-exhaustive list :

It is a gain of time because you don't have to run each command manually, both for building, testing and deploying your apps
It is automated, making builds and deploys repeatable.
It allows for continuous verification of many variables (successful compilation, succeeding tests, etc) and allows for giving constant feedback to the dev team : are there any warnings for potential regressions ?

Anyway, you must use a CI & CD platform.

This is not really a Flutter-specific topic but I had to mention it. The only specificity with Flutter builds are related to iOS because iOS apps can only be built on a MacOS runner... yeah 🙌

Other than that, you can use pretty much any platform or combination of platforms (and I have used a lot on my various clients projects) as long as you can implement a decent build and a deploy (those steps can be separated though because releases are usually manual) : GitLab CI, Github Actions, Bamboo, Bitrise, Azure Devops, Codemagic...

By the way, I wrote an article about 2 years ago (might not be up-to-date now...) on how to deploy apps to the stores using Codemagic in combination with Fastlane.

Ideally you want to try and give as much as feedback to your fellow developers on each build : lint checks result, failed tests including golden failures, tests results, code coverage, etc. A very promising Dart package called Danger.dart does just that, and here is an article that explains the process a little bit more.

Code quality measurement

If you are using the tools presented in this article you should by now have pretty good measures of your app code quality. Linters, tests, code coverage, etc.

However, some people might still want some kind of dashboard to gather all code quality related measures into one place.

Ring any bell ? Don't be shy, I know that you are thinking of Sonarqube !

And luckily for us, there is now a Flutter / Dart package available to plug into Sonarqube for our loved apps analysis !

This library called sonar-flutter generates the well-known Sonarqube dashboard after taking into account all Dart & Flutter related measures :

Conclusion

In this article we saw that code quality is very important to make a Flutter app more safe, reliable and maintainable. We also had a look at a few tools that can make our life easier when trying to improve code quality in our apps.

Please note that there are many more techniques that I did not mention earlier such as pair-programming, code reviews, documentation or TDD. Because they are general techniques, I chose to focus on specific Dart or Flutter tools here to try and give you a good overview of what you can do. Obviously you can use any known technique in addition to the specific tools if you feel the need to :

Your team is the only master of your app quality !

CLI applications made easy with Dart & dcli

Olivier Revial — Tue, 02 Nov 2021 09:15:12 +0000

I've been developing Flutter applications for a while now and I love Flutter and Dart, in particular since Dart 2.12 / Flutter 2.0 when null safety was officially released !

Sometimes however, you need a good ol' bash script to run some chore stuff or for your CI... and I have to admit, I suck at writing clean bash scripts.

💭 If only I could write my scripts using the same language I use everyday, the very language I'm the most familiar with at the moment ? 🤔

Today I will show you how we can write command-line applications using only Dart with a few librairies.

What we will do

The goal of our application will be to simply release a new version of our app by updating the version number in our project's pubspec.yaml. This is something that could later be executed by a CI to automate this, we will do it by hand here for the sake of the example. Also, you can find the code in this repository.

The main steps we will follow are :

Write our CLI
Run our CLI within Dart environment
Create a standalone CLI that can be run as a simple script

Let's start by generating a simple Dart application :

dart create dart-release-cli
dart-release-cli

Great, we now have a sample project that we can work from :

It's now time to add actual CLI-related code !

To do that, we could use Dart's out-of-the-box support for command line code (e.g. reading input or printing output, I/O related operations and so on). The process to do a bunch of CLI-related operations is very well documented by Dart team in this great Dart tutorial "[Write command-line apps]"(https://dart.dev/tutorials/server/cmdline). However I find that default libraries are not so simple to use and I don't like to have a lot of boilerplate code to do simple CLI things such as reading an input or writing to a file.

Fortunately I'm not the only one and so people have created a dedicated wrapper around Dart native CLI code : a package named dcli. And, bonus, the documentation is pretty cool and rather comprehensive !

Enough talking, let's add this package as a dev dependency. We will also add yaml package, we will need it to easily read our pubspec.

#pubspec.yaml:

dev_dependencies:
  dcli: ^1.9.6
  yaml: ^3.1.0

Of course, don't forget to run pub get to update dependencies.

We can now write our CLI to bin/release.dart file:

#! /usr/bin/env dcli

import 'dart:io';

import 'package:dcli/dcli.dart';
import 'package:yaml/yaml.dart';

void main() {
  // Display output to console
  print('''# ------------------------------
# 🚀 Release new version
# ------------------------------''');

  // Ask for user input
  final newVersion = ask(
    'Enter new version number:',
    validator: Ask.regExp(
      r'^\d+\.\d+\.\d+$',
      error: 'You must pass a valid semver version in the form X.Y.Z',
    ),
  );

  try {
    // Read a file
    final pubspec = File('./pubspec.yaml').readAsStringSync();

    // Parse yaml content to retrieve current version number
    final currentVersion = loadYaml(pubspec)['version'];

    print('# Updating from version $currentVersion to version $newVersion...');

    // Write to a file
    'pubspec.yaml'.write(pubspec.replaceFirst(
      'version: $currentVersion',
      'version: $newVersion',
    ));

    print('# ✅ Pubspec updated to version $newVersion');
  } catch (e) {
    // Print or treat error as needed
    printerr('# ❌ Failed to update pubspec to version $newVersion\nError: $e');
  }
}

And that's it !

I won't go through all the code because it's pretty basic stuff, but a few things can be noted:

The very first line is a shebang that allows us to run our dart file as if it were a simple shell script. To do that you first need to globally install DCli tools. Note that this is dcli recommended solution but it works just as well with a simple dart shebang (#! /usr/bin/env dart) 😉
print (Dart's one) and printerr (DCli's one) will both output to the console, respectively to stdout and stderr of course. 📝
DCli offers cool APIs such as the ask operator, a one-line solution to ask for user input and optionally validate user input at the same time. Another example is the write command we use, a String extension that wraps Dart file handling.

Alright, time to run our CLI ! We have quite a few options here.

The first is to run our file using Dart VM :

dart bin/release.dart

The second solution, as explained above, is to use globally installed DCli tools or dart shebang to run our Dart file as if it were a simple script, so we just have have to execute it :

# Make sure your file is executable first
chmod +x bin/release.dart

# Run as a simple script
./bin/release.dart

Hopefully you had expected output by now ! This is great and allows for quick testing, however you may have noticed that it takes quite some time before the first line is actually displayed. Well, it's normal because the code needs to be compiled before it can be run. Fortunately, Dart offers the ability to generate a platform executable easily using dart compile command:

dart compile exe bin/release.dart -o release

🕙 Info: Compiling with sound null safety
🕙 Generated: dart-release-cli/release

# Time to run !
./release

We now have an executable that run instantly. In a real world it would probably be the CI job to compile and produce the executable. Also, note that the only drawback is that the executable is much larger than a simple Dart file (about 5MB for our example CLI).

Alright, that's about it ! Here's a demo of the running CLI :

I hope you enjoyed this article where I tried to show you a simple way to generate powerful CLI applications using Dart and a few libraries. And if you want to test the CLI, head over to the Github repository 👋

References:

Flutter: flavors launcher icons made simple

Olivier Revial — Mon, 11 Jan 2021 08:43:14 +0000

Anyone that has already dealt with flavors in Flutter knows that they are not such an easy thing to implement.

You usually end up reading a lot of articles, going back and forth between your favorite IDE and Xcode, looking at configuration in your build.gradle and then crying over Xcode build configs...

But one thing is pretty easy to do when using flavors : generating appropriate launcher icons for both platforms, for all resolutions and obviously each flavor.

Supposing we have development, integration and production flavor, what we want to do in the end is have all these icons for Android:

And the same goes for iOS:

I don't know about you, but I really don't want to generate all these icons by hand... so we will instead use the great Flutter plugin flutter_launcher_icons !

As it is just a generator that we will use once, we don't need it at runtime so we can add it to our pubspec.yaml dev_dependencies:

dev_dependencies:
  ...
  flutter_launcher_icons: ^0.8.1

Now we will need to configure the assets location for each flavor:

flutter_launcher_icons-development.yaml:

flutter_icons:
  android: true
  ios: true
  image_path: "assets/launcher_icon/launcher-icon-dev.png"

flutter_launcher_icons-integration.yaml:

flutter_icons:
  android: true
  ios: true
  image_path: "assets/launcher_icon/launcher-icon-int.png"

flutter_launcher_icons-production.yaml:

flutter_icons:
  android: true
  ios: true
  image_path: "assets/launcher_icon/launcher-icon-prod.png"

Now that we have our plugin configured for each flavor, it's time to add our actual icons in our assets:

As you probably understood already, the next step will generate all the icons shapes and resolutions as needed by each platform. This process will take our source icon for each flavor and will downscale it to create all resolutions. This is important because it means you should provide the highest resolution of 1024x1024 for source icons.

Note: at this point our assets are not related to anything, i.e. not used by either Android or iOS, so we need to translate them in their respective format:

Let's do just that by invoking a run of our plugin:

flutter pub get
flutter pub run flutter_launcher_icons:main -f flutter_launcher_icons*

After a few seconds, you should get a result similar to this one:

════════════════════════════════════════════
   FLUTTER LAUNCHER ICONS (v0.8.0)
════════════════════════════════════════════

Flavor: development
• Creating default icons Android
• Overwriting the default Android launcher icon with a new icon
• Building iOS launcher icon for development

Flavor: integration
• Creating default icons Android
• Overwriting the default Android launcher icon with a new icon
• Building iOS launcher icon for integration

Flavor: production
• Creating default icons Android
• Overwriting the default Android launcher icon with a new icon
• Building iOS launcher icon for production

✓ Successfully generated launcher icons for flavors

And voilà, you now have brand new icons for all platforms, flavors and resolutions conbinations.

Pretty neat, right ?

As a side note, the plugin is also able to generate adaptative Android icons by providing adaptive_icon_background and adaptive_icon_foreground for Android, which I highly recommend !

References

Flutter, a wonderful journey from 0 to app stores - CI featuring Codemagic and Fastlane

Olivier Revial — Tue, 03 Nov 2020 17:54:33 +0000

So you just started using Flutter and already love its wonderful and numerous widgets, but as your project is being serious you're wondering how to sign your Android & iOS apps and how to deploy them to both stores ? Oh, and as a ~~lazy~~ good developer, you're obvisouly wondering how to automate your builds and deployments to reduce manual actions (and errors) and free up some of your precious time to code your beautiful app ?

Well, me too. At first I thought it would be very simple and I'd just have to configure the awesome Codemagic, use its 500 free minutes per month... and relax 🛁

After all, it's supposed to be magic 🧙 !

In this article I will explain step-by-step how to easily automate app building and beta deployment to Android & iOS stores.

❗️📝

I am not claiming to have the best approach of all, but I needed a reminder of all the steps needed and I thought it could benefit others... tell me if you have better ways to do it or if I can improve my solution !
In this article we will only explore how to deploy to testing tracks (beta track for iOS and internal tests for Android), but the process should be pretty much the same for other tracks.

The various chapters we will detail are the following:

👥 Prerequisites: all the accounts and things you need before setting up the CI
🧙‍♂️ Codemagic setup: setup of the CI workflow(s)
🤖 Android setup
🍏📱 iOS setup

📁 You can find my demo repository on Github.

Prerequisites

Hopefully you are reading this article before you actually need the CI, because asking all people in the company to create the various accounts may take some time.

👥 Accounts

In order to connect to different platforms through Codemagic you will need a few accounts and keys to be setup before you start implementing actual CI pipeline.

In particular you need the following accounts and keys:

Apple enrolled account. You basically need a paid Apple Developer account, either enrolling as an individual or as an organization.
- Example: support@your-company.com
Apple Developer account: Used to handle iOS profiles & certificates and to deploy iOS builds to TestFlight.
- This account must have App Manager role (not juste Developer role).
- This account will be the same as the enrolled account if you're an individual, otherwise you should probably keep things separate and create this account as a "technical account".
- Example: apple-technical@your-company.com
Google Play Developer enrolled account. Again, you need a paid Google Play Developer account if you want to publish your apps.
- Example: support@your-company.com
Google Play Console service account key. A JSON key used to deploy Android builds to Google Play beta store.
- This account must have Release Manager role.
- This key will later be used in Google Play service account key chapter.
Google Play Console account: your personal account
- This account must have at least Release Manager role.
- Strictly speaking this account won't be used by Codemagic (it will only use service account JSON key mentionned above) but you will need it to manually upload your first build or to to fill app information in Google Play Console, promote the build to production and so on.
- Example: Google developer account like your.name@your-company.com
Gitlab/Github/Bitbucket account. Used by Codemagic to checkout Flutter app's repository.
- Make sure this account has access to your repository by checking repository Members (e.g. in Gitlab members)
- You need to generate SSH key pair and upload the public key to your profile (e.g. Gitlab SSH keys). You can name the public key anything, I like to put something like Codemagic app XXX
- Keep the private key, you will need to encrypt it and pass it to Codemagic later on !
- Example: codemagic@your-company.com
Codemagic account. Used to configure Codemagic pipeline itself, Codemagic account is required to run any build as each account has a limited number of build minutes available (500 by default).
- For simplicity reasons I use the same Gitlab account mentioned above to connect to Codemagic (e.g. codemagic@your-company.com Gitlab account).

Configuration

Once you have your personal accounts for both Apple Connect and Google Play Console, you should take some time to create the application page for both stores as you won't be able to deploy Android or iOS test apps before you filled a minimum information:

Google Play Console : create a new application in the console.
- Make sure you enter a correct package name, it's definitive !
Google Play Console : fill-in basic information. Start at app's dashboard and follow the guide to fill all required steps (there are a lot of steps but it's pretty simple)
- You will need to upload at least an app icon (make sure you respect appropriate format), a Play Store commercial banner and 2 screenshots of the app before you can upload your first APK (yes, even for internal tests).
Apple Developer identifiers : create a new identifier of type App IDs and follow the steps.
- Note generated bundleId and App team id, you will need them later.
- Make sure you enter a correct bundle id, it's definitive !
Appstore Connect : create a new application in the console. You will need to link your new app to a bundleId... the one you created in previous step 🤘
Appstore Connect : fill-in basic information.

🧙‍♂️ Codemagic setup

Codemagic use multiple files to do its "magic":

codemagic.yaml main file: defines various worflow for iOS, Android and both apps deployments.
Codemagic post-clone setup file: various installation steps for both Android and iOS apps
Gemfile for Android and for iOS. Used to automate gem installation on Codemagic using bundle install command.
Fastlane for Android and Fastlane for iOS: we use Fastlane for all build and deploy steps to help us handle certificate management, app signing and similar boring stuff.

Let's start right away with Codemagic main file, codemagic.yaml:

workflows:
  internal-deployment:
    name: Internal deployment
    max_build_duration: 90
    environment:
      vars:
        BUILD_NAME: 1.2.3
        GOOGLE_PLAY_STORE_JSON_BASE64: Encrypted(var)
        ANDROID_KEYSTORE: Encrypted(var)
        ANDROID_KEYSTORE_PASSWORD: Encrypted(var)
        ANDROID_KEY_ALIAS: Encrypted(var)
        ANDROID_KEY_PASSWORD: Encrypted(var)
        IOS_APP_ID: com.company.somename
        APPLE_DEVELOPER_TEAM_ID: 123ABCD456
        FASTLANE_EMAIL: apple-technical@your-company.com
        FASTLANE_PASSWORD: Encrypted(var)
        MATCH_PASSWORD: Encrypted(var)
        SSH_KEY_FOR_FASTLANE_MATCH_BASE64: Encrypted(var)
        FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD: Encrypted(var)
      flutter: stable
      xcode: latest
      cocoapods: default
    cache:
      cache_paths:
        - $HOME/Library/Caches/CocoaPods
        - $HOME/.gradle/caches
        - $FLUTTER_ROOT/.pub-cache
    triggering:
      events:
        - tag
    scripts:
      - name: Post clone setup for Android & iOS
        script: |
          #!/usr/bin/env sh
            /bin/sh $FCI_BUILD_DIR/codemagic/post-clone.sh all
      - name: Build & Deploy Android app
        script: |
          cd $FCI_BUILD_DIR/android

          bundle exec fastlane supply init --track internal --package_name com.company.somename
      - name: Build & Deploy iOS app
        script: |
          cd $FCI_BUILD_DIR/ios

          bundle exec fastlane ios beta
    artifacts:
      - build/**/outputs/**/*.apk
      - build/**/outputs/**/*.aab
      - build/ios/ipa/*.ipa
      - /tmp/xcodebuild_logs/*.log
      - flutter_drive.log
    publishing:
      slack:
        channel: '#flutter-codemagic-demo'
        notify_on_build_start: false

If you're familiar with Gitlab CI or YAML in general, you shouldn't be too shocked reading this file. Let's describe the behaviour of your one and only workflow, internal-deployment:

🔫 max_build_duration: a timeout duration, because sometimes Codemagic/Android/iOS builds get stuck.
✓ environment: all environment variables that will be used by our scripts. More on this in Android and iOS setup. Also defines the versions of Flutter, Xcode and Cocoapods that Codemagic should use. It is good practice to use latest stable versions.
🏎 cache: files that should be cached and reused across multiple builds to save some precious build minutes.
🔘 triggering: how the build should be triggered. I decided that mine should be triggered every time I release a new tag in my Gitlab project. Note that I had to add a Gitlab webhook in order to automatically trigger Codemagic on tag creation.
▶️ scripts: the core of our workflow: what we should do when this Codemagic workflow is triggered. As you can see here we only delegate the work to Fastlane for both Android and iOS, as we will see in a minute.
📦 artifacts: the artifacts we want to publish on Codemagic. Not solely needed as artifacts will be published to iOS and Android consoles but I like to keep them here in case the build fails and I need to debug.
📩 publishing: I like to be notified when my build fails or succeeds so I configured a Slack notification here.

The first script called by our worflow is a "post-clone" script, let's see this codemagic/post-clone.sh file:

#!/usr/bin/env sh

set -e # exit on first failed commandset

installGems() {
  echo "Installing gems..."

  bundle install
  bundle update fastlane
  bundle update signet
}

androidSteps() {
  echo "========================================"
  echo "|       Android post-clone steps       |"
  echo "========================================"

  # set up key.properties
  echo $ANDROID_KEYSTORE | base64 --decode > /tmp/keystore.keystore
  cat >> "$FCI_BUILD_DIR/android/key.properties" <<EOF
storePassword=$ANDROID_KEYSTORE_PASSWORD
keyAlias=$ANDROID_KEY_ALIAS
keyPassword=$ANDROID_KEY_PASSWORD
storeFile=/tmp/keystore.keystore
EOF

  # set up local properties
  echo "flutter.sdk=$HOME/programs/flutter" > "$FCI_BUILD_DIR/android/local.properties"

  echo "--- Generate Google Service key for Android"
  GOOGLE_PLAY_STORE_JSON_KEY_PATH="$FCI_BUILD_DIR/android/app/google-play-store.json"
  echo $GOOGLE_PLAY_STORE_JSON_BASE64 | base64 --decode > $GOOGLE_PLAY_STORE_JSON_KEY_PATH

  cd $FCI_BUILD_DIR/android
  installGems
}

iosSteps() {
  echo "========================================"
  echo "|         iOS post-clone steps         |"
  echo "========================================"

  echo "--- Generate SSH key for Gitlab access from Fastlane Match"
  echo $SSH_KEY_FOR_FASTLANE_MATCH_BASE64 | base64 --decode > /tmp/bkey

  # adding custom ssh key to access private repository
  chmod 600 /tmp/bkey
  cp /tmp/bkey ~/.ssh/bkey
  ssh-add ~/.ssh/bkey

  cd $FCI_BUILD_DIR/ios
  installGems
}

androidSteps
iosSteps

This script is pretty boring but what it does is install a bunch of gems and setup some files that iOS and Android will need for building, signing and publishing the apps.

As you can see there is a bundle install that will use Android Gemfile and iOS Gemfile to automatically install required gems:

Android Gemfile

source "https://rubygems.org"

gem "fastlane"

iOS Gemfile

source "https://rubygems.org"

gem "fastlane"
gem "cocoapods"

Let's now see Android build script in android/fastlane/Fastfile file:

default_platform(:android)

platform :android do
  desc "Deploy a new internal build to Google Play"
  lane :internal do
    Dir.chdir "../.." do
      sh("flutter", "packages", "get")
      sh("flutter", "clean")
      sh("flutter", "build", "appbundle", "--build-number=#{ENV["PROJECT_BUILD_NUMBER"]}", "--build-name=#{ENV["BUILD_NAME"]}")
    end
    upload_to_play_store(
        track: 'internal',
        aab: '../build/app/outputs/bundle/release/app-release.aab'
    )
  end
end

This script basically uses Fastlane to build and deploy the app to Google Play Console. First it defines a "lane", which is just Fastlane way of defining a "worflow". Then we use shell to initialize and build our flutter app, by passing it build name and number from environment variables.

Once the signed application is ready, we use Fastlane once again to publish this app to the internal Android track through upload_to_play_store task.

Pretty simple right ? 🎉

Let's now see the equivalent for iOS:

default_platform(:ios)

platform :ios do
  before_all do
      # Create a local keychain that will later store iOS profiles and certificates
      if is_ci?
          puts "This is CI run. Setting custom keychain."
          create_keychain(
              name: 'Temp.codemagic_keychain',
              password: 'Temp.codemagic_keychain_password',
              default_keychain: true,
              unlock: true,
              timeout: 3600,
          )
      end
  end

  desc "Push a new beta build to TestFlight"
  lane :beta do
    # Synchronize profiles & certificates from Git repo using Match
    match(
        type: "appstore",
        readonly: is_ci,
        keychain_name: 'Temp.codemagic_keychain',
        keychain_password: 'Temp.codemagic_keychain_password'
    )
    # Disable automatic code signing as we will use custom signing method later on
    update_code_signing_settings(
      use_automatic_signing: false
    )
    # Update Xcode provisioning profile with the one we got from Git repo using Match
    update_project_provisioning(
      # https://github.com/fastlane/fastlane/issues/15926
      profile: ENV["sigh_#{ENV["IOS_APP_ID"]}_appstore_profile-path"],
      build_configuration: "Release",
      code_signing_identity: "iPhone Distribution",
      xcodeproj: "Runner.xcodeproj",
    )
    # Replace version number with Codemagic build number
    set_info_plist_value(
      path: "Runner/Info.plist",
      key: "CFBundleVersion",
      value: ENV["PROJECT_BUILD_NUMBER"]
    )
    # Replace version name with our semver version
    set_info_plist_value(
      path: "Runner/Info.plist",
      key: "CFBundleShortVersionString",
      value: ENV["BUILD_NAME"]
    )
    # Run a first Flutter build with code signing disabled
    Dir.chdir "../.." do
      sh("flutter", "packages", "get")
      sh("flutter", "clean")
      sh("flutter", "build", "ios", "--release", "--no-codesign")
    end
    # Run a second Flutter build with custom code signing
    build_app(
        workspace: "Runner.xcworkspace",
        scheme: "Runner",
        configuration: "Release",
        xcargs: "-allowProvisioningUpdates",
        export_options: {
            signingStyle: "manual",
            method: "app-store",
            provisioningProfiles: {
                "#{ENV["IOS_APP_ID"]}": "match AppStore #{ENV["IOS_APP_ID"]}"
            }
        },
    )
    # Upload our build to TestFlight (Beta track)
    upload_to_testflight(
        skip_waiting_for_build_processing: true,
        apple_id: "123456789"
    )
  end
end

Alright, let's decrypt this file step-by-step:

before_all : Before all steps run we create a local keychain that will later store iOS profile and certificates for app signting. This keychain will only live the time of the Codemagic build so we don't really care what the password is.
match : We start the beta lane with profile and certificates synchronization. Codemagic configuration for iOS deployment can be rather complicated, in order to avoid this complexity by handling certificates and provisioning profiles ourselves, we use Fastlane and match for code signing. Here we basically tell match to retrieve all stored profiles and certificates from the repo we stored them previously, and store them into the keychain we created in the previous step. We tell match to get things for appstore type, i.e. release profile and certificates. We will see later on in iOS setup how to initialize match to create and store profile and certificates.
update_code_signing_settings : We disable automatic code signing because we want to handle it manually using match-retrieved profile/certificates from previous step
update_project_provisioning : This step is essential as it tells Xcode to use our generated profile and certificates instead of its default (automatic) signing files. We tell it to use match retrieved provisioning profile instead.
set_info_plist_value : We replace Xcode version values with the correct version:
- PROJECT_BUILD_NUMBER is the Codemagic global build number (global to your application, no matter which workflow you run). Useful to deploy the same version multiple times and from different workflows.
- BUILD_NAME is the actual version of our project, in semver terminology. Note that it is up to you to handle the bump of this variable on each of your app's releases. In my case I use Gitlab pipeine with a custom script to automatically bump BUILD_NAME environment variable inside codemagic.yaml and commit this as a release commit.
flutter build ios --release --no-codesign : Flutter setup with a full packages/clean and build process, but with codesign disabled. As stated in Flutter official CI/CD documentation, we need to initialize our project wiht a first Flutter build before we can actually run the signing build step with Fastlane (see next step)
build_app : We use Fastlane once again to build a signed version of our iOS application. Of course we tell it to use the provisioning profile we retrieved in previous step with match.
upload_to_testflight : We can finally use Fastlane upload_to_testflight task to automatically push our new .ipa application to Apple TestFlight (for beta testing). The two parameters are set to avoid waiting for Apple build processing to end Codemagic build. It means that you are not 100% sure that your build has correctly been received (and is valid) by Apple but in the other hand it has 2 great advantages:
- It consumes less Codemagic free minutes waiting for an external response that is not really part of the build itself
- It is the only way of only using an Application Specific Password to push your app. Otherwise you would need 2FA with a short-lived session token, and you don't want that on your CI. Really, believe me 🙄

🤖 Android setup

Codemagic configuration for Android deployment is pretty straightforward and is rather well described in code signing documentations of both Flutter and Codemagic

You will need to create a few things:

Create a service account key to interact with Google Play Console
Create a keystore for Android app signing
Configure Android project to use code signing for release mode, with appropriate keystore
Locally generate a signed Android application and upload this first release to Google Play Console

1. Google Play service account key

Let's start by creating the service account key.
These steps are very well described in Codemagic documentation (in Google Play section).

This variable is going to be translated into Android google-play-store.json by our codemagic/post-clone.sh script:

GOOGLE_PLAY_STORE_JSON_KEY_PATH#"$FCI_BUILD_DIR/android/app/google-play-store.json"
echo $GOOGLE_PLAY_STORE_JSON_BASE64 | base64 --decode > $GOOGLE_PLAY_STORE_JSON_KEY_PATH

...which will used by Fastlane further down the road when deploying our application to Google Play Console.

⚠️ Only the app owner of Google Play Console project is able to create a service account key, make sure you ask this person to create (and send you) the JSON key !

We now need to create a file named android/fastlane/Appfile to give Fastslane our Google Play service key:

json_key_file("app/google-play-store.json")
package_name("com.company.awesome_app")

Of course, you should put your Android package name here (lowercase letters and underscores only !)

ℹ️ the actual JSON key will be populated later on by post-clone.sh script

2. Keystore

Generate a keystore for current project:

keytool -genkey -v -keystore your_app.jks -alias your_app_key -keyalg RSA -keysize 2048 -validity 10000 -storetype JKS

You will prompted for a password.

⚠️ Don't forget to securely store the password and share it with your team if they need to sign the app using the keystore (e.g. for CI builds). 🔐

ℹ️ -storetype JKS option is required for Java 9+ otherwise you will not be prompted for alias key/password and the keystore will not be compatible for code signing anyway...

Now that you have an Android keystore, you need to encode your keystore to base64.

base64 -i your_app.jks

3. Project configuration

Now that we have created util stuff needed to sign and publish our app to Google Play, we only need to tweak our configuration to reference info.

First open codemagic.yaml and define the following variables:

GOOGLE_PLAY_STORE_JSON_BASE64 : [Codemagic-encrypted] - base64 encoded version of JSON key downloaded from Google Play Console in step 1 above.
ANDROID_KEYSTORE : [Codemagic-encrypted] - base64 encoded version of the keystore generated in step 2 above.
ANDROID_KEYSTORE_PASSWORD : [Codemagic-encrypted] - password of the generated store
ANDROID_KEY_ALIAS : [Codemagic-encrypted] - alias of the generated key
ANDROID_KEY_PASSWORD : [Codemagic-encrypted] - password of the generated key alias

⚠️ Variables that are marked as [Codemagic-encrypted] must be encrypted using Codemagic user interface.
Go to any project > configuration wheel > Encrypt environment variable and paste the variable you want to encrypt, and copy it back to codemagic.yaml.
Keep the Encrypted(VAR) around your variable !

These variables are going to be translated into Android key.properties by our codemagic/post-clone.sh script:

echo $ANDROID_KEYSTORE | base64 --decode > /tmp/keystore.keystore
cat >> "$FCI_BUILD_DIR/android/key.properties" <<EOF
storePassword#$ANDROID_KEYSTORE_PASSWORD
keyAlias=$ANDROID_KEY_ALIAS
keyPassword=$ANDROID_KEY_PASSWORD
storeFile=/tmp/keystore.keystore
EOF

Finally, we can configure our android/app/build.gradle to sign our app using provided keystore. First add these lines above android tag:

def keystoreProperties = new Properties()
def keystorePropertiesFile = rootProject.file('key.properties')
if (keystorePropertiesFile.exists()) {
    keystoreProperties.load(new FileInputStream(keystorePropertiesFile))
}

These lines tell Android to read keystore properties from our specific file. We can now add signing configs to the build (replace existing lines with these new ones):



signingConfigs {
    release {
        keyAlias keystoreProperties['keyAlias']
        keyPassword keystoreProperties['keyPassword']
        storeFile keystoreProperties['storeFile'] ? file(keystoreProperties['storeFile']) : null
        storePassword keystoreProperties['storePassword']
    }
}

buildTypes {
    release {
        signingConfig signingConfigs.release
    }
}

You're ready to sign your Android app !

4. Signed release & first upload

In order to push automated releases of your Android app, you need to create a first release manually on the Google Play Console interface and upload a first signed bundle/apk.

You can follow these steps:

Add a file named /android/key.properties with the following content:

storePassword=$ANDROID_KEYSTORE_PASSWORD
keyAlias=$ANDROID_KEY_ALIAS
keyPassword=$ANDROID_KEY_PASSWORD
storeFile=/tmp/keystore.keystore

Obvisouly you need to replace the values of each property with the actual value you generated (same values as you set for corresponding environment variable in Project Configuration section).

Make sure /android/key.properties is excluded in your /android/.gitignore. Flutter does this by default when creating the app, you should see the following line:

key.properties

If you don't have this line, obviously add it, it is crucial that you do not push secrets to your distant Git repository !

Build and package the application with signing enabled (release mode):

flutter build appbundle --build-number 1 --build-name 0.0.1 --release

Once the signed application is ready, it's time to upload it to Google Play Console: create a new release under "Internal tests" and drop your appbundle there, you should be able to continue with your brand new release.

Once your first release is accessible for internal tests, you can now upload automated releases through CI ! Yeah 🎉

⚠️ Make sure you filled all necessary information on Google Play Console for internal tests (Dashboard will tell you what steps you need to complete first).

🍏📱 iOS setup

Codemagic configuration for iOS deployment can be rather complicated.
To avoid handling certificates and provisioning profiles ourselves, we'll be using Fastlane and match for code signing, as explained in Prerequisites.

First make sure you use latest Fastlane version:

sudo gem install fastlane

Also create an empty Git repository called certificates somewhere in your Gitlab/Github/Bitbucket. We can no run a few commands to setup codesigning for iOS so make sure you are in ios subdirectory:

cd ios/

Time to init match to point to our certificate repository:

match init
> Select git
> URL of the Git repo:
git@gitlab.com:your-company/your-app-group-folder/certificates.git

This should generate a Matchfile in .fastlane/Matchfile with the following content:

git_url("git@gitlab.com:your-company/your-app-group-folder/certificates.git")

storage_mode("git")

type("development") # The default type, can be: appstore, adhoc, enterprise or development

# app_identifier(["tools.fastlane.app", "tools.fastlane.app2"])
# username("user@fastlane.tools") # Your Apple Developer Portal username

# For all available options run `fastlane match --help`
# Remove the # in the beginning of the line to enable the other options

# The docs are available on https://docs.fastlane.tools/actions/match

⚠️ Use a SSH url for the repository (rather than the HTTPS url) as we will use an SSH key later to enable Codemagic fetching our repository.

Now we have to create ios/fatslane/Appfile to fill out a few needed information for Fastlane (and match too):

app_identifier("com.company.awesomeApp") # The bundle identifier of your app
apple_id("apple-technical@your-company.com") # Your Apple email address for the technical account

team_id("123ABC1DE") # Developer Portal Team ID
itc_team_id("123456789") # App Store Connect Team ID

ℹ️ To easily output these variables you can run fastlane produce and you should get all these variables:

app_identifier is your bundle id as created in Prerequisites - Config and can be found in your [Apple identifier](https://developer.apple.com/account/resources/identifiers/bundleId
apple_id is the technical account email address (e.g. apple-technical@your-company.com)
team_id is your Developer portal team ID and can be found in your Apple identifier
itc_team_id is an integer team id derived from above team_id

Once all variables are set, we can use match to create new certificates and profiles.
Please make sure that the Apple account you set in apple_id above has "App manager" permissions and not just "Developer" or you won't have permissions to create certificates and profiles.

Let's run the command to generate everything:

fastlane match appstore

Enter password for apple-technical@your-company.com.

If the command executed successfully you should now have new, generated certificates and [profiles]https://developer.apple.com/account/resources/profiles/list).
Also these certificates should now be stored encrypted into your Git certificates repository.

⚠️ Don't forget to securely store ios keychain and match passwords and share them with your team if they need to regenerate certificates ! 🔐

Now that you have generated profile and certificate, it's time to complete Codemagic workflow by setting all environment variables such as Fastlane environment variables:

BUILD_NAME : The version that you wish to deploy, in semantic versioning notation (e.g. 1.2.3, or more generally X.Y.Z)
IOS_APP_ID : the app identifier as set in Certificates, Identifiers & Profiles.
- Example: com.company.awesomeApp
APPLE_DEVELOPER_TEAM_ID : Apple Developer Team Id (e.g. 123ABC4DE)
FASTLANE_EMAIL : Apple Developer account email (e.g. apple-technical@your-company.com)
FASTLANE_PASSWORD : Apple Developer account password
MATCH_PASSWORD : [Codemagic-encrypted] - Password used when generating fastlane match appstore above using match.
SSH_KEY_FOR_FASTLANE_MATCH_BASE64 : [Codemagic-encrypted] - Base64 encoded private SSH key of Codemagic user for Gitlab. You can do the following to generate SSH keys :
- Generate a new SSH key pair : ssh-keygen -t ed25519 -C "SSH key description"
- Copy public key and add it under Gitlab's SSH keys for Codemagic user.
- Copy private key and encrypt it using Codemagic (see note below)
- Delete SSH key from your local computer
- If you ever need to see the key... you can't, just revoke the old one and generate a new pair !
FASTLANE_APPLE_APPLICATION_SPECIFIC_PASSWORD : [Codemagic-encrypted] - Apple Application specific password as described in Fastlane documentation. You can follow these steps:
- Go to Apple manage account page with your Apple account corresponding to FASTLANE_EMAIL.
- Generate a new application specific password
- Encrypt the password using Codemagic (see note below)

ℹ️ You can run fastlane produce to output some of these information (app_identifier, team_id, itc_team_id).

⚠️ Variables that are marked as [Codemagic-encrypted] must be encrypted using Codemagic user interface.
Go to any project > ⚙️ > Encrypt environment variable and paste the variable you want to encrypt, and copy it back to codemagic.yaml.
Keep the Encrypted(VAR) around your variable !

iOS Project configuration

Just a few more things we need to setup:

In ios/Runner.xcodeproj/project.pbxproj, set all DEVELOPMENT_TEAM variables to your actual Apple Development team (must be the same as APPLE_DEVELOPER_TEAM_ID env var above). If development team variables do not appear in ios/Runner.xcodeproj/project.pbxproj you can setup manually by doing following steps:
- Open Xcode
- Select Runner in left pannel
- In central panel select Targets > Runner
- Open "Signing & Capabilities" tab
- Under "All" sub-tab, disable automatic code siging and select your team from the appropriate field.

If no team is found you can also import a provisioning profile after downloading it from Apple profiles

In ios/Runner/Runner.entitlements (create the file if you don't have it), setup required Apple entitlements for your application.
- If you don't use any entitlements, you should still declare it by leaving the file with an empty list:
```
  <?xml version="1.0" encoding="UTF-8"?>
  <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
  <plist version="1.0">
      <dict/>
  </plist>
```

Phew, you should be all set for iOS automated deployment 🎉

Conclusion

As you can see setting up CI/CD automation for Flutter projects to go from the ground up to deployed beta versions of both Android and iOS applications requires a bit of patience, but in the end once you understand the various components interacting with each other, it's almost magic ! 🧙‍♀️

My advice : take the time to set up CI/CD when your Flutter project starts, you will be grateful to have it all automated when comes the mandatory rush of your project 🙂

Flutter UTF8 TextField length limiter and char counter

Olivier Revial — Fri, 23 Oct 2020 13:34:23 +0000

Flutter `TextField` is awesome, but...

I love Flutter and when it comes to TextFields, it's pretty easy to enforce a maximum length and have a counter display that indicates the current number of characters and the maximum length. All you have to do is:

TextField(maxLength: 20)

And you get a beautiful char-limited text field:

Pretty easy, right ?

Not so fast !

What happens if you are using special characters such as emojis, accent chars or graphenes ? Let's see:

Hmm 🤔, your text is 15 characters long, what's the problem ?

Well... sometimes you don't want to limit text length to visible characters but you want to limit length to the actual number of bytes of your text. For example, if you are sending text to a Bluetooth device or a fixed-length field in a database, you need to make sure that you won't send too many UTF-8 encoded characters. And here is the problem my friends: while a letter is coded on one byte, special characters and emojis can be coded on 2 to 8 bytes !

In this case, above example "I ❤️ Flutter 😻😍🤩" would actually take 29 bytes when UTF-8 encoded... a bit more than the 15 displayed by Flutter counter.

And by the way, Flutter official TextField documentation is pretty clear on this exact issue:

Limitations
The text field does not currently count Unicode grapheme clusters (i.e. characters visible to the user), it counts Unicode scalar values, which leaves out a number of useful possible characters (like many emoji and composed characters), so this will be inaccurate in the presence of those characters. If you expect to encounter these kinds of characters, be generous in the maxLength used.

...we need an UTF-8 length limiter...

When reading documentation above it becomes clear that we need our own implementation of TextField length limiter. Fortunately for us, this is pretty easy to do as TextField has a param called inputFormatters that takes a list of input formatters:

Optional input validation and formatting overrides.

Formatters are run in the provided order when the text input changes.

By the way Flutter default text length limiter works by using a specific input formatter called LengthLimitingTextInputFormatter, so now we will just mimic its behavior to get our very own length limiter that will limit on bytes count:

class _Utf8LengthLimitingTextInputFormatter extends TextInputFormatter {
  _Utf8LengthLimitingTextInputFormatter(this.maxLength)
      : assert(maxLength == null || maxLength == -1 || maxLength > 0);

  final int maxLength;

  @override
  TextEditingValue formatEditUpdate(
      TextEditingValue oldValue,
      TextEditingValue newValue,
      ) {
    if (maxLength != null &&
        maxLength > 0 &&
        bytesLength(newValue.text) > maxLength) {
      // If already at the maximum and tried to enter even more, keep the old value.
      if (bytesLength(oldValue.text) == maxLength) {
        return oldValue;
      }
      return truncate(newValue, maxLength);
    }
    return newValue;
  }

  static TextEditingValue truncate(TextEditingValue value, int maxLength) {
    var newValue = '';
    if (bytesLength(value.text) > maxLength) {
      var length = 0;

      value.text.characters.takeWhile((char) {
        var nbBytes = bytesLength(char);
        if (length + nbBytes <= maxLength) {
          newValue += char;
          length += nbBytes;
          return true;
        }
        return false;
      });
    }
    return TextEditingValue(
      text: newValue,
      selection: value.selection.copyWith(
        baseOffset: min(value.selection.start, newValue.length),
        extentOffset: min(value.selection.end, newValue.length),
      ),
      composing: TextRange.empty,
    );
  }

  static int bytesLength(String value) {
    return utf8.encode(value).length;
  }
}

This class is pretty simple : it checks whether we are exceeding maximum length (see formatEditUpdate method). If we already are at maximum bytes length and try to type in another char, this char will simply be rejected. In case we suddenly exceed the char limit with more than 1 char (e.g. if multiple chars are being pasted to the text field), we will call truncate method to get our string back to maxLength bytes.
Also note the important method here:

static int bytesLength(String value) {
  return utf8.encode(value).length;
}

This method simply returns the length of a String in bytes rather than in Unicode characters as done in Flutter default's length limiter.

Great, let's use this new input formatter in our TextField:

TextField(
  maxLength: 20,
  inputFormatters: [
     _Utf8LengthLimitingTextInputFormatter(20),
  ],
)

NOTE: In this example length limiter class must be in the same file as your widget because I declared the formatter class private to the current file.

All good ? 🤔 Hum, actually if we test now our text field input will be effectively limited to 20 (or N) bytes and we won't be able to enter a new "character" (letter, emoji, etc) if this new character would make the total bytes length exceed our limit. So even if you are currently at 18/20, you won't be able to type in an awesome 4-bytes emoji 🥳. However, the counter is still TextField's default counter so in my previous example I will be blocked but the counter will be wrong, as shown in the capture below:

Counter should actually be at 15 because I entered 15 bytes worth of data with the heart emoji coded as 4 bytes.

Let's customize Flutter TextField char counter !

...and a custom chars counter

To display the correct number of bytes typed in the text input, we need to override default char counter and make sure it's displayed every time a character is typed, as the standard char counter would do.

To do this we can use buildCounter of TextField class and return our own widget. By default the builder callback takes a parameter named currentLength which is the wrong length, so we will ignore this parameter and calculate string bytes length using UTF-8 encoding instead. To do this, we need a reference to the current value of our TextField, so let's create a TextEditingController in our widget state:

TextEditingController _utf8TextController = TextEditingController();

Finally, let's set the buildCounter callback that uses our text controller to check the UTF-8 bytes length every time the user enters/pastes some text:

TextField(
  controller: _utf8TextController,
  maxLength: 20,
  buildCounter: (
    context, { currentLength, isFocused, maxLength }) {
      int utf8Length = utf8.encode(_utf8TextController.text).length;
      return Container(
        child: Text(
          '$utf8Length/$maxLength',
          style: Theme.of(context).textTheme.caption,
          ),
        );
    },
 inputFormatters: [              
   _Utf8LengthLimitingTextInputFormatter(maxLength),            
 ],
)

And voilà 🥳 ! We now have a beautiful text field input limited to 20 bytes, not characters... whatever that means.

Notes:

I added a caption style to the counter to mimic counter original look.
This technique would obviously work with CupertinoTextField as well as it presents exactly the same parameters.

Full code samples are available in this Github demo repo.

Native images with Micronaut and GraalVM

Olivier Revial — Mon, 01 Jun 2020 13:04:43 +0000

After a Micronaut introduction we saw in a previous article how we can create a simple and powerful CLI application using Micronaut and Picocli, but our application was too slow for a CLI... it's time to take it to the next level by GraalVM native compilation support to this app to run a very (very) fast CLI application 🏎 !

Understanding native compilation

If you are used to the JVM environment, you might be wondering what native compilation is and how it differs from JVM compilation. Let's forget Micronaut for a while and focus on this.

JVM compilation

When we compile a JVM application (say a Java or Kotlin app for example), we usually package it as a JAR (Java ARchive), which is made of "bytecode", our well-known ".class" files. In order to run a Java application we have to run it on a Java Virtual Machine, i.e. a virtual machine that has been specifically developed to interpret our Java bytecode. The great thing about the JVM is that JVMs exist for all systems (Unix, Mac, Windows), meaning that as a developer you only need to care about compiling your app to a jar, and then the JVM will run it, no matter which system you're on. In short, the JVM idea comes from this 1995 slogan:

Write once, run anywhere

You can write and compile your application code on your Windows machine (using the Windows-compatible Java Development Kit) and deploy your jar to your Debian server (using the Unix-compatible Java Runtime Environment).

So far so good, isn't it ? Well, if it's working so well, why in earth do we need another compilation system for our apps ? Well, as always things are not black and white. JVM interoperability is great but comes at the cost: the cost of adding a layer between the operating system and the application: yes, you guessed it, the JVM itself. This cost is paid in terms of memory footprint and startup times. This is due to the JVM weight but also to the fact that most JVM applications heavily rely on runtime reflection, making the application startup slower and slower (and the memory footprint bigger and bigger) as the codebase grows.

Native compilation

Under this mysterious "native compilation" is actually a quite simple principle: you translate your source program into native code - i.e. code composed of instructions directly recognized by your CPU - using tools such as an assembler or a linker. And these are exactly the same tools used to make the "native applications" that run on your computer. The native code is therefore linked to a particular family of processors (and thus a family of operating systems) sharing the same set of instructions.
C language is a very good example of native compilation: when you run gcc what you do is compiling your code into an exe on Windows or an executable file on Unix systems:

> gcc -o helloworld helloworld.c
> ls -lah                                                                    

.rwxr-xr-x   12k olivier 23 May 19:39  helloworld

Although the interoperability problem might be a concern, native compilation offers various advantages:

applications will start much faster
applications will have a much lower memory footprint

Also note that the interoperability becomes less of a problem with containers as we are now able to compile native code using a container (with a specific operating system) and run the native image in a very lightweight version of the same operating system (only basic system libs such as file or networks libs are needed).

Is the difference so big ?

Well, let's test it with this Hello World application:

class HelloWorld {
    public static void main(String[] args)
    {
        System.out.println("Hello World !");
    }

}

Let's first compile it and run it with the JVM:

> javac HelloWorld.java
> time "java HelloWorld"

Benchmark #1: java HelloWorld
  Time (mean ± σ):      92.9 ms ±   9.0 ms    [User: 94.4 ms, System: 23.2 ms]
  Range (min … max):    81.5 ms … 122.1 ms    29 runs

And now let's do the same compiling with GraalVM (more on that later):

> native-image HelloWorld
> time "./helloworld"

Benchmark #1: ./helloworld
  Time (mean ± σ):       5.8 ms ±  14.6 ms    [User: 1.6 ms, System: 1.4 ms]
  Range (min … max):     2.3 ms …  86.8 ms    33 runs

Yes, even the most simple Hello World starts more than 10 times faster when compiled natively 🏎 !

ℹ️ I used hyperfine here instead of time to run a benchmark on commands

Do I need to run fast (and small) ?

Right now you may be asking yourself whether you need to run an app in tenth of milliseconds instead of a second.

I'll give the easy answer: it depends on your use case. If you are only deploying a few microservices a few times a month on big servers, you most probably don't care much about the speed, memory footprint or size of your application. On the other hand, there a few use cases where native images can be useful or even essential:

when deploying the app as a serverless function, time is of the essence as you want to reduce cold starts as much as possible.
when developing a CLI application (.e.g with Picocli), a very fast startup time is mandatory if you want to keep a good and enjoyable user experience.
when you care about the size of your app and its memory footprint. This is particularly true when deploying dozens of microservices packaged into containers onto a small cluster: the smaller the better. Native image, as they do not need a full JVM to run, are not contained on ~300MB JVM containers but rather in ~50MB containers.
when security is a concern. When it comes to security of your containerized applications, the less the better. As native images require less libraries to run (basically they just require a very basic system with only networking libs and such) instead of an heavy JVM, attack surface is reduced.

Micronaut and GraalVM

Now that we know what native image compilation is, let's see how it works on a real application. In this article we will reuse weather-cli application from previous article and add it native compilation support to see the differences between a JVM application and a native binary.

⁉️ What ?

The app I'm talking about is a weather CLI application that uses Micronaut and Picocli and interacts with Weatherbit.io API to provide the following commands:

weather command will return the current weather for a given location (city and country)
forecast command will return a forecast for the next 1 to 16 days for a given location (city and country)

❓ Why Micronaut ?

In a strict sense we don't need a specific framework to use GraalVM and compile an application down to a native binary. However Micronaut provides powerful features and out of the box support for native compilation ! That's the great thing about Micronaut : it doesn't use runtime reflection or proxies, making it an excellent candidate for compiling native images to run very (very) fast microservices !

Repository

Code sample for this article can be found in this Github repository.

Configuring our application for native compilation

GraalVM configuration

As mentioned earlier we will reuse an existing (Java) Micronaut app and add it native image support. Note however that if we were to create our native app from scratch we could use Micronaut mn command to generate our app skeleton to pre-configure our build with Picocli and GraalVM dependencies, as mentionned in our Micronaut 101 article:

> mn create-cli-app weather-cli --features graal-native-image,http-client

The resulting build.gradle build file would have been similar to our previous build file expect for the added GraalVM dependency.

➡️ Let's add GraalVM dependency and Micronaut specific graal support to our build.gradle (in the dependencies block)

dependencies {
...
    annotationProcessor "io.micronaut:micronaut-graal"
    compileOnly "org.graalvm.nativeimage:svm"
...
}

In reality adding graal-native-image feature to Micronaut command-line generation did more than just adding dependencies to the build and also added some needed configuration files for native compilation.

There is one file we need to add to our project because GraalVM needs it and will try to pick it up: native-image.properties.

➡️ Add a new file in a new directory under src/main/resources/META-INF/native-image/{package.name}/{application-name}/native-image.properties.

Note: In our case the file must be at the path src/main/resources/META-INF/native-image/weather.cli/weather-cli-application/native-image.properties.

➡️ Add the following content to this file:

Args = -H:IncludeResources=logback.xml|application.yml|bootstrap.yml \
       -H:Name=weather-cli \
       -H:Class=weather.cli.WeatherCliCommand

Basically we tell GraalVM to include some resources files to our compiled application, and we tell it where to find the main class once the app has been compiled.

Alright, that's pretty much it for GraalVM configuration as far as Micronaut is concerned !

Picocli native configuration

Although our Micronaut/GraalVM configuration is correct, if we try to compile our app using native-image right now, it wouldn't compile (or would crash at runtime) because Picocli uses reflection at its core, however reflection is not possible on a GraalVM runtime application. In other words, we need a way to tell native-image tool that our Picocli application uses certain classes using reflection and thus that the tool should include them in the native binary.

GraalVM is able to read a file named reflection-config.json - that should be under the same META-INF directory as native-image.properties - that tells it what classes in our app are usually loaded using reflection, to make sure GraalVM includes them at compile time, and is able to know what fields and methods it should include.

The file for our app basically looks this:

[ {
  "name" : "weather.cli.commands.ForecastSubcommand",
  "methods" : [ {
    "name" : "setNbDays",
    "parameterTypes" : [ "int" ]
  }, {
    "name" : "run",
    "parameterTypes" : [ ]
  } ]
}, {
  "name" : "weather.cli.commands.LocalizedCommand",
  "fields" : [ {
    "name" : "spec"
  }, {
    "name" : "spec"
  }, {
    "name" : "spec"
  } ],
  "methods" : [ {
    "name" : "setCountry",
    "parameterTypes" : [ "java.lang.String" ]
  }, {
    "name" : "setCity",
    "parameterTypes" : [ "java.lang.String" ]
  }, {
    "name" : "run",
    "parameterTypes" : [ ]
  } ]
}, {
  "name" : "weather.cli.WeatherApplication",
  "methods" : [ {
    "name" : "main",
    "parameterTypes" : [ "java.lang.String[]" ]
  }, {
    "name" : "run",
    "parameterTypes" : [ ]
  } ]
}, {
  "name" : "weather.cli.commands.CurrentWeatherSubcommand",
  "methods" : [ {
    "name" : "run",
    "parameterTypes" : [ ]
  } ]
} ]

😱 Beautiful isn't it ?

Well of course if you are as lazy as I am you are now sweating while realizing this file should be updated each time you had a Picocli-related class !

But don't worry, Picocli provides a tool named picocli-codegen to automatically include this reflection configuration file in our project build. Told you Picocli was awesome 🙃

➡️ Let's add Picocli annotation processor to our build.gradle, inside the dependencies block:

annotationProcessor 'info.picocli:picocli-codegen:4.0.3'

➡️ We can now repackage our app:

>./gradlew assemble

...
Note: ReflectConfigGen writing to: CLASS_OUTPUT/META-INF/native-image/picocli-generated/reflect-config.json
Note: ResourceConfigGen writing to: CLASS_OUTPUT/META-INF/native-image/picocli-generated/resource-config.json
Note: ProxyConfigGen writing to: CLASS_OUTPUT/META-INF/native-image/picocli-generated/proxy-config.json

Phew, Picocli just generated the reflection-config.json file for us ! What it did is basically just adding Picocli commands and subcommands as needed classes for native compilation.

Well, are we ready to compile our app down to a native image ? Well, actually we still have two problems.

The first resides in the fact that dependency injection will only be added in the reflection config when using constructor injection or public fields. In other words, our private injected WeatherAPIClient in CurrentWeatherSubcommand and ForecastSubcommand won't work while doing native compilation.

@Inject
private WeatherAPIClient weatherAPIClient;

We have two choices to fix this: either make this field public or use constructor injection. Although I would always recommend the constructor injection, Picocli subcommands require a no-args constructor so we can't do that.

➡️ For simplicy, let's do it the ugly way (remember to do this in both subcommands!):

@Inject
public WeatherAPIClient weatherAPIClient;

ℹ️ the actual good way of doing this would be to use a custom factory to be able to call an arg-constructor, but I'd rather keep the code simple here.

Our last problem resides in our mapping API beans whose constructor and methods won't be found by Jackson after native compilation, as in the example below:

Type [weather.cli.api.WeatherAPIClient$Intercepted] executed with error: Error decoding HTTP response body: Error decoding stream for type [class weather.cli.api.WeatherResponse]: Cannot construct instance of `weather.cli.api.WeatherResponse` (no Creators, like default construct, exist): cannot deserialize from Object value (no delegate- or property-based Creator)

➡️ To solve this problem all we need to do is to annotate our four POJO with @Introspected annotation:

@Introspected
public class ForecastResponse {
    // ...
}

➡️ Repeat the operation for the four beans: ForecastObservation, ForecastResponse, WeatherObservation, WeatherResponse.

Phew, our configuration is now ready, we can launch native compilation.

Native compilation

Native compilation is done in two steps:

➡️ First build the uber-jar as usual:

>./gradlew assemble

➡️ Now launch native compilation using GraalVM native-image tool:

> native-image --no-server --class-path build/libs/weather-cli-0.1-all.jar                                           
[weather-cli:23973]    classlist:   6,031.94 ms,  1.52 GB
[weather-cli:23973]        (cap):   2,442.65 ms,  1.52 GB
[weather-cli:23973]        setup:   4,441.13 ms,  1.52 GB
[weather-cli:23973]   (typeflow):  67,023.02 ms, 10.51 GB
[weather-cli:23973]    (objects):  47,327.83 ms, 10.51 GB
[weather-cli:23973]   (features):   4,982.65 ms, 10.51 GB
[weather-cli:23973]     analysis: 123,953.65 ms, 10.51 GB
[weather-cli:23973]     (clinit):   1,745.52 ms, 10.71 GB
[weather-cli:23973]     universe:   5,704.32 ms, 10.71 GB
[weather-cli:23973]      (parse):  11,405.65 ms, 10.71 GB
[weather-cli:23973]     (inline):  19,854.90 ms, 11.61 GB
[weather-cli:23973]    (compile): 102,471.28 ms, 11.61 GB
[weather-cli:23973]      compile: 137,496.58 ms, 11.61 GB
[weather-cli:23973]        image:   9,296.04 ms, 11.61 GB
[weather-cli:23973]        write:   3,119.01 ms, 11.61 GB
[weather-cli:23973]      [total]: 290,640.34 ms, 11.61 GB

As you can see native compilation takes quite a while to complete (almost 5 minutes on my machine), but if everything goes well you should see an output similar to the one above !

A few notes:

We run the command with --no-server because we don't need server-based image building
We pass our built jar as a build input using --class-path argument
We could have used Docker to build our native image instead of the native-image tool (Micronaut provides a docker-build.sh script) but I wanted to show the "hard-way" so you know what is actually going on 😉

And now, the best part of this article: time to run the binary app !

First let's verify that we have a binary:

> ls -lah

...
.rwxr-xr-x   50M olivier 23 Mar 18:38  weather-cli*

As you can see, this 50MB file is our executable, standalone binary application, that needs no other dependency to be run (SubstrateVM is already included in this binary). Pretty cool isn't it ?

➡️ Time to run it:

# Run the app and ask for Montreal weather
> ./weather-cli --country CA --city montreal

Established active environments: [cli]
Asking weather for city montreal and country ca
=====================================
Current weather in montreal:
  - temperature: 1.9°C
  - wind speed: 11.268 km/h
  - cloud coverage: 100.0%
  - precipitation: 0.0 mm/h
=====================================

# Ask for Paris 3-days forecast
> /weather-cli forecast --city Paris --days 3

Established active environments: [cli]
Asking forecast for city paris and country fr
Using default city paris...
Using default country fr...
=====================================
Forecast in paris on day 2020-03-23:
  - average temperature: 7.8°C
  - min temperature: 4.2°C
  - max temperature: 10.8°C
  - wind speed: 12.081204 km/h
  - probability of precipitation: 0.0%
  - average cloud coverage: 0.0%
Forecast in paris on day 2020-03-24:
  - average temperature: 5.1°C
  - min temperature: 0.5°C
  - max temperature: 10.8°C
  - wind speed: 8.629308 km/h
  - probability of precipitation: 0.0%
  - average cloud coverage: 0.0%
Forecast in paris on day 2020-03-25:
  - average temperature: 4.0°C
  - min temperature: 0.1°C
  - max temperature: 8.7°C
  - wind speed: 9.142452 km/h
  - probability of precipitation: 0.0%
  - average cloud coverage: 0.0%
=====================================

Of course the results are no different than previous outputs, but it's always good to see that our app is actually working when launched as a standalone binary !

Performance

Time to measure execution time and compare it to our previous JVM execution !

🥁🥁🥁 Drumroll please 🥁🥁🥁

> time "./weather-cli --help"

Benchmark #1: ./weather-cli --help
  Time (mean ± σ):      39.5 ms ±   5.8 ms    [User: 16.1 ms, System: 15.7 ms]
  Range (min … max):    32.0 ms …  57.8 ms    67 runs

Wow, our response time just dropped from ~2 seconds down to an average 40ms. Pretty awesome right ?

ℹ️ Again I used hyperfine here instead of time to run a benchmark on the help command

Conclusion

In this article we saw the differences between JVM and native compilation. We implemented a real-world use-case to see how to actually use native compilation to compile an application down to a native image.

The main takeaway of this is that an application startup time and memory footprint can be dramatically improved by using native compilation instead of traditional JVM compilation, but also that it comes at a certain cost: not everything can be done the way it was when simply compiling for the JVM, and even when possible it can sometimes require an extra configuration step before an app can be run natively.

And the great thing is: frameworks and libraries such as Micronaut and Picocli remove the need for handling reflection configuration manually by providing a set of tools to automatically generate appropriate reflection files !

...beyond Micronaut

Micronaut is not the only one to propose a simplified way of writing complex apps that can be compiled down to native images with GraalVM support. Here a few examples of frameworks that support or partially support GraalVM native compilation:

Quarkus by RedHat, to write "Supersonic Subatomic Java"
Helidon by Oracle, a "collection of Java librairies for writing microservices"
MicroProfile by Eclipse, a project "aimed at optimizing Enterprise Java for the microservices architecture"
SpringBoot by Pivotal, with experimental (and hopefully future) GraalVM native images support

References:

I can only recommend the great Micronaut documentation that provides most of the information you need. You can basically find it in two places:

Micronaut user guide that covers most of the features Micronaut provides
Micronaut documentation home page that provides links to specific guides. Very useful when you need to work on specific topics such as Micronaut with GCP, Kafka or Views.

CLI applications with Micronaut and Picocli

Olivier Revial — Mon, 01 Jun 2020 12:52:52 +0000

In a previous article we introduced Micronaut framework with a simple hello-world application. Today we will see how we can easily create powerful CLI applications using Micronaut and Picocli. In a future article we will see how we can compile our app down to a native binary using Micronaut and GraalVM !

A great use case for Micronaut is to use it with the wonderful Picocli library to build very simple yet very powerful Command Line Interface (CLI) applications. Picocli️ ❤️ is an awesome library that allows creating rich command line applications for the JVM, and we can use Micronaut to enrich our application with auto-generated web clients, auto-configuration and such.

So what exactly are we talking about here ? 🤔

The exercice today will be to develop a simple weather CLI application that will connect to a distant weather API to give us current weather and forecast for a given location.

We will use Picocli️ features to easily respond to CLI commands and we will use Micronaut to interact with the distant weather API, with Micronaut http-client library. Of course for this simple example it could make more sense to use Java 11 native HTTP client but for the sake of this exercise, let's stick with Micronaut 🚀. It will allow us to see how Micronaut works in this context (and of course it could be extended with richer features such as auto-configuration) and this simple CLI application will also be a good playground for the native compilation coming in the next article 😉

In this exercice we will use Weatherbit.io API to add two commands to our CLI application:

weather command will return the current weather for a given location (city and country)
forecast command will return a forecast for the nehugxt 1 to 16 days for a given location (city and country)

👩‍💻👨‍💻 Code sample for this article can be found in this Github repository.

Generate a CLI Micronaut application

As Micronaut applications are quite simple we could create ours from scratch, but as explained in our Micronaut 101 article we will instead use Micronaut mn command to generate our app skeleton, mostly to pre-configure our build with Picocli dependency.

➡️ Run the following command to generate the application:

> mn create-cli-app weather-cli --features http-client

ℹ️ We told Micronaut to create a CLI application, thus we avoid the use of an HTTP server (useless in our case).

➡️ Let's now have a look at generated build file build.gradle:

dependencies {
    annotationProcessor platform("io.micronaut:micronaut-bom:$micronautVersion")
    annotationProcessor "io.micronaut.configuration:micronaut-picocli"
    annotationProcessor "io.micronaut:micronaut-inject-java"
    annotationProcessor "io.micronaut:micronaut-validation"
    implementation platform("io.micronaut:micronaut-bom:$micronautVersion")
    implementation "io.micronaut:micronaut-runtime"
    implementation "info.picocli:picocli"
    implementation "io.micronaut.configuration:micronaut-picocli"
    implementation "io.micronaut:micronaut-inject"
    implementation "io.micronaut:micronaut-validation"
    implementation "io.micronaut:micronaut-http-client"
    runtimeOnly "ch.qos.logback:logback-classic:1.2.3"
    testAnnotationProcessor platform("io.micronaut:micronaut-bom:$micronautVersion")
    testAnnotationProcessor "io.micronaut:micronaut-inject-java"
    testImplementation platform("io.micronaut:micronaut-bom:$micronautVersion")
    testImplementation "org.junit.jupiter:junit-jupiter-api"
    testImplementation "io.micronaut.test:micronaut-test-junit5"
    testImplementation "io.micronaut:micronaut-inject-java"
    testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine"
}

Amongst other things we can see that Micronaut added a few dependencies, including Micronaut core dependencies (java-inject, validation, runtime, etc.) along with Picocli dependency.

Add Picocli commands and Micronaut HTTP client

➡️ First let's look at WeatherCliCommand, the default main class that was generated for us:

@Command(name = "weather-cli", description = "...",
        mixinStandardHelpOptions = true)
public class WeatherCliCommand implements Runnable {

    @Option(names = {"-v", "--verbose"}, description = "...")
    boolean verbose;

    public static void main(String[] args) throws Exception {
        PicocliRunner.run(WeatherCliCommand.class, args);
    }

    public void run() {
        // business logic here
        if (verbose) {
            System.out.println("Hi!");
        }
    }
}

If you run the application now you should see a very basic CLI application that works but doesn't do much (except printing "Hi!" if called with the -v option). We will now add the weather and forecast subcommands to add behavior.

Working with Picocli commands

As both subcommands will need to take similar city and country inputs, let's factorize this.

➡️ Create a LocalizedCommand class that simply declares a city and a country, and validates these inputs:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import picocli.CommandLine;
import picocli.CommandLine.Option;
import picocli.CommandLine.Spec;

public abstract class LocalizedCommand implements Runnable {
    private static final Logger LOG = LoggerFactory.getLogger(ForecastSubcommand.class);

    @Spec
    protected CommandLine.Model.CommandSpec spec;

    private static final String DEFAULT_CITY = "paris";
    private static final String DEFAULT_COUNTRY_CODE = "fr";

    protected String city;
    protected String country;

    @Option(names = {"--country"},
        description = "the 2-letters country code",
        defaultValue = DEFAULT_COUNTRY_CODE,
        showDefaultValue = ALWAYS)
    public void setCountry(String country) {
        if (country.length() >= 2) {
            this.country = country.substring(0, 2).toLowerCase();
        } else {
            throw new CommandLine.ParameterException(
                spec.commandLine(),
                "Country parameter must be a 2-letters code");
        }
    }

    @Option(names = {"--city"},
        description = "the city name",
        defaultValue = DEFAULT_CITY,
        showDefaultValue = ALWAYS)
    public void setCity(String city) {
        this.city = city.toLowerCase();
    }

    @Override
    public void run() {
        LOG.info("Asking forecast for city {} and country {}", city, country);
        if (DEFAULT_CITY.equals(city)) {
            LOG.warn("Using default city {}...", city);
        }
        if (DEFAULT_COUNTRY_CODE.equals(country)) {
            LOG.warn("Using default country {}...", country);
        }
    }
}

Perfect, we now have the base parameters ready. Time to create real subcommands !

➡️ First, let's create the weather subcommand that simply returns the result of the Weather API call:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import picocli.CommandLine.Command;
import weather.cli.api.WeatherAPIClient;
import weather.cli.api.WeatherResponse;

@Command(name = "weather", description = "gives weather for a given location")
public class CurrentWeatherSubcommand extends LocalizedCommand {
    private static final Logger LOG = LoggerFactory.getLogger(CurrentWeatherSubcommand.class);

    @Inject
    private WeatherAPIClient weatherAPIClient;

    @Override
    public void run() {
        super.run();

        WeatherResponse weather = weatherAPIClient.weather(city, country);
        LOG.info("=====================================");
        LOG.info("Current weather in {}: \n{}", city, weather.getData().get(0).toString());
        LOG.info("=====================================");
    }
}

Of course this will not compile as we don't have the API client interface yet, nor the models. You can create the interface now if you want but we will see this interface in a minute. API models can be found in source repository.

➡️ We can now create the forecast command:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Option;
import weather.cli.api.ForecastObservation;
import weather.cli.api.ForecastResponse;
import weather.cli.api.WeatherAPIClient;

import javax.inject.Inject;

import static java.util.Comparator.comparing;
import static picocli.CommandLine.Help.Visibility.ALWAYS;

@Command(name = "forecast", description = "gives forecast for given city")
public class ForecastSubcommand extends LocalizedCommand {
    private static final Logger LOG = LoggerFactory.getLogger(ForecastSubcommand.class);

    @Inject
    private WeatherAPIClient weatherAPIClient;

    private int nbDays;

    @Option(names = {"-d", "--days"},
        description = "the number of forecast days to fetch (between 1 and 16)",
        defaultValue = "1",
        showDefaultValue = ALWAYS)
    public void setNbDays(int nbDays) {
        if (nbDays < 1 || nbDays > 16) {
            throw new CommandLine.ParameterException(
                spec.commandLine(),
                "Forecast must be between 1 and 16 days");
        }
        this.nbDays = nbDays;
    }

    @Override
    public void run() {
        super.run();

        ForecastResponse forecast = weatherAPIClient.forecast(city, country, nbDays);
        LOG.info("=====================================");
        forecast.getData().stream()
            .sorted(comparing(ForecastObservation::getForecastDate))
            .forEach(forecastObservation -> LOG.info("Forecast in {} on day {}: \n{}",
                city, forecastObservation.getForecastDate(),
                forecastObservation.toString()));
        LOG.info("=====================================");
    }
}

This class is similar to the weather subcommand except that it takes (and validates) an extra --days parameter that corresponds to the number of days we want the forecast for, and loop on the API response to display forecast of each coming day.

➡️ Great, we now have our two subcommands, let's add them to the "main" command, we can now open WeatherCliCommand:

import io.micronaut.configuration.picocli.PicocliRunner;
import picocli.CommandLine.Command;
import weather.cli.commands.CurrentWeatherSubcommand;
import weather.cli.commands.ForecastSubcommand;

@Command(subcommands = {
    CurrentWeatherSubcommand.class,
    ForecastSubcommand.class
})
public class WeatherCliCommand implements Runnable {

    public static void main(String[] args) throws Exception {
        PicocliRunner.run(WeatherCliCommand.class, args);
    }

    public void run() {
        System.out.println("Welcome to weather app...");
    }
}

Working with Micronaut HTTP Client

Creating an HTTP client with Micronaut is dead-simple. Before creating it, let's look at Weatherbit API endpoints:

Current weather endpoint:
- Requires an API key
- Takes a city and a country to localize the request
16 days forecast endpoint:
- Requires an API key
- Takes a city and a country to localize the request
- Takes an optional days parameter to specify the number of days to include in the forecast (defaults to 16)

➡️ Alright, time to create the WeatherAPIClient:

import io.micronaut.http.annotation.Get;
import io.micronaut.http.annotation.QueryValue;
import io.micronaut.http.client.annotation.Client;

@Client("${weather.api.url}")
public interface WeatherAPIClient {
    @Get("/current?key=${weather.api.key}")
    WeatherResponse weather(@QueryValue String city,
                   @QueryValue String country);

    @Get("/forecast/daily?key=${weather.api.key}")
    ForecastResponse forecast(@QueryValue String city,
                    @QueryValue String country,
                    @QueryValue(defaultValue = "1") int days);
}

What can we see here ?

We just need an interface. No need to write a client implementation, Micronaut will inject an implementation for us when needed.
@Client annotation declares a Micronaut HTTP client, and we pass it a configuration key named weather.api.url as the server base URL. Micronaut will resolve this configuration at runtime.
@Get annotation declares a HTTP Get endpoint and we tell it to add a query parameter from the configuration key weather.api.key. We pass it by configuration to avoid putting our API key directly in the code (so you don't steal my API key 🔒👮)
Both methods declare @QueryValue parameters that will be added to the URL by Micronaut at runtime.
Both methods returns a simple POJO response that will be automatically converted from the API json response by Micronaut with the help of Jackson ObjectMapper.

As said above, we added a query parameter named weather.api.key that we want to be filled in our configuration.

➡️ Let's open the application.yml file in main/resources and modify it so it matches the following content:

micronaut:
  application:
    name: weather-cli

weather:
  api:
    url: https://api.weatherbit.io/v2.0/
    key: invalid-api-key

But right now you might think that I'm crazy to put my configuration key in a configuration file that is going to be versioned in my public Github repository... well, I won't put my real key. To avoid this we could tell Micronaut that this key uses an environment variable by defining our property as such :

weather.api.key: ${WEATHER_API_KEY:invalid-api-key}

In fact we don't need this extra definition as Micronaut uses "Property Value Binding", which means that WEATHER_API_KEY environment variable value will automatically be assigned and override our property key weather.api.key.

Running our application

➡️ Alright, time to run our application and generate a uber-jar:

> ./gradlew assemble

➡️ We can now run it and ask for current weather in Montreal:

# First export WEATHER_API_KEY (replace with your own key)
> export WEATHER_API_KEY=WEATHER-BIT-IO-GENERATED-API-KEY

# Run the app and ask for Montreal weather
> java -jar build/libs/weather-cli-0.1-all.jar weather --country CA --city montreal

Established active environments: [cli]
Asking weather for city montreal and country ca
=====================================
Current weather in montreal:
  - temperature: 8.3°C
  - wind speed: 3.6 km/h
  - cloud coverage: 100.0%
  - precipitation: 6.0 mm/h
=====================================

➡️ Let's now ask for Paris 3-days forecast:

# Ask for Paris 3-days forecast
> java -jar build/libs/weather-cli-0.1-all.jar forecast --city Paris --days 3

Established active environments: [cli]
Asking forecast for city paris and country fr
Using default city paris...
Using default country fr...
=====================================
Forecast in paris on day 2020-04-30:
  - average temperature: 13.4°C
  - min temperature: 11.6°C
  - max temperature: 14.9°C
  - wind speed: 17.962884000000003 km/h
  - probability of precipitation: 75.0%
  - average cloud coverage: 86.0%
Forecast in paris on day 2020-05-01:
  - average temperature: 13.1°C
  - min temperature: 10.0°C
  - max temperature: 17.7°C
  - wind speed: 13.19382 km/h
  - probability of precipitation: 75.0%
  - average cloud coverage: 74.0%
Forecast in paris on day 2020-05-02:
  - average temperature: 12.5°C
  - min temperature: 9.1°C
  - max temperature: 16.9°C
  - wind speed: 10.229436 km/h
  - probability of precipitation: 0.0%
  - average cloud coverage: 60.0%
=====================================

Great, everything works as expected 😊

👏 Congrats, you have now successfully created a CLI application using Micronaut and Picocli !

A note on performance...

Let's time the execution time of the help command (we don't time our subcommands because they use network time to talk to the API and this would make any comparison flawed) to have a better idea of the startup time:

> time "java -jar build/libs/weather-cli-0.1-all.jar --help"

Benchmark #1: java -jar build/libs/weather-cli-0.1-all.jar --help
  Time (mean ± σ):      2.279 s ±  0.172 s    [User: 2.764 s, System: 0.216 s]
  Range (min … max):    2.075 s …  2.732 s    10 runs

ℹ️ I used hyperfine here instead of time to run a benchmark on the help command

We have a mean answer time of approximately 2 seconds. Although it can be satisfying for a JVM app, it's not great for a CLI application.

So how do we improve that ?

The answer: GraalVM native compilation !

A Micronaut introduction

Olivier Revial — Mon, 01 Jun 2020 08:09:12 +0000

Note: this article was originally published on Stack Labs blog.

Until not too long ago, when we wanted to develop JVM microservices, there wasn't much choice beyond the almighty Spring Boot. This is not true anymore with the arrival of frameworks such as Eclipse Microprofile, Helidon, Quarkus or Micronaut. In this article we will focus on Micronaut framework to discover how we can use it to develop microservices through a simple hello-world app example.

Micronaut goal

You might be wondering why using another JVM framework when Spring Boot is already working quite well for us.

If we read Micronaut official documentation introduction, it becomes clear what Micronaut claims to provide:

Fast startup time
Reduced memory footprint
Minimal use of reflection
Minimal use of proxies
Easy unit testing

By providing these points Micronaut clearly takes a completely different approach to the one taken by older frameworks like Spring Boot, Spring or Grails, and Micronaut developers know what they are talking about because they are... former Grails framework developers !

Although there are downsides to famous historical frameworks, a lot of things work well (well, that's why frameworks like Spring Boot became so popular, isn't it ?) and Micronaut intends to keep all these well-working things, such as dependency injection, auto-configuration, service discovery and HTTP clients and servers.

Now that we know what Micronaut goal is, let's dive into a real Micronaut application to see concretely how it goes for us developers ⌨️️

Creating a fully-featured app skeleton with Micronaut CLI

All the great frameworks provide a quick way to create a pre-configured app skeleton to enable for a very quick start. The idea behind this being that you spend less time initializing and configuring your app and more time actually coding and adding business value.

If you've ever seen one of Pivotal best speaker Josh Long, you already know "start dot spring dot io" where you can select the various features and other configurations for your brand new app.

Well Micronaut does not provide a web UI but a simple CLI called mn to allow easy app creation. You can easily install it using Sdkman or directly with Homebrew on a Mac.

Mac install:

> brew update && brew install micronaut

Unix install:

# With Sdkman installed
> sdk update && sdk install micronaut

➡️ Let's see what features Micronaut provides by running the following command:

> mn profile-info service

Resolving dependencies..
| Profile: service
--------------------
The service profile

| Provided Commands:
--------------------
  create-bean              Creates a singleton bean
  create-client            Creates a client interface
  create-controller        Creates a controller and associated test
  create-job               Creates a job with scheduled method
  create-repository        Creates a repository and associated test
  create-test              Creates a simple test for the project's testing framework
  create-websocket-client  Creates a Websocket client
  create-websocket-server  Creates a Websocket server
  help                     Prints help information for a specific command

| Provided Features:
| (+) denotes features included by default.
--------------------
  annotation-api (+)        Adds Java annotation API
  application (+)           Facilitates creating an executable JVM application and adds support for creating fat/uber JARs
  asciidoctor               Adds Asciidoctor documentation
  aws-api-gateway           Adds support for AWS API Gateway
  aws-api-gateway-graal     Creates an AWS API Gateway Proxy Lambda with Graal Native Image
  cassandra                 Adds support for Cassandra in the application
  config-consul             Adds support for Distributed Configuration with Consul (https://www.consul.io)
  data-hibernate-jpa        Adds support for Micronaut Data Hibernate/JPA
  data-jdbc                 Adds support for Micronaut Data JDBC
  discovery-consul          Adds support for Service Discovery with Consul (https://www.consul.io)
  discovery-eureka          Adds support for Service Discovery with Eureka
  ehcache                   Adds support for Ehcache (https://www.ehcache.org/)
  elasticsearch             Adds support for Elasticsearch in the application
  file-watch                Adds automatic restarts and file watch
  flyway                    Adds support for Flyway database migrations (https://flywaydb.org/)
  graal-native-image        Allows Building a GraalVM Native Image
  graphql                   Adds support for GraphQL in the application
  groovy                    Creates a Groovy application
  hazelcast                 Adds support for Hazelcast (https://hazelcast.org)
  hibernate-gorm            Adds support for GORM persistence framework
  hibernate-jpa             Adds support for Hibernate/JPA
  http-client (+)           Adds support for creating HTTP clients
  http-server (+)           Adds support for running a Netty server
  java                      Creates a Java application
  jdbc-dbcp                 Configures SQL DataSource instances using Commons DBCP
  jdbc-hikari               Configures SQL DataSource instances using Hikari Connection Pool
  jdbc-tomcat               Configures SQL DataSource instances using Tomcat Connection Pool
  jib                       Adds support for Jib builds
  jrebel                    Adds support for class reloading with JRebel (requires separate JRebel installation)
  junit                     Adds support for the JUnit 5 testing framework
  kafka                     Adds support for Kafka
  kafka-streams             Adds support for Kafka Streams
  kotlin                    Creates a Kotlin application
  kotlintest                Adds support for the KotlinTest testing framework
  kubernetes                Adds support for Kubernetes
  liquibase                 Adds support for Liquibase database migrations (http://www.liquibase.org/)
  log4j2                    Adds Log4j2 Logging
  logback (+)               Adds Logback Logging
  management                Adds support for management endpoints
  micrometer                Adds support for Micrometer metrics
  micrometer-appoptics      Adds support for Micrometer metrics (w/ AppOptics reporter)
  micrometer-atlas          Adds support for Micrometer metrics (w/ Atlas reporter)
  micrometer-azure-monitor  Adds support for Micrometer metrics (w/ Azure Monitor reporter)
  micrometer-cloudwatch     Adds support for Micrometer metrics (w/ AWS Cloudwatch reporter)
  micrometer-datadog        Adds support for Micrometer metrics (w/ Datadog reporter)
  micrometer-dynatrace      Adds support for Micrometer metrics (w/ Dynatrace reporter)
  micrometer-elastic        Adds support for Micrometer metrics (w/ Elastic reporter)
  micrometer-ganglia        Adds support for Micrometer metrics (w/ Ganglia reporter)
  micrometer-graphite       Adds support for Micrometer metrics (w/ Graphite reporter)
  micrometer-humio          Adds support for Micrometer metrics (w/ Humio reporter)
  micrometer-influx         Adds support for Micrometer metrics (w/ Influx reporter)
  micrometer-jmx            Adds support for Micrometer metrics (w/ Jmx reporter)
  micrometer-kairos         Adds support for Micrometer metrics (w/ Kairos reporter)
  micrometer-new-relic      Adds support for Micrometer metrics (w/ New Relic reporter)
  micrometer-prometheus     Adds support for Micrometer metrics (w/ Prometheus reporter)
  micrometer-signalfx       Adds support for Micrometer metrics (w/ SignalFx reporter)
  micrometer-stackdriver    Adds support for Micrometer metrics (w/ Stackdriver reporter)
  micrometer-statsd         Adds support for Micrometer metrics (w/ Statsd reporter)
  micrometer-wavefront      Adds support for Micrometer metrics (w/ Wavefront reporter)
  mongo-gorm                Configures GORM for MongoDB for Groovy applications
  mongo-reactive            Adds support for the Mongo Reactive Streams Driver
  neo4j-bolt                Adds support for the Neo4j Bolt Driver
  neo4j-gorm                Configures GORM for Neo4j for Groovy applications
  netflix-archaius          Adds support for Netflix Archaius in the application
  netflix-hystrix           Adds support for Netflix Hystrix in the application
  netflix-ribbon            Adds support for Netflix Ribbon in the application
  picocli                   Adds support for command line parsing (http://picocli.info)
  postgres-reactive         Adds support for the Reactive Postgres driver in the application
  rabbitmq                  Adds support for RabbitMQ in the application
  redis-lettuce             Configures the Lettuce driver for Redis
  security-jwt              Adds support for JWT (JSON Web Token) based Authentication
  security-session          Adds support for Session based Authentication
  spek                      Adds support for the Spek testing framework
  spock                     Adds support for the Spock testing framework
  springloaded              Adds support for class reloading with Spring-Loaded
  swagger-groovy            Configures Swagger (OpenAPI) Integration for Groovy
  swagger-java              Configures Swagger (OpenAPI) Integration for Java
  swagger-kotlin            Configures Swagger (OpenAPI) Integration for Kotlin
  tracing-jaeger            Adds support for distributed tracing with Jaeger (https://www.jaegertracing.io)
  tracing-zipkin            Adds support for distributed tracing with Zipkin (https://zipkin.io)
  vertx-mysql-client        Add support for the Reactive MySQL Client in the application
  vertx-pg-client           Add support for the Reactive Postgres Client in the application

Great, we can see all the features provided by Micronaut. By the way we can note that some features such as annotation-api, application, http-client, http-server or logback are included by default in any application created through mn command as they are considered basic features for any real-world application 🙂

➡️ Let's now create a hello-world Java application with a few features (plus features included by default) and with Maven build tool (mn defaults to Gradle):

> mn create-app com.stacklabs.micronaut.hello-world --lang=java --build=maven --features=junit 
| Generating Java project...
| Application created at /Users/path/to/the/micronaut-workshop/hello-world

Great, we now have our app skeleton, time to code !

Note: Micronaut CLI is great for initializing an app and avoiding the creation of Maven/Gradle boilerplate, but it is pretty basic and you cannot use it to add new features to your app once it has been initialized. If you have forgotten features, you will have to add them manually on your existing app build.

Creating a simple controller

The controller

We will now create a controller, but before that take a look at the main class generated for us by mn command:

public class Application {

    public static void main(String[] args) {
        Micronaut.run(Application.class);
    }
}

Not too complicated right ?

➡️ Now, to create our controller we can use mn command again:

> mn create-controller Hello
| Rendered template Controller.java to destination src/main/java/fr/stacklabs/HelloController.java
| Rendered template Spec.groovy to destination src/test/groovy/fr/stacklabs/HelloControllerSpec.groovy

As we can see, the command line generated a Java class (because we created a Java app) for our controller and its test counterpart. Let's have a look at this controller:

import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Get;
import io.micronaut.http.HttpStatus;

@Controller("/hello")
public class HelloController {

    @Get("/")
    public HttpStatus index() {
        return HttpStatus.OK;
    }
}

Pretty basic, the default skeleton creates a controller with a simple endpoint returning a HTTP 200. Note that the @Controller and @Get annotations are not JAX-RS annotations but Micronaut specific annotations.

➡️ Let's modify it so it adds "Hello Micronaut !" to our HTTP 200 response body:

@Controller("/hello")
public class HelloController {

    @Get("/")
    public String index() {
        return "Hello Micronaut !";
    }
}

The test

➡️ Let's now open the test and check the default content:

import io.micronaut.context.ApplicationContext;
import io.micronaut.http.HttpStatus;
import io.micronaut.http.client.RxHttpClient;
import io.micronaut.runtime.server.EmbeddedServer;
import io.micronaut.test.annotation.MicronautTest;
import org.junit.jupiter.api.Test;
import javax.inject.Inject;
import static org.junit.jupiter.api.Assertions.assertEquals;

@MicronautTest
public class HelloControllerTest {

    @Inject
    EmbeddedServer embeddedServer;

    @Test
    public void testIndex() throws Exception {
        try(RxHttpClient client = embeddedServer.getApplicationContext().createBean(RxHttpClient.class, embeddedServer.getURL())) {
            assertEquals(HttpStatus.OK, client.toBlocking().exchange("/hello").status());
        }
    }
}

What do we do have here ?

@MicronautTest annotation tells Micronaut that it needs to make the ApplicationContext available for our test
We use a constructor injection to inject an embedded server into our test
@test is a simple JUnit 5 annotation, nothing new here
Using the application context of the embedded server, we will create a simple HTTP Client bean and we configure it to connect to our embedded server (dynamic) url
We then simply execute a blocking GET request to test our very first controller method

➡️ Although this test will work (we are returning a string within an HTTP 200), let's modify it to test our "hello" string response instead:

@MicronautTest
public class HelloControllerTest {

    @Inject
    EmbeddedServer embeddedServer;

    @Test
    public void testIndex() throws Exception {
        try(RxHttpClient client = embeddedServer.getApplicationContext().createBean(RxHttpClient.class, embeddedServer.getURL())) {
            assertEquals("Hello Micronaut !", client.retrieve("/hello").blockingFirst());
        }
    }
}

We are now ready to run the test using maven wrapper:

> ./mvnw test

Once finished, you should see an output similar to:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.stacklabs.micronaut.workshop.agency.HelloControllerTest
23:18:45.538 [main] INFO  i.m.context.env.DefaultEnvironment - Established active environments: [test]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.62 s - in com.stacklabs.micronaut.workshop.agency.HelloControllerTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Running the app

You probably love tests but you might want to run the app, so let's do that.

➡️ First let's compile and package our application:

> ./mvnw clean install

By default Micronaut will add Shadow Jar dependencies and plugins to our build so it generates two jars, the "original" one that only contain
our application classes, and a self-contained executable "uber-jar" (or fat-jar) that we can deploy as is.

➡️ Let's check the output of our build:

> ls -l target/
-rw-rw-r-- 1 user group  13M mars  10 13:14 hello-world-0.1.jar
-rw-rw-r-- 1 user group  11K mars  10 13:14 original-hello-world-0.1.jar

➡️ We can now launch our app using the fat-jar and interact with it using our browser or curl:

> java -jar target/hello-world-0.1.jar
13:24:15.361 [main] INFO  io.micronaut.runtime.Micronaut - Startup completed in 968ms. Server Running: http://localhost:8080
> curl http://localhost:8080/hello
Hello Micronaut !

That's it, we now have a working REST application that we can test, compile and run !

Going further...

...with Micronaut

Of course we only created and developed a simple hello-world application so you might be a bit frustrated, but the goal of this article was simply showing how simple it is to both create and develop a Micronaut application.

You may ask yourself: where should I start to dive further into Micronaut framework ? Well it all depends on what you are trying to achieve but here are some hints:

Native compilation with GraalVM, to generate native binaries that run in a few milliseconds.
Complete REST servers with URL and body params, reactive APIs and HTTP clients to both test your servers and interact with external services such as remote HTTP APIs.
Configuration and auto-configuration
Data integration with Micronaut Data for relational databases or support for various data provider such as MongoDB, Redis or Cassandra
Microservices observability with Management & Monitoring
Cloud native features with support for various cloud through auto-configuration and such, service discovery or serverless functions.
Standalone CLI applications with the great Picocli ♥️ parser integration !
... and so much more !

...beyond Micronaut

As stated in the introduction we now have quite a lot of new JVM frameworks that aim to answer today's microservices needs. Although we won't compare them to Micronaut in this article (maybe in a future article ?), I'll still leave some links here in case you want to dig into one of them:

Quarkus by RedHat, to write "Supersonic Subatomic Java"
Helidon by Oracle, a "collection of Java librairies for writing microservices"
MicroProfile by Eclipse, a project "aimed at optimizing Enterprise Java for the microservices architecture"
SpringBoot by Pivotal, with experimental (and hopefully future) GraalVM native images support

References:

I can only recommend the great Micronaut documentation that provides most of the information you need. You can basically find it in two places:

Micronaut user guide that covers most of the features Micronaut provides
Micronaut documentation home page that provides links to specific guides. Very useful when you need to work on specific topics such as Micronaut with GCP, Kafka or Views.

DEV Community: Olivier Revial

Flutter Vikings - Day 2

The cookbook of Flutter testing

Parallel Beauty with Isolates into the Mandelbrot set

Writing Flutter apps in Lego style

Building better Bingo backends

Game Development in Flutter

Flutter Vikings - Day 1

Deep dive into Flutter theming

Fun with code generation

Flutter At Scale - Experience From 25+ Flutter Devs Working Together On A Mobile Banking App

Responsive UI for auth flows

Understanding Flutter's handling of gestures

Riverpod 2.0

Flutter : a journey to higher-quality apps

You said code quality ? 🤔

Increasing confidence & reputation 👍

Increasing code reuse ⌨️

What metrics make for a good code quality app ? 📊

Code quality in Flutter 📱

Linters ℹ️

Refactoring & IDE features 🧑‍💻

Tests ✅ ❌

Code coverage

CI & CD (Continuous Integration & Continuous Deployment)

Code quality measurement

Conclusion

CLI applications made easy with Dart & dcli

What we will do

Flutter: flavors launcher icons made simple

References

Flutter, a wonderful journey from 0 to app stores - CI featuring Codemagic and Fastlane

Prerequisites

👥 Accounts

Configuration

🧙‍♂️ Codemagic setup

🤖 Android setup

1. Google Play service account key

2. Keystore

3. Project configuration

4. Signed release & first upload

🍏📱 iOS setup

iOS Project configuration

Conclusion

Flutter UTF8 TextField length limiter and char counter

Flutter TextField is awesome, but...

...we need an UTF-8 length limiter...

...and a custom chars counter

Native images with Micronaut and GraalVM

Understanding native compilation

JVM compilation

Native compilation

Is the difference so big ?

Do I need to run fast (and small) ?

Micronaut and GraalVM

Repository

Configuring our application for native compilation

GraalVM configuration

Picocli native configuration

Native compilation

Performance

Conclusion

...beyond Micronaut

CLI applications with Micronaut and Picocli

Generate a CLI Micronaut application

Add Picocli commands and Micronaut HTTP client

Working with Picocli commands

Working with Micronaut HTTP Client

Running our application

A note on performance...

A Micronaut introduction

Micronaut goal

Creating a fully-featured app skeleton with Micronaut CLI

Creating a simple controller

The controller

The test

Running the app

Going further...

...with Micronaut

...beyond Micronaut

Flutter `TextField` is awesome, but...