DEV Community: Mathieu Kerjouan

Flutter and SQLite with sqflite

Mathieu Kerjouan — Mon, 20 Jul 2026 10:00:00 +0000

SQLite is an incredible embedded database. Anyone is using it, when you are browsing internet with your browser, or when you are using your smartphone. A library for Flutter called sqflite offers an interface to manage SQLite database on all supported platform.

Bootstrapping

As usual, let create a new project from scratch to test this package.

$ flutter create mydb
Creating project mydb...
Resolving dependencies in `mydb`... 
Downloading packages... 
Got dependencies in `mydb`.
Wrote 131 files.

All done!
You can find general documentation for Flutter at: https://docs.flutter.dev/
Detailed API documentation is available at: https://api.flutter.dev/
If you prefer video documentation, consider: https://www.youtube.com/c/flutterdev

In order to run your application, type:

  $ cd mydb
  $ flutter run

Your application code is in mydb/lib/main.dart.

$ cd mydb

sqflite can be installed with flutter pub add command. The amount of dependencies is correct.

$ flutter pub add sqflite
Resolving dependencies... 
Downloading packages... 
  matcher 0.12.19 (0.12.20 available)
  meta 1.18.0 (1.19.0 available)
+ platform 3.1.6
+ plugin_platform_interface 2.1.8
+ sqflite 2.4.3
+ sqflite_android 2.4.3
+ sqflite_common 2.5.11
+ sqflite_darwin 2.4.3+1
+ sqflite_platform_interface 2.4.1
+ synchronized 3.4.1
  test_api 0.7.11 (0.7.13 available)
  vector_math 2.2.0 (2.4.0 available)
Changed 8 dependencies!
4 packages have newer versions incompatible with dependency constraints.
Try `flutter pub outdated` for more information.

If you are working on Linux, MacOS or Windows, you will need sqlite_ffi package. It will be especially helpful for the testing part.

$ flutter pub add sqflite_ffi
Resolving dependencies... 
Downloading packages... 
+ code_assets 1.2.1
+ crypto 3.0.7
+ ffi 2.2.0
+ file 7.0.1
+ glob 2.1.3
+ hooks 2.0.2
+ logging 1.3.0
  matcher 0.12.19 (0.12.20 available)
  meta 1.18.0 (1.19.0 available)
+ native_toolchain_c 0.19.2
+ pub_semver 2.2.0
+ record_use 0.6.0
+ sqflite_common_ffi 2.4.2
+ sqflite_ffi 0.1.0
+ sqlite3 3.4.0
  test_api 0.7.11 (0.7.13 available)
+ typed_data 1.4.0
  vector_math 2.2.0 (2.4.0 available)
+ web 1.1.1
+ yaml 3.1.3
Changed 16 dependencies!
4 packages have newer versions incompatible with dependency constraints.
Try `flutter pub outdated` for more information.

It seems we are ready to code. Before starting, ensure you have the sqlite3 installed locally, it can be helpful for designing or even debugging SQLite database.

$ sudo apt-get install sqlite3

Simple Cache Schema

Before doing anything with SQLite, a schema will be required. Indeed, SQLite is like any other SQL database, and data must be structured before being stored. For a first test, creating a kind of cache system with SQLite could be interesting. The following schema will be stored in a file called cache.sql.

CREATE TABLE cache (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  key BLOB UNIQUE NOT NULL,
  value BLOB NOT NULL,
  created_at INTEGER NOT NULL DEFAULT CURRENT_TIMESTAMP,
  updated_at INTEGER
);

Few notes regarding this table, BLOB type is being used for the key and the value column because they will receive raw data from Flutter. I would like to see how SQLite will manage them. The created_at and updated_at should use TIMESTAMP, but while reading sqflite documentation, this type was not implemented. The documentation also advise the developer to use INTEGER instead. Why not. BOOL is not supported as well and INTEGER must be used instead. On this last type, SQLite BOOL type is already represented by an INTEGER.

Before integrating this schema with Flutter, let have some fun with an in-memory SQLite database.

$ sqlite3
SQLite version 3.46.1 2024-08-13 09:16:08
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.

The .read command can be used to import a schema, if you are lazy, you can also just do a copy/paste of the previous schema, it's still okay for small schemas.

sqlite> .help read
.read FILE               Read input from FILE or command output
    If FILE begins with "|", it is a command that generates the input.

sqlite> .read cache.sql

To check if the schema was correctly created, the .schema command can be used as well. It will return the current schema of the database.

sqlite> .help schema
.schema ?PATTERN?        Show the CREATE statements matching PATTERN
   Options:
      --indent             Try to pretty-print the schema
      --nosys              Omit objects whose names start with "sqlite_"

sqlite> .schema
CREATE TABLE cache (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  key BLOB UNIQUE NOT NULL,
  value BLOB NOT NULL,
  created_at INTEGER NOT NULL DEFAULT CURRENT_TIMESTAMP,
  updated_at INTEGER
);
CREATE TABLE sqlite_sequence(name,seq);

When dealing with complex schema, it can be helpful to have headers displayed when checking the content of a table. For doing that, one can use the .headers command.

sqlite> .help headers
.headers on|off          Turn display of headers on or off

sqlite> .headers on

The environment is now ready. Let insert some values in the cache table.

sqlite> INSERT INTO cache (key, value) VALUES ('key', 'value');

It looks good, let check if the data has been correctly inserted.

sqlite> SELECT * from cache;
id|key|value|created_at|updated_at
1|key|value|2026-07-13 12:31:41|

Integration

The schema is ready, but it needs to be integrated in the Flutter application previously created. Let start with the headers, the device target will be a desktop Linux application, and sqflite_ffi will be required.

import 'package:flutter/material.dart';
import 'package:sqflite/sqflite.dart' as sqflite;
import 'package:sqflite_ffi/sqflite_ffi.dart';

Entry-Point

When the application is starting, the main() entry-point is called. If something goes wrong during this part of the code, the application must crash quick. The SQLite database is opened with the help of the sqflite.openDatabase() function and will be stored in memory by setting the path to the inMemoryDatabasePath constant (in fact, this is a String containing the value :memory).

The database is then passed to the CacheInterface.init() method. The CacheInterface class will be described later, but in short, it contains the attributes, parameters and methods to deal with the cache table present in the database.

The last step here is to start the application by calling the runApp() Flutter function with our Application() object instantiated with the cache object previously created.

void main() async {
  final db = await sqflite.openDatabase(inMemoryDatabasePath);
  final CacheInterface cache = CacheInterface.init(db: db);
  runApp(Application(cache: cache));
}

The database can be opened with custom options via the OpenDatabaseOptions. A quick note regarding this part, if the database is coming from the filesystem, its presence should be checked first.

Material Application

The Application class is extending a StatefulWidget. Why? Because it was choice I did in the first version of the tests, but a StatelessWidget would have done the job perfectly.

class Application extends StatefulWidget {
  final CacheInterface cache;

  const Application({
    super.key,
    required CacheInterface this.cache,
  });

  static ApplicationState? of(BuildContext context) {
    return context.findAncestorStateOfType<ApplicationState>();
  }

  @override
  State<Application> createState() => ApplicationState(cache: cache);
}

One interesting function is the of() one. Indeed, this is a static function used to wrap the findAncestorStateOfType() method and then help to find the ApplicationState across the whole application tree. It was mostly a test to see how this function was implemented and how it was working. If someone wants to share a state across the application, it would be easier to use an InheritedWidget.

The second part of this code is the state definition. In our case, it will store the reference of the CacheInterface() as well. The build() function is returning a MaterialApp() object, where the home page is created using the Home class.

class ApplicationState extends State<Application> {
  final CacheInterface cache;

  ApplicationState({ required CacheInterface this.cache });

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'MyDB',
      theme: ThemeData(
        colorScheme: .fromSeed(seedColor: Colors.deepPurple),
      ),
      home: const Home()
    );
  }
}

A quick note on this part, the state is created there because it's one of the first widget created on the tree. This means the state can easily be shared across the rest of the application.

Home Screen

The Home screen will also be a StatefulWidget (why? because I need to have the habit to create them, and that's a good exercise). So, the Home class is not really complex and does not contain any specific methods or attributes.

class Home extends StatefulWidget {
  const Home({super.key});

  @override
  State<Home> createState() => _HomeState();
}

The _HomeState class is a bit more complex though. The attributes length and items will respectively store the length of the items from the database and the items themselves. In fact the length can simply be a getter like int get length => items.length; or something similar.

class _HomeState extends State<Home> {
  int length = 0;
  List<Map<String, Object?>> items = [];

The build method will be split in small part, because lot needs to be say there.

  @override
  Widget build(BuildContext context) {

Firstly, let's talk a bit of the cache variable, this one is created by finding the CacheInterface state using the Application.of() method defined in the Application class. It will then return the cache object containing the database. A Scaffold object is returned. The appBar parameter is set with a title, and that's all.

    CacheInterface? cache = Application.of(context)?.cache;
    return Scaffold(
      appBar: AppBar(
        title: Text("MyDB"),
        actions: []
      ),

Scaffold's body is a bit more complex though. The first object created there is a NotificationListener(), this element is new on my side. The idea is to listen for Notification sent by Widgets below in the tree. Those Notifications are generated by the Notification().dispatch() method and will bubble to the root of the tree. The NotificationListener() Widget can stop this bubbling by returning true when the onNotification callback function is called.

Regarding the callback, it will only set the length and the items attributes when the DatabaseInputNotification notification is received. cache.getLength() and cache.list() methods are defined later in the code, but they are both asynchronous function, they are then both returning a Future object. To deal with that, I'm using Future.then() method and then call the setState(). This is probably not the best method, using a StreamBuilder object would be maybe a bit better, but I discovered it only recently.

      body: NotificationListener(
        onNotification: (DatabaseInputNotification din) {
          cache!.getLength()
            .then((x) {
              setState(() {
                this.length = x;
              });
            });
          cache!.list()
            .then((items) {
              setState(() {
                this.items = items!;
              });
            });
          return true;
        },

The direct child of the NotificationListener Widget is a Container, the most interesting part of this code is the ListView.builder() part, where the itemCount parameter as int is using the length attribute and the itemBuilder parameter using the items attribute from the state.

The ListView Widget can then dynamically build its list using the data from items. This is not an efficient method though, because every time a new item is created or deleted, the full content of the database is stored in the items attributes. It does the work though, many works around can be created there, for example, by storing only the id of the item in the items attribute for example, or, even better, create a fully interface to the database able to return both the length but also an item based on an index.

        child: Container(
          padding: EdgeInsets.all(10),
          child: Column(
            spacing: 10,
            children: [
              Flexible(
                child: DatabaseInput()
              ),
              Expanded(
                child: ListView.builder(
                  itemCount: this.length,
                  itemBuilder: (BuildContext context, int index) {
                    return Item(item: items[index]);
                  },
                )
              )
            ]
          )
        )
      )
    );
  }
}

Inputs

Adding data into the database requires some kind of inputs. Those data can be added from a remote server, but in our case, they will be added by the user directly. A StatefulWidget will be used again.

class DatabaseInput extends StatefulWidget {
  DatabaseInput({super.key});

  @override
  State<DatabaseInput> createState() => _DatabaseInputState();
}

The state of the object will encapsulate key and value as String. Those will be contain user's data.

class _DatabaseInputState extends State<DatabaseInput> {
  String key = "";
  String value = "";

To make things "easier", the TextFields widget have their own methods. The _buildKey() method will create a TextField dedicated for the key. As you can see, the onChanged() anonymous function is able to configure the key, in short, when an user is updating the content of the TextField (by adding/removing some letters) the key attribute will reflect it.

  Widget _buildKey(context) {
    final onChanged = (String key) {
      setState(() {
        this.key = key;
      });
    };
    return Row(
      spacing: 10,
      children: [
        Text('key:'),
        Expanded(
          child: TextField(onChanged: onChanged)
        )
      ]
    );
  }

The same is done for the value, by using a method called _buildValue().

  Widget _buildValue(BuildContext context) {
    final onChanged = (String key) {
      setState(() {
        this.value = value;
      });
    };
    return Row(
      spacing: 10,
      children: [
        Text('value:'),
        Expanded(
          child: TextField(onChanged: onChanged)
        )
      ]
    );
  }

When the user has done with the key/value, the data can be sent to the database when pressing a TextButton. It will add these two elements with the help of the cache.add() method, and then, will send a Notification via DatabaseInputNotification().dispatch() to notify other Widget to rebuild their screen if needed.

  Widget _buildButton(BuildContext context) {
    final CacheInterface? cache = Application.of(context)?.cache;
    return TextButton(
      onPressed: () {
        setState(() {
          cache!.add(this.key, this.value);
          DatabaseInputNotification().dispatch(context);
        });
      },
      child: Text('add')
    );
  }

Finally, the build() method is defined.

  @override
  Widget build(BuildContext context) {
    return Container(
      child: Row(
        spacing: 10,
        children: [
          Expanded(child: _buildKey(context)),
          Expanded(child: _buildValue(context)),
          _buildButton(context),
        ]
      )
    );
  }
}

Notifications

Remember the NotificationListener widget from the Home class? Well, a Notification object is required to notify it, but the Notification class is abstract. A new class extending it is then needed. In our case, it is called DatabaseInputNotification. When I created it, I was thinking if it's a good practice to add some methods in it to directly embed the actions to do when some updates are made.

class DatabaseInputNotification extends Notification {}

Items

Every items retrieved from the SQLite database are typed as Map<String,Objects?> and usuallyed returned as List. The SQL request used here is returning all the content of the objects without any kind of filtering. So, an item can be represented by the Item class extending a StatelessWidget.

This is a very poor design, the Item class should have all attribute defined inside the class itself, and should not use a dynamic Item. The constructor should then be used to initialize all required fields. When the build() method is called, those attributes can then be used safely.

class Item extends StatelessWidget {
  final dynamic item;
  Item({
    super.key,
    required dynamic this.item
  });

An instantiated item object embed all required informatio, but if an efficient implementation was required, only the id is really mandatory. Other fields can be easily retrieved from the database by using this unique identifier. The only interesting part here is the related to the TextButton widget used to delete an item. It follows the same principle seen previously, by dispatching a Notification after having removed the item from the database by calling the cache.delete() method.

  Widget build(BuildContext context) {
    final CacheInterface? cache = Application.of(context)?.cache;
    final id = item["id"];
    final key = item["key"];
    final value = item["value"];
    final locked = item["locked"];
    final created_at = item["created_at"];
    final updated_at = item["updated_at"];
    return ListTile(
      leading: Text("$id"),
      title: Text('${key} (${locked})'),
      subtitle: Text('${value}'),
      trailing: Column(
        children: [
          Text('created_at: ${created_at}'),
          Text(updated_at == null ? '' : 'updated_at ${updated_at}'),
          Expanded(child: TextButton(
            onPressed: () {
              cache!.delete(id);
              DatabaseInputNotification().dispatch(context);
            },
            child: Text('delete'),
          )),
        ],
      )
    );
  }
}

Database Interface

The "most" import part of the this application is the interface to the SQLite database using sqflite.

class CacheInterface {
  Database db;

The SQL schema containing the cache table is directly embedded in the CacheInterface class. This schema is straightforward, it creates a new class cache containing a primary key id, a column key as BLOB, a column value as BLOB, a locked column as BOOL and two columns to add a timestamp the data (created_at and updated_at).

  final String schema = '''
    CREATE TABLE cache (
      id INTEGER PRIMARY KEY AUTOINCREMENT,
      key BLOB UNIQUE NOT NULL,
      value BLOB NOT NULL,
      locked BOOL NOT NULL DEFAULT FALSE,
      created_at INTEGER NOT NULL DEFAULT CURRENT_TIMESTAMP,
      updated_at INTEGER
    );
  ''';

The main constructor requires only a valid opened database, but another one called CacheInterface.init() can also deploy the previous schema to the opened database.

  CacheInterface({required Database this.db});

  CacheInterface.init({required Database this.db, bool createSchema = true}) {
    if (createSchema == true)
      db.execute(this.schema);
  }

add() is a wrapper around Database.rawInsert() method. This is more an UPSERT than an INSERT, a key/value couple is inserted inside the database, in case of conflict, the value field is overwritten, and the update_at field updated. All SQL requests crafted with sqflite can use the ? parameter to insert value. Please avoid String concatenation here to avoid SQL injection.

  Future<int> add(String key, String value) async {
    final ret = await db.rawInsert('''
        INSERT INTO cache (key, value) VALUES (?, ?)
          ON CONFLICT (key) DO UPDATE SET
            value=?,
            updated_at=CURRENT_TIMESTAMP;
      ''',
      [key, value, value]
    );
    return ret;
  }

The delete() method is used to delete an item by using its unique id (primary key).

  Future<List<Map<String, Object?>>> delete(int id) async {
    final ret = await db.rawQuery('''
        DELETE FROM cache WHERE id=?;
      ''',
      [id]
    );
    return ret;
  }

The lock() method set the locked field to True. It was simply to test the BOOL/INTEGER types in SQLite.

  Future<List<Map<String, Object?>>> lock(int id) async {
    final ret = await db.rawQuery('''
        UPDATE FROM cache SET locked=true WHERE id=?;
      ''',
      [id]
    );
    return ret;
  }

The unlock() method set the locked field to False.

  Future<List<Map<String, Object?>>> unlock(int id) async {
    final ret = await db.rawQuery('''
        UPDATE FROM cache SET locked=false 
          WHERE id=?
          RETURNING id;
      ''',
      [id]
    );
    return ret;
  }

The list() method returns the complete list of items from the database. This is not a really efficient way of doing that, if the database contains to much items, it can have a huge impact on the memory. A better solution would probably to use LIMIT and OFFSET to only select a subset of the items when needed. This method can also have another impact, this time on the CPU and the I/O, but the amount of memory used should be negligible.

  Future<List<Map<String, Object?>>> list() async {
    final ret = await db.rawQuery('''
      SELECT * FROM cache ORDER BY id;
    ''');
    return ret;
  }

The getLength() method returns the total number of items present in the cache table.

  Future<int> getLength() async {
    final ret = await db.rawQuery('''
      SELECT count(id) as count FROM cache;
    ''');
    int l = 0;
    if (ret.length == 1) {
      l = ret[0]['count'] as int;
    }
    return l;
  }

Interfaces Summary

sqflite reflects the complexity of SQLite by offering many different interfaces to interact with the database, because a simple example like the previous one can't cover everything, a small non-exhaustive list of the interesting functions and methods can be seen below.

sqflite.databaseExists(): this function checks if a database exists (or not);
sqflite.inMemoryDatabasePath: this is a constant, used to open an in-memory SQLite database. The same behavior can be done by passing the String :memory: when opening a database;
sqflite.OpenDatabaseOptions(): an object containing extra database options for sqflite. For example, if someone needs to migrate the database, the function callbacks will be defined there;
sqflite.openReadOnlyDatabase(): it can be useful to open a database in read-only, for example after a crash or simply to protect the content of the database from the user;
sqflite.openDatabase(): this is the default function used to open a database;
sqflite.deleteDatabase(): if a database can be created, it can also be deleted. If for some reasons, the database is not required anymore (e.g. temporary data), it can be deleted using this function;
Database.close(): when the application is terminated, or if the database is not required anymore, it can be closed using this method;
Database.execute(): execute a query on the opened database. It can be used to create the schema of the database or configure some SQLite pragmas. Optional arguments can be passed as a List;

db.execute(
  '''
    PRAGMA foreign_keys = TRUE;
    CREATE TABLE t(
      id INTEGER PRIMARY KEY AUTOINCREMENT,
      value STRING UNIQUE
    );
  '''
);

Database.rawQuery(): execute a raw query and returns the result of it. It works like all the raw queries methods;

db.rawQuery(
  '''
    SELECT * FROM cache WHERE id = ?;
  ''', 
  [id]
);

Database.rawInsert(): execute an INSERT and returns its result, works like other raw queries methods;

db.rawInsert(
  '''
    INSERT INTO cache (key, value)
      VALUES (?, ?);
  ''', 
  ['mykey', 'myvalue']
);

Database.rawUpdate(): execute an UPDATE and returns its result;

db.rawUpdate(
  '''
    UPDATE FROM cache
      SET value = ?
      WHERE key = ?
  ''', 
  ['my_new_value', 'mykey']
);

Database.rawDelete(): a method used to use DELETE against the database, returning the changes;

db.rawDelete(
  '''
    DELETE FROM cache
      WHERE key = ?
  ''', 
  ['mykey']
);

Database.query(): this method is a function helper to select data from a table, by setting the different elements of a query as class parameters. It was not used in the previous application example, but it is a nice tool to deal with data;

final int id = 1;
db.query(
  'cache',
  columns: ['id'],
  where: 'id = ?',
  whereArgs: [id]
);

Database.insert(): this method is another helper, this time to insert data into a table. In case of conflict on the table, a ConflictAlgorithm can be selected (rollback, abort, fail, ignore and replace);

final item = {
  'key': 'mykey',
  'value': 'myvalue',
};
db.insert(
  'cache',
  item,
  conflictAlgorithm: ConflictAlgorithm.replace
);

Database.update(): another function helper to update data from a table;

final item = {
  'key': 'mykey',
  'value': 'myvalue2',
};

db.update(
  'cache',
  item,
  where: 'key = ?',
  whereArgs: [item.key],
  conflictAlgorithm: ConflictAlgorithm.rollback
);

Database.delete(): another method to help deleting data from a table;

db.delete(
  'cache',
  where: 'id = ?',
  whereArgs: [item.id]
);

Database.transaction(): sqflite support SQL transactions, a really important feature when inserting complex data in a database. The transaction uses an anonymous function with one argument (the transaction id as Transaction() object), if something goes wrong during the execution of this function, the data are not commited;

db.transaction((txid) {
  final int id = txid.insert(
    'cache',
    {'key': 'test', 'value': 'data'}
  );

  txid.delete(
    'cache',
     where: 'id = ?',
     whereArgs: [id]
  );
});

Database.batch(): even if sqflite does not support the WHERE IN statement, it's still possible to insert many data at the time with a batch. Those data can be inserted atomically with the Batch.commit() method or one by one by calling the Batch.apply() method;

var batch = db.batch();
batch.insert('cache', {'key': '1', 'value': '1'});
batch.insert('cache', {'key': '2', 'value': '2'});
batch.insert('cache', {'key': '3', 'value': '3'});
var results = await batch.commit();

Conclusion

The code source of this application should not be used in production environment. It is a draft and was used as sandbox to test some of the interfaces available on the sqflite library.

sqflite is the most used SQLite library on Flutter, and it does most of the given tasks. After few days of testing though, lot of important features are missing from my point of view.

The first one is the lack of WHERE IN statement support, and the different "solutions" offered by the community are sometimes totally unsafe (SQL injection¹) or complete hacks (requests crafting²). If you can't use another library, you will probably need to use a loop inside a transaction.

db.transaction(txid) {
  for (var item in items) {
    txid.rawQuery('DELETE FROM items WHERE id=?'), [item.id]);
  }
}

Another idea is to mark all data to be removed by adding a column to_delete and then use a recursive query. When marked as deleted, another tasks can be used to cleanup the database asynchronously.

ALTER TABLE items ADD COLUMN to_delete BOOL NOT NULL DEFAULT FALSE;

db.rawQuery('''
  WITH d AS (SELECT id FROM items WHERE to_delete IS TRUE)
  DELETE FROM items WHERE items.id IN d;
''');

Those are still hacky solutions, but it should be better than crafting queries and risking SQL injections. More workarounds exist, like creating a temporary table to store those elements to delete for example.

The second missing feature is the lack of prepared statement ³. In fact, this feature could have been a good alternative to deal with multi item deletion or batch queries, but well, no support. The solution? Using transactions (or batch) instead.

Another annoying behavior from this package is the overuse of asynchronous functions to query the database. I don't think a library should enforce asynchronous functions call everywhere, but should let the developer to do synchronous or asynchronous call. For example, opening a database should be a synchronous call, if the database can't be opened, the application retries or crashes. This step is usually done at the startup of the application, when no one cares are about asynchronous call because if you don't have your database is not there, the rest of the application can't work.

The best way to fix that, is to improve sqflite. Unfortunately, it's not in my todo list, if some of you wants to create a PR, you are welcome!

Finally, what do I really think of sqflite now... Well, it can be used to create proof of concepts, small prototypes or even small application with a limited amount of SQL requirements. In case of bigger app, I would probably recommend to switch to sqlite3 package.

This topic is not a small one, covering both SQL, SQLite, Dart and Flutter in the same article is not an easy task. In fact, it took me 2 or 3 days to write this dirty application while reading the documentation and the source code. It seems I'm not an expert with Flutter yet! Anyway many others developers wrote about that, here a list of resources I consulted:

Persist data with SQLite from Flutter documentation;
sqflite package on pub.dev;
Official sqflite API Documentation on pub.dev;
Official sqflite_ffi API Documentation on pub.dev;
Official sqflite git Repository at Github;
sqflite Opening a database Example at Github;
sqflite Migration Example at Github;
SQLite in Flutter: Guide with CRUD Example by Bouargalne Hamid on Medium;
SQLite in Flutter: The Complete Guide by @arslanyousaf12.

Many open-source application using sqflite can be found on Github or elsewhere, here a short list of them:

Flutter Tutorial - Database Storage Using Sqlite & Sqflite: Persist data with Flutter's Sqflite Database locally on your Android or iOS device by using Raw SQL statements;
Flutter SQFLITE MVP LOGIN APP: An Login Page App in Flutter with MVP and SQFLITE ( SQLITE) Implementation;
Flutter Sqflite Manager: Visually manage your Flutter Sqflite database;
flutter-notes-app: A notes taking app written in Flutter;
contacts: A flutter project with Implementation of a Contacts app in 4 ways (API, Custom, Preferences and Sqflite);

Have fun and Happy Hacking!

Cover Image by Theodore Black on Unsplash

please don't do what the author of the answer is saying. It's a huge risk of SQL injection, and can have huge consequences on your code. https://stackoverflow.com/questions/59215540/sqflite-in-operator-with-and-operator ↩
The first comments are from the authors of the library and should not be followed (sql injection). The last answers are probably the best because they are crafting the SQL requests by using ? parameters, but it's still crafting, and crafting SQL requests can be dangerous. https://github.com/tekartik/sqflite/issues/15#issuecomment-350979779 ↩
the author of the module said prepared statement is not supported. https://github.com/tekartik/sqflite/issues/253#issuecomment-518954845 ↩

Dealing with null in Dart

Mathieu Kerjouan — Fri, 17 Jul 2026 08:00:00 +0000

Into the void of silence, into the empty space of nothing, the joy of life is unfurled.

-- C. S. Lewis

Usually, a developer will focus on errors and exceptions to increase the stability of its application. It can be with defensive programming patterns, or like for Erlang, embracing errors and accepts them with the Let it crash philosophy (see also crash-only systems). In Dart... it was weird to see more information about null than errors management. In fact, one of the longest article on dart.dev seems to be dedicated to null safety.

How to deal with errors, exceptions and... null, in Dart? It will probably be the thing I will hate the most with Dart. Dealing with null seems to be really complex, at least, when no external packages are used.

What is null anyway?

The reserved word null evaluates to the null object.
   ⟨nullLiteral⟩ ::= null
The null object is the sole instance of the built-in class Null. Attempting to instantiate Null causes a compile-time error. It is a compile-time error for a
class to extend, mix in or implement Null. The Null class extends the Object class and declares no methods except those also declared by Object. The null object has primitive equality (10.2.3). The static type of null is the Null type
-- Dart Specification, section 17.4, page 103

So, null is an unique object only instantiated once during - probably - the initialization.

The treatment of null merits some discussion. Consider null + 2. This expression always causes an error. We could have chosen not to treat it as a constant expression (and in general, not to allow null as a subexpression of numeric or boolean constant expressions). There are two arguments for including it: First, it is constant so we can evaluate it at compile time. Second, it seems more useful to give the error stemming from the evaluation explicitly.
-- Dart Specification, page 101

Errors generate by nulls can be evaluated at compile time, this is helpful for static analysis.

Bootstrapping

For this article, a new project called n will be created. It will be used to play with null in Dart.

$ dart create n
Creating n using template console...

  .gitignore
  analysis_options.yaml
  CHANGELOG.md
  pubspec.yaml
  README.md
  bin/n.dart
  lib/n.dart
  test/n_test.dart

Running pub get...                     2.7s
  Resolving dependencies...
  Downloading packages...
  Changed 48 dependencies!

Created project n in n! In order to get started, run the following commands:

  cd n
  dart run

$ cd n

Dissecting Null

The Null object got only two attributes, hashCode and runtimeType. The hashCode returns always 0 (zero) and the runtimeType returns the Type of the object, Null in this case. Only two methods are also public, noSuchMethod() and toString().

void main() {
  print(null.hashCode);
  print(null.runtimeType);
}

0
Null

Let have a quick look on the source code.

@pragma("vm:entry-point")
final class Null {
  factory Null._uninstantiable() {
    throw UnsupportedError('class Null cannot be instantiated');
  }

  external int get hashCode;

  String toString() => "null";
}

The Null class is implemented in sdk/lib/core/null.dart as a final class. It uses a vm:entry-point pragma, confirming its presence during initialization. the _uninstantiable constructor is a bit odd to me. I don't know if it's convention or something specified somewhere, but because a factory is set, I would assume it is a way to protect against creating new instance during runtime. It could be nice to find some ways to call this method.

Operators

Let have a quick look on the Null operators. The first one to check will be the ! operator. It is only used if you are sure a variable will never be null, if the data is null, then it will raise an exception Null check operator used on a null value.

void t1(dynamic? v) {
  var v1 = v!;
  print("t1: ${v1}");
}

void main() {
  t1(1);
  t1(null);
}

$ dart run
Building package executable... 
Built n:n.
t1: 1
Unhandled exception:
Null check operator used on a null value
#0      t1 (file:///home/user/tmp/dart/n/bin/n.dart:9:13)
#1      main (file:///home/user/tmp/dart/n/bin/n.dart:5:3)
#2      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:313:19)
#3      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)

What to do if the value is null but a default value can be used instead? In this situation, the ?? (if-null) operator can be used.

An if-null expression evaluates an expression and if the result is the null object (17.4), evaluates another.

-- Dart Language Specification, Section 17.25, page 176

void t2(dynamic? v) {
  var v1 = v ?? "test";
  print(v1);
}

void main() {
  t2(1);
  t2(null);
}

$ dart run
Building package executable... 
Built n:n.
t2: 1
t2: default value

void t3(dynamic? v) {
  var v3 = null;
  v3 ??= v;
  print("t3: ${v3}");
}

void main() {
  t3(1);
  t3(null);
}

$ dart run
Building package executable... 
Built n:n.
t3: 1
t3: null

void t4(dynamic? v) {
  print(v?.toString());
}

void main() {
  t4(1);
  t4("test");
  t4(null);
}

$ dart run
Building package executable... 
Built n:n.
1
test
null

void t5(dynamic? v) {
  print([...? v]);
  print([... v]);
}

void main() {
  t5([1]);
  t5(["test"]);
  t5(null);
}

$ dart run
Building package executable... 
Built n:n.
[1]
[1]
[test]
[test]
[]
Unhandled exception:
type 'Null' is not a subtype of type 'Iterable<dynamic>'
#0      t5 (file:///home/user/tmp/dart/n/bin/n.dart:31:14)
#1      main (file:///home/user/tmp/dart/n/bin/n.dart:6:3)
#2      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:313:19)
#3      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)

It's All About Syntactic Sugar

I consume too much sugar. It’s a problem, I need to stop.

-- Kevin Jonas

Most of the previous examples are syntactic sugar around the conditional expression. This can be seen in the Figure 3 in the 17.20 section of the Dart Language specification.

A conditional expression evaluates one of two expressions based on a boolean condition

-- Dart Language Specification, Section 7.24, page 175

Let check these expression by first defining a new class R containing one attribute id and one method f. Nothing complex there, it's just to have a custom object.

class R {
  dynamic id = null;
  R(dynamic this.id);
  dynamic f(dynamic args) => args;
}

The first syntactic sugar to test is r?.attribute. If r is null then it will return null, else r.id will be used.

(dynamic, dynamic, bool) sugar1(dynamic r) {
  var a = r?.id;
  var b = r?.id == null ? null : r.id;
  final bool res = a == b;
  final ret = (a,b,res);
  print("sugar1: $ret");
  return ret;
}

The second one is the r?.attribute = e syntactic sugar. If r is null, nothing is set, but if it's not the case, the value e is set to the r.id attribute.

(dynamic, dynamic, dynamic) sugar2(dynamic r, dynamic e) {
  var a = r?.id = e;
  var b = r == null ? null : r.id = e;
  final bool res = a == b;
  final ret = (a,b,res);
  print("sugar2: $ret");
  return ret;
}

The third one is about method calling r?.f(args). If r is null then it will return null, else the method f() is called.

(dynamic, dynamic, dynamic) sugar3(dynamic r, dynamic args) {
  var a = r?.f(args);
  var b = r == null ? null : r.f(args);
  final bool res = a == b;
  final ret = (a,b,res);
  print("sugar3: $ret");
  return ret;
}

The fourth one is about List: r?[e]. If r is null then null is returned else, the index is used on the list r.

(dynamic, dynamic, dynamic) sugar4(dynamic r, dynamic e) {
  var a = r?[e];
  var b = r == null ? null : r[e];
  final bool res = a == b;
  final ret = (a,b,res);
  print("sugar4: $ret");
  return ret;
}

The fifth one is still about list, this time, if r is null, nothing can be done on the list, but if r is not null, the index e1 will be used to set the value e2.

(dynamic, dynamic, dynamic) sugar5(dynamic r, dynamic e1, dynamic e2) {
  var a = r?[e1] = e2;
  var b = r == null ? null : r[e1] = e2;
  final bool res = a == b;
  final ret = (a,b,res);
  print("sugar5: $ret");
  return ret;
}

Those functions can be added in our entry-point...

void main() {
  sugar1(R(1));
  sugar1(R(null));

  sugar2(R(1), 2);
  sugar2(R(null), 2);

  sugar3(R(1), 2);
  sugar3(R(null), 2);

  sugar4([1,2,3], 0);
  sugar4(null, 0);

  sugar5([1,2,3], 0, 1);
  sugar5(null, 0, 1);
}

... And finally invoked.

$ run bin/n.dart
sugar1: (1, 1, true)
sugar1: (null, null, true)
sugar2: (2, 2, true)
sugar2: (2, 2, true)
sugar3: (2, 2, true)
sugar3: (2, 2, true)
sugar4: (1, 1, true)
sugar4: (null, null, true)
sugar5: (1, 1, true)
sugar5: (null, null, true)

The results are similar. I know, this is a dummy test, something more accurate could have been done, but it was just to give the equivalent syntax one can use.

Conclusion

Nothing exists but atoms and the void.

-- Democritus

After a bit more than 1 month of coding with Dart, I found the null part exhausting. To me, this part of the language is close to the annoying error catching in Go... Perhaps a little bit better than that though. Adding more and more syntactic sugar to fix a problem like this one can lead to confusing syntax, and it seems to be the case. Using ?, ?? or ! in different part of the code makes everything more complex.

As usual, I am not the only one to write about this topic, and it would be a shape to not share all the resources I used to learn about null safety in Dart. Here the list:

Understanding null safety from the Official Dart documentation;
Sound null safety from the Official Dart Documentation;
Null Safety and Variables from the Official Dart Documentation;
Why nullable types? from the Official Dart Blog;
The road to Dart 3: A fully sound, null safe language from the Official Dart Blog;
Announcing Dart null safety beta on the Official Dart Blog;
Null class API documentation;
Null class implementation source code at Github;
Native NULL Assertions from Dart SDK on Github;
Null safety in Dart from leancode;
Dart Lesson 10: Null Safety (Part 1) — Why is Null Safety Needed? by @jige2025;
Dart Lesson 11: Null Safety (Part 2) — Safe Operators Explained by @jige2025;
Type Promotion and Nullability in Dart by Alexander Obregon on Substack;
Understanding Null Safety in Dart: Why It Matters and How to Use It by Ugama Benedicta Kelechi on Medium;
Dart: How to Write Shorter Null, False, Empty String, and Zero Checks from CodeStudy.net;
Mastering Dart's Null Safety: From ? to ! and Everything In Between by @hiteshm_devapp;
Master Null Safety in Dart: A Step-by-Step by Abhay Kumar Bhumihar on Medium.
STATIC TYPE CHECKER TOOLS FOR DART by Snigdha Mokkapati
Type Unsoundness in Practice: An Empirical Study of Dart by Gianluca Mezzetti, Anders Møller and Fabio Strocco
Type Soundness in the Dart Programming Language by Fabio Strocco

Have fun! And Hack well!

Cover Image by aa aaa on Unsplash

Kanell/0, Yet Another Pipeline-like Library

Mathieu Kerjouan — Mon, 13 Jul 2026 08:00:00 +0000

When I started to work with Dart and Flutter, few weeks ago, I was looking for something like Erlang finite state machine or Elixir Plug. The first one is most about dealing with state change and events, the second is to easily compose data-structures over functions. In both case, when a developer starts to use one of them, it is impossible to come back, and one will try to reproduce it in any language (in my case, I tried to reimplement them in Perl and Prolog).

What's the plan then? Well, if you are reading my publications, you probably saw few posts on reverse engineering. While I was writing them, I was also looking for functional oriented programming in Dart to make my life easier. I discovered a lot of nice libraries I will show you soon, but, I did not find something close to Erlang/Elixir design patterns. Indeed, it's fascinating how reverse engineering protocols is easy with Erlang and Elixir - mostly due to the syntax - and having a way to manage a state easily could be really helpful.

Erlang Pipeline

Every Unix/Linux user should already know what a pipeline is, the output of a function is the input of the next one. When using Unix/Linux and a shell, input or output are generally strings, but when you switch to another languages, you can have different kind of data-structure. My own definition of pipeline in Erlang is perhaps not the same than other developers, and is coming from the KISS (Keep It Simple Stupid) philosophy from Unix early days. Before showing you an example in Dart, I will show you an Erlang snippet to explain how I see a pipeline. First, let define a module called pipeline which will be used as behavior later.

-module(pipeline).
-export([run/2, run/3]).
-export([start/2, start/3]).
-export([start_link/2, start_link/3]).

Type definitions can make our life easier when creating behaviors in Erlang, it's not mandatory though.

-type args() :: term().
-type state() :: term().
-type reason() :: term().
-type result() :: {ok, state()}
                | {keep, state()}
                | {next, function(), state()}
                | {next, module(), function(), state()}
                | {stop, reason(), state()}.
-callback init(args()) -> result().
-callback 'StateName'(state()) -> result().
-optional_callback(['StateName'/1]).

A way to start the pipeline is required, two functions will be created for that, run/2 and run/3. Those functions will run the pipeline in the current process.

-spec run(Module, Args) -> Return when
  Module :: module(),
  Args :: term(),
  Return :: result().

run(Module, Args) ->
  run(Module, init, Args).

-spec run(Module, Function, Args) -> Return when
  Module :: module(),
  Function :: atom(),
  Args :: term(),
  Return :: result().

run(Module, Function, Args) ->
  do_loop(Module, Function, Args).

Erlang is great to isolate things inside processes, let create new functions called start/2 and start/3 to start the pipeline in an isolated process, outside of the main one.

-spec start(Module, Args) -> Return when
  Module :: module(),
  Args :: term(),
  Return :: result().

start(Module, Args) ->
  start(Module, init, Args).

-spec start(Module, Function, Args) -> Return when
  Module :: module(),
  Function :: atom(),
  Args :: term(),
  Return :: result().

start(Module, Function, Args) ->
  spawn(fun() ->
    run(Module, Function, Args)
  end).

We can eventually do the same with a linked or monitored process, let call those functions start_link/2 and start_link/3.

-spec start_link(Module, Args) -> Return when
  Module :: module(),
  Args :: term(),
  Return :: any().

start_link(Module, Args) ->
  start_link(Module, init, Args).

-spec start_link(Module, Function, Args) -> Return when
  Module :: module(),
  Function :: atom(),
  Args :: term(),
  Return :: any().

start_link(Module, Function, Args) ->
  spawn_link(fun() ->
    run(Module, Function, Args)
  end).

A router for the returned data from the callback functions is now required, this function will be called do_route/3. When a valid returned value is returned by do_loop/3, a new function is called with the updated state.

-spec do_route(Module, Function, Result) -> Return when
  Module :: module(),
  Function :: atom(),
  Result :: result(),
  Return :: result().

do_route(_Module, _Function, {ok, NewState}) ->
  {ok, NewState};
do_route(_Module, _Function, {stop, Reason, NewState}) ->
  {stop, Reason, NewState};
do_route(Module, Function, {keep, NewState}) ->
  do_loop(Module, Function, NewState);
do_route(Module, _Function, {next, NewFunction, NewState}) ->
  do_loop(Module, NewFunction, NewState);
do_route(_Module, _Functon, {next, NewModule, NewFunction, NewState}) ->
  do_loop(NewModule, NewFunction, NewState);
do_route(Module, Function, Result) ->
  throw({Module, Function, Result}).

Finally, the do_loop/3 function will use erlang:apply/3 function to execute the module/function returned by a function callback, with the state provided.

-spec do_loop(Module, Function, State) -> Return when
  Module :: module(),
  Function :: atom(),
  State :: term(),
  Return :: result().

do_loop(Module, Function, State) ->
  Result = erlang:apply(Module, Function, [State]),
  do_route(Module, Function, Result).

If you don't have the habit to code with Erlang or dynamic language, the idea is really let the developer creating a flow of actions. But an example is probably a better way to understand the concept. The checker module is using the pipeline behavior previously defined, and will export few functions. The run/1 function is a wrapper around pipeline:run/1. The init/1 is the function entry-point for the pipeline, it will decide which will be the next function callback to call. Finally the increment/1, decrement/1 and final/1 are the callback functions, they will receive the state and modify it until the end of the pipeline.

-module(checker).
-behavior(pipeline).
-export([run/1]).
-export([init/1]).
-export([increment/1, decrement/1, final/1]).

run(Args) ->
  pipeline:run(?MODULE, Args).

init(_Args) ->
  io:format("init~n"),
  {next, increment, 0}.

increment(State) ->
  io:format("increment~n"),
  {next, decrement, State+1}.

decrement(State) ->
  io:format("decrement~n"),
  {next, final, State-1}.

final(State) ->
  io:format("final~n"),
  {ok, State}.

What's happening when the checker:run/1 function is called?

1> c(pipeline).
{ok, pipeline}

2> c(checker).
{ok, checker}

3> checker:run(0).
init
increment
decrement
final
{ok,0}

As you can see, every functions are executed in order, and the final result is returned. Can you see the power of these modules? One small function for one small action, returning one structured data-structure. It will help to create better specification, documentation, test coverage and better understanding of the whole code. If the procedure needs to be updated, creating a new function and modifying the routing will be the only task to do. Here an extended version of the checker module.

%%%===================================================================
%%% @doc A simple pipeline-like module, inspired by `gen_statem'.
%%% @end
%%%===================================================================
-module(checker).
-behavior(pipeline).
-export([run/1]).
-export([init/1]).
-export([increment/1, decrement/1, final/1]).
-include_lib("eunit/include/eunit.hrl").

%%--------------------------------------------------------------------
%% @doc A wrapper around `pipeline:run/2'.
%% @end
%%--------------------------------------------------------------------
-spec run(Args) -> Return when
  Args :: term(),
  Return :: pipeline:result().

run(Args) ->
  pipeline:run(?MODULE, Args).

%%--------------------------------------------------------------------
%% @doc Entry-point.
%% @end
%%--------------------------------------------------------------------
-spec init(Args) -> Return when
  Args :: term(),
  Return :: pipeline:result().

init(_Args) ->
  io:format("init~n"),
  {next, increment, 0}.

init_test() ->
  ?assertEqual({next, increment, 0}, init([])).

%%--------------------------------------------------------------------
%% @doc An increment function to increment an `integer()'.
%% @end
%%--------------------------------------------------------------------
-spec increment(State) -> Return when
  State :: integer(),
  Return :: {next, decrement, State}.

increment(State) ->
  io:format("increment~n"),
  {next, decrement, State+1}.

increment_test() ->
  ?assertEqual({next, decrement, 1}, increment(0)).

%%--------------------------------------------------------------------
%% @doc A decrement function to decrement an `integer()'.
%% @end
%%--------------------------------------------------------------------
-spec decrement(State) -> Return when
  State :: integer(),
  Return :: {next, final, State}.

decrement(State) ->
  io:format("decrement~n"),
  {next, final, State-1}.

decrement_test() ->
  ?assertEqual({next, final, 0}, decrement(1)).

%%--------------------------------------------------------------------
%% @doc The final function, returning the final state (as `integer()').
%% @end
%%--------------------------------------------------------------------
-spec final(State) -> Return when
  State :: term(),
  Return :: {ok, State}.

final(State) ->
  io:format("final~n"),
  {ok, State}.

final_test() ->
  ?assertEqual({ok, 0}, final(0)).

Just to be sure, we can compile it and run the test suite.

4> c(checker).
{ok,checker}

5> checker:test().
  All 4 tests passed.
ok

All successful, but the idea is to have one function doing one small tasks and enough place to document it. To me, this is the cleanest way of coding one can do. It's elegant, self-documented, and specified. Mocking features is also easy, because the input and the output are fully in-control. It's flexible, one can easily create a loop between those functions... So, is it possible to create something similar with Dart?

Kanell Pipeline

Dart is statically typed, it will be hard to have the same flexibility than Erlang or Elixir, but it's still possible to do cool stuff. Before starting with a complex implementation including routing or asynchronous call, a simple linear pipeline should do the job.

Most of the steps of the pipeline will be defined during compile time, while in Erlang, it could be possible to define them during runtime (even if the previous example does not show that, it's possible to do it). I would like to also avoid unnecessary object creation, then, static functions or lambda function will be used. Only the state will pass between the different stage of the pipeline, and - at this time - only the pipeline will be instantiated at the beginning. An example can be seen below.

import 'package:kanell/kanell.dart';

void main() {
  var pipeline = Pipeline();

  pipeline.add(_increment);
  pipeline.add(_increment);
  pipeline.add(_increment);

  Flow flow = Flow(data: 0);

  var result = pipeline.apply(flow);
  print(result)
}

Flow _increment(Flow flow) {
  flow.data += 1;
  return flow;
}

As you can see, the code is quite different. I'm currently trying to deal with the feature itself, without types (I find the Dart types annotation annoying and confusing), but the idea is already present. Let explain a bit this code.

A pipeline is created using the Pipeline class from kanell module (be careful, Dart already got a class called Pipe). This object is a simple wrapper around a list defined by List<Function(Flow)> (a list containing functions accepting Flow object).

A function can be added to the pipeline with the pipeline.add() method. In this example, _increment() static function is added three time.

A Flow object is also required, this is the equivalent of the State variable in the Erlang example. It will contain a Status, mimicking the Erlang atoms ok, error or stop, and the data to be passed through the pipeline (via the data parameter).

When the pipeline is ready, one can call pipeline.apply() method with a Flow object, it will then return the final value as result. This is not like what we have in Erlang, but that's the first step. In fact, we could eventually create named Pipe, an object containing a name as String and a Function(Flow) stored somewhere. When a Flow is returned, it could contain the name of the function to call after.

What's the Point?

One reading this article would think it's just another way to deal with state, no real advantage. In fact, if you remember the first paragraphs of this post, this method is really helpful to deal with unstable interfaces, or undocumented interfaces. Every steps are documented, in case of failure, only a small part of the pipeline can be fixed or a new pipe can be added between actual pipes.

Conclusion

I hope this article is the first one of a long series. It's also my first Dart package. I'm not really sure if this is a good idea to implement this kind of high level procedure from a functional language like Erlang in Dart. For example, asynchronous calls, or reusable named pipeline can probably have lot of side effects. At this time, this is just a test, another way to learn more of Dart and to try to make my developer experience better.

The kanell package is currently in active development (version 0.1.0 released) but can be seen on my Github profile. It will evolve, perhaps with your help, but also with few publications to explain why and how I did those choices.

As last note, the Erlang snippet presented here is a smaller version of the one provided by the automata project created for Erlang-Punch. An article will be published on this topic, so, stay tuned. Wants to know a bit more about my research?

pipeliner module can be an alternative to my package in Dart;
pipeline_plus module is yet another alternative in Dart;
Erlang gen_statem behavior, the main source of inspiration;
Erlang gen_fsm behavior, an old finite state machine implementation in Erlang;
Elixir Plug module, the best example for a middleware/pipeline feature.

Enjoy and Happy Hack!

Cover Image by the blowup on Unsplash

SignalApp UI Chat Screen but with Flutter

Mathieu Kerjouan — Fri, 10 Jul 2026 08:00:00 +0000

If your competitors start copying you then you are doing something right!
-- Jay Baer

The previous article was focused on the SignalApp Home screen, on this one, we will try to reproduce the Chat screen, where all messages from an user are displayed. Like the last time, we will first analyze the design, and then re-implement it using only Flutter available features.

Chats Screen

If you copy my dreams, you automatically will copy my challenges too. My success will give you a crown. My challenges will make you frown.
-- Israelmore Ayivor

This article will focus on the second screenshot on the right. When an User is tapping on one active chat available from the home screen, the action is to open a new screen containing the list of messages related to this conversation.

Chat Bar

The top navigation bar using the AppBar widget is made of 3 parts, the leading is displaying a left arrow also called arrow_back. When clicking on it, the application should go to the home screen. The title parameter is now displaying the icon/avatar of the contact and its name. The Avatar part can be created via the CircleAvatar widget. The actions parameter contains 3 buttons, one to start a video call using a videocam_rounded icon, one to start a phone call using a phone_rounder icon and the last one to open a menu used to configure the parameter of the chat using a more_vert icon. The background color of this element is blue (#2b6cee), but the Colors.blue constant will be used instead.


class ChatBar extends StatelessWidget {
  const ChatBar({super.key, PreferredSizeWidget? bottom});

  @override
  AppBar build(BuildContext context) {
    return AppBar(
      backgroundColor: Colors.blue[700],
      leading: Center(child: Icon(Icons.arrow_back, color: Colors.white)),
      title: Row(
        children: [
          Padding(
            padding: EdgeInsetsGeometry.directional(end: 10),
            child: CircleAvatar(),
          ),
          Text(
            "Maya Dorai",
            style: TextStyle(color: Colors.white, fontWeight: FontWeight.bold),
          ),
        ],
      ),
      actions: [
        IconButton(
          icon: Icon(Icons.videocam_rounded),
          onPressed: () {},
          color: Colors.white,
        ),
        IconButton(
          icon: Icon(Icons.phone_rounded),
          onPressed: () {},
          color: Colors.white,
        ),
        IconButton(
          icon: Icon(Icons.more_vert),
          onPressed: () {},
          color: Colors.white,
        ),
      ],
    );
  }
}

This is not the most complex part, at least for the design. All buttons must be configured to react to user actions. For example, one user tapping on the video button should open a new screen to start a call with video support. The screenshot below shows this AppBar on Android:

Chat Messages

The next chunk of the screen is containing the list of messages. I suppose those are Card widgets, probably added in a ListView. A message can have different design depending on many parameters. The most visible one is the position of each messages, when a message is sent by the user, it is on right side of the screen and if it's received from a contact, it is put on the left side of the screen. A message with attachments will not have the same shape than a message containing only text. Messages containing only emojis will not be displayed the same as well. In short, it will be painful to design that, so, I will only try to implement only a small subset of all features.

In short, a message can contain text as String, attachments (e.g. images, video, audio) as List<Image>, emoji(s) as String or can be a sticker (signal-like emoji feature) as Image. It must have an origin (Sender or Receiver) and can have labels using emojis.

Attachments

Let start implementing the messages representation when attachments are shared. You can see on the screenshot 3 pictures of a cat, but the number of attachments can change the whole shape and design if this message. For example, with 1 attachment, the image will use all the space. with 2 attachments, both images will share the space in half, with 5+ attachments, only 5 attachments are displayed, with the one on the top taking half the space. To be honest, this part was kinda hard, and I was curious to see how it was implemented in SignalApp.

The SignalApp source code can be seen in app/src/main/java/org/thoughtcrime/securesms/components/AlbumThumbnailView.java (goddamn it, Java paths are crazy). The implemention is not so complex but look like a big hack, their idea is to create a list of attachment, and use a representation for each case (1, 2, 3... 5 attachments). It works well, I think we can reuse it.

The first step is to create a new class called AttachmentView extending a StatelessWidget. This new class will only store the list of images and how to display them.

class AttachmentView extends StatelessWidget {
  final List<Image> images;

  const AttachmentView({super.key, images}) : this.images = images;

Each images are "encapsulated" in their own AnimatedContainer (currently not active, but it was to add an animation when someone is tapping on the image). I did - too - many test to find the "best" way to display an image, but it seems the easy path was to use a BoxDecoration and BoxFit.cover. With these parameters, the image will always use the space available.

  Widget _container(Image image) {
    return AnimatedContainer(
      duration: Duration(seconds: 2),
      decoration: BoxDecoration(
        image: DecorationImage(image: image.image, fit: BoxFit.cover),
      ),
    );
  }

Many Row will be required, then using a custom method called _row can be helpful to avoid duplicated code. Nothing fancy there, a row is created after having modified via a map the list of images passed as arguments.

  Widget _row(List<Image> images) {
    final rows = images.map((x) {
      return Expanded(child: _container(x));
    }).toList();
    return Row(spacing: 2, children: rows);
  }

The same is done for Column, a _column method is created, taking a list of images are arguments and doing the applying the same procedure than the one used for the rows.

  Widget _column(List<Image> images) {
    final columns = images.map((x) {
      return Expanded(child: _container(x));
    }).toList();
    return Column(spacing: 2, children: columns);
  }

The images must be displayed in different way based on the amount of images passed as arguments. The method _displayImages() is doing that for us, based on the length of the list of image. The previous defined methods are reused to deal with the view.

  Widget _displayImages() {
    if (images.length == 1) {
      return _container(images[0]);
    }

    if (images.length == 2) {
      return _row(images);
    }

    if (images.length == 3) {
      return Row(
        spacing: 2,
        children: [
          Expanded(child: _container(images[0])),
          Expanded(child: _column([images[1], images[2]])),
        ],
      );
    }

    if (images.length == 4) {
      return Row(
        spacing: 2,
        children: [
          Expanded(child: _column([images[0], images[1]])),
          Expanded(child: _column([images[2], images[3]])),
        ],
      );
    }

    if (images.length == 5) {
      return Row(
        spacing: 2,
        children: [
          Expanded(child: _column([images[0], images[1]])),
          Expanded(child: _column([images[2], images[3], images[4]])),
        ],
      );
    }

    if (images.length > 5) {
      return Row(
        spacing: 2,
        children: [
          Expanded(child: _column([images[0], images[1]])),
          Expanded(child: _column([images[2], images[3], images[4]])),
        ],
      );
    }

    return Container(child: Text("nothing to see there..."));
  }

Finally, the build() method returns a SizedBox widget with the return of _displayImages() method as child. The height if here is quite important and will be used to define the final size of the images displayed. Without it, nothing will be printed and an error will be generated. I did not find a good way right now to avoid that.

  @override
  Widget build(BuildContext context) {
    return SizedBox(
      height: 300,
      child: _displayImages()
    );
  }
}

A quick note on Flutter's assets, I downloaded some random cat images from unsplash and put them in assets/images. After that, the pubspec.yaml file must be updated, like mentioned in Flutter documentation.

# ...
flutter:
  # ...
  assets:
    - assets/images/loan-7AIDE8PrvA0-unsplash.jpg
    - assets/images/manja-vitolic-gKXKBY-C-Dk-unsplash.jpg
    - assets/images/mikhail-vasilyev-IFxjDdqK_0U-unsplash.jpg
    - assets/images/kate-stone-matheson-uy5t-CJuIK4-unsplash.jpg
    - assets/images/amber-kipp-75715CVEJhI-unsplash.jpg

Then, those assets can be used inside the code, let check that with our AttachmentView class.

AttachmentView(
  images: [
    Image.asset(
      "assets/images/loan-7AIDE8PrvA0-unsplash.jpg",
    ),
    Image.asset(
      "assets/images/manja-vitolic-gKXKBY-C-Dk-unsplash.jpg",
    ),
    Image.asset(
      "assets/images/mikhail-vasilyev-IFxjDdqK_0U-unsplash.jpg",
    ),
    Image.asset(
      "assets/images/kate-stone-matheson-uy5t-CJuIK4-unsplash.jpg",
    ),
    Image.asset("assets/images/amber-kipp-75715CVEJhI-unsplash.jpg"),
  ],
)

Below the result of this snippet.

The behavior is a bit different than SignalApp, when configured 5 images, they are using a Column and not a Row as main widget, resulting in the 2 top images being the biggest ones.

Messages

Warning: dirty code. It was not expected to take so much time due to the complexity of the different layer. I offer only a partial solution there, a better implementation will be created in another post.

The remote contact can send a message containing only text, emoji or sticker. It can be created using a Card Widget. The following code is quite complex because I tried to include all the features in one class. The design is dirty, but it "works". The main issue I got is the different parts of a message that can be modified due to the differences between a message received or sent.

  Message({
    super.key,
    final Alignment alignment = Alignment.centerLeft,
    final Color color = Colors.blue,
    final Color textColor = Colors.white,
    final String? text,
    final String? date,
    final String? label,
    final bool received = false,
    final List<Image> attachments = const [],
  }) : this.alignment = alignment,
       this.color = color,
       this.textColor = textColor,
       this.text = text,
       this.date = date,
       this.label = label,
       this.received = received,
       this.attachments = attachments;

  Widget _text() {
    return text == null
        ? Container()
        : Text(
            text ?? "null",
            style: TextStyle(
              color: alignment == Alignment.centerLeft
                  ? textColor
                  : Colors.black,
              fontWeight: FontWeight.w500,
            ),
          );
  }

  Widget _date() {
    return date == null
        ? Container()
        : Text(
            date ?? "??",
            style: TextStyle(
              color: alignment == Alignment.centerLeft
                  ? textColor
                  : Colors.black,
              fontWeight: FontWeight.w100,
              fontSize: 10,
            ),
          );
  }

  Widget _attachment() {
    return attachments.isEmpty
        ? Container()
        : Row(
            children: [Expanded(child: AttachmentView(images: attachments))],
          );
  }

  Widget _label() {
    return Container(
      padding: EdgeInsets.all(1),
      margin: EdgeInsets.all(1),
      child: label == null
          ? Container()
          : Text(label!, style: TextStyle(fontSize: 14)),
    );
  }

  @override
  Widget build(BuildContext context) {
    return Padding(
      padding: EdgeInsetsGeometry.only(
        right: alignment == Alignment.centerLeft ? 40 : 10,
        left: alignment == Alignment.centerLeft ? 10 : 40,
      ),
      child: Badge(
        backgroundColor: Colors.grey[400],
        offset: alignment == Alignment.centerLeft
            ? Offset(-40, -20)
            : Offset(20, -20),
        alignment: alignment == Alignment.centerLeft
            ? AlignmentGeometry.bottomEnd
            : AlignmentGeometry.bottomStart,
        isLabelVisible: label == null ? false : true,
        label: _label(),
        child: Container(
          clipBehavior: Clip.antiAlias,
          margin: EdgeInsets.all(10),
          decoration: BoxDecoration(
            color: Colors.transparent,
            borderRadius: BorderRadius.circular(12),
          ),
          child: Column(
            children: [
              _attachment(),
              text == null
                  ? Container()
                  : Container(
                      // margin: EdgeInsets.all(10),
                      padding: EdgeInsets.all(10),
                      alignment: alignment,
                      decoration: BoxDecoration(
                        color: alignment == Alignment.centerLeft
                            ? color
                            : Colors.grey[300],
                        borderRadius: BorderRadius.only(
                          topLeft: attachments.isEmpty
                              ? Radius.circular(12)
                              : Radius.zero,
                          topRight: attachments.isEmpty
                              ? Radius.circular(12)
                              : Radius.zero,
                          bottomLeft: Radius.circular(12),
                          bottomRight: Radius.circular(12),
                        ),
                      ),
                      child: Column(
                        spacing: 5,
                        children: [
                          Align(alignment: Alignment.topLeft, child: _text()),
                          Align(
                            alignment: alignment == Alignment.centerLeft
                                ? Alignment.bottomLeft
                                : Alignment.bottomRight,
                            child: _date(),
                          ),
                        ],
                      ),
                    ),
            ],
          ),
        ),
      ),
    );
  }

I waste a lot of time also on the shape of the different objects, until I discovered the Clip.hardEdge and Clip.antiAlias. Indeed, when I was adding attachments, I was losing the rounded borders, it was quickly fixed by enabling those parameters in the parent containers.

The same can be done for an User's message. Unfortunately, as you can see, the same message can contain different kind of data (1) an emoji (2) text only (3) a small image called sticker (see Bandit the Cat) if I remember correctly. This will not be implemented though.

User Chat Input

Finally, the last part of this screen is the place where the user can write messages. It is split in 2 parts, the left side is to let the user write a message via a text input, this one is also made of 3 buttons, one on the left to add emojis, and 2 on the right, to insert a picture or record a vocal messages. The last button on the left is used to send the message to the contact. The background color is grey (#fefefe).

The user's input can be divided in 3 parts, the leading part containing an IconButton with an emoji. The body part containing a TextField. The actions part containing more IconButton.

The final button of this part of the application is the one used to send the message.

class ChatBox extends StatelessWidget {
  const ChatBox({super.key});

  @override
  Widget build(BuildContext context) {
    return Padding(
      padding: EdgeInsetsGeometry.all(10),
      child: Row(
        crossAxisAlignment: CrossAxisAlignment.end,
        mainAxisSize: MainAxisSize.max,
        mainAxisAlignment: MainAxisAlignment.end,
        spacing: 10,
        children: [
          Expanded(child: ChatInput()),
          SendButton(),
        ],
      ),
    );
  }
}

class SendButton extends StatelessWidget {
  const SendButton({super.key});

  @override
  Widget build(BuildContext context) {
    return Align(
      alignment: Alignment.bottomCenter,
      child: Container(
        margin: EdgeInsets.only(bottom: 14),
        alignment: Alignment.bottomCenter,
        decoration: BoxDecoration(
          color: Colors.blue,
          borderRadius: BorderRadius.circular(100),
        ),
        child: IconButton(
          onPressed: () {},
          icon: Icon(Icons.add),
          color: Colors.white,
        ),
      ),
    );
  }
}

class ChatInput extends StatelessWidget {
  const ChatInput({super.key});

  @override
  Widget build(BuildContext context) {
    return Container(
      alignment: Alignment.bottomCenter,
      margin: EdgeInsetsGeometry.only(bottom: 10),
      decoration: BoxDecoration(
        color: Colors.grey[200],
        borderRadius: BorderRadius.circular(20),
      ),
      child: Align(
        alignment: Alignment.bottomCenter,
        child: Padding(
          padding: EdgeInsetsGeometry.only(left: 10, right: 10),
          child: Row(
            crossAxisAlignment: CrossAxisAlignment.end,
            mainAxisSize: MainAxisSize.max,
            mainAxisAlignment: MainAxisAlignment.end,
            spacing: 10,
            children: [
              EmojiButton(),
              Expanded(child: ChatInputText()),
              ActionButtons(),
            ],
          ),
        ),
      ),
    );
  }
}

class ChatInputText extends StatelessWidget {
  const ChatInputText({super.key});

  @override
  Widget build(BuildContext context) {
    return Container(
      child: TextField(
        onTap: () {
          print("tapped");
        },
        onChanged: (str) {
          print("$str");
        },
        minLines: 1,
        maxLines: 6,
        decoration: InputDecoration(
          border: InputBorder.none,
          hintText: "Signal message",
          contentPadding: EdgeInsets.all(10),
        ),
      ),
    );
  }
}

class EmojiButton extends StatelessWidget {
  const EmojiButton({super.key});

  @override
  Widget build(BuildContext context) {
    return Container(
      margin: EdgeInsets.only(bottom: 4),
      decoration: BoxDecoration(
        borderRadius: BorderRadius.circular(100),
      ),
      alignment: Alignment.bottomCenter,
      child: IconButton(
        onPressed: () {},
        icon: Icon(Icons.emoji_emotions_outlined),
      ),
    );
  }
}

class ActionButtons extends StatelessWidget {
  const ActionButtons({super.key});

  @override
  Widget build(BuildContext context) {
    return Container(
      margin: EdgeInsets.only(bottom: 4),
      alignment: Alignment.topCenter,
      child: Row(
        spacing: 2,
        children: [
          Container(
            child: IconButton(
              onPressed: () {},
              icon: Icon(Icons.camera_outlined),
            ),
          ),
          Container(
            // decoration: BoxDecoration(borderRadius: BorderRadius.circular(100)),
            child: IconButton(onPressed: () {}, icon: Icon(Icons.mic_outlined)),
          ),
        ],
      ),
    );
  }
}

Here a screenshot of a ChatBox object on Android:

Final Result

Design can be art. Design can be aesthetics. Design is so simple, that's why it is so complicated.
-- Paul Rand

Learning design is a long journey but the pieces are starting to fit together. The final result is not perfect, but it will reflect my current level. All widgets previously created can be mixed together.

The ChatBody class will create a ListView to deal with all the messages. Actually, those messages are created manually, including all the assets, the label and the raw text. In a real world application, those messages are fetched from a server, a local database or a cache.

class ChatBody extends StatelessWidget {
  const ChatBody({super.key});

  @override
  Widget build(BuildContext context) {
    return ListView(
      children: [
        Message(
          label: "❤️",
          alignment: Alignment.centerRight,
          attachments: [
            Image.asset(
              "assets/images/loan-7AIDE8PrvA0-unsplash.jpg",
              // fit: BoxFit.cover,
            ),
            Image.asset(
              "assets/images/manja-vitolic-gKXKBY-C-Dk-unsplash.jpg",
              // fit: BoxFit.fill,
            ),
            Image.asset(
              "assets/images/mikhail-vasilyev-IFxjDdqK_0U-unsplash.jpg",
              // fit: BoxFit.fill,
            ),
          ],
        ),
        Message(
          text:
              "The rain is pouring down and I'm sitting here just listening to it",
          label: "👍",
          date: "30m",
        ),
        // Message(alignment: Alignment.centerRight, text: "☺️"),
        Message(
          alignment: Alignment.centerRight,
          text: "That's what I did this morning too.",
        ),
        Message(
          text:
              "The rain is pouring down and I'm sitting here just listening to it",
          date: "30m",
          attachments: [
            Image.asset(
              "assets/images/loan-7AIDE8PrvA0-unsplash.jpg",
              // fit: BoxFit.cover,
            ),
            Image.asset(
              "assets/images/manja-vitolic-gKXKBY-C-Dk-unsplash.jpg",
              // fit: BoxFit.fill,
            ),
            Image.asset(
              "assets/images/mikhail-vasilyev-IFxjDdqK_0U-unsplash.jpg",
              // fit: BoxFit.fill,
            ),
            Image.asset(
              "assets/images/kate-stone-matheson-uy5t-CJuIK4-unsplash.jpg",
            ),
            Image.asset("assets/images/amber-kipp-75715CVEJhI-unsplash.jpg"),
          ],
        ),
      ],
    );
  }
}

Then, the ChatScreen will contain the ChatBody widget (containing all messages) and the ChatBox (containing the input). A Scaffold and a Column widgets are being used there.

class ChatScreen extends StatelessWidget {
  const ChatScreen({super.key});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: ChatBar().build(context),
      body: Column(
        children: [
          Expanded(child: ChatBody()),
          ChatBox(),
        ],
      ),
    );
  }
}

The main widget called MyApp can be created, it will just be used to manage the ChatScreen object.

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'SignalCopyCat',
      home: ChatScreen(),
      theme: ThemeData(fontFamily: "Roboto"),
      themeMode: ThemeMode.light,
      debugShowCheckedModeBanner: false,
    );
  }
}

Finally, the application can be started from the main entry-point.

void main() {
  runApp(const MyApp());
}

The final result can be seen in the following screenshot.

Lot of features are missing, but it could be a good challenge to try to reproduce that in 6 months or 1 year, to see if I can produce better code.

Conclusion

Only God creates. The rest of us just copy.
-- Michelangelo

When I started this post, I thought it will be done in few hours, perhaps few days. It was not the case. The level of complexity to design a "simple" is crazy. I know, I'm no expert in Flutter yet, but based on the amount of opened tabs in my web browser, I can be sure it was not an easy task.

In fact, it's not a surprise. SignalApp is quite old now, and the design evolved alongside new features. Reproducing everything in one long coding session is then complex, even if the Java/Kotlin source code of the application is available on Github. Just to be clear, this article does not include state management, animations and interactivity.

One question we can ask thought... Many projects are using the same patterns to represent messages, Flutter is a kind of "low level" design framework, then why not creating a project reproducing the same design from Signal, WhatsApp and so on? It could be great to mix great ideas together.

As usual, if you want to know a bit more about this publication, here few resources:

Flutter Learning resources from the Official Flutter website;
Create a grid list from the Official Flutter documentation;
WhatsUp Flutter Appplication on Github;
Chat Screen from the WhatsUp Application on Github;
Chat from the WhatsUp Application on Github;
Chat Field from the WhatsUp Application on Github;
Attachment Renderers from the WhatsUp Application on Github;
WhatsApp Flutter Application on Github;
Flutter Row and Column: MainAxis, CrossAxis, and Spacing Explained by Sagnik Bhattacharya;
SignalApp UI source code on Github;
SignalApp MMS (Multimedia Messages?) source code on Github;
SignalApp Album Thumbnail View on Github;
The equivalent of wrap_content and match_parent in flutter?
on StackOverflow;
How to make one side circular border of widget with flutter? on StackOverflow;
How to change height of a card in flutter on StackOverflow;
How do I stretch an image to fit the whole background (100% height x 100% width) in Flutter? on StackOverflow;
Why isn't my image filling the space with BoxFit.fill in Flutter? on StackOverflow;
Flutter BoxFit.cover - Show top of image, not center on StackOverflow;
Flutter images in a row on StackOverflow;
How do I crop an image in Flutter? on StackOverflow;
border radius not apply inside container widget on StackOverflow.

That's a lot of links, right? Well, my web browser was full of tabs... I just extracted few of them, the ones I found the most interesting.

Have fun and Hack the planet!

Cover Image by Kelly Sikkema on Unsplash

QubeOS Tips and Tricks

Mathieu Kerjouan — Thu, 09 Jul 2026 10:00:00 +0000

Foreword: these notes are coming from a draft. I was planning to create an article on QubeOS for a secure business oriented environment. Unfortunately, because of the complex upgrade procedure, I thought it was not the right system for doing business. The following sections have been taken from the FAQ part.

QubeOS was my professional operating system for 2 years. I did not have great experience. In fact, it works, but when it comes to update/upgrade, it's painful. I still own one laptop running with QubeOS for test purpose only and also to see if the upgrade process is evolving in the right way... It's not the case.

Anyway, here few tips and tricks I used in the past.

Copying files between dom0 to domU

A file can be copied from dom0 to domU using the qvm-copy-to-vm command.

qvm-copy-to-vm

Copying files between domU

A file can be copied from domU to domU using the qvm-copy command.

qvm-copy

Executing Commands in Minimal Qubes as root

In minimal template, only user user is created, without any kind of right. To install/upgrade software or configure sudo you will need to have shell with root access. Here an example:

qube=debian-11-minimal
qvm-run --user=root ${qube} xterm
# or with short flags -u
# qvm-run -u root ${qube} xterm

Xen Store Dom0 view

Xen Store is a shared environment containing meta-data and other cool stuff. This store can be managed with the xenstore-* commands.

xenstore-ls
xenstore-read
xenstore-write

Xen Store DomU view

Xen Store is not only available from dom0 but also from domU, even if the view can change a bit.

# return local vm information based on uuid
xenstore-ls /vm/${uuid}

# return global domain information
xenstore-ls /local/domain/${domain}

# return vbd information
xenstore-ls /local/domain/${vbd}/backend/vif/${domain}/0

# information can be read/write using:
xenstore-read
xenstore-write

Qubesdb

Qubesdb is an interface to xenstore, it gives you all required information by using only / key without looking for other keys/values in the store. You can get ip address, netmask or gateway.

qubesdb-list /
qubesdb-read /type
qubesdb-read /qubes-ip
qubesdb-read /qubes-netmask

Restarting the Network

Network interfaces are isolated from other qubes in dedicated virtual machine called sys-net. Sometime, wireless interfaces can be stuck due to NetworkManager application. In this case, you can run this command to restart the service from dom0:

qvm-run -u root sys-net "systemctl restart NetworkManager"

This will also restart XFCE widgets and should solve simple network issues.

Local Network Script with rc.local

You can set a local network configuration using /rw/rc.local. This script will be executed at startup ONLY for your current qubes.

# set network stack
ip set link dev eth0 up
qubesdb-read /qubes-ip \
  | xargs -I%d ip address add %d/32 dev eth0
qubesdb-read /qubes-gateway\
  | xargs -I%g \
sh -c 'ip route add %g dev eth0 scope link; \
  ip route add default via %g'

Global Network Script

ou can also enable /etc/rc.local with systemd. Take the previous script and enable it following this procedure:

chmod +x /etc/rc.local
systemctl daemon-reload
systemctl start rc-local
systemctl enable rc-local

Creating Ubuntu Template

Note: you will need to execute these commands in a qube set with fedora.

# install required dependencies
dnf install gnupg git createrepo rpm-build make wget rpmdevtools python3-sh dialog rpm-sign dpkg-dev debootstrap python3-pyyaml devscripts perl-Digest-MD5 perl-Digest-SHA

# clone qubes-builder repository from QubeOS official repository
git clone https://github.com/QubesOS/qubes-builder
cd qubes-builder

# install dependencies
./setup
make install-deps
make get-sources
make qubes-vm
make template

You will need to edit few of the generated packages due to missing dependencies. In my case, qubes-src/meta-packages/debian/control required to be edited and some dependencies needed to be removed.

It is now possible to install this template on dom0.

qubes-src/meta-packages/debian/control

Starting a disposable VM from standard VM

A disposable VM can be started on a standard Qube with the help of the qvm-run-vm command.

qvm-run-vm --disp ${disp_vm} xterm

Starting a disposable VM from Dom0

A disposable VM can be started from Dom0 using the qvm-run command.

qvm-run --disp ${disp_vm} xterm

Conclusion

These commands have been collected during a small period of times. I'm not an active QubeOS user anymore, OpenBSD is still and will remain my favorite Operating System.

Cover Image by shraga kopstein on Unsplash

Working with Elm

Mathieu Kerjouan — Wed, 08 Jul 2026 10:00:00 +0000

Foreword: this article is from my attic. It was planned to be published in 2024, it's probably a bit old... But I don't think Elm and Phoenix framework changed a lot in 2 years.

I usually use OpenBSD when programming in Erlang, C or Elixir. Unfortunately, doing front-end development with node for example or even doing some back-end stuff in Rust can lead to a disaster. Indeed, OpenBSD is a great piece of software but not everything is working by default, and you will keep hacking around. Hacking is always a good way to learn, but, you don't always have time for doing that. When developing front-end application, but also application for iOS or Android, you should use something highly compatible and with not a lot of bad surprises.

Introduction

Front-end programming is something a bit weird. In fact, front-end development is present in my long list of IT nightmare, alongside with Object Oriented Programming (in particular Java), systemd, blobs and other highly complex framework you can find in system administration.

Anyway, when it comes to design a web-site, it costs me time, to create the Javascript part and the CSS. Because the web is evolving fast -- perhaps too fast -- and because I'm not a front-end developer, my toolbox stayed in the past. How to find stability in this environment? Well, it seems Elm is quite stable. I followed this project since 2016 or 2017 without seeing critical changes in the syntax, that's a very good point. The standard library and external modules are looking stable as well. The only alternative I know that could help to build front-end is clojure-script.

My own projects led me to build web applications and I don't have time to maintain them. Dealing with stability and strict programming rules is a game changer. I don't know if a framework using the same principles than Erlang exists in Front-end world though. Anyway, let integrates Elm in an Elixir stack with Phoenix framework.

Phoenix Project from Scratch

If you already have used Phoenix Framework in the past, no secret there. I assume you already have mix available on your system. You can use any solution for that, like asdf or debian packages.

# install hex locally
mix local.hex --if-missing

# install phx_new templating module
mix archive.install  hex phx_new

# create a new project without assets,
# we will add them manually later
mix phx.new project --no-assets
cd project

Elm Project from Scratch

Elm can be installed using npm or directly by downloading on the official website. We will first download it to create the project and then move to npm solution during the integration step.

mkdir -p assets/elm
cd assets/elm

To init a project, elm init command needs to be executed. It will generate a elm.json file and a src folder.

elm init

Based on the different projects I watched before doing that, it seems the convention is to create a src/Main.elm file containing the entry-point.

touch src/Main.elm

This file will contain our dummy application for the moment.

module Main exposing (..)

import Browser
import Css exposing (..)
import Html exposing (..)
import Html.Attributes exposing (..)
import Html.Events exposing (onClick, onInput)

-- This is our main function, our entry-point.
main =
  Browser.element
    { init = initialModel
    , view = view
    , subscriptions = \_ Sub.none
    , update = update
    }

-- Our model type (our state) will be empty
type alias Model = ()

-- this is our state function
initialModel : () -> ( Model, Cmd Msg )
initialModel _ =
  ( ()
  , Cmd.none
  )

-- this is our update (loop?) function, where all events
-- are received
update msg model =
  model

-- this is our view function, where all the html/css black
-- magic is created and rendered
view model =
  div [] []

Note: Don't forget to install elm-mode in emacs!

Esbuild and NPM Bootstrapping

npm install command can be executing in assets/elm, it will produce a generic package.json file. This one can be modified to fit our needs. elm, esbuild and esbuild-plugin-elm modules are required. Two scripts are also created, one to build the project in one time fashion (build), and the other one watching for modification (watch).

{
    "name": "project",
    "version": "0.1.0",
    "main": "src/index.js",
    "license": "MIT",
    "dependencies": {
        "elm": "^0.19.1-6",
        "esbuild": "0.17.11",
        "esbuild-plugin-elm": "^0.0.12"
    },
    "devDependencies": {
        "elm": "^0.19.1-6",
        "esbuild": "0.19.11",
        "esbuild-plugin-elm": "^0.0.12"
    },
    "scripts": {
        "build": "node ./build.js",
        "watch": "node ./build.js --watch"
    }
}

Phoenix supported esbuild for a long time now, but integrating it with other language like Elm is a bit hard. That's why we will use esbuild API in a small script, instead of hacking Elixir esbuild module. This snippet is not from me, but from esbuild-plugin-elm and Ben Koshy's blog.

const path = require('path')
const esbuild = require('esbuild');
const ElmPlugin = require('esbuild-plugin-elm');

esbuild.build({
    entryPoints: [
        "./src/index.js"
    ],
    bundle: true,
    // outfile: "app.js",
    target: "es2017",
    outdir: "../../priv/static/assets",
    minify: false,
    plugins: [
        ElmPlugin({
            debug: true,
            verbose: true,
        }),
    ],
}).catch(_e => process.exit(1))

The last bootstrap is to create a script to link Elm modules in the bundle produced by esbuild. This scrip

import { Elm } from './Main.elm';

// look for an element with the id application set
// and use it as mount point for our Elm application.
var $root = document.getElementById('application');
Elm.Main.init({
  node: $root
});

Bootstrapping Phoenix Framework

defmodule Project.MixProject do
  use Mix.Project

    # ...

    defp aliases do
    [
      # ...
      "assets.setup": [
        "esbuild.install --if-missing",
        "cmd --cd assets/elm npm install"],
      "assets.build": [
        "cmd --cd assets/elm node build.js"
      ],
      "assets.deploy": [
        "cmd --cd assets/elm node build.js --deploy",
        "phx.digest"
      ]
    ]
  end
end

# install esbuild and install node dependencies
mix assets.setup

# build assets
mix assets.build

# build assets and create digest
mix assets.deploy

We should have now a file index.js in priv/assets directory, ready to be injected in our HTML, just add the following code at the end of your page.

<script src={~p"/assets/index.js"}></script>

Development Process

Create your Elm application
Run mix assets.build, your browser should reboot automatically if changes have been made
Modify your Phoenix controllers or views.
Don't forget to create or update .gitignore file, you don't want to deal with all assets produced in your repository.

Configuring Backend Hostname

Elm accept flags feature, that means when your application will start it initialization phase, some external arguments will be passed to the entry-point. When starting Phoenix server, this one will usually start to listen on TCP/4000 port. This value can be set as string in flag (or create a JSON object containing all value you want to put there). Based on the previous configuration and bootstrap we did, flags must be set in assets/elm/src/index.js.

import { Elm } from './Main.elm';

var $root = document.getElementById('application');
Elm.Main.init({
    node: $root,
    flags: "http://localhost:4000/"
});

Then, the arguments passed to the entry-point function in the application must also be modified. Here an example when sending a string.

init : String -> ( Model, Cmd Msg)
init target_ = 
  ( { target = target_ }, Cmd.none )

If an URL is passed, it could probably better to use a JSON object, decoded with Json.Decode, decomposed it, to finally create a new clean and safe Url.

Managing Url with `Browser.element`

Bernardoow already created an interesting gist on this topic.

Conclusion

It's not so "hard" to use Elm with Phoenix Framework. In fact, except some issue with Esbuild configuration and some odd behaviors with npm, it was just working fine. Many other developers did also the same process - or a similar one.

References and Resources

Awesome Elm: A curated list of useful Elm tutorials, libraries and software. Inspired by awesome list. Feel free to contribute.

The Structure of an Elm Application

How to Use Elm at Work

JavaScript Interop

Elm, Elixir, and Phoenix: Reflecting on a Functional Full-Stack Project

phoenix-elm-starter

Elm Destructuring (or Pattern Matching) cheatsheet

Cover Image by Lyrax on Unsplash

Chored/0: an alternative task manager

Mathieu Kerjouan — Tue, 07 Jul 2026 10:00:00 +0000

The answer is out there, Neo, and it's looking for you, and it will find you if you want it to.
The Matrix, Trinity

Dealing with production - and personal - servers is an exhausting task, even with automation tools and scripts. Many projects came to my hands to manage infrastructure, Ansible, SaltStack, Capistrano, Fabric, ReX, Nomad or Kubernetes. Most of them are great but they are not the answer I was looking for. A server should act like a finite state machine, reacting to some kind of events. If you are working on Unix/Linux systems, daemons should not be the only way to do concurrent/parallel computations in background. In fact, a daemon in charge of executing different kind of tasks without scheduling, and being able to reuse the layers of secure configured on the system.

Unfortunately, after 20 years of experiences I did not find this pearl. One of the closed to this idea was Nomad (autonomous application), then SaltStack (remote agent) and finally Ansible (simplicity). This publication is the first one of a long series about a pet project called chored, an open-source software to deal with dynamic long living process written in pure C and following all OpenBSD best-practices. It can be seen as a cron like application on steroïd, with an optional scheduling part.

Introduction

How to deal with a remote (long living) task? Well, lot of solutions are currently existing, but those not fit some of my own requirements:

Configuring a service managed by the init process like systemd service, runit, systemv script, rcctl on OpenBSD or sysrc on FreeBSD. If an user does not have enough privilege to create this configuration, supervisord can be used.
On some system, especially on Linux distributions, one can use daemon command.
Using an application simulating a tty like screen, tmux, tmate or dtach
Using an application starting automatically a job at a specific time like at and cron
Using an existing agent like the one present in SaltStack called salt-minion.

Another kind of software can deal with long living task: init managers, including runit, sys-v init, supervisord or systemd. supervisord is particularly interesting in this case due its versatility; but this is coded in Python. Even if Python is a great C abstraction, it's still a huge pile of dependencies.

If one wants to run a long living task on a remote server, one needs to have an agent or a way to control the task without his help. The first listed programs have been created to start long living shell session, but they are not offering a low level interface with an API to deal with spawned state.

Then, why not creating a small program to deal with long running task, designed for low memory and embedded systems? Following POSIX and Unix principles should not be so hard, and one can easily creates something portable enough to be executed anywhere. Even more, why not creating an interface to ansible or other automation tool to deal with long living processes?

Constraints and Requirements

This project is written for OpenBSD and other BSD systems, but will be ported to Linux after. One of the goal is to offer a really stable and secure solution before doing an official release. Why OpenBSD? Because the OpenBSD source code is pure high quality code.

Design

When coding in C, Unix philosophy is clear: do one thing and do it well. One can also add the KISS principle (Keep it Stupid, Simple). chored should be small and simple, not an easy task. chored can be started as a service with an isolated users, or directly by any user of the system.

Usage

How to use chored? I like to start thinking of the interface to control a program first, just to have an idea how to design it and which features will be critical.

starting chored. chored is the main daemon, it can be started in two different ways; (1) as root with privilege separation, a bit like sshd (2) as standard user with the same privileges.

# start chored as privilege daemon
# it will use /etc/chored.conf and
# try to find the user _chored and the
# group _chored for the privilege
# separation, but it also means it will
# be able to run any kind of program
# from any user/group present on the
# system
doas chored

# start chored as unprivileged user. This
# time it will look in ~/.chored.conf
# to load the configuration. It can eventually
# invoke command from another user only
# via doas or sudo.
chored

# start in debug/foreground mode;
# chored will not run in the backgroud
chored -d

configuring chored. The configuration must be easy to understand, easy to modify and easy to parse. Using something similar to the OpenBSD httpd daemon or opensmtpd could do the job.

# create the privilege chored configuration
cat > /etc/chored.conf << EOF
# set parameter_name my_value
set user _chored;
set group _chored;
set chroot /;

# job job_name {}
job job_name {
  command /usr/game/pom;
  user _chored;
  group _chored;
  chroot /;
  ulimit "";
  umask "";
  pledge stdio;
  unveil /usr/lib:r;
}

EOF

creating a new job. A job creation should be easy to do.

# create a new job called job_name
# this job is not started, it is simply
# stored in memory, waiting to be invoked
# the command returns the name of the job
# and its state
$ chore create job_name -- command
job_name ready
$ echo $?
0

executing a job. A job can be executed just after the creation or started when an external events happen.

# executing an existing job
# it will return a random UUID
$ chore execute job_name
job_name ${pid} 4fd67c41-fd21-40db-8778-b03d9842f93d

monitoring a job execution. When a long living job is running, it can be nice to have few information on it, like its memory or cpu usage, the last logs and so on.

$ cat /var/chored/jobs/job_name.conf
...

$ tail /var/chored/jobs/job_name/4fd67c41-fd21-40db-8778-b03d9842f93d/stdout
...

$ tail /var/chored/jobs/job_name/4fd67c41-fd21-40db-8778-b03d9842f93d/stderr
...

$ echo 1 | tee /var/chored/jobs/job_name/4fd67c41-fd21-40db-8778-b03d9842f93d/stdin
...

$ cat /var/chored/jobs/job_name/4fd67c41-fd21-40db-8778-b03d9842f93d/ret
...

checking a job. In most situation, we don't really care about monitoring, a sysadmin just wants to know the state of the process, if it's still running, if it's stopped or crashed.

$ chore status 4fd67c41-fd21-40db-8778-b03d9842f93d

stopping a job. A long living job can also be stopped manually, via a signal or from any other methods defined by chored.

$ chore stop 4fd67c41-fd21-40db-8778-b03d9842f93d

killing a job. If the job is stuck for some reason (e.g. hardware issue, bad designed software), a sysadmin must be able to kill it.

$ chore kill 4fd67c41-fd21-40db-8778-b03d9842f93d

remove a job. Finally, a job definition can also be removed from chored memory to avoid starting it by mistake.

$ chore remove job_name

Ideas

job template: when a dynamic job is created it can automatically load a job template configuration containing extra parameters.

template job_template {
  chroot /home/my_user;
  user my_user;
  group my_group;
  # other custom parameters ...
}

post action: when the job is done, another script/program with the job name, the job UUID and the return code as argument

Conclusion

This is a pet project, I would like to use it to fix one time for all some part of my missing knowledge in Unix/Posix system design and C programming. The project is called chore on Github. This is not something ready, yet, but you have now an idea of the goal. No dead-line though, I will take my time!

Have fun!

Cover Image by Chloe Skinner on Unsplash

Persistent Data Storage with sembast

Mathieu Kerjouan — Mon, 06 Jul 2026 10:00:00 +0000

As a general rule, the most successful man in life is the man who has the best information.
-- Benjamin Disraeli

It's pretty rare to find an application without state, in fact, I don't even think it's possible to find that except for prototype or demos. My future application(s) will require a way to store temporarily data locally (e.g. cache), and it could be nice to have something flexible to do that. In Erlang/Elixir, in-memory cache can be created with ETS, for persistent state, DETS can also be used. More recently, a rocksdb backend for Mnesia has been created and seems stable enough to be used by blockchains like aeternity.

Anyway, we are not writing Erlang code there, it's about Dart, and we have a lot of choice from pub.dev, here a quick list of the popular ones:

sembast: NoSQL persistent embedded file system document-based database for Dart VM and Flutter with encryption support;
objectbox: Flutter database for super-fast NoSQL ACID compliant object persistence;
tostore: Fast distributed AI vector database and persistent local storage engine. High-performance key-value store supporting SQL, NoSQL, offline cache and encrypted data;
localstore: A JSON file-based storage package provides a persistent repository for simple NoSQL database;
isar: Extremely fast, easy to use, and fully async NoSQL database for Flutter;
mimir: Extremely powerful, reactive NoSQL database with typo-tolerant full-text search and declarative queries;
stash: a key-value store abstraction with plain and cache driven semantics and support for a pluggable backend architecture;
orange: super fast and esay modern key-value store. working everywhere.and no need build_runner;
hive: Lightweight and blazing fast key-value database written in pure Dart. Strongly encrypted using AES-256.

All of them look great, but for this first article, we will test sembast. The source code is available on Github, and the API Documentation can be found on pub.dev. More examples and documentation can also be found in the doc directory from the repository.

Bootstrapping

As usual, we will create a small sandbox project, called sb this time.

$ dart create sb
Creating sb using template console...

  .gitignore                         
  analysis_options.yaml
  CHANGELOG.md                             
  pubspec.yaml
  README.md    
  bin/sb.dart                                                                                  
  lib/sb.dart                                                                                  
  test/sb_test.dart                          

Running pub get...                     4.1s                                                                                                                                                   
  Resolving dependencies...                                                                    
  Downloading packages...
  Changed 48 dependencies!

Created project sb in sb! In order to get started, run the following commands:

  cd sb                                                                                        
  dart run                                                                                     

$ cd sb

$ dart pub add sembast
Resolving dependencies... 
Downloading packages... 
+ sembast 3.8.9+1
+ synchronized 3.4.1
Changed 2 dependencies!

Persistent Local Store

The NICS database has holes big enough to drive a Mack truck through.
-- John Kennedy

A sembast database needs to be initialized like any other database. sembast.dart and sembast_io.dart are required. The first one will offer the methods to deal with the database, the second one will give access to

import 'package:sembast/sembast.dart';
import 'package:sembast/sembast_io.dart' as sio;

To help us for the initialization part, we can create a dedicated function called initDb(). sembast can support different kind of method to store the database, in this example, we will use an IO database stored on the filesystem with the help of databaseFactoryIo method. It returns a new DatabaseFactory() object we can directly use to open the database via the openDatabase() method.

Future<Database> initDb({dbPath = "./sample.db"}) async {
  String dbPath = './sample.db';
  DatabaseFactory dbFactory = sio.databaseFactoryIo;
  return await dbFactory.openDatabase(dbPath);
}

This function can be called from the main() entry-point. At the end of the execution, the database can be closed by calling the close() method.

void main(List<String> arguments) {
  Database db = await initDb();
  await db.close();
}

Finally, the application can be invoked with dart run.

$ dart run
Building package executable... 
Built sb:sb.

After the execution, you should find a sample.db file in your project directory. This is the sembast database. As you can see, it contains JSON objects.

$ file sample.db 
sample.db: JSON text data

$ cat sample.db 
{"version":1,"sembast":1}

Before opening a database, some checks must be applied, especially if the local database is persistent. Indeed, if a database is already existing, opening it with sembast will drop its content. To avoid this issue, a DatabaseMode can be configured when calling DatabaseFactory.openDatabase() method by setting the mode parameter with the database mode of your choice. As reminder, 6 modes are currently available:

Furthermore, before initializing or opening a database, one should check its existence. It can be achieved via the DatabaseFactory.databaseExists() method. If the database is already present on the filesystem, opening it without removing the data already stored there can be a good choice for a persistent data-store.

In-Memory Store

Database: the information you lose when your memory crashes.
-- Dave Barry

sembast is flexible, and many different backend can used. For example, if you have to deal with temporary data,

import 'package:sembast/sembast_memory.dart' as smem;

Future<Database> initMemDb({dbId = "main"}) async {
  DatabaseFactory dbFactory = smem.databaseFactoryMemory;
  return await dbFactory.openDatabase(dbId);
}

https://pub.dev/documentation/sembast/latest/sembast_memory/

Data Store Usage

If you want a database, you don't go out and say you're going to write it. I see platforms as going in that direction.
-- Parker Harris

Now we know how to create a database with sembast, we can use it. Any data added, updated or removed from the database needs a StoreRef() object, one object from this class can be instantiated using with the StoreRef.main() constructor.

var store = StoreRef.main();

A new value can be added via the add() method. If no record has been defined, then the key will be an auto-incremented integer. Assuming the database is a fresh new one, the returned key will be set to 1.

var k = await store.add(db, "1234");
  print("k: ${k}");
var key = store.add(db, "test");

k: 5

In case one would like to set a different key, for example using a String or a different kind of data, a new record can be created via the record() method. In the following example, the value mykey will be printed.

var k2 = await store.record("mykey").add(db, "test");
print("k2: ${k2}");

k2: mykey

At this point in the database, two key/value have been added, one with the key 1 and the second with the key mykey. To list all the keys available, one can use the findKeys() method. It is possible to filter the keys by configuring the finder parameter with a Finder() object, if not configured, all keys will be returned.

var keys = await store.findKeys(db);
print("keys: ${keys}");

keys: [1, mykey]

If one wants to check if a key is present, the findKey() method can be used instead. Like the previous method, finder parameter can be set to filter the keys stored in the store. Let defines one here for the example.

var keyA = await store.findKey(db,
  finder: Finder(
    filter: Filter.byKey("mykey")
  )
);
print("keyA: ${keyA}");

keyA: mykey

A Finder() object is used to filter the key/value from the store, a Filter() object can then be instantiated to find the desired key or value; for example, to check if a key equals another key via the Filter.byKey() constructor.

A more global method called find() can also be used to find a key, a value or both, using the same interface.

The data stored can obviously be returned via the get() method.

var value1 = await store.record(1).get(db);
print("value1: ${value1}");
var value2 = await store.record('mykey').get(db);
print("value2: ${value2}");

value1: 1234
value2: test

sembast can also react to changes on records when using the onSnapshot() method. It will return a Stream where one can listen to the events. Based on the documentation, the dart:async module is needed to invoke the unawaited() function to avoid memory leaks.

var subscription = store.record('mykey').onSnapshot(db).listen((snapshot) {
    print("listen snapshot: ${snapshot}");
    print("listen key: ${snapshot?.key}");
    print("listen value: ${snapshot?.value}");
  });
await store.record('mykey').put(db, 'update');
unawaited(subscription?.cancel());

listen snapshot: Record(_main, mykey) test
listen key: mykey
listen value: test
listen snapshot: {key: mykey, value: update, revision: 3}
listen key: mykey
listen value: update

As you can see above, the key mykey has been updated, it is also possible to remove records using the delete() method.

var kdelete = await store.record('mykey').delete(db);
print("kdelete: ${kdelete}");
var f = await store.record('mykey').get(db);
print("f: ${f}");

kdelete: mykey
f: null

sembast also supports transactions via the Database.transaction() method. A transaction reference to the database will be passed to an asynchronous lambda function, containing the list of actions to execute and alter the database accordingly.

await db.transaction((txn) async {
  await store.add(txn, 'transaction_data');
  await store.record('transaction').put(txn, "test");
});
var vtx1 = await store.record('transaction').get(db);
print("vtx1: ${vtx1}");

vtx1: test

All data can be wiped-out when using dropAll() method.

await db.dropAll();

The full application source code looks like that:

import 'package:sembast/sembast.dart';
import 'package:sembast/sembast_io.dart' as sio;
import 'package:sembast/sembast_memory.dart' as smem;
import 'dart:async';

void main(List<String> arguments) async {
  Database db = await initDb();
  var store = StoreRef.main();

  var k = await store.add(db, "1234");
  print("k: ${k}");

  var k2 = await store.record("mykey").add(db, "test");
  print("k2: ${k2}");

  var keys = await store.findKeys(db);
  print("keys: ${keys}");

  var keyA = await store.findKey(db,
    finder: Finder(
      filter: Filter.byKey("mykey")
    )
  );
  print("keyA: ${keyA}");

  var value1 = await store.record(1).get(db);
  print("value1: ${value1}");

  var value2 = await store.record('mykey').get(db);
  print("value2: ${value2}");

  var subscription = store.record('mykey').onSnapshot(db).listen((snapshot) {
    print("listen snapshot: ${snapshot}");
    print("listen key: ${snapshot?.key}");
    print("listen value: ${snapshot?.value}");
  });
  await store.record('mykey').put(db, 'update');
  unawaited(subscription?.cancel());

  var kdelete = await store.record('mykey').delete(db);
  print("kdelete: ${kdelete}");
  var f = await store.record('mykey').get(db);
  print("f: ${f}");

  await db.transaction((txn) async {
    await store.add(txn, 'transaction_data');
    await store.record('transaction').put(txn, "test");
  });
  var vtx1 = await store.record('transaction').get(db);
  print("vtx1: ${vtx1}");

  await db.dropAll();

  var close = await db.close();
  print("close: ${close}");
}

Future<int> addUser(Database db, String name) async {
  var store = StoreRef<int, String>.main();
  int key = await store.add(db, name);
  return key;
}

Future<Database> initDb({dbPath = "./sample.db"}) async {
  String dbPath = './sample.db';
  DatabaseFactory dbFactory = sio.databaseFactoryIo;
  return await dbFactory.openDatabase(dbPath);
}

Future<Database> initMemDb({dbId = "main"}) async {
  DatabaseFactory dbFactory = smem.databaseFactoryMemory;
  return await dbFactory.openDatabase(dbId);
}

Corrupted Database

JSON format is not really safe, no checksum by default, no real recovery procedure in case of corruption, and no way to know if the data received is the correct one. What could happen if the JSON objects stored in the database are corrupted? Let check that.

The first step is simple, how sembast will react when a database file is fully corrupted with random data. One can use /dev/random or openssl rand function.

$ cat /dev/random | dd of=sample.db count=1 bs=8k
1+0 records in
1+0 records out
8192 bytes (8.2 kB, 8.0 KiB) copied, 0.00407324 s, 2.0 MB/s

$ file sample.db 
sample.db: data

$ dart run
Building package executable... 
Built sb:sb.
k: 1
k2: mykey
keys: [1, mykey]
keyA: mykey
value1: 1234
value2: test
listen snapshot: Record(_main, mykey) test
listen key: mykey
listen value: test
listen snapshot: {key: mykey, value: update, revision: 3}
listen key: mykey
listen value: update
kdelete: mykey
f: null
vtx1: test
close: null

$ hexdump -C sample.db  | head -n 10
00000000  9c 50 9e 52 be 47 ca 73  5c 5f 62 44 8a 6f ab a5  |.P.R.G.s\_bD.o..|
00000010  be 77 b8 f3 a0 2b 55 ed  2f 7e d9 36 0c 04 15 96  |.w...+U./~.6....|
00000020  f5 d0 42 0a c5 32 40 ef  5c 21 cb 96 26 1a a0 ee  |..B..2@.\!..&...|
00000030  c8 d8 66 1e 17 f5 00 3e  e7 ae fd 94 e6 94 c2 f7  |..f....>........|
00000040  fb 90 2f 20 cc 1e f7 92  e9 bf 6d 38 09 32 a3 00  |../ ......m8.2..|
00000050  06 c2 5d bb f1 15 30 40  bf 45 56 ae 00 01 3c 87  |..]...0@.EV...<.|
00000060  ba cb ff 40 19 e9 76 69  3a 0d 65 38 bb 67 f6 30  |...@..vi:.e8.g.0|
00000070  c0 6a 98 9a e1 46 43 9d  7b d7 6e 3c e9 8c 15 8d  |.j...FC.{.n<....|
00000080  00 d7 22 8c 87 ac ea 20  0b b9 8a c0 1b 24 83 98  |..".... .....$..|
00000090  1b 5b 99 80 ac 41 f9 82  97 09 7e fd e3 9b 0c 66  |.[...A....~....f|

Interesting. The first part of the file is still random, but the end is containing the database. After another run, the random data at the beginning of the file is removed. The application used is the one created in the previous section, nothing really complex.

With the current database saved, we can append some random data at the end of the file, just to have an idea of what will happen.

$ cat /dev/random | dd count=1 bs=8k | tee -a sample.db > /dev/null
1+0 records in
1+0 records out
8192 bytes (8.2 kB, 8.0 KiB) copied, 0.0102849 s, 797 kB/s

$ dart run
Building package executable... 
Built sb:sb.
Unhandled exception:
FormatException: Unexpected character (at character 1)
&
^

#0      _ChunkedJsonParser.fail (dart:convert-patch/convert_patch.dart:1467:5)
#1      _ChunkedJsonParser.parseNumber (dart:convert-patch/convert_patch.dart:1333:9)
#2      _ChunkedJsonParser.parse (dart:convert-patch/convert_patch.dart:935:22)
#3      _parseJson (dart:convert-patch/convert_patch.dart:35:10)
#4      JsonDecoder.convert (dart:convert/json.dart:640:36)
#5      JsonCodec.decode (dart:convert/json.dart:219:41)
#6      SembastDatabase._fsFullMetaAndImport (package:sembast/src/database_impl.dart:861:23)
<asynchronous suspension>
#7      SembastDatabase.open.<anonymous closure>.import (package:sembast/src/database_impl.dart:1101:28)
<asynchronous suspension>
#8      SembastDatabase.open.<anonymous closure> (package:sembast/src/database_impl.dart:1115:15)
<asynchronous suspension>
#9      BasicLock.synchronized (package:synchronized/src/basic_lock.dart:38:16)
<asynchronous suspension>
#10     SembastDatabase.open (package:sembast/src/database_impl.dart:971:5)
<asynchronous suspension>
#11     DatabaseOpenHelper.openDatabase.<anonymous closure> (package:sembast/src/database_open_helper.dart:63:7)
<asynchronous suspension>
#12     BasicLock.synchronized (package:synchronized/src/basic_lock.dart:38:16)
<asynchronous suspension>
#13     SembastDatabaseFactoryMixin.openDatabaseWithOptions (package:sembast/src/database_factory_mixin.dart:55:12)
<asynchronous suspension>
#14     initDb (file:///home/user/tmp/dart/sb/bin/sb.dart:67:10)
<asynchronous suspension>
#15     main (file:///home/user/tmp/dart/sb/bin/sb.dart:7:17)
<asynchronous suspension>

Dart JSON parser can't parse the database anymore. Can we recover it? In this case, yes, because all the random data can be found at the end of the file, the JSON format can still be delimited. Unfortunately, what could happen if only a subset of the JSON object is modified? I think you already have the answer. The format used by sembast does not support data integrity, the metadata stored on the first line of the database does not contain for example a checksum. Every JSON objects stored are stored as it, without checksum as well.

Export/Import Database

sembast can import or export database with the help of import_export.dart module. For example, this can be used to import a database from the filesystem into memory. This feature can also be seen from the test suite in test/sembast_import_export_io_test.dart. Let start this section by adding the sembast_import_export module. The database file is the one generated from the first part of this article.

import 'package:sembast/utils/sembast_import_export.dart';

All the next snippets can be added in a custom main() function, just for doing some test. The first part of procedure to import/export a database is similar to our previous examples.

String dbPath = './sample.db';
DatabaseFactory dbFactory = sio.databaseFactoryIo;
var store = StoreRef.main();

The local database is opened with the openDatabase() method from a DatabaseFactory instance.

var db = await dbFactory.openDatabase(dbPath);
print(db);

Then we call the exportDatabase() function. It will return the data in a specific format, ready to be imported somewhere else. To be sure the feature is working, we can close() the database.

var exp = await exportDatabase(db);
print(exp);
await db.close();

In this example, the exported database is imported in an in-memory store via the importDatabase() function. The first argument is the exported database object, the second is the kind of database we want (e.g. databaseFactoryMemory) and finally, we can give a path, but I think this is not mandatory for this factory.

db = await importDatabase(
  exp,
  smem.databaseFactoryMemory,
  dbPath
);  
print(db);

The previously added data - here with the transaction key - can be retrieved directly from the memory.

print(await store.record("transaction").get(db));

This feature is great if one wants to import/export database from one factory to another. It can also be used to dump in-memory database (e.g. cache) locally for a fast-boot after an application shutdown.

Encryption and Integrity

War is 90% information.
-- Napoleon Bonaparte

We have seen in the corrupted database section sembast does not support data integrity. In case of corrupted data, nothing can be done to find what happened, then, we can't really trusted the content of the database. So, yes, transaction is supported, but we trust blindly the data saved on the HDD or in memory. Is it possible to fix that partially by creating our own codec, it can include a checksum and encrypt the data as well.

The document mention this feature, but forgot to give us something working. In fact, the implementation example can be found in the test suite, in sembast_test/lib/encrypt_codec.dart and in sembast_test/test/database_encrypt_codec_io_test.dart. If you want to use it, you should be aware the code there is not production-ready:

/// FOR DEMONSTRATION PURPOSES ONLY -- do not use in production as-is!

Anyway, it gives us a good idea of how things are working. Before encrypting data, validating it could be way better. It will also the first time I will create a Codec from scratch. To do that, dart:convert and dart:typed_data modules will be required.

import 'dart:convert';
import 'dart:typed_data';

We also need a way to create a checksum, for this test, I think using SHA256 from the crypto module will be enough.

import 'package:crypto/crypto.dart';

This is an external module, dart pub add is then required.

$ dart pub add crypto
...

Now we have our checksum function, we can create an interface to use it. The code above will take any kind of data as String and will return the checksum as hexadecimal String.

String _checksum(String s) {
  final json_encoded = json.encode(s);
  final utf8_encoded = utf8.encode(json_encoded);
  return sha256.convert(utf8_encoded).toString();
}

A Codec requires a decoder and a encoder attributes. Those attributes are created when the object is instantiated via the _IntegrityCoded() constructor. As you can see, two custom objects called _IntegrityEncoder() and _IntegrityDecoder() as then set to the corresponding attributes.

class _IntegrityCodec extends Codec<Object?, String> {
  late _IntegrityEncoder _encoder;
  late _IntegrityDecoder _decoder;

  _IntegrityCodec() {
    _encoder = _IntegrityEncoder();
    _decoder = _IntegrityDecoder();
  }

  @override
  Converter<String, Object?> get decoder => _decoder;

  @override
  Converter<Object?, String> get encoder => _encoder;
}

The _IntegrityEncoder class extends the Converter class. This class is used to return another representation of some kind of data. The only method to recreate is the convert() one. Below the raw working code snippet created to deal with the encoding part. When sembast is opening the database, it will pass the signature key in the encoder, then, the same signature must be returned by the coded. Then, if some keys/values are being added into the database, the key/value is passed to the encoder. The final returned value will be the line stored into the database. It seems a try/catch is present somewhere in sembast, hiding the errors and the exceptions if something bad is happening in the encoder, so, if you are creating your own encoder, one should be careful with that.

class _IntegrityEncoder extends Converter<Object?, String> {
  @override
  String convert(dynamic input) {
    print("encoder: $input");
    try {
      switch (input) {
        case {"signature": String signature}: return signature;
        case {"key": dynamic key, "value": dynamic value}:
          return json.encode({
            "key": key,
            "value": value,
            "checksum": _checksum(value)
          });
        default:
          return "error";
      }
    }
    catch (e,s) {
      print("$e, $s");
      throw(e);
    }
  }
}

A decoder is now required, let create the _IntegrityDecoder class. Like the previous one, it extends the Converter class and override the convert() method. The data to decode will be JSON strings, except for the signature. My code is a bit dirty, but I'm trying to be explicit. When a String is given, then it is decoded, the value and checksum values are extracted. A new checksum is generated to check if the value is correct, if it's the case, then the line is returned, else, an error is thrown.

class _IntegrityDecoder extends Converter<String, Object?> {
  @override
  dynamic convert(String input) {
    print("decoder: $input");
    try {
      switch (input) {
        case "integrity": return {"signature": "integrity"};
        case String s:
          final j = json.decode(s);
          final checksum = j["checksum"];
          final value = j["value"];
          final checksum_value = _checksum(value);
          if (checksum == checksum_value) return j;
          else throw("checksum error");
        default: return input;
      }
    }
    catch (e,s) {
      print("$e, $s");
      throw(e);
    }
  }
}

The codec is ready, but a SembastCodec object is required by sembast. The code present in the test suite creates a function to instantiate this kind of object, and I will follow this path. SembastCodec needs at least two parameters, signature representing the name of the codec is used, and codec where the custom Codec will be stored.

SembastCodec getIntegritySembastCodec() {
  return SembastCodec(
    signature: 'integrity',
    codec: _IntegrityCodec()
  );
}

A quick modification to the initDb() function is also required. The custom codec is also required when we are opening a database.

Future<Database> initDb({dbPath = "./sample.db", SembastCodec? codec}) async {
  String dbPath = './sample.db';
  DatabaseFactory dbFactory = sio.databaseFactoryIo;
  return await dbFactory.openDatabase(
    dbPath,
    codec: codec
  );
}

Finally, let modify the main() entry-point. Nothing fancy there, few data are inserted and then retrieved.

void main(List<String> arguments) async {
  Database db = await initDb(codec: getIntegritySembastCodec());
  var store = StoreRef.main();

  await store.record("test").put(db, "test");
  await store.record("test1").put(db, "test");
  await store.record("test2").put(db, "test");
  await store.record("test3").put(db, "test");

  print(await store.record("test").get(db));
  print(await store.record("test1").get(db));
  print(await store.record("test2").get(db));
  print(await store.record("test3").get(db));

  await db.close();
}

Okay, it looks good. What will happen when this application is executed?

$ > sample.db

$ dart run
Building package executable... 
Built sb:sb.
encoder: {signature: integrity}
encoder: {signature: integrity}
encoder: {signature: integrity}
encoder: {key: test, value: test}
encoder: {key: test1, value: test}
encoder: {key: test2, value: test}
encoder: {key: test3, value: test}
test
test
test
test

$ cat sample.db
{"version":1,"sembast":1,"codec":"integrity"}
{"key":"test","value":"test","checksum":"4d967a30111bf29f0eba01c448b375c1629b2fed01cdfcc3aed91f1b57d5dd5e"}
{"key":"test1","value":"test","checksum":"4d967a30111bf29f0eba01c448b375c1629b2fed01cdfcc3aed91f1b57d5dd5e"}
{"key":"test2","value":"test","checksum":"4d967a30111bf29f0eba01c448b375c1629b2fed01cdfcc3aed91f1b57d5dd5e"}
{"key":"test3","value":"test","checksum":"4d967a30111bf29f0eba01c448b375c1629b2fed01cdfcc3aed91f1b57d5dd5e"}

It works! If we modify a checksum or a value, the program should then crash. Let check that, I will modify the value stored with the key test3, and comment the insertion from the main() function.

$ sed -iE '/test3/s/"value":"test"/"value":"corrupted"/' sample.db 

$ cat sample.db
{"version":1,"sembast":1,"codec":"integrity"}
{"key":"test","value":"test","checksum":"4d967a30111bf29f0eba01c448b375c1629b2fed01cdfcc3aed91f1b57d5dd5e"}
{"key":"test1","value":"test","checksum":"4d967a30111bf29f0eba01c448b375c1629b2fed01cdfcc3aed91f1b57d5dd5e"}
{"key":"test2","value":"test","checksum":"4d967a30111bf29f0eba01c448b375c1629b2fed01cdfcc3aed91f1b57d5dd5e"}
{"key":"test3","value":"corrupted","checksum":"4d967a30111bf29f0eba01c448b375c1629b2fed01cdfcc3aed91f1b57d5dd5e"}

Let rerun the application with this modification.

$ dart run
...
decoder: {"key":"test3","value":"corrupted","checksum":"4d967a30111bf29f0eba01c448b375c1629b2fed01cdfcc3aed91f1b57d5dd5e"}
checksum error, #0      _IntegrityDecoder.convert (file:///home/user/tmp/dart/sb/bin/sb.dart:93:16)
#1      Codec.decode (dart:convert/codec.dart:30:34)
#2      SembastContentCodecAsyncSupport.decodeContentSync (package:sembast/src/async_content_codec.dart:128:30)
#3      SembastDatabase.decodeRecordLineStringSync (package:sembast/src/database_impl.dart:262:50)
#4      SembastDatabase._fsFullMetaAndImport (package:sembast/src/database_impl.dart:856:17)
<asynchronous suspension>
#5      SembastDatabase.open.<anonymous closure>.import (package:sembast/src/database_impl.dart:1101:28)
<asynchronous suspension>
#6      SembastDatabase.open.<anonymous closure> (package:sembast/src/database_impl.dart:1115:15)
<asynchronous suspension>
#7      BasicLock.synchronized (package:synchronized/src/basic_lock.dart:38:16)
<asynchronous suspension>
#8      SembastDatabase.open (package:sembast/src/database_impl.dart:971:5)
<asynchronous suspension>
#9      DatabaseOpenHelper.openDatabase.<anonymous closure> (package:sembast/src/database_open_helper.dart:63:7)
<asynchronous suspension>
#10     BasicLock.synchronized (package:synchronized/src/basic_lock.dart:38:16)
<asynchronous suspension>
#11     SembastDatabaseFactoryMixin.openDatabaseWithOptions (package:sembast/src/database_factory_mixin.dart:55:12)
<asynchronous suspension>
#12     initDb (file:///home/user/tmp/dart/sb/bin/sb.dart:166:10)
<asynchronous suspension>
#13     main (file:///home/user/tmp/dart/sb/bin/sb.dart:11:17)
<asynchronous suspension>

It crashed because the checksums are not the same. This code is just a draft, a recovery method should probably be used, a faster checksum like blake2 or blake3 would be better for this task. Anyway, the codec to encrypt can be seen in the test suite, it can be improved, but now, you have an idea of the flow to reproduce this feature.

Database Migration

In the 21st century, the database is the marketplace.
-- Stan Rapp

sembast supports versioned store, the first line of the file containing the data contains a version field. When opening a database, sembast expects to find a specific version number, if it's not the case, a migration procedure can be applied. No headers are required there, only the sembast.dart one. Let re-open again our sample.db database.

String dbPath = './sample.db';
DatabaseFactory dbFactory = sio.databaseFactoryIo;

A VersionChange() function will be required, here I will create a lambda function stored in a variable, but this function can be named somewhere else. The idea is too dynamically change/update the data from the database based on a version number. If you already did some SQL migration in the past, this is mostly the same thing, every time a new version of the schema is found, changes are applied.

final versionChange = (Database db, int oldVersion, int newVersion) {
  print("db: $db");
  print("oldVersion: $oldVersion");
  print("newVersion: $newVersion");
  print("--");
  while (oldVersion <= newVersion) {
    switch (oldVersion) {
      case 1:
        print("version 1 update");
        // do some check here
      case 2:  
        print("version 2 update");
        // do some update here
      case 3:
        print("version 3 update");
        // do some update here
      case 4:
        print("version 4 update");
        // do some update here
      case 5:  
        print("version 5 update");
        // do some update here
      case 6:  
        print("version 6 update");
        // do some update here
      default:
        throw("unsupported version $oldVersion");
    }   
    oldVersion += 1;
  }   
};

Finally, we can open the database and defines the version we are expecting from it (version 6). The versionChange() function is passed to the onVersionChanged parameter.

var db = await dbFactory.openDatabase(
  dbPath,
  version: 6,
  onVersionChanged: versionChange
);  

await db.close();

Now, we can execute the application, and you will see the different update procedure being executed.

$ dart run
Building package executable... 
Built sb:sb.
db: {path: /home/user/tmp/dart/sb/sample.db, version: 1, stores: [{name: _main, count: 5}], exportStat: {lineCount: 12, obsoleteLineCount: 4, compactCount: 0}}
oldVersion: 1
newVersion: 6
--
version 1 update
version 2 update
version 3 update
version 4 update
version 5 update
version 6 update

A migration procedure is always a nice feature, especially when it comes to databases. Indeed, in a world where applications are changing fast, a model should follow these changes without breaking. Adding an explicit format and a version to a database can help to do smooth upgrade without losing data (or service). More examples are available from the examples

Conclusion

life is short and information endless: nobody has time for everything
-- Aldous Huxley

sembast is a complete NoSQL database store, offering a wide range of features. It is easy to use, it supports transactions, multi-format and can be used with reactive programming pattern. Many different backend exists, allowing a developer to store data locally, in-memory or above more stable platform like sqlite.

Nothing is perfect though, and one should probably avoid using it if some custom specific formats are being used by its application. Using the filesystem as data-store can also have its own limitations (e.g. race condition, size limit...) and should be avoid when working with important volume of data. In fact, sembast should not be used for that, and one should expect good performance if the store is limited for a small amount of inserts.

One thing I fear the most with database is data-corruption, and sembast does not seem to support a way to recover data in case corrupted data. It's not really surprise, the database has been created using JSON, where each objects are stored line by line. sembast user must keep that in mind, and should probably use sembast_sqflite to avoid eventual corrupted data.

Anyway, to be honest, the sembast experience was great, and looks to be a good choice for cache management or for non-critical data. The effort to improve this library for other needs is perhaps to important though, and another choice - like SQLite - could be better. As usual, more resources can be found here:

Official sembast API documentation;
sembast source code at Github;
sembast_sqflite package from pub.dev;
sembast_web package from pub.dev;
stash_sembast from pub.dev;
Flutter/Dart ORM Benchmarking: Drift vs Floor vs Isar vs SQLite3 by Sidharth Mudgil on Medium;
Comparing Hive to other Flutter app database options by Onuoha Ifeanyi at LogRocker;
DB Benchmarks at Github;
Flutter — Persisting data locally using sembast by Shivmaurya on Medium.

Have fun and Happy Hack!

PS: it was a really huge article and it took me a lot of time to write it. I was expecting to find more documentation or examples on this module... But it was not the case. I started to read the source code and the test to understand more how sembast was working. Furthermore, writing articles during summer in France is kinda hard. ;)

Cover Image by Maksym Kaharlytskyi on Unsplash

Protocol Buffers in Elixir

Mathieu Kerjouan — Fri, 26 Jun 2026 10:00:00 +0000

Serializing data is an important topic. The main goal is to convert some kind of data in another format compatible for some tasks, usually, to send them over the network. JSON is probably one of the most used serializer, but because of its simplicity, can be slow and can lead to other issues. Indeed, JSON is using a text-like format, and disallow usage of binary in any form - except if they are converted in base64 string for example. Furthermore, JSON does not optimize its payload and does not care by default of the data containing in it.

Today, we will talk a bit of Protocol Buffer or protobuf, a binary serializer format created by Google for their internal use. Instead of JSON, protobuf payload is using a binary format and its data format is specified. One can't add a new field in the payload without having created the specification for it.

Requirements

protoc is required. One can install it from the latest release on Github, or use one's package manager distribution. protoc is used to compile protocol buffer data structure definition in a specific language. In our case, we will need to install an external plugins to help protoc to generate Elixir modules.

$ mix escript.install hex protobuf 0.16.0
...

Bootstrapping

$ mix new pb
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/pb.ex
* creating test
* creating test/test_helper.exs
* creating test/pb_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd pb
    mix test

Run "mix help" for more commands

$ cd pb

$ cat > mix.exs <<EOF
defmodule Pb.MixProject do
  use Mix.Project

  def project do
    [
      app: :pb,
      version: "0.1.0",
      elixir: "~> 1.19",
      start_permanent: Mix.env() == :prod,
      deps: deps()
    ]
  end

  def application, do: [extra_applications: [:logger]]

  defp deps, do: [{:protobuf, "~> 0.17.0"}]
end

$ mix deps.get
Resolving Hex dependencies...
Resolution completed in 0.024s
New:
  protobuf 0.17.0
* Getting protobuf (Hex package)
EOF

$ mix escript.install hex protobuf 0.16.0
...

$ export PATH=${PATH}:${HOME}/.mix/escripts

Payload Specification

Let reuse again the rock-paper-scissors idea to design a new protobuf data-structure. The file will be stored in ./proto/v1/rps.proto, but it could be perhaps better to store it in ./priv/proto/v1/rps.proto, not sure yet. Actually, we don't really care for now, because this protocol buffer file will be compiled into Elixir module. The first step is to define the syntax we are using, in our case, proto3.

syntax = "proto3";

Then, we use the package keyword to create a new namespace. In our case, it will create an Elixir module prefixed by Rps.

package rps;

We have 3 shapes to create, an enum should do the job. By convention, the enumerated values should be prefixed to avoid naming conflict. When we will create a new Shape enum, the default value used will be SHAPE_UNKNOWN, because it's the first one defined.

enum Shape {
  SHAPE_UNKNOWN = 0;
  SHAPE_ROCK = 1;
  SHAPE_PAPER = 2;
  SHAPE_SCISSORS = 3;
}

Our second main object we will use is the result of the game. We have 3 values to define, win, draw and loss; they can be defined as another enum following the same previously shown conventions.

enum Result {
  RESULT_UNKNOWN = 0;
  RESULT_WIN = 1;
  RESULT_DRAW = 2;
  RESULT_LOSS = 3;
}

Raw enums are kinda useless without using them inside messages. The first one we can create is the data-structure sent by a player to a server, it will only contain - for now - a Shape. If you are not familiar with the protobuf syntax, like me when I started this publication, the type of the data is on far-left followed by the name of the field. The assignment is not an integer value, but the field number used by the protocol buffer wire format. In this case, the shape field will have the number 1 assigned.

message Play {
  Shape shape = 1;
}

Now, let try to define another data-structure for a player. A player will have an id as integer, and an optional name as string. Protobuf is offering a wide range of scalar types, the best one(s) will mostly depend of our needs.

message Player {
  int64 id = 1;
  optional string name = 2;
}

When a client is asking for an opponent, the servers should return a list of available players. Protocol buffer does not define something like a list of an array, instead, it uses the repeated keyword to create a pseudo-list of objects while encoding or decoding the payload.

message Players {
  repeated Player players = 1;
}

The client select one player from the previous list and start to play with an other player as opponent. The data-structure will then contain a player object and a shape object.

message Play {
  Player player = 1;
  Shape shape = 2;
}

Finally, the server is returning the result, containing an unique id as string, player's informations and the final result for the player. I don't think this is the best data-structure to return because the result can lead to some confusion (who's the winner or loser?) but it will do the job for this example.

message PlayResult {
  string id = 1;
  Player player = 2;
  Player opponent = 3;
  Result result = 4;
}

If everything looks good, we can compile this protocol buffer file into an Elixir module with protoc. It will create the ./lib/proto/v1/rps.pb.ex file.

$ protoc --elixir_out=./lib proto/v1/rps.proto

Let invoke iex to play with these structures. To encode a struct in protobuf, we can use Protobuf.encode/1, it will return a binary. Here few examples from the message specifications defined above.

$ iex -S mix
iex(1)> encoded_player = Protobuf.encode(
  %Rps.Player{
    id: 1,
    name: "test" 
  })
<<8, 1, 18, 4, 116, 101, 115, 116>>

iex(2)> Protobuf.encode(
  %Rps.Player{
    id: 2 
  })
<<8, 2>>

iex(3)> Protobuf.encode(
  %Rps.Players{
    players: []
  })
""

iex(4)> encoded_players = Protobuf.encode(
  %Rps.Players{
    players: [
      %Rps.Player{id: 1},
      %Rps.Player{id: 2, name: "test"}
    ]
  })
<<10, 2, 8, 1, 10, 8, 8, 2, 18, 4, 116, 101, 115, 116>>

iex(5)> encoded_play = Protobuf.encode(
  %Rps.Play{
    player: %Rps.Player{ id: 1},
    shape: 1 
  })
<<10, 2, 8, 1, 16, 1>>

iex(6)> encoded_play = Protobuf.encode(
  %Rps.Play{
    player: %Rps.Player{ id: 1},
    shape: :SHAPE_ROCK
  })
<<10, 2, 8, 1, 16, 1>>

iex(7)> encoded_playresult = Protobuf.encode(
  %Rps.PlayResult{
    id: "random_string",
    result: :RESULT_WIN,
    player: %Rps.Player{id: 1},
    opponent: %Rps.Player{id: 2}
  })
<<10, 13, 114, 97, 110, 100, 111, 109,
  95, 115, 116, 114, 105, 110, 103, 18,
  2, 8, 1, 26, 2, 8, 2, 32, 1>>

When it comes to decode, one can use Protobuf.decode/2 or Protobuf.decode/3. The first argument is the encoded binary and the second is the Elixir module required to decode the payload.

iex(8)> Protobuf.decode(encoded_player, Rps.Player)
%Rps.Player{
  id: 1,
  name: "test",
  __unknown_fields__: [],
  __protobuf__: true
}

iex(9)> Protobuf.decode(encoded_players, Rps.Players)
%Rps.Players{
  players: [
    %Rps.Player{id: 1, name: nil, __unknown_fields__: [], __protobuf__: true},
    %Rps.Player{id: 2, name: "test", __unknown_fields__: [], __protobuf__: true}
  ],
  __unknown_fields__: [],
  __protobuf__: true
}

iex(10)> Protobuf.decode(encoded_playresult, Rps.PlayResult)
%Rps.PlayResult{
  id: "random_string",
  player: %Rps.Player{
    id: 1,
    name: nil,
    __unknown_fields__: [],
    __protobuf__: true
  },
  opponent: %Rps.Player{
    id: 2,
    name: nil,
    __unknown_fields__: [],
    __protobuf__: true
  },
  result: :RESULT_WIN,
  __unknown_fields__: [],
  __protobuf__: true
}

Finally, Here the generated Elixir module by protoc you can find in ./lib/proto/v1/rps.pb.ex:

defmodule Rps.Shape do
  @moduledoc false

  use Protobuf,
    enum: true,
    full_name: "rps.Shape",
    protoc_gen_elixir_version: "0.16.0",
    syntax: :proto3

  field :SHAPE_UNKONWN, 0
  field :SHAPE_ROCK, 1
  field :SHAPE_PAPER, 2
  field :SHAPE_SCISSORS, 3
end

defmodule Rps.Result do
  @moduledoc false

  use Protobuf,
    enum: true,
    full_name: "rps.Result",
    protoc_gen_elixir_version: "0.16.0",
    syntax: :proto3

  field :RESULT_UNKONWN, 0
  field :RESULT_WIN, 1
  field :RESULT_DRAW, 2
  field :RESULT_LOSS, 3
end

defmodule Rps.Player do
  @moduledoc false

  use Protobuf, full_name: "rps.Player", protoc_gen_elixir_version: "0.16.0", syntax: :proto3

  field :id, 1, type: :int64
  field :name, 2, proto3_optional: true, type: :string
end

defmodule Rps.Play do
  @moduledoc false

  use Protobuf, full_name: "rps.Play", protoc_gen_elixir_version: "0.16.0", syntax: :proto3

  field :player, 1, type: Rps.Player
  field :shape, 2, type: Rps.Shape, enum: true
end

defmodule Rps.PlayResult do
  @moduledoc false

  use Protobuf, full_name: "rps.PlayResult", protoc_gen_elixir_version: "0.16.0", syntax: :proto3

  field :id, 1, type: :string
  field :player, 2, type: Rps.Player
  field :opponent, 3, type: Rps.Player
  field :result, 4, type: Rps.Result, enum: true
end

defmodule Rps.Players do
  @moduledoc false

  use Protobuf, full_name: "rps.Players", protoc_gen_elixir_version: "0.16.0", syntax: :proto3

  field :players, 1, repeated: true, type: Rps.Player
end

Layered Design

While doing all those tests, I was thinking if it was a "good practices" or even possible to use protobuf with layers, a bit like a network packet. For example, a first payload is containing a protobuf header with the last part encoded as bytes. This header could then be used to help the decoder to know what kind of data is stored.

package layer;

message header {
  // checksum = "sha256-checksum"
  required string checksum = 1;

  // type = "application/protobuf/players"
  required string type = 2;

  // payload = protobuf binary for players object
  required bytes payload = 3;
}

Let compile it...

$ protoc --elixir_out=./lib/ proto/v1/layer.proto

... And start a new shell to play with that.

iex(1)> encoded_players = Protobuf.encode(%Rps.Players{ players: [%Rps.Player{ id: 1}] })
<<10, 2, 8, 1>>

iex(2)> encoded_headers =  Protobuf.encode(
  %Layer.Header{
    checksum: Base.encode64(
      :crypto.hash(:sha256, encoded_players)
    ),
    type: "application/protobuf/players",
    payload: encoded_players
  })
<<10, 44, 87, 90, 83, 69, 79, 81, 90, 102, 75, 87, 71, 101, 57, 66, 75, 65, 121,
  55, 107, 121, 118, 108, 76, 70, 98, 90, 110, 70, 108, 109, 116, 108, 52, 66,
  69, 83, 79, 102, 67, 89, 117, 43, 56, 61, 18, 28, 97, 112, 112, 108, 105, 99,
  97, 116, 105, 111, 110, 47, 112, 114, 111, 116, 111, 98, 117, 102, 47, 112,
  108, 97, 121, 101, 114, 115, 26, 4, 10, 2, 8, 1>>

With this kind of structure, we can then encapsulate different kind of data without parsing them. In fact, we can also put more metadata in the headers to help the application to know what kind of format is stored in the payload. Even more, one could also add some encryption feature there. Unfortunately, I don't know if it's a good practice, because it can probably make things a bit confusing, but also more flexible. Here the code generated in lib/proto/v1/layer.pb.ex:

defmodule Layer.Header do
  @moduledoc false

  use Protobuf, full_name: "layer.header", protoc_gen_elixir_version: "0.16.0", syntax: :proto2

  field :checksum, 1, required: true, type: :string
  field :type, 2, required: true, type: :string
  field :payload, 3, required: true, type: :bytes
end

Conclusion

This article was just a warmup, a kind of sandbox to see the capabilities of Protocol Buffer. At the same time, I was also looking to the FlatBuffers protocol trying to identify the difference between both format. I would prefer to use CBOR for my projects, but I would also like to avoid using too much dependencies from unknown repositories. Furthermore, protobuf Dart package is already used by Dart and Flutter (for metrics IIRC), it can also be used for messaging as well.

The previous code is containing a lot of design flaws, they can be "easily" fixed by redesigning a bit the data-structures and adding few of them. The procedure should look like that:

A connected client (with an active session) to the server list the available Players;
The client ask the server to create a new Arena for with one available Player;
The server returns an Arena containing an unique identifier, the name of the two players and arena's state (waiting, declined, ready, active, done). A waiting state is when the arena is waiting for the 2 players to become ready. A ready state is when both player are ready and can start playing together. An active state is when both players are playing. A done state is when the game is over, and one player won. A declined state is when the opponent declined the invitation;
When the server is in ready or active state, both players can send their Shape. The server returns a Result. If one win, the Arena's state switch to done.

Designing even a small game is not easy, but it can lead to really interesting result. We have a concrete example of distributed complexity there, with one user sending some data, waiting for result, and one actor (the server) keeping a private state to deal with both users.

Anyway, it was the first article on Protobuf and probably not the last one. I know you would like to dig a bit more on this topic, as usual, a list of resources:

protobuf Elixir module on hex.pm;
protobuf Elixir source code on Github;
protobuf Elixir module test source code;
Official Protobuf Best Practices;
Official Protobuf Google Group;
erlang_protobuffs Erlang module on github;
Protocol Buffer Best Practices on JsonToTable.org;
Roadmap to Learning Protobuf & Best Practices by Yusuf Savaş at Medium;
How to Handle Protocol Buffer Evolution from OneUptime.com;
Protocol Buffers by Chris Currier from Mobile Forensics – The File Format Handbook;
Cheatsheet - Protocol Buffers (Protobuf) from Hacking Notes;
Protocol Buffers for (Coding) Dummies on Github
Protocol Buffers Handbook by Clément Jean.

Happy Hack and Have Fun!

Cover Image by Mateusz Szerszyński on Unsplash

JSON Schema in Dart

Mathieu Kerjouan — Thu, 25 Jun 2026 08:00:00 +0000

Everything which distinguishes man from the animals depends upon this ability to volatilize perceptual metaphors in a schema, and thus to dissolve an image into a concept.
-- Friedrich Niezsche

When you know how to encode and decode JSON data, it can become annoying to deal with the structure every time. Worse, it can become a nightmare when some kind of data structure are being upgraded. That why some people created JSON Schema. The idea is to define what kind of fields and values should be contained in JSON objects. A package called json_schema was created to implement this standard.

Bootstrapping

$ dart create validate
Creating validate using template console...

  .gitignore
  analysis_options.yaml
  CHANGELOG.md
  pubspec.yaml
  README.md
  bin/validate.dart
  lib/validate.dart
  test/validate_test.dart

Running pub get...                     0.3s
  Resolving dependencies...
  Downloading packages...
  Changed 48 dependencies!
  3 packages have newer versions incompatible with dependency constraints.
  Try `dart pub outdated` for more information.

Created project validate in validate! In order to get started, run the following commands:

  cd validate
  dart run

$ cd validate

$ dart pub add json_schema
Resolving dependencies... 
Downloading packages... 
  _fe_analyzer_shared 103.0.0 (104.0.0 available)
  analyzer 13.3.0 (14.0.0 available)
+ http 1.6.0
+ json_schema 5.2.2
  package_config 2.2.0 (3.0.0 available)
+ quiver 3.2.2
+ rfc_6901 0.2.1
+ uri 1.0.0
Changed 5 dependencies!
3 packages have newer versions incompatible with dependency constraints.
Try `dart pub outdated` for more information.

We can edit bin/validate.dart and import the json_schema package.

import 'package:json_schema/json_schema.dart';

Schema Conception

Don't trust, just verify.
-- Steven Levitt

For this example, we will use the examples present in the JSON Schema Getting Started tutorial. Their first JSON object looks like that:

{
  "productId": 1,
  "productName": "A green door",
  "price": 12.50,
  "tags": [
    "home",
    "green"
  ]
}

To help us with the tests, we can create a function helper instantiating a schema and validating data. Let call it validate(). To instantiate a new JsonSchema object, the JsonSchema.create() static function can be invoked. Then, the JsonSchema.validate() method will be called to check if the data passed is correct.

void validate(Map<String, dynamic> schema, Map<String, dynamic> data, {String? msg}) {
  if (msg!=null) print("${msg}:");
  final _schema = JsonSchema.create(schema);
  final validate = _schema.validate(data);
  print("  ${validate}: ${data}");
  print("");
}

Before doing the whole specification for this object, we can start to specify only one of those fields, like the productId. json_schema module can use both a Map<String, dynamic> or a JSON<String> to load this. To make our life easier, we will create functions wrapper returning the schema we want as a Map.

Map<String, dynamic> _productId() {
  return {                                          
    "description": "The unique identifier for a product",
    "type": "integer",                              
  };                                                
}

A JSON schema describe an object and its properties. A productId is an integer. The description field is not used by the validator, it's only a way to help developers and designer to understand the structure. An object can be defined by many types:

null (e.g. null);
boolean (e.g. true or false);
number (e.g. 1 or -1.23);
integer (e.g 1 or 1024);
string (e.g. "test");
object (e.g. {});
array (e.g. []).

All of them are coming from the JSON specification itself.

Map<String, dynamic> _productName() {               
  return {                                          
    "description": "Name of the product",           
    "type": "string"                                
  };                                                
}

The productName is a string. Nothing complex here, the schema is similar to the productId object.

Map<String, dynamic> _price() {
  return {                                          
    "description": "The price of the product",
    "type": "number",                               
    "exclusiveMinimum": 0                           
  };                                                
}

A price is a number. To avoid negative numbers, we are also enforcing the specification with the exclusiveMinimum parameter. If a price is negative, the schema becomes invalid.

Map<String, dynamic> _tags() {
  return {                                          
    "description": "Tags for the product",
    "type": "array",                                
    "items": {                                      
      "type": "string"                              
    },                                              
    "minitems": 1,                                  
    "uniqueItems": true                             
  };                                                
}

Compound terms can also be created. The tags object is defined as an array or string. It must have at least 1 items (see minitems parameter) and must contain only unique items (see uniqueItems parameter).

Map<String, dynamic> _dimensions() {
  return {                                          
    "type": "object",                               
    "properties": {                                 
      "length": {                                   
        "type": "number"                            
      },                                            
      "width": {                                    
        "type": "number"                            
      },                                            
      "height": {                                   
        "type": "number"                            
      }                                             
    },                                              
    "required": [ "length", "width", "height" ]
  };                                                
}

Another object can be defined. It is also a compound term. In our case, the dimensions of an item must have a length as number, a width as number and a height as number too. All those fields are required (see required).

Map<String, dynamic> fullSchema() {
  return {
    "\$schema": "https://json-schema.org/draft/2020-12/schema",
    "\$id": "https://example.com/product.schema.json",
    "title": "Product",             
    "description": "A product in the catalog",
    "type": "object",
    "required": [  
      "productId",
      "productName",    
      "price",
    ],          
    "properties": {     
      "productId": _productId(),
      "productName": _productName(),
      "price": _price(),
      "tags": _tags(),
      "dimensions": _dimensions()
    }                                          
  };
}

Finally, the function fullSchema() is containing the whole JSON schema. Lot of fields here a kinda mandatory and are directly related to the JSON Schema specification. Let test that in our main entry-point.

void main(List<String> arguments) {
  validate(fullSchema(), {}, msg: "empty data");

  validate(fullSchema(), {
    "productId": 1
  }, msg: "productId only");

  validate(fullSchema(), {
    "productId": 1,
    "productName": "A green door",
    "price": 12.50
  }, msg: "minimal valid data");

  validate(fullSchema(), {
    "productId": 1,
    "productName": "A green door",
    "price": 12.50,
    "tags": [
      "home",
      "green"
    ]
  }, msg: "valid data with tags");

  validate(fullSchema(), {
    "productId": 1,
    "productName": "A green door",
    "price": 12.50,
    "tags": [
      "home",
      "green"
    ],
    "dimensions": {
      "length": 1,
      "width": 2,
      "height": 3,
    }
  }, msg: "valid data with tags and dimensions");
}

We can now run this application to see the result.

$ dart run
Resolving dependencies in `/tmp/dart/validate`... 
Downloading packages... 
Got dependencies in `/tmp/dart/validate`.
Building package executable... 
Built validate:validate.

empty data:
  INVALID, Errors: [# (root): required prop missing: productId from {}, /productId: required prop missing: productId from {}, # (root): required prop missing: productName from {}, /productName: required prop missing: productName from {}, # (root): required prop missing: price from {}, /price: required prop missing: price from {}]: {}

productId only:
  INVALID, Errors: [# (root): required prop missing: productName from {productId: 1}, /productName: required prop missing: productName from {productId: 1}, # (root): required prop missing: price from {productId: 1}, /price: required prop missing: price from {productId: 1}]: {productId: 1}

minimal valid data:
  VALID: {productId: 1, productName: A green door, price: 12.5}

valid data with tags:
  VALID: {productId: 1, productName: A green door, price: 12.5, tags: [home, green]}

valid data with tags and dimensions:
  VALID: {productId: 1, productName: A green door, price: 12.5, tags: [home, green], dimensions: {length: 1, width: 2, height: 3}}

As you can see, if the data received is missing some fields, the data is considered invalid. In other hands, if the requirements are there, the data is considered valid. Great, right? We now have a way to control the data received (or to control what we are sending).

Conclusion

Without approval and without scorn, but carefully studying the sentences word by word, one should trace them in the Discourses and verify them by the Discipline. If they are neither traceable in the Discourses nor verifiable by the Discipline, one must conclude thus: 'Certainly, this is not the Blessed One's utterance; this has been misunderstood by that bhikkhu - or by that community, or by those elders, or by that elder.' In that way, bhikkhus, you should reject it.
-- Siddharta Gautama

JSON Schema is a good way to make your application resilient and stable. It can be used as documentation tool, it will help to create tests, and it will also ensure the data received are legit. If you are working with public APIs, it can also help you to create the OpenAPI specifications. Finally, it will help you to design your application by thinking on what it needs, it will lead to a lean application containing only the necessary values.

JSON Schema can also be used to help us to reverse engineer private API interfaces by documenting what kind of data is working and which ones are not working. Creating specifications is great, Enjoy.

Wants to know more about JSON Schema? Here few interesting resources for you.

Official JSON Schema website;
json_schema package on pub.dev;
json_schema API documentation on pub.dev;
json_schema source code on Github;
json_schema examples on Github;
JSON Schema Explained with Examples: The Complete Guide by 楊東霖 on dev.to;
How to Validate JSON Schema: A Complete Developer Guide by 楊東霖 on dev.to;
JSON Schema in 10 Minutes — Validation, Types & Real Examples by Anh Quân Nguyễn on dev.to;
JSON Schema Examples - Real-World Schema Patterns by Saurabh Goyal.

Happy Hack and Have fun!

Cover Image by Alex wong on Unsplash

OpenVPN Daemon Double Free Bug

Mathieu Kerjouan — Wed, 24 Jun 2026 10:00:00 +0000

Since 2016, I started manage my own VPN infrastructure for my own usage and also for testing purpose. My VPN is not always started, in fact, I use it rarely. After the latest upgrade of OpenBSD to 7.9, I started to see the openvpn vanishing after few minutes of usage. It was not really important, but recently, I needed someone else to use the VPN for debugging purpose and... Every 3 or 4 minutes, the service crashed. Not really cool for a debugging session, fortunately, having tmux (or tmate) running on the other side is life saving. Let stop the openvpn service and invoke it as a standalone service instead of a daemon.

$ doas openvpn --config /etc/openvpn/server.conf
2026-06-20 08:27:44 us=727112 TCP/UDP: Closing socket                                                                                                        
2026-06-20 08:27:46 us=638261 MULTI: multi_create_instance called                                                                                            
openvpn(90124) in free(): double free 0x3626945d540
Abort trap

Indeed, something bad is happening with this application. It's not a good thing to see an Abort trap or a double free exception. Let check the release note from OpenBSD 7.9, but it did not mention anything regarding OpenVPN, is some case, OpenBSD team put some notes their if something really important about a package needs to be updated. Maybe this information can be found in the package's README but it was not the case. I already using most of the recommendations from this file.

OpenVPN package on OpenBSD supports some flavors, like the mbedTLS one. Perhaps installing this version could fix the issue, it could be an easy workaround. (installing a flavor with pkg_add can be done by postfix the package name with -- followed by a list of flavors separated by commas).

$ doas pkg_delete openvpn
...

$ doas pkg_add openvpn--mbedtls
...

Let start it again.

$ openvpn --configuration myconfig.conf
2026-06-20 09:09:43 us=709832 TCP/UDP: Closing socket                                                                                                       
openvpn(87978) in free(): double free 0x1957f910620                                                                                                           
Abort trap

Hmm, this is not a problem with this part of the application then. Let cleanup the openvpn configuration to see if we can find the culprit by removing all extra parameters and only keeping the mandatory ones.

ca /...
cert /...
chroot /var/openvpn
client-config-dir ccd
dev tun0
dh /...
dhcp-option DNS 10.1.0.1
dhcp-option DOMAIN ...
group _openvpn
key /...
local 1.2.3.4
port 1194
proto udp
server 10.1.0.0 255.255.255.0
tls-auth /... 0
topology subnet
user _openvpn
verb 11

This is basically the smallest configuration I can have for this service and it still crashes. Another great thing, we can install a debug version of openvpn on OpenBSD, so, why not?

$ doas pkg_delete openvpn--mbedtls
...

$ doas pkg_add openvpn debug-openvpn
...

$ doas openvpn --config /etc/openvpn/server.conf
2026-06-21 17:10:47 us=495655 Connection Attempt UDPv4 WRITE [54] to [AF_INET]1.2.3.4:32858: DATA ...
2026-06-21 17:10:47 us=496625 Connection Attempt UDPv4 write returned 54
2026-06-21 17:10:47 us=497180 Connection Attempt PO_CTL rwflags=0x0001 ev=4 arg=0xe6d6ae83e28
2026-06-21 17:10:47 us=497204 Connection Attempt PO_CTL rwflags=0x0001 ev=3 arg=0x00000002
2026-06-21 17:10:47 us=529222 Connection Attempt PO_WAIT[0,0] fd=4 rev=0x00000001 rwflags=0x0001 arg=0xe6d6ae83e28 [scalable]
openvpn(34334) in free(): double free 0xe6d6ae97000
2026-06-21 17:10:47 us=529957 Connection Attempt UDPv4 read returned 1222
2026-06-21 17:10:47 us=530561 Connection Attempt ACK read ID 1 (buf->len=1168)
2026-06-21 17:10:47 us=530609 Connection Attempt Valid packet (P_CONTROL_V1) with HMAC challenge from peer ([AF_INET]1.2.3.4:32858), accepting new connection.
2026-06-21 17:10:47 us=530628 Connection Attempt MULTI: multi_create_instance called
Abort trap

After this last issue, I decided to test the configuration on another server running on OpenBSD as well. Same behavior. I currently don't have any servers on Linux to install OpenVPN on it and do the correct configuration, but I assume this is a bug from OpenVPN. Let have a look at the version.

$ openvpn --version
OpenVPN 2.7.2 x86_64-unknown-openbsd7.9 [SSL (mbed TLS)] [LZO] [LZ4] [MH/RECVDA] [AEAD]
library versions: mbed TLS 3.6.6, LZO 2.10
Originally developed by James Yonan
Copyright (C) 2002-2026 OpenVPN Inc <sales@openvpn.net>
Compile time defines: enable_async_push=no enable_comp_stub=no enable_crypto_ofb_cfb=yes enable_dco=no enable_debug=yes enable_dlopen=unknown enable_dlopen_self=unknown enable_dlopen_self_static=unknown enable_dns_updown_by_default=yes enable_fast_install=needless enable_fragment=yes enable_gtk_doc=no enable_iproute2=no enable_libtool_lock=yes enable_lz4=yes enable_lzo=yes enable_management=yes enable_ntlm=yes enable_pam_dlopen=no enable_pedantic=no enable_pkcs11=no enable_plugin_auth_pam=no enable_plugin_down_root=yes enable_plugins=yes enable_port_share=yes enable_selinux=no enable_shared=yes enable_shared_with_static_runtimes=no enable_silent_rules=no enable_small=no enable_static=yes enable_strict=no enable_strict_options=no enable_systemd=no enable_werror=no enable_win32_dll=yes enable_wolfssl_options_h=yes with_aix_soname=aix with_crypto_library=mbedtls with_gnu_ld=yes with_mem_check=no with_openssl_engine=no with_sysroot=no

Now, we can create an issue on OpenVPN bugtracker on Github. Thanks to @cron2, he asked me to use gdb to see what's happening.

$ doas gdb openvpn
GNU gdb 6.3                                                                                                                                                   
Copyright 2004 Free Software Foundation, Inc.                                                                                                                                                                                                                                                                               
GDB is free software, covered by the GNU General Public License, and you are                                                                                                                                                                                                                                                
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-unknown-openbsd7.9"...(no debugging symbols found)

(gdb) run /etc/openvpn/server.conf
Starting program: /usr/local/sbin/openvpn /etc/openvpn/server.conf
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
...
2026-06-21 18:08:29 us=538009 Connection Attempt MULTI: multi_create_instance called
openvpn(83681) in free(): double free 0xcd8b677f2a0

Program received signal SIGABRT, Aborted.

(gdb) where
2026-06-21 18:08:29 us=538009 Connection Attempt MULTI: multi_create_instance called
openvpn(83681) in free(): double free 0xcd8b677f2a0

Program received signal SIGABRT, Aborted.
thrkill () at /tmp/-:2
2       /tmp/-: No such file or directory.
        in /tmp/-
Current language:  auto; currently asm
#0  thrkill () at /tmp/-:2
#1  0x87e238fe7c29df29 in ?? ()
#2  0x00000cd91200f8bb in _libc_abort () at /usr/src/lib/libc/stdlib/abort.c:51
#3  0x00000cd912014714 in wrterror (d=Variable "d" is not available.
) at /usr/src/lib/libc/stdlib/malloc.c:378
#4  0x00000cd91201a681 in find_chunknum (d=Variable "d" is not available.
) at include/machine/tcb.h:43
#5  0x00000cd912015cb0 in ofree (argpool=0x7e622a409040, p=0xcd8b677f2a0, clear=Variable "clear" is not available.
) at /usr/src/lib/libc/stdlib/malloc.c:1659
#6  0x00000cd912015933 in _libc_free (ptr=0xcd8b677f2a0) at /usr/src/lib/libc/stdlib/malloc.c:1729
#7  0x00000cd6a5471978 in x_gc_free () from /usr/local/sbin/openvpn
#8  0x00000cd6a54b1bb8 in pre_connect_restore () from /usr/local/sbin/openvpn
#9  0x00000cd6a548d7a9 in init_instance () from /usr/local/sbin/openvpn
#10 0x00000cd6a548fc21 in inherit_context_child () from /usr/local/sbin/openvpn
#11 0x00000cd6a54a1b94 in multi_create_instance () from /usr/local/sbin/openvpn
#12 0x00000cd6a54a0a06 in multi_get_create_instance_udp () from /usr/local/sbin/openvpn
#13 0x00000cd6a54a39e0 in multi_process_incoming_link () from /usr/local/sbin/openvpn
#14 0x00000cd6a54a0d12 in multi_process_io_udp () from /usr/local/sbin/openvpn
#15 0x00000cd6a54a9626 in multi_io_process_io () from /usr/local/sbin/openvpn
#16 0x00000cd6a54a7181 in tunnel_server () from /usr/local/sbin/openvpn
#17 0x00000cd6a54ad978 in main () from /usr/local/sbin/openvpn

After an investigation by cron2, it seems the bug was from the DHCP/DNS options management.

   3186
   3187         /* Free DNS options and reset them to pre-pull state */
   3188         gc_free(&o->dns_options.gc);
   3189         struct gc_arena dns_gc = gc_new();
   3190         o->dns_options = clone_dns_options(&pp->dns_options, &dns_gc);
   3191         o->dns_options.gc = dns_gc;

Great! We have found a bug, and the OpenVPN team already fixed it. Regarding the workarounds, for now... I don't have a lot of solutions:

configure dhcpd instead of using the DHCP service from OpenVPN
configure a local DNS server on the client configuration side
configure transparent DNS connections forwarding via a firewall (e.g. pf in this case).

In fact, this is a great timing, I need to upgrade my VPN infrastructure, too many useless services are running on it and I would like to have something lean. I would also like to remove tinc (it was used to interconnect others servers) and use wireguard instead. The whole story of this bug can be seen on Github issue @1060.

Happy Hack and have fun!

Cover Image by Carla Quario on Unsplash

SignalApp UI Home Screen but with Flutter

Mathieu Kerjouan — Tue, 23 Jun 2026 08:00:00 +0000

He who can copy can do.
-- Leonardo Da Vinci

My UI skill is very low, I'm not really good with colors or themes and it's always a pain to design a full interface by myself... In short: I'm not an artist and it needs to be fixed. SignalApp is one of my main messenger, the interface is simple and functional. The application source code is also available on Github, and the Android application is using Material3. I think it can be a good target.

Trying to reproduce it completely will take a while, only a small subset of the interface will be re-implemented, the home and the user chat screens, both can be seen from the screenshot above.

Home Screen

Good artists copy, great artists steal.
-- Pablo Picasso

The Home screen will be the entry-point of the application, where the active conversations can be seen by the user. It is also the place where SignalApp can be configured, when tapping on the User avatar on the top left of the application, a Drawer menu should open. This menu is not displayed on the screenshot, then, only an empty one will be created.

The first easily identifiable element of the application is the top bar. I will assume this one can be created with the help of the AppBar class, where the leading parameter will contain the user picture (rounded). This element can be created with a CircleAvatar class. The title parameter will display the text "Signal" in bold, a TextStyle class can be set properly inside the Text class style attribute. The parameters actions will be set with 2 icons, one icon will represent a camera with Icons.camera and the last one will represent a pencil using the pencil constant. Finally, the background color of this element is grey (#fafafa).

AppBar homeBar(BuildContext context) {
  return AppBar(
    backgroundColor: Colors.grey[50],
    titleTextStyle: TextStyle(
      color: Colors.black,
      fontWeight: FontWeight.bold
    ),
    leading: Padding(
      padding: EdgeInsets.all(8.0),
      child: CircleAvatar(
        backgroundColor: Colors.amber,
        child: Text('AH')
      ),
    ),
    title: Center(child: Text('SignalCopyCat')),
    actions: [
      IconButton(
        onPressed: () {}, 
        icon: Icon(Icons.camera_alt_outlined)
      ),
      IconButton(
        onPressed: () {},
        icon: Icon(Icons.edit_outlined)
      ),
    ],
  );
}

The top Bar using by Signal is not the most challenging part of the design, at least, when we don't have state to manage. The homeBar() function created above is returning an AppBar() object with some custom parameters.

The backgroundColor parameter is set to grey. The titleTextStyle parameter has been set with a custom Color and FontWeight, this should be applied on the rest of the widgets instantiated below this bar.

The leading parameter (the right part of the bar) is using a Padding widget to reduce the side of its child element with the help of an EdgeInsets object. The offset configured (8.0) is applied on all sides, thanks to the EdgeInsets.all() constructor. Its only child is CircleAvatar widget containing only the initials of the users as a Text() widget.

The title parameter is only containing the title of the application as in a Text() object, centered via a Center() widget.

The actions parameter is using a list of IconButton widget, which one set with an Icon(). The number of icons available with Material is astonishing... I took me a moment to find what I was looking for, Icons.camera_alt_outlined for the camera and Icons.edit_outlined for the pencil. Other icons can also be used via the CubertinoIcons class (usually designed for iOS though).

No callback functions have been set in the onPressed parameters, this is not the goal of this article.

The next big part of the Home screen is taking the remaining place. This section can be scrolled. It is made of many similar elements probably derived from a ListTile class. Those are split in 3 part, from the left to the right, the first part is an icon containing a picture of a remote user where we can reuse the CircleAvatar class. The second part contains the name of the user (in bold) above the last messages (if any). The last part contains the date of the last message received (with an attachment or an acknowledgement if any). When the user is tapping on one of these elements, it is redirected to a Chat Screen. The background color of this part of the application is white (#ffffff).

String initials(String str) {
  return str
    .split(" ")
    .map((x) => x.isEmpty ? "?" : x[0])
    .take(2)
    .join("");
}

Firstly, I think creating a function to convert a first name and last name (or any kind of string) to their initials can be helpful for the next part. This is a quick and dirty implementation, the string must be sanitized first and only characters (including UTF8 characters) should be used. It will do the job for know though, it could be nice to see what kind of algorithm SignalApp is using, I think the getAbbreviation() method from org.thoughtcrime.securesms.util.NameUtil is the one we are looking for.

ListTile tile({
  String? title,
  String? subtitle,
  String? trailing,
  MaterialColor? avatarColor,
  String? notifications,
  bool? received,
}) {
  return ListTile(
    leading: Badge(
      label: Text(notifications ?? "0"),
      isLabelVisible: notifications == null ? false : true,
      textColor: Colors.white,
      backgroundColor: Colors.blue,
      child: CircleAvatar(
        backgroundColor: avatarColor ?? Colors.amber,
        child: Container(
          decoration: BoxDecoration(shape: BoxShape.circle),
          child: Text(title == null ? "??" : initials(title)),
        ),
      ),
    ),
    title: Flex(
      direction: Axis.horizontal,
      mainAxisAlignment: MainAxisAlignment.spaceBetween,
      children: [
        Align(
          alignment: Alignment.topLeft,
          child: Text(
            title ?? "John Smith",
            style: TextStyle(fontWeight: FontWeight.bold, fontFamily: "Roboto")
            )
          ),
        Align(
          alignment: Alignment.topRight,
          child: Text(
            trailing ?? "11:11 AM",
            style: notifications != null 
              ? TextStyle(fontWeight: FontWeight.bold, fontSize: 12, fontFamily: "Roboto")
              : TextStyle(fontWeight: FontWeight.normal, fontSize: 12, fontFamily: "Roboto"),
          ),
        ),
      ],
    ),
    subtitle: Flex(
      direction: Axis.horizontal,
      mainAxisAlignment: MainAxisAlignment.spaceBetween,
      children: [
        Expanded(
          flex: 1,
          child: Text(
            subtitle ?? "Last message...",
            overflow: TextOverflow.ellipsis,
            style: notifications != null 
              ? TextStyle(fontWeight: FontWeight.bold, fontSize: 14, fontFamily: "Roboto")
              : TextStyle(fontWeight: FontWeight.normal, fontSize:14, fontFamily: "Roboto")
          ),
        ),
        received == true
            ? Icon(Icons.library_add_check_rounded, size: 16)
            : Container(),
      ],
    ),
    onTap: () {},
  );
}

Another function helper called tile() will help us to generate each ListTile() objects. This function can take 6 optional arguments, if those are not set, some default values are used instead.

The leading parameter is containing the user avatar, material gives us the Badge() object to deal with that. This one contains a label (the blue circle containing the number of notifications). I tried to add a white border on this label, without success, perhaps my future me will fix that. If there is no notification, the isLabelVisible parameter can be set to false, in this case, the label will be removed. The only child is a CircleAvatar using a specific BoxDecoration to produce a circle via BoxShape.circle. By default, the initials of the title variable will be used.

Regarding badges, 2 modules can probably fix most of my issues here:

info_label: A high-performance Flutter label widget built on CustomPainter. Background, border, text, and overlay indicators are painted directly on canvas for minimal widget overhead;
badges: A package for creating badges. Badges can be used for an additional marker for any widget, e.g. show a number of items in a shopping cart.

The title parameter was configured to set the user name on the left and the date on the right. I thought at first I could use the trailing parameter to do that, but it was kinda messy, and way more complex. Anyway, a Flex widget is used to let us have a flexible positioning across the elements. Indeed, we want the widgets on an horizontal Axis (see Axis.horizontal) with a space between them (see MainAxisAlignment.spaceBetween). Then the 2 widgets children can be aligned using the Alignment.topLeft and Alignment.topRight constants. Nothing smart here, both of them are Text() widgets.

The subtitle is using the same hack than the title parameter previously described. An element to left (last message from the remote user) and another one to the right (message ack).

onTap is not configured yet, but the idea when someone is tapping on one tile, he will be redirected to the chat screen.

class Conversations extends StatelessWidget {
  const Conversations({super.key});

  @override
  Widget build(BuildContext context) {
    return ListView(
      children: [
        tile(
          avatarColor: Colors.blueGrey,
          title: "Tina Ukuku",
          subtitle: "You set disappearing message time to 1...",
          avatarText: "TU",
        ),
        tile(
          avatarColor: Colors.indigo,
          title: "Chairman Meow",
          subtitle: "Missed call",
          avatarText: "CM",
          notifications: "2",
        ),
        tile(
          avatarColor: Colors.lightGreen,
          title: "Myles Larson",
          subtitle: "🎤 Voice Message",
          trailing: "11:07 AM",
          notifications: "7",
        ),
        tile(
          avatarColor: Colors.lime,
          title: "Cat Chat 🐈 🐱",
          subtitle: "This is the instruction manual. 📎 Attachment",
          trailing: "11:02 AM",
        ),
        tile(
          avatarColor: Colors.blueGrey,
          title: "Ali Smith",
          subtitle: "📷 Attachment",
          trailing: "10:38 AM",
          received: true,
        ),
        tile(
          avatarColor: Colors.indigo,
          title: "Kirk Family",
          subtitle: "Happy birthday to you. Happy birthday to you!",
          trailing: "9:13 AM",
          notifications: "1",
        ),
        tile(
          avatarColor: Colors.lightGreen,
          title: "Jordan B.",
          subtitle: "Sticker Message",
          received: true,
        ),
        tile(
          avatarColor: Colors.lime,
          title: "Sunsets 🌅",
          subtitle: "View-once media",
          trailing: "TUE",
          received: true,
        ),
        tile(
          avatarColor: Colors.blueGrey,
          title: "🧗‍♂️ Rock Climbers",
          subtitle: "Which route should we take?",
          trailing: "TUE",
        ),
        tile(
          avatarColor: Colors.indigo,
          title: "Nikki R.",
          subtitle: "Thanks! What a wonderful message to read in the morning!",
          trailing: "TUE",
        ),
        tile(
          avatarColor: Colors.lightGreen,
          title: "Weather Forecasts",
          subtitle: "Raining all day 📷 Attachment",
          trailing: "MON",
        ),
        tile(avatarColor: Colors.lime),
        tile(avatarColor: Colors.blueGrey),
        tile(avatarColor: Colors.indigo),
        tile(avatarColor: Colors.lightGreen),
        tile(avatarColor: Colors.lime),
      ],
    );
  }
}

Now our tile() function is ready, we can create our Conversations() object containing the list of user chats. This part creates a ListView object, containing a list of ListTile created by our helper function. This part seems complex, but only one function is used many time to generated content and mimic the screenshot.

class Home extends StatelessWidget {
  const Home({super.key});

  Widget build(BuildContext context) {
    return Scaffold(
      appBar: homeBar(context),
      body: Conversations()
    );
  }
}

Everything looks good, we then mix everything together inside an Home class returning a Scaffold() object.

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'SignalCopyCat',
      home: Home(),
      theme: ThemeData(
        fontFamily: "Roboto"
      ),
      themeMode: ThemeMode.light,
      debugShowCheckedModeBanner: false,
    );
  }
}

void main() {
  runApp(const MyApp());
}

Finally, the last step, we can create also our application using a MaterialApp() object.

Above, a screenshot of this application from the Android emulator. The images/pictures are missing on purpose, I was not able to found the "correct" images. As you can see, the emojis are not correctly rendered as well. Still few bugs to fix, but I did that in few hours.

I forgot something though, a big part of my code is using custom fonts (Roboto). Fonts are kind of assets and must be downloaded then configured in the project. The Use a custom font article on the official Flutter documentation is doing a great job to explain that.

Conclusion

I’ve been imitated so well, I’ve heard people copy my mistakes.
-- Jimi Hendrix

This is not the perfect 1:1 reproduction of the SignalApp Home screen... I still need more experience with Flutter. The terms used are also new to me, finding one icon similar to the one from the screenshot is taking time for example. The infinite amount of solution to display something is also time consuming. Here a list of few resources I checked during this experience.

Flutter - Wrapping text on StackOverflow;
Lay out multiple widgets vertically and horizontally from Flutter Documentation;
How to show a notification badge on the app icon using flutter? on StackOverflow;
Flutter CIrcle Avatar With Border on StackOverflow
Flutter give container rounded border on StackOverflow
How to make a widget maximum size be equal to the flex value in a row on StackOverflow
Roboto fonts from Google fonts, an alternative font to the one used by Signal

Anyway, copying and reproducing is a great way to learn. It also train the mind to decompose every applications in small pieces. The same could have been done for Briar, SimpleX or Telegram applications, all of them are open-source and available on Github, Gitlab or somewhere else. Some of them are supporting mobile, desktop and web platform, in short, they are great targets. To summarize, if you want to learn how to design UI, try to recreate the design and the style of your favorite application.

The next step? Why not create an open-source library in Dart/Flutter exposing these styles? Perhaps one day...

Happy Hack and Have fun!

Cover Image by Compagnons on Unsplash

DEV Community: Mathieu Kerjouan

Flutter and SQLite with sqflite

Bootstrapping

Simple Cache Schema

Integration

Entry-Point

Material Application

Home Screen

Inputs

Notifications

Items

Database Interface

Interfaces Summary

Conclusion

Dealing with null in Dart

Bootstrapping

Dissecting Null

Operators

It's All About Syntactic Sugar

Conclusion

Kanell/0, Yet Another Pipeline-like Library

Erlang Pipeline

Kanell Pipeline

What's the Point?

Conclusion

SignalApp UI Chat Screen but with Flutter

Chats Screen

Chat Bar

Chat Messages

Attachments

Messages

User Chat Input

Final Result

Conclusion

QubeOS Tips and Tricks

Copying files between dom0 to domU

Copying files between domU

Executing Commands in Minimal Qubes as root

Xen Store Dom0 view

Xen Store DomU view

Qubesdb

Restarting the Network

Local Network Script with rc.local

Global Network Script

Creating Ubuntu Template

Starting a disposable VM from standard VM

Starting a disposable VM from Dom0

Conclusion

Working with Elm

Introduction

Phoenix Project from Scratch

Elm Project from Scratch

Esbuild and NPM Bootstrapping

Bootstrapping Phoenix Framework

Development Process

Configuring Backend Hostname

Managing Url with Browser.element

Conclusion

References and Resources

Chored/0: an alternative task manager

Introduction

Constraints and Requirements

Design

Usage

Ideas

Conclusion

Persistent Data Storage with sembast

Bootstrapping

Persistent Local Store

In-Memory Store

Data Store Usage

Corrupted Database

Export/Import Database

Encryption and Integrity

Database Migration

Conclusion

Protocol Buffers in Elixir

Requirements

Bootstrapping

Payload Specification

Managing Url with `Browser.element`