DEV Community: Simon Shine

JSON Schema in Haskell using AutoDoCodec

Simon Shine — Sat, 26 Jul 2025 21:02:21 +0000

In April 2022 I made a review of Haskell libraries for JSON Schema.

The outcome of my research was that there weren't really any good libraries for dealing with JSON Schema from within Haskell. That was a surprising disappointment, because Haskell often has very good libraries (although they may be hard to use). The reason is that the JSON Schema specification itself isn't very well-made.

I eventually gave up using any of them.

Today Albert Krewinkel wrote me and told me that you can actually use AutoDoCodec to both generate and validate JSON schemas. The Haskell package autodocodec was announced in Haskell Weekly #291 in November 2021 by Tom Syd Kerckhove. I knew this library, but had never made the connection that it'd serve all my JSON schema needs. Since my original blog post felt like a dud, here is another post with a happy ending.

AutoDoCodec provides a unified approach to defining codecs that can both serialize/deserialize JSON and generate corresponding JSON schemas automatically. It eliminates the common problem of having separate validation logic and schema definitions that can drift out of sync over time.

Since the JSON schema application as an example is a little dispersed, here's a full example of how one can define a data type, generate its schema, and validate a JSON object against it. You can read the full source code for the example on GitHub.

Step 1: Define your data type and codec

First, let's define a simple data type with a string, integer, and boolean field, along with its AutoDoCodec instance:

{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE DerivingStrategies #-}
{-# LANGUAGE OverloadedStrings #-}

import Autodocodec
import Data.Aeson (ToJSON(..), FromJSON(..))
import GHC.Generics (Generic)

data Person = Person
  { personName :: String
  , personAge :: Int
  , personIsActive :: Bool
  } deriving stock (Show, Eq, Generic)

instance HasCodec Person where
  codec = object "Person" $
    Person
      <$> requiredField "name" "The person's name" .= personName
      <*> requiredField "age" "The person's age in years" .= personAge
      <*> requiredField "isActive" "Whether the person is currently active" .= personIsActive

instance ToJSON Person where
  toJSON = toJSONViaCodec

instance FromJSON Person where
  parseJSON = parseJSONViaCodec

The HasCodec instance defines both the JSON structure and provides field descriptions that will appear in the generated schema. The ToJSON and FromJSON instances are automatically derived from the codec definition.

Step 2: Generate the JSON schema

Now we can generate a complete JSON schema from our data type:

import Autodocodec.Schema (jsonSchemaViaCodec)
import Data.Aeson.Encode.Pretty (encodePretty)
import Data.ByteString.Lazy.Char8 qualified as L8

main :: IO ()
main = do
  let schema = jsonSchemaViaCodec @Person
  putStrLn "Generated JSON Schema:"
  L8.putStrLn $ encodePretty schema

This produces a complete JSON schema with type information, descriptions, and validation rules:

{
    "$comment": "Person",
    "properties": {
        "age": {
            "$comment": "The person's age in years",
            "maximum": 9223372036854775807,
            "minimum": -9223372036854775808,
            "type": "integer"
        },
        "isActive": {
            "$comment": "Whether the person is currently active",
            "type": "boolean"
        },
        "name": {
            "$comment": "The person's name",
            "type": "string"
        }
    },
    "required": ["isActive", "age", "name"],
    "type": "object"
}

Step 3: Validate JSON against the schema

AutoDoCodec provides built-in validation functions to check JSON against the generated schema:

import Autodocodec.Schema (validateAccordingTo)
import Data.Aeson qualified as Aeson

-- Valid JSON example
let validJson = Aeson.object
      [ "name" Aeson..= ("Alice" :: String)
      , "age" Aeson..= (30 :: Int)
      , "isActive" Aeson..= True
      ]

-- Invalid JSON example (age as string instead of number)
let invalidJson = Aeson.object
      [ "name" Aeson..= ("Bob" :: String)
      , "age" Aeson..= ("not-a-number" :: String)
      ]

-- Validate using the schema
if validateAccordingTo validJson schema
  then putStrLn "Valid JSON accepted"
  else putStrLn "Valid JSON rejected"

if validateAccordingTo invalidJson schema
  then putStrLn "Invalid JSON incorrectly accepted"
  else putStrLn "Invalid JSON correctly rejected"

For more detailed error messages, you can use the parsing functions directly:

case Aeson.fromJSON invalidJson of
  Aeson.Success (_ :: Person) -> putStrLn "Parsed successfully"
  Aeson.Error err -> putStrLn $ "Parsing failed: " ++ err
-- Output: "Parsing failed: parsing Scientific failed, expected Number, but encountered String"

Adding validation constraints

AutoDoCodec provides built-in support for adding validation constraints to both numeric and string fields.

Numeric constraints

To constrain the age field to reasonable values between 0 and 200:

-- Define age bounds: 0 to 200 years
ageBounds :: Bounds Integer
ageBounds = Bounds (Just 0) (Just 200)

String length constraints

To ensure the name field is not empty:

-- Name length bounds: at least 1 character (non-empty)
nameBounds :: Bounds Word
nameBounds = Bounds (Just 1) Nothing

Updated codec with constraints

instance HasCodec Person where
  codec = object "Person" $
    Person
      <$> requiredFieldWith "name" (textWithLengthBoundsCodec nameBounds) "The person's name (non-empty)" .= personName
      <*> requiredFieldWith "age" (dimapCodec fromIntegral fromIntegral (integerWithBoundsCodec ageBounds)) "The person's age (0-200 years)" .= personAge
      <*> requiredField "isActive" "Whether the person is currently active" .= personIsActive

This automatically generates JSON schema with proper validation constraints:

{
  "name": {
    "$comment": "The person's name (non-empty)",
    "minLength": 1,
    "type": "string"
  },
  "age": {
    "$comment": "The person's age (0-200 years)",
    "maximum": 200,
    "minimum": 0,
    "type": "integer"
  }
}

The constraints are enforced both during JSON parsing and schema validation, ensuring invalid values like empty names or ages outside 0-200 are automatically rejected.

The beauty of AutoDoCodec is that your codec definition serves as both the source of truth for JSON serialization/deserialization and schema generation. This eliminates the common problem where validation logic and schema definitions drift apart over time, ensuring they always stay in sync.

Minimal Sufficient Groups

Simon Shine — Sun, 02 Feb 2025 16:36:48 +0000

This is a programming interview question I got some years ago.

The problem

When a certain company creates a new project, they enumerate the skills required for the project, and they form a project group where each of those skills are represented. Project groups have two rules:

Sufficient: Each required skill should be represented with at least one group member.
Minimal: Each group member should be essential, so removing any one member from the team should make the rest of the team no longer sufficient. Members can have overlapping skills, as long as everyone is essential in some capacity.

The task is, given a set of employees and their skill sets, and a set of required skills, provide all possible minimal, sufficient project groups.

My solution

{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ScopedTypeVariables #-}

module MinimalProjectGroups where

import Data.Map.Strict (Map)
import qualified Data.Map.Strict as Map
import Data.Set (Set)
import qualified Data.Set as Set
import Data.Text (Text)
import qualified Data.Text as Text

minimalSufficientGroups
  :: forall employee. forall skill. (Ord employee, Ord skill)
  => Map employee (Set skill) -> Set skill -> Set (Set employee)
minimalSufficientGroups staff requirements =
  Set.filter (\group -> isSufficient group && isMinimal group) allGroups
  where
    allGroups :: Ord employee => Set (Set employee)
    allGroups = Set.powerSet (Map.keysSet staff)

    isSufficient :: Ord employee => Set employee -> Bool
    isSufficient group =
      requirements `Set.isSubsetOf` Set.unions (Set.map skillsOf group)

    skillsOf :: (Ord employee, Ord skill) => employee -> Set skill
    skillsOf employee = Map.findWithDefault Set.empty employee staff

    isMinimal :: Ord employee => Set employee -> Bool
    isMinimal group = all (`isEssentialFor` group) group

    isEssentialFor :: Ord employee => employee -> Set employee -> Bool
    isEssentialFor employee group =
      not (isSufficient (Set.delete employee group))

I optimized for natural readability and type generality.

Solving it today I would have factored in the total memory usage of the program, since power sets are exponential. While laziness helps a little, I have come to appreciate explicit control of program memory.

Why you should be careful with the Default trait/typeclass

Simon Shine — Tue, 25 Oct 2022 07:27:54 +0000

You heard it. Default considered harmful!

tl;dr:

Default has no defining or testable properties
Default is not even guaranteed to be the same between runs
Defaults are context-specific: One type does not always have one default
Defaults are useful for transitive derivation, but context drift risks causing errors
Your Default instance is dangerous if you cannot carelessly replace it with any other value
Haskellers: Use the default* pattern instead
Rustaceans: Use Default with care and respect

`Default` in Rust and in Haskell

Rust's standard library has a Default trait that lets you provide a default value for any type:

pub trait Default {
    fn default() -> Self;
}
Returns the "default value" for a type.

Default values are often some kind of initial value, identity value, or anything else that may make sense as a default. Sometimes, you want to fall back to some kind of default value, and don't particularly care what it is.

The Default type class never made it to Haskell's standard library; one lives in the data-default package, and another one with much more pleasing defaults (sic) lives in acme-default.

When asking Haskellers why, the typical reason is that "type classes should reflect mathematical properties (invariants), and Default does not have any properties": Testing an instance of Default without making additional assumptions about the type gives you no good things to assert.

The Rust ecosystem does not have a similar mathematical vigilance around the use of traits: You are much more free to define traits in Rust than type classes in Haskell, and they don't have to reflect mathematical depth. Both of these attitudes are fine, so to dig into why Default is also bad in Rust, we must dig a little deeper.

Defaults are useful, but `Default` is sketchy

Software configuration relies heavily on default values.

Defaults let you run complicated software without having read the complete manual.

The features that require custom configuration (such as external APIs) can be disabled by default.

The default database can be SQLite for evaluating the software or for a lighter setup.
The default logging can be on-screen or in an operating system default directory.
The default host/port can be 127.0.0.1:3000, so you don't accidentally expose the service when toying around on semi-public networks.
Using your current working directory as the default directory sometimes works.

The implied context here is "a user will run the software and react to any unintended behavior caused by defaults". When the notion of configuration defaults is taken to a programmatic extreme, the implied context is not necessarily true.

What is the default integer?

3000 seems like a good choice if it's a generic service port number. 0 or 1 both seem like good choices if it's a monoid. If it's a service port number, 0 will mean "kernel picks a randomly unused port number", which does not produce deterministic behavior. If it's an IPv4 address, listening on 0.0.0.0 means you will listen on all interfaces, including public ones. Should the default HTTP client parameters be with or without SSL? For simplicity: without. For security: with. It depends on context. You may think that your type has a well-defined default, but nesting it inside a bigger structure may cause that to no longer be the case, because defaultness does not transcend.

An example of this drift from the real world is when XMonad's grabKey grabbed all unbound keys, because "KeySym 0 (NoSymbol) gets mapped to every unbound KeyCode, since that's what XKeycodeToKeysym returns for those." So while KeySym 0 (NoSymbol) is an obvious pick, it isn't so good inside XKeycodeToKeysym.

Another example is when a default prime-field element, 0, caused .mul_inverse() to fail. What a poor implementation of multiplicative inverses when it doesn't even work for the default prime-field element! The default, I tell you! (...and I haven't told you much, because defaults are not characterised by anything in general.)

The bad examples of defaults are general to any programming language.

Does your `Default` do anything?

To check if you understand and appreciate how arbitrary defaults should be capable of being, try and see if you agree with some of the instances found in acme-default package:

The Default False is the answer to the question whether mniip has a favourite Bool.
The Default Char is '→' because arrows look fancy when you use them in a chat.
The Default Double is 1.1102230246251565e-16 as the difference between 1 and sum (replicate 10 0.1).
The Default Float is 388.38, which is approximately equal to twice the molar mass of caffeine in grams per mol.
The Default Int is 18871, which is the product of a Sophie-Germain prime and a safe prime. You know, for safety and for Sophie.
The Default Int8 is 29 for obvious reasons.

The mockery here serves to highlight the conceptual mistake that defaults are often embedded with meaning that is not warranted in light of being a default, because defaults don't have special meaning.

Rust: A `Default` case-study

An example of when Default is used in Rust can be found in the Axum web framework:

impl<S, B> Default for Router<S, B>
where
    B: HttpBody + Send + 'static,
    S: Default + Clone + Send + Sync + 'static,
{
    fn default() -> Self {
        Self::with_state(S::default())
    }
}

It says that a web app's state must impl Default.

This makes you able to start a web app "from scratch".

Is this made carefully?

Because Axum is a web framework and not a web app, providing a generic way to pass the default starting state also passes on the responsibility of defining sensible defaults to the web app developer. Different web apps may use "default" to mean "initial", "empty", or "demo" at their own discretion.

Default's lack of any properties seems to make it a legit placeholder here.

Summary

Implicit assumptions about the properties of a default for some specific type may hold in some contexts. They may also eventually propagate into errors, like in the examples above. Any legal value of a type should be a valid default.

You may prefer one default over another, but you shouldn't have to.

The recommended pattern in Haskell is to have named value bindings:

-- | The default configuration for trying out the software
defaultConfig :: Config
defaultConfig = ...

-- | The default shortcut key for interacting with the software
defaultModMask :: ModMask
defaultModMask = controlMask

so that any special context can be referred to in the name of the binding.

In Rust we're stuck with having Default in the standard library, so we cannot treat it with the same degree of aversion, since it may occur in the wild. Instead we have to treat it with respect and care, because its only pseudo-property is fragile: It shouldn't matter what the default is.

While avoiding Default in Rust is not a general recommendation, the practice of naming things transcends languages. If an impl Default demonstrates other properties (i.e. needs to be some value to work, or breaks something when changed), it is probably more than a default. It doesn't have to be the impl of any trait, and naming it may be a better option.

Do you have examples of when a sketchy Default caused problems down the line?

(Cover art generated by DreamStudio.ai asking it to draw "default")

A review of JSON Schema libraries for Haskell

Simon Shine — Sun, 10 Apr 2022 14:14:14 +0000

Update 2025-07-26: This review is obsolete. Consider using AutoDoCodec.

JSON Schema is a JSON format that describes JSON formats.

It is mostly used to validate that a JSON value has the correct format, but it can also be used to generate random values that fit a schema definition. This may be useful for testing.

The latest version of JSON Schema is called "Draft 2020-12", and the way to define a schema using this draft:

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    ...
}

Previous versions were called Draft-03 (2010), Draft-04 (2013), ... up to Draft-08 which was renamed into Draft 2019-09. When exploring Haskell libraries that handle JSON Schema definitions, they tend to have stalled on an earlier draft. There seems to be a pattern, and Juspay's medea package has already summarised what's going on in their README section Why Medea?; where their answer goes in depth with some sober roasting, I'll summarise:

The JSON Schema standard is complex

It [...] covers considerably more than simply validating JSON documents

JSON Schema requires arbitrary URI resolution

This reminds me of Bloodhound's support for ElasticSearch version 1 and 5 (and why the library doesn't support version 6, 7, or 8): The complexity and constant release of new ElasticSearch API versions makes it difficult to make a typed library around it. I'm not sure exactly how to phrase it, but Haskell seems like a bad fit for this type of highly volatile interface. Adding recursive, unbounded network I/O as part of the validation makes a Haskeller less likely to pursue a full implementation.

Never the less. I did a deep scan for the word "schema" on https://hackage.haskell.org/packages/ and procured the following list of JSON Schema-specific packages. They fall into one of two categories.

When a spec is too complex, a few things can happen

First, there's the "we tried and gave up" category; they all have in common that they're attempts to make a working library for an early version of JSON Schema, and they all have in common that they're abandoned; such is the fate of open source sometimes.

aeson-schema: Only Draft-03, recommends hjsonschema instead.
hjsonschema: Only Draft-04, announced deprecated in an attempt to be too modular.
jsonschema-gen, jsons-to-schema: Only Draft-04, neither does validate, generates schemas based on values, some limitations apply.

Then there are the libraries that appear to be JSON Schema-related libraries judging by their name, but they are, in fact, all variations that explicitly do not attempt to support JSON Schema, but build something similar with more limited assumptions:

json-schema: "It's haskell specific and has no relation to json-schema.org."
aeson-schemas: Last updated in 2022! Type-safe schema language using Template Haskell. But it doesn't come with an option to load a JSON Schema .json file. So they're schemas in JSON, but not JSON Schema schemas.
hschema-aeson: Last updated in 2022! A similar project that lets me specify schemas for Haskell data types and encode them as JSON. So they're schemas in JSON, but not JSON Schema schemas.
schematic: Last updated in 2021. "It can be thought of as a subset of JSON Schema", "Schematic schemas can be exported to json-schema".
medea: Last updated in 2021. "Medea is a schema language for JSON document structure. It is similar to JSON Schema, but is designed to be simpler and more self-contained."
(quick-schema: Last updated 2015. Minimalistic JSON schema language. Not maintained, and not exactly up to par.)

Summary

If you wish to mostly support a 10-12 year old draft of JSON Schema using an unmaintained package that currently fails to build in a modern build environment, you have two options: aeson-schema, and hjsonschema. While one recommends the other, and the other is self-deprecated, I'm not completely sarcastic when I say that aeson-schema could work. It appears well made, and you may want to support a super old JSON Schema definition before it got complex, or even extend it to a later draft.

If you wish for any coverage of Draft 2020-12, you're in bad luck. Haskellers simply gave up and wrote alternative JSON schema libraries. If you're not trying to release a JSON specification into the public domain, and you're just picking a good internal pipe language that supports JSON, any of aeson-schemas, hschema-aeson, schematic, and medea may be good choices. Or you may look at entirely different serialisation frameworks.

While I cannot currently prioritise evaluating each of these, since I am in the process of releasing a specification for my new pet project, JSON Flashcard, and I do need for a specification format and not just a library, I'm trying my luck with schematic, because it allows me to export to JSON Schema, without trying to support it fully.

Has-style traits in Rust

Simon Shine — Thu, 31 Mar 2022 06:37:01 +0000

I recently experimented with Rust trait instances on bounded generic types. This is the fancy way of saying impl<T: SomeTrait> AnotherTrait for T, i.e. any number of AnotherTrait instances created as a result of the presence of a SomeTrait instance.

Here is a similar but slightly more complicated example of that.

I have a number of things that all have a common base set of properties. I'd like for the common base set to be described with one trait, and all the ways in which the things are different to be described with another trait that may depend on the common things.

/// First, a simple base struct:
///
/// It contains things that are common to a set of structs.
#[derive(Debug, Clone)]
struct Base {
    foo: u64,
    bar: u64,
}

impl Base {
    fn new(foo: u64, bar: u64) -> Self {
        Self { foo, bar }
    }
}

/// Then, a trait that describes it
///
/// This is an abstract interface over the same struct. It allows us to
/// access other structs as if they were `Base`, as long as they have
/// some way to access a `Base`, e.g. by having a field of type `Base`.
trait BaseTrait {
    fn foo(&self) -> u64;
    fn bar(&self) -> u64;
}

/// Next, a trait that describes things that contain `Base`s
trait HasBase {
    fn get_base(&self) -> Base;
}

/// ...of which the simplest one is that `Base` has itself;
/// but to avoid infinite recursion, we `impl BaseTrait` for
/// `Base`.
impl BaseTrait for Base {
    fn foo(&self) -> u64 {
        self.foo
    }

    fn bar(&self) -> u64 {
        self.bar
    }
}

/// This is the crux:
///
/// A recursive trait instance that enables `BaseTrait`
/// calls on any thing that `HasBase` (i.e. has a `Base`).
impl<T: HasBase> BaseTrait for T {
    fn foo(&self) -> u64 {
        self.get_base().foo()
    }

    fn bar(&self) -> u64 {
        self.get_base().bar()
    }
}

/// Now, a `Thing` with a `Base`
#[derive(Debug, Clone)]
struct Thing {
    base: Base,
    baz: u64,
}

impl Thing {
    pub fn new(base: Base, baz: u64) -> Self {
        Thing { base, baz }
    }
}

/// If `BaseTrait` had many functions, we'd save a lot of lines here.
///
/// This `impl HasBase for Thing` implies `impl BaseTrait for Thing`.
impl HasBase for Thing {
    fn get_base(&self) -> Base {
        self.base.clone()
    }
}

/// So far, this is all great!
///
/// Now I want to make another trait that depends on BaseTrait
trait ExtensionTrait: BaseTrait {
    fn foo_bar_sum(&self) -> u64 {
        self.foo() + self.bar()
    }

    fn bar_baz_sum(&self) -> u64 {
        self.bar() + self.baz()
    }

    fn baz(&self) -> u64;
}

/// And what's more natural than give this trait to `Thing`
impl ExtensionTrait for Thing {
    fn baz(&self) -> u64 {
        self.baz
    }
}

fn main() {
    let some_thing = Thing::new(Base::new(2, 3), 5);

    println!(
        "foo = {}, bar = {}, baz = {}",
        some_thing.foo(),
        some_thing.bar(),
        some_thing.baz(),
    );

    println!(
        "foo + bar = {}, bar + baz = {}",
        some_thing.foo_bar_sum(),
        some_thing.bar_baz_sum()
    );
}

Writing code on doors

Simon Shine — Tue, 29 Mar 2022 18:18:06 +0000

Did you ever see pseudo-code in job ads that looked like

if ($you.like_to_work($here) === True) {
    $job.apply();
}

and thought to yourself a bunch of things about variables not being in scope, and whether their actual code has utterly specific library functions? Did you ever try to design an authentic piece of pseudo-code and found how difficult it is to remain concise enough to get the point across, but also idiomatic enough to even care expressing this as code?

At my local office, a sign was recently posted with instructions to lock the office.

Shortly after, another sign followed.

Analysis

Having policies listed on the door is a bit messy, but it gets the point across. Perhaps some of the mess can be reduced by analysing the process through which they ended up here, and produce a shorter version.

Growing an office policy is an iterative process, and two separate methods of adding new rules were used: Adding to the paper with the existing policy, and hanging up new paper. The fact that a new rule is added with a pen is not actually a problem, but is the greatest feature of paper tech that we have yet to reinvent elsewhere.

Both methods introduce redundancy: The hand-written "Turn off coffee machine" and all but one of the printed bullets are listed twice. Since the lists are short, it is easy to determine what the key point is: Start the dishwasher before weekends begin.

Being new to Rust, I wanted to write the stupid pseudo-code version of this. My first iteration simply tried to reduce code duplication by making the weekend-dishwasher rule conditional:

impl Shutdown for Office {
    fn leave(&mut self, now: DateTime<Local>) {
        self.lights(Off);
        self.lock(On);
        self.coffee_machine.stop();

        if now.weekday() == Weekday::Fri {
            self.dishwasher.start();
        }
    }
}

The basic thing I like about this code is that it looks like Rust, but doesn't go much into what a Shutdown or an Office actually is.

Ambiguity

The lovely and sometimes frustrating thing about human language is that it leaves room for ambiguity. You can say multiple things at once, say different things to different people with one message, and leave room for interpretation.

The moment you write human instructions as programming code, the ambiguity becomes glaringly apparent. Once I had read the instructions as programming code, I immediately thought, "Wait, what if Friday is a bank holiday?" I actually want a softer definition of weekend.

Abusing ambiguity

So wait...

I'm starting the dishwasher on Fridays, no matter what, just because it's weekend? This seems like an X-Y problem. I know how dishwashers work. We don't want to accumulate smell over the weekend, and we like to tidy our workspace so that arriving to it gives you energy. But we also don't want to pointlessly start dishwashers on Fridays. What are the actual criteria?

impl Shutdown for Office {
    fn leave(&mut self, now: DateTime<Local>) {
        self.lights(Off);
        self.lock(On);
        self.coffee_machine.stop();

        if now.is_beginning_of_weekend() && self.dishwasher.is_full() {
            self.dishwasher.start();
        }
    }
}

So wait...

We only start the dishwasher on Fridays if it's full?
What if it's half-full, or it's full and isn't a Friday?
What if the dishwasher has already started? Do we assume that .start() is idempotent, do we stop the machine just to start it again, or do we busy-wait for the machine to stop just to start it a second time? It is Friday, after all!

A problem of joining the two pieces of paper was that they carried different implicit assumptions; joining them, the absence of any non-Friday dishwasher logic should not imply a hard rule against using the dishwasher on non-Fridays. We could be more explicit about overlapping event handlers, but that sounds like boilerplate code.

You start empty dishwashers?

It was at this time that I changed the policy and let go of the "Dishwasher Friday" rule. I know, I know. Cleaning nazi! Party pooper! But really, as these rules are explicated further, I am sure that people will appreciate how rarely they actually have to start the dishwasher once they wield the powers of formal logic.

Let's not start empty dishwashers just because we can. And let's add soap.
A lady came by and suggested that we also want to turn off the AC when we leave.
Since we're programming, and Rust has sequential evaluation semantics, let's not turn off the lights and lock the door before we've performed the tasks that are better performed with unimpaired vision and being able to leave.

impl Shutdown for Office {
    fn leave(&mut self) {
        self.air_condition(Off);
        self.coffee_machine.stop();

        if !self.dish_washer.is_empty() &&
           !self.dishwasher.is_running() &&
           !self.dish_washer.is_complete()
        {
            self.dish_washer.add_soap();
            self.dish_washer.start();
        }

        self.lights(Off);
        self.lock(On);
    }
}

Implement Rust trait for all types that have another trait

Simon Shine — Tue, 15 Feb 2022 01:30:04 +0000

Rust traits are powerful. A particularly neat thing you can do is implement a trait for several types at once. To give a simple example, a trait for squaring numbers mainly depends on multiplying them:

trait Square {
    fn square(&self) -> Self;
}

We could implement this trait for a single type like so:

impl Square for u64 {
    fn square(&self) -> Self {
        self * self
    }
}

But instead of adding impl Square for ... for each type we want to .square(), this impl can be generalised to all types that impl Mul:

impl<T: Mul<Output = T> + Copy> Square for T {
    fn square(&self) -> Self {
        *self * *self
    }
}

The key here is impl<T: ...> Square for T { ... }, that is, for T, the type that the impl takes as parameter. When eventually the constraints on T become too many, they can be moved to a where clause:

impl<T> Square for T
where
    T: Mul<Output = T> + Copy,
{
    fn square(&self) -> Self {
        *self * *self
    }
}

Here's an example of .square() being used for multiple types that already impl Mul:

fn main() {
    println!(
        "{}, {}, {}, {}",
        10u8.square(),
        3u16.square(),
        4u32.square(),
        5u64.square()
    );
}

A Terraform DNS moment

Simon Shine — Sat, 29 Jan 2022 02:53:40 +0000

tl;dr: I stuck the IPv4 and IPv6 address of my VPS into an SPF record field text.

I manage my DNS with Terraform. This means that past domain registration and changing nameservers, I manage DNS with the Terraform command-line interface and git. Today I experienced a moment of happiness when two pieces of personal infrastructure fitted.

The scenario: I'm setting up a send-only email server on a VPS for Gitea notifications. If this works out well, I will consider using it for a newsletter. Doing this on a commercial cloud VPS means that my server's IPv4 address gets a bad score. One way to improve on that score is to add an SPF record.

For my personal email hosted at FastMail, the SPF record is simpler:

$ dig +short TXT simonshine.dk
"v=spf1 include:spf.messagingengine.com -all"

Their SPF record contains a bunch of ip4:... rules.

To let the world know that I'm sending emails from my VPS, I simply list its IP address:

resource "digitalocean_record" "spf_example_com" {
  domain = digitalocean_domain.example_com.name
  type   = "TXT"
  name   = "@"
  value  = "v=spf1 ${format("ip4:%s ip6:%s",
    digitalocean_droplet.example_droplet.ipv4_address,
    digitalocean_droplet.example_droplet.ipv6_address)} -all"
}

This encourages receivers of emails from this domain to reject the email if it didn't originate from this server. The part that got me blogging is that I didn't hardcode the IP addresses of my server, but rather, the value was taken from the deployment configuration of the concrete server.

Fiat currencies are not one thing

Simon Shine — Fri, 10 Dec 2021 12:10:33 +0000

When we discuss the potential future usefulness of cryptocurrency, we compare against existing use-cases of fiat currency. Given a particular fiat currency -- its name, its unit, and its exchange rate -- our language enforces us to think of it as one currency.

This is a misconception that will lead to poorly designed alternatives with strong, but unaligned monetary properties at the wrong time. If we don't think of a fiat currency as many currencies within one brand, then any attempt to re-model those use-cases with cryptocurrencies will come short of being practically useful.

The classic shapes of fiat currencies are:

Physical cash money
Credit card money
Bank loan money
Company money
Point system money
New digital cash money

The reason why thinking of these as separate currencies within the same brand is that they do not have the same monetary properties, or even the same purposes, at all. And if you consider the cost of exchanging one shape for another as part of the exchange rate, they don't even share that.

When Bitcoin were still getting popular, the following six historic monetary properties were used to justify being "real money":

Scarcity (has limited supply)
Durability (doesn't break)
Acceptability (can pay with)
Portability (can carry)
Divisibility (can buy big/small things)
Fungibility (all coins are the same)

Now cryptocurrencies are immensely popular, but still mostly useless. While these monetary properties have gradually become true for money historically, cryptocurrencies generally lack some of them; in particular, mainstream acceptability and practical durability (recover lost funds).

We could write that off as work in progress. But why bother changing currencies if all you get is the same, but with fewer consumer guarantees (e.g. conditional transaction reversibility) and a worse user experience?

Perhaps, like not all historic money had all desirable monetary properties in the same degree, or at all, the newer monetary properties of cryptocurrencies have yet to be understood and valued because they still fail basic usability.

jq hack #2: curl'ing the right binary on GitHub

Simon Shine — Sat, 04 Dec 2021 17:53:45 +0000

Something I see quite often is a combination of grep and cut to extract URLs from JSON objects, for example:

$ curl -s https://api.github.com/repos/go-gitea/gitea/releases/latest \
  | grep browser_download_url \
  | cut -d '"' -f 4 \
  | grep '\linux-amd64$' \
  | wget -i -

This is neat because grep and cut are almost always available on Linux. And the approach works for all kinds of plaintext documents with a semi-expectable format, not just JSON, so the approach is worth keeping around. But for JSON in particular, this approach is unnecessarily fragile because of a few unreasonable assumptions:

Whitespace formatting: If, for example, GitHub decides to start compacting their JSON output, this script would stop working, since it assumes "browser_download_url" is on a separate line with its URL value.
JSON escape sequences: If, for example, the queried output contains JSON escape sequences, then those will not be evaluated, so \u.... would become six characters rather than one Unicode character.

For a jq version that is stable across legal JSON syntax variation, one could do this instead:

$ curl -s https://api.github.com/repos/go-gitea/gitea/releases/latest \
  | jq -r '.assets[]
           | .browser_download_url
           | select(endswith("linux-amd64"))'
https://github.com/go-gitea/gitea/releases/download/v1.15.7/gitea-1.15.7-linux-amd64

That's the tl;dr.

Explaining how this query works

curl -s only prints the file contents, not debug information about the download.
jq -r outputs plaintext instead of JSON if and only if the object you're printing is a JSON string.
.assets[] enters the "assets" sub-field and splits the list it contains into multiple results,
.browser_download_url reduces each of those multiple results to its sub-field,
select(endswith("linux-amd64")) removes results that don't match.

Here is a variation of the query that produce the same result:

$ curl -s https://api.github.com/repos/go-gitea/gitea/releases/latest \
  | jq -r '.assets
           | map(.browser_download_url
                 | select(endswith("linux-amd64")))
           | .[]'
https://github.com/go-gitea/gitea/releases/download/v1.15.7/gitea-1.15.7-linux-amd64

The difference is that here, .assets instead of .assets[] does not split the sub-field into multiple results. Instead, map(...) runs ... on each element in the one result being a list (and not the many results being the list's elements).

The query itself in both cases is

.browser_download_url | select(endswith("linux-amd64"))

which first turns each list element into its sub-field, being only the URL, and then removes that URL from the inside of the list if it isn't the right one. Lastly, .[] takes the list of remaining URLs and turns them into multiple string results.

This map() variation is worth considering because it also lets us filter based on criteria made on the full asset object, and not just the URL itself. For example, to remove any uploads from the result that were not made by the GiteaBot user,

$ curl -s https://api.github.com/repos/go-gitea/gitea/releases/latest \
  | jq -r '.assets | map(select(.uploader.login == "GiteaBot")
                         | .browser_download_url
                         | select(endswith("linux-amd64")))
                   | .[]'
https://github.com/go-gitea/gitea/releases/download/v1.15.7/gitea-1.15.7-linux-amd64

Exploring a big JSON document with jq/less

In the process of deriving the final jq query, I relied on using jq and less as an ad-hoc JSON browser using the following steps:

$ curl -s https://api.github.com/repos/go-gitea/gitea/releases/latest \
  | jq -C . | less -R

Thought: So there's an .assets list, and .browser_download_url is a direct child field in each underlying asset object... how many elements does this .assets have?

$ curl -s https://api.github.com/repos/go-gitea/gitea/releases/latest \
  | jq '.assets | length'
72

Thought: And how many assets are actually related to "linux-amd64"?

$ curl -s https://api.github.com/repos/go-gitea/gitea/releases/latest \
  | jq -r '.assets[] | .browser_download_url | select(contains("linux-amd64"))'
https://github.com/go-gitea/gitea/releases/download/v1.15.7/gitea-1.15.7-linux-amd64
https://github.com/go-gitea/gitea/releases/download/v1.15.7/gitea-1.15.7-linux-amd64.asc
https://github.com/go-gitea/gitea/releases/download/v1.15.7/gitea-1.15.7-linux-amd64.sha256
https://github.com/go-gitea/gitea/releases/download/v1.15.7/gitea-1.15.7-linux-amd64.xz
https://github.com/go-gitea/gitea/releases/download/v1.15.7/gitea-1.15.7-linux-amd64.xz.asc
https://github.com/go-gitea/gitea/releases/download/v1.15.7/gitea-1.15.7-linux-amd64.xz.sha256

Hvorfor må jeg ikke eje en svensker?

Simon Shine — Sun, 28 Nov 2021 16:04:13 +0000

Written January 1, 2003

The following essay is an adaptation of a letter titled Why Can’t I Own a Canadian? sent to the religious radio host Dr. Laura Schlessinger in October 2002, as a response to a statement that homosexuality is an abomination according to Leviticus 18:22, and cannot be condoned under any circumstance. This Danish adaptation was originally posted by me to the Atheist Society of Denmark web forums, but was later lost from there and copied without attribution.

Kære kristen

Tak fordi du gør så meget for at prædike Guds ord. Jeg har lært en masse ved at gå i kirke, og jeg prøver at dele min viden med så mange som muligt. Når nogen for eksempel forsvarer homoseksualitet henviser jeg dem bare til 3. Mosebog kapitel 18 vers 22 der siger at du ikke må have samleje med en mand, som man har samleje med en kvinde. Det er en vederstyggelighed. Diskussion slut. Jeg har dog nogle spørgsmål om nogle andre love og hvordan man bør følge dem:

Når jeg brænder en tyr ved alteret, ved jeg ifølge 3. Mosebog kapitel 1 vers 9 at det skaber en liflig duft for Herren. Problemet er imidlertid mine naboer, der siger at de ikke kan lide lugten. Skal jeg dræbe dem som Bibelen foreslår?

Jeg har overvejet at sælge min datter væk som slave som stadfæstet i 2. Mosebog kapitel 21 vers 7: Når en mand sælger sin datter som trælkvinde, skal hun ikke frigives på samme måde som trællene. Hvad ville en fair pris være for hende nu om dage?

Jeg ved at jeg ikke må have kontakt med kvinder i deres menstruationsperiode som skrevet i 3. Mosebog kapitel 19-24 (Hun er uren i syv dage, hvis jeg rører hende eller ved noget hun har rørt skal jeg vaske mig og er uren indtil om aftenen, alt hvad hun rører ved er urent og jeg må heller ikke have samleje med hende.) Problemet her er bare, hvordan ved jeg om kvinder menstruerer? Jeg har prøvet at spørge, men de fleste kvinder bliver fornærmede.

Jeg må ifølge 3. Mosebog kapitel 25 vers 44 godt eje slaver: De trælle og trælkvinder, du vil have, kan I købe af folkene omkring jer. Både mænd og kvinder, hvis bare de er købt fra nabolande. En af mine venner siger at det er rigtig nok for polakker og pakistanere, men ikke for svenskere. Kan du uddybe? Hvorfor må jeg ikke eje en svensker?

Min nabo arbejder i Aldi, der som måske bekendt har åbent om søndagen.. 2. Mosebog kapitel 35 vers 2 siger klart at I seks dage må der udføres arbejde, men den syvende dag skal være hellig for jer; I skal holde fuldstændig hvile for Herren. Enhver, der udfører arbejde på den dag, skal lide døden. Er jeg moralsk forpligtet til at dræbe ham og alle andre der arbejder i Aldi? Yderligere siger den at I ikke må tænde ild på sabbatsdagen, hvor end I bor, og jeg ved med sikkerhed at min nabo laver varm mad om søndagen.

En af mine venner siger at selvom det er en vederstyggelighed at spise skaldyr (3. Mosebog kapitel 11 vers 10), er det en mindre vederstyggelighed end at være homoseksuel. Jeg er ikke enig. Kan du afgøre dette for os?

I 3. Mosebog kapitel 21 vers 20 står at pukkelryggede, dværge, folk med dårligt syn eller folk der har fnat eller beskadigede testikler ikke må nærme sig Guds alter. Jeg må indrømme at mit syn ikke er helt fejlfrit, men er der plads til lidt albuerum her? Ifølge vers 19 må mænd med brækkede arme heller ikke, så jeg har bedt min bror om at blive hjemme fra kirke de sidste tre måneder.

De fleste af mine mandlige venner får deres hår klippet, inklusiv ved deres tindinger, selvom dette eksplicit er forbudt i 3. Mosebog: I må ikke klippe håret kort, og du må ikke studse skægget. Hvordan skal mine venner dræbes?

Jeg ved fra 3. Mosebog kapitel 11 vers 6-8 at jeg ikke må røre ved en død gris’ hud, men må jeg godt spille fodbold alligevel hvis jeg bærer handsker?

Min onkel har en bondegård. Han overskrider 3. Mosebog kapitel 19 vers 19 ved at tilså to slags korn på sin mark, og det gør hans kone også ved at bære tøj, der er vævet af to slags garn. Deres søn bander også en del. Er det virkelig nødvendigt at besvære sig med at samle hele byen for at stene dem? (3. Mosebog kapitel 24 vers 10-16) Kunne vi ikke bare brænde dem til døde som et andet privat familieforetagende ligesom vi gør med folk der har samleje med deres svigermoder? (3. Mosebog kapitel 20 vers 14)

Jeg ved at du som kristen har studeret disse og mange andre problemstillinger som Bibelen bringer os, og jeg er sikker på at du kan hjælpe. Tak for at minde os om at Guds ord er evigt og uforanderligt.

Hilsen Simon

Why rewriting version control history matters

Simon Shine — Thu, 25 Nov 2021 22:54:41 +0000

Life is understood backwards, but must be lived forwards.
--- Søren Kierkegaard

A summary on good commit messages

Because version control systems, like git and GitHub, are at the heart of modern software development workflows, most workflows are heavily impacted by and defined in terms of how to operate and cooperate around version control.

Making use of issue trackers, pull requests, review tools, and commit messages, there are many right ways to distribute the communication that revolves around a change in source code. So...

Why good commit messages?

Code is read much more often than it is written. As Raymond Chen says:

Even if you don't intend anybody else to read your code, there's still a very good chance that somebody will have to stare at your code and figure out what it does: That person is probably going to be you, twelve months from now.

The same reason extends to commit messages. As Chris Beans says:

... a well-crafted Git commit message is the best way to communicate context about a change to fellow developers (and indeed to their future selves). A diff will tell you what changed, but only the commit message can properly tell you why.

When an entire commit message consists of messages like "wip", "small fix", or "added code", Peter Hutterer suggests this is due to a lack of training in reading code:

If you haven't given much thought to what makes a great Git commit message, it may be the case that you haven't spent much time using git log and related tools. There is a vicious cycle here: because the commit history is unstructured and inconsistent, one doesn't spend much time using or taking care of it. And because it doesn't get used or taken care of, it remains unstructured and inconsistent.

The cost of rewriting under collaboration

When discovering the solution to a problem, the explored path is rarely the shortest path. Placing good explanations in git history makes it ideal for being read, but there are several costs when one has to "go back" and place them while at the same time collaborate with others:

Cost of learning new tools: Rewriting git history requires some of the more difficult git commands, but there are simple approaches, too:
- GitHub lets you squash pull requests; in the web UI, you can squash all commits into a single one and edit the commit message in a textarea. This works best when your pull requests are small.
- git commit --fixup and git rebase -i --autosquash are easy to learn and use. Read the --fixup & --autosquash guide by Florent Lebreton.
- git reset --soft lets you re-commit everything anew. Read Git Basics: Rewriting a branch's commit history from scratch by Igor Marques da Silva.
- You can always create a local copy of a branch before attempting to rewrite history. You can even view the history of your copy (using git log <branch> and git show <hash>) while simultaneously rebuilding the history (using git add -p and git commit -v).
Cost of wording: The time and patience to write a good explanation, and the expectation that it will eventually be read. Writing messages to your future self can work as a legitimate motivator.
Cost of merge conflicts: When rewriting history and force-pushing to a remote branch, this forces collaborators on that branch out of sync by removing conflict-free merge paths, and it risks overwriting contributions.
- Clearly, push --force is bad, mm'kay?! Fortunately, Steve Smith has something to say:
The problem here is that when doing a force push Bob doesn’t know why his changes have been rejected, so he assumes that it’s due to the rebase, not due to Alice’s changes. This is why --force on shared branches is an absolute no-no; and with the central-repository workflow any branch can potentially be shared.

But --force has a lesser-known sibling that partially protects against damaging forced updates; this is --force-with-lease.

What --force-with-lease does is refuse to update a branch unless it is the state that we expect; i.e. nobody has updated the branch upstream. In practice this works by checking that the upstream ref is what we expect, because refs are hashes, and implicitly encode the chain of parents into their value.

Even when your collaborator is only reviewing or evaluating code, and not pushing changes, a merge conflict caused by force-pushing can cause extra time spent syncing. If this bothers a collaborator, perhaps rewriting history can happen only before and after their role is fulfilled.

The explorative path might truthfully involve a lot of "small fix", "trying", and "oops" steps, but these are not interesting in 6 months. We may be interested in more than perfect hindsight, so leaving a trail of comments in an issue tracker, in a pull request thread, or in a code review is a way to deepen the understanding.

How to write Good Commit Messages: Quick Reference

How to write a git commit message, by Chris Beams, focuses on the formatting:

Separate subject from body with a blank line

Limit the subject line to 50 characters,

Capitalize the subject line

Do not end the subject line with a period

Use the imperative mood in the subject line

Wrap the body at 72 characters

Use the body to explain what and why vs. how

On commit messages, by Peter Hutterer, focuses on the what and the why:

Why is it necessary? It may fix a bug, it may add a feature, it may improve performance, reliabilty, stability, or just be a change for the sake of correctness.

How does it address the issue? For short obvious patches this part can be omitted, but it should be a high level description of what the approach was.

What effects does the patch have? (In addition to the obvious ones, this may include benchmarks, side effects, etc.)

Some real-world examples

Imagine receiving a pull request with the following commits:

commit edc9ff5febb698a05f68650d6d58828202c24daf
Author: John Doe <john.doe@example.com>
Date:   Mon Nov 22 16:23:24 2021 +0100

    WIP

This message is clearly a working title that should be rephrased when the author is comfortable. Sometimes you want to commit and push half-done work, and sometimes you want to wait with providing a good explanation.

When collaborating, establish a standard way of saying "don't do X yet," where X could be look at, review, merge, contribute to, or any other activity that creates potential conflicts between contributors. Typically the word "WIP" means that.

This warrants git history rewriting before merging.

commit 11f8a068bec49abdddd7bdf203b77ecbfe48451f
Author: John Doe <john.doe@example.com>
Date:   Thu Nov 4 10:19:40 2021 +0100

    Added cronjob

Added what cronjob, and why?

commit da8362e0b43ea3b18013f9245a614cf00ce43b60
Author: John Doe <john.doe@example.com>
Date:   Thu Nov 4 12:49:43 2021 +0100

    fixup! Added cronjob

The author is signalling that they intend to squash the change. With this intent in mind, this clearly warrants git history rewriting before merging.

commit 57e0b283f69c576db48003dc1ec3b2f492f79540
Author: John Doe <john.doe@example.com>
Date:   Mon Nov 22 14:30:12 2021 +0100

    small fix

This commit should either be squashed into some other commit, or explain what small fix this is. This either warrants rewriting the commit message, or squashing the commit.

commit 438cc1ce87dec47081b75d4cbd6ce4abca40b8c0
Author: John Doe <john.doe@example.com>
Date:   Thu Oct 7 19:42:13 2021 +0200

    Added indexed "cachedStatus" field on KnowledgeNodeStore which is automatically filled with the value of KnowledgeNode->data['status']['state']. This might help the speed of common KnowledgeNodeQuery calls. Migration included that needs to run before we can remove the old filtering logic in KnowledgeNodeQuery.

Yes, that is a 308-character subject field. Besides coming up with a short, descriptive title, and arguably word-wrapping, this commit message does detail what and why rather than how. The included migration that is mentioned could be split into a separate commit.

DEV Community: Simon Shine

JSON Schema in Haskell using AutoDoCodec

Step 1: Define your data type and codec

Step 2: Generate the JSON schema

Step 3: Validate JSON against the schema

Adding validation constraints

Numeric constraints

String length constraints

Updated codec with constraints

Minimal Sufficient Groups

The problem

My solution

Why you should be careful with the Default trait/typeclass

Default in Rust and in Haskell

Defaults are useful, but Default is sketchy

What is the default integer?

Does your Default do anything?

Rust: A Default case-study

Summary

A review of JSON Schema libraries for Haskell

When a spec is too complex, a few things can happen

Summary

Has-style traits in Rust

Writing code on doors

Analysis

Ambiguity

Abusing ambiguity

You start empty dishwashers?

Implement Rust trait for all types that have another trait

A Terraform DNS moment

Fiat currencies are not one thing

jq hack #2: curl'ing the right binary on GitHub

Explaining how this query works

Exploring a big JSON document with jq/less

Hvorfor må jeg ikke eje en svensker?

Why rewriting version control history matters

A summary on good commit messages

The cost of rewriting under collaboration

How to write Good Commit Messages: Quick Reference

Some real-world examples

`Default` in Rust and in Haskell

Defaults are useful, but `Default` is sketchy

Does your `Default` do anything?

Rust: A `Default` case-study