DEV Community: Tomáš Zemanovič

State machine testing with Proptest

Tomáš Zemanovič — Tue, 28 Mar 2023 00:00:00 +0000

I’m excited to share the state machine testing support for Proptest crate that I’m trying to contribute back into upstream 🧡 and to include the contents of this post as a guide in the Proptest Book.

Rust is very practical. It allows me to stay productive by guiding me through code with its type-checker and borrow-checker. In general though, more code means more opportunities for bugs to sneak in and I often surprise myself with just how many of them I miss when I pretend that I can interpret code in my head. As the old wisdom goes, types are no substitute for tests (even in e.g. pure functional languages with more expressive types) and Rust has some great facilities for testing. I love using Proptest as it enables me to test code thoroughly and go through cases that I wouldn’t have thought of myself.

I’ve been using the state machine tests in Namada project. I’m all for OSS development, knowledge sharing to help others to form their opinion and improving the general state of the software development industry (don’t we all regularly encounter issues in software), but please don’t mistake this for an endorsement of blockchains.

Before this gets through 🤞, you can find the branch of Proptest that contains the state machine testing feature here https://github.com/proptest-rs/proptest/pull/310.

Cicindela Aurulenta by Portioid / CC BY-SA 2.0

Before using state machine testing, it is recommended to be at least familiar with the basic concepts of Proptest itself as it’s built on its essential foundations. That is:

Strategies are composed and used to generate inputs to a state machine test.
A test will attempt to shrink the generated transitions sequence on a discovery of a case that breaks some properties.
It will capture regressions file with a seed that can be used to deterministically repeat the found case.

Be cautious that it may not be fully self-explanatory how to use it from its API and there are some opportunities for paper cuts. What follows is a more detailed description that will guide you through it.

In short, use AbstractStateMachine and StateMachineTest to implement your state machine test and prop_state_machine! macro to run it.

If you just want to get started quickly, take a look at one of the examples:

state_machine_heap.rs - a simple model to test an API of a heap data structure
state_machine_echo_server.rs - a more advanced model for an echo server with multiple clients talking to it

To see what transitions are being applied in standard output as the state machine test executes, run these with e.g. PROPTEST_VERBOSE=1 cargo run --example state_machine_heap.

The state machine testing support for Proptest is heavily inspired by the Erlang’s eqc_statem (see the paper Finding Race Conditions in Erlang with QuickCheck and PULSE) with some key differences. Most notably:

Currently, only sequential strategy is supported, but a concurrent strategy is planned to be added at later point.
There are no “symbolic” variables like in eqc_statem. The state for the abstract state machine is separate from the concrete state machine.
The post-conditions are not defined in their own function. Instead, they are part of the apply_concrete function.

When to use State Machine testing?

State machine testing automates the checking of properties of a system under test (SUT) against an abstract state machine definition. It does this by trying to discover a counter-example that breaks the defined properties of the system and attempts to shrink it to a minimal sequence of transitions that still reproduce the issue.

State machines are a very useful abstraction for reasoning about code. Many things from low-level to high-level logic and anywhere in between can be modelled as a state machine. They are very effective for modelling effectful code, that is code that performs some state changes that can be too hard to test thoroughly with a more manual approach or too complex to verify formally.

Some fitting examples to give you an idea include (by no means exhaustive):

A data structure with an API that mutates its state
An API for a database
Interactions between a client(s) and a server

There is some initial investment needed to set the test up and it usually takes a bit more time to run than simple prop tests, but if correctness is important for your use case, you’ll be rewarded with a test that is so effective at discovering bugs it might feel almost magical, but as you’ll see, you could have easily implemented it yourself. Also, once you have the test setup, it is much easier to extend it and add new properties to check.

How to use it

State machine testing is made up of two parts, an abstract state machine definition that drives the inputs to a test and a test definition for a SUT that replicates the same transitions as the abstract state machine to find any possible divergence or conditions under which the defined properties (in here post-conditions and invariants) start to break.

Abstract state machine strategy

You can get started with state machine testing by implementing trait AbstractStateMachine, which is used to drive the generation of a sequence of transitions and can also be compared against the state of the SUT. At the minimum, this trait requires two associated types:

type State that represents the state of the abstract state machine.
type Transition with possible transitions of the state machine. This is typically an enum with its variants containing input parameters for the transitions, if any.

You also have to implement three associated functions:

To initialize the abstract state machine:
To generate transitions:
To apply the given transition on the abstract state:

Additionally, you may want to override the default implementation of:

fn preconditions(state: &Self::State, transition: &Self::Transition) -> bool

By default, this simply returns true, which implies that there are no pre-conditions. Pre-conditions are a way of restricting what transitions are valid for a given state and you’ll only need to restrict the transitions whose validity depends on the current state. This ensures that the abstract state machine will only produce and shrink to a sequence of valid valid transitions. It may not be immediately apparent that the current state may be affected by shrinking. With the example of selecting of keys of a hash map for fn transitions, you’ll need to check that the transition’s key is still present in the hash map, which may no longer be true after some shrinking is applied.

You can either implement AbstractStateMachine for:

A data structure that will represent your abstract state machine and set the associated type State = Self; or
An empty struct, which may be more convenient than making a wrapper type if you’re using a foreign type for the type State

Definition of a state machine test

With that out of the way, you can go ahead and implement StateMachineTest. This also requires two associated types:

type ConcreteState which is the type that represents the SUT.
type Abstract with the type for which you implemented the AbstractStateMachine.

There are also three associated functions to be implemented here (some types are slightly simplified for clarity):

Initialize the concrete state:
Apply the transition on the concrete state:
Check properties that apply in any state:

Make the state machine test runnable

Finally, to run the StateMachineTest, you can use the prop_state_machine! macro. For example:

prop_state_machine! {
  #[test]
  fn name_of_the_test(sequential 1..20 => MyStateMachineTest);
}

You pick a name_of_the_test and a single numerical value or a range after the sequential keyword for a number of transitions to be generated for the state machine execution. The MyStateMachineTest is whatever you’ve implemented the StateMachineTest for.

And that’s it. You can run the test, perhaps with cargo watch as you develop it further, and see if it can find some interesting counter-examples to your properties.

Extra tips

Because a state machine test may be heavier than regular prop tests, if you’re running your tests in a CI you may want to override the default proptest_config’s cases to include fewer cases in a single run. You can also use PROPTEST_CASES environment variable and during development it is preferable to override this to run many cases to get a better chance of catching those pesky ~~bugs~~ , erm, defects.

Given that there are thought to be in the region of another four million species that we have not yet even named, there is no doubt that scientists will be kept happily occupied studying them for millennia, so long as the insects remain to be studied. Would the world not be less rich, less surprising, less wonderful, if these peculiar creatures did not exist?

– Dave Goulson, Silent Earth

So let’s leave bugs alone and only squash defects instead!

Because the output of a failed test case can be a bit hard to read, it is often convenient to print the transitions. You can do that by simply setting the proptest_config’s verbose to 1 or higher. Again, if you don’t want to keep this in your test’s config or if you’d prefer to override the config, you could also use the PROPTEST_VERBOSE environment variable instead.

Another helpful config option that is good to know about is timeout (PROPTEST_TIMEOUT via an env var) for tests that may take longer to execute.

How does it work

This section goes into the inner workings of how the state machine is implemented, omitting some less interesting details. If you’re only interested in using it, you can consider this section an optional read.

The AbstractStateMachine::sequential_strategy sets up a Sequential strategy that generates a sequence of transitions from the definition of the AbstractStateMachine. The acceptability of each transition in the sequence depends on the current state of the state machine and AbstractStateMachine::preconditions, if any. The state is updated by the transitions with the AbstractStateMachine::apply_abstract function.

The Sequential strategy is then fed into Proptest like any other strategy via the prop_state_machine! macro and it produces a Vec<Transition> that gets passed into StateMachineTest::test_sequential where it is applied one by one to the SUT. Its post-conditions and invariants are checked during this process and if a failing case is found, the shrinking process kicks in until it can shrink no longer.

The shrinking strategy which is defined by the associated type Tree = SequentialValueTree of the Sequential strategy is to iteratively apply Shrink::InitialState, Shrink::DeleteTransition and Shrink::Transition (this can be found in proptest/src/strategy/state_machine.rs):

We start by trying to delete transitions from the back of the list until we can do so no further (the list has reached the min_size - that is the variable that gets set from the chosen range for the number of transitions in the prop_state_machine! invocation).
Then, we again iteratively attempt to shrink the individual transitions, but this time starting from the front of the list from the first transition to be applied.
Finally, we try to shrink the initial state until it’s not possible to shrink it any further.

The last applied shrink gets stored in the SequentialValueTree, so that if the shrinking process ends up in a case that no longer reproduces the discovered issue, the call to complicate in the ValueTree implementation of the SequentialValueTree can attempt to undo it.

Closing thoughts

If you have some use case where state machine testing can be applied I hope that this guide will make it easier to get you started. As always, I’d appreciate any feedback and if you have any issues with using this Proptest feature or some interesting exhibit discovered with its help, please don’t hesitate to share it!

Rust coding style

Tomáš Zemanovič — Fri, 03 Mar 2023 00:00:00 +0000

This post is about a high-level Rust coding style (as in it doesn’t go into specific details), partly inspired by data-oriented design and partly by ML-based functional programming languages, which imho fits quite naturally into Rust and makes for ergonomic, flexible and easily extensible APIs. It’s nothing advanced, but I hope this would be useful for people coming to Rust, perhaps with some background in one of the common OOP languages.

I’ll try to sum up the main points into a handful of guiding principles, highlighted in block quotes for easy skim reading. If you don’t agree with something, I invite you to read the supporting arguments and if you still don’t agree I’d love to hear your opinion.

Data-oriented design originated in games development and is very well suited for performance and opens the door to better optimization. When you pick Rust over some higher-level language in which you, for example, don’t need to think about ownership, you probably care about performance and might want to optimize your code at some point and even if you don’t, you’ll reasonably expect good out-of-box performance. But of course, there are many other great reasons to use Rust too, not least its great community!

Somewhat analogously to the rule of least power, I think that fns with struct, enum , type aliases and mods are not only sufficient for most things we commonly do but sticking to them is advantageous. In most modern computer architectures, data layout and locality are of the utmost importance for performance.

Pay a lot of attention to your data structures, they have a huge impact on what you’ll be able to do.

It’s not just for performance sake, data structures are very helpful in getting a good understanding of the problem and implementing a solution in a clear, readable and maintainable way. To quote Mike Acton “if you don’t understand the data, you don’t understand the problem”.

In functional programming, we take advantage of the fact that sum types (enums in Rust) allow you to express data of arbitrary cardinality (number of possible variants), so you can:

Make invalid states impossible to represent with your data types.

If it’s not clear how to do that, there are great articles and talks you can find on the topic.

A very simple example: Say you have a type (or arguments to a function) in which you want to have either A, B, or neither of them. If you simply use two Option types (NotXorOpts below), it’s possible to represent a value with both of A and B present. But you can very quickly define a custom enum type that rules that out:

struct A;
struct B;

type NotXorOpts = (Option<A>, Option<B>);

enum XorOpts {
  A(A),
  B(B),
  Neither,
}

The essence of it is to think of types as sets. In particular, Rust enums act as closed sets - open sets can be represented with traits. Often, traits are used in places where closed sets are sufficient.

Prefer to only use traits if and when you need open sets.

That is, only use them when you don’t know or cannot forsee all the possible variants.

You can often see encapsulation being achieved by keeping fields of data structures private, which looks satisfying because it can prevent the consumer from doing modifications that would break some abstractions. In a struct, fields are private by default so you do get directed into this, but there are other ways to achieve encapsulation. Here, I want to advocate:

Never hide data fields of types that appear in your public API.

When you’re using a crate and its abstraction doesn’t exactly fit with what you’re doing, having all public data types may allow to make it suit your use case without having to wrangle it with some transmutations, fork it or worse abandon it altogether. As we know, pretty much all the abstractions in software are leaky. Furthermore, when you do care about the layout of your data and some abstraction is trying to hide it from you, it is only getting in the way of that.

Instead of putting your invariants on data types, place them on functions, document them, and use debug_assert!s generously.

Another pain point which might ruin your day is that when you follow the data hiding encapsulation technique, you’ll be limited in what you can do when you’re using a method that requires you to mutably borrow the data you’re working with (&mut self). Before the mutable borrow is returned (to avoid any confusion, I do not mean the keyword return, but rather the moment when the mutable reference of the thing that’s being borrowed is released), you will not be able to call other methods on it that require immutable borrowing (&self), even if you can reason yourself that the fields that are being mutated are different from the fields that you want to access immutably. While there are some proposals on how to allow such code to be expressed, when the fields are public, you’re all good to partially borrow them directly! Simpler and easier.

Let’s look at an example that may be given as a motivation for hiding data to achieve encapsulation. Say we have some list of integers, which can get large and we’ll want to often use its average without having to recompute it every time, so we might want to store it with the data. In an OOP way, this may looks like this:

#[derive(Default)]
struct AvgVec {
  data: Vec<i64>,
  avg: i64,
}

impl AvgVec {
  pub fn get_avg(&self) -> i64 { self.avg }

  pub fn push(&mut self, val: i64) {
    self.data.push(val);
    self.update_avg();
  }

  pub fn pop(&mut self) -> Option<i64> {
    let res = self.data.pop()?;
    self.update_avg();
    Some(res)
  }

  fn update_avg(&mut self) { todo!() }
}

Whenever the data changes, which here is only possible via the public push and pop methods, we call update_avg and so we internally enforce data consistency, which is not a bad thing and after making it generic you might consider the problem solved. But we can achieve the same without hiding any data:

struct AvgVec(pub Vec<i64>);

impl AvgVec {
  /// Push a value and return the new average.
  pub fn push(&mut self, val: i64) -> i64 {
    self.0.push(val);
    self.avg()
  }

  /// If the data is not empty, returns a pair of the popped value and the new average.
  pub fn pop(&mut self) -> Option<(i64, i64)> {
    let res = self.0.pop()?;
    let new_avg = self.avg();
    Some((res, new_avg))
  }

  /// Compute the average value. Returns `0` if the data is empty.
  pub fn avg(&self) -> i64 { todo!() }
}

Admittedly, this does feel a bit trivial and contrived, but I hope it illustrates the point. If you can think of a better (counter-)example, I want to hear from you!

A nice side effect of this style is that unlike in the first version if you forget to return the new average, it’s a type error. If you remove the call to self.update_avg(); or you forget to use it in a new function that mutates the state, you don’t get much help.

Of course, it’s still a good practice to minimize how much you rely on the internals of some data type outside of its module(s), as you will be less likely affected if and when the data type changes, but even if you do, the type system has your back.

Use the module system to encapsulate logic.

For more complex modules, you can split it up into (private) sub-modules and then re-export (pub use) the public parts from the parent module. The Rust compiler will even check for you when you accidentally expose something that is using another thing in its API that itself isn’t public.

In Rust methods are just syntax sugar providing some convenience, but often, there is over-reliance on using methods for stuff that doesn’t need it, which I suspect is at least partly habitual from OOP languages. Taking the example from above, we could just as well do this:

pub type Data = Vec<i64>;

/// Push a value and return the new average.
pub fn push(data: &mut Data, val: i64) -> i64 {
  data.push(val);
  avg(&data)
}

/// If the data is not empty, returns a pair of the popped value and the new average.
pub fn pop(data: &mut Data) -> Option<(i64, i64)> {
  let res = data.pop()?;
  let new_avg = avg(data);
  Some((res, new_avg))
}

/// Compute the average value. Returns `0` if the data is empty.
pub fn avg(data: &Data) -> i64 { todo!() }

Not everything needs to be a method, in fact, often a method can carry a lot of baggage when you only need a handful of things from self.

Strive to reduce the inputs to your functions to the bare minimum needed to implement them.

This makes it easier to test such functions very thoroughly in isolation from things that are not relevant to them. You can build very readable code with plain functions using qualified symbols with similar usability to methods. For example:

// This `mod` might be a file `space.rs`
pub mod space {
  pub struct Ship;

  pub fn yeet(ship: Ship) {}
  pub fn is_the_place() -> bool { true }
}

fn main() {
  let space_karen = space::Ship;
  space::yeet(space_karen);
  assert!(space::is_the_place());
}

When you’re used to having a drop-down of possible methods to come up after you type . following a name of a variable, switching to :: following a module name works much the same.

Last but not least. While Rust’s type system is powerful and it will help you to find how to do things right and prevent many bugs, it also makes it very cheap to add tests and sometimes it can be easier to test against issues than it is to prevent them at type level.

Stay pragmatic, not everything needs to be solved at type level.

Abstraction levels in functional programming

Tomáš Zemanovič — Thu, 27 Dec 2018 00:00:00 +0000

Update on December 29, 2018: I misinterpreted what was meant by the implementation inhabitants, so I scratched the sentence that talked about it. Thanks, Brian!

Teaching and learning functional programming can be challenging. Many people will already have some experience of programming, which is usually very different from FP concepts. Analogies to other languages can sometimes do more damage than good (you’ve probably heard that you have to “unlearn” stuff when you’re starting to learn FP). I am delighted to see more people learning functional programming and find the effort that goes into making it accessible to a wider audience encouraging. I think that Elm has a great role to play in it.

I’m a fan of the work Brian McKenna does with Haskell on that front (my personal favourite are his streams on building Sonic in Haskell). His recent post on Higher Kinded Parametricity made me think about abstraction and its role in teaching and learning FP. The argument concerns Functor type class and the following function (not to be confused with the void type that originated in Algol, C and found in many popular OOP languages):

void ::
  Functor f =>
  f a
  -> f ()

To anyone versed in Haskell, this is a very basic example as Functor is a fundamental building block that appears in many places. To those who are less familiar, this could pose many new challenges.

Upon opening hoogle, you might find the definition of Functor.

The Functor class is used for types that can be mapped over…

Alright, so Functor is a class, but once again, these classes are nothing like OOP classes. Assuming some familiarity with the concept of Maybe data type (in some languages called Optional), what would happen if we instead started from the bottom of the abstraction ladder and worked our way up?

Haddock tells us that, for example, void replaces the contents of a Maybe Int with unit (unit is denoted with () ). In a basic form, we can express this as:

void :: Maybe Int -> Maybe ()
void Nothing = Nothing
void (Just _) = Just ()

But for the purpose of this function, it’s irrelevant if the parameter was Maybe Int, Maybe Bool or Maybe a, so we introduce parametric polymorphism:

void :: Maybe a -> Maybe ()
void Nothing = Nothing
void (Just _) = Just ()

A type starting with a lower case letter is polymorphic, meaning we can put another type in its place. Indeed, if we replace a with Int we’ll get exactly what we had in the previous level.

Next, we could show that the kind of operation on Maybe type where we do something with its contents is very common, so we define:

fmap :: (a -> b) -> Maybe a -> Maybe b
fmap _ Nothing = Nothing
fmap f (Just a) = Just (f a)

The first parameter accepts a function that gets applied to the content of the second argument, if it has any content.

We can now use this to implement our void function:

void :: Maybe a -> Maybe ()
void = fmap $ const ()

In Elm, this is where we reach the ceiling of abstraction (note that in Elm, Haskell’s const is named always and $ is named <| ) :

void : Maybe a -> Maybe ()
void = Maybe.map <| always ()

Or for the List data type:

void : List a -> List ()
void = List.map <| always ()

You would have to implement this for each data type for which you’d want to use this function, which can be tedious. But, is this really more complicated than in Haskell? ~~Our void implementation for Maybe still has only a single inhabitant and the same with List.~~

Does this stop Elm from being incredibly practical or could this be a viable alternative for the currently most prominent web library React? I have reasons to think it exceeds React on many fronts¹². Those who pick up Elm will feel encouraged by being able to build great things using pure FP.

Haskell can, of course, take the abstraction to another level thanks to the wonderful concept of type classes first introduced by the living legend Philip Wadler and Stephen Blott in 1988 as a way of taming ad-hoc polymorphism:

void :: Functor f => f a -> f ()
void x = () <$ x

And it can go much further with that. After all, type classes are built on category theory, the language of mathematics. But even mathematicians can struggle with abstraction. I recommend the work of Eugenia Cheng, who explains mathematics and category theory in the most accessible down to earth way³. The following quote comes from her How to bake PI: An Edible Exploration of the Mathematics of Mathematics.

A moment where advanced mathematicians sometimes reach their abstract limit is category theory. They react in much the same way that teenagers do when they need x’s and y’s - they say they don’t see the point, and resist any further abstraction.

As programmers we are lucky to have the context for learning about these concepts.

If type classes are an essential part of your toolbox, then you’ll possibly find using Elm restricting. However, as this can be a scary territory for newcomers, it’s important to be sympathetic, especially if you teach FP using Haskell. Regardless, it shouldn’t have to be an exclusive choice. If and when Elm developers get the need to pursue the abstraction further, they’ll most definitely circle back to Haskell, PureScript or anything else with a more advanced type system. But I’m hopeful Elm will evolve to implement type classes in future.

We all have different ways in which we learn most effectively. Did your previous experience affect your learning of functional programming? Wouldn’t it be more constructive if FP communities were more united over the common goal of teaching FP? Please share your thoughts.

New, simpler and user-friendlier page

Tomáš Zemanovič — Tue, 11 Dec 2018 00:00:00 +0000

As we say in Czech “blacksmith’s mare walks barefoot” (or “the shoemaker’s children go barefoot” in English), my previous homepage with its Bootstrap based style looked admittedly so terrible that I myself didn’t enjoy looking at it anymore.

I finally found some time to give it much needed revamp. Also, many current developments in our industry are, to put it lightly, less than ideal and I wanted to make sure I am not just adding more fuel to the fire. It’s not just the things we can see straight away¹, which are easier to laugh off, but more importantly the things that are hidden from most people’s sight². In that spirit, all the content on this site is now self-hosted (well, except for the comments, 🤔, maybe I’ll setup something like this self-hosted commenting system one day).

To keep things simple, without making it too good, I’ve used Pure.css with its small, responsive modules. It is nice, but quite frankly, even though I’ve been making websites since my childhood, going back to CSS has been a total pain in the ass (even for something as simple as this website) after being spoilt with the awesome elm-ui (previously style-elements) that I’ve been using at work for well over a year. But the best thing about using Elm has really been its welcoming, friendly and inclusive community.

I’ve also added a simple way to estimate reading time for the blog posts. If you’re interested, you can see the source for this page on GitLab. Feel free to use any of it for your own page.

RealWorld example with Haskell Yesod

Tomáš Zemanovič — Tue, 14 Aug 2018 00:00:00 +0000

This is an overview of an example implementation of the RealWorld backend built using the powerful Yesod framework written in Haskell.

This project is essentially a RESTful API to a database and even though Yesod can do much more than that, you still get a lot of nice things that help along the way. For example, Yesod comes with some helper commands for common tasks (e.g. you can add a new handler using yesod add-handler) and you can start a new project using one of the stack templates. I’ve started this project with stack new yesod-sqlite which scaffolded application with SQLite database connection, whose schema is created from persistent entity file. The scaffolded project is setup with:

development server setup using ghcid with automatic rebuild and fast reloads
hpack for a nicer alternative to the Cabal package format
behaviour specification using hspec
fast and scalable logging using fast-logger for requests and database queries
log functions from monad-logger that includes source code location in the log entries
the fast Warp web server
classy-prelude as an alternative prelude that removes all partial functions
persistent for type-safe database-agnostic data modelling and querying with automatic migrations in development
aeson for fast JSON parsing and encoding
keter deployment system

If you want to learn more about Yesod, you can read the excellent Developing Web Applications with Haskell and Yesod for free. Some other dependencies used in this project are:

esqueleto - type-safe EDSL for SQL queries on persistent backends
forma - parse and validate form input
jwt - JSON Web Token (JWT) decoding and encoding
weeder - to detect dead code

Project structure

app
├── app
├── package.yaml
│   ├── main.hs                     "The main entry point for production build."
│   ├── devel.hs                    "The main for development server."
│   └── DevelMain.hs                "The main for running inside GHCi."
├── config
│   ├── models                      "Persistent entity file with the database schema declaration."
│   ├── routes                      "Routes declaration for our RESTful API."
│   ├── settings.yml                "App runtime settings."
│   └── test-settings.yml           "Settings overrides for tests."
├── src
│   ├── Application.hs              "Loads the settings to initalize the app with a HTTP manager,"
│   │                               "logger and database connection pool."
│   │
│   ├── Foundation.hs               "Declares the foundation 'App' datatype which is accessible"
│   │                               "from every handler, type for routes loaded from `config/routes`"
│   │                               "and authentication requirements for each route."
│   │
│   ├── Settings.hs                 "Declares and loads the app settings, which can be loaded from"
│   │                               "various sources."
│   │
│   ├── Import.hs                   "Commonly used module imports grouped for convenience."
│   ├── Model.hs                    "Generates types for models loaded from `config/models`."
│   ├── Database.hs                 "Database queries."
│   ├── Pagination.hs
│   ├── Handler                     "Handlers for the RESTful API."
│   │   ├── Articles.hs
│   │   ├── Profiles.hs
│   │   └── User.hs
│   ├── Database                    "Helpers for dealing with database."
│   │   └── Persist
│   │       ├── Extended.hs
│   │       └── Types
│   │           ├── Email           "Persistent field type for email, which is"
│   │           │   └── Internal.hs "stored in a case-insensitive string."
│   │           ├── Email.hs
│   │           ├── Password        "Persistent field type for password,"
│   │           │   └── Internal.hs "stored as a hash."
│   │           └── Password.hs
│   ├── Auth                        "Authentication using JWT token."
│   │   └── JWT.hs
│   └── Web                         "User input validation."
│       └── Forma
│           └── Extended.hs
├── test                            "The specs for user and profiles handlers."
│   ├── Handler
│   │   ├── ProfilesSpec.hs
│   │   └── UserSpec.hs
│   ├── Spec.hs
│   └── TestImport.hs

Some notes

The handlers are implemented according to the API spec:

The Profiles module is the most simple one with handlers using persistent queries
The User handlers deal with user input that is parsed and validated using the forma library
The Articles handlers contain more complex queries and pagination for the article feed. Because their queries deal with multiple tables, I reached out to the esqueleto library, which provides a nice eDSL (a SQL language embedded in Haskell) for querying the database.

Functional programming for beginners

Tomáš Zemanovič — Mon, 19 Mar 2018 00:00:00 +0000

Recently, I attended Manuel Chakravarty’s YOW! night talk titled Demystifying Functional Programming, which inspired me to write this post.

From my work, I gained some intuition that trying to teach FP by explaining the theory first didn’t work well. In hindsight, this seems almost obvious. A lot of us were taught math this way and sadly it has left many of us misunderstanding math or even despising it, even though it might be very useful in their chosen field¹.

What worked better was to show how elegantly FP solves many problems we were focusing on; one that required a use of parsers or another for which we were using reactive programming to process and analyse large quantities of data in real-time. These were some of the concepts I’ve learned in Haskell and was able to apply in Scala. They were harder to implement in the imperative way, which only complected them and that also made them harder to understand.

My intuition really aligned with the content of Manuel’s talk. With his long-time experience and commitment to working with and teaching Haskell, Manuel discussed what we can do to advance FP from the stage of innovators and early adopters to early majority², which he summarised as:

Examples first
Teach design patterns
Tight feedback loop
Visualisation can be an effective tool

Another seasoned Haskeller Gabriel Gonzalez has written some great advice for Haskell beginners³, in which he recommends to:

Avoid big-design-up-front
Start programming at term-level (using simple values and functions)
Build something useful

If you gave Haskell a try, but still find it too intimidating, I would recommend you having a look at Elm. Elm (which itself is written in Haskell) is a very small language in terms of the size of its grammar, which makes it a perfect fit for beginners. Not only is it very good at all the points above, it can also compete with other front-end frameworks like React. The whole notion of languages being general-purpose can be quite misleading⁴, Elm is intentionally not a general-purpose language⁵ and it serves its intended purpose very well. You don’t have to take my word for it though, instead, you can watch a talk by Richard Feldman, who’s been using both React and Elm side-by-side in production for over a year.

Cool, you got me, but…

Where do you start?

Of course, not everyone learns the same way. As a general recommendation I would say:

Try not to learn too many things at once, keep things simple. This is especially important if you come to FP with prior experience from imperative languages, which doesn’t translate well into FP concepts. You might have heard functional programmers say that when you start, you will have to unlearn things, and this is what they’re referring to. Honestly, you will need patience, because everything might seem slow at first. That’s okay, don’t give up and keep at it!

Try to find someone who already has experience with FP and ask them for help with writing something simple. When you see FP applied to solving a real problem, you are very likely to learn faster. There are people who say they were able to pick up Elm this way within a week, but don’t set your expectations too high. Once again, everyone’s learning path is different. Look for FP meetups in your area, join Elm on Slack, read the comprehensive Elm Guide. Don’t start trying to write a full-blown single page app, there are ways to gradually introduce Elm at work today. Or you can just play around directly from your browser, you don’t even have to install anything.

Lastly, don’t be afraid to make mistakes, for it’s an important part of any learning process.

One year of Elm in production

Tomáš Zemanovič — Tue, 06 Mar 2018 00:00:00 +0000

About a year ago, I started a new job where I build tools aimed at helping others provide a better user experience. As is common in the software industry, I inherited lots of poor vanilla JavaScript code. There were no tests, very little code structure or code re-use, mixed coding styles and as expected, a full menagerie of bugs, which frequently cropped up resulting in very poor user experience. With such a sorry state of things, it only made sense to try to take a wholly different approach - stop writing JavaScript and instead pick a higher level language that would make sure all the things are kept in check.

As a functional programming enthusiast, I’ve been a keen user of Elm for some time and although I conceptually liked it better before version 0.17 with its higher-level abstraction of rendering visual elements contained in elm-graphics library, the inclusion of HTML library made it possible to integrate it into existing front-end ecosystem with much less friction to the point where it actually became one of the top choices.

If Elm code looks terrifyingly unfamiliar to you, fear not. It builds on top of very mature ML (Meta Language) family of programming languages. Elm itself is written in Haskell, which is as old as programming itself, but you can find a plethora of languages in this family:


PureScript	front-end language, more similar to Haskell
Idris	general purpose language similar to Haskell, but with dependent types
Scala	fusion of FP with OOP for the JVM
Frege	Haskell for the JVM
Eta	Haskell for the JVM
OCaml	FP with OOP
F#	Microsoft’s FP with OOP
Hobbes	custom language built for embedding in C++
Standard ML	general purpose popular language for compilers

As processors scaling predicted by Moore’s law is close to reaching its hard physical limits (as Herb Sutter famously put it the free lunch is over¹), the software industry will have to reach to tools that are better suited for dealing with concurrency. And these tools are functional programming languages. The interesting thing is that even if your application has no need to deal with concurrency, it can still benefit from FP, its foundation in mathematical concepts scares a lot people off, but it is its strong point. If the word mathematical puts you off, know that you don’t actually have to know about this to take advantage of it, just like you won’t fly off a rollercoaster if you don’t know its physics (anecdotally, it will probably be your last ride if the rollercoaster has been built with complete ignorance of physics).

As most of software development education and training focuses on OOP, there are lots of developers who get the impression that thinking about things in terms of objects is somehow natural. But the limitation of this mindset becomes very apparent in a very popular OOP topic - Gang of Four’s design patterns². It also becomes apparent when implementation becomes so complex that no one can or even wants to work with it anymore. But where does this complexity come from, is it intrinsic to the problem domain? Most programmers know that tight coupling between unrelated concepts creates problems, in the same spirit I would say that the interleaving of data with behavior inside classes can create more problems than it solves. The declarative nature of FP languages removes much of the incidental complexity, the elephant we’re dragging around³, introduced by imperative programs because we don’t need to think about execution to understand what is going on. In more conceptual words of professor Philip Wadler, some languages were discovered while others were invented⁴ and you can tell.

The nice thing about Elm is that the community that has formed around it is very inclusive and welcoming. The language itself is very beginner friendly, to the point where some mistake its simplicity for a toy language. Don’t make the same mistake though, because Elm is up there with big names such as React, in fact, it is much more a complete solution than React. The simplicity of Elm is rooted in hard work that has been put into its design.

Once you familiarize yourself with it, the Elm compiler will be your best friend. Its error messages are so clear and helpful that they’ve influenced other languages, such as Rust. The language comes with a package manager elm-package and the libraries released in Elm automatically adhere to semantic versioning. You can even use it to check what has changed between different versions of a given package. For interactive coding, there is elm-repl and elm-reactor. The performance of Elm generated code is great and when needed optimization is a simple function call away.

So far, Elm’s simple yet powerful design has helped us to stay focused on our goal of building a great product and we expect that the payoff will be even greater in the long run. At the moment, we have about 16k Elm lines of code in production, steadily translating pleasant development experience into pleasant user experience and we have more on their way.