In Search of the Best Functional Programming Back-End: 2021 Update

Introduction

For 3 years I’ve been searching for an enjoyable language to use in writing back-end code, for API’s & Serverless. Specifically a strictly typed, functional one.

I haven’t found it.

However, I have learned a lot, and though I’d share a summary of that below. Both to hopefully inspire readers, and in hopes someone can either fill in some gaps, or shed light in a new direction I should look.

What is Back-end?

For back-end, I mean not front-end. I’ve settled on Elm for building front-end web applications. I don’t do mobile or game development professionally, so I’m not sure what I’d use in those environments.

The type of work I do for back-end includes REST API’s and serverless data parsing work, typically deployed on AWS in Lambda and Step Functions. I’m not allowed to use API Gateway so I use a lot of Application Load Balancers instead with Lambdas to trigger things that’d you typically use API Gateway for. While AWS has its favorite languages, they provide a lot of tools and documentation on how you can use other languages.

This also includes Command Line tooling to support the DevOps of the above.

I no longer do freelance nor consulting for the past 5 years. I’m 110% devoted to my salaried position. Which means I’ll do extra projects for work specifically to either learn or make things easier for myself, my team, my line of business, or the company as a whole. This includes innersourcing, prototyping, and general R&D on AWS architecture. That 10% is where the free time is now devoted.

Why Functional Programming?

Based on what I’ve learned and seen through practice the past few years, it seems to result in code that is more simple compared to Object Oriented Programming. The rules aren’t debatable either. A lot of rules in Object Oriented Programming, despite being many years old, are extremely high level and lack testable parts. Time and again I’ve seen extremely smart & experienced individuals debate the true meaning of some rule.

No one debates the meaning of a pure function. It’s clear what it is, what traits it has, and how to test for it.

Secondly, the code appears to be easier to test. Not having to utilize mocks greatly reduces to testing time. Stubs are still verbose, but much easier than mocks to write and maintain.

Third, if you’re willing to find a strictly typed language that has a “soundness” property, you don’t even need unit tests, just property tests and functional tests. Even if it’s not sound, a whole class of bugs just disappears when you use types, so it’s worth it if the compile times & maintenance costs are low.

Fourth, and very important to me: it’s easier to teach. Explaining how to make pure functions, test them, and build programs with them is much easier than going down the OOP rabbit hole. Even simple things like mocks/spies vs. stubs just resonate better.

“You call this thing, which your spy will record, then you ask the spy to verify it called this fake thing with those inputs.”

vs.

“Your fake function returns 2; assert it is 2”.

Fifth, stateless architectures result in more testable architectures, and in turn this leads to things being more independently testable. This allows you to update a large architecture with more confidence. To create stateless architectures, it helps if you’re language helps with things like immutability and clarity where the side effects are.

Sixth, it’s weird coding UI’s in Elm, then completely going to “Chaos Land” in the back-end like JavaScript or Python. You end up trying to make JavaScript/Python Functional to compensate so you can quickly switch between the two when building applications.

Haskell

I bought a book, and read parts of free books online. The goal was to use “the mother” of all strongly typed FP. In the past 3 years, I’ve made very little progress. While I’ve done the basics using Stack, I always run into the same set of problems that you’ll see repeated in this article.

First, no one can articulately explain the value of Category Theory in a way I can understand. I need this as motivation to continue learning. So far, the only 2 things I’ve learned are:

1. Instead of List.map and Array.map in Elm, higher kinded types give you map; it works with everything!
2. State Monads make it easier to store information between pipelines vs. the closure / variadic tuple insanity you deal with in JavaScript/TypeScript Promise chains.

That’s it. I know I’ve seen the benefits using libraries such as Folktale for example, which has a wonderful Maybe, Result, Validation, and Union set of types. However, how you go from those Algebraic Data Types to Category Theory is this 46 page PDF that I just cannot finish, even on 4,000 grams of Ritalin.

I want to believe. I just keep running out of steam since it’s hard to see the value.

Second, as soon as I want to do a COMPLETELY COMMON, NORMAL THING like a REST call, the complexity wagon comes in and I just move on.

Third, I can’t install Haskell packages at my company. Like Elm, we rewrite our SSL certificates. When the language doesn’t allow you to bypass this, like Node does ( process.env["NODE_TLS_REJECT_UNAUTHORIZED"] = 0 ), then you simply can’t install packages. You’re then left with 2 choices. Use ANOTHER language to install packages for you, or just not use it. Both are disappointing.

PureScript

The original draw here was that I’d get Haskell powers, but could utilize the same CICD pipelines for Node.js. However, even with the wonderful JavaScript FP community creating easier to use libraries, EXACT same problems as Haskell. Ajax is hard, and I can’t install packages using Spago because of SSL rewrites breaking the installer.

Scala

I’ve specifically had 2 job offers internally at my company because of this language. First with Cats and Scalaz and now with ZIO, Scala has taken the best parts of Haskell, the best parts of Scala, and made it really nice to work with. You can barely see the OOP leftovers.

I don’t even know Scala, nor Java for the matter, but people who “know” you’re an FP’er know you can learn that type of stuff. Either my Java prejudice, or my concerns with how slow SBT / Maven can be keep me away? I did have a positive experience playing with Gatling. Maybe I know all realistic roads lead to here, and that’s why I avoid it?

Rust

I’m a high level programmer. I have no experience in C, get really confused when Objective C / Go start talking about pointers, and never really run into performance problems. Most of my code is parsing which can horizontally scale, or just simple string parsing for orchestration API’s.

Despite that, Rust has pretty much everything you’d think someone like me would want. Specifically, almost everything is a Result. As an AWS aficionado, the concurrency abilities do not call to me at all; I’m biased that it is an infrastructure and not a code problem. I’ve seen some of the code examples in Advent of Code 2019/2020, and they vary greatly. Those from imperative/OOP backgrounds use mut copiously, whereas those from FP backgrounds have a much more pipeline style approach.

While I have grave concerns around the slow compiler, and how crates work with Artifactory (do they even?), one of these days I’ll give her a try. That C style syntax just puts me off, making me feel it’s overkill for what I need; next to Scala, maybe I have no choice?

For now, passion is just not there.

F#

I had high hopes for F#. I’m a big fan of Scott Wlaschin’s talks, and love his writing style. I’m also seriously impressed with the amount of work the open source community has done with Ionide for VSCode. Like Reason/ReScript, the syntax is this nice “no types” style, but smart enough to “know what you meant”. This results in really terse code that still retains all FP features I want like function currying and pipelines. Another nice touch is that it’s original design goal to compile to the .NET / CIL bytecode, means it still supports null pointers and class syntax. While I personally loathe that stuff, it means onboarding new people from a variety of backgrounds can be done. That’s amazing.

I was originally motivated to explore it after Quin, maker of Folktale for JavaScript, had plans of building a new and better TypeScript, specifically one that had soundness guarantees, called Purr.

Sadly, despite the insane amount of work from Krzysztof Cieślak, and Microsoft creating wonderful cli tools for me to deploy to AWS Lambda, I found F# extremely hard to advance past the beginner stage, assuming you get that to work at all.

First off, the docs aren’t that great. When you search on Microsoft’s site, it’ll give the option to see the C# or F# version of a class/module for .NET. Many, MANY of the F# examples are non-existent. Worse, many of the F# core features, such as async streaming, have no docs. Like at all. Some will give you a basic example explaining nothing. I KNOW they’re there somewhere ,though, because Ionide gives me insane amounts of type hints, implying a wealth of functionality is there. If you switch to C# and look around, it’s clear where the documentation money is going as some of the C# docs are quite impressive. However, I can’t seem to find non-MS docs on Google either like example blog posts.

Second, the format for .NET on AWS SDK API’s are atrocious; I can’t navigate like you would for Python/JavaScript, i.e pick a service like “S3” then read about “getObject”. Maybe this is because it’s C# aka “act like Java, and approach everything like OOP Soup”. It’s disappointing because F# could be architected quite differently without classes, but apparently is lumped in there. If I want to do a basic s3.getObject, it’s quite the link navigation effort. For example, assuming you found getObject amongst the nest of classes, clicking it goes to AWS docs, not code docs like the rest of the SDK’s… wat?

Third, as someone who knows nothing of .NET, F# seems to have family baggage that brings it down. The language and the people involved seem amazing, but I don’t care about any of that. I don’t build monolith solutions using Windows deployed Azure. I just build simple FP Lambdas deployed to AWS. Yet the docs expect you “grew up in .NET and C#, so you should know your way around the nomenclature and class libraries, so your brain switches OOP to FP when you see a C# class if there are no docs for F#”. Uh… no thanks.

Fourth, the only tools I used on Mac that weren’t horrible were Ionide. Even Rider seemed to assume “you know how to configure .NET, right?”. It seems .NET in general is from the monolith era. I just want to deploy little functions. Worse, all the docs and videos I’ve seen targeted at .NET / C# show this glorious life if you use Windows and install Visual Studio. The day I go back to Windows is the day hell freezes over.

Overall, it’s really hard to domain model in F# because the feedback loop is so slow. I was hoping if I got the F# scripting to work, it’d improve. Unlike OCAML, what F# is based on, the compile times were NOT fast even for small code. One of these months I’ll grab a few beers and try again. F# feels like it’s worth it. I really wish someone would throw a few million at that Krzysztof & his Ionide crew.

Reason/ReScript

Worse branding ever, but extremely good basic API docs. I lump Reason and ReScript together because when I started, Reason was the way to do what I wanted; write simple, strongly typed functions that had a sound type system, but still use the existing Node.js CICD pipelines for AWS Lambda. Now that’s ReScript.

The other draw was that not only was ReScript the fastest compiler I’d ever used next to MTASC (oh, OCAML, surprise surprise…), it’s type system was sound. This meant that compared to TypeScript, not only was it faster, but it would ensure when you compile, you had a higher chance of it being correct.

Like TypeScript, it has an interesting way to bind to various existing libraries in JavaScript so you get typing in a programmatic way. It’s extremely hard to debug this when it breaks, but when it works, it’s quite concise compared to the damn TypeScript declarations which require a separate file, and are typically installed via your package manager.

Sadly, I have the same problems here I have with Haskell/PureScript. Basic things are really hard, specifically AJAX. This is made worse by the community being fragmented over Bucklescript/Reason/ReScript. For example, the bs-fetch library is quite good; while I’m still learning the various ways to use it’s types to make strongly typed PUT calls (read: haven’t been successful yet) that’s where things get weird. I’m writing ReScript… but I install a ReasonML (not Reason, heh, Google doesn’t get it) package… using npm… which is for Node.js… but the library has a bs prefix… which is Bucklescript.

Secondly, the compiler error messages have a huge learning curve. Elm v0.19.1 is night and day more friendly.

Third, JavaScript Promises aren’t native to the language yet. This makes using them extremely verbose unless you wrap it in your own typed functions. It’s not quite there using the pipeline operator yet, so it’s not even close to the beauty you get in something like F#.

Branding insanity aside, the tooling is just not there yet for the compiler, language, and IDE tools. ReScript needs to bake more, but the ACTUAL code it produces, quickly, is amazing. I’ve made more progress with F#, but based on this community continually working on it, maybe ReScript will be my go too?

Conclusions

Sadly, I keep coming back to JavaScript using things like Folktale and Lodash, or Python using returns and PyDash. If I were to ship to production, I’d add TypeScript on top AFTER the fact. Using JavaScript to explore your domain model (i.e. figure out how you’re going to solve your programming problem) just seems to be the fastest way to get things done in AWS. While my team is heavily invested in Python, as soon as you want to do concurrency, things go downhill quickly with boto3. It doesn’t supporting native Python 3 async/await, and having to use thread pool and worry about thread safe data types if you don’t offload those concurrency concerns to AWS infra is just bananas.

I keep thinking I’ll reach some aha moment with F#, forget all the “dll on Mac locations, wat” insanity, and it’ll be my go to… but I have a feeling I should probably just bite the bullet and charge into Scala holding tightly to ZIO and pray.

What do you think I should try?

Comments

[Max Brokhman]

Pretty cool article, thanks! What about elixir? Have you tried it?

[Jesse Warden]

Yes, it was actually one of the first true functional languages I tried. A lot has changed in the years I tried it:

1. I love modeling with types, and the bugs types prevent, and Elixir doesn't have those. However, one of these weekends I'll give Gleam a try (typed Elixir)
2. I loathe Servers; I'm a Serverless kid. I'd rather let AWS handle my concurrency problems. I don't enjoy managing EC2's or SSH'ing into some box to upgrade a script like you'd do with Erlang. I just find Lambdas and Step Functions way more fun.

[Max Brokhman]

Thanks for reply and link!