DEV Community: Benoit Ruiz

Advices from a Software Engineer with 8 Years of Experience

Benoit Ruiz — Tue, 14 Mar 2023 13:03:21 +0000

My career evolution
Things I wish I had started doing earlier:
- Write a work log
- Leave the comfort zone
- Be curious about other teams and projects
- Join the oncall team
- Change teams
- Write blog posts
Things I wish I had done differently:
- Be careful when introducing new things to the team
- Do not let your emotions take over in front of the team
- Dip a foot into the hiring market
End thoughts

Hello and welcome!

My name is Benoit, I have been a Software Engineer for the past 8 years. I stayed at my previous (and first) company for 7.5 years, then I joined a new one early 2022.

This blog post comes from a self-reflection I recently had, on the things I wish I had started doing earlier in my career, and things I wish I had done differently.

What I am sharing here may be useful to any junior to mid-level developer that wishes to improve, and progress toward the title of senior and beyond.

My career evolution

Before diving into the main subject, here is my career evolution:

I started working as an intern for 3 months at a startup (which quickly became a scale-up) company.
After that, I did a full year of work-study where I spent 3 months at school and 9 months at work.
Then, I got hired as a full-time Software Engineer, and I kept this title for 3.5 years.
Quickly after the introduction of the tech career ladder, I got promoted to Senior Software Engineer. I kept this title for 3 years until I left the company, at which point the tech teams accounted for approximately 200 people.
I joined as a Software Engineer 2 at a company with thousands of tech employees. Despite the title downgrade at the second company (cf. Big Tech Hiring is Conservative - But Why?), I have been trying to keep the same responsibilities as before.

At the beginning, I was part of the Frontend team. The tech organisation was split between Backend and Frontend developers. At that time, we were no more than 30 engineers. When our new CTO arrived a year later, he introduced an organisation based on feature teams: the Spotify Model. Although there was some friction at the start (people don't like change), this reorganisation definitely turned out to be for the better.

I stayed for more than 5 years in the same feature team. I was there at its inception, so throughout the years, I became the tech referent of the project. Eventually I joined another team, where I worked until I left for a whole new adventure, a year later.

Alright, enough with the context. I hope you'll enjoy reading the rest, and that the following advices will be actionable for your career progression!

Things I wish I had started doing earlier

Write a work log

A work log is a document that contains the list of tasks you accomplished. The granularity and the type of the tasks don't matter, as long as you keep track of what you did.

You can fill in this document at the frequency you want. I would advise doing that on a weekly basis. Tasks done during the week are still fresh on Friday, so you won't struggle writing them down.

Why is this work log important? For 2 reasons:

To remind yourself of all the things you have done over the past 6 to 12 months. This is very valuable during performance reviews, to show your manager what you have accomplished, and why you deserve that raise or promotion.
To keep track of projects, notable responsibilities, and critical numbers (e.g. decreased latency by X% for a critical service) that you had over your career. This is great for completing your resume, whenever you want to venture in the waters of the hiring market.

I started writing a work log roughly 2 years before leaving my first company. So, over the past 8.5 years, my work log contains only 3 years of data (with some gaps here and there). When I had to write my resume in late 2021, I had to rely on my memories to remember what I did during the first 5 years of my career. To say the least, it took me some time to remember everything valuable, and I'm sure I forgot some of them.

You can use a work log template if you want to (here is an example). Personally, I have been using Microsoft Notes for the first 2 years, then I switched to a Google Docs with bullet points (1 list for each month of the year).

Leave the comfort zone

This is the best way to learn and become a better developer. The comfort zone is the scope, environment in which you feel comfortable doing your job. It is the teammates you already know and work with daily, the projects you have been working on for years, the responsibilities you have been carrying, and so on.

But why would someone advise you to leave this wonderful situation? Because this environment is not suitable for evolution.

Sure, if you stay in this bubble, you are an efficient person. You already know who to talk to about a specific subject, and what file in the codebase has to be changed. But what is better than one efficient person? Several efficient people!

Once you have reached the comfort zone on a particular topic, you should look for:

Mentoring people so in turn they become comfortable in this topic.
Look for something new to do, outside of your comfort zone.

Mentoring is one of the responsibilities that is expected from a senior position. It is a great way of helping our coworkers to be more efficient quickly. You become a force multiplier.

As for the new things to do, it could be anything. Here is a non-exhaustive list, for inspiration:

Contribute to a project of the team or organisation that you never had the chance to touch before, e.g. because it was always assigned to the same person (who was in their comfort zone).
Write documentation about a topic you are comfortable with. The objective is to share your knowledge and indirectly mentor people so they can get this knowledge faster than you did. Also, writing is a great skill to learn and improve, be it for documentation, emails, instant messages, RFCs (Requests For Comments), blog posts, etc.
Volunteer to participate in cross-team projects. You can even take a leadership position during these projects, to feed two birds with one hand.
Take care of improving tooling, monitoring, or team/organisation processes.
Participate in meetups organised by the company.
Join a community/guild at the company to work on org-level, cross-teams projects.
Help the hiring team by conducting technical interviews, and/or checking take-home exercises from candidates.

The goal is to learn something new. Performance reviews can help you define what to do, and how to do it when stepping outside the comfort zone. But you don't have to wait for this moment to make the move. You can do that at any time, as long as your manager is aware of it. For instance, you can talk about it during a 1-1 meeting. One of the core objectives of your manager is to help you progress in your career, so I highly recommend talking with them before leaving the comfort zone.

There may be some subjects that you don't really care about. In my case, for a long time I didn't want to learn anything related to machine learning and data analytics. But, my curiosity and thirst for knowledge eventually led me on the path of these topics. Even though I didn't get the chance to work on company projects based on these fields, I am glad I learned about them. It is great to have meaningful conversations with my peers, and it can even help me find ideas I couldn't get without this knowledge.

If you want to progress in your career, I strongly encourage you to leave your comfort zone, and learn new things every time you get the chance. And I'm pretty sure this advice also applies to one's personal life too!

Be curious about other teams and projects

This one is close to the previous one, though you don't have to endorse additional responsibility.

For a long time, I did not care about teams and projects outside my team's scope. Our product had some dependencies with services owned by other teams, but as long as the API between them and us was clearly defined, we didn't have to know anything about their services.

The only time we had to open the box and see how things worked was when we had to contribute to these projects. As we were organised in feature teams, if we needed some changes in one of the other projects of the company, we had to make these changes ourselves. Each feature team had its own roadmap, so we couldn't ask another team to do the work for us. Although we were slow at first, with the help of the other team, we progressively became more efficient when contributing to their projects.

But, for the other services that we didn't directly interact with, I had no idea how they worked, and what was their precise role in the system. It was when I joined the oncall team, years later, that I really started having a look at all the services of the company. When I finally got the full picture, it felt great:

I could have a better understanding of what could be the culprit when things went wrong.
I would know why we needed that particular service, or why we made that choice of data store.
Finally piecing everything together, after years of bringing value to the company, provided an awesome feeling of "Ohhh now I get it".

If you can, take a look at the other projects and teams at your company. Read their internal wiki pages, attend to their demos, read their public documentation. This is something you can do from time to time, it doesn't have to be a sustained effort. Bonus: if you can draw a diagram and write some documentation about the "full picture", do it. Chances are a lot of people in the organisation will thank you for that!

Notice that you do not have to produce anything here, compared to the previous advice regarding the comfort zone. All you need is to be curious, read documentation from the other teams, and ask questions. You might meet people from other teams along the way, and you may discover projects that you really want to work on.

Join the oncall team

This one may feel controversial, and it may not even be possible at your company. Of course, you should consider this advice only if your company has a healthy oncall environment (cf. Oncall Compensation for Software Engineers).

The oncall team is composed of people who are willing to intervene if something goes wrong during and outside business hours, i.e. at night, during weekends, and bank holidays. Your company may have a dedicated team of SREs (Site Reliability Engineers) for it, and/or your team may not be responsible for DevOps work.

But, if you have the possibility to join the oncall team, whether it's for the product you work on, or for the whole company (depending on its size), I would suggest doing it.

I see the following advantages of joining this team:

You learn a lot about the "behind the scenes" of the products and services that make the company thrive.
You feel more empathetic to your coworkers, as you experience the weight of responsibility whenever something bad occurs, especially at night.
You feel more engaged with the company, as you invest more of your time into making sure the products and services work as expected for the customers.

Again, a healthy oncall environment is required before embracing these responsibilities.

At my first company, I joined the oncall team (who was responsible for all the services) approximately 2 months before leaving. I wish I had joined earlier, as I learned a lot during these few months, and this additional responsibility was well compensated.

At my second and current company, I joined the oncall team (who is responsible for the services of a single product) 2 to 3 months after my first day. For now, I am only intervening during business hours, but eventually I will be able to respond to pages during the night and the weekends.

Change teams

I can see 3 reasons why you would move to another team:

You are too comfortable at your current position, and you would like to go out of your comfort zone.
You don't really like the projects/scope of the team, and you wish to work on projects that you enjoy more.
The relations with your coworkers and/or manager has deteriorated, and you want some fresh air while still being part of the company.

If you see yourself in one of these situations, then I encourage you to consider a new team instead of resigning and looking for a new company.

Changing company is exhausting, and you may lose things that you really appreciated, such as coworkers, the engineering culture, or employee benefits.

I think team hopping is great for the following reasons:

The organisation of the new team may be different (rituals, ways of working together) so you get more experience in this field.
You can bring positive changes that you learned from the previous team (improve the code review process, tools, libraries, rituals), thus becoming a good practices advocate at your company.
You can help your new teammates when they have to work on the projects owned by your previous team (i.e. knowledge spreading efficiently from one team to the other).
You can learn new tools, languages, libraries, architectures, ways of solving problems. In other words, become a better developer.
Possibly, you get to work in better conditions, if your change is due to reasons 2 or 3 mentioned earlier.

One year before leaving my first company, I decided to move to another team. Several teams asked me to join them, and if I could have split myself into multiple parts, I would have gladly joined them all.

When I joined the new team, I felt like my senior title was not legitimate anymore. I had to learn new codebases, tools, and practices. Sure, I kept my soft skills and my knowledge about the business/products, but my technical skills took a hit. Learning something new was great of course, but I wasn't the technical referent of the team anymore. Though, whenever the team had to contribute to the projects of my previous team, I could help them in a more efficient way. Through time, the feeling of "not deserving my title" faded away, and I became even better as I gathered more skills.

That being said, I think changing teams should not happen too frequently. I stayed at my first team for more than 5 years, then switched to the new one for 1 year, and eventually left the company for reasons unrelated to this new team.

If you feel like your situation fits in one of these 3 reasons, I would advise you to consider changing, but only if you stayed at least 1 full year with your current team. I think 1 year is a reasonable amount of time to feel if you belong with your current team or not. If you cannot wait for a whole year, then it means the situation is quite critical, and I would suggest involving your manager, and/or their manager to address the urgency.

Write blog posts

This one should just be "Write", but it felt too short. Writing is one of the most important skills a developer should have. A lot of our daily work involves writing: code, messages, emails, documentation, RFCs, meeting notes, incident post-mortems, etc.

Writing is one of the best asynchronous way of communicating with people. They can read your messages whenever it suits them, they are not interrupted in the middle of their task and can focus on it. Of course, in some situations, synchronous communication is a better way of communication: video call, in-person meetings, to address some urgency or remove ambiguity and misinterpretations. But in my experience, developers are more exposed to writing than talking on a daily basis.

Writing blog posts is interesting for the following reasons:

Practice makes perfect. The only way to improve a skill is by practising it. If you are not sure you are doing it right, you may ask for help from someone who does it well in your opinion, so they can mentor you on this topic. You can also read documentation and blog posts about it. That being said, the most important thing is to start practicing, even if your first articles have some flaws.
It forces you to know the subject you are talking about. This is a great way of actually learning things, by diving deeper than usual into a specific subject.
It develops your personal brand. The more people are interested in your blog posts, the more followers you get, and the more influential you become.

You can write articles on your personal blog, and/or on your company's blog. Writing for your company is great at the beginning, because it already has a base of readers and followers. However, you have less freedom on the subject you want to talk about, as it's the company's choice.

Do not expect to become popular after the first blog post. It takes a long time to become influential. You might even never reach that moment, and that's fine. You should write for YOU, to improve your writing skills, and share your discoveries with the community. You should not care about how many likes or followers you get. Yes, it is a great moral boost to get that type of notification, but your goal should not be to increase these numbers. Your goal should always be to improve your writing and share the knowledge.

I started this journey by publishing on my first company's engineering blog: The most accurate way to schedule a function in a web browser. It even got a mention in the JavaScript Weekly newsletter, which felt awesome.

Then, in March 2021, I started writing blog posts on Dev.to, and I continue doing so from time to time. I also posted an article on HackerNoon, but I didn't really like their editing on the title, and I felt like Dev.to would be my main blog medium, at least for now.

Things I wish I had done differently

Be careful when introducing new things to the team

This is especially true the more senior you become, though even juniors should feel capable of introducing new things to the team, be it libraries, languages, paradigms, ways of working together, and so on. As a junior, you will be more easily challenged by the senior folks of your team. However the more senior you become, the easier it gets to convince people, especially juniors, to be onboard.

Back at my first company, a few months before moving to another team, I was in a position where I could propose... ambitious changes to the codebase. I had been learning about the Functional Programming paradigm for a couple of years at that point, and I was convinced of its advantages.

Over the course of a few months, I introduced functional concepts to 2 teams, including mine. For the record, these presentations occurred before I introduced the heavy changes to the codebase.

I thought these training sessions were sufficient to convince people to adopt this new paradigm. And at that moment, it was true: people were nodding their heads up and down. They understood the new concepts, the pros and cons, and what we could use to improve our projects.

At one point after these presentations, we saw an opportunity:

It was at the beginning of the summer, so business activity was going to be quite slow for a couple of months.
Over the first half of the year, we encountered a few bugs and had to use dirty workarounds in one of our oldest systems.
We could drastically improve the testability and developer experience of that system, using functional concepts and the types of TypeScript (a.k.a TS). That old system was initially developed at an early version of TS that did not provide great type features.

In other words, it was the perfect time to do some refactoring! I showed a proof of concept to the team, using functional and type-level programming with TS, to greatly improve that system. We were all convinced of its benefits, and decided to go further with a production-ready implementation. I was in charge of creating the tasks, planning what could be done in parallel, etc. I posted regular updates to a Slack channel where stakeholders could follow the progress.

I developed the v2 of the system as a library within our own codebase, making it an independent module where it was possible to test every possible edge case we could think of, with unit tests since side effects were finally under control. Despite introducing a lot of new concepts, functions, and code changes, everyone was fine with it. I got approvals during code reviews.

I was heavily inspired by the fp-ts library, which has a way of coding that differs from "regular JavaScript", some could say. We couldn't simply import the library in our codebase due to constraints I am not going to mention here, so I had to reintroduce some of its functions and data types myself, with minor adaptations. I shared more presentations about these changes, and got positive feedback.

The lack of opposition led me into continuing deeper into the rabbit hole.

Once the new system was finished and tested, we had to replace the old system that was scattered everywhere in the codebase. It was used in a lot of places, so making the migration from the old system to the new proved to be very tedious.

I had never overworked so much in my life. I loved refactoring the old code to what I thought was a better version, so I didn't count the hours. Over the span of 2 months, I must have spent around 10 hours a day working on it, including weekends.

Once the work was done, I took 2 weeks of time off (they were already planned for a long time). While I was gone, the team was unfortunate to get a quite important incident. They struggled to identify the root cause, and to find a fix for it. The issue was located somewhere inside the new system, which didn't look at all like the rest of the codebase. The team wasn't really familiar with it, as I was pretty much the only one who developed it.

During the presentations I shared, people understood these new tools and practices, and agreed with the changes. But, once they were alone in the middle of all this novelty, they rightfully felt lost.

Presentations are a thing, but if you don't practice what you just learned, you will eventually forget about it. Plus, presenting each concept separately is not the same as using them all together. It adds complexity to an already-difficult subject to learn.

Long story short, I introduced a lot of new concepts to my team, and while they were on board during my presentations, the changes I brought to the project greatly damaged their ability to fix a critical issue while I was gone.

When I got back and learned about this incident, I offered to share more presentations with them, so they could get more familiar with the new code.

In reality, I should have never gone down the rabbit hole in the first place. It was not a pragmatic solution. The type improvements were great, but the whole new functional system was a mistake.

You should not bring important changes to the team just because you are comfortable with these changes. You have to keep the bus factor in mind. I thought I was going to stay in that team for long enough that everyone would eventually be familiar with the new code, that I could teach them little by little.

But, a few months after these events, I joined another team. Looking back, I feel bad for dropping this hard-to-maintain module a few months before leaving them, in addition to almost burning myself out because of it.

Whether you are an individual contributor (a.k.a IC) or a manager, if a senior teammate introduces important changes like these ones, I strongly advise you to really challenge their proposition. I am sure a better tradeoff could have been found in my case.

If you are in the same position as I was, I encourage you to think twice before committing. Is this really a pragmatic solution? Are you sure the scope is well defined, and rabbit holes well identified? Are you doing this for the good of the team, or because you like it? Will the team be comfortable with it when you won't be around to help them?

Do not let your emotions take over in front of the team

I've had moments where I strongly disagreed with whatever a manager or coworker was sharing with the team, during a meeting. I let everyone in the room know that it bothered me, thus starting a "conflict" publicly.

I wanted to show my peers that it was fine to disagree with someone else's decision. However, this type of behaviour is not healthy:

People may feel uncomfortable when a conflict becomes visible/obvious, like in these situations. It's not fair to put them into these situations.
This may create cleavage within the team, where some agree with person A, and others agree with person B. A team must remain united to be efficient.
Generally speaking, it shows some lack of self-control and discipline, and some inability to take a step back and think about the situation.

I am not saying that it's easy to contain our emotions. After all, we are human beings, not machines. But, it is also important to show respect to our peers, and avoid disturbing the course of the meeting.

In my opinion, the right thing to do is to wait for the end of the meeting, then immediately:

Go talk to the other person in a 1-1 meeting.
Or, talk to your manager about the situation, and find a way to fix it.

It is important to trigger this 1-1 immediately after the meeting that made you feel this rush of emotions. The more time passes, the worse the situation gets.

In my experience, talking with the other person always helped in improving the situation. Communication is the key here. Without social interactions, we cannot go very far in a company (or in our personal lives).

Dip a foot into the hiring market

When I joined the first company as an intern, I had a 30 minutes interview with my future manager. And that's it. I did a quick "culture-fit" interview, and I was accepted.

After that, I didn't set a foot on the hiring market for 7.5 years. Once I decided to leave, my single and brief hiring market experience was clearly not representative of what I was about to deal with. The hiring process when you apply for senior positions with approximately 8 years of experience is definitely not composed of just a "30 minutes behavioural interview".

After reading The Most Heated Tech Job Market in History, I felt I was ready to give it a try. I updated my LinkedIn profile, then set myself as "open to work".

I felt overwhelmed. Dealing with all the job propositions that rain down on me was exhausting. From September 2021 to the end of October 2021, I must have received between 5 to 10 propositions per business day. I had to take PTOs (Personal Time Off) to address the situation, and I spent a significant amount of my weekends dealing with it.

Initially I was able to answer every recruiter message. But, when I was in a situation where:

I had to work all day, as I was still employed at my current company at that time.
I was in the middle of moving to another city, which was quite stressful for me.
I was engaged in the hiring process of several companies.
I kept receiving new job propositions.

It became increasingly hard to answer all of the new messages from recruiters.

I ended up writing my own message template, containing my wishes for the next job. I shared as many details as I could: size of the company, engineering culture, remote work, compensation, technical challenges, and so on. But despite this, I couldn't keep up with all the messages.

To all the recruiters that contacted me at that time, and to whom I never got the chance to answer: I am sorry. I was simply overwhelmed by the situation.

But dealing with LinkedIn messages was not the most difficult part of it. The most exhausting part was actually doing the interviews. I never got the chance to train DSA (Data Structures and Algorithms) before, as I never felt the need of changing company.

I trained mainly on leetcode, and I also bought a few books to help me:

Cracking the Coding Interview by Gayle Laakmann McDowell.
System Design Interview Volume 1 by Alex Xu.

In addition, I had to remember and present some of the most impactful projects I worked on, in detail. This is where the work log we mentioned previously could be a very interesting asset.

When I finally accepted an offer, I handed my resignation to my manager. If I remember correctly, I got a +25% base salary increase with this new offer, and there was some additional compensation included, and overall better employee benefits for my situation (working remotely).

The reason why I encourage you to try interviewing from time to time is for this:

You will get some experience along the way, so when the time comes to finally join a new company, you will feel less overwhelmed.
You can check what is your worth on the market, and potentially ask for some compensation adjustments at your current company. If you can get a job offer, chances are it will help you even more at getting what you think you deserve at your current position.

At my first company, I got good salary raises (between 7% and 10% year-on-year), except for the last year. Despite not being satisfied with the one I got in the last year, I thought I would have gotten an even better raise the next year anyway.

Since I was getting good raises for most of my career at that company, I never felt like checking what I was worth in the hiring market. I wish I had done that earlier. That doesn't necessarily mean I would have left the company earlier, but at least I would have gotten some experience, and I could have possibly asked for a salary adjustment instead of a raise.

Don't get me wrong: getting a 10% raise is great. But if your salary is 15% to 20% below what the market tells you what you are worth, in the end that 10% raise isn't good enough. And how can you know if you have the compensation you deserve? By experiencing that market yourself. Also, by staying in touch with former coworkers that experienced the market themselves.

Doing interviews doesn't force you to leave the company. You can do tens of interviews while staying at your current company, as long as you don't sacrifice your work for doing interviews. This is why I had to take PTOs, and I had to do interviews early in the morning, and during lunch breaks.

End thoughts

First of all, thank you for reading so far! I hope this article was useful to you. I think you noticed that all these advices are not really technical. Most of them are about soft skills. Maybe I'll write another article with advices that are more about technical skills, or hard skills. Let me know if you would be interested in such content.

One last thing I wanted to share with you: stay in touch with some of your former coworkers, especially the ones that inspired you. They are people you admired, because they did a great job according to your own standards. They may have been excellent managers, or influential technical leaders that you really appreciated. This is important because they will help you improve and become a better developer. They may even convince you to join their company/team, which is healthy to do as long as the frequency is reasonable. Everyone should build a network that helps them become a better person.

If you have the occasion to try these advices out, and if it ends up working out for you, please share your experience. I am really curious to get feedback about these advices, and see if they can apply to other situations than my own.

As always, feel free to share your opinion in the comments! See you next time :)

Photo by Simon English on Unsplash.

Data immutability

Benoit Ruiz — Wed, 04 May 2022 15:08:40 +0000

Introduction
Characteristics of data immutability
Summary

Introduction

Data immutability is a concept that applies to values that are created once, and cannot be modified afterwards. Values are in read-only mode, frozen in time.

If we want to change a value, we have to create a copy of it, then change this copy. The newly created value becomes immutable in turn, thus carrying this read-only property.

Data immutability comes in direct opposition to data mutability. A mutable value is in read-write mode, i.e. it can be altered by anyone, at any time.

An example of a mutable value could be an instance of a class whose methods change the value of its properties:

class User {
  constructor(private name: string) {}
  setName(newName: string): void { this.name = newName }
  getName(): string { return this.name }
}

const user = new User('Bob')
console.log(user.getName()) // "Bob"
user.setName('Henri')
console.log(user.getName()) // "Henri"

The fact that a property may change in time makes the code less predictable, and harder to understand, test, and debug.

Mutability does not only apply to imperative paradigms such as Object-Oriented Programming. We could use code that looks functional, and still have mutability:

interface User { name: string }

function setName(user: User, newName: string): User {
  user.name = newName
  return user
}

const user: User = { name: 'Bob' }
console.log(user.name) // "Bob"
const newUser = setName('Henri')
console.log(user.name, user === newUser) // "Henri", true

Note that in both cases, we used const to declare the user variable, though we were still able to mutate their name property. In JavaScript, the const keyword ensures that we cannot assign a new value to the variable, but the value inside the variable can be changed, as long as it is not a primitive type such as string, number, or boolean.

Here, user is an object, i.e. a non-primitive type, so we can freely mutate its properties. There are no exceptions thrown by the compiler at compile time, nor by the JavaScript engine at runtime.

There are ways to ensure immutability at both compile and run times for non-primitive values, though we will not discuss these in this article.

In TypeScript/JavaScript specifically, feel free to look for:

Immutability in TS, at compile time, using as const, readonly, and Readonly<A> type syntax.
Immutability in JS, at runtime, using Object.freeze on both arrays and objects.
Immutability in JS, at runtime, using a third-party library such as Immutable.js.
Immutability in JS, at runtime, using immutable records and tuples (more on that in the next paragraph).

In the latest State of JavaScript 2021, one of the most wanted features in JS that people would like to use is Immutable Data Structures such as Record and Tuple.

The JavaScript Records & Tuples Proposal, which is currently in stage 2 out of 4, should allow developers to use deeply immutable object-like and array-like structures, using respectively #{ x: 1, y: 2} and #[1, 2, 3].

This shows that people (or at least JS/TS developers) are really interested in data immutability.

That being said, we do not need immutability enforced by the language, or a library, to actually write code that deals with immutable values.

Data immutability is a matter of not mutating values. Whether these values are technically protected against changes by the compiler/library or not, at the end of the day, it is our responsibility as developers to keep these values unaltered.

Data immutability depends on the developers' discipline to not mutate values. We can be helped by technology to enforce this property, but it is not a prerequisite. Though, I would advise using features enforcing immutability as much as possible, as it can be tempting to take shortcuts and mutate values to go faster.

Let's see what are the advantages and drawbacks of using immutable data in our programs. The list from the next chapter herafter is non-exhaustive; feel free to share your opinion.

Characteristics of data immutability

Code is more predictable

Once some piece of data has been created, it cannot change anymore. We do not have to worry about changes happening behind our backs. We do not have to search the entire codebase to see if it is safe, or not, to use that particular value.

If that value contains the information we need, then we can use it safely. We can let our guard down a little, and relax our defensive programming mindset. Once a value has been verified to contain all the information it should contain, then it is valid indefinitely. There cannot be any surprises, or undesired behavior.

When dealing with mutable data though, we have to be extra cautious. Suddenly, our program is filled with conditions and assertions to make sure we are using a value that has the expected shape.

Furthermore, the type of a value cannot help us understand where it is used in the timeline of events. A piece of data that changes over time must hold a type that works no matter its state. Thus, we end up using types that are quite generic (e.g. with lots of optional properties), and that are not great at helping us understand what is going on in a specific part of the codebase.

Let's take an example. Here is a program representation, where squares are modules, ellipses are mutable values, and arrows are interactions between modules and these values (from value to module = read, from module to value = write):

Can you guess what is the order in which these arrows happen?

We cannot accurately predict what will be the actual data flow of this program. We can make some guesses or assumptions, for example:

A → B → C → D → F → G → I → E → H
B → A → F → D → C → I → G → H → E
A → B → C → F → D → I → E → H → G

If we really want to know the answer, we have to actually read the code, or run the program to find out.

Now, let's make these values immutable. In other words, arrows from modules to values (i.e. write operations) are impossible. The modified program looks like this:

Here, because the data flows in a linear direction, we can actually have a sense of timeline of events happening in the program. With this information, it is much easier to predict the path that will be taken:

In this new illustration: A → B → (C → D → E)
On the original one from above: A → B → C → F → D → I → H → E → G

Additionally, the type of these values can be defined more accurately. For example, in the left-most module, we know that the green value has the shape {a, b, e}. In other words, we know e is defined and we do not have to make assertions later in the program. From this point and onward, the type is {a, b, e}, and not {a, b, e?} like we had in the original program.

Thread-safety

As we already mentioned in a previous article, data immutability allows to program with thread-safety baked in. We do not have to worry about race conditions, since we do not mutate any shared state. Reading from a read-only value is multithreading-friendly.

Threads may use a local mutable state, as long as this state is not accessed by any other thread. The coordinator is in charge of gathering the results from the threads, then create a new, immutable state based on these results.

If we were to implement a program with multiple threads using a shared mutable state, we would have to use complex mechanisms to have the same advantages of using immutable data.

Some examples of these mechanisms could be:

A granular locking mechanism to safely access some parts of the shared state.
These locks should have a timeout mechanism, in case a thread dies unexpectedly, to release the lock and make the resource available again.
Another service to listen to transactions, and keep a history of all the state changes, e.g. for audit or compliance purposes.

Time Travel Debugging

As Microsoft says:

Time Travel Debugging (TTD) can help you debug issues easier by letting you "rewind" your debugger session, instead of having to reproduce the issue until you find the bug.

A lot of actions in the software have consequences on the state of the program. Let's take a basic example: a "to do list" application. This program exposes a list of tasks to do. We can add, modify, or remove tasks to/from this list, and we can also mark some of these tasks as "done".

If we manage to:

Save the initial state, e.g. an empty "to do" list
After each action, save a snapshot (or copy) of the action performed, the state at that time, and the resulting state following the action

Then we can implement Time Travel Debugging quite easily.

This allows us to replay the session, step by step, helping us identify which combination of action and state led to a bug, or if something unexpected happened between 2 actions.

A nice side effect (not to be confused with side effects) is that we can very easily implement undo/redo actions. All we have to do is travel back or forward in time, i.e. restore a previous state.

If you are familiar with frontend development using TypeScript or JavaScript, then you might have heard about Redux. It is a library for state management, often used with React, whose particularity is to use reducers to update the state of the program. A reducer is a pure function that takes an action and the state as arguments, and returns a new state. We can easily plug a middleware to keep track of every reducer call, allowing us to build a Time Travel Debugging tool, such as Redux DevTools.

More memory allocation

As a reminder, if we want to change a value, we have to create a copy of it, then apply the changes on that copy. What happens if we have a huge list of values, and we want to add a new element? Or, what happens if we have an object with a lot of depth, and we want to change the value of a deeply-nested property?

We have to duplicate the entire value before applying the changes, that is the rule. As a consequence, our program has to run on a device that has more memory than it actually needs to perform correctly. (disclaimer: I guess today's engines are smart enough to make optimizations in this area, but I don't have sufficient knowledge to make such a claim. Feel free to share if you know more about it!)

In the majority of cases, the programs we write are used on devices that have a lot of memory. Plus, the engines that run the code have mechanisms such as Garbage Collection, a.k.a GC, to free unused memory up. Unless we need to keep track of previous values (e.g. for Time Travel Debugging, history, auditing...), the previous value that got copied becomes useless, so it can be safely removed from the memory by the GC.

However, there are devices where the memory is not that abundant. This is the case for IoT (Internet of Things), or programs run on a Raspberry Pi, or similar. In these cases, immutability may not even be an option for large values. Furthermore, developers' discipline as we mentioned earlier may not even apply: the limited amount of memory may force us to purposely mutate values, as the memory is scarce.

May be cumbersome to update deeply-nested values

Let's take the following User model:

interface User {
  name: string
  job: Job
}

interface Job {
  title: string
  company: Company
}

interface Company {
  name: string
  address: Address
}

interface Address {
  street: AddressStreet
  zipCode: string
  country: string
}

interface AddressStreet {
  name: string
  nb: number
  special?: string
}

Granted, we could have used a simple string for the company's address, but this is an academic example. Furthermore, people might want to (or are constrained to) use a complex solution to model the address, such as this one.

So, keeping data immutability in mind, how would we update the name of the street?

We could use the spread operator to rebuild the User object, while applying the change(s) we want:

declare const user: User

const userWithNewCompanyAddress: User = {
  ...user,
  job: {
    ...user.job,
    company: {
      ...user.job.company,
      address: {
        ...user.job.company.address,
        street: {
          ...user.job.company.address.street,
          name: 'Awesome avenue'
        }
      }
    }
  }
}

But wait, you said we had to clone/duplicate the value before altering it. You don't duplicate the whole object here?

Indeed. Using the spread operator, we are making shallow copies of every intermediate object. This means that, if user.job.title was an object, then userWithNewCompanyAddress.job.title would be the exact same object (same reference), not a copy of it.

Ok then, let's use a solution that truly clones the whole value:

declare function deepCopy<A>(obj: A): A

declare const user: User

const clonedUser = deepCopy(user)
clonedUser.job.company.address.street.name = 'Awesome avenue'

I must admit, I am not fond of this approach:

We need some deepCopy utilility function to clone objects (and possibly arrays). It is not very hard to implement if we use pure data: something such as JSON.parse(JSON.stringify(obj)) should do the trick, although it has its limitations. Nonetheless, such a function is not available in the standard library.
- [side note] I recently heard about the native structuredClone function to deep-copy an object, though it is only supported on recent browser and Node versions.
It has some runtime performance impact. For a single object, it is probably negligeable. Though, what if we iterated over hundreds or thousands of objects that would be more complex than this one?
We still have a mutation step, even if it applies on a copy of the initial value. It may feel odd to discourage/forbid mutations, then see this type of lines of code in the codebase.

This is why I prefer the first approach:

It is a one-shot step: only one value assignment to a variable
It preserves the original sub-objects and their properties that are not changed: better memory footprint and less CPU utilization (please, correct me if I am wrong here)

However, as you can see, the major drawback is that it is quite verbose if we are changing a deeply-nested value.

In the functional world, there is a solution to that: optics. You might see the word "lens" (or "lenses") come up more often than "optics". A lens is a type of optic that, in my experience, is the most used compared to other optics such as iso, prism, or traversal.

Without going into too many details, an optic is a composable and pure getter/setter.

We might talk about optics in this series later, in a bonus article. For now, here is how we could leverage optics to improve readability in our case, using monocle-ts:

import { Lens } from 'monocle-ts'

declare const user: User

const companyStreetName = Lens.fromPath<User>()([
  'job', 'company', 'address', 'street', 'name'
])

const userWithNewCompanyAddress: User =
  companyStreetName.set('Awesome avenue')(user)

Finally, to demonstrate the power of optics, let's imagine that the company has several addresses, and we would like to change all their street names to lowercase:

const newUser = {
  ...user,
  job: {
    ...user.job,
    company: {
      ...user.job.company,
      addresses: user.job.company.addresses.map(address => ({
        ...address,
        street: {
          ...address.street,
          name: address.street.name.toLowerCase()
        }
      }))
    }
  }
}

When we start mixing objects and arrays, it gets messy quite quickly. Using optics, this would become more readable, and more composable as well:

import { fromTraversable, Lens, Traversal } from 'monocle-ts'
import { Traversable } from 'fp-ts/Array'

// optic to get the name of the street, from an address
const streetNameL = Lens.fromPath<Address>()(['street', 'name'])

// optic to get an address from a list of addresses
const companyAddressesT: Traversal<Address[], Address> =
  fromTraversable(Traversable)<Address>()

// optic to get the names of the street, from a list of addresses
const companyStreetNamesT: Traversal<Address[], string> =
  companyAddressesT.composeLens(streetNameL)

// optic to get the names of the street of the company, from a user
const userCompanyStreetNamesT: Traversal<User, string> =
  Lens.fromPath<User>()(
    ['job', 'company', 'addresses']
  ).composeTraversal(companyStreetNamesT)

const lowerCaseCompanyStreets: (u: User) => User =
  userCompanyStreetNamesT.modify(name => name.toLowerCase())

const newUser = lowerCaseCompanyStreets(user)

The most interesting part (and more declarative as well IMO) being:

userCompanyStreetNamesT.modify(name => name.toLowerCase())

Immutability syntax may bloat the code

The standard library of some languages exposes mutable data structures by default. This is the case in TypeScript, with arrays and objects. This means that, if we want to enforce immutability, we have to use additional syntax, or use data structures imported from a third-party library.

In TypeScript, adding keywords such as readonly, as const, and Readonly<> everywhere (on top of existing types) can lead to code that gets more difficult to read and understand.

Which one of the following is easier to read?

const actions = ['a', 'b', 'c']

type Action = 'a' | 'b' | 'c'

interface User {
  name: string
  actions: Action[]
}

function makePairs<A, B>(arr1: A[], arr2: B[]): [A, B][] {
  if (arr1.length !== arr2.length) {
    return []
  }
  return arr1.reduce(
    (acc, val, index) => [...acc, [val, arr2[index]]],
    [] as [A, B][]
  )
}

const user1: User = { name: 'Bob', actions: ['a', 'a', 'c'] }
const user2: User = { name: 'Henri', actions: ['b', 'a'] }
const arr1: User[] = [user1]
const arr2: User[] = [user2]

const res = makePairs(arr1, arr2)
// const res: [User, User][]

const actions = ['a', 'b', 'c'] as const

type Action = (typeof actions)[number]

interface User extends Readonly<{
  name: string
  actions: readonly Action[]
}> {}

function makePairs<A extends Readonly<any>, B extends Readonly<any>>(
  arr1: readonly A[],
  arr2: readonly B[]
): ReadonlyArray<readonly [A, B]> {
  if (arr1.length !== arr2.length) {
    return []
  }
  return arr1.reduce(
    (acc, val, index) => [...acc, [val, arr2[index]]],
    [] as ReadonlyArray<readonly [A, B]>
  )
}

const user1: User = { name: 'Bob', actions: ['a', 'a', 'c'] }
const user2: User = { name: 'Henri', actions: ['b', 'a'] }
const arr1: readonly User[] = [user1]
const arr2: readonly User[] = [user2]

const res = makePairs(arr1, arr2)
// const res: readonly (readonly [User, User])[]

I think you will agree with me that the first version is more readable, though less safe. It has 30% fewer characters than the version with immutable types. Again, you might want to rely on developers' discipline and not on the language's syntax to make the code less bloated.

Keep in mind that, in a more complex codebase, it could be difficult to see that values (such as arr1, arr2 or the User objects they contain) could be mutated anywhere, leading to undesired side effects. Using TypeScript syntax or a third-party library could prevent these kinds of effects to occur. As always in our jobs, it is a matter of tradeoff between safety and readability.

Maybe some day, TypeScript will release a new compiler option "readonlyByDefault", and a new type operator mutable, that would allow us to use immutable data by default (though the migration of the codebase to this "mode" would be probably painful!).

Summary

Data immutability is great for many reasons as we have seen in this article. It has some drawbacks, but thankfully they can be mitigated, or they do not apply in the majority of cases.

For me, the most important part is the predictability it offers. I think it's great to be able to read a function and be certain that the values it uses cannot be changed anywhere else (e.g. because of some arbitrary event I don't know about).

If I want to know how the values are used, I can do the following:

If it's not returned by the function, then it means that:
- Either the value (e.g. of type Foo) is only used by the function I am currently reading => local scope, I can just focus on this particular function and not worry about the rest,
- Or it is a global immutable state that will never change, and always have the same type (e.g. Foo). So I know exactly what the function is able to do with it, or if it needs more information to work properly.
If it's returned by the function, then I can search for the places where the function is called, and follow the paths from there to see how the data flows in the program.

Plus, it removes a big chunk of lines induced by defensive programming, so the code feels more readable and focuses on the most important parts.

In my opinion, following the "breadcrumbs" in a linear way is great for understanding the codebase, and makes debugging the code easier.

If we zoom out from the code, we can see that one of the most trending pieces of technology of today, which may revolutionize the World in the near future, uses data immutability: the Blockchain.

Additionally, anything that needs some traceability, such as financial operations or database/service accesses, has to implement some kind of ledger or auditing mechanism to better understand (and justify) when something goes wrong. This is only possible with data immutability.

Finally, I think the following quote from Archis Gore written on Quora sums up pretty well how to approach this subject in our day-to-day work:

Shared state is fine if it is immutable. Mutable state is fine if it is not shared.

Thank you for reading this far! In the next article, we will talk about currying, partial application, and tacit programming (also called point-free style). See you next time!

Side note:
Originally, I wanted to include a "how to deal with mutability" chapter where I would take some typical examples (e.g. global mutable state, object instance whose properties are partially defined) and try to make them immutable. Though, I didn't anticipate that I would write so much in the characteristics chapter! So, I decided not to write another chapter here. Let me know if you would be interested though, and I might write another article specifically for this! :)

Photo by Xavi Cabrera on Unsplash.

Pictures made with Excalidraw.

Function purity and referential transparency

Benoit Ruiz — Tue, 12 Apr 2022 11:00:20 +0000

Function purity
Referential transparency
How to deal with impure functions?
Practical example (long chapter)

Function purity

A function is considered pure if it does not have any side effects. To understand what a side effect is, please refer to the previous article of this series.

In other words, given some input, a function will always return the same output. External factors, such as network conditions or global state changes, have no impact on the returned value of a pure function.

Let's suppose we have a greet function. The impure version performs a HTTP request to some API endpoint in order to retrieve the final greeting message. The pure version does a simple string concatenation. When the network is no longer available, the function with side effect does not work anymore, whereas the pure one still does.

We want to work with as many pure functions as possible because they are predictable, thus making it easy to understand and test their behaviors.

Additionally, pure functions ensure that we can only care about their returned values. We do not have to worry about how the value was calculated. Hence, we can treat pure functions as black boxes, and just use their results to reason about the program.

Referential transparency

Referential transparency is the ability to replace an expression with its result, without changing the meaning/behavior of the program.

It is a property of expressions, which applies to pure functions, since these functions are essentially parametrized expressions.

As a simple example, we can use the expression 2 + 3. This expression is pure (i.e. no side effects), so it is referentially transparent. We can replace 2 + 3 with its result 5 without changing the behavior of the program. For example, if the program was computing the following expression: ((2 + 3) + 2) / 7, then computing ((5) + 2) / 7 would yield the same result, 1. The behavior of the program is preserved, although its definition has been simplified.

It does not matter how many times you evaluate an expression: it will always yield the same result.

For functions, we can represent this property as a mapping between the "function reference + its inputs" and "its output":

We can use the referential transparency property for optimization purposes (both developers and compilers), such as using lazy evaluation, memoization, or parallelization.

The opposite of referential transparency is referential opacity. This means that we cannot replace the expression or function call with its result, we have to run it.

Any function that performs side effects is referentially opaque. As mentioned in the previous article, we need side effects in our programs to make them useful. This means that we cannot have only pure functions, though we want as many of these functions as possible.

How to deal with impure functions?

It is very easy to write impure functions. As soon as we need some external dependency, such as a global state, making a network request, or reading from a file, we can simply reference these dependencies in the middle of our functions to use them.

However, in my experience, this can eventually make the code look like a plate of spaghetti, and it makes it harder to understand what the function actually does as a whole.

In addition, when it comes time to test these functions, we often have to mock the dependencies before each test case. This can lead to having complex before/after hooks to set up the tests, and restore the state prior to running these tests.

We cannot get rid of the side effects though, so how can we deal with this situation?

We have to identify the parts of the function that are pure, and the ones that are not. Then, we can move the pure and not-so-pure parts into their own functions. Finally, we can rewrite the original function by using the pure ones directly, and by explicitly providing the (side-effectful) dependencies as an argument, using Dependency Injection (a.k.a DI).

I am not going to talk about DI frameworks here. We are going to use the simplest version of DI: passing a dependency as one of the function's arguments.

The final result is still an impure function, since a function that needs at least one side effect cannot be pure. But, this time the pure parts are separated and can be tested easily, while the impure parts are explicitly passed as arguments and can be easily replaced with lab values to write tests.

That being said, if you have read the previous article, you have seen one way to deal with side effects: using intermediate functions (or "thunks") to make them lazy, thus making the function artificially pure.

Practical example

I was not sure if I should include this chapter in this article. I think it adds an interesting value at this point of the serie, though I may move this chapter in a dedicated article in the future.

Let's consider the following function, whose goal is to get the definition of a term provided by a user. It does the following:

Keep track of the terms searched
Make sure the term is valid
Get the definition of the term from a cache, if available
If not, make a request to a web service, then cache the result in a file

Hereafter is its definition, using the TypeScript language:



import { promises as fs } from 'fs'
import fetch from 'node-fetch'

const searchedTerms = new Set<string>()

export async function getTermDefinition(term: unknown): Promise<string> {
  if (!term || typeof term !== 'string') {
    throw new Error(`Invalid term: ${term}`)
  }

  const lcTerm = term.toLowerCase()

  if (searchedTerms.has(lcTerm)) {
    return fs.readFile(`definitions/${lcTerm}.txt`, 'utf8')
  }

  const response = await fetch(`/api/definition?term=${lcTerm}`)
  const { definition } = await response.json()

  await fs.writeFile(`definitions/${lcTerm}.txt`, definition)
  searchedTerms.add(lcTerm)

  return definition
}

As you know, there are hundreds of ways to write a function with the features listed above. I intentionally wrote it this way for academic purposes, though it may not be that far from an actual production implementation.

We know it is impure because it has the following side effects:

It reads from a global state: searchedTerms
It reads from a file to get the definition of a term: fs.readFile
It calls an API endpoint using HTTP: fetch
It updates the global state: searchedTerms.add
It writes to a file: fs.writeFile

Here are the pure parts of this function:

Validate the term, making sure it is a non-empty string
- Technically, since we throw an error, this part is not really pure. We could have chosen to return a Either/Validation/Result data type instead. But, for the sake of simplicity, let's keep the error throw and consider this part pure.
Transform the term by making it lower case
- We could add more transformation steps here, such as sanitizing the string to remove special characters.

Testing the function

To test this function using Jest, we have to rely heavily on mocking:



jest.mock('fs', () => ({
  promises: {
    readFile: jest.fn(),
    writeFile: jest.fn()
  }
}))
import { promises as fs } from 'fs'

jest.mock('node-fetch')
import fetch from 'node-fetch'
const { Response } = jest.requireActual('node-fetch')

import { getTermDefinition } from './getTermDefinition'

describe('getTermDefinition', () => {
  type MockedReadFile = jest.MockedFunction<typeof fs.readFile>
  type MockedWriteFile = jest.MockedFunction<typeof fs.writeFile>
  type MockedFetch = jest.MockedFunction<typeof fetch>

  beforeEach(() => {
    ;(fs.readFile as MockedReadFile).mockReset()
    ;(fs.writeFile as MockedWriteFile).mockReset()
    ;(fetch as MockedFetch).mockReset()
  })

  test('invalid term', async () => {
    await expect(getTermDefinition(42)).rejects.toEqual(
      new Error('Invalid term: 42')
    )
  })

  test('valid term "foo", cache miss', async () => {
    ;(fetch as MockedFetch).mockResolvedValue(
      new Response(JSON.stringify({ definition: 'description' }))
    )
    expect(await getTermDefinition('foo')).toBe('description')
    expect(fs.writeFile).toHaveBeenCalledWith(expect.any(String), 'description')
  })

  test('valid term "foo", cache hit', async () => {
    ;(fs.readFile as MockedReadFile).mockResolvedValue('description')
    expect(await getTermDefinition('foo')).toBe('description')
    expect(fetch).not.toHaveBeenCalled()
    expect(fs.writeFile).not.toHaveBeenCalled()
  })
})

The mocking parts represent approximately 40% of the lines written to test the getTermDefinition function.

Given the current implementation of the module exposing the tested function, we cannot control the searchedTerms list. Here, to test the "cache hit" part, we have to:

First call the function with a given term "foo" to cache the result, and update the searchedTerms list
Then call the function a second time, with the exact same term "foo" to hit the cache

A test case should not depend on another. We should be able to copy/paste the test(...) blocks wherever we like.

To do that, we need to control the searchedTerms list. One way of doing that would be to export it from the module, then set it up in the tests:



- import { getTermDefinition } from './getTermDefinition'
+ import { searchedTerms, getTermDefinition } from './getTermDefinition'

describe('getTermDefinition', () => {
  beforeEach(() => {
+   searchedTerms.clear()
  })

  test('valid term "foo", cache hit', async () => {
+   searchedTerms.add('foo')
    ;(fs.readFile as MockedReadFile).mockResolvedValue('description')
    expect(await getTermDefinition('foo')).toBe('description')
    expect(fetch).not.toHaveBeenCalled()
    expect(fs.writeFile).not.toHaveBeenCalled()
  })
})

There, we managed to test this function, despite its number of side effects. Though, to do that, we had to write a lot of code to mock the external dependencies, and we had to expose the searchedTerms list to control it.

Furthermore, if we want to move to a different HTTP or file system library in the future, we would have to:

Adapt the implementation of this function
Adapt all the mocking parts of the tests

Let's try to move the different pure and impure parts into dedicated functions, then use dependency injection and compose these functions together to rebuild getTermDefinition.

Splitting the function

Let's move the pure parts into 2 separate functions: validateTerm and transformTerm.

For the other parts, we are going to use a Dependencies object passed as the second parameter of the getTermDefinition function. This object will contain the following elements:

The list of searched terms: searchedTerms
A function to read from a file: readFile
A function to write to a file: writefile
A function to fetch the definition of a term: fetchDefinition

We are also going to provide default values for these dependencies, using the fs and node-fetch modules, for the live environments (i.e. development and production).

Note that the searchedTerms list is not exported anymore, as we can provide one using the Dependencies object.



import { promises as fs } from 'fs'
import fetch from 'node-fetch'

const searchedTerms = new Set<string>()

function validateTerm(term: unknown): asserts term is string {
  if (!term || typeof term !== 'string') {
    throw new Error(`Invalid term: ${term}`)
  }
}

function transformTerm(term: string): string {
  return term.toLowerCase()
}

async function readFile(path: string): Promise<string> {
  return fs.readFile(path, 'utf8')
}

async function writeFile(path: string, content: string): Promise<void> {
  return fs.writeFile(path, content)
}

async function fetchDefinition(term: string): Promise<string> {
  const response = await fetch(`/api/definition?term=${term}`)
  const { definition } = await response.json()
  return definition
}

export interface Dependencies {
  searchedTerms: Set<string>
  readFile: (path: string) => Promise<string>
  writeFile: (path: string, content: string) => Promise<void>
  fetchDefinition: (term: string) => Promise<string>
}

const defaultDependencies: Dependencies = {
  searchedTerms,
  readFile,
  writeFile,
  fetchDefinition
}

export async function getTermDefinition(
  term: unknown,
  { searchedTerms, readFile, writeFile, fetchDefinition } = defaultDependencies
): Promise<string> {
  validateTerm(term)

  const lcTerm = transformTerm(term)

  if (searchedTerms.has(lcTerm)) {
    return readFile(`definitions/${lcTerm}.txt`)
  }

  const definition = await fetchDefinition(lcTerm)

  await writeFile(`definitions/${lcTerm}.txt`, definition)
  searchedTerms.add(lcTerm)

  return definition
}

Testing the new version

As we have added the Dependencies parameter to the function, we can provide values and mocks, depending on the test case. We are still using mocked functions, but this time we are not mocking entire modules anymore.

Note that we did not export the pure functions from this module. Here, I made the choice to test them indirectly when testing the main function getTermDefinition. In addition, I did not add any test case for invalid terms such as 42 for simplicity, but it should be added for a real case.



import { getTermDefinition, Dependencies } from './getTermDefinition'

describe('getTermDefinition', () => {
  const baseDependencies: Dependencies = {
    searchedTerms: new Set<string>(),
    readFile: jest.fn(),
    writeFile: jest.fn(),
    fetchDefinition: jest.fn()
  }

  test('invalid term', async () => {
    await expect(getTermDefinition(42, baseDependencies)).rejects.toEqual(
      new Error('Invalid term: 42')
    )
  })

  test('valid term "Foo", cache hit', async () => {
    const dependencies = {
      ...baseDependencies,
      searchedTerms: new Set<string>(['foo']),
      readFile: jest.fn().mockResolvedValue('definition')
    }

    const definition = await getTermDefinition('Foo', dependencies)

    expect(definition).toEqual('definition')
    expect(dependencies.fetchDefinition).not.toHaveBeenCalled()
    expect(dependencies.writeFile).not.toHaveBeenCalled()
  })

  test('valid term "foo", cache miss', async () => {
    const dependencies = {
      ...baseDependencies,
      fetchDefinition: jest.fn().mockResolvedValue('definition')
    }
    expect(dependencies.searchedTerms.size).toBe(0)

    const definition = await getTermDefinition('foo', dependencies)

    expect(definition).toEqual('definition')
    expect(dependencies.writeFile).toHaveBeenCalled()
    expect([...dependencies.searchedTerms]).toEqual(['foo'])
  })
})

The changes we made allow us to refactor the code to use a different HTTP or file system module, simply by changing the implementations of readFile, writeFile, and fetchDefinition. Plus, we do not have to update the tests at all, because the Dependencies interface does not change.

This example was not entirely about "purifying" an impure function. We have seen how to handle side effects in the previous article of this series. Here, I tried to show you how we can organize our code to split the pure and impure parts into dedicated functions. Then, we can use dependency injection to make these side-effectful dependencies explicit, and provide fully-controlled values to write the tests. Hopefully you enjoyed this chapter!

Thank you for reading this far! As always, feel free to share your opinion in the comments :) Next time we will talk about data immutability. See you!

Photo by Xavi Cabrera on Unsplash.

Pictures made with Excalidraw.

Equivalent of Scala's for-comprehension using fp-ts

Benoit Ruiz — Thu, 17 Feb 2022 09:10:54 +0000

Introduction
Prerequisites
Using fp-ts
Using fp-ts-contrib
Summary

If you are only interested in the final result with some takeaways, feel free to jump to the Summary section.

Introduction

Hello and welcome!

If you come from the Scala world, and you are wondering how you can replicate a for comprehension expression in TypeScript, then you may be interested in this article.

Today, we are going to see how we can write such an expression using TypeScript and the fp-ts ecosystem.

The objective of this article is to transform the following Scala blocks of code:

Case 1

def foo(): Option[String] = ???
def bar(a: String): Option[Int] = ???
def baz(b: Int): Option[Int] = ???

for {
  a <- foo()
  b <- bar(a)
  c <- baz(b) if b >= 42
} yield (b + c)

// Option[Int]

Case 2

// Here, we are using some arbitrary `IO` data type.
// This could come from the cats-effect library for example.

def foo(): IO[String] = ???
def bar(): IO[Unit] = ???
def baz(a: String): IO[Int] = ???
def qux(b: Int): IO[Unit] = ???

for {
  a <- foo()
  _ <- bar()
  b <- baz(a)
  _ <- qux(b)
} yield (b + 1)

// IO[Int]

Both have some common behaviors: bindings to intermediate references such as a and b, and yielding a final value. In addition, the first one has a filtering step, and the second one has some discarded results (i.e. running side effects that do not return any value).

Depending on the version of fp-ts you are using, you might have to follow either the fp-ts or the fp-ts-contrib way. Please refer to the Prerequisites section for more information.

In both cases, we are going to use the do notation.

Prerequisites

fp-ts v2.8 or above
Or, fp-ts-contrib v0.0.2 or above

For both ways, here are the function declarations we are going to use:

Case 1

import * as O from 'fp-ts/Option'

declare function foo(): O.Option<string>
declare function bar(a: string): O.Option<number>
declare function baz(b: number): O.Option<number>

Case 2

import * as I from 'fp-ts/IO'

declare function foo(): I.IO<string>
declare function bar(): I.IO<void>
declare function baz(a: string): I.IO<number>
declare function qux(b: number): I.IO<void>

Using fp-ts

Case 1
Case 2

Case 1

Let us build the for comprehension expression step by step.

First, we have to start the do notation. There are 2 ways of doing this:

Using Do and bind:

import * as O from 'fp-ts/Option'
import { pipe } from 'fp-ts/function'

pipe(
  O.Do,
  O.bind('a', () => foo())
)

Calling foo, then using bindTo:

import * as O from 'fp-ts/Option'
import { pipe } from 'fp-ts/function'

pipe(
  foo(),
  O.bindTo('a')
)

There is no preferred way, although I personally like to make it explicit with the Do step.

At this point, the type of this expression is Option<{ a: string }>.

Tip: on VS Code, you can inspect the type inferred by TypeScript with the help of an identity function and ctrl + mouseover the parameter (cmd + mouseover on Mac):

I use this identity function quite a lot to make sure I have made the correct value transformations, step by step.

The next step is b <- bar(a). For this, we are going to use bind, while accessing the previous value bound to the a reference:

pipe(
  ...,
+  O.bind('b', ({ a }) => bar(a))
)

In this step, the second argument of bind is a function whose parameter is a { a: string }, i.e. whatever is inside the Option at this point. This allows us to read the previous value to call the bar function, and get a new value bound to the b reference.

Following this step, the type has become Option<{ a: string, b: number }>.

The next line, c <- baz(b) if b >= 42, is the most complicated one. It is composed of 2 steps: a filter, and some binding of a value.

At the time of writing these lines (fp-ts v2.11.8), there is no function available that combines both these steps, so we have to write them explicitly:

pipe(
  ...,
+  O.filter(({ b }) => b >= 42),
+  O.bind('c', ({ b }) => baz(b))
)

We could create an intermediate function, bindFilter, that would combine these 2 steps:

import { flow } from 'fp-ts/function'

const bindFilter: <N extends string, A, B>(
  name: Exclude<N, keyof A>,
  f: (a: A) => O.Option<B>,
  predicate: (a: A) => boolean
) => (ma: O.Option<A>) => O.Option<{ readonly [K in N | keyof A]: K extends keyof A ? A[K] : B }> =
  (name, f, predicate) => flow(O.filter(predicate), O.bind(name, f))

pipe(
  O.Do,
  ...,
  bindFilter('c', ({ b }) => baz(b), ({ b }) => b >= 42)
)

However, we would have to write a different version of bindFilter for each instance of Monad used in our code, e.g. for Either, IO, Task... This could be quite tedious just to abstract the filter + bind calls.

Alternatively, we could use flow to regroup these 2 steps into a single one:

+ import { flow } from 'fp-ts/function'

pipe(
  ...,
+  flow(O.filter(({ b }) => b >= 42), O.bind('c', ({ b }) => baz(b)))
)

Following this new addition, the type of our expression has changed to Option<{ a: string, b: number, c: number }>.

Finally, we have the yield (b + c) part. This one can be achieved using map:

pipe(
  ...,
+  O.map(({ b, c }) => b + c)
)

The final type has become Option<number>, which is what we were expecting to get.

Final result:

import * as O from 'fp-ts/Option'
import { pipe } from 'fp-ts/function'

pipe(
  O.Do,
  O.bind('a', () => foo()),
  O.bind('b', ({ a }) => bar(a)),
  O.filter(({ b }) => b >= 42),
  O.bind('c', ({ b }) => baz(b)),
  O.map(({ b, c }) => b + c)
)

Case 2

A lot of what we are going to use has already been seen in the previous section, for the first case. The only difference here is that we have some steps that do not return any value.

First, let us start the do notation:

import * as I from 'fp-ts/IO'
import { pipe } from 'fp-ts/function'

pipe(
  I.Do
)

Then, we can add the a <- foo() step:

pipe(
  I.Do,
+  I.bind('a', () => foo())
)

So far, the type of our expression is IO<{ a: string }>.

Following this, we have our first case where we do not care about the returned value: _ <- bar(). Since we ignore the result of calling bar, we are going to use chainFirst:

pipe(
  ...,
+  I.chainFirst(() => bar())
)

Here, we still have the same type for this expression (i.e. IO<{ a: string }>) since we ignored whatever bar() was returning.

Now comes the b <- baz(a) step. As seen in the previous section, we are going to use bind, while accessing the previous value bound to the a reference:

pipe(
  ...,
+  I.bind('b', ({ a }) => baz(a))
)

Following this step, the type has become IO<{ a: string, b: number }>.

The next step is another effect where we ignore the value it returns. However, now we need to access a previously bound value:

pipe(
  ...,
+  I.chainFirst(({ b }) => qux(b))
)

The type of the expression is still IO<{ a: string, b: number }>, since we did not bind any new value.

Finally, we have the yield (b + 1) part, that we can handle using map:

pipe(
  ...,
+  I.map(({ b }) => b + 1)
)

The final type has become IO<number>, which is what we were expecting to get.

Final result:

import * as I from 'fp-ts/IO'
import { pipe } from 'fp-ts/function'

pipe(
  I.Do,
  I.bind('a', () => foo()),
  I.chainFirst(() => bar()),
  I.bind('b', ({ a }) => baz(a)),
  I.chainFirst(({ b }) => qux(b)),
  I.map(({ b }) => b + 1)
)

Using fp-ts-contrib

Case 1
Case 2

Case 1

Let us build the for comprehension expression step by step.

First, we have to start the do notation:

import { Do } from 'fp-ts-contrib/Do'
import * as O from 'fp-ts/Option'

Do(O.Monad) // or, Do(O.option) for fp-ts prior v2.11

Then comes the first step of this expression, a <- foo(). For this, we can use the bind method:

Do(O.Monad)
+  .bind('a', foo())

Next, we have b <- bar(a), which requires us to use the bindL method to access the a reference:

Do(O.Monad)
  ...
+  .bindL('b', ({ a }) => bar(a))

Following this, we have the filtering part, c <- baz(b) if b >= 42. This is going to feel a bit "hacky" since, at the time of writing these lines (fp-ts-contrib v0.1.26), there is no filter method available in the Do builder.

To add this filter, we have to end the do expression to get a Option<{ a: string, b: number }>, then apply the filter on it, then chain the filtered Option with a new do expression to continue the computation:

+import { pipe } from 'fp-ts/function'
+
+pipe(
  Do(O.Monad)
    .bind('a', foo())
    .bindL('b', ({ a }) => bar(a))
+    .done(),
+  O.filter(({ b }) => b >= 42),
+  O.chain(({ a, b }) => Do(O.Monad)
+    .bind('c', baz(b))
+    .return(({ c }) => b + c)
+  )
+)

In the fp-ts world, chain is the same thing as flatMap.

Here are some explanations to understand what is happening there:

We end the first do notation with .done(), which yields Option<{ a: string, b: number }>.
We filter this value, using O.filter and a predicate.
At this point, we still have a Option<{ a: string, b: number }>, but we need to go back in a Do builder. For that, the trick is to chain the option we have with a new option that we will get with a new Do builder.
By chaining, we have access to the previous references a and b. Be careful though, as they are not available in the new Do builder.
We use the .return(...) method to yield an Option<number>, which is the type of value we are expecting.

Maybe someday we will have a DoFilterable builder that will feel more natural, as suggested in an open issue on the fp-ts-contrib repository.

Final result:

import { Do } from 'fp-ts-contrib/Do'
import * as O from 'fp-ts/Option'
import { pipe } from 'fp-ts/function'

pipe(
  Do(O.Monad)
    .bind('a', foo())
    .bindL('b', ({ a }) => bar(a))
    .done(),
  O.filter(({ b }) => b >= 42),
  O.chain(({ a, b }) => Do(O.Monad)
    .bind('c', baz(b))
    .return(({ c }) => b + c)
  )
)

Case 2

As usual, let us start the do notation:

import { Do } from 'fp-ts-contrib/Do'
import * as I from 'fp-ts/IO'

Do(I.Monad) // or, Do(I.io) for fp-ts prior v2.11

The next step, a <- foo(), is pretty straightforward:

Do(I.Monad)
+  .bind('a', foo())

Following this, we have the first case where we ignore the returned value. For this, we can use the do method:

Do(I.Monad)
  ...
+  .do(bar())

Then, we can use bindL to implement the b <- baz(a) part:

Do(I.Monad)
  ...
+  .bindL('b', ({ a }) => baz(a))

After that, we have a second case where the returned value is ignored. However this time, we need a previous value: _ <- qux(b). For that, we can use the doL method:

Do(I.Monad)
  ...
+  .doL(({ b }) => qux(b))

The last step, yield (b + 1), can be achieved using the return method:

Do(I.Monad)
  ...
+  .return(({ b }) => b + 1)

Final result:

import { Do } from 'fp-ts-contrib/Do'
import * as I from 'fp-ts/IO'

Do(I.Monad)
  .bind('a', foo())
  .do(bar())
  .bindL('b', ({ a }) => baz(a))
  .doL(({ b }) => qux(b))
  .return(({ b }) => b + 1)

Summary

Sadly, there is no syntactic sugar available for monad composition in TypeScript. As fp-ts is one of the most popular functional libraries in this language, chances are you will use it to build functional programs.

Hopefully, this article can help you transpose your Scala knowledge regarding for comprehension expressions into a TypeScript functional project, using the do notation.

Please, let me know if this is of any help :)

Thank you for reading this far, and have a wonderful day!

Using fp-ts

Case 1

for {
  a <- foo()
  b <- bar(a)
  c <- baz(b) if b >= 42
} yield (b + c)

pipe(
  O.Do,
  O.bind('a', () => foo()),
  O.bind('b', ({ a }) => bar(a)),
  O.filter(({ b }) => b >= 42),
  O.bind('c', ({ b }) => baz(b)),
  O.map(({ b, c }) => b + c)
)

Case 2

for {
  a <- foo()
  _ <- bar()
  b <- baz(a)
  _ <- qux(b)
} yield (b + 1)

pipe(
  I.Do,
  I.bind('a', () => foo()),
  I.chainFirst(() => bar()),
  I.bind('b', ({ a }) => baz(a)),
  I.chainFirst(({ b }) => qux(b)),
  I.map(({ b }) => b + 1)
)

Takeaways

Start the do notation with Do + bind
To discard the result, e.g. _ <- foo(), use chainFirst
To use a filter, e.g. x <- foo() if y, use filter + bind
End the do notation (yield x) with map

Using fp-ts-contrib

Case 1

for {
  a <- foo()
  b <- bar(a)
  c <- baz(b) if b >= 42
} yield (b + c)

pipe(
  Do(O.Monad)
    .bind('a', foo())
    .bindL('b', ({ a }) => bar(a))
    .done(),
  O.filter(({ b }) => b >= 42),
  O.chain(({ a, b }) => Do(O.Monad)
    .bind('c', baz(b))
    .return(({ c }) => b + c)
  )
)

Case 2

for {
  a <- foo()
  _ <- bar()
  b <- baz(a)
  _ <- qux(b)
} yield (b + 1)

Do(I.Monad)
  .bind('a', foo())
  .do(bar())
  .bindL('b', ({ a }) => baz(a))
  .doL(({ b }) => qux(b))
  .return(({ b }) => b + 1)

Takeaways

Start the do notation with Do(monadInstance)
To discard the result, e.g. _ <- foo(), use doL if you require a previous value, or do if you do not
End the do notation (yield x) with return if you need to compute some value, or done if you want to preserve the "context" object containing all the references
To use a filter, e.g. x <- foo() if y, end the do notation with done, then use filter, then chain with a new do notation, with chain and Do(monadInstance)

More information regarding this do notation on the following article: Do syntax in TypeScript.

Side effects

Benoit Ruiz — Wed, 16 Feb 2022 13:03:48 +0000

Introduction
How to deal with side effects
Chaining side effects

Introduction

Initially, I wanted to talk about function purity and referential transparency at this point of the series. However, when preparing the article, I used a significant amount of references to the term "side effect". This is why I chose to introduce side effects before talking about function purity. Both concepts are related to one another, but they deserve their separate article.

So, what is a side effect? A side effect is an interaction with the outside world.

Now you might be wondering, what is the "outside world" then? It is anything that is not part of the domain layer of the software. Here are some examples of these types of interactions:

Reading a file, and/or writing to a file
Making a network request (calling an API, downloading a file...)
Reading from a global state (e.g. global variable, a parameter from the parent's closure...)
Throwing/intercepting an exception
For web applications, using DOM objects and methods
Logging messages to the console

The domain layer contains all the models, logic, and rules of the domain. If you are familiar with the "hexagonal architecture", also known as "ports and adapters architecture", then this layer is at the very center of the diagram.

Generally speaking, all the functions from the domain layer can be tested with unit tests, as there are no side effects. This is the only layer that is pure, all the others encompassing it have side effects.

In functional programs, functions have to be pure, i.e. side-effect-free. Why? Because side effects are not deterministic, and they are hard to reason about.

When we want to validate that a function works as expected, we have to be able to understand it and use automated tests. Introducing non-determinism and external references into the function makes it hard to understand it simply, and test its behavior.

That being said, a program that has no side effects is not very useful. We need interactions with the outside world, such as getting input from a user, writing to a database, or displaying something to the screen.

So, on the one hand, side effects are bad, but on the other hand, they are mandatory to build useful programs. What can we do about it?

How to deal with side effects

One particularity of a side effect is that it just happens. In our imperative programs, a side effect does not return anything. In other words, we cannot store it in a variable, or pass it as a parameter of a function.

We can easily identify functions that perform side effects, as they are the ones that do not return any value. For example:



function trustMeNoSideEffectHere(): void {
  ...
}

This function's name is misleading as its return type does not match its name. The void type indicates that this function does not return any value. Therefore, if it does not compute any value, it is either useless or performs at least a side effect.

Of course, we can still have functions that return a value, but also perform side effects:



function trustMeNoSideEffectHere(): number {
  let n = 21
  window.foo.bar++ // mutating the global state = side effect
  return n * 2
}

In an imperative program with no defined rules regarding side effects, we must read the definition of the function to understand what it does, thus losing the type-based reasoning benefit.

Furthermore, the moment we call the trustMeNoSideEffectHere function, the side effect immediately happens. We have to make sure the environment is ready right before calling the function, otherwise, we are exposed to undesired behaviors.

In functional programs, since functions are pure (i.e. no side effects), they must return a value, otherwise, they are pretty much useless. In addition, determinism is a key aspect of functional programming. We cannot allow a function to trigger a side effect just by calling it. As developers, we want to control when the side effect happens (remember, side effects must happen at some point, otherwise the program is pointless).

How can we make the trustMeNoSideEffectHere function pure, and control when the side effect (i.e. global state mutation) happens?

One way of doing it would be to wrap some parts of this function in another function, making it "lazy":



function trustMeNoSideEffectHere(): () => number {
  let n = 21
  return () => {
    window.foo.bar++ // mutating the global state = side effect
    return n * 2
  }
}

What happens when we call trustMeNoSideEffectHere()? We get a value of type () => number, that we can store in a variable:



const res: () => number = trustMeNoSideEffectHere()

Did anything happen? No. The global state has not been mutated yet since we have not called res() yet. Though, we changed the definition of trustMeNoSideEffectHere to make it pure: it always returns the same value, a function.

Granted, if we call the function twice, we will get different references for the inner function: trustMeNoSideEffectHere() === trustMeNoSideEffectHere() would be false. So technically, we do not return the "same" value. Still, since functions equality is a complex topic, let us assume that the values are equivalent here, and move on.

Now that the function is deterministic, we can write a unit test, although it is not very useful in this case:



expect(trustMeNoSideEffectHere()).to.be.a('function')

Imagine you are testing the launchNuclearWarheads function. It is pretty nice to make sure that calling launchNuclearWarheads() does not do anything, and that it requires an extra step to actually end life on Earth.

The moment we are calling res(), the side effect will happen, thus propagating the non-determinism to the parent functions.

At this point, it is not possible to write unit tests for the main, serviceA, and serviceB functions. What can we do to make these functions deterministic? We can apply the same strategy as before: returning functions that wrap the side effect (cf. the blue wrapping function on the following picture).

We managed to make the serviceA and serviceB functions side-effect-free. Only the main function is impure and performs side effects, which is convenient: now we know where all the side effects actually happen in the program.

Chaining side effects

What happens when we want to chain side effects? For example:

First we want to ask for the user's name
Then we want to perform some API call with the user's name (e.g. to retrieve some information regarding its origin)
Finally we want to display the result on the screen

Again, we can use the same technique: make any side effect lazy by wrapping it inside a function.

(here we are assuming the code is run in a browser)



function askForUserName(): () => string {
  return () => prompt('What is your name?')
}

function getUserNameOrigin(userName: string): () => Promise<string> {
  const sanitizedName = userName.toLowerCase()
  return () => fetch(`/get-origin/${sanitizedName}`)
    .then(res => res.json())
    .then(res => res.userNameOriginText)
}

function displayResult(text: string): () => void {
  return () => {
    document.getElementById('origin-text')?.innerText = text
  }
}

function originNameService(): () => Promise<void> {
  return () => {
    const lazyUserName = askForUserName()
    const lazyGetUserNameOrigin = getUserNameOrigin(lazyUserName())
    return lazyGetUserNameOrigin()
      .then(originText => {
        const lazyDisplayResult = displayResult(originText)
        lazyDisplayResult()
      }) 
  }
}

function main(): void {
  const lazyOriginNameService = originNameService()
  lazyOriginNameService()
}

Here, only the main function is non-deterministic, the remaining functions are all side-effect-free, thanks to the wrapper functions.

That being said, writing tests for these pure functions would not be very useful, as we would not test their actual behavior:



expect(askForUserName()).to.be.a('function')
expect(getUserNameOrigin('foo')).to.be.a('function')
expect(displayResult('bar')).to.be.a('function')

To overcome this, we could use Dependency Injection, a.k.a DI. This is a bit out-of-scope for this article, but essentially we would pass the actual effect (i.e. prompt, fetch...) as a dependency of the function, and we would use a mocked/controlled version of this effect for writing the unit tests.

For example:



interface Dependencies {
  prompt: (text: string) => string
}

function askForUserName({ prompt }: Dependencies): () => string {
  return () => prompt('What is your name?')
}

const lazyUserName: () => string =
  askForUserName({ prompt: window.prompt })



const fakePrompt = (text: string) => 'foo'

const lazyUserName: () => string =
  askForUserName({ prompt: fakePrompt })

expect(lazyUserName).to.be.a('function')
expect(lazyUserName()).to.equal('foo')

What if we wrote a simple type alias for the introduction of laziness?



type IO<A> = () => A

function askForUserName(): IO<string> { ... }
function getUserNameOrigin(userName: string): IO<Promise<string>> { ... }
function displayResult(text: string): IO<void> { ... }
function originNameService(): IO<Promise<void>> { ... }
function main(): void { ... }

And what about a special alias for the lazy promises?



type Task<A> = IO<Promise<A>>

function askForUserName(): IO<string> { ... }
function getUserNameOrigin(userName: string): Task<string> { ... }
function displayResult(text: string): IO<void> { ... }
function originNameService(): Task<void> { ... }
function main(): void { ... }

The type of main is () => void, so in other words, main: IO<void>. Also, askForUserName and originNameService do not take any argument, so they are already lazy. We can simplify their definition with:



const askForUserName: IO<string> = ...
const originNameService: Task<void> = ...
const main: IO<void> = ...

Now imagine if these IO and Task types had some behavior attached to them, allowing us to do something such as:



const originNameService: Task<void> =
  Task.fromIO(askForUserName)
    .andThen(getUserNameOrigin)
    .andThen(text => Task.fromIO(displayResult(text)))

This is how we can contain side effect declarations inside values, that we can compose with other values, thus describing more and more complex side effects. To run the side effects, we can simply run the final value obtained from the composition:



// assuming we have some behavior attached to our Task<void> value

originNameService.runEffect()

Here, I presented one approach we can use to handle side effects.

We have separated their declaration from their execution, giving us more flexibility to combine these effects, and more control over what happens in our program.

Functional languages and libraries provide "standard" tools to deal with side effects, so we do not have to build them ourselves. Haskell has a built-in IO data type. The fp-ts TypeScript library provides both IO for synchronous side effects, and Task for asynchronous ones. The cats-effect Scala library provides the IO data type as well.

Writing functional programs will undoubtedly lead you to use these data types to deal with side effects.

The goal is to have as many side-effect-free functions as possible in our programs, by leveraging side effects declarations, and isolating side effects execution.

This concludes the chapter on side effects. Hopefully, you enjoyed it and understood what side effects are, and how you can deal with them.

The next step is to talk about function purity and referential transparency. As always, feel free to leave a comment :)

See you then!

Special thanks to Tristan Sallé for reviewing the draft of this article.

Photo by Xavi Cabrera on Unsplash.

Pictures made with Excalidraw.

Declarative vs imperative

Benoit Ruiz — Thu, 07 Oct 2021 16:01:40 +0000

Introduction
Making a chocolate cake
Some examples
When to use declarative code
Conclusion

Introduction

Functional Programming is a declarative programming paradigm, in contrast to imperative programming paradigms.

Declarative programming is a paradigm describing WHAT the program does, without explicitly specifying its control flow.

Imperative programming is a paradigm describing HOW the program should do something by explicitly specifying each instruction (or statement) step by step, which mutate the program's state.

This "what vs how" is often used to compare both of these approaches because... Well, it is actually a good way to describe them.

Granted, at the end of the day, everything compiles to instructions for the CPU. So in a way, declarative programming is a layer of abstraction on top of imperative programming.

At some point, the state of the program must be changed in order for things to happen, and these changes can only occur with instructions moving data from one location (cache, memory, hard drive...) to another. But we are not here to talk about low-level programming, so let's focus on high-level languages instead.

The transformation from declarative to "imperative code" is generally made by engines, interpreters, or compilers.

For example, SQL is a declarative language. When using the SELECT * FROM users WHERE id <= 100 query, we are expressing (or declaring) what we want: the first 100 users ever registered in the database. The way how these rows are retrieved is completely delegated to the SQL engine: can it use an index to accelerate the query? Should/Can it use multiple CPU cores to finish earlier?

From a developer's point of view, we have no idea how these data are actually retrieved. And we don't really care, unless we are investigating some performance issues. All we care about is telling the program what data we want to retrieve, and not how to do it. The engine/compiler is smart enough to find the most optimal way to do that anyway.

For languages that use a declarative paradigm (e.g. Haskell, SQL), this "underlying imperative world" is abstracted/hidden to the developers. It is something we don't have to worry about.

For languages that are multi-paradigms (e.g. JavaScript, Scala), there is still the possibility to write imperative code. This allows us to write declarative code based on imperative code that we wrote ourselves. This can be useful to support FP features that are not built-into the language for example, or just to make the code more "declarative", which makes it more readable and understandable, in my opinion.

The imperative code is abstracted by the declarative one, which is the one used by the developers to actually write the software. The imperative part becomes an implementation detail of the software.

Making a chocolate cake

Let's take an example from the real world: we would like to make a chocolate cake. How would that look like with these 2 paradigms?

The imperative way

First, turn on the oven to preheat it at 180°C.
Next, add flour, sugar, cocoa powder, baking soda and salt to a large bowl, then stir the mixture with a paddle.
Then, add milk, vegetable oil, eggs and vanilla extract to the mixture, and mix together on medium speed until well combined.
Distribute the cake batter evenly in a large cake pan, then bake it for approx. 30 minutes.
Remove the pan from the oven with a pot holder, let it cool for 10 minutes.
Finally, remove the cake from the pan with the tapping method, and frost it evenly with chocolate frosting.

The declarative way

You have to preheat the oven to 180 °C.
You have to mix dry ingredients in a bowl.
Once dry ingredients are mixed, you have to add wet ingredients to the mixture, and mix together to form the cake batter.
Once the oven and batter are ready, you have to put the batter in a pan, then bake it for 30 minutes.
Once baked, you have to remove the pan from the oven and let it cool for 10 minutes.
Finally, you have to remove the cake from the pan, and frost it.
Ready? Go!

Analysis

In the imperative way, we are told what to do, and more importantly how to do it: use a large bowl, mix with a paddle, mix at medium speed, use a large pan, distribute batter evenly, remove pan with a pot holder, use the tapping method, frost evenly.

These details are great when actually making a cake, especially as a beginner. But when describing how to make one, on a "higher level" of abstraction, we don't need all these information.

Furthermore, we are actually doing something at each step, i.e. we are changing the world around us, step by step. If we choose to stop at an intermediate step, then we basically "wasted" all the tools and ingredients from the previous steps.

In the declarative way, we are told what we will have to do to make the cake. Nothing actually happens until the last step, i.e. the world doesn't change until we have reached the 7th step.

In other words, we are preparing all the steps in advance, then at the very end, we are doing what was described. How do we perform the actions described in these steps though? It's abstracted: all the "how" parts are provided as later as possible, between the "Ready?" and "Go!", either by the developer (for multi-paradigms languages) or by the engine/compiler. For example, this is where the binding between "remove the pan from the oven" and "using a pot holder" is done. We could also bind it to "using the pan handle", without changing the definition of the 5th step.

Some examples

Let's say we want to double every value of a given list of numbers. There are plenty of ways to iterate over a list and transform each of its elements in JavaScript:

Declarative: recursive function, or functions already available such as the map and reduce methods of arrays
Imperative: for loop, while loop

To demonstrate that imperative code can be abstracted by declarative code, we could use a for loop and hide it inside a transformEachElement function:



// "hidden" in a utils/helper/whatever module, or library-like
function transformEachElement<A, B>(
  elements: A,
  action: (element: A) => B
): B[] {
  const result = []
  for (let i = 0; i < elements.length: i++) {
    result.push(action(elements[i]))
  }
  return result
}

// What do we want? Double each number of a given list
const res = transformEachElement([1, 2, 3], n => n * 2)

But we could use map directly as it's already declarative, and widely known for this type of use case:



const res = [1, 2, 3].map(n => n * 2)

Here is another example, where we want to target the text from an element of a web page. This element's location is a few levels down in the elements hierarchy (called the DOM tree). The twist is that each of these elements may not exist in practice.

So, each time we progress by one node in the tree, we have to check if the next node is available or not.

The imperative way could look like this:



function getMainTitle(): string | null {
  const main = document.getElementById('main')
  if (main !== null) {
    const title = main.querySelector('.title')
    if (title !== null) {
      const text = title.querySelector<HTMLElement>('.title-text')
      if (text !== null) {
        return text.innerText
      } else {
        return null
      }
    } else {
      return null
    }
  } else {
    return null
  }
}

This is pretty verbose, and the more depth there is to reach an element, the bigger the pyramid of doom gets.

Additionally, we have leaked an implementation detail: a node that doesn't exist has the value null. It could have been undefined, or 'nothing', or something else entirely. The point is that we have to understand that null is the magic value expressing the absence of an element in the tree here. It should not be necessary to know that to understand what this function does.

Here is a more declarative approach:



const main: Option<Element> =
  Option(document.getElementById('main'))

function getTitle(main: Element): Option<Element> {
  return Option(main.querySelector('.title'))
}

function getTitleText(title: "Element): Option<HTMLElement> {"
  return Option(
    title.querySelector<HTMLElement>('.title-text')
  )
}

function getMainTitle(): Option<string> {
  return main
    .flatMap(getTitle)
    .flatMap(getTitleText)
    .map(text => text.innerText)
}

In this second version, all we care about is accessing an element in the tree, where each intermediate element could be missing. In other words, we have written "what" to do in order to access the element containing the text we are looking for.

This supposes that we have access to some Option data structure in our code base. There are plenty of articles available on the Internet that talk about this Option (also known as Maybe) data type. Essentially, it allows us to express the possible absence of a value, transform it if the value is available, and combine it with other possible missing values, all that in a declarative way.

In fact, this data type is so useful that some languages already provide it in their standard library (e.g. Scala, Haskell, F#), even the more mature ones (e.g. Optional in Java, C++).

The flatMap and map terms may seem "mystical" at this point. We will talk about them by the end of this series, in the article about algebraic data structures and type classes. In functional programs, you will often encounter these functions or their equivalent, depending on the language:

map is also known as fmap, lift, <$>

flatMap is also known as bind, chain, >>=

A couple of years ago (Dec. 2019), the optional operator proposal reached stage 4 in the EcmaScript specification, used for both JavaScript and TypeScript. This allows us to greatly simplify the code from above, without relying on any library:



function getMainTitle(): string | null {
  return document.getElementById('main')
    ?.querySelector('.title')
    ?.querySelector<HTMLElement>('.title-text')
    ?.innerText
}

This still "leaks" the fact that either null or undefined values should be used to mark an element as missing, but it is still way more expressive than the first imperative version from earlier.

When to use declarative code

This section applies only to muli-paradigms languages. Obviously, if you are using a functional language such as Haskell, you are always using declarative code.

So, it is possible to make imperative code look like declarative code, to some extent. In such case, I would suggest isolating the imperative parts from the rest of the code base, to make sure developers use the "declarative" functions instead.

In multi-paradigms languages, the scale between declarative and imperative is not a clear black/white separation, but rather multiple shades of grey. It is up to us to determine which shade is the best for our projects and teams.

Here is a non-exhaustive list of pros and cons for each of these approaches, based on my experience:

Declarative

Pros	Cons
Better readability and understanding of the code	More lines of code, where a potential bug could hide
Better control over the actual execution of the changes to the world	Potential loss of performance, due to more memory allocation and intermediate function calls
	Longer debugging, due to bigger stack traces
	Developers are usually less comfortable with this way of programming

Imperative

Pros	Cons
Less code overall, as there is no need to wrap imperative code inside declarative functions	More time taken to read and understand what the code does
Shorter debugging, due to smaller stack traces	But harder debugging overall, due to state mutations and "less-controlled" changes to the world
Developers are usually more comfortable with this way of programming

Since code is destined to be read and understood by human beings, I think it is a good practice to use more declarative programming in our softwares.

Sometimes, performance is critical and requires the use of imperative programming (we are talking about multi-paradigms languages here). In such cases, comments and documentation are crucial to understand the code base. Otherwise, some exceptions put aside, code should be self-explanatory through good naming and declarative steps, and should not require comments to understand it well.

For strictly-declarative languages such as Haskell and SQL, the compiler/engine makes the best optimizations possible; so there is no need (and no way anyway) to write imperative code to improve performance.

Conclusion

In this article, I tried to illustrate (with some examples) the difference between these 2 approaches, and the advantages of the declarative way. The biggest benefit is making the code more readable and understandable.

Misunderstanding the responsibility of some part of the code base is one of the most common reasons why bugs are introduced in the first place. It is also one of the reasons why adding improvements and features takes more time, as we need to first understand what the code does before making any changes.

Functional Programming is about expressing "what" we want to do with data, but not actually doing anything until the very last moment. Doing something requires changing state and running statements. These parts are handled by engines/interpreters/compilers, since they know "how" to efficiently do "what" we wrote in the code base.

It is not a requirement to fully understand this way of writing code, because it will come naturally the more functional code you write. By going through the articles of this series, you will see that declarative programming is ubiquitous, despite not being mentioned explicitly.

Thank you for reading this far! As always, feel free to leave a comment if need be. The next article will talk about pure functions and referential transparency. See you there!

Special thanks to Tristan Sallé for reviewing the draft of this article.

Photo by Xavi Cabrera on Unsplash.

Pictures made with Excalidraw.

Function composition and higher-order function

Benoit Ruiz — Thu, 16 Sep 2021 16:32:36 +0000

Function composition
How far we can go with it
Higher-order function

Function composition

Function composition is a simple yet powerful key concept in Functional Programming.

What is it?

In a nutshell, it's the ability to combine behaviors together in a specific order, transforming data little by little from an initial shape into a new one.

More concretely, it's the combination of functions where the output (returned value) of the previous function becomes the input (argument) of the next one, and so on until the last function.

We can visualize it as a pipeline where some data of a given shape enters on one side, then comes out with a new (same or different) shape on the other side.

Composing functions yields... Another function! Therefore a pipeline is, in turn, a function. We don't know exactly how it's defined from an external point of view, and that's why it's so powerful: it's an abstraction that hides implementation details. The intermediate steps to transform a BlueSquare into a PurpleHexagon are hidden/abstracted.

Given the following 3 functions f, g, and h:

Can we compose these functions to create the pipeline from above? Let's see:

f takes a BlueSquare as input, then returns a GreenCircle
g takes a GreenCircle as input (therefore, it can be composed with f, if f comes first), then returns a RedDiamond
h takes a RedDiamond as input (therefore, it can be composed with g, if g comes first), then returns a PurpleHexagon

By aligning f, then g and finally h, we end up with a pipeline that takes a BlueSquare as input, and returns a PurpleHexagon, which is exactly what we were looking for!

The weird notation with circles is the way function composition in mathematics is written: h ∘ g ∘ f can be read as "h round g round f", and is applied from right to left: h(g(f(x))).

How far we can go with it

Since pipelines are functions, it means we can compose them with other functions as well. This is the essence of software programming: composing smaller pieces together to end up with a complex system that fulfills all the requirements.

Suppose we need to transform some GreenCircle into a PurpleHexagon somewhere in our code base. Do we have to implement this function by ourselves, or can we simply compose functions that already exist?

Here, we are assuming the actual implementations of the functions from above suit our needs. In reality, there could be several implementations for a function transforming a GreenCircle into a PurpleHexagon. For example, isEven and isOdd are both functions that transform numbers (green circles) into booleans (purple hexagons), but with different implementations.

Of course we can, by composing g with h:

And since this pipeline is a function, we can actually define the previous pipeline using this one and f:

This way, we have reusable units/functions in our code base that can be composed anywhere to form new functions/pipelines that are more and more complex.

Shapes are nice, but if you're looking for something more concrete, here's an example with simple functions in TypeScript:

const getNumberOfCharacters =
  (text: string): number => text.length

const increment = 
  (n: number): number => n + 1

const isGreaterThan5 =
  (n: number): boolean => n > 5

// pipeline definition
const isTextLongEnough: (text: string) => boolean =
  flow(getNumberOfCharacters, increment, isGreaterThan5)

console.log(isTextLongEnough('abc')) // false
console.log(isTextLongEnough('abcde')) // true

I'm using the flow function from the fp-ts library, which is used specifically to compose functions, from left to right. I could've written it the following way, but it's less readable in my opinion:

const isTextLongEnough: (text: string) => boolean =
  text => isGreaterThan5(increment(getNumberOfCharacters(text)))

The advantage of having small units/functions is that we can reuse some of them to create different pipelines. For example:

const isEven =
  (n: number): boolean => n % 2 === 0

const hasEvenNumberOfCharacters: (text: string) => boolean =
  flow(getNumberOfCharacters, isEven)

console.log(hasEvenNumberOfCharacters('abc')) // false
console.log(hasEvenNumberOfCharacters('abcd')) // true

Wait a minute... All these functions have only 1 argument, also called unary functions. What happens if we want to compose functions that take multiple arguments? For example:

Or more concretely:

const isGreaterThan =
  (n: number, limit: number): boolean => n > limit

The answer lies in currying and partial application, which are 2 concepts we'll see later in this series. Stay tuned!

Higher-order function

A higher-order function, or HOF for short, is a function that takes at least a function as its argument(s), and/or returns a new function.

If you have used the map and filter methods on an array in JavaScript, then you have already used HOFs, because these 2 methods take a function as their arguments:

const evenNumbers = [1, 2, 3, 4].filter(n => n % 2 === 0)
// or the shorter version: [1, 2, 3, 4].filter(isEven)

console.log(evenNumbers) // [2, 4]

const doubledNumbers = [1, 2, 3].map(n => n * 2)

console.log(doubledNumbers) // [2, 4, 6]

We can create our own higher-order functions as well. Let's have a look at a simple example, in TypeScript and Scala:

const enum Shape {
  BlueSquare,
  GreenCircle,
  RedDiamond
}

function createShapes(
  volume: number,
  generateShape: () => Shape
): Shape[] {
  return [...new Array(volume)].map(() => generateShape())
}

sealed trait Shape
case object BlueSquare extends Shape
case object GreenCircle extends Shape
case object RedDiamond extends Shape

def createShapes(
  volume:        Int,
  generateShape: () => Shape
): List[Shape] =
  List.range(0, volume).map(_ => generateShape())

Here the createShapes function is a HOF, because it takes a function as one of its arguments: generateShape.

If you are familiar with design patterns from Object-Oriented Programming, then you might have thought about the Strategy pattern here, and you would've been correct. In fact, a lot of OO design patterns can be implemented using HOFs. That being said, some of the OO patterns don't make sense in FP, since we are not trying to solve problems with classes and objects, but rather with data and functions.

The flow function that we saw earlier is also a HOF, since it takes multiple functions as its arguments, and returns a function at the end.

We can define a higher-order function in TypeScript to emulate some form of pattern matching on these shapes:

function matchShape(
  onBlueSquare:  (shape: BlueSquare) => void,
  onGreenCircle: (shape: GreenCircle) => void,
  onRedDiamond:  (shape: RedDiamond) => void
) {
  return (shape: Shape): void => {
    switch (shape) {
      case BlueSquare:
        return onBlueSquare(shape)
      case GreenCircle:
        return onGreenCircle(shape)
      case RedDiamond:
        return onRedDiamond(shape)
    }
  }
}

matchShape(
  shape => console.log('Blue is my favorite color!', shape),
  shape => console.log('I love green tea!', shape),
  shape => console.log('I have a red car!', shape)
)(GreenCircle)

In this example, matchShape is a HOF as it takes 3 arguments that are functions, and returns a new function that takes a Shape as its argument.

If you are from the web app development world, you might have crossed path with Higher-Order Components, or HOCs. In React, a HOC is a function that takes a component as its argument, and returns a new component. We can clearly see the similarities with HOFs here.

Alright, we've reached the end of this chapter.

Summary

Function composition is the ability to combine functions together, in a specific order, to transform some shape into a new shape.
A higher-order function, or HOF, is a function that takes at least 1 function as its argument(s), and may return a new function.

Thank you for reading this far! I hope everything was clear and you learned something. If you have any question or need some clarifications, feel free to leave a comment :)

The next article in this series will be about "declarative vs imperative" programming. See you next time!

Photo by Xavi Cabrera on Unsplash.

Pictures created with Excalidraw.

Why should we learn and use FP?

Benoit Ruiz — Fri, 10 Sep 2021 15:56:39 +0000

Before trying and adopting a new paradigm, it is only legitimate to ask ourselves this question: why should we spend some of our precious time learning this thing?

Allow me to present my list of pros and cons of doing FP when writing software.

Composition over inheritance
Extensibility
Testability
Type-based reasoning
Concurrency and parallelism
Debugging
Adoption rising among libraries and frameworks
Something new to learn
It's hard to get on board
Some languages are more adapted than others

Composition over inheritance

One of the problems with inheritance is that it leads to less and less code reusability and flexibility. To quote Joe Armstrong, author of the Coders at Work book:

I think the lack of reusability comes in object-oriented languages, not functional languages. Because the problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.

(emphasis added)

Let's take a case study involving animals.

Inheritance approach

abstract class Animal {
  constructor(private readonly name: string) {}
  eat() {}
}

abstract class WalkingAnimal extends Animal {
  walk() {}
}

abstract class SwimmingAnimal extends Animal {
  swim() {}
}

class Dog extends WalkingAnimal {
  constructor() { super('dog') }
  bark() {}
}

class Dolphin extends SwimmingAnimal {
  constructor() { super('dolphin') }
  playWithPufferFish() {}
}

Composition approach

const withName = (name: string) => ({ name })
const canEat = { eat: () => {} }
const canWalk = { walk: () => {} }
const canSwim = { swim: () => {} }
const canBark = { bark: () => {} }
const canPlayWithPufferFish = { playWithPufferFish: () => {} }

const createDog = () => ({
  ...withName('dog'), ...canEat,
  ...canWalk, ...canBark
})
const createDolphin = () => ({
  ...withName('dolphin'), ...canEat,
  ...canSwim, ...canPlayWithPufferFish
})

What if we want to have an animal that can bark and swim?

We can't use the inheritance approach without duplicating some code. However, with the composition approach, such animal becomes trivial to implement: the existing code can be reused without any duplication. Furthermore, having granular behaviors allows for more flexibility when composing these behaviors together, and creating new types of animals.

const createSeaGoodBoy = () => ({
  ...withName('good boy of the seven seas'),
  ...canEat, ...canBark, ...canSwim
})

Functional Programming favors composition over inheritance. I am not saying we can't do composition with OOP (in fact we can use interfaces and delegation for that), but it is more tempting to use inheritance, which leads to the problems we just talked about.

Extensibility

What if you want to add more functionality to an existing type? For instance in JavaScript, adding new behaviors to the Array or String data types.

Well, you could do this:

Array.prototype.getEvenNumbers = function getEvenNumbers() {
  return this.filter(_ => _ % 2 === 0)
}

String.prototype.containsFoo = function containsFoo() {
  return /foo/i.test(this)
}

const res0 = [1, 2, 3, 4, 5].getEvenNumbers() // [2, 4]
const res1 = 'Hello, World!'.containsFoo() // false

But I wouldn't recommend it, as it could alter the behavior of other scripts using the same scope. Plus, in the majority of languages, it is impossible to modify existing classes anyway, whether they are coming from the standard library or third-party libraries.

In FP, the data and the functions are 2 distinct entities. In fact, functions take data (and additional arguments if needed) as input, then return data as output.

This is in contrast to OOP where data (properties) are put together with functions (methods) under the same entity (object).

Separating data and functions allows us to add new functionalities quite easily: we simply have to add a function that takes the data as one of its arguments.

const getEvenNumbers =
  (ns: number[]): number[] => ns.filter(_ => _ % 2 === 0)

const containsFoo = (s: string): boolean => /foo/i.test(s)

const res0 = getEvenNumbers([1, 2, 3, 4, 5]) // [2, 4]
const res1 = containsFoo('Hello, World!') // false
const res2 = getEvenNumbers(['foo', 'bar', 'baz']) // compiler error

This pattern may be known as the visitor pattern to some OOP developers.

In some languages such as Scala, there are techniques used to keep a fluent API*, despite separating the data from the functions.

In F# for example, this would be the pipeline operator. In fact, as this approach is becoming more and more popular, there is a pipeline operator proposal for JavaScript in active development (it actually reached stage 2 quite recently, which means it's serious business).

* A fluent API is a way to chain function calls on some data, e.g. myData.f().g().h() or myData |> f |> g |> h, instead of h(g(f(myData))).

Testability

One of the key concepts of FP is isolating side-effects and composing pure functions. We'll see what these concepts are in the next articles of this series, but essentially, this allows us to easily write unit tests to cover all the possible cases of our functions.

Let's take a simple example: we have a module that, given some threshold, allows to log a message or not (i.e. logs are sampled).

An initial implementation could look like this:

const logThreshold = 0.05 // 5% sampling

export function canLog(): boolean {
  return Math.random() <= logThreshold
}

And this is how we could write unit tests:

describe('canLog', () => {
  const originalMathRandom = Math.random

  afterEach(() => Math.random = originalMathRandom)

  it('should allow logging', () => {
    Math.random = () => 0.01
    expect(canLog()).toBe(true)
  })

  it('should not allow logging', () => {
    Math.random = () => 0.5
    expect(canLog()).toBe(false)
  })
})

It works, but we had to perform extra steps to make sure to:

Mock the Math.random function, so it always returns the values we want in order to test the behavior of the module in a consistent way,
and restore the original value of Math.random, to avoid breaking other unit tests based on this global function.

Here, the canLog function is impure as it's performing a side-effect: calling the global Math.random function that returns a random number. In order to prevent this, we can do the following:

export function canLog(rng: () => number): boolean {
  return rng() <= logThreshold
}

Now writing unit tests is actually easier and straightforward:

describe('canLog', () => {
  it('should allow logging', () => {
    expect(canLog(() => 0.01)).toBe(true)
  })

  it('should not allow logging', () => {
    expect(canLog(() => 0.5)).toBe(false)
  })
})

Type-based reasoning

This applies only to programming languages that provide a static type system, which is often the case for languages that have a compiler, in my experience.

Type-based reasoning is the ability to understand what the program does only by reading the types and function signatures. One does not have to read and understand the actual implementation in order to understand what the program does.

This is quite a powerful feature, and I would say it's mandatory for doing Domain-Driven Design, since domain-specific rules can be encoded at the type level of a program. I actually wrote a series about Domain-Driven Design in TypeScript, feel free to have a look.

Let's take an example.

declare function isNumber(n: unknown): n is number
declare function isInteger(n: number): n is Integer
declare function isStrictlyPositiveInteger(
  n: Integer
): n is StrictlyPositiveInteger
declare function isOddInteger(
  n: StrictlyPositiveInteger
): n is OddInteger
declare function oddIntegersSum(ns: OddInteger[]): OddInteger

declare function program(
  values: unknown[]
): OddInteger | undefined

Here, I'm only defining the function signatures. There is no actual runtime code written anywhere and yet, we can understand what is going on, or at least have a good idea of what the program does.

It looks like the program takes a list of unknown values, and it returns either a single OddInteger, or undefined (probably if there's no odd integer in the list of random values provided). We can see among the function signatures above that this program should filter only the odd integers from the list, then sum them to return a single OddInteger, if odd integers are available.

The actual implementation of program could be different, for example it could simply return the first OddInteger of the list, instead of their sum. But at least we have some degree of understanding about this program, only by reading the types. Of course, having a meaningful name such as getOddIntegersSum or getFirstOddInteger instead of program would also help a lot.

And since these functions are pure, we won't be surprised by a "HTTP" call or some database operation in the middle of the implementation. If such an event should happen, then it must be "documented" in the types used in the function signature. We'll certainly talk about this in more details in the article about side-effects.

Concurrency and parallelism

Since data is immutable in a program written in FP, the entire class of problems related to race conditions (from the imperative world) don't apply. This leaves the developers with fewer error cases to check if the program doesn't work as expected.

Adapting a single-threaded program into a multi-threaded one should be way easier to do with FP than with any form of imperative programming.

Debugging

Debugging code becomes easier. Finding a bug is much more direct as you can clearly define the inputs and check the outputs of the functions that compose the program.

There is no shared state, no global / external variables used in the functions. If a piece of the software doesn't behave as expected, then we can isolate it and test it with different inputs, until eventually finding the unexpected behavior / output.

Adoption rising among libraries and frameworks

This is a trend I've mostly noticed in the web frontend world. But I think it's also spreading in the mobile and backend world, with languages (or language's features) such as Kotlin, Swift, and Rust lately.

React added hooks a few years ago, which allow building complex apps relying only on functional components.

The pipeline operator proposal has recently reached the stage 2, and there are more undergoing FP-related proposals, such as:

Immutable records and tuples (stage 2)
Pattern matching (stage 1)
Do expressions (stage 1)
Partial application (stage 1)
First-class protocols (stage 1)

In the State of JavaScript 2020, to the question "What do you feel is currently missing from JavaScript?", we can see some FP-related answers at the top:

Pattern Matching
Pipe Operator
functions (not sure what that means?)
Immutable Data Structure

All this tends to show that FP is becoming more and more ubiquitous in our lives, so we might as well learn it to understand the libraries, frameworks and language features that are already there, or coming in the near future.

Something new to learn

I don't know about you, but one of the most exciting things in being a developer is that there's always something new to learn.

I love learning, and if what I'm learning is interesting and can make me a better developer, then sign me in!

I think it's valuable to learn this paradigm, even if you don't intend to use it. It should make you more critical about your own code, and hopefully improve it to make it more readable, testable, extendable.

One does not have to embrace this paradigm at 100%, but there are some very powerful concepts and tools that are worth the time spent learning them.

It's hard to get on board

As I mentioned in the introduction, there's a big entry barrier when trying to learn Functional Programming. The vocabulary/jargon can be scary at first.

I'm writing this series to make it easier for people to discover this world.

Some languages are more adapted than others

I've had the opportunity to write FP code in TypeScript, Elm, and Scala for the past years. I've also read blog posts and articles that shared examples in F#, PureScript, and Haskell.

I must say that writing FP code feels more natural with languages that have some degree of built-in support for FP.

It is still possible to write FP code in languages that don't offer all these built-in concepts and tools, but it requires the developers to use some boilerplate to mimic the same tools. I'll give you 2 examples in TypeScript.

The first one is about algebraic data types (ADTs). We'll cover this topic in more details later in this series, but basically here's how we can define a sum type in TypeScript and in Haskell:

// data type
type Option<A> =
  | { readonly type: 'None' }
  | { readonly type: 'Some', readonly value: A }

// constructors
const none: Option<never> = { type: 'None' }

function some<A>(value: A): Option<A> {
  return { type: 'Some', value }
}

// matcher / fold
export function fold<A, R>(
  onNone: () => R,
  onSome: (value: A) => R,
  fa: Option<A>
): R => {
    switch (fa.type) {
      case 'None': return onNone()
      case 'Some': return onSome(fa.value)
    }
  }
}

data Option a = None | Some a

As you can see, there's much more boilerplate in the TypeScript version, since ADTs are not built-into the language, so we have to define the constructors and matcher functions.

Another example is the ability to implement data structures. Again, we'll see what these structures are much later in this series. TypeScript doesn't support higher-kinded types (HKTs), but some people have tried emulating HKTs with the current type-level features of the language.

Here's how we can implement the Functor data structure using fp-ts, one of the most popular TypeScript libraries to write FP code:

import { HKT, Kind, URIS } from 'fp-ts/lib/HKT'

export interface Functor<F> {
  readonly URI: F
  readonly map: <A, B>(f: (a: A) => B, fa: HKT<F, A>) => HKT<F, B>
}

export interface Functor1<F extends URIS> {
  readonly URI: F
  readonly map: <A, B>(f: (a: A) => B, fa: Kind<F, A>) => Kind<F, B>
}

And here's how we can do the same in Haskell, where data structures are built-into the language using type classes:

class Functor f where 
   fmap :: (a -> b) -> f a -> f b

(Actually we don't have to define Functor as it's already available in Haskell)

And finally, here's how we can create an instance of these data structures, for the type Option<A> we defined earlier:

const URI = 'Option'
type URI = typeof URI

const map = <A, B>(f: (a: A) => B, fa: Option<A>): Option<B> =>
  isNone(fa) ? none : some(f(fa.value))

const OptionFunctor: Functor1<URI> = { URI, map }

instance Functor Option where
  fmap _ None = None
  fmap f (Some a) = Some (f a)

You don't have to understand everything in these examples, as I'll explain these concepts later in the series. The point is that there is way less code with Haskell than with TypeScript.

Nonetheless, it's still possible to write FP code in both, it just feels more "natural" with Haskell because the language has better "native" support, it doesn't require any library or emulation to achieve the same results.

So, here we are! Thank you for reading this far. Hopefully the pros outperform the cons to you, and you are willing to give FP a try. As of the next articles, we'll dive into the FP concepts and tools I've been mentioning since the introduction.

Feel free to share your pros and cons in the comments!

Photo by Xavi Cabrera on Unsplash.

What is Functional Programming?

Benoit Ruiz — Fri, 10 Sep 2021 15:56:05 +0000

Functional Programming is a declarative programming paradigm where developers write software by:

Composing pure functions,
While avoiding mutable data and shared state,
And pushing side-effects to the edge of the program.

Don't worry about all these terms for now, I'll explain them later in this series, with dedicated articles.

Hmmm... Ok sure, but what is a "paradigm", and what does declarative mean?

A programming paradigm is a way to classify a programming language based on some features. Basically, a programming paradigm is a way to write programs.

Some languages use only 1 paradigm, while others can use multiple paradigms. For example, Haskell uses a single paradigm: functional programming. JavaScript uses multiple paradigms, as we can write programs using procedural programming, "object-oriented" programming (OOP for short), functional programming, or even reactive programming.

For single paradigm languages, since there's only 1 way of writing programs, it's easy to "stay on the track". For multi-paradigms languages, it's harder to stay on the same track, as the frontier between 2 paradigms depends solely on the developer writing the program.

One piece of the software can be written in FP, while the other can be written in OOP. Both pieces can coexist, as long as the code is adapted to jump from one way to the other. This flexibility can be both a good and a bad thing, but we won't cover this topic here.

I think it's important to say that FP is not a replacement for object-oriented programming, or any other paradigm whatsoever. It's just a different way of writing programs, which comes with some advantages we'll see in the next article.

Well that covers the paradigm part, what about the declarative part?

I won't go into many details here since I've planned on writing an article about declarative vs imperative code, later in this series.

In a nutshell, declarative programming is a style of building the structure and elements of a program by describing what the program does, and not how it does it (that's the imperative way).

A declarative program expresses the logic of a computation ("what") whereas an imperative program expresses the steps to perform a computation, explicitly ("how").

Don't worry if this is still blurry, we'll cover this part more thoroughly later.

Requirement

All languages are not born equal. There is actually one requirement for writing FP code: functions must be first-class citizens of the language.

First-class what now?

A first-class citizen is an entity of the language that supports the following operations:

Storing the entity in a variable
Passing the entity as an argument of a function
Returning the entity when calling a function

Let's take an example: are functions first-class citizens in JavaScript?

Can they* be stored in variables? Yes.
Can they be passed as arguments of other functions? Yes.
Can they be returned when calling other functions? Yes.

Therefore, functions are first-class citizens in JavaScript, meaning we can write FP programs using this language.

* Small detail: we are not talking about the functions directly here, but rather their references.

If the language you use every day fulfills this requirement, then you can definitely implement some - if not all - of the concepts and tools we'll talk about in this series.

If that's not the case (e.g. Java < 8), then you can still learn through this series, but you probably won't be able to play with these concepts in your favorite language. And there's nothing better than some practice to actually learn new things.

Last but not least, Functional Programming comes with some "restrictions" that are better enforced with a compiler. So my recommendation would be to use a programming language that has a compiler that offers static typing.

You can still write FP programs in JavaScript - an interpreted language - for example, but you won't benefit from the greatness provided by a typed language such as TypeScript.

Photo by Xavi Cabrera on Unsplash.

Introduction

Benoit Ruiz — Fri, 10 Sep 2021 15:55:54 +0000

Hello and welcome!

Today I'm introducing a new series about Functional Programming, or FP for short.

Functional Programming is an amazing world, but it can be quite challenging to walk on these lands and learn its concepts without feeling overwhelmed. At least that's how I felt when I discovered this paradigm.

When trying to learn FP, one of the first walls we bump into is the vocabulary. Functor, monad, algebraic data types, referential transparency, side-effects, higher-kinded types. All these buzzwords sounded mystical to me. It took some time (several years actually) to learn what they mean, how I could use them, and what were the relationships that tight them together.

Although there are quite a lot of articles about functors, monads, and the likes, I think there is still a lack of content regarding all the "surroundings". More specifically, there are root concepts that are essential and used everywhere when doing FP.

In this series, I will try my best at introducing and explaining the following:

What is FP, and why we should learn and use it
Function composition and higher-order functions
Declarative vs imperative
Side-effects
Function purity and referential transparency
Data immutability
Function currying, partial application and tacit programming
Parametric polymorphism (aka "generics")
Inversion of control
Algebraic Data Types (ADTs)
Kinds and higher-kinded types (HKTs)
A word on category and set theories
Algebraic structures and type classes
Bonus (possibly): map filter reduce, tagless final, optics, transducers

I will not talk about (or maybe briefly) functors, monads, applicatives, etc. as I believe there is already a lot of great material out there explaining these great tools.

Hopefully by the end of this series, you will have all the means necessary to walk on these lands peacefully.

Who am I?

Before diving into the subject, let me tell you about my experience with FP, and what you can expect from the quality and depth of this series.

My adventure began about 4 years ago, in 2017. I am still climbing the learning curve, I haven't mastered this paradigm. Though, I don't feel like I'm at the bottom anymore, which is nice.

I have been applying these concepts in TypeScript and Scala for the past years, as they are the main languages I have been using. I have seen the limitations of TypeScript compared to Scala or Haskell for example. Nonetheless, it is still possible to do FP in TypeScript, although it may feel less natural than with a language that has built-in FP concepts.

I have read a lot of articles and blog posts, and I still do. These are mainly the ones that helped me in this journey:

Composing Software by Eric Elliott
Mostly Adequate Guide to Functional Programming by Brian Lonsdorf, aka DrBoolean
John De Goes' blog
Fantas, Eel, and Specification by Tom Harding
James Sinclair's blog

I started reading Category Theory for Programmers (Scala Edition) by Bartosz Milewski a few years ago, although I never finished the book. I stopped at around 30%. I would not advise reading this book at the beginning of the journey. As the book name suggests, there is a lot of theory in these pages! That being said, once you have grasped the main concepts, it is definitely a good book to dive deeper into the essential "components" that make FP possible.

Since my learning revolves mainly around TypeScript/JavaScript and Scala, the examples I will share will be written in these languages. However, this should not be an issue as the majority of these concepts are language-agnostic and can be implemented, or applied, in pretty much any language.

So, as you see, I am not some FP guru. I am still not able to write Haskell code like it's child's play. Some of my explanations may be incomplete, or partially wrong. If that is the case then please leave a comment so I can learn and fix these mistakes.

I may lack some formality in my explanations, though I think it is better to present these concepts in a more informal way, if one wants to convince people to learn and try FP.

I hope this series will help you dip a toe in the water, and convince you that Functional Programming is not that mystical after all!

Photo by Xavi Cabrera on Unsplash.

My takes on Dan Abramov's "Goodbye, Clean Code"

Benoit Ruiz — Wed, 04 Aug 2021 16:28:20 +0000

Hello and welcome!

First of all, I have to say I highly respect Dan Abramov for his contributions to the JavaScript community. I think he is a strong influencer and thus, every time he speaks about something, it will affect a lot of people. When I recently came across his tweet referencing Goodbye, Clean Code, I couldn't help myself but react to it:

// Detect dark theme var iframe = document.getElementById('tweet-1216110520891334660-58'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1216110520891334660&theme=dark" }

I believe that having multiple opinions/points of view on a specific subject is very important to understand it as a whole, so... Here are my takes on this matter!

TL;DR the real problem is not removing code duplication, it's changing the code of a colleague unbeknownst to them, and not questioning the abstractions chosen to solve the previous problems.

The definition of clean code really depends from one developer to the other. In Dan's article, it looks like clean code refers to applying the DRY principle: Don't Repeat Yourself. In my opinion, there are way more elements that compose the definition of "clean code", but let's focus solely on avoiding code duplication, as it appears to be the main subject here (this, and another issue regarding teamwork, but we'll talk about it as well).

The context

The story starts when a coworker implements a solution to a problem, but it involves a lot of repetitive lines of code. This code duplication bothers the author, so he refactores the code to avoid duplication and make it DRY. The new code is half the size of the previous one and has no repetitions.

In my opinion, the result is better: the logic (i.e. math formulas) can easily be unit-tested, and only once. In addition, the behaviors can be composed to generate new behaviors that fix the initial problem. There are also fewer lines of code, which means fewer places where nasty little bugs can hide.

Once the author was pretty happy with the refactoring, he committed the changes on the main branch, then went to bed. The next morning, he was told by his boss to revert the changes.

The phase

Now comes the second part of the article where I (partially) disagree with Dan.

It looks like he claims that writing clean code is "a phase" that has an end, which I disagree with. The definition of code "cleanliness" may differ through time, but it's still a good practice to adopt, no matter the years of experience one has. I don't believe that it has an end.

To me, the "phase" (if any) is the one that goes from "discovering the wonders of abstraction" as a junior developer, to "applying abstraction at every single line of code" as a more experienced developer. Although this is the red zone, it doesn't have to end there. Maybe that's what Dan meant in the first paragraphs of the "It’s a phase" section, but the end of the article "Then let it go" suggests that developers should let clean code go enterily at the end of this phase, and that's where I disagree.

It's important to go through this phase to learn about abstractions and clean code, and it's even more important to know the right threshold of abstraction once this phase is over. First gather experience by over-abstracting things and seeing the consequences, then use this valuable experience to use the right abstraction, to write clean code.

There is a reference to The Wrong Abstraction article in there. I highly recommend you to read it if you haven't done it already. Essentially, it tells us that removing code duplication, by using abstractions, eventually leads to more complications in the future (e.g. when there are new requirements), because we don't dare remove the existing code. However, there is a very important part in this article that is often overlooked in my opinion:

Once you completely remove the old abstraction you can start anew, re-isolating duplication and re-extracting abstractions.

I feel like a lot of people that reference this article (for the sake of saying "hey duplication is actually good" maybe?) seem to forget this crucial part. It doesn't tell us that duplication is actually good. It tells us that it's fine to break/remove the abstraction and go back to duplicating the code temporarily when implementing new requirements. However, once the new requirements are handled, we have to think about a new abstraction (given the new state of the code) that will feel appropriate, and remove the code duplication.

The reasons why the refactoring was unsuccessful

The article continues with the enumeration of 2 reasons explaining why the refactoring was not appropriate.

Reason 1

The first reason is because he didn’t talk to the person that wrote the initial implementation. I completely agree on this one.

I don't believe in sabotage. When I write code, I believe that the solution I find is a good fit, given the context I am at that moment: my experience as a developer, my understanding of the code base and the requirements, the time granted to solve the problem. It might actually not be as "clean" as I think it is, because of multiple reasons such as code convention and best practices of the project, team knowledge about some paradigm or library I'm using, or lack of tests to name a few. But in my mind, I'm convinced I'm writing the best code I can think of, given the context.

I would feel bad if I spent time writing code to solve a problem, went to bed, then woke up seeing that it got almost enterily rewritten by someone else. I would think "Well, I guess it wasn't that good then... But why? Why did this other person think it was ugly? I spent some time writing this, and it was actually useless". It would destroy my motivation, and my self-esteem. I would be "scared" to write anything, or spend too much time on it, since it could be completely rewritten over night.

This is not a critic about the author, since he acknowledges that he shouldn't have done it as the first reason. This is an attempt to make people realize that this type of behavior can have deep consequences on the mental health and productivity of their coworkers. Please, if you work in a team, do not do this.

What can you do then? If the code written by someone else is bothering you for some reason, e.g. you don't think it's "clean", then I encourage you to do the following:

Talk to your team by explaining what clean code is to you.
If everyone agrees (or at least a vast majority), then write some documentation about code conventions and best practices.
You will probably compromise, as some people will think that "this part of the definition is too extreme, too demanding". It's up to you to decide what to do, but my recommendation is that the team is greater than player: if the team thinks that your definition of clean code is too restrictive, then you should find a suitable tradeoff.
Clean code won't be written magically by everyone as of the next day. A period of adaptation will take place. It could last weeks, or months. It's up to you to educate your teammates by leaving appropriate comments during code reviews, with examples and references to the documentation aforementioned.

Collaboration is key in a team working on the same project. Talk to them, teach them what clean code is, and why it should be written that way. Chances are you will learn things by listening to their points of view, and you will adapt your definition of clean code, for the better.

Reason 2

The second reason mentioned is one that bothers me. The author states that his refactoring would have made the new requirements difficult to implement, so in the end the original one with a bunch of duplication made it pretty easy to solve the new problems. The thing is, code changes over time, no matter the code. It changes because users have different needs and requirements, bugs are found, performance needs to be improved, or technologies must be updated/upgraded.

Code, whether it's duplicated or abstracted, will change.

Saying that an abstraction makes new requirements difficult to implement implies that the code written in the past cannot be questioned, challenged, changed, or adapted. Again, let me quote this part from the "The Wrong Abstraction" article:

Once you completely remove the old abstraction you can start anew, re-isolating duplication and re-extracting abstractions.

This means that the abstraction implemented with the refactoring to remove the code duplication could have been replaced with a better one, taking into account the new requirements. One does not have to choose between "code duplication + easy future implementations" and "abstraction + difficult future implementations". We can have "abstraction + easy future implementations", but for that, we need to be comfortable in undoing previous abstractions and adding temporary code duplication to come up with a new, better abstraction.

Should you write "dirty" code? As mentioned in the article, no.

Should you find your own definition of "clean" code? Yes, definitely.

Can this definition change depending on the people/project you are working with/on? Yes, definitely.

Should you be a clean code zealot? As mentioned as well, no. Nothing good ever happens in the extremes of any scale.

Should you let clean code go? No, never.

Maybe I am still in "The Phase" and I am simply not aware of it (yet?). I have been developing for 7 years professionally. I have known some moments of "clean code zealot", but I didn't stay there. These few moments made me a better developer and team player, because they made me realize that I was not alone working on these projects, and the team is greater than the player. I will probably encounter more of these moments in the future, but that's fine because I will grow a better developer. I will not let clean code go, because I'm convinced of its advantages.

A few months ago, I wrote about a way to solve problems while limiting the technical debt, which is introduced in a project because of a lack of clean code, in my opinion. I talk about clean code and how it can be used in a team there, feel free to have a look: A Method To Deliver Updates Quickly Without Compromising the Future.

Thank you for reading this far! I hope you enjoyed the ride. Feel free to share your opinion! :)

Cover photo by Kalle Schmitz on Unsplash.

Using fp-ts and io-ts: types and implementation

Benoit Ruiz — Wed, 24 Mar 2021 12:00:33 +0000

The idea behind io-ts is to create a value with the type Type<A, Output, Input> (also called a "codec") that is the runtime representation of the static type A.

In other words, this codec allows to:

Parse/Deserialize an Input and validate that it's an A (e.g. parse an unknown and validate that it's a NonEmptyString50). This part is handled by the Decoder side of the codec.
Serialize an A into an Output (e.g. serialize an UnverifiedUser into a string). This part is handled by the Encoder side of the codec.

We are going to use only the first part, i.e. the Decoder, since we want to take values coming from outside our domain, validate them, then use them inside our business logic.

In this article, I am not going to use the experimental features. I'll use what is available with the following import as of v2.2.16:

import * as t from 'io-ts'

When decoding an input, the codec returns an Either<ValidationError[], A>, which looks very similar to the Validation<A> type we wrote in the previous article of this series. Actually, the library exposes a Validation<A> type that is an alias to Either<ValidationError[], A>.

Previously, we defined the types then we wrote the implementation. Here, we are going to do the opposite: write the implementation, then derive the types from it using the TypeOf mapped type provided by io-ts.

First and last names

The equivalent of a "newtype" created with newtype-ts is a "branded type" in io-ts. We can use the t.brand function to create a codec for a branded type:

interface NonEmptyString50Brand {
  readonly NonEmptyString50: unique symbol
}

const NonEmptyString50 = t.brand(
  t.string,
  (s: string): s is t.Branded<string, NonEmptyString50Brand> => s.length > 0 && s.length <= 50,
  'NonEmptyString50'
)

type NonEmptyString50 = t.TypeOf<typeof NonEmptyString50>

First we create the NonEmptyString50Brand brand. Next, we create the codec by providing 3 parameters:

The codec for the "underlying" type of the branded type (here, string)
The type guard function, or "refinement" function
The name of the codec (optional)

Let's look at the default error message reported for this codec when an invalid input is provided:

import { PathReporter } from 'io-ts/PathReporter'

PathReporter.report(NonEmptyString50.decode(42))
// ['Invalid value 42 supplied to : NonEmptyString50']

If we keep the same logic regarding the errors handling as we did in the previous article, then this message is not particularly "user-friendly". We want a better description of the expected value (string whose size is between 1 and 50 chars). For that, we can use a little helper function provided by io-ts-types:

import { withMessage } from 'io-ts-types'

const FirstName = withMessage(
  NonEmptyString50,
  input => `First name value must be a string (size between 1 and 50 chars), got: ${input}`
)

const LastName = withMessage(
  NonEmptyString50,
  input => `Last name value must be a string (size between 1 and 50 chars), got: ${input}`
)

Let's look at the error message reported:

import { PathReporter } from 'io-ts/PathReporter'

PathReporter.report(FirstName.decode(42))
// ['First name value must be a string (size between 1 and 50 chars), got: 42']

We end up with the same error message we had in the previous article, with very little effort thanks to withMessage!

Email address

Nothing fancy here:

interface EmailAddressBrand {
  readonly EmailAddress: unique symbol
}

// https://stackoverflow.com/a/201378/5202773
const emailPattern = /(?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9]))\.){3}(?:(2(5[0-5]|[0-4][0-9])|1[0-9][0-9]|[1-9]?[0-9])|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])/i
const EmailAddress = withMessage(
  t.brand(
    t.string,
    (s: string): s is t.Branded<string, EmailAddressBrand> => emailPattern.test(s),
    'EmailAddress'
  ),
  input => `Email address value must be a valid email address, got: ${input}`
)

type EmailAddress = t.TypeOf<typeof EmailAddress>

Middle name initial

We need to create a Char codec:

interface CharBrand {
  readonly Char: unique symbol
}

const Char = t.brand(
  t.string,
  (s: string): s is t.Branded<string, CharBrand> => s.length === 1,
  'Char'
)

type Char = t.TypeOf<typeof Char>

Then create a MiddleNameInitial codec from it:

import { optionFromNullable } from 'io-ts-types'

const MiddleNameInitial = withMessage(
  optionFromNullable(Char),
  input => `Middle name initial value must be a single character, got: ${input}`
)

This is the same codec as Char, but we made it optional with the optionFromNullable helper, and we set a custom error message.

Remaining readings

The io-ts library provides a codec for integers, but not for positive integers like we had in newtype-ts. We need to create this type:

interface PositiveIntBrand {
  readonly PositiveInt: unique symbol
}

const PositiveInt = t.brand(
  t.Int,
  (n: t.Int): n is t.Branded<t.Int, PositiveIntBrand> => n >= 0,
  'PositiveInt'
)

type PositiveInt = t.TypeOf<typeof PositiveInt>

As you noticed, we can create branded types from other branded types: t.Branded<t.Int, PositiveIntBrand>.

Let's define a RemainingReadings codec, which is a PositiveInt codec with a custom error message:

const RemainingReadings = withMessage(
  PositiveInt,
  input => `Remaining readings value must be a positive integer, got: ${input}`
)

Verified date

Last but not least, we need a Timestamp codec for the verified date:

interface TimestampBrand {
  readonly Timestamp: unique symbol
}

const Timestamp = t.brand(
  t.Int,
  (t: t.Int): t is t.Branded<t.Int, TimestampBrand> => t >= -8640000000000000 && t <= 8640000000000000,
  'Timestamp'
)

type Timestamp = t.TypeOf<typeof Timestamp>

The VerifiedDate codec is a Timestamp with a custom error message:

const VerifiedDate = withMessage(
  Timestamp,
  input =>
    `Timestamp value must be a valid timestamp (integer between -8640000000000000 and 8640000000000000), got: ${input}`
)

User types

If you remember from the previous article, we wrote 2 intermediate types before getting a User: UserLike and UserLikePartiallyValid.

To create UserLike, we can do the following:

const UserLike = t.intersection([
  t.type({
    firstName: t.unknown,
    lastName: t.unknown,
    emailAddress: t.unknown
  }),
  t.partial({
    middleNameInitial: t.unknown,
    verifiedDate: t.unknown,
    remainingReadings: t.unknown
  })
])

type UserLike = t.TypeOf<typeof UserLike>

The only way to make some properties of an object optional is to make the intersection between an object with required properties (type) and an object with all the optional properties (partial).

Next, we can use some codecs previously defined to build the UserLikePartiallyValid codec:

const UserLikePartiallyValid = t.strict({
  firstName: FirstName,
  lastName: LastName,
  emailAddress: EmailAddress,
  middleNameInitial: MiddleNameInitial
})

type UserLikePartiallyValid = t.TypeOf<typeof UserLikePartiallyValid>

I used strict here (as opposed to type) to make sure any extra property of the input is discarded from a UserLikePartiallyValid data object.

Now we can write both UnverifiedUser and VerifiedUser codecs.

const UntaggedUnverifiedUser = t.intersection(
  [
    UserLikePartiallyValid,
    t.strict({
      remainingReadings: RemainingReadings
    })
  ],
  'UntaggedUnverifiedUser'
)

type UntaggedUnverifiedUser = t.TypeOf<typeof UntaggedUnverifiedUser>

type UnverifiedUser = UntaggedUnverifiedUser & { readonly type: 'UnverifiedUser' }

We first build an UntaggedUnverifiedUser because we don't want to include the validation of the type property that is used only to create the User sum type in TypeScript. Then, we create the UnverifiedUser type by adding the type property.

Notice that it's only a type definition, there's no codec associated because there's no need to validate external data: we (the developers) are the ones adding the type property via the constructor functions (defined a bit later).

We can do the same for the UntaggedVerifiedUser codec:

const UntaggedVerifiedUser = t.intersection(
  [
    UserLikePartiallyValid,
    t.strict({
      verifiedDate: VerifiedDate
    })
  ],
  'UntaggedVerifiedUser'
)

type UntaggedVerifiedUser = t.TypeOf<typeof UntaggedVerifiedUser>

type VerifiedUser = UntaggedVerifiedUser & { readonly type: 'VerifiedUser' }

Now that we have both UnverifiedUser and VerifiedUser types, we can create the User type simply with:

type User = UnverifiedUser | VerifiedUser

And the constructor functions:

const unverifiedUser = (fields: UntaggedUnverifiedUser): User => ({ ...fields, type: 'UnverifiedUser' })

const verifiedUser = (fields: UntaggedVerifiedUser): User => ({ ...fields, type: 'VerifiedUser' })

There's one last function we need before (finally) writing the parseUser function. We need to detect if a user-like object looks like a verified user or not. In the previous article, we wrote the detectUserVerification function. Here, we are going to write a similar function, but instead of taking a UserLikePartiallyValid input, it will take a UserLike input:

const detectUserType = <A>({
  onUnverified,
  onVerified
}: {
  onUnverified: (userLikeObject: UserLike) => A
  onVerified: (userLikeObject: UserLike & { verifiedDate: unknown }) => A
}) => ({ verifiedDate, ...rest }: UserLike): A =>
  pipe(
    O.fromNullable(verifiedDate),
    O.fold(
      () => onUnverified(rest),
      verifiedDate => onVerified({ ...rest, verifiedDate })
    )
  )

This is because we are going to use the decoders of either UntaggedUnverifiedUser or UntaggedVerifiedUser codecs that already contain the validation steps for a UserLikePartiallyValid object:

const parseUser: (input: unknown) => t.Validation<User> = flow(
  UserLike.decode,
  E.chain(
    detectUserType({
      onUnverified: flow(UntaggedUnverifiedUser.decode, E.map(unverifiedUser)),
      onVerified:   flow(UntaggedVerifiedUser.decode,   E.map(verifiedUser))
    })
  )
)

And that's it! The logic for parseUser is slightly different compared to the one we wrote in the previous article, but overall it looks very similar. And, we wrote fewer lines of code for the same result, which is nice (fewer lines = less chances for a bug to be introduced).

The source code is available on the ruizb/domain-modeling-ts GitHub repository.

The io-ts library allows us to create codecs that we can combine to build even more complex codecs.

One key difference with the previous method is that the type definition for User is not clearly readable for the developers without relying on IntelliSense, and even then, it doesn't show the whole type definition:

// examples of types displayed with IntelliSense

type UserLikePartiallyValid = t.TypeOf<typeof UserLikePartiallyValid>
/*
type UserLikePartiallyValid = {
  firstName: t.Branded<string, NonEmptyString50Brand>;
  lastName: t.Branded<string, NonEmptyString50Brand>;
  emailAddress: t.Branded<...>;
  middleNameInitial: O.Option<...>;
}
*/

type UnverifiedUser = UntaggedUnverifiedUser & { readonly type: 'UnverifiedUser' }
/*
type UnverifiedUser = {
  firstName: t.Branded<string, NonEmptyString50Brand>;
  lastName: t.Branded<string, NonEmptyString50Brand>;
  emailAddress: t.Branded<...>;
  middleNameInitial: O.Option<...>;
} & {
  ...;
} & {
  ...;
}
*/

There is an open issue to address this problem in the VS Code editor.

To solve this, we could've first defined the type like we did in the 3rd article of this series, then use io-ts for the implementation part only, and not use TypeOf to define the types of users.

Final thoughts

We used these codecs to validate data coming from the external world to use them in our domain. We can safely use these data in the functions holding the business logic, at the core of the project. We didn't write these functions in this series though, I chose to focus on the "let valid data enter our domain" part.

If you are familiar with the "onion architecture" (or "ports and adapters architecture") then these codecs take place in the circle wrapping the most-inner one that has the business logic.

This approach allows us to document the code easily by describing and enforcing domain constraints and logic at the type level.

I hope I convinced you to try Domain Driven Design in TypeScript using the fp-ts ecosystem!

DEV Community: Benoit Ruiz

Advices from a Software Engineer with 8 Years of Experience

Table of contents

My career evolution

Things I wish I had started doing earlier

Write a work log

Leave the comfort zone

Be curious about other teams and projects

Join the oncall team

Change teams

Write blog posts

Things I wish I had done differently

Be careful when introducing new things to the team

Do not let your emotions take over in front of the team

Dip a foot into the hiring market

End thoughts

Data immutability

Table of contents

Introduction

Characteristics of data immutability

Code is more predictable

Thread-safety

Time Travel Debugging

More memory allocation

May be cumbersome to update deeply-nested values

Immutability syntax may bloat the code

Summary

Function purity and referential transparency

Table of contents

Function purity

Referential transparency

How to deal with impure functions?

Practical example

Testing the function

Splitting the function

Testing the new version

Equivalent of Scala's for-comprehension using fp-ts

Table of contents

Introduction

Case 1

Case 2

Prerequisites

Case 1

Case 2

Using fp-ts

Case 1

Case 2

Using fp-ts-contrib

Case 1

Case 2

Summary

Using fp-ts

Case 1

Case 2

Takeaways

Using fp-ts-contrib

Case 1

Case 2

Takeaways

Side effects

Table of contents

Introduction

How to deal with side effects

Chaining side effects

Declarative vs imperative

Table of contents

Introduction

Making a chocolate cake

The imperative way

The declarative way

Analysis

Some examples

When to use declarative code

Declarative

Imperative

Conclusion

Function composition and higher-order function

Table of contents

Function composition

What is it?