DEV Community: Daniel Bartholomae

Easily identify technical debt in any repository

Daniel Bartholomae — Tue, 07 Mar 2023 15:23:11 +0000

Technical debt is a term used a lot whenever a codebase is not to our liking, and there's never enough time to fix all of it. What actually is technical debt, how do you quickly prioritize what to work on - and what are these GitHub Blocks that people are talking about?

If you are interested in more articles and news about web product development and entrepreneurship, please feel free
to subscribe to new articles via email, via RSS, or to follow me on Twitter.

What is technical debt?

"Technical debt" is often used as a term for basically any kind of code we don't like, be it difficult to understand or just written by someone who since left the company.
And often, it is considered as something to avoid at all cost.

But that's not what the term was meant to be originally, and neither how it is most useful. So let's start with a definition from Wikipedia:

Technical debt is the implied cost of future reworking required when choosing an easy but limited solution instead of a better approach that could take more time.

So unless there was an explicit choice to take an easier route, it's not technical debt, it is accidental complexity.
Technical debt, on the other side, is a tool that can be used to move work into the future: Instead of taking the time-consuming path now, we take an easy route, knowing full well that it might lead to more work in the future.

A typical example is around scalability: Especially at a startup, we often build features to attract users, and while we hope for many of them, we only have maybe a handful in the beginning.

This also means that, most of the time, technical debt is a good thing: We don't know whether or how a feature will be used, before it actually is used, so if we can push work into the future, we might not even need to do it at all.
And we can still fix the feature to be more scalable once this is actually needed (or, ideally, once we know we will need it to be scalable, but right before it actually is needed).

Technical debt can run us into trouble in two ways, though:

If we are in an organization that is willing to take the trade-off, but then can't manage the cost of cleaning up when needed.
If the cost of fixing the problem later massively out-scales the cost of fixing it from the get-go.

The first one can be reduced by having explicit timelines, e.g. building a feature and then, 3 months later, either removing it again if it isn't used, or cleaning it up otherwise.

The second one can be avoided by writing down the expected costs explicitly and think them through to make an explicit trade-off. It's less common, though, and I've seen way more situations where it was used as an argument against technical debt even though technical debt would have been the better choice.

So what about accidental complexity?

In reality, the term "technical debt" is often used for accidental complexity. So let's define that as well (this time my one as there is no Wikipedia article):

accidental complexity is any complexity in the code that is not needed to model actual complexity in the business domain.

In practice, accidental complexity mainly comes from the following:

Not understanding the domain well enough, e.g. adding features and special cases that are never or almost never used by our users
Not understanding the technology we use well enough, e.g. writing complex custom solutions for a problem that could be solved more easily with features already in the framework we use
Trying to appear smart by writing complex code, e.g. to make oneself less replaceable

All of that have an implicit continuation of "and not fixing the code once we learn more (about the domain/the technology/...)".

So how to fix technical debt?

Once it is in place, technical debt is indistinguishable from accidental complexity, as the terms differentiate between how both get created, not what they actually are.
This means that the following is true both for fixing technical debt and reducing accidental complexity.

Fixing the complexity usually is obvious: The reason why we identify something as technical debt usually is that we have an idea of a "better" way to do it.
Instead, the hard part is to know where to start, as reducing complexity will take a lot of time, and we usually don't have enough time to reduce all of it at once.

So what should we focus on first? We need to find out

which files have the most accidental complexity, and
which ones are most likely to need to be changed in the near future.

For complexity, there are many metrics, e.g. cyclomatic complexity, Halstead, or simply counting lines of code.
Fortunately, they are all strongly correlated, so for practical purposes, looking at
lines of code is often enough. In practice, these are also closely correlated with file size (which is roughly proportional to number of lines times average line length).

Figuring out which files are most likely to be changed is more of a communication exercise: We can talk to management, product managers, and
other team members to figure out which services are most likely needed for upcoming projects. Or we can just look at the service that we need to work on next.

How to identify the most critical files inside a service? Thanks to git, we have a lot of data at our fingertips.
The easiest way to predict which files will need to be changed, is to look at the files that were changed most recently.
We can do this by counting how many commits touched a file.

Also, not all files are equal. E.g. a lock-file might be big and changing often, but since it is autogenerated,
there is not much complexity.

Fortunately, we don't need to do all of this ourselves. There's a GitHub block for this.

Interlude: What are GitHub blocks?

GitHub Blocks is an update to the way we can interact with repositories.
It allows to write custom "blocks" which get data about either a file or a folder and can display this information,
enriched with other information e.g. from the GitHub API, any way we want.
As of Q1 2023, it is in technical preview.

This is potentially a game changer for GitHub, as it allows them to basically outsource the creation of a great UI.
For developers, it opens up a full new world of interacting with their code. I'll write a short blog post about this soon,
so if you want to know more, feel free to subscribe via email, via RSS,
or to follow me on Twitter.

The technical debt block

The first thing I thought about when I read about GitHub blocks was a way to identify technical debt in a folder:
The Technical Debt Block was born.

If you have access to GitHub Blocks, you can use it to view a folder and get a list of files in that folder,
sorted by technical debt as described above. Only thing that remains to do is to actually clean the top files.

Conclusion

We talked about technical debt, which is born out of a conscious trade-off decision and usually is a good thing,
as well as accidental complexity, which usually isn't. We looked at how to best prioritize which to clean with help of
information readily available in Git, and a GitHub Block that helps to visualize this.

Hit me up on Twitter:
How do you prioritize technical debt? And which files did you find with this method that surprised you the most?

Rewriting my blog from scratch with NextJS

Daniel Bartholomae — Fri, 30 Dec 2022 15:14:03 +0000

"Writing a blog" is usually taken as "writing articles for a blog".
I did this for a while, using Ghost to host the blog.
And it mostly worked, but there were multiple reasons for me to change it:

Have an actually running open-source application, both for experimenting, and for my CV
Try out NextJS, rush, and other technologies
Be able to automate whatever I want around the blog

What I thought would be a project of maybe a month, turned out to take me almost a full year.
Here's what I did and what I learned on the way.

If you are interested in more articles and news about web product development and entrepreneurship, please feel free
to subscribe to new articles via email, via RSS, or to follow me on Twitter.

Overview

When building the blog to the current state, I ran through the following phases:

Setting up the blog in NextJS
Adding an RSS feed
Hosting on Amplify
Starting with analytics
Moving to a monorepo
Using OIDC to work with AWS
Publishing to dev.to
The current structure
What's next

If you want to follow along in code, this blog is open-source now, so you can roam through
its commit history
and follow along. I will mention relevant commits in each chapter.

Setting up the blog in NextJS

Commits start here.

When I decided to rewrite my blog, one of the reasons was that I wanted to learn more about NextJS.
It had already gotten some traction then, and seemed like a good fit for a blog. It was a bit overkill compared
to other solutions like Gatsby, but I liked the idea of learning something
that would allow me to quickly create even more complicated prototypes thanks to API routes.

I decided quite quickly to store blog posts in markdown-format with yaml front matter for meta-data.
On the one hand, this made migrating from Ghost easy, as my blog posts where already stored there as markdown files.
On the other hand, this also allowed me to store all relevant information in one file.

What didn't become obvious immediately was that storing the slug inside the file instead of using the filename was beneficial.
This creates an additional burden, as I need to make sure by hand that I don't use the same slug twice, but it allows me to potentially change the file name I use without losing SEO juice for the existing slug.

The folder structure inside NestJS also evolved over time, mainly due to more extensive use of Storybook for visual tdd.

content contains the markdown content for the blog articles
public contains additional assets like images
src/pages has the NextJS pages, but tries to be as lean as possible
It uses functions from src/loading-posts for server-side rendering, and
src/presentation/templates for the actual pages
src/presentation in general contains the full component library

Adding an RSS feed

Commits start with chore: add empty script file to create RSS feed

I'm a big fan of RSS feeds and use them to stay up-to-date on many blogs and sites, so offering an RSS feed myself was a hard requirement for the migration.
Fortunately, Ghost has an open-source, MIT-licensed implementation of creating an RSS xml file that I could look at to ensure that my implementation does not break anything for people who might already be following via RSS.
The actual code turned out to be quite simple, but it took me some time to understand, which elements where needed and why.
This way, I also learned about the <link rel="alternate" type="application/rss+xml" title="The Startup CTO" href="https://startup-cto.net/rss.xml"> tag.

Hosting on Amplify

Commits start with chore: add amplify config to repo

When I first started, I tried hosting the blog on GitHub Pages.
Unfortunately, I didn't find a way to make this work with a custom domain that was not hosted on a subdomain, as GitHub Pages requires the use of a CNAME record, and these can't be set on a root domain like startup-cto.net.
I could have decided for a subdomain like www.startup-cto.net, but this would have required an additional redirect or rewrite solution, or put me in danger of losing all my existing SEO juice for startup-cto.net.
I had anyway thought about trying out AWS Amplify at that point, so that's what I did.

The hosting itself went surprisingly smoothly.
I needed to tweak the amplify.yml config file a bit, especially once I moved my repository to be a monorepo, but apart from needing to build twice, it just worked.
I decided against running tests in Amplify, and instead to run the tests in GitHub, to separate CI and CD a bit and keep my development workflow in one place - hoping that I won't need to debug why a run fails on Amplify even though it succeeded on GitHub any time soon.

Starting with analytics

Commits start with chore: add first experimental CDK web analytics setup

This was definitively one of the most interesting parts of my journey.
I'm a big fan of building for the user, and for blog posts, the best way to understand the user - which is you, the reader - is to use analytics.
I did not want to collect any unnecessary data, though.
Most importantly, I did not want to help others on collecting unnecessary data.

At this point, I looked into some existing analytics solutions like Google Analytics and Plausible, but they either collected unnecessary information, or were costing money.
I could have self-hosted some of them, but given the little traffic I had so far, all of these options would have been significantly more expensive than what I went with now:
Writing my own little serverless analytics tool.
The main reason for this was to learn more about DynamoDB, though.

The architecture behind my little analytics tool is quite simple:

An API gateway offers access to two endpoints:
- One to collect an analytics event, consisting of a path and potentially utm parameters
- Another one to retrieve the events from the last month or so
A DynamoDB to save the events
Some frontend code to collect analytics events
A very simple page that shows me ugly graphs with the data

All the infrastructure is defined via AWS CDK, a library on top of CloudFormation.
The code is now in its own project.

Most of the code for the backend lives in its own CDK construct.
The first interesting realization was that I can easily link handlers to infrastructure code via environmental variables, e.g. the table name, but also key names for DynamoDB.

I then realized that this is even truer for validation logic.
This way, a single file can define validation logic that is used in the ApiGateway validator as well as in the handler itself.
It's a bit unnecessary to duplicate it in this case, but the performance hit is small enough for me to keep it in just to show the principle.

Speaking of performance:
I also validate the data I load from the database.
I'm not sure whether I will keep this, as it does create a performance hit.
The performance hit was significantly greater, though, when I accidentally recompiled the validation schema on each validation.

One last interesting aspect here is the DynamoDB setup with hash and sort keys.
A NoSQL database, to be honest, isn't the best choice for a database structure for an analytics solution like this one.
Yes, analytics databases get more write than read requests, and NoSQL databases can be better suited to writing a lot of document-like data than SQL databases.
But, honestly, my blog getting enough hits for this to matter is far from likely.
On the other hand, querying the data based on different combinations of paths and utm parameters is not a strength of NoSQL databases.

Other systems either use SQL databases for this purpose, or multi-step pipelines that use different tools for storing and querying the data.
I didn't want to use either solution, on the one hand, because I didn't find free solutions for low traffic, and on the other the hand because I anyway wanted to experiment a bit with DynamoDB.

Specifically, DynamoDB can only be queried by fixed hash keys and parts of sort keys.
They need to be designed with the intended queries in mind.
In our case, we intend to query by date and maybe by path or utm parameters.
The number of different paths will be growing slowly (very slowly, at the rate I currently write blog posts) while dates will be seasonal but roughly equally distributed, and utm parameters will have a few very frequent and a lot very infrequent entries.
Therefore, the paths could be a good candidate for hash keys.

On the other hand, there are limits for write-throughput in the same hash key.
To solve this, I added a scatter value as the hash key and randomly distribute all analytics events across the hash key.
This way, I can scale up when needed, but don't need to worry about the complexity yet.

I used UTC date strings as the sort key, mainly to be able to easily query all events for a certain month by just checking for keys that start with e.g. 2022-12.
A more common way would be to save the date time as an epoch and check for numbers between the epochs of the beginning and the end of the month, but I hadn't thought about that when I wrote the code, and it hasn't yet been important enough to refactor.

Moving to a monorepo

Commits start with Move package into sub-directory

With the analytics project in place, code became a bit more complicated and intertwined.
To solve this — and to learn more about monorepos in practice — I decided to split the projects into multiple directories and manage them with Rush.

The most interesting challenge was how to manage the CI pipeline.
I decided to have one common pipeline for all packages, and call individual scripts per package.
This way, each package can still manage the details of its pipeline, but the central pipeline manages dependencies and when to run which task of which project.

Using OIDC to work with AWS

Commits start with Add package to manage access

Another small change was to use OIDC to authorize GitHub towards AWS.
Thanks to the monorepo setup, the code is in its own access package.
While OIDC was quite straightforward, I did learn a few more things about AWS CDK.
Most importantly, that infrastructure should always live in a Stage, so that the construct ids do not change when the infrastructure is deployed via a pipeline.

Publishing to dev.to

Commits start with refactor: add DraftPost

The last addition to the blog (for now) is the concept of publishing posts.
Up to this point, the deployment pipeline worked like a side-effect-free pure function:
Take the current state of the repo from GitHub, find all articles that are in a published state, and convert them to HTML files.

But a blog also has side effects that should happen only once when a post is published.
So that's what I implemented next.

The structure ended up quite simple.
I cleaned up the different types of posts and introduced a ToPublishPost, which is a post that has all the metadata needed to be published, but no publishedAt yet.
Then a script that finds all these toPublishPosts, adds the current date as a publishedAt, commits it to repo, and runs the side effects.
For now, there are only two side effects:

Publishing the post to Dev.to
Send a newsletter email via Buttondown

The current structure

Since the blog is open-source now you can look at its current structure on GitHub.
It still follows the structure I introduced when moving to a monorepo:

.github for CI config,
access to manage access credentials to AWS,
analytics with all the code for my custom analytics solution,
blog with the actual blog, and
util with helper packages used in multiple projects.

There are more and smaller changes that I didn't mention in the rest of the article, but if you read up to this point,
you should have a good grasp of the overall situation.

What's next

There are obviously a lot more ideas that I want to do with the
blog. The blog isn't only a way for me to manage and share my own knowledge, but now it is also a sandbox
for trying out different web technologies in a production app that won't cause me much pain if it is down due to my mistakes.

For now, I will most likely focus on writing articles again, but there are some changes I might work on
from time to time, like automatically publishing new articles on Twitter. I'm sure what I learn there
will lead to new articles on its own.

If you want to see what else will happen, you can subscribe to new articles
via email, via RSS, or to follow me on Twitter.

How to read any startup's source code

Daniel Bartholomae — Sat, 13 Feb 2021 11:00:22 +0000

So you are in interviews as an engineer with an SaaS-startup and want to impress them with your deep knowledge of their tech stack? Or you have been working on learning projects but want to see some real-world code? Here are some tricks I have used in the past to know more about a startup's technology than their recruiter.

You could also use these tricks for market research and spying on your competition, just keep in mind that in early stage it is way more important to understand your customer than your competitors. And that some things might be plainly illegal.

I also want to point out that I personally wouldn't consider these loopholes a security problem by itself, but they can be if you are not careful, so you might want to avoid some of them in your own company. And yes, there are ways to avoid them, so, contrary to the click-baity title, you might run into situations, where they don't work.

If you are interested in more articles and news about web product development and entrepreneurship, please feel free to follow me on Twitter.

To understand a startup's tech stack, I

find all (sub-)domains for all the applications they use,
analyze them for tell-tale signs of their tech stack, and
skim their source code.

Finding their apps

Usually, there will be at least some of the following:

A landing-page builder for landing pages
A content management system for a blog
An authentication system
A web app for the actual product
A backend for the actual product
One or multiple native apps

A good starting point is the homepage itself. This often is created in a landing-page builder or a similar tool like WordPress, or with a static page builder.

Sometimes, blog or landing pages are hosted separately. The blog is usually linked from the homepage. A typical subdomain is blog.

Finding the landing-page builder can be slightly more tricky. Try to find ads, e. g. by using a traffic analysis tool like SimilarWeb, or a specific ads tool like the Google Ads keyword plnaner if you suspect them to use these kinds of ads. Then search for the keywords from these tools. If you see an ad, this will lead to a landing page on their landing page builder. Keep in mind that not all startups rely on these kinds of apps and not all use landing page builders.

Next is getting to the authentication system and to the web app. Just search for a login button on the homepage. Most startups I've seen do not use a separate authentication system, but simply integrate it into their web app, so when you see the login screen, you are already on the right subdomain. If they use an external solution like AWS Cognito, then you might have to create an account to find the actual subdomain. A typical subdomain name for the authentication system is auth, and for the web app is app.

This is also a good point to check for a staging environment that might be hosted e. g. at test, dev, or staging, and sometimes on a sub-level of the app, so e. g. dev.app or app.dev. The staging environment will most likely be very similar code but might have more debugger friendly settings.

When you open the web app, this also allows you to find the backend by opening the dev tools and monitoring XHR requests going out. A typical subdomain name for the backend is api.

Finding the native apps usually is quite easy: They are either mentioned on the homepage, or you find them by entering the startup's name in the app store search.

Analyzing their infrastructure

Most tools leave distinctive telltale signals when they are in use. Fortunately, there are companies out there which specialize in analyzing these. I personally use Wappalyzer to analyze each subdomain found above. There is also a free plugin for Chrome and Firefox, just keep in mind that if something is free, then you are the product: It most likely tracks the pages you visit to add more information to the Wappalyzer database.

This step gives you insights into which frameworks are used in the frontend, which kind of backend is running, and even more. But there is one more step.

Reading their source code

It should not be possible for you to access any backend source code from a different company. If it is, immediately let them know. This is a massive security problem.

Frontend code, on the other hand, is executed in the browser. Therefore it is accessible to anyone who can open the web app. Just open your DevTools, go to the sources tab, and there it is. On your way, you can even check for warnings or errors that the app emits.

But there is one more trick to it: The source code you see is most likely bundled and minified, which puts everything into one file and replaces all variable names with single letters. This makes it really hard to understand the code.

This also makes it really hard to understand error messages. That's why many companies have source maps enabled, which link each character in the scrambled file to the corresponding character in the original source. This way, an error that points to line 1 character 542 in the scrambled file can instead be traced to line 30 character 12 in the Login.tsx file.
These source maps are either used together with the original code in another tool or, and in my experience, this is more common, the source code is saved as inline source maps next to the minified code.

What does this mean for us? It means that the original source code is shipped with the application. You can find it when opening the web app in your browser's dev tools and open the "Sources" tab. You might have to search around a bit, but somewhere in there, you can find the source maps with all the juicy code.

Here's an example from my little todo app:

Unfortunately, sometimes this doesn't work. But don't give up your hope yet! It might be that source maps are disabled on production. But did you try the staging environment you found in the first step? It might have source maps enabled.
And maybe the code is lazy loaded in, so you don't have access to all source maps right away, but if you create an account and navigate through the app, you should get more and more insights.

What kind of things can you learn from here? Since the advent of Single Page Applications, a lot of business logic is living in web applications. You will be able to understand code structure, tools used, important features, features in development that are only rolled out to some of the users, backend structure, external tools used, ...

What did you learn with help of these methods? Feel free to follow me on Twitter and reach out with your learnings.

Why take-home assignments might not be as bad as their reputation

Daniel Bartholomae — Sun, 07 Feb 2021 19:13:17 +0000

I have dropped out of recruiting processes in the past after they asked me to do a take-home assignment. Still, a take-home assignment is part of almost every recruiting process I define. Here I'll write about why most take-home assignments suck, and how I design my own to avoid the same mistakes.

I'm focusing mostly on hiring developers in this blog post, but in my experience, most will hold true also for hiring other people for in-demand jobs.

If you are interested in more articles and news about web product development and entrepreneurship, please feel free to follow me on Twitter.

Why I reject most take-home assignments

Most take-home assignments suck. You spend hours, days, sometimes weeks, working on a problem no one cares about, just to be told you didn't get the job from someone who might not even have looked at your code. And then do it all over again for the next job application.

I've done my share of take-home assignments. The worst was algorithmic challenges. I work on web applications, and, except for maybe database queries, readability of the code usually is significantly more important than writing performant algorithms. But there were also some generic "create a full-stack app in 2 hours" tasks. Not as bad as the algorithmic ones. But still usually a waste of time.

And I am not only talking about myself. Especially the algorithmic challenges are not only a waste of the candidates' time but also a dangerous signal to the company hiring. I do not doubt that there is a correlation between performance on an algorithmic task and later on the job, but it is an indirect one. Those who studied computer science or specifically prepare for interviews, and spend a lot of time on their current occupation will both perform well on these challenges, as well as most likely invest time to learn the skills for their specific new job.

But it will also sort out those with unorthodox careers, limited time for preparation, and, worst of all (from the company's perspective), those that have the specific skills for the job and no longer need to spend time on grooming interview-only skills.

And why I still use them in recruiting

So why do I still use take-home assignments when hiring myself? If done correctly, they are actually not only effective but also help to avoid weeding out candidates that fit the job but don't fit the usual recruiting process. There's a couple of guidelines for good take-home assignments that I try to follow:

Right candidate motivation

Recruiting is like any kind of product work: If the customers (the candidates) think that your product is too expensive (they don't want to spend the time on a take-home challenge), then what this really tells you is that the product doesn't deliver enough value to them (they don't see a good reason to change from their current job to this new one).

Therefore the most important part of using take-home assignments is to first make sure that the job you are offering is really of interest to your candidates, and that they know this.

Relevant task

Instead of some made-up challenge, I choose a task we had to solve in the past at the company, and that the candidate would have worked on, had they already been there back then. I make sure that the task description matches what we thought the task was about back then, not what we figured out along the way.

So far I have found no better predictor on how well someone will perform on their job than seeing them perform on the exact job.

Relevant work environment

I usually tell the candidates to act for the challenge as if they were already working for the company, just from home. This includes googling, using tools, asking questions, and everything else they would be doing on the job normally. When they ask questions, I don't use that as an opportunity to test their knowledge, but try to help as much as possible instead. The same as I would for a colleague or employee.

There are some caveats to this, as the time a candidate works on a task usually isn't during working hours. So far I have mostly bit the bullet and answered those questions outside of my usual working hours, but this has challenges of its own, including the appearance that outside-hours work might be normal at the company.

Small scope

I slim down the task until it takes me less than 1 hour to do it. For someone without background, this can still mean a multi-hour task. I believe that when working with a company for a year or longer, it makes sense for the candidate to spend some time on the company. But there are limits, especially when hiring someone who already has a job and might not easily find a full day that early in the recruiting process.

With the right scope, this also brings the advantage of being more accessible to those who otherwise would get nervous in a one-on-one or group interview setting.

How this fits in the overall recruiting process

I am usually hiring in startup contexts. Usually the process I set up looks something like this:

I always have a really detailed job description including salary ranges, tools used, typical tasks, and as much information as possible to make sure that candidates know even before applying whether the job is for them.
For each candidate I check the CV whether it could match. Here I am quite liberal. It usually would be worse for me to sort out a good candidate too early than to sort out a bad candidate "too late". My estimate is that about 50% of candidates make it to the next step.
Next is a (video-)call. The sole purpose of this call is to convince the candidate that they should work for us. I also ask questions, because that is what candidates expect and without these, they would be uncomfortable, but I don’t really care about them. I’ve in the past actually outsourced this to a sales agent with quite a good success. Here I lose about 5% of candidates who don’t want to do the next step.
Now comes the take-home assignment. I tell the candidate about a dozen times (literally, it is really important to stress this) that the task can only be solved if they ask questions, and that this is totally fine and that they should just imagine they were already working for us and would be working from home for one day. This is the main cut-off, only about 5% of candidates make it through this.
Next is a trial half-day of working with the team. Before the 'Rona this happened in the office, with plenty of breaks, lunch, and opportunities to socialize. Remotely it is one or two longer video calls. The candidate needs to sign an NDA. The tasks here are not made up either, but just whatever the team is currently working on. This step loses another 5-10%.
At his point, I’m offering a contract and hire the candidate.

Each of the steps is required for take-home assignments to make sense. If the candidates do not know enough about the job or are obviously a bad fit, then asking them to do a take-home assignment isn't fair to them and their time. And even though the take-home assignment is in my experience quite a good measure for job performance, there is still a need to ensure team fit.

What are your experiences with take-home assignments from both sides? Feel free to jump into a discussion with me on Twitter.

10 bad TypeScript habits to break this year

Daniel Bartholomae — Thu, 21 Jan 2021 21:01:40 +0000

TypeScript and JavaScript have steadily evolved over the last years, and some of the habits we built over the last decades have become obsolete. Some might never have been meaningful. Here's a list of 10 habits that we all should break.

If you are interested in more articles and news about web product development and entrepreneurship, please feel free to follow me on Twitter.

Onto the examples! Please note that each "What it should look like" only fixes the issue discussed, even if there are further code smells that should be addressed.

1. Not using `strict` mode

What it looks like

Using a tsconfig.json without strict mode.

{
  "compilerOptions": {
    "target": "ES2015",
    "module": "commonjs"
  }
}

What it should look like

Just enable strict mode:

{
  "compilerOptions": {
    "target": "ES2015",
    "module": "commonjs",
    "strict": true
  }
}

Why we do it

Introducing stricter rules in an existing codebase takes time.

Why we shouldn't

Stricter rules will make it easier to change code in the future, so the time spent on fixing the code is returned and then some when working on the repository in the future.

2. Defining default values with `||`

What it looks like

Falling back with || for optional values:

function createBlogPost (text: string, author: string, date?: Date) {
  return {
    text: text,
    author: author,
    date: date || new Date()
  }
}

What it should look like

Use the new ?? operator, or, even better, define the fallback right at the parameter level.

function createBlogPost (text: string, author: string, date: Date = new Date())
  return {
    text: text,
    author: author,
    date: date
  }
}

Why we do it

The ?? operator has just been introduced last year, and when using values in the middle of a long function it might be hard to set them already as parameter defaults.

Why we shouldn't

??, unlike ||, falls back only for null or undefined, not for all falsy values. Also, if your functions are so long that you cannot define defaults at the beginning, then splitting them might be a good idea.

3. Using `any` as type

What it looks like

Using any for data when you are unsure about the structure.

async function loadProducts(): Promise<Product[]> {
  const response = await fetch('https://api.mysite.com/products')
  const products: any = await response.json()
  return products
}

What it should look like

In almost every situation where you type something as any, you should type it as unknown instead.

async function loadProducts(): Promise<Product[]> {
  const response = await fetch('https://api.mysite.com/products')
  const products: unknown = await response.json()
  return products as Product[]
}

Why we do it

any is convenient, as it basically disables all type checks. Often, any is used even in official typings (e. g. response.json() from the example above is typed as Promise<any> by the TypeScript team).

Why we shouldn't

It basically disables all type checks. Anything that comes in via any will completely forego any type-checks. This leads to hard to catch bugs, as code will fail only when our assumptions about type structure are relevant to the runtime code.

4. `val as SomeType`

What it looks like

Forcefully telling the compiler about a type that it cannot infer.

async function loadProducts(): Promise<Product[]> {
  const response = await fetch('https://api.mysite.com/products')
  const products: unknown = await response.json()
  return products as Product[]
}

What it should look like

That's what type guards are for.

function isArrayOfProducts (obj: unknown): obj is Product[] {
  return Array.isArray(obj) && obj.every(isProduct)
}

function isProduct (obj: unknown): obj is Product {
  return obj != null
    && typeof (obj as Product).id === 'string'
}

async function loadProducts(): Promise<Product[]> {
  const response = await fetch('https://api.mysite.com/products')
  const products: unknown = await response.json()
  if (!isArrayOfProducts(products)) {
    throw new TypeError('Received malformed products API response')
  }
  return products
}

Why we do it

When converting from JavaScript to TypeScript, the existing codebase often makes assumptions about types that cannot be deduced automatically by the TypeScript compiler. In these cases, throwing in a quick as SomeOtherType can speed up the conversion without having to loosen the settings in tsconfig.

Why we shouldn't

Even if the assertion might be save right now, this might change when someone moves code around. The type guard will ensure that all checks are explicit.

5. `as any` in tests

What it looks like

Creating incomplete stand-ins when writing tests.

interface User {
  id: string
  firstName: string
  lastName: string
  email: string
}

test('createEmailText returns text that greats the user by first name', () => {
  const user: User = {
    firstName: 'John'
  } as any

  expect(createEmailText(user)).toContain(user.firstName)
}

What it should look like

If you need to mock data for your tests, move the mocking logic next to the thing you mock and make it reusable.

interface User {
  id: string
  firstName: string
  lastName: string
  email: string
}

class MockUser implements User {
  id = 'id'
  firstName = 'John'
  lastName = 'Doe'
  email = 'john@doe.com'
}

test('createEmailText returns text that greats the user by first name', () => {
  const user = new MockUser()

  expect(createEmailText(user)).toContain(user.firstName)
}

Why we do it

When writing tests in a codebase that doesn't have great test coverage yet, there are often complicated big data structures, but only parts of it are needed for the specific functionality under test. Not having to worry about the other properties is easier in the short term.

Why we shouldn't

Foregoing the creation of a mock will bite us, latest when one of the properties changes and we need to change it in all tests instead of one central location. Also, there will be situations where the code under test relies on properties that we did not deem important before, and then all tests for that functionality need to be updated.

6. Optional properties

What it looks like

Marking properties as optional that are sometimes there and sometimes not.

interface Product {
  id: string
  type: 'digital' | 'physical'
  weightInKg?: number
  sizeInMb?: number
}

What it should look like

Explicitly model which combinations exist and which don't.

interface Product {
  id: string
  type: 'digital' | 'physical'
}

interface DigitalProduct extends Product {
  type: 'digital'
  sizeInMb: number
}

interface PhysicalProduct extends Product {
  type: 'physical'
  weightInKg: number
}

Why we do it

Marking properties as optional instead of splitting out types is easier and produces less code. It also requires a deeper understanding of the product being built and might limit usage of code if assumptions about the product change.

Why we shouldn't

The big benefit of type systems is that they can replace runtime checks with compile-time checks. With more explicit typing, it is possible to get compile-time checks for bugs that otherwise might have gotten unnoticed, e. g. by making sure that every DigitalProduct has a sizeInMb.

7. One letter generics

What it looks like

Naming a generic with one letter

function head<T> (arr: T[]): T | undefined {
  return arr[0]
}

What it should look like

Giving a full descriptive type name.

function head<Element> (arr: Element[]): Element | undefined {
  return arr[0]
}

Why we do it

This habit grew I guess because even the official docs use one-letter names. It is also quicker to type and requires less thinking to press T instead of writing a full name.

Why we shouldn't

Generic type variables are variables, like any other. We have abandoned the idea of describing the technicalities of variables in their names when IDEs started to just show us these technicalities. E. g. instead of const strName = 'Daniel' we now only write const name = 'Daniel'. Also, one letter variable names are generally frowned upon because it can be hard to decipher what they mean without looking at their declaration.

8. Non-boolean boolean checks

What it looks like

Checking whether a value is defined by passing the value directly to an if statement.

function createNewMessagesResponse (countOfNewMessages?: number) {
  if (countOfNewMessages) {
    return `You have ${countOfNewMessages} new messages`
  }
  return 'Error: Could not retrieve number of new messages'
}

What it should look like

Explicitly checking for the condition we care about.

function createNewMessagesResponse (countOfNewMessages?: number) {
  if (countOfNewMessages !== undefined) {
    return `You have ${countOfNewMessages} new messages`
  }
  return 'Error: Could not retrieve number of new messages'
}

Why we do it

Writing the check in short looks more succinct and allows us to avoid thinking about what we actually want to check.

Why we shouldn't

Maybe we should think about what we actually want to check. The examples above for example handle the case of countOfNewMessages being 0 differently.

9. The Bang Bang operator

What it looks like

Converting a non-boolean value to boolean.

function createNewMessagesResponse (countOfNewMessages?: number) {
  if (!!countOfNewMessages) {
    return `You have ${countOfNewMessages} new messages`
  }
  return 'Error: Could not retrieve number of new messages'
}

What it should look like

Explicitly checking for the condition we care about.

function createNewMessagesResponse (countOfNewMessages?: number) {
  if (countOfNewMessages !== undefined) {
    return `You have ${countOfNewMessages} new messages`
  }
  return 'Error: Could not retrieve number of new messages'
}

Why we do it

To some, understanding !! is like an initiation ritual to the world of JavaScript. It looks short and succinct, and if you are already used to it, then you know what it is about. It is a shortcut to convert any value to a boolean. Especially if, in a codebase, there is no clear semantic separation between falsy values like null, undefined, and ''.

Why we shouldn't

Like many shortcuts and initiation rituals, using !! obfuscates the true meaning of the code by promoting insider knowledge. This makes the codebase less accessible to new developers, be it new to developing in general, or just new to JavaScript. It's also quite easy to introduce subtle bugs. The problem with countOfNewMessages being 0 from "Non-boolean boolean checks" persists with !!.

10. `!= null`

What it looks like

The little sister of the bang bang operator, != null allows us to check for null and undefined at the same time.

function createNewMessagesResponse (countOfNewMessages?: number) {
  if (countOfNewMessages != null) {
    return `You have ${countOfNewMessages} new messages`
  }
  return 'Error: Could not retrieve number of new messages'
}

What it should look like

Explicitly checking for the condition we care about.

function createNewMessagesResponse (countOfNewMessages?: number) {
  if (countOfNewMessages !== undefined) {
    return `You have ${countOfNewMessages} new messages`
  }
  return 'Error: Could not retrieve number of new messages'
}

Why we do it

If you got here, your codebase and your skills are already in quite good shape. Even most linting rulesets that enforce using !== over != make an exemption for != null. If there is no clear distinction in the codebase between null and undefined, then != null helps to shorten a check for both possibilities.

Why we shouldn't

While null values were quite a hassle in the early days of JavaScript, with TypeScript in strict mode, they can become a valuable member of the language's tool belt. A common pattern I have seen is defining null values as things not being there, and undefined as not things that are unknown, e. g. user.firstName === null might mean that the user literally does not have a first name, while user.firstName === undefined just means that we haven't asked that user yet (and user.firstName === '' would mean the first name literally is '' - you'd be surprised which kind of names actually exist).

TDD in a React frontend

Daniel Bartholomae — Tue, 19 Jan 2021 22:39:43 +0000

Nowadays, only a few professional developers are left that seriously doubt the value of test-driven-development and test-driven-design (tdd). But the reality of many codebases I have seen is that tdd is often limited to the backend, where the "business logic" lives.

Part of this is due to a stigma that frontend development is not "real software development", even though in most cases a fully functional backend is completely unusable without the matching frontend. But part of it is due to missing skills on how to tdd in the frontend. This is what this article is about.

I'm taking React as an example because it is the framework I am most familiar with and the declarative style makes it easier to some of the tests than when using pure JavaScript, HTML, and CSS. But most of the ideas from this article hold also in other contexts.

If you are interested in more articles and news about web product development and entrepreneurship, please feel free to follow me on Twitter.

Why is frontend testing harder than backend?

It is not always laziness that drives frontend engineers away from tdd. This becomes especially obvious when watching full-stack engineers, who religiously practice tdd for their backend code, not write a single test in the frontend.

In my experience, the differences boil down to three points:

In the frontend, features usually have significantly bigger interfaces. While a backend API in its simplest version might be defined by a simple JSON structure, even the simplest frontend feature will be defined not only by functionality but also by often thousands of pixels rendered to the screen.
Even worse, we don't yet have a good way to explain to a machine which of these pixels matter. For some, changing the pixels doesn't really make any difference, but change the wrong ones, and the feature becomes completely unusable.
For a long time, tooling did not allow for integration tests that run in seconds. Instead, tests either had to be limited to pure business logic or run in the browser with often minutes of setup time.

So how do we fix this?

Writing testable frontend code

Similar to how you often need to split backend code and introduce dependency injection to be able to test it, frontend code also should be split to make it easier to test. There are roughly three categories of frontend code, each of them with a different way to test them.

Let's take a classical React todo app as an example. I recommend to open the repository on a second screen and follow along. I have added code excerpts to this article for those who might read on the mobile phone or otherwise don't have access to the repository while reading.

Glue code

The App component and the useTodos hook are what I like to call glue code. It "glues" together the rest of the code to bring the functionality to life:

const TodoApp: FunctionComponent = () => {
  const { todos, addTodo, completeTodo, deleteTodo } = useTodos([]);

  return (
    <>
      <TodoList
        todos={todos}
        onCompleteTodo={completeTodo}
        onDeleteTodo={deleteTodo}
      />
      <AddTodo onAdd={addTodo} />
    </>
  );
};

export function useTodos(initialTodos: Todo[]) {
  const [todos, dispatch] = useReducer(todosReducer, initialTodos);
  return {
    todos,
    addTodo: (description: string) =>
      dispatch(createAddTodoAction(description)),
    completeTodo: (id: Todo["id"]) => dispatch(createCompleteTodoAction(id)),
    deleteTodo: (id: Todo["id"]) => dispatch(createDeleteTodoAction(id)),
  };
}

Similar to a controller in the backend, this is best tested with integration tests:

describe("TodoApp", () => {
  it("shows an added todo", async () => {
    render(<App />);

    const todoInput = screen.getByLabelText("New todo");
    const todoDescription = "My new todo";
    userEvent.type(todoInput, todoDescription);
    const addTodoButton = screen.getByText("Add todo");
    userEvent.click(addTodoButton);

    expect(await screen.findByText(todoDescription)).toBeInTheDocument();
  });
});

The reason why I am talking about these tests first is that this is usually the first kind of test that I write. The difference between a web app and a landing page is that the web app, without any of its functionality and just with its looks, has no value. These tests describe the behavior and allow me to keep focused so that I only implement what is needed.

These kinds of integration tests should be as independent of the technology used as possible. The test examples above are dependent on React (if I were to rewrite the app without React, I would have to change the tests as well), but that's about it. The same tests would work irrespective of whether I am using functional components, class components, Redux state management, an external form library, or whether I use 3 or 300 components to build the todo app. This is very important, as it means that I can safely refactor the code without touching the tests.

The reason for this is that the tests are written from a user perspective: Find something labeled "New todo", type the new todo into it, press the "Add todo" button and check that the todo I just wrote now shows up on the screen.

Business logic

These are the tests that folks coming from backend testing are most familiar with. The business logic of our todo app takes care of creating, removing, and marking todos as done. The exact same could also be used in the backend.

export function todosReducer(todos: Todo[], action: TodoAction) {
  switch (action.type) {
    case TodoActionType.AddTodo:
      return [...todos, action.payload];
    case TodoActionType.CompleteTodo:
      return todos.map((todo) =>
        todo.id === action.payload.id ? { ...todo, completed: true } : todo
      );
    case TodoActionType.DeleteTodo:
      return todos.filter((todo) => todo.id !== action.payload.id);
  }
}

Tests for this kind of code are deceivingly simple:

describe("todo reducer", () => {
  describe("addTodoAction", () => {
    it("adds a new todo to the list", () => {
      const description = "This is a todo";
      expect(todosReducer([], createAddTodoAction(description))).toContainEqual(
        expect.objectContaining({ description })
      );
    });

    it("does not remove an existing todo", () => {
      const existingTodo = new TodoMock();
      expect(
        todosReducer([existingTodo], createAddTodoAction("This is a todo"))
      ).toContainEqual(existingTodo);
    });
  });
});

The hard part about testing business logic is not to write the tests, but to separate the business logic from the rest of the code. Let's have a look at useTodos, which is the glue code bringing this reducer into React:

export function useTodos(initialTodos: Todo[]) {
  const [todos, dispatch] = useReducer(todosReducer, initialTodos);
  return {
    todos,
    addTodo: (description: string) =>
      dispatch(createAddTodoAction(description)),
    completeTodo: (id: Todo["id"]) => dispatch(createCompleteTodoAction(id)),
    deleteTodo: (id: Todo["id"]) => dispatch(createDeleteTodoAction(id)),
  };
}

The danger here would be to write the business logic so that it can only be tested by testing the full hook. Using the hook just to glue together the reducer and action creators with React logic saves us from all that pain.

Presentational components

Last, but not least, let's look at the presentational code. These components define the interface to the user, but do not contain any business logic on their own. This is where most of the problems I mentioned at the beginning of the article come to pass. And, to be quite honest, I haven't found a perfect solution to all of them. But there is a concept that gets close:

A story is the visual equivalent of a unit test. The main remaining shortcoming is that the step of asserting whether or not the test was successful has to be done manually.

Here's a story for a button:

const Template: Story<Props> = (args) => <Button {...args} />;

const actionArgs = {
  onClick: action("onClick"),
};

export const Default = Template.bind({});

Default.args = {
  ...actionArgs,
  children: "Click me!",
  color: ButtonColor.Success,
};

and here is the button itself:

export enum ButtonColor {
  Alert = "Alert",
  Success = "Success",
}

export enum ButtonType {
  Submit = "submit",
  Reset = "reset",
  Button = "button",
}

export interface Props {
  children: ReactNode;
  color: ButtonColor;
  onClick?: () => void;
  type?: ButtonType;
}

export const Button: FunctionComponent<Props> = ({
  children,
  color,
  onClick,
  type,
}) => {
  const colorStyles = {
    [ButtonColor.Alert]: {
      border: "#b33 solid 1px",
      borderRadius: "4px",
      boxShadow: "2px 2px 2px rgba(100,0,0,0.8)",
      color: "white",
      backgroundColor: "#a00",
    },
    [ButtonColor.Success]: {
      border: "#3b3 solid 1px",
      borderRadius: "4px",
      boxShadow: "2px 2px 2px rgba(0,100,0,0.8)",
      color: "white",
      backgroundColor: "#0a0",
    },
  };
  return (
    <button
      style={{
        ...colorStyles[color],
        padding: "0.2rem 0.5rem",
      }}
      onClick={onClick}
      type={type}
    >
      {children}
    </button>
  );
};

The story renders the button in isolation. I can first write the story, which allows me to think about the intended interface for this component, and only implement the component itself afterward. If any implementation details change, then as long as the interface stays the same, I won't have to change the story. And I can look at the rendered story in isolation whenever I want to verify that it still looks as intended (this is the "manual" part I mentioned above). As soon as I have a version I am happy with, I can even set up automated regression testing with help of a visual regression tool.

The All Together

What would it look like in practice, developing this todo app tdd-style?

Write an integration test that the text "No todos" should be visible if there are no todos
Fulfill the test by implementing the App component so that it just returns "No todos"
Extract "No todos" into its own component
Add a story for it
Use the story to drive visual changes until the "No todos" part looks like it should
Add an integration test about adding a todo
Start implementing the test and realize that I will need some kind of state management
Comment out the integration test
Write a unit test for the state reducer
Fulfill the test by writing a simple first version of the reducer
Write a story for displaying the list of todos
Use the story to drive the implementation of a TodoList component
Comment the integration test back in
Fulfill the integration test by gluing together the reducer and the component
...

Obviously, there are many other ways to go about this. But hopefully, this shows one potential workflow to use tdd in the frontend.

If you are interested in more articles and news about web product development and entrepreneurship, please feel free to follow me on Twitter. And please send me a tweet about your experiences with tdd in the frontend!

Automating Open-Source on GitHub

Daniel Bartholomae — Fri, 08 Jan 2021 20:59:01 +0000

Maintaining open source is a noble, but can be time-consuming endeavor. It doesn't have to be, though. During Hacktoberfest 2020 I used one of my repositories to experiment with automation around contribution and maintenance of jsx-readme. This is what I learned.

Make it easy to contribute

The benefit of open-source project is that whenever someone has a feature request or finds a bug, they can fix it themselves by raising a PR. At least in theory. In practice, it is really hard to contribute to many of the open-source projects out there. To be able to contribute, you need to

understand the codebase,
make it run locally, and
know which standards to adhere to.

All of this becomes significantly simpler with a good CONTRIBUTING.md. I've looked through multiple, both from big as well as small projects, and created a simple one.

But this is not all. The code should also be structured so that it is easy to understand, and well tested so that contributors can be sure they didn't break anything.
Furthermore, having as many rules as possible codified in an automated fashion, e. g. via linting, makes it easy for a contributor to just open a PR and get feedback. This is why in my projects I always have a pre-commit hook that runs eslint --cache --fix with a config that includes prettier. Which brings us to...

Review PRs automatically

Being able to review a PR without having to check out and run the code locally, makes the whole process significantly better. With my setup, I think I have so far only had one PR which I had to check out locally to do some adjustments on my own, and for the rest I felt confident in accepting them after just reading the code changes. Why?

I enforce that there has to be an issue before a PR. This way I can first discuss what to do and use the PR only to check the how.
There is 100% test coverage, and it is enforced automatically. Usually, 100% test coverage might not be that helpful, but for open source it means that contributors will have to test-cover the code they add. The CI will tell me both if they covered 100%, as well as whether their tests pass. So when reviewing, I usually focus on the tests first: Do they test everything discussed in the issue? Then I review the rest of the code for maintainability. As a last step, I go back to the tests, because sometimes reading the code brings to mind some edge cases that I might not be sure about. With 100% test coverage, propablity is high that they are covered.
I do not review code for style, this is taken care already by linting and prettification.

How does the automation work in practice? I've used TravisCI and CircleCI in the past, but have switched to GitHub Workflows last year. I run tsc, eslint, jest, and pkg-ok against every PR. tsc builds and compiles everything and surfaces type errors. eslint checks not only for linting, but also for style, via the prettier plugin. jest runs both unit tests, which test the source code, as well as integration tests, which run on the compiled code to make sure that everything is compiled into the right spot. And pkg-ok makes sure that the entry points defined in package.json exist.

Synchronize documentation and code

When writing an open source library, usually the hope is for others to use it. This requires a good external documentation. A good starting point is a README file with an example. But for most libraries you will need more.
Unfortunately, documentation has the tendency of getting out-of-sync with the code it is describing quite quickly. And when maintaining an open-source repo, ideally you can make changes quickly without having to worry about documentation too much.
So for jsx-readme, I used TypeDoc, a TypeScript documentation library based on the JSDoc standard (which also lead me to discover something about how React and JSX work together). This way, my documentation is next to the code:

/**
 * Defines a regex to search with and a string that should be used
 * to replace the results found.
 */
interface Replacement {
  find: RegExp;
  replace: string;
}

/** @internal */
interface Props {
  fileName: string;
  children: string;
  replacements?: Replacement[];
}

/** Displays a code file with a heading and a codeblock. */
export const CodeFile: Component<Props> = ({
  /** The file's content. */
  children,
  fileName,
  /**
   * A list of replacements to be made in the file's content,
   * e. g. for replacing relative import paths.
   * */
  replacements = [],
}) => {
  function executeReplacements(str: string): string {
    return replacements.reduce(
      (replacedStr, replacement) =>
        replacedStr.replace(replacement.find, replacement.replace),
      str
    );
  }

  return (
    <Fragment>
      <Heading level={3}>{fileName}</Heading>
      <CodeBlock language={path.parse(fileName).ext.slice(1)}>
        {executeReplacements(children.trimEnd())}
      </CodeBlock>
      <LineBreak />
    </Fragment>
  );
};

Thanks to TypeScript and good naming, not everything needs a long description. A fileName that is a string should be understandable without additional documentation. For the rest, we can add comments right next to the code, and the documentation will be created accordingly. This way, when I (or a contributor that comes in for the first time), e. g. adds a new prop, it will show up in the documentation, and the existing documenting comment right next to it will inspire me to add all needed documentation there as well.

But even more important than the documentation deeper down in the code, is the README on top of it. This is where jsx-readme itself comets in:

const Readme: Component = () => (
  <Fragment>
    {/* Create a header with title, badges and description inferred from package.json */}
    <TitleFromPkg pkg={pkg} />
    <BadgesFromPkg pkg={pkg} />
    {/* Add additional badges. */}
    <LineBreak />
    <DescriptionFromPkg pkg={pkg} />
    {/* Create an example section based on all files from the example directory set up in package.json */}
    <ExamplesFromPkg pkg={pkg} />
    {/* Create a section linking to the homepage from package.json */}
    <HomepageFromPkg pkg={pkg} />
    {/* Create a section linking to the contributing guidelines file */}
    <ContributingSection />
    {/* Create a section linking to the license file. */}
    <LicenseFromPkg pkg={pkg} />
  </Fragment>
);

void renderToFile("./README.md", <Readme />);

This will be autocreated from package.json and the examples file, so if either one changes, the README changes with them. In addition, I used the example files as part of the integration tests, so I always know that the examples actually work.

Last but not least, building the documentation should happen on each merge automatically. This is done with help of a script

typedoc && touch docs/.nojekyll && ts-node examples/README.md.tsx

which is run via a GitHub Workflow that we will look at later in this article.

One thing that I haven't yet set up, is an automatic check of documentation coverage. I had used that in previous open-source projects, but it lead to meaningless documentation just to fulfill the coverage.

Automatically create changelogs

Another part of documentation is the changelog. To simplify processes here, I am using semantic versioning which means that I describe all changes in a machine-readable format in my commit messages. The big advantage is, that I do not need to maintain additional documentation: I create the changelog automatically based of the commits.

This also has a disadvantage, though: Git isn't the most user-friendly tool, and writing meaningful commit messages is a rare skill even for an experienced developer. I have a check in the CI to ensure that all commit messages of a PR are in the right format, but updating commits has so far been the most time-consuming part of PR discussions.

Release on push

Now that I know that the PR is mergable, all I have to do is approve the review. Everything else is automated:

mergify waits for the CI to be green and the PR to be approved, and then automatically merges it.

Semantic Release reads through the commit messages and figures out whether a new release is warranted (if at least one commit contains a new feature, a fix, or a breaking change) and which kind of version bump should happen. It also creates the changelog, commits all changed files into a release commit and tags the commit with the release version. Last but not least it releases to npm. All of this aided by a plethora of plugins.

Keep dependencies up-to-date

Last but not least, maintaining an open-source repository also means keeping dependencies up-to-date. The best way around this obviously is to not have dependencies in the first place, and especially in the JavaScript world this approach is still underused: For each dependency think about whether it is really needed, as it will not only show up in your project, but then in every project that uses your library. So for each dependency that your users will have to install, think twice whether you really need it.

For the remaining dependencies, the setup we have so far make it really easy to keep up-to-date: I set up dependabot to automatically create PRs with new package versions. Thanks to 100% test-coverage I know that if all tests still pass, I can just merge the PR. Which means, that I can actually configure mergify to do this for me automatically. And then the GitHub workflow will automatically create the appropriate release.

What's next?

Overall, the experiment during Hacktoberfest was quite interesting, and I am overall really happy with my setup. My next step on this will be to create a project template out of it to make it easier for me to create new open-source projects. If you don't yet, feel free to follow me on Twitter for the latest news on that and for more on web technology and entrepreneurship.
If you do, please tweet to me what your experiences are on maintaining open-source projects, and how to make it as painless as possible?

What writing my own JSX renderer taught me about React

Daniel Bartholomae — Tue, 05 Jan 2021 22:13:40 +0000

One of the reasons that React got so popular is definitively the syntax it introduced: Writing HTML-like code to declaratively describe components just feels good. But this brought me to the question: Why is this syntax only used for React, basically for describing HTML?

My first realization hit after a few minutes of research: It isn't. And at least one of them you have most likely already seen in action: Ink. It is used to declaratively build CLIs, e. g. for Gatsby, Yarn or Terraform.

This gave me the courage to try something like this on my own, a journey that lead to jsx-readme and the underlying jsx-md. In this post, I will lead you along my journey and the learnings this sparked about JSX, and about React.

If you are interested in more on tech, entrepreneurship, and how to bring these two together, feel free to follow me on Twitter.

Our goal

We want to be able to write code like

function Readme() {
  return (
    <Fragment>
      <Heading level={1}>{pkg.name}</Heading>
      <LineBreak />
      <Text>{pkg.description}</Text>
    </Fragment>
  );
}

writeFile("examples/README.md", <Readme />);

to create markdown like

# jsx-md
Generate markdown files with a React\-like syntax.

Later on, this will allow us to write more complicated components from these building blocks. But now: Let's start with some fundamentals. You can skip everything you know and follow the headlines.

Markdown

Markdown is a markup language. It allows to add formatting like italics or bold with help of text characters. E. g. this blog post is written in Markdown:

## Markdown

Markdown is a markup language. It allows to add formatting like *italics* or __bold__ with help of text characters. E. g. this blog post is written in Markdown:

It is also used for documentation of code, e. g. README files.

JSX

JSX is syntactic sugar for JavaScript. It is compiled down to pure JavaScript, and therefore can only be used if a compile step is available, for example via webpack or TypeScript. For compiling down, it needs to know which pragma to use. By default, most compilers use React, e. g.

<article>
  <h1 id='primary'>Writing Markdown with JSX</h1>
  <p>One of the reasons that React got so popular...</p>
</article>

becomes

React.createElement(
  'article',
  {},
  React.createElement('h1', { id: 'primary' }, 'Writing Markdown with JSX'),
  React.createElement('p', {}, 'One of the reasons that React got so popular...')
)

but you can tell the compiler to use a different function someOtherFunction instead of React.createElement by adding the pragma /** @jsx someOtherFunction */ to the beginning of the file.

In the end, JSX is just syntactic sugar for function calls

First try: Returning plain strings

So if JSX is syntactiv sugar for functions, and markdown is just a special kind of string, then why not just write a function that returns a string? This was the first approach I tried and lead to code like this:

/* @jsx createElement */

function createElement (typeOrComponent, attributes, ...children): string {
  if (typeof typeOrComponent === 'function') {
    return typeOrComponent({ ...(attributes ?? {}), children })
  }
  return children.join('')
}

function Heading ({ children, level }: Props) {
  return <md-text>{'#'.repeat(level)} {children}</md-text>
}

assert.strictEqual(<Heading level={1}>Test</Heading>, '# Test')

And I got this to a working version. But why, then, is there no release with this?

Before releasing, I wanted to add documentation. And for documentation, I wanted to use TypeDoc, so I can put the documentation right in the code and create pretty HTML files automatically.

The problem? TypeDoc uses React and imports the React types into global space. So with <Heading>Test</Heading> returning a string, I was met with a TypeScript error.
This left me with two options:

Get rid of TypeDoc and hope that no one ever tries using my new library in the same repository as React
Adjust my structure to the way that React does it

Interlude: How does React do it?

To figure out how React actually does this, I looked into two sources:

To summarize: <div className='test'>Test</div> (which is syntactic sugar for React.createElement('div', { className: 'test' }, 'Test')) returns a so called element:

{
  type: 'div',
  props: {
    children: 'Test',
    className: 'test'
  },
}

For nested props (e. g. children), the element simply becomes a nested JSON structure.

The render function then takes this structure and converts it into HTML, or, in case of the string renderer, into a string containing this HTML.

Second try: Returning elements

Instead of directly returning the markdown string from createElement, we now return an element and then render the element in a separate render function:

/* @jsx createElement */

function createElement (type, attributes, ...children) {
  return {
    type,
    props: {
      ...(attributes ?? {}),
      children: children.length <= 1 ? children[0] : children,
    },
    key: null,
  };
}

function render(element): string {
  if (element === null || element === undefined || element === false) {
    return "";
  }
  if (typeof element === "string") {
    return element;
  }
  if (typeof element === "number") {
    return element.toString();
  }
  if (Array.isArray(element)) {
    return element.map((el) => renderNode(el)).join("");
  }
  if (typeof element.type === "function") {
    return render(element.type(element.props));
  }
  throw new Error("Invalid element");
}

function Heading ({ children, level }: Props) {
  return <md-text>{'#'.repeat(level)} {children}</md-text>
}

assert.strictEqual(render(<Heading level={1}>Test</Heading>), '# Test')

Yoyu can find the full, unabbreviated code as version 1.1.0.

Real-life application of jsx-md

When starting with jsx-md, I already had an application in mind. One of the first open source projects I wrote, was a script in CoffeeScript that created README files for open source projects. With jsx-md, I can now describe the components of a README file declaratively:

import package from './package.json'

const DescriptionFromPkg: Component<Props> = ({
  pkg: { description },
}: Props) => {
  if (description === undefined) {
    return null;
  }
  return (
    <Fragment>
      <Text>{description}</Text>
      <LineBreak />
      <LineBreak />
    </Fragment>
  );
};

writeFileSync('README.md', render(<DescriptionFromPkg pkg={package} />))

Overall this lead me to write jsx-readme, a library for describing README files in JSX.

But what about hooks?

Nowadays, it is quite uncommon to write about React and never mention hooks even once. So what about hooks?

Hooks are a solution by React to solve two problems: First, that the declarative description of a component is executed on every render, but some side effects shouldn't. And second, that there should be a way to tell an individual component to rerender with new data, without having to pass this information through the full component tree.

Both are not that relevant to rendering a static markdown file - it doesn't really have side effects, and changing it happens on timeframes far too great to have the render function running continuously. But when working on jsx-readme, I did run into something that would be solved with hooks in React and that I couldn't yet solve with jsx-md:

What if the data that I need to first render a component needs to be fetched asynchronously?

This, fortunately, did no require a full implementation of hooks, or even of context. Instead, all I had to do, was make the rendering asynchronous and allow promises as children of elements:

/* @jsx createElement */

function createElement (type, attributes, ...children) {
  return {
    type,
    props: {
      ...(attributes ?? {}),
      children: children.length <= 1 ? children[0] : children,
    },
    key: null,
  };
}

function renderAsync(element): Promise<string> {
  if (element === null || element === undefined || element === false) {
    return Promise.resolv("");
  }
  if (typeof element === "string") {
    return Promise.resolv(element);
  }
  if (typeof element === "number") {
    return Promise.resolv(element.toString());
  }
  if (Array.isArray(element)) {
    return Promise.resolv(element.map((el) => renderNode(el)).join(""));
  }
  if (typeof element.type === "function") {
    return render(element.type(element.props));
  }
  if (element.type === 'mdAwait') {
    return element.props.children;
  }
  throw new Error("Invalid element");
}

function Heading ({ children, level }: Props) {
  return <md-text>{'#'.repeat(level)} {children}</md-text>
}

renderAsync(<Heading level={1}>Test</Heading>).then((result) =>
  assert.strictEqual(result, '# Test')
);

What now?

First of all, if you found the article interesting and would like to hear more about tech, entrepreneurship, and how to bring the two together, then please feel free to follow me on Twitter.

Both jsx-md as well as jsx-readme are open source and hopefully in a state where the code is easy to understand, so feel free to roam around a bit.

And if you are interested – maybe you want to contribute to these repositories to allow even more markdown shenangians, and learn about the core of React on the way?

Summarized – Tech and business books I read in 2020

Daniel Bartholomae — Mon, 04 Jan 2021 20:29:58 +0000

In 2020 I thought I would find less time to read, due to the missing commute, my main reading time for 2019. This lead me to explicitly take time to read, and actually finish more books than in the year before. And most of them were great books, too! Here's a summary of what I learned from each.

Overall there's three themes in the books I read:

Ideation and Customer Development, which I read up on due to my time testing out different startup ideas for half a year,
Agile Software Architecture, based on my general interest in how to get away without any planning and still not end up in a hot mess, and
Product/Engineering Organization, which I used mostly end of year when working with startups to scale their development teams.

I've added affiliate links for everyone who wants to support me, but please do not feel pressured to use them. Feel free to also follow me on Twitter for more startup and tech related content.

Ideation and Customer Development

Lean Customer Development (by Cindy Alvarez)

The first book I read about how to get to a working idea was also the best one from my perspective. The main idea is to talk to potential customers to find and validate ideas. It goes into detail how to find customers to talk to, which questions to ask and how to best ask them, how to find an MVP, and how to validate it with minimal effort. From a content perspective it is similar to The Mom Test (see below), but I found it to be a bit more practical.

If you are building product in an early-stage startup (or any other agile organization), this is a must-read.

Loonshots (by Safi Bahcall)

While the other books in this category are aimed (at least partially) at startups, Loonshots by Safi Bahcall is taking more of a bird's-eye view on the topic of innovation. The main idea from the book is that an organization can only grow to a certain size and remain innovative, because managing a big organization requires a different mindset. It proposes to split your organization in two, one part that is innovative and one that is operational, and tie them together close enough so ideas can permeate.

It's an interesting read due to the examples for innovation, but not really necessary if you are building a startup.

Play Bigger (by Al Ramadan)

Play Bigger introduces the idea of category design. Instead of just building a product, you build a new market demand, while at the same time building a product that fulfills the market demand and an organization that builds the product. The idea of doing everything at the same time didn't resonate that well with me, but finding a customer problem and then promoting the problem instead of your own solution is a very powerful tool.

This book can bring you in the right mindset for selling a big vision as a CEO.

The Mom Test (by Rob Fitzpatrick)

The Mom Test is on the recommended reading list by EF, the program I joined in March 2020. It's basically the same content as Lean Customer Development: Ask your customers what to build, but ask them in a way that does not incentivize them to lie. This is achieved by asking about how they currently do things and extrapolating from there, instead of asking how they would do things in the future.

It's a good alternative to Lean Customer Development. It's a bit more famous, but personal I liked it slightly less. Feel free to choose either one.

Agile Software Architecture

Domain-Driven Design (by Eric Evans)

A modern classic in software development, Domain-Driven Design introduces ideas on how to structure code based on the business domain. There is a lot of very good insights, but for me the strongest idea is the domain language: Things in the code should not be called by technical terms, but by business terms instead. Ideally, the code written on the business logic should be understandable by someone who knows the business, but does not know how to code.

It's a good fundamental read, but there are some signs of stiff Java structures in their that might not be that relevant with modern programming languages.

The Clean Coder (by Robert C. Martin)

The Clean Coder by Uncle Bob is one of the wider known standards in computer literature. Still, it is the only book in this list that I will not add a link to. The reason for this lie in the person of Robert C. Martin who I do not want to support financially in any way due to sexism and racism (see for example "The Politics" in this article). Unfortunately I only became aware of this after I had already bought this book.

After having read Clean Code (there's a good JavaScript based summary if you haven't read that one and don't want to buy it), this was on my reading list for some time. The main ideas are true, and easily summarized: Be professional and don't let non-coders take code decisions for you. Say no if a manager pushes you to churn out code quickly that you know will break. And rely on good practices like tdd and practice.

Even though I like the ideas mentioned in the book, I currently cannot recommend it for the reasons given above. My goal for this year is to find a book on this topic from a different author that I can recommend.

Refactoring Databases (by Scott W. Ambler)

One of the biggest obstacles I had over the last years when working in an agile way on projects, was adapting state-full applications like databases. Other than a state-less application, these cannot simply be replaced, instead they have to be migrated. This lead me to read Refactoring Databases. I did not fully read it, as it went into detail that I did not yet need, but the general ideas of how to refactor databases still stick with me, especially using views and database procedures to support multiple different interfaces (e. g. table structures) at the same time.

If you mainly work with databases, it is a must read. Otherwise, just read it if you struggle in this specific area.

Building Evolutionary Architectures (by Neal Ford)

The other obstacle I still face a lot when working agile is the question of how much architecture needs to be defined top-down and how much can develop organically. Building Evolutionary Architectures by Neal Ford applies the idea of test-driven design to architecture, but instead of testing functionality, it introduces the idea of fitness functions to test for the boundary conditions that should come from the architecture, like performance, cost, or coupling. This way, assumptions and reasoning are made transparent, and the CI pipeline can enforce future adherence to these rules, while leaving open the opportunity for developers to evolve the architecture within these bounds.

It's a very good read for anyone working as software architect, with the risk of changing how you view your job.

Patterns of Enterprise Application Architecture (by Martin Fowler)

Another classic in this list, Patterns of Enterprise Application Architecture unfortunately was quite a dull read for me. Similar to Domain-Driven Design, the stale stench of Java emitted from the book took some getting used to, and that it is more a list of patterns than an interwoven story made it even harder to read. I still think that I learned a lot from it, so I would recommend to at least have a look at the book before dismissing it completely, just be prepared for a tough read.

Living Documentation (by Cyrille Martraire)

The last book I read from this category was way more specialized than the others, but also the best of the bunch. In Cyrille Martraire's Living Documentation he poses the question how to transfer knowledge in an organization and which role documentation can play. The main idea is to not separate documentation from code, but instead to bring them as close together as possible, so that it becomes as hard as possible to change one without the other. Some ideas that especially stuck with me:

Automatically create diagrams from code
Add onboarding examples to the codebase to show how things are done around here
Do not duplicate documentation, but instead link to it, e. g. instead of describing the build process in the readme, link to the commented build config file

This is a must read for anyone who needs to introduce others to a codebase more than once per year.

Product/Engineering Organizations

Shape Up (by Ryan Singer)

Basecamp is known for some unconventional management techniques. With Shape Up, they published a free book on their product management philosophy. It revolves around 6 week blocks of time, so called bets, where teams work on their own on topics and find the best solution possible within this limited time. The main idea is to fully commit to an idea for 6 weeks and give full freedom to the team to implement during this time, but to reevaluate priorities without thinking about sunk cost after.

The book is a short read, and if you are building a product organization will give you helpful ideas.

Team Topologies (by Matthew Skelton)

While Shape Up is already almost old, Team Topologies is still the rage in tech organization literature. There are already quite a few blog posts summarizing the book (e. g. this one), but in a sentence, it is about the following ideas: Build teams that can create full customer solutions start to finish on their own and teams that create tools for enabling the first group.
If you are building a product development organization of more than 10 people, read this book before it becomes too late.

Radical Candor (by Kim Scott)

Those who know me (or have read my GitHub profile) know how highly I value transparency and honesty. This book is right up my alley: As a leader, be honest and direct about your feedback to your employees. Be polite, and, more importantly, empathic, but bring out any problems you might encounter as soon as possible, so people actually get a chance to change.

This is a must-read for anyone in a leadership position.

Dynamic Reteaming (by Heidi Helfand)

The last book I read this year I stumbled upon via this infographic which mentioned a lot of good books I had already read – and this one. Even though it talks about tech, it is actually a quite general book on organization and how to form and disband teams. It's a collection of methods to use when splitting, merging, building, or disbanding teams to make the process more effective and reduce the time needed for teams to grow together. Apart from some specific workshop formats, this also emphasized the idea for me that teams do not only need a ritual when forming but also when disbanding.

If you are in the process of reorganizing your org, or if you have a lot of team churn, you should have a look.

Others

Form Design Patterns (by Adam Silver)

The first outlier this year is a book about writing forms in HTML, CSS, and JavaScript. I stumbled upon it in Smashing Magazine (who also published it) and fell in love with the idea of an in-depth book about a small part of software engineering. There are a lot of great learnings in the book: How to make form elements that work in HTML, but work better if JavaScript is enabled. Or how to deal with accessibility. But it is not all sunshine. The HTML examples are hard to follow (I was reading it on my phone as all other books), so I skipped them. And some opinions are strongly colored by the author working mainly in frontend design, e. g. when he argues that usability trumps security.

If you are interested in frontend development, this is a good read.

Never Split the Difference (by Chris Voss)

The last book in this list is one of the, if not the best book I read in 2020. Never Split the Difference by Chris Voss is a book about negotiation, written by an FBI hostage negotiator. The main idea: Just say no. As long as you know you will get an offer that is at least as good, decline offer after offer until you get an offer that is too good to say no to. There's obviously more to it, but for the details, you will have to read the book.

It's a must-read for everyone, especially if you are negotiating your salary for a new job.

Outlook

For next year, I have a couple of books on my digital shelf. Currently, I'm reading the AWS Certified Solutions Architect Study Guide, and there are reading samples for a lot more books waiting for me. If you want to follow along, feel free to follow me on Twitter for more startup and tech related content. And if you do, please tweet to me: What books did you read in 2020? And what did you learn from them?

Preparing your repositories for Hacktoberfest

Daniel Bartholomae — Sat, 26 Sep 2020 13:46:50 +0000

Hacktoberfest is upon us. Starting 1st of October, we will all be celebrating Open Source contributions with tons of swag. But while contributors still have to wait for the 1st of October to get their contributions counted, now is still Preptember: The time to prepare your repositories.

Why you should participate

Open source lives from contribution, but contributing to repositories can be daunting. Hacktoberfest is a great opportunity to clean up your repository and make it as easy as possible for others to contribute.

It is a great chance to get people who just entered our field into open source and use this to maybe also teach about some best practices like well-written commit messages or test coverage.

Also, it might help you to get some more hands on features or bugs you just didn't get around to fix yet.

How to prepare your repo

What is officially needed

You can find all official requirements on the Hacktoberfest homepage.

Basically there is no need to do any special setup to be eligable for Hacktoberfest. Any PR to an open-source project will count for contributors towards their swag-goal of 4 PRs, as it isn't obviously playing the rules or gets invalidated by the repo owner with help of the invalid label.

Labels

It is recommended to mark issues with the right labels, though, so participants can discover them more easily. So far I have seen mainly four labels in use:

hacktoberfest is the label recommended by the Hacktoberfest team to mark any issue that is meant to be worked on during Hacktoberfest. Personally I've set the label color to #FF8AE2 which seems to be the main brand color used this year.
help wanted indicates that this issue is open to outsiders and not only to existing maintainers of your repo
good first issue shows that this issue is specifically meant for new contributors. Not all issues avaiable to outsiders fall in this category and you should put extra care into writing them to make it easy to get into your repo

Make your repository ready for contribution

When you have been working on your repository for a while, you have everything set up and are already familiar with all quirks. Neither is true for a fresh contributor.

The established way to communicate this information is via a CONTRIBUTING.md file in the repositories root directory. If you don't have one yet, now is a good opportunity to create one. Here's one of mine for inspiration.

But even if you already have contributing guidelines - are you sure they are up to date? Best to fully erase the repository from your disk, uninstall all special tools, and then follow your contributing guidelines yourself, until you have it running (including all steps required in your build pipeline like linting, building and testing). This will show you steps missing.

Create good first issues

Creating a "good first issue" is easy, but creating a good "good first issue" is hard. You have to put yourself in the shoes of someone who just discovered your repository and knows literally nothing about it. Your issue description needs to

Give them enough background to understand what your project tries to achieve,
explain in detail what the goal of this specific issue is,
show them a starting point where they can start their work, and
sell them to the idea of taking on this issue in particular.

Here's a couple of examples I wrote specifically for Hacktoberfest:

https://github.com/dbartholomae/jsx-readme/issues/7

https://github.com/dbartholomae/jsx-readme/issues/14

https://github.com/dbartholomae/jsx-md/issues/1

Additional idea : If you can take the time, then setting up a link where contributors can schedule a quick call with you for questions, or even 30 minutes of pair programming, might make it even easier for others to join in.

The legal stuff

Depending on your organization's policies you might have to also set up Contributor License Agreements. I'm currently using CLA assistant for this, but there are also other bots.

Promoting your repo

There already is a good guide on promoting open source, so in this article I will focus on Hacktoberfest specifically.

Add a hacktoberfest label to the issues you want to be found.
Post your project in the official Discord in the #issue-sharing channel.
Tweet about it including #Hacktoberfest

Have fun!

In the end, even though it is noble to make open source contribution easier to others, remember that by maintaining an open source repo you already volunteered your time to the cause of open source. It is admirable if you can do even more by making it easy for others to contribute, but don't beat yourself up over it if you don't find the time to do everything mentioned in the article.

The most important part is that you have fun with your work and do your best :)

Automatically generating README files with jsx-readme

Daniel Bartholomae — Thu, 24 Sep 2020 15:52:38 +0000

It's the first thing that greets you when you look at an npm package, be it on npm itself or in the source on GitHub: The README.md file. But if you are like me, then writing documentation never was your strong suite - let alone maintaining and updating it.

What if, instead of having to update the README file manually whenever your code changes, the README could just stay up-to-date auto-magically? What if instead of having to remember which badges you need to add, this would all be done for you?

Introducing jsx-readme

Here's an example of using jsx-readme:

// We need to tell the JSX transpiler that in this file,
// instead of React we use the custom createElement and Fragment
// functions from jsx-readme
/* @jsx MD */
/* @jsxFrag Fragment */
import type { Component } from "jsx-readme";
import MD, {
  BadgesFromPkg,
  CodecovBadge,
  DescriptionFromPkg,
  ExamplesFromPkg,
  Fragment,
  GithubWorkflowBadge,
  HomepageFromPkg,
  renderToFile,
  TitleFromPkg,
} from "jsx-readme";
import { Heading, InlineCode, LineBreak } from "jsx-md";
import pkg from "./package.json";

const Readme: Component = () => (
  <Fragment>
    {/* Create a header with title, badges and description inferred from package.json */}
    <TitleFromPkg pkg={pkg} />
    <BadgesFromPkg pkg={pkg} />
    {/* Add additional badges. */}
    <CodecovBadge pkg={pkg} />
    <GithubWorkflowBadge pkg={pkg} workflowName="Build and deploy" />
    <LineBreak />
    <DescriptionFromPkg pkg={pkg} />
    {/* You can use the components from jsx-md to build custom markdown. */}
    <Heading level={2}>Installation</Heading>
    Add <InlineCode>jsx-readme</InlineCode> to your{" "}
    <InlineCode>devDependencies</InlineCode> and install it. I recommend using
    it with <InlineCode>ts-node</InlineCode>. Then all you need to do is add a
    file like in the example below and run it via{" "}
    <InlineCode>ts-node</InlineCode> whenever you want to create a new version
    of the <InlineCode>README</InlineCode>.
    <LineBreak />
    <LineBreak />
    {/* Create an example section based on all files from the example directory set up in package.json */}
    <ExamplesFromPkg pkg={pkg} />
    {/* Create a section linking to the homepage from package.json */}
    <HomepageFromPkg pkg={pkg} />
  </Fragment>
);

void renderToFile("./README.md", <Readme />);

As you can see, jsx-readme uses a React-like JSX syntax to define Markdown. It is built on top of jsx-md which you can also use independently, or together with jsx-readme, depending on your needs.

Since JSX is basically JavaScript, we can do in this file whatever we want. In this simple case we just import the local package.json file to read from it, but why not e. g. query the GitHub API to figure out who contributed and then adding them to your Readme?

And since it is just a TypeScript file, you can simply run it via e. g. ts-node instead of having to learn yet another CLI tool.

So what does it look like?

The best way to understand the result of the script above is to just look at the package itself - jsx-readme uses it to create its own README.

Moist code - Why code should not be completely DRY

Daniel Bartholomae — Fri, 21 Aug 2020 13:08:00 +0000

At some point, each developer encounters the term “dry code”. It comes from the acronym DRY for “Don’t repeat yourself”. But if code is too dry, it easily can become brittle. To keep the code moldable, it is helpful to leave in a bit of repetition - a concept I personally like to call “Moist code”.

A simple example

The core idea behind “Don’t repeat yourself” is, as the name suggests, to not repeat code. Let’s look at a simple example in TypeScript React:

export function App () {
  return <>
    <nav><h2>Navigation</h2></nav>
    <main>This is the homepage</main>
  </>
}

So far, there isn't much repetition. Let's add a contact page:

export function App () {
  const location = useLocation()
  if (location === '/contact') {
    return <>
      <nav>
        <h2>Navigation</h2>
        <ul>
          <li><Link to='/'>Home</Link></li>
          <li><Link to='/contact'>Contact</Link></li>
        </ul>
      </nav>
      <main>
        This is the homepage
      </main>
    </>
  }

  return <>
    <nav>
      <h2>Navigation</h2>
      <ul>
        <li><Link to='/'>Home</Link></li>
        <li><Link to='/contact'>Contact</Link></li>
      </ul>
    </nav>
    <main>
      Contact us!
    </main>
  </>
}

Now we have a a bit of duplication. Time to split our paths and see the difference between dry and moist code.

Dry code

Let's remove as much repetition as possible:

export function App () {
  const location = useLocation()
  return <>
    <Navigation />
    <Router location={location} />
  </>
 }

function Navigation () {
  return (
    <nav>
      <h2>Navigation</h2>
      <ul>
        {
          [{
            children: 'Home',
            to: '/'
          }, {
            children: 'Contact',
            to: '/contact'
          }].map(props => <NavLink ...props key={to} />)
        }
     </ul>
    </nav>
  )
}

function NavLink ({ to, children }) {
  return <li><Link to={to}>{children}</Link></li>
}

function Router ({ location }) {
  return <main><Content location={location} /></main>
}

function Content ({ location }) {
  if (location === '/profile') {
    return 'Contact us!'
  }
  return 'This is the homepage'
}

We have removed every repetition we could find. Time to add the next feature. We want a login page, and users that are not logged in, should automatically see the login form instead of any other page:

export function App () {
  const location = useLocation()
  return <>
    <Navigation />
    <Router location={location} />
  </>
 }

function Navigation () {
  return (
    <nav>
      <h2>Navigation</h2>
      <ul>
        {
          [{
            children: 'Home',
            to: '/'
          }, {
            children: 'Contact',
            to: '/contact'
          }, {
            children: 'Login',
            to: '/login'
          }].map(props => <NavLink ...props key={to} />)
        }
     </ul>
    </nav>
  )
}

function NavLink ({ to, children }) {
  return <li><Link to={to}>{children}</Link></li>
}

function Router ({ location }) {
  return <main><Content location={location} /></main>
}

function Content ({ location }) {
  const isUserLoggedIn = useLoggedInState()
  if (!isUserLoggedIn || location === '/login') {
    return 'This is a login form'
  }
  if (location === '/profile') {
    return 'Contact us!'
  }
 return 'This is the homepage'
}

Thanks to the strict removal of duplication, we only had to add another link object in the NavBar and another if statement with new content in the Content component.

Our app is growing, and we have a UX team now. After some user testing they figured out that it is really confusing to see the Navigation on the login screen, even though seemingly none of the links work. So let's fix this by not showing any links in the NavBar on the login page:

export function App () {
  const location = useLocation()
  return <>
    <Navigation location={location} />
    <Router location={location} />
  </>
 }

function Navigation ({ location }) {
 return (
    <nav>
      <h2>Navigation</h2>
      { location !== '/login' && <NavLinks /> }
   </nav>
  )
}

function NavLinks () {
  return <ul>
    {
      [{
        children: 'Home',
        to: '/'
      }, {
        children: 'Contact',
        to: '/contact'
      }, {
        children: 'Login',
        to: '/login'
      }].map(props => <NavLink ...props key={to} />)
    }
  </ul>
}

function NavLink ({ to, children }) {
  return <li><Link to={to}>{children}</Link></li>
}

function Router ({ location }) {
  return <main><Content location={location} /></main>
}

function Content ({ location }) {
  const isUserLoggedIn = useLoggedInState()
  if (!isUserLoggedIn || location === '/login') {
    return 'This is a login form'
  }
  if (location === '/contact') {
    return 'Contact us!'
  }
 return 'This is the homepage'
}

Again, thanks to the small amount of duplication, the change is small and easy. But something starts to feel off about the code. Let's see what happens after the next feature.

Our marketing team just called. They had this great idea for a special easter promotion: They want to hide a special easter egg on the page, where if you click the login link on the contacts page, it will instead redirect you to the promotion website. It's just a change in a link, so how hard can it be?

export function App () {
  const location = useLocation()
  return <>
    <Navigation location={location} />
    <Router location={location} />
  </>
 }

function Navigation ({ location }) {
 return (
    <nav>
      <h2>Navigation</h2>
      { location !== '/login' && <NavLinks location={location} /> }
   </nav>
  )
}

function NavLinks ({ location }) {
  return <ul>
    {
      [{
        children: 'Home',
        to: '/'
      }, {
        children: 'Contact',
        to: '/contact'
      }, {
        children: 'Login',
        to: location === '/contact'
          ? 'https:///our-special-promotion.com'
          : '/login'
      }].map(props => <NavLink ...props />)
    }
  </ul>
}

function NavLink ({ to, children }) {
  return <li><Link to={to}>{children}</Link></li>
}

function Router ({ location }) {
  return <main><Content location={location} /></main>
}

function Content ({ location }) {
  const isUserLoggedIn = useLoggedInState()
  if (!isUserLoggedIn || location === '/login') {
    return 'This is a login form'
  }
  if (location === '/contact') {
    return 'Contact us!'
  }
 return 'This is the homepage'
}

Let's look at our moist code team, to see how they are doing.

Moist code

In moist space, we still have the starting code:

export function App () {
  const location = useLocation()
  if (location === '/contact') {
    return <>
      <nav>
        <h2>Navigation</h2>
        <ul>
          <li><Link to='/'>Home</Link></li>
          <li><Link to='/contact'>Contact</Link></li>
        </ul>
      </nav>
      <main>
        This is the homepage
      </main>
    </>
  }

  return <>
    <nav>
      <h2>Navigation</h2>
      <ul>
        <li><Link to='/'>Home</Link></li>
        <li><Link to='/contact'>Contact</Link></li>
      </ul>
    </nav>
    <main>
      Contact us!
    </main>
  </>
}

We don't mind a bit of repetition, so instead of refactoring right away, let's add the next feature first, the login page:

export function App () {
  const location = useLocation()
  const isUserLoggedIn = useLoggedInState()
  if (!isUserLoggedIn) {
    return <>
      <nav>
        <h2>Navigation</h2>
      </nav>
      <main>
        This is a login form
      </main>
    </>
  }

  if (location === '/login') {
    return <>
      <nav>
        <h2>Navigation</h2>
      </nav>
      <main>
        This is a login form
      </main>
    </>
  }

  if (location === '/contact') {
    return <>
      <nav>
        <h2>Navigation</h2>
        <ul>
          <li><Link to='/'>Home</Link></li>
          <li><Link to='/contact'>Contact</Link></li>
          <li><Link to='/login'>Login</Link></li>
        </ul>
      </nav>
      <main>
        Contact us!
      </main>
    </>
  }

  return <>
    <nav>
      <h2>Navigation</h2>
      <ul>
        <li><Link to='/'>Home</Link></li>
        <li><Link to='/contact'>Contact</Link></li>
        <li><Link to='/login'>Login</Link></li>
      </ul>
    </nav>
    <main>
      This is the homepage
    </main>
  </>
}

Let's extract some of the concepts that seem to emerge, but not too many:

export function App () {
  const location = useLocation()
  const isUserLoggedIn = useLoggedInState()
  if (!isUserLoggedIn) {
    return <LoginPage />
  }

  return <Router />
}

function Router ({ location }) {
  if (location === '/login') {
    return <LoginPage />
  }

  if (location === '/contact') {
    return <ContactPage />
  }

  return <HomePage />
}

function LoginPage () {
  return <>
    <nav>
      <h2>Navigation</h2>
      <ul>
        <li><Link to='/'>Home</Link></li>
        <li><Link to='/contact'>Contact</Link></li>
        <li><Link to='/login'>Login</Link></li>
      </ul>
   </nav>
    <main>
      This is a login form
    </main>
  </>
}

function ContactPage () {
  return <>
    <nav>
      <h2>Navigation</h2>
      <ul>
        <li><Link to='/'>Home</Link></li>
        <li><Link to='/contact'>Contact</Link></li>
        <li><Link to='/login'>Login</Link></li>
      </ul>
    </nav>
    <main>
      Contact us!
    </main>
  </>
}

function HomePage () {
  return <>
    <nav>
      <h2>Navigation</h2>
      <ul>
        <li><Link to='/'>Home</Link></li>
        <li><Link to='/contact'>Contact</Link></li>
        <li><Link to='/login'>Login</Link></li>
      </ul>
    </nav>
    <main>
      This is the homepage
    </main>
  </>
}

There is still some repetition left and our fingers might be itching to remove it, but let's add the next feature first: Not showing links on the login page:

export function App () {
  const location = useLocation()
  const isUserLoggedIn = useLoggedInState()
  if (!isUserLoggedIn) {
    return <LoginPage />
  }

  return <Router />
}

function Router ({ location }) {
  if (location === '/login') {
    return <LoginPage />
  }

  if (location === '/contact') {
    return <ContactPage />
  }

  return <HomePage />
}

function LoginPage () {
  return <>
    <nav>
      <h2>Navigation</h2>
   </nav>
    <main>
      This is a login form
    </main>
  </>
}

function ContactPage () {
  return <>
    <nav>
      <h2>Navigation</h2>
      <ul>
        <li><Link to='/'>Home</Link></li>
        <li><Link to='/contact'>Contact</Link></li>
        <li><Link to='/login'>Login</Link></li>
      </ul>
    </nav>
    <main>
      Contact us!
    </main>
  </>
}

function HomePage () {
  return <>
    <nav>
      <h2>Navigation</h2>
      <ul>
        <li><Link to='/'>Home</Link></li>
        <li><Link to='/contact'>Contact</Link></li>
        <li><Link to='/login'>Login</Link></li>
      </ul>
    </nav>
    <main>
      This is the homepage
    </main>
  </>
}

That was easy. We literally just had to delete a couple lines. With our new knowledge about how our Navigation can change, let's now extract it:

export function App () {
  const location = useLocation()
  const isUserLoggedIn = useLoggedInState()
  if (!isUserLoggedIn) {
    return <LoginPage />
  }

  return <Router />
}

function Router ({ location }) {
  if (location === '/login') {
    return <LoginPage />
  }

  if (location === '/contact') {
    return <ContactPage />
  }

  return <HomePage />
}

function LoginPage () {
  return <>
    <Navigation />
    <main>
      This is a login form
    </main>
  </>
}

function ContactPage () {
  return <>
    <Navigation
      links={[{
        children='Home'
        to='/'
      }, {
        children='Contact'
        to='/contact'
      }, {
        children='Login'
        to='/login'
      }]}
    />
    <main>
      Contact us!
    </main>
  </>
}

function HomePage () {
  return <>
    <Navigation
      links={[{
        children='Home'
        to='/'
      }, {
        children='Contact'
        to='/contact'
      }, {
        children='Login'
        to='/login'
      }]}
    />
    <main>
      This is the homepage
    </main>
  </>
}

function Navigation ({ links }) {
  return (
    <nav>
      <h2>Navigation</h2>
      { links &&
        <ul>
          {
            links.map({ children, to } =>
              <li key={to}><Link to={to}>{children}</Link></li>)
          }
       </ul>
      }
    </nav>
  )
}

Even though there is a lot more repetition in this code then in the last version (e. g. each page having to explicitly use main), it already feels a bit cleaner and more lightweight. Let's see what happens with the marketing request to have a special link for the easter egg hunt:

export function App () {
  const location = useLocation()
  const isUserLoggedIn = useLoggedInState()
  if (!isUserLoggedIn) {
    return <LoginPage />
  }

  return <Router />
}

function Router ({ location }) {
  if (location === '/login') {
    return <LoginPage />
  }

  if (location === '/contact') {
    return <ContactPage />
  }

  return <HomePage />
}

function LoginPage () {
  return <>
    <Navigation />
    <main>
      This is a login form
    </main>
  </>
}

function ContactPage () {
  return <>
    <Navigation
      links={[{
        children='Home'
        to='/'
      }, {
        children='Contact'
        to='/contact'
      }, {
        children='Login'
        to='https:///our-special-promotion.com'
      }]}
    />
    <main>
      Contact us!
    </main>
  </>
}

function HomePage () {
  return <>
    <Navigation
      links={[{
        children='Home'
        to='/'
      }, {
        children='Contact'
        to='/contact'
      }, {
        children='Login'
        to='/login'
      }]}
    />
    <main>
      This is the homepage
    </main>
  </>
}

function Navigation ({ links }) {
  return (
    <nav>
      <h2>Navigation</h2>
      { links &&
        <ul>
          {
            links.map({ children, to } =>
              <li key={to}><Link to={to}>{children}</Link></li>)
          }
       </ul>
      }
    </nav>
  )
}

One link replaced. That's it.

What happened?

The main difference between our two approaches is that with the moist code approach, we gave the code more time to dry. Instead of forcing it into a specific form early on based on assumptions only, we first collected a bit more requirements and only then started removing repetition.

This lead us to use the pattern of composition instead of inheritance for the Navigation, which made it easier to individualize the Navigation by route and gave us more flexibility for changing the code in the places that actually mattered for our new features.

Conclusions

When writing code we usually don't know from the beginning every requirement that will come ahead. Especially for web applications the requirements might never fully settled and always keep in motion. If we optimize our code only step-by-step as we discover the requirements, we can keep it moist and malleable. If we aim for perfectly DRY code, it will become brittle and inflexible to change.

DEV Community: Daniel Bartholomae

Easily identify technical debt in any repository

What is technical debt?

So what about accidental complexity?

So how to fix technical debt?

Interlude: What are GitHub blocks?

The technical debt block

Conclusion

Rewriting my blog from scratch with NextJS

Overview

Setting up the blog in NextJS

Adding an RSS feed

Hosting on Amplify

Starting with analytics

Moving to a monorepo

Using OIDC to work with AWS

Publishing to dev.to

The current structure

What's next

How to read any startup's source code

Finding their apps

Analyzing their infrastructure

Reading their source code

Why take-home assignments might not be as bad as their reputation

Why I reject most take-home assignments

And why I still use them in recruiting

Right candidate motivation

Relevant task

Relevant work environment

Small scope

How this fits in the overall recruiting process

10 bad TypeScript habits to break this year

1. Not using strict mode

What it looks like

What it should look like

Why we do it

Why we shouldn't

2. Defining default values with ||

What it looks like

What it should look like

Why we do it

Why we shouldn't

3. Using any as type

What it looks like

What it should look like

Why we do it

Why we shouldn't

4. val as SomeType

What it looks like

What it should look like

Why we do it

Why we shouldn't

5. as any in tests

What it looks like

What it should look like

Why we do it

Why we shouldn't

6. Optional properties

What it looks like

What it should look like

Why we do it

Why we shouldn't

7. One letter generics

What it looks like

What it should look like

Why we do it

Why we shouldn't

8. Non-boolean boolean checks

What it looks like

What it should look like

Why we do it

Why we shouldn't

9. The Bang Bang operator

What it looks like

What it should look like

Why we do it

Why we shouldn't

10. != null

What it looks like

What it should look like

1. Not using `strict` mode

2. Defining default values with `||`

3. Using `any` as type

4. `val as SomeType`

5. `as any` in tests

10. `!= null`