DEV Community: Stephen Mizell

Building Quality APIs by Managing Complexity as We Go

Stephen Mizell — Tue, 05 Oct 2021 13:22:50 +0000

Quality APIs don't happen on their own. We know this from experience. Bad code doesn't get better after we run git commit, and poor design choices don't improve while we're out to lunch. So if we want to design and build better APIs, we have to be intentional about how we do it.

We've all used poor-quality APIs. They're hard to use. They're difficult to understand. We had to spend extra time digging through documentation and piecing together clues to find the data we needed. If an experience is bad enough, we went looking for a better API. We want to try to avoid this for our APIs if we can.

I recently released an ebook called API by Design that's about building quality APIs. Actually, it's about the opposite of quality—complexity—and how, if we're ever going to improve the way we design APIs, we're going to have to find a way to manage complexity as we go. Otherwise, we're left to hoping that our decisions won't lead to complex, hard-to-use APIs.

If complexity gets bad enough to frustrate us or slow us down, we've waited too long to deal with it. Maybe we cut a few corners as we went or took some shortcuts in the beginning, thinking we'd come back and clean things up. But we often get busy and never find the time. Those tiny choices we make to increase tech debt are high-interest loans. They grow and compound over time until they put us over our heads in debt. Eventually, it becomes easier to rewrite the code than to change what's there.

This is especially true for APIs because APIs are good at hiding complexity. We can't change them however we want because we don't want to break existing clients. So we have to be cautious with how we evolve them. We have to work around what's there, adding and never removing. Without a plan to deal with the accumulation of features, we risk creating a poor user experience.

We need ways to manage complexity as we design and develop our APIs. We need to know how much complexity we add with every API change.

If we can measure and catch complexity early in the development process, we can design and build quality APIs. Managing complexity goes together with managing quality.

Amplifying Others

Stephen Mizell — Thu, 18 Apr 2019 21:22:54 +0000

I want to share a way that I elevate others and give visibility to those who may not be visible. It's proven to be a way for me to build trust with others that I work with and to make sure others are seen and heard.

I try to use "we" when talking about team accomplishments. Even if I did the bulk of the work or the most visible work, I try to put the group in the spotlight.
I try to use the person's name when talking about a teammate's work. Rather than use "we," I directly give that person credit for their work and ideas.
I try to use "I" when taking responsibility. If something fell through the cracks that was my fault, or if something didn't go as we planned, I put it on my back.

It usually works well, and it sometimes feels contagious. Others will often amplify each other’s work or ideas the same way. I feel it lets people share without the fear of someone else taking the credit. This is healthy for a team.

I must add a disclaimer. I mention these tips from a place of privilege. I can give away visibility or credit and be fine. This may not be true for underrepresented groups or people with low visibility. They may have to use "I" where I try to use "we." It's on me to recognize that and amplify their work and voice even more.

Using GraphQL With REST APIs

Stephen Mizell — Wed, 20 Mar 2019 11:00:00 +0000

People choose GraphQL because of all the details it lets them forget about. It's tough to go back to modern REST approaches after working with the powerful patterns and well-built tools of the GraphQL ecosystem. But maybe we don't have to choose between GraphQL and REST. Maybe we can get the benefits of both approaches.

To explore this, I've worked on a proof of concept. I called it Graphable JSON. It's a JavaScript library that takes a GraphQL query and lazily fetches all the requested data from a REST API by following links. It allows API consumers to move beyond resources, URLs, HTTP methods, and status codes by specifying their required data in GraphQL's query syntax. All of this happens in the client rather than sending the query to an API somewhere. This is an important distinction to remember.

An example of how it works

I feel it's easiest to explain this approach by way of examples. We'll use an order API for these examples, and we'll write a GraphQL query to fetch data from it. The API is a REST API with no GraphQL endpoint, so the Graphable JSON library will have to figure out how to get the data on its own.

The API is made up of several different examples for representing order data. For each example, our GraphQL query will be the same. This will show that we can have different kinds of responses and representations of data and evolve the API without breaking the client. We'll also see how client developers can specify the data they want without worrying about the details of the REST API.

Here's our GraphQL query to fetch order data.

{
  order {
    order_number
    total
  }
}

No orders in the response

We'll start with the first example which is an empty object with no order data, {}.

Since there are no orders, the result for the query will be:

{ "order": [] }

Graphable JSON will handle null the same way, meaning we'll never have to worry about null and undefined values. The shape of the data will always be the same.

A single order in the response

The second example has a single order. The library will first look for a single value and yield that value if it exists.

{
  "comment": "A single embedded order object",
  "url": "https://graphablejsonapi.glitch.me/examples/example1",
  "order": {
    "url": "https://graphablejsonapi.glitch.me/orders/1000",
    "order_number": "1000",
    "total": 150,
    "unit": "USD"
  }
}

When we run our query, we get just the data we requested.

{
  "order": [
    {
      "order_number": "1000",
      "total": 150,
    }
  ]
}

Even though there was a single order, we see an array of order items in the result. There are a couple of things to mention about this before moving on.

First, the Graphable JSON library makes use of asynchronous generators. Without getting too far into those details, it means that it will return a function that can yield many values rather than returning a single value. This little piece of magic lets us treat zero values, single values, and—as we'll see in the next sections—multiple values the same way.

Second, it may feel odd to have the singular word order referring to an array. It's best to think of each property as describing a relationship between the given object and the value or values of the object. The object returned can have zero or many order items, though there is only one concept of an order relationship in this context. Treating relationships this way let's us evolve the API and not break clients when we need to expand the number of related items.

The goal is to return the same shape of the data in the client even though the API may change how that data is represented. Separating Graphable JSON's response from the representation of the API's data is the key to making this idea work.

Multiple orders in the response

I've shortened the third example for the case of brevity. It shows multiple orders in a single response.

{
  "comment": "All order objects embedded",
  "url": "https://graphablejsonapi.glitch.me/examples/example2",
  "order": [
    {
      "url": "https://graphablejsonapi.glitch.me/orders/1000",
      "order_number": "1000",
      "total": 150,
      "unit": "USD"
    },
    {
      "url": "https://graphablejsonapi.glitch.me/orders/1001",
      "order_number": "1001",
      "total": 25,
      "unit": "USD"
    },
  ]
}

As we would expect, we get the same shape as above, only with more orders.

{
  "order": [
    {
      "order_number": "1000",
      "total": 150,
    },
    {
      "order_number": "1001",
      "total": 25,
    }
  ]
}

Linked orders

What happens when we want to give each order its own URL? How can we provide ways to cache all or parts of the data from the API? The way we do that is by adding links in the responses and letting the Graphable JSON client follow those links. Consider our "single order" example above. Instead of embedding the order, we can link it with an order_url property.

{
  "comment": "Links to each order rather than embedding",
  "url": "https://graphablejsonapi.glitch.me/examples/example3",
  "order_url": "https://graphablejsonapi.glitch.me/orders/1000"
}

Graphable JSON would return the same response as the "single order" example. Here's how it works. The library first looks for order. If it's not found, it looks for either order_url or orderUrl. If found, it requests the link. In both cases, the result of the query is the same. This works for multiple links as well.

{
  "comment": "Links to each order rather than embedding",
  "url": "https://graphablejsonapi.glitch.me/examples/example3",
  "order_url": [
    "https://graphablejsonapi.glitch.me/orders/1000",
    "https://graphablejsonapi.glitch.me/orders/1001",
    "https://graphablejsonapi.glitch.me/orders/1002",
    "https://graphablejsonapi.glitch.me/orders/1003"
  ]
}

The library will request each order link and provide the response as if it were included.

Collections of orders

Lastly, there are times where we'll want to paginate data in the API. Most GraphQL approaches require the consumer to deal with pagination in the query. Graphable JSON does pagination by looking for collections of data and following links if they exist—it happens automatically.

In our example, if the order value contains an $item property, it will respond with each item and follow a next link if it exists. Here is a response with a link to an order, but this time it's to an order collection.

{
  "comment": "Links to a paginated order collection",
  "url": "https://graphablejsonapi.glitch.me/examples/example4",
  "order_url": "https://graphablejsonapi.glitch.me/ordersCollection"
}

The linked order collection looks something liket his below.

{
  "$item": [
    {
      "url": "https://graphablejsonapi.glitch.me/orders/1000",
      "order_number": "1000",
      "total": 150,
      "unit": "USD"
    }
  ],
  "next_url": "https://graphablejsonapi.glitch.me/ordersCollection?page=2"
}

The library will return all of the items then move on to the next page until there were no longer any next links. Like all the examples above, this results in the same shape of the data, all using the same query. All of this is hidden from the client developers.

The benefits of this approach

Though Graphable JSON doesn't solve the problem of overfetching and underfetching, it does provide other benefits that may outweigh that issue.

API designers can think about relationships of data rather than strict structures of data.
API developers can evolve the API based on technical needs without breaking client implementations.
API developers can optimize responses based on client needs.
API consumers don't have to piece together resources to understand the interconnectedness of the API—the links show the way.
Any data in the API can be given a URL, making it possible to refresh smaller batches of data later without requesting larger queries.
Clients are able to cache data based on cache settings in the response. For the browser, this implementation comes for free.

Graphable JSON is just a proof of concept. But maybe it will spark ideas for others. There are many benefits to REST, and we lose some of these benefits as we move to things like GraphQL. Maybe there are ways to get the best of both worlds.

Application-Driven API Design

Stephen Mizell — Thu, 14 Mar 2019 05:09:52 +0000

When designing APIs, it's important to take a design-first approach to ensure the right API gets built. However, I don't think the first step should be to write an OpenAPI or API Blueprint document. I think it should be to design a working application.

To take this application-driven approach to API design, API designers should start the API design process by creating an application that allows them to explore the domain and try out the design all without considering data storage, web interactions, or architecture. It produces a better design and is a better step for beginners to take.

An example of this approach

Let's say we've been asked to design an API for managing tasks. We'll keep it basic for now. We'll say requirements are to provide a way to manage tasks that have a description and a way to mark them as complete. We'll need to filter the tasks on the their status. Instead of starting with an API description for this, we'll first write a tiny application in Python—any language would do.

from dataclasses import dataclass

task_storage = []

@dataclass
class Task:
    description: str
    is_complete: bool = False

def mark_task_complete(task):
    task.is_done = True

def mark_task_incomplete(task):
    task.is_complete = False

def store_task(task):
    task_storage.append(task)

def all_tasks():
    return task_storage

def completed_tasks():
    return [task for task in task_storage if task.is_complete]

def incomplete_tasks():
    return [task for task in task_storage if not task.is_complete]

At this point, we could drop into a REPL and explore the design. Does it feel right? Is our vocabulary correct? Are we missing some functionality? Through this we'll see operations that are safe and others that alter state and are unsafe. This is all useful knowledge that can drive the API design.

Also look at all the things we're not doing in this step. We're not installing mock servers, JSON schema validators, or documentation tools. We're not forcing ourselves to think in the context of URLs and HTTP. We can push these details to later because—while important—they come second to the application design.

How this affects beginners

Our code above would be simple enough to talk through with people completely new to programming. We would be able to discuss the problem and the domain with beginners and not have to start with details related to building an HTTP API. Once we have a good feel for our application, we can start asking the questions about how we might persist the data or make the functionality available over the web. It's a much better starting place than URLs and HTTP requests.

I have a hunch that experienced API designers do this application design in their head as they write API definitions. They've been through the process and used APIs enough to know what works and what doesn't. However, I've found there is a lot to gain by writing applications first to drive the API design.

This is nothing new

This process for writing code to explore a domain isn't new. There is a great presentation called Domain Modeling Made Functional where the presenter walks through how he uses F# and static types to define the domain. He concludes, "The design is the code and the code is the design." His point is that you can capture the domain in code and use the tools around F# to explore the design.

We could do the same with API design if our first step is to define and explore the domain. If we can do this in code, we can give our application a test drive before ever thinking about the API specifics.

API Design and Beginners

Stephen Mizell — Wed, 13 Mar 2019 17:45:36 +0000

I've been working through How to Design Programs recently. It's interesting to see how they walk people—especially newcomers—through the process of learning how to design programs. It got me thinking about how newcomers experience the world of API design.

Design-first and beginners

Many agree the design-first approach to API design is preferred—including experts and API design companies alike. The idea is to write an API definition like OpenAPI before writing a line of code. This stands in contrast to a code-first approach where API definitions are generated later from code. And though design-first is preferred, code-first is probably most common.

Consider what a beginner would need for a design-first approach.

Experience and knowledge for designing applications (more on this later)
An understanding of common URL patterns used in the industry
A firm grasp on how to use HTTP methods and status codes along with their many nuances
Knowledge of designing JSON documents through JSON schema
Familiarity with formats like OpenAPI or API Blueprint
Exposure to tooling for these formats

Beginners have to gain experience, knowledge, and understanding in API design before they can do design-first well. Good practices are spread across books, websites, and videos with varying degrees of consensus among practitioners on what truly is good. This takes lots of time and patience.

A quick search of the web for "how to design an API" will give recommendations like using plural names in URLs and HTTP methods as defined, but these fail to give direction for good design. It's up to the beginner to sift through the noise.

Pet-store-first driven API design

I've been working with APIs for several years now, and I'm sorry to admit I still have to copy and paste the Pet Store example to start writing an OpenAPI document. The same is true for me and API Blueprint's Polls example. I still need some visible guidance on both the syntax but also what is possible to define for the API.

But there is something deeper here than syntax. While these formats are great, they don't prescribe a way to design an API. You can design a wonderful API or a horrible API with any format.

This feels similar to the way How to Design Programs considers program design more important than programming. We can't teach newcomers things like variables, functions, loops, classes, and so on and expect them to be able to design a good program. Rather, we should teach methods for coming up with a good design.

Application-first driven API design instead

We talk about writing API definitions first, but I don't think it's possible to design a good API without understanding how to design a good application (an API is a programming interface to an application after all). Conversely, a well-design application can be used to drive good API design. If this is true, beginners should begin learning how to design applications and only then learn how to make those applications work across the web by way of an interface. Design the application first, then write the API definition.

It doesn't stop for beginners. This isn't something you move on from once you get good at it. This should be true for experts as well. Experts should be able to sit with domain experts and users and walk everyone through the design of an application before ever considering what the API looks like.

Everyone can design APIs

The authors claim in the preface of How to Design Programs that "everyone can design programs." They say the skills from program design can transcend industries. Good program design will shape a person's approach to any problems by giving them a systematic process for exploring the design. It makes me think the API design community could borrow these principles and practices to help lower the barrier of entry for newcomers.

If you want to see a good explanation of this approach, check out the video How to teach programming to loved ones. He goes through some of the reasoning behind how they teach programming design and then shows a demo of it in practice.

Why GraphQL Thinks People Should Like GraphQL

Stephen Mizell — Mon, 18 Feb 2019 08:26:20 +0000

I recently wrote that I believe people choose GraphQL because of the way it addresses common frustrations with web APIs. I'd like to look at this from the perspective of GraphQL to see why they think people should like GraphQL by looking at their marketing site. Though they mention technical reasons throughout the page, they're clear about the pain points they try to resolve.

When are there moments of frustration?

There is a moment in time right before a person decides to look for a solution where a frustration or need drives them to action. Good marketing will take people to these moments, tell a compelling story, and offer relief. Basecamp is a company that does this well on their landing page.

All growing businesses run into the same fundamental problems. Hair on fire, buried under email, stuff everywhere. The good news? Basecamp solves them.

Instead of focusing on their product, Basecamp focuses on the frustrations people feel when dealing notifications coming from all directions at work. They want to help create a calmer work experience, and they do so by providing a project management tool. These moments of frustration tell us why they think people should like Basecamp.

The frustrations addressed by GraphQL

The GraphQL landing page also takes this approach, though it's not as explicit. If we dig into what they are saying, we'll find the frustrating moments they feel GraphQL addresses. We'll look at the main points on the GraphQL landing page and consider what a person might be feeling before choosing GraphQL.

"Why can't I have more control over the API?"

The site says, "Ask for what you need, get exactly that." Client developers may be taken back to times when they had to sift through payloads looking for the data they needed. Though GraphQL results in smaller initial responses, I'm not sure this is the driving factor for most. People know what data they need. Why should they deal with more? Why do they have to go through the extra work when they could use GraphQL and get what they ask for?

"I can't figure out how this data is related."

"Get many resources in a single request," they say. But we've had solutions for ages that do this. Why is GraphQL different? They give us a hint.

GraphQL queries access not just the properties of one resource but also smoothly follow references between them.

GraphQL tries to expose the interconnectedness of the data. For many web APIs, it's a difficult task to understand how all of resources are related, but with GraphQL, it's one of the primary ways you interact with and understand the data.

"It's difficult to write usable and helpful docs."

When they say, "Describe what’s possible with a type system," they are making a statement about what wasn't possible without types. You might have thought while reading API docs, "I read it all, and I still don't know what I can do with this API." The type system gives developers direction and becomes it's own documentation.

"My tools require me to do a lot."

They say, "Move faster with powerful developer tools." They want to have tools that promote speed and agility in development while being simple to use. Instead of jumping between docs and tools to piece together multiple requests, GraphiQL lets you navigate the docs and build requests in the same tool. Most people didn't realize how badly they wanted this until they saw it.

"It's hard to reliably make changes to my API."

It takes work to evolve an API. Most people reach for versioning as a way to manage API changes. It's one of the first discussions people have when starting a new API—"How are we going to version our API? In the URL? Headers? Are the versions global?" The GraphQL site addresses versioning in their best practices:

When there's limited control over the data that's returned from an API endpoint, any change can be considered a breaking change, and breaking changes require a new version.

In their landing page they say, "Evolve your API without versions." They want you to forget everything you know about versioning. Can you evolve REST APIs without versions? Sure. But it's still uncommon to see, so much so that it's a main point of their marketing.

A recipe for traditional web APIs

If you're building tooling for traditional web APIs, I feel you could focus on these same frustrations and address the same users who are switching to GraphQL. I'd love to see people create tools and practices to address these issues within the REST paradigm—bonus points for working with common API best practices.

There are many tradeoffs in the GraphQL vs. REST discussion that need to be worked through (read more here and here). Maybe we can find ways get the benefits of REST with the ease and simplicity of GraphQL.

Why People Like GraphQL

Stephen Mizell — Fri, 15 Feb 2019 18:54:37 +0000

People like GraphQL, but I'm unconvinced it's primarily for technical reasons. I think it's something more human-related. People choose tools because they resolve some underlying frustration, and the current implementations of GraphQL solve some of the bigger frustrations people feel working with APIs.

The beginning of the web

Tim Berners-Lee provides an answer to the question "What made you think of the WWW?":

Well, I found it frustrating that in those days, there was different information on different computers, but you had to log on to different computers to get at it. Also, sometimes you had to learn a different program on each computer. So finding out how things worked was really difficult. Often it was just easier to go and ask people when they were having coffee.

It's interesting—though not surprising—the web was born out of frustration. Berners-Lee was tired of writing software to interact with so many different systems. "Can't we convert every information system so that it looks like part of some imaginary information system which everyone can read?" he asked. He says this is what became the WWW.

GitHub's reason for GraphQL

There have been lots of writing about technical reasons to choose it, but I believe those are secondary to the frustrations people feel working with APIs. Look at why GitHub moved to GraphQL.

[We] heard from integrators that our REST API also wasn’t very flexible. It sometimes required two or three separate calls to assemble a complete view of a resource. It seemed like our responses simultaneously sent too much data and didn’t include data that consumers needed.

The technical issue described here is what's called "overfetching" or "underfetching." But if you read into the quote, you can almost hear their customers saying, "I know what I want. Why do I have to do all this work to get it?" Or better yet, "Can't you convert all of your API resources so that they look like part of some imaginary information system which everyone can read?"

Each web API today feels different. They all have their own SDK, their own client, their own generated tools, and their own nuances for exposing data over HTTP. Even within them, it can feel repetitive requesting a list of resources to solve common goals. GraphQL tries to resolve this, and GitHub saw this.

GraphQL unifies a fragmented world

GraphQL lets people think about data and how it's related. It saves developers from piecing together a list of resources to understand how to get related data. It allows consumers to forget about the nuances of HTTP methods. People ask for what they want and that's what they get. This comes with the cost of losing out on some of HTTP's time-tested superpowers, but for most people, it's worth the cost.

The next generation of web APIs

I hope the next step for web APIs will be to follow these steps and solve some of these frustrations. I think that step will be more pragmatic and more human-centered than it will be technically superior.

Boundaries are the Key to Working Remotely

Stephen Mizell — Mon, 28 Jan 2019 16:52:08 +0000

I've been working remotely for over four years now, and I've had successes and failures in making it a good fit. Overall it's been a great experience, and I'll share what I feel is the key to remote work—setting up good boundaries.

When I talk to people about working remotely, their response is on the spectrum from "I would love to do that" to "I would never get any work done." People are interested in the freedoms that remote work can give, but they are worried they won't be productive or able to keep a healthy work-life balance.

Good boundaries are the key. Good boundaries are about creating a space to work so you can focus. Good boundaries are about defining a time when you'll work and when you won't so you won't work all the time. Good boundaries are about being honest about the nature of remote work so your coworkers and you can work well together.

Set up a place to work and only work

If I could only give one piece of advice, it would be to try to designate a place in your home for work and make it the only place you work. Create a workspace separate from your living space if possible, because I've found it's too easy to work on the couch or at the kitchen table. That leads to bad habits such as working when you don't want to work. Your workspace can be an entire room or a desk somewhere away from everyone. Whatever you do, try to find a workspace where you can focus.

Also try not to do personal tasks in your workspace. Though it makes it harder to set aside space only for work, you'll find it helps make the work-life separation. I found it was too easy to get drawn into doing work tasks even if I was doing personal tasks in the workspace.

For me, I didn't do well keeping my work in the workspace in the beginning, and it made it tough. I had a separate space, but it wasn't comfortable and I ended up working in other areas around the house. This made productivity difficult. Things got better once I improved the workspace and got use to spending my time in the workspace.

As a note, I'm not saying you should never leave your workspace. I'm rather suggesting you to not mix living and working spaces. Definitely take breaks to see your family, go for a walk to think, grab lunch with a friend, or find a coffee shop you enjoy to work in. I've found I need those times away because otherwise I wouldn't leave the house.

When you're at work, you're at work

The next key for a good work-life balance is to have designated times for working. This is tough to do working from home, and even tougher if you can't create a separate space. But if you can get the workspace boundary and time boundary figured out, you'll be in great shape.

For one, it's important to think of going into your work area as going to work. If you've worked in a physical office, you got up at a certain time, got dressed and ready, went to the office, worked for a bit, and then went home. Try to continue this pattern. Even if no one will see you while working, get prepared for the day—it helps create a separation and helps with self-care. When I wait until later in the day to get ready, I never get that feeling like I started work. The whole morning blurs together.

When you're in your workspace, you're working. Your family and friends have to treat you that way. You aren't available for chores, errands, or helping someone out. It's easy for people to treat you as available because you are physically around more, but you'll have to work to set this boundary. To everyone but your coworkers, you're unavailable.

My kids and I had to learn this boundary together over time. Early on, they'd barge in my office to say hello, which I loved, but it would break focus and many times interrupt meetings. They now know I'm working, and they'll quietly slide notes under my door. That works a lot better and is always a fun surprise.

There are exceptions where you need to leave work for a personal errand. If you have a regular task like picking up your kids everyday during work hours, talk to your manager first. They will probably be understanding, and if you've already built up the trust that you get work done working remotely, your manager shouldn't have a problem with it.

When you're not working, you're not working

When you're off work and you've left your space, the opposite of above is true—your coworkers should treat you as unavailable. The best step to take to establish this is to be strict about your working time. You shouldn't be expected to work more than 40 hours a week, and if you are, it should be an exception. Define a start time and finish time for your work and stick to it, otherwise you'll have trouble stopping work for the day. Your coworkers will catch on to your pattern and will have it in mind while they work. But it also requires that you respect their personal time and expect they finish at a certain time of the day. Trust works both ways.

Most calendar tools can help with start and stop times. For instance, Outlook lets you set working hours. I also block off time for lunch because it shows up as busy when people try to schedule meetings. The scheduled break lets you get away from lunch so you don't have to eat at your desk.

Block out your schedule to get work done

Video calls are the primary way people talk remotely. They works well, but the meetings can take up a big portion of your day if you have lots them. If you work at a place where anyone can drop meetings onto your calendar, block out time to get some things done if your work is piling up. People will understand and will be happy to schedule if it's not urgent.

Use your work computer only for work

If you get a nice computer from your employer, it'll be tempting to use it for personal tasks. Try to resist. If you are doing personal tasks on your computer after hours, work will be in your focus the entire time. Emails, notification, and opened documents will be trying to steal you away from your personal time. Even a quick glance at email can get your mind thinking about work.

The answer for me was to get a personal computer for personal tasks only. I don't use it in my workspace, and I don't do anything work-related on it. It's an extra expense, but it helps separate work and life.

Just to note, there are some legal issues you don't have to worry about if you keep a personal machine. If you build something of value on your work machine, there is risk that your employer can claim some ownership of your work. You may not fear this with your employer. However, I think it's safer to keep things separate if you can. Talk to a lawyer about this before starting remote work.

Keep work apps off your phone if you can

I struggle to keep work email or chat apps off my phone, but if you can, try not to add them. It makes it too easy to reach for your phone as soon as you wake up in the morning, putting your mind into work mode. But I've had the need to check my email or calendar when I'm out and about, so I always end up installing them. The answer here may be to practice discipline around when you look at work stuff rather than completely removing it. There is always balance.

Keeping work-related apps on your phone can easily break your boundaries without you noticing it. You might find yourself working on the couch after hours by picking up your phone and opening your email. The phone gives you a false sense that you're not working—you're just checking in. If you do it, your family will change their expectations and your coworkers will start to think you work at different hours. Avoid the phone trap if you can.

Success is dependent on others

These boundaries are ones you set for yourself. For success though, it requires others to respect them. Of course, you will also need to work to respect their boundaries and build trust over time. You'll have to remind people of the boundaries you've created. You can say small things like, "Would you mind if we move the meeting? It's during my lunch," and people will understand. Even refraining from working after hours will help give people a sense of your boundaries. This to me is key to developing a remote-first or remote-friendly culture.

But what if you can't set these boundaries?

You may not be able to create a separate space for your work. You may not have the funds to buy a personal computer. I think you can still be successful working remotely, but it may take more discipline. You may have to do more mental separation for your work than physical separation. Either way, keep healthy boundaries in mind.

You also might not be able to work 40 hours a week as the norm. If you are constantly asked to work beyond a normal work week and you have a good relationship with your manager, talk to them about it. Ask for their help in creating a better work-life separation. Good managers are always eager to help their employees be successful.

I recommend the experience

For me, working remotely has been great. I don't have a commute, I get to see my family in the morning, I get to live in a small town that is calm and quiet, and I get to work with great people all over the world. I've been fortunate to have people in my life who help me with these boundaries—especially my wonderful wife—and I have worked with and for people who make the work a lot easier. If you have the opportunity and the means to make it happen, you should give it a shot.

Test Coverage is a Lie

Stephen Mizell — Mon, 14 Jan 2019 19:07:24 +0000

I have some bad news about test coverage—it doesn’t tell us how much of our code is covered by tests, even though the name says it does. It doesn’t help us improve the quality of our software. It doesn’t help us find bugs. It feels like it’s a lie. And maybe it is.

What is Test Coverage?

Wikipedia has an underwhelming—though in my opinion correct—definition of test coverage.

In computer science, test coverage is a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs.

As its defined here, test coverage does not tell us which lines in the code have been tested. It does not tell us whether all possible conditions have been tested. It only tells which lines of code were executed during a test run.

The definition makes no mention of quality, though it does go on to talk about bugs.

A program with high test coverage, measured as a percentage, has had more of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs compared to a program with low test coverage.

This idea presented is that more coverage correlates with fewer bugs. Correct or not, this is not the same as using coverage as a tool to find existing bugs or preventing bugs from making it to production. It only tells us there is a lower chance of bugs because of the test coverage.

The Reason Test Coverage Lies

Let’s look at a short example to see how test coverage gives us a false sense of how well-tested our code is. Consider the code below. It has four branches to test.

// This function always returns 12
// ...or does it?
exports.alwaysTwelve = function(a, b) {
  let y, z;

  if (a === true) y = 4;
  else y = 6;

  if (b === true) z = 3;
  else z = 2;

  return y * z;
}

And here are some tests for it.

const assert = require('assert');
const { alwaysTwelve } = require('.');

describe('Always Twelve', function () {
  context('when true', function () {
    it('returns 12', function () {
      const value = alwaysTwelve(true, true);
      assert.equal(value, 12);
    });
  });

  context('when false', function () {
    it('returns 12', function () {
      const value = alwaysTwelve(false, false);
      assert.equal(value, 12);
    });
  });
});

The code and tests above result in 100% test coverage because the two tests are enough to cover the four branches in the code. However, they are not enough to cover all possible paths through the application code. They don’t help us catch an obvious bug in the logic of our code.

We can calculate the number of tests we need to execute all the possible paths through the code. The possible paths is the result of taking 2ⁿ where n is the number of conditions in the code. Therefore, in our code above with two conditions, we need four tests, not two. As we have it, our tests only test the inputs:

true and true (resulting in 12)
false and false (resulting in 12)

However, they should also test the inputs:

true and false (resulting in 8)
false and true (resulting in 18)

If we had tested these conditions, we would have found our tests did not completely test our logic, resulting in a bug. Sadly, with passing tests and 100% coverage, we may have shipped this code and had to find these bugs later in production. That’s no fun.

I've created a repository showing this example which you can checkout and try. You'll see the tests passing with 100% coverage.

Code Executed Isn’t Code Covered

If test coverage only tells us what code is executed, it means we could execute code and never test it. Let’s convert our test suite from above to show this scenario.

const assert = require('assert');
const { alwaysTwelve } = require('.');

describe('Always Twelve', function () {
  context('when called', function () {
    it('returns 12', function () {
      // Called twice, though nothing is asserted
      alwaysTwelve(true, true);
      alwaysTwelve(false, false);
    });
  });
});

This bad test will pass and result in 100% test coverage. Not only does test coverage say “100%” when we may need to test more, it shows us “100%” when we thought we tested everything but didn’t.

What This Tells Us About Test Coverage

There are some unfortunate conclusions we can take from these examples.

Executed and tested are two different concepts. While we may know what code executed through test coverage, it doesn’t mean those lines are properly tested or tested at all.
The percentage is a best case scenario. If the test coverage says 80%, that tells us between 0-80% of the code is properly tested.
Once a line of code is marked as covered, it ceases to be a helpful designation. As shown above, unless we know why a line of code is considered covered, we can’t say it’s been properly tested.
The number of lines unexecuted is the only useful metric. This measurement can tell us what code needs tests or what areas of code are weak on testing. The percentage of test covered tells us little more than which code was executed, which has some value but not much. It's only useful in telling us what code wasn't executed.

Test coverage doesn’t help us improve code quality, explore the design of our code, discover bugs, prevent bugs, or test the logic of the code. It can only tell us that our code executed without errors with the inputs we gave it. That’s it really. That’s all we get from test coverage. Maybe that’s worth something.

A Better Way

Good testing starts with good software architecture. Gary Bernhardt has a great talk called Boundaries where he discusses a model for thinking about and designing software that can lead to helpful testing practices. The video does a much better job than I can at explaining his thoughts. But there are some things I’d like to pull from it to talk about testing.

Gary has two categories of code. One category contains all of the paths and conditionals but has no dependencies. This category of code includes business logic and policies but doesn’t access the outside world like databases, the internet, or disks. It lends itself well to isolated unit testing, and it allows developers to test all possible paths quickly (because 2ⁿ tests adds up). He calls the category the functional core.

The other category of code contains almost no paths yet defines all of the dependencies. This category lends itself well to integration testing since integration tests wire together databases, internet, and disks and ensure they work. He calls this category the imperative shell.

With these categories in mind, we can look at code and look at whether there are outside dependencies and whether all paths are well-tested. We can write many fast unit tests since the paths are isolated from the outside world and a few integration tests to ensure the world is wired together correctly.

Therefore, code reviewers can move from relying on test coverage to thinking about paths and dependencies.

Have we introduced a dependency where it shouldn’t be?
Have we added paths in our code where they shouldn’t be?
Are we testing all of the conditionals in the code?
Are we writing small functions so we can reduce our testing surface area and create solid boundaries?

I’d also like to point to another video that I think gives many good thoughts on testing well called Please Don’t Mock Me. It is a good addition to Gary Bernhardt’s Boundaries video and explores poor ways to use tests and one good way to use them.

How Should We Use Test Coverage?

There is some usefulness to this measurement when it gives feedback on unexecuted code during the development process. It can show TDD practitioners where they are getting ahead of themselves. It can show developers where they forgot to test before they open a pull request. It can show open source maintainers areas missing tests by a contribution. The measurement for the project may not say much as a whole, but during the changes to the code, the unexecuted value can give some insights.

And that is the important takeaway—the value is in knowing what code went unexecuted during our test run. It's better to think of the measurement as "10% of my code was unexecuted during my tests" rather than "90% of my code is covered by tests" (which as we've seen may not be true). Or better yet, "I've added 15 lines of code that aren't executed during my tests." This to me is the more helpful—and more honest—approach, and it helps us change code with some added confidence.