DEV Community: John Hotterbeekx

Meet team Heisenberg, an inside look in a great team

John Hotterbeekx — Tue, 30 Jul 2019 18:17:19 +0000

I'd love to introduce you to the team I've proudly been a part of for the last three years. We've had our shares of ups and downs working together. Streamlining our processes to increase not only the amount of work we get done, but also the quality of the it. Learning what to do, and of course what not to do. If you are interested, sit down, strap yourself to your seat and join me while I take you through the journey of team Heisenberg.

Who are we?

Three years ago a restructuring within our product and engineering department put a group of six people together. We came from all corners of the world, well sort of, if you consider the floor of our building a world, which it basically is because working on a software department is a world on it's own. Anyway, most of us were already working for the company when we formed the team, but we had some new people as well.

The team consists of Mike, our product owner. Who's been fighting to get the features built that need to be built, define a clear road map and vision for our products and most importantly making sure that the level of jokes being told are as low as they can possibly be. We also had Niels, our scrum master. Not only teaching us to work agile using scrum, but also taking us on a mentoring journey to increase the quality of the products that we build and increasing the quality of the team at the same time. And last but not least there are four software engineers. All of us making sure things were built in the right way, making sure everything keeps working as expected and making sure our product owners emotions kept tightly strapped in the roller coaster we put them in.

As all developers do, we all have quirks of our own. Take Johan who keeps trying to convince everybody that we have to speak Dutch, or Jo who is a Belgium guy with a love for BBQ, weapons and bad movies. It is like having your own cross-over between Chuck Norris and Jo Bonten in your team. Our third developer started of with our good friend Bert, who was so fond of telling us to add comments saying arrange, act and assert to every test or that we had to remove unused usings in our C# files, that even after having left the company two years ago we still refer to these things as Bertjes. Nowadays his place has been taken by our Brazilian friend Geraldo, who's awesome logic sense to simplify chances of anything happening to 50/50 is always great to have around. Lastly there is myself, I'm not going to praise myself here, but if you'd stop by there are probably enough weird things my team mates can tell you about me. This band of misfits is us, team Heisenberg.

But why Heisenberg you might be wondering. Well starting a new team should always start with the best thing you'll ever do...coming up with a name. I remember we wrote a whole list of names down. TV Shows, childhood cartoons, a lot of names with the word code in it. Slowly reducing them to just a few and eventually voting for Heisenberg. Of course thinking of the somewhat famous German physicist Werner Karl Heisenberg... well not really, we all just loved the TV show. I mean come one, how awesome is it to say "I am the one who knocks" when entering a room? And next to that having a name that's almost the same as the feared Heisenbug, we couldn't get more awesome than this, right?

Full stack and more

While I hear different ways software engineers define roles in their teams, we've found our own way in this. Since the beginning our agreement has been that every engineer in our team should be able to do everything we need to build and maintain our products. You might not be great at it, but you should understand the basics and be able to work with it. While the basics of this knowledge is our full stack, meaning our React front-end and our C# back-end, we go further than that. We all write our own tests, monitor our running products, maintain our infrastructure in the cloud, maintain our CI/CD pipelines, do our designing basically, manage our databases, run our data warehouse, anything that is part of building our product. And in the last year, after our scrum master left the team, we've also started rotating the role of scrum master between team members every week. While it is a lot of work in keeping up with the everything, this way of working gives us great advantages in swarming stories and reducing dependencies on each other. At this point any three developers can go on holiday and the fourth one is able to take care of everything he needs to.

How do we work?

We've been doing agile scrum since the start of the team. This means we do all the regular scrum stuff like stand-ups, sprint reviews, sprint planning and retrospectives. We work with stories, a backlog and sprints. But it get's more interesting as we dive into how we handle these stories.

The first thing we do when starting a story is a 'Design Discussion'. This is where all the engineers gather up and discuss how exactly we are going to build it. This usually ends up in several, pretty detailed tasks, which helps us in swarming the story. In the design discussion we also think about which end-to-end, monitoring and data-validation tests might need to be added, all other tests are part of the regular tasks. Next up we make something that we call a 'Test Plan'. One of us makes a mind-map with everything we want to manually test for the story, to make sure it works. While most of it is covered with automated tests we keep doing this step because even though automating tests is great, nothing is better to give you a good final overview of the workings of a feature as manual testing. You can compare it with exploratory testing, but with some guidance. This plan is then looked at by a second person who adds more steps if possible while reviewing it.

This takes us to the tasks which are always handled twice. One person picks it up, by putting his initial on it and once done, which includes writing all tests for it, checks it off. Then another person picks the same tasks and reviews everything the other person did. So no task is completed unless two people looked at it.

Architecture review

Once all tasks are done, including all tests, we have one task left. We call it the 'Architecture review'. We sit together with all developers from the team and go through everything we touched in the code. Now we pay extra attention to the way we named entities, the place we put them, dependencies between them. We discuss direction where we want to go with the code and structure. Usually this means we will change some things to make steps in the way we want to go. This keeps the architecture of our code in good health and gives us a point to regularly think about it.

Now we push our code to our test environment, let the automated tests run and pick up our test plan. At this point two different people use the test plan and manually perform and validate all the steps from our test plan on the test environment. Trying our best to find at least one issue, because a wise man once told us if you test good enough you should always be able to find at least one thing wrong. And you will find one, we tend to find quite some issues we would have otherwise overlooked using this method. Once we developers agree the story is complete, we demo it to our product owner and if he signs off we push everything to production and drop our feature branches.

Things we learned

When you work together as a team you keep adjusting the way you work to increase the quality and performance of the team. We've tried a lot, some things worked great, some of them didn't. I'd love to go over a few of them.

Batman and Robin

Next to our function we have several roles in the team. You can be scrum master for that week, you can be designated to do the demo at the next sprint review, and lastly you can be either Batman or Robin...queue bat-music!

These roles are relatively new in our team, but work really well. So what's the deal with them you might ask. Well if you are Batman, the dark knight, the caped crusader, then it is your task to not only protect Gotham, but also to protect the flow of the team. You'll be doing this by taking care if any bug or other incident that comes in our path. And you won't be doing this alone, should you need help you can always count on your trusty sidekick Robin. Sounds awesome doesn't it? It is, and it really works!

If you work in a team you probably know that there are always interruptions. This can be bugs that are reported, import information requests that need to be handled, basically anything that wasn't planned in a sprint but can't wait until the next one. Our batman takes this task, should he need help he'll ask Robin. This makes sure the other two developers can stay in their flow. Once the task is taken care of, Batman retires and our Robin becomes Batman. He then picks a new Robin and they will stay guardian until the next issue arrives. This way it's divided between the whole team, most of the team does not get interrupted and you can be a cool superhero once in a while.

Zero bug policy

Like every team we've had our fair share of struggles when it comes to bugs. Do you always give them priority over new features? How do you ensure they are found and fixed quickly? What can you do to reduce the chance on bugs?

For most question we follow the best practices. We pay attention to quality, build a lot of automated tests, do manual testing, have monitoring and more. But bugs can, will and should happen. If you don't have any bugs, you probably spent too much time making the code quality to high. Interesting opinion right? Maybe I'll write something about that one day. Anyway one of the things we do is maintain a our own version of a zero bug policy.

Most definitions of zero policies are pretty straight forward, if there is a known bug you fix it before you work on new features. While it sounds awesome, in real life this loses a little bit on the pragmatic side. This is how we came up with our version of the zero bug policy. Any found or reported bug will be investigated, assessed and discussed. Is the impact big enough it will become highest priority in our current sprint. Is the impact not that high, it will go on the backlog and will be highest priority in the next sprint. This means all bugs are usually solved within the time span of a sprint, which is two weeks for us. We find that this solution gives us the best balance between providing the quality we aim to achieve and the amount of disturbance in the work on new features.

Work together

This one might sound simple and unnecessary to mention, but it isn't. I can't express how important it is to work together. Within everything in our team you can tell that we pay attention to this. We do our stand-ups, swarm our stories, have tasks like 'Design discussion' and 'Architecture reviews' where we all sit together. And like I mentioned before, every task is done by at least two people independent from each other so a lot of information is shared that way as well.

Next to this there is always a lot of talking going on in our office. Not everything is work related, but it keeps the lines open which is very important. Because now when somebody is stuck, has some doubts or anything else that could use some help it's relatively easy to just ask it out loudly in our room. I can not express enough how important it is to work on creating this atmosphere in your team. When you enter the room you should feel as a team, not an individual working in a team. When something breaks you should feel responsible, as a team. When you complete your sprint you should celebrate, as a team. Take lunches together, do fun stuff together, create a place where people aren't afraid to talk about stuff they're struggling with, be it work or personal related. That's when you know you've built a good team. That's what we've built in team Heisenberg.

Team learning events

Not only do we work together, but we also learn together as well. Every two weeks we have an hour and a half scheduled where we book a meeting room and watch a video on a topic that interests us. After watching the video we'll discuss it and see if there any new steps that we want to take in the way we work. This way you keep learning new stuff, keep challenging yourself and keep discovering new and interesting concepts. We started off watching videos from Uncle Bob, but nowadays we write down any interesting video we come across and pick one.

Mobbing kick-offs

Most development teams pair every now and then, as do we. But we also use a concept that I like to call a mob kick-off. When we start something new, usually a big new concept where we are not sure how to implement it yet. We'll book a meeting room for a day or two and start mobbing. Mobbing is basically the same as pairing, only with more people. We have one keyboard and mouse, which we pass on to the next person after 10 minutes. All of this while working on a machine that is connected to a TV, so everybody is involved. Since this involves all the developers in the team, you share knowledge instantly, everybody has to understand what is going on and you'll get a solution that fits the team. It really is a great way to kick-start a new epic that you are not sure about how to implement yet.

You are not your product

Another thing I've learned throughout the years. I've been in many teams, seen them come and go and usually a team was tied to a single product. Once the product was done or help in another product was needed you moved on. Slowly I've seen a transition making the link between a product and a team less and less. This made me realize that there is a really important difference between working with several people on a product and working in a team. A good team has their strength in working together, not working on a certain project. Can you have a single project? Yeah, sure. But it should not define your team. Don't be afraid to work on other projects, but work on them as a team. This is where you are at your best.

Testing is hard

A little more technical than the other lessons, but like the title says testing is hard. And not just writing the tests can be a challenge, but thinking about what to tests, not testing to much, using TDD or not, naming your tests, keeping end-to-end tests stable, keeping tests up to date. And how about this one... not letting tests prevent easy refactoring while writing tests to allow easy refactoring... oh yes, I just said that...

I'd love to say that we've found the perfect solution for this, but I don't think there is any. At least not a perfect technical solution. The important tools you need are experience, communication and pragmatism. Experience; keep trying. You'll make too many tests, too little, name them wrong. But keep trying, the more you practice the better you'll get. Communication; keep talking with your team about it. Challenge each other, experiment, but do it together. Pragmatism; don't over do it. Don't spent 3 days writing tests for a feature that has zero impact when it breaks. Stay practical.

Thank you

I'd like to end this article with a thank you towards the team. Thank you for letting me be part of the team, thank you for helping me grow, thank you for this amazing journey.

This article has also been posted on my own blog.

Let's Build a Chrome Extension!

John Hotterbeekx — Sat, 06 Apr 2019 16:21:28 +0000

Recently I've created my first chrome extension. It's called Catify, and it replaces every image on your page with an image of a cat. Now every cat worshiper can get even more support while praying to his or her cat gods. It was my first attempt at a Chrome extension and it was a fun journey. Would you like to create your own Chrome extension as well? Awesome, I'm going to show you how. Let's build it together, so put on your hobbit feet and loudly shout "I'm going on an adventure!". And don't worry, you'll be done before second breakfast.

What are Chrome extensions

Although Chrome is a great browser on its own, you can make it even better. By adding extensions to it. To check which extensions are available you can go to the Chrome Web Store. Just go and check out all the amazing extensions available. And now imagine your own extension in that store as well. Let's make that happen.

What are we building?

While you probably already have dozens of ideas of great extensions to build, we should take a look at the basics first. So for our first extension we'll take one of the basic examples from Google themselves, page redder, and we spice it up a little. We'll create an extension that turns the background color of the page to a random color every time you press the extension icon.

Getting started, create the manifest

Make sure you set up your source control and have your favorite editor pointed to the right location. We start by creating a manifest, this is the file that tells Chrome everything it needs to know about your extension. Things like the name, the icon, but also which permission it requires and where its code is located. Let's create one, create a new file called manifest.json and fill it with the following:

{
  "name": "Make it rain(bow)",
  "description": "Embrace the inner unicorn and reflect on the page background.",
  "version": "0.0.1",
  "manifest_version": 2
}

So, what do we see here? Firstly the name. This is how the extension gets shown in the store, in the extension overview and unless specified otherwise what you get to see when you hover the icon inside the browser. Then there is description, this is just like the title, a description shown in the store and the extension overview. Followed by the version of your extension. You should use semantic versioning on your extension and increase this every time you update your extension. Lastly we see manifest_version, this tells Chrome this file is written in manifest version 2. Should you want to support Chrome before version 18, you should use manifest version 1, but you'll lose some of the newer functionality. Now that the basics are in place, we can slowly add more stuff.

Change the title when hovering the icon

By default it shows the name of the extension, but it doesn't have to be the same. Let's change it! Add the following to the root of your manifest.json.

"browser_action": {
  "default_title": "Unleash the unicorn dust!"
},

Now when the user hovers the icon, it will show the text Unleash the unicorn dust!.

Permissions and the script

Before we can build the actual code for the extension, we need to add two more things to the manifest. Firstly, we need to define the permissions it needs to have. In this case we only need one permission, which is access to the current tab. Let's define this. Add the following to the root of your manifest.json.

"permissions": [
  "activeTab"
],

Next we need some script to run which performs the actual changing of the background. We will be performing this in a background script, which we need to define in the manifest.json as well. Add this to the root.

"background": {
  "scripts": ["background.js"],
  "persistent": false
},

So we will define our logic in a file called background.js. Next to that it will not be persistence, you should only make it persistent if the extension uses chrome.webRequest API to block or modify network request. Time to build the actual logic!

Changing the page background color

Okay, so we told Chrome that the logic is placed in background.js, so let's make this file and build the logic.

chrome.browserAction.onClicked.addListener(function(tab) {
  const colors = ['red', 'orange', 'yellow', 'green', 'blue', 'violet'];

  const colorPicker = () => {
    const randomIndex = Math.floor(Math.random() * colors.length);
    return colors[randomIndex];
  }

  chrome.tabs.executeScript({
    code: 'document.body.style.backgroundColor="' + colorPicker() + '"',
  });
});
});

Let's take a short look at the code. The first line tells us that we added a listener to the onClick event of the browserAction. What, what, what?!? The browserAction is the button you see in chrome when you add an extension, the onClick is the event that happens when you click that button and adding a listener means this triggers when that click action is performed. So this method is executed when you click the button inside chrome.

The code itself isn't that exiting. We have a list of colors, a method that picks a random color from that list and a script execution that changes the background color. We do this by executing a piece of javascript inside the browser tab that performs this action inside the actual page.

Add an icon

Before we are going to try out our extension, we'll give it something nice to look at. We are going to define the icon that is shown in the top of your browser for this extension. Start by creating any 128x128 image you want. Now you'll want to save this image into several formats:

128x128, which is used in the chrome store
48x48, which is used during installation
32x32, which is often used by windows
16x16, which is used in chrome itself as the icon that is shown at the top of your screen.

To add these images, we change the following in our manifest.json. Add this to the browser_action section:

"default_icon": "icon16.png"

We only need to specify the 16x16 image here, since that icon is always 16x16 on any device. And add this piece to the root:

"icons": { 
  "16": "icon16.png",
  "32": "icon32.png",
  "48": "icon48.png",
  "128": "icon128.png" 
},

These are the icons that can be used from your applications and which have all the required sizes available.

Lets try it out!

Are you excited? You should be, because we are going to test our extension in our own browser! Fire up your chrome and go to your extensions by pressing the menu button and choosing More Tools > Extensions. The first thing you should do is enable developer mode. You find this in the top right corner. Enable this.

Now you should see three buttons appear in the top left of your page. Allowing you to load an unpacked extension, pack an extension or force updates. Click on the first button, to load an unpacked extension.

Now browse to the folder where you've create the extension and press Select folder. Your extension will be installed now, how exciting! After it's installed you should see it on your extension page and at the top of your browser.

Lets try it out! Open a new tab, go to dev.to and press the rainbow icon. And press it again...and again...and again...taste the rainbow!

It works!

Publishing your extension

There is only one thing left to do, and that is publishing your amazing Make it rain(bow) extension. Let's walk through the steps.

Create a zip file that contains all the files you've made. So the manifest.json, the background.js and all the icons should be in there.
Go to https://chrome.google.com/webstore/devconsole/ and sign in with your google account.

After you accept the licenses you can continue with your extension.

Press the button that says NEW ITEM in the top right, which opens a dialog where you should chose the .zip file you created in step 1.
After uploading a form will open which requires some details before you can make your extension available. You will at least need to fill out:
- Title
- Summary
- Detailed description
- Category
- Language
- One screenshot So let's fill these out.

Press the 'SAVE DRAFT' and if everything is filled out correctly you should now be able to press 'PUBLISH ITEM'. Click it, now only a confirmation awaits. While you search your inner unicorn you feel your stomach growling for second breakfast, so you quickly press PUBLISH and your adventure is done. You just finished your first extension!

Finishing up

Thank you for joining me on this adventure, I hope you learned something from it. As you can see, creating a Chrome extension isn't that hard. Coming up with a good idea for an extension that isn't available yet, that's a whole different story. I can only wish you good luck and hope to see one of your extensions very soon!

Resources

While there is a lot to find on the internet about extensions, I mainly used the google documentation as it is very clear and complete. There are also a lot of working examples in there:

https://developer.chrome.com/extensions

And you can find the example code used here:
https://github.com/JHotterbeekx/make-it-rainbow-chrome-extension

Testing asynchronous lifecycle methods with Jest

John Hotterbeekx — Sun, 17 Mar 2019 16:14:28 +0000

While helping out a colleague recently we wandered into the land of promises, slowly drifting away from the yellow brick asynchronous road and getting stuck in the swap of broken promises, awaiting for our fate to resolve. If you have some experience with writing asynchronous code in javascript you might have gotten the references. If not, don't worry I'll go into them later on, so you too can learn about promises and async / await, but mostly how can you write unit tests when your lifecycle methods use asynchronous code. Because this is the swamp where we ended up and got stuck. However slowly but surely we got out and I'd love to share my learnings of this experience with you.

Before I start with the actual code, this article expects you to have basic knowledge of react, jest and enzyme. I'll explain some basic parts of it but not the whole base, since that would be a pretty big area to cover. I've created a demo locally and placed it on github, you can find it at https://github.com/JHotterbeekx/demo-jest-testing-with-async-lifecycle.

Promises

What are promises? And I'm not talking about those things you say to your spouse, or the ones you don't need to keep as long as you cross your fingers. I'm talking about javascript promises. They are actually not that different from the promises you make in real life, you make them, they can be fulfilled which we call resolving, or they can be broken which we call rejecting. A promise method is a method that defines an action it's going to perform and how it's going to handle the result and / or failure of that action. The actual actions is not performed yet, but will be. Getting confused yet? Let's look at some code to shed some light on it. Let's bring out the console logs!

function add(x, y) {
  return x + y
}
add(12, 13);

What does this do? It writes the number 25 to the console. Nothing fancy about this one. Let's turn it into a promise.

function add(x, y) {
  return new Promise((resolve) => resolve(x + y));
}
console.log(add(12, 13));

What do we see now in the console. If you are fast, very fast, I mean seriously fast like the flash or have the time slowing abilties people in the matrix seem to have you'll see a quick log showing:

Promise {<pending>}

This means we gave a promise that it would do something, in this case add the numbers, but it didn't fulfill that promise yet. Now if you wait microsecond of two the promise will resolve. You'll noticed the log changed. Changed? Yes changed, since it's still the same promise it will all happen in the same console log. Your super hero self will now see what all other humans saw almost instantly:

Promise {<resolved>: 25}

This means the promise was resolved, and the value it returned is 25. But what if we want to log that value, how can we get to that value? Hold on, we'll get you in the matrix, or let's just handle the result of the promise.

function add(x, y) {
  return new Promise((resolve) => resolve(x + y));
}
add(12, 13).then((sum) => console.log(sum));

What do you see now? 25? Wow, is that magic? Close, but not really, although it's the next best thing I promise you. You see what we changed here? We added then .then() part after the add(). So that tells the engine what to do with the result of the promise once it resolves, which in this case is logging it to the console. Awesome right?!?
But I feel the question burning on the tip of your tongue, like we have a mental connection across the world wide web, it feels like your asking me; "Can you make it even more awesome? Please?!?". Yes, we can! Watch this:

function add(x, y) {
  return new Promise((resolve) => resolve(x + y));
}
add(12, 13)
  .then((sum) => {
    console.log(sum)
    return sum + 10;
  })
  .then((sum) => {
    console.log(sum)
    return sum + 7;
  })
  .then((sum) => {
    console.log(sum)
  })

What does your console show you now? Could it be?

25
35
42

We found the meaning of life! But how is that possible? This is the magic called promise chaining. This means when you return a result form your resolve handler, which is the code inside the then, you spawn another resolve. Which can be handled and spawn another one, and so on and so on. In this case we kept increasing the value of the result, which shows in the logs.

Is that it for promises? Not by a long shot, but it's enough of a base for this article. Should you want to read more I'd love to recommend you this article: https://codeburst.io/javascript-learn-promises-f1eaa00c5461 or use google since promises has been along for a while, there is a lot of material you can find. For us, we go on to the next level, async / await.

Async / Await

While promises have been around for a while, around two years ago with the release of ES2017, we got a new present from Santa. A lot of complains had been going around about the messiness of the code when using promises, nested promises and promise chaining. Some people are never satisfied, you get awesome asynchronous code possibilities and they abuse it and make it messy again... But they were right, it does get messy pretty quickly, just imagine having to handle two or three promises in a certain order, the horror! That's why we got async / await. Lets look at an example with three promises, where I've written the last example to keep using the same promise:

function add(x, y) {
  return new Promise((resolve) => resolve(x + y));
}
add(12, 13)
  .then((sum) => {
    console.log(sum);
    return add(sum, 10);
  })
  .then((sum) => {
    console.log(sum);
    return add(sum, 7);
  })
  .then((sum) => {
    console.log(sum);
  });

While the result is amazing, reading the code isn't that easy is it? Let's use await and see what changes.

function add(x, y) {
  return new Promise(resolve => resolve(x + y));
}

async function showMeaningOfLife() {
  let sum = 0;
  sum = await add(12, 13);
  sum = await add(sum, 10);
  sum = await add(sum, 7);
  console.log(sum);
}

showMeaningOfLife();

This is a lot cleaner, right? We did have to wrap the asynchronous calls in a method to be able to use the await, but all the chaining and nesting is gone, which makes it a lot easier to read.

Basically there is not a whole not more to async/await, it's a way to make promises easier and more readable. Should you want to read more about the differences and advantages, I'd recommend you read: https://codeburst.io/javascript-es-2017-learn-async-await-by-example-48acc58bad65. We will leave it at this for now, it's time to go look at testing!

Real use-case

Okay let's assume we have a application that has two components. First of a DataDisplayer, this displays the result that it retrieves from a DataRetriever. However this retriever works asynchronously, so we can't just use the result, we have to handle this correctly. I've added comments to the code to try and explain what we do, let's look at DataDisplayer.

import React from "react";
import RetrieveData from "./DataRetriever";

export default class DataDisplayer extends React.Component {
  constructor(props) {
    super(props);

    // We initialize the state with a property that contains a boolean telling us if data is
    // available, which will be set to 'true' once the data is available. And a data
    // property which will be filled with a title.
    this.state = {
      dataAvailable: false,
      data: null
    };
  }

  // We use the componentDidMount to trigger the retrieval of the data once the component is
  // mounted. Which means the component first mounts with its default state and than triggers
  // this method so data is retrieved. We make the method asynchronous so we are able to use
  // await. This gives us a better readable and debuggable way to handle the promise received
  // from RetrieveData().
  async componentDidMount () {
    // We call the retrieve method and wait for the promise to resolve, the result of this resolved
    // promise will be the title, which is placed in the variable title. We validate if we indeed
    // got a title before updating the state and marking the data as available.
    const title = await RetrieveData();
    if(title){
      this.setState({
        dataAvailable: true,
        data: title
      });
    }
  }

  // This render method will initially render the text 'Data not available', because in the 
  // initial state the property dataAvailable is false. Once data is retrieved and the callback
  // async code has resolved the state will update, which triggers a re-render, so the render
  // is executed again. Now the dataAvailable will be true and the content in data will be shown.
  render() {
    if (!this.state.dataAvailable) return <div>Data not available</div>;
    return (
      <div>
        Data value: <strong>{this.state.data}</strong>
      </div>
    );
  }
}

Okay, lets look at the basic functionality of the page. It renders the page with 'Data not available', on the mount of the component it triggers a call to the retriever. This is a asynchronous method, so we chose to use await to wait for the result. Because we want to use await, we need to make componentDidMount async as well.

Now instead of await, we could also have processed the promise. To do this the content of componentDidMount would have looked like:

RetrieveData().then(title => {
      if(title){
        this.setState({
          dataAvailable: true,
          data: title
        });
      }
    });

Almost the same code, only nesting the handling inside of a function inside then then handler.

Let's take a look at the DataRetriever, which retrieves the data asynchronously for us.

// This demo method calls an open API, then translates the response to JSON. Once that is done
// it returns the 'title' property from this data. So when the API gives us { title: 'myTitle' },
// the code will return 'myTitle'. Since we want to use await to give us a readable way to handle
// the promises we encounter, we have to make the method itselfs asynchronous. Which results in 
// the actual return value being a promise that resolves to the title.
export default async() => {
  const todoData = await fetch("https://jsonplaceholder.typicode.com/todos/1");
  const todoDataJson = await todoData.json();
  return todoDataJson.title;
}

This code reads pretty easy right? We do a call to a rest API, wait for the result to come in. We convert the result as JSON, wait for that to be done. The only thing left is to return the title from it. We could have written this as a promise, which would've made it look like this.

// This demo method calls an open API, then translates the response to JSON. Once that is done
// it calls the passed in callbackMethod with the title property as parameter. So when the API
// gives us { title: 'myTitle' }, the code will perform callbackMethod('myTitle')
export default () => {
  return fetch("https://jsonplaceholder.typicode.com/todos/1")
    .then(response => {
      return response.json();
    })
    .then(responseJson => responseJson.title);
}

Still decently readable, but not as pretty as the await syntax. But you've seen both options now. Time to figure out how to test this.

Testing asynchronous lifecycle methods

No matter if we use the promise style handling or the async / await, the stuff going on in componentDidMount is asynchronous. While I'd usually do a mount with enzyme to trigger the lifecycle methods, it won't work in this case. Why doesn't that work? Great question! The lifecycle method will be triggered when mounting, but since the code is called asynchronously, the execution will continue and not wait for the promise to resolve. So by the time you'll reach your expect calls, the promise is still resolving and the data will not be processed.

Then how can we wait for the promises to resolve? It's actually not that hard. Let's figure it out. We'll start by building a scaffold for the tests.

import React from "react";
import { shallow } from "enzyme";
import DataDisplayer from "./DataDisplayer";

// We want to test DataDisplayer in an isolated state, but DataDisplayer uses DataRetriever.
// To keep the isolation we will need to mock out the DataRetriever. This way we control 
// what this component does and we can predict the outcome. To do this we need to do a manual
// mock, we can do this by importing the component we want to mock, and then defining a mock
// om that import.
import * as DataRetriever from "./DataRetriever";
DataRetriever.default = jest.fn();

describe("DataDisplayer", () => {
  beforeEach(() => {
    // Before each test we want to reset the state of the mocked component, so each test can
    // mock the component in the way it needs to be mocked. Should you have any default mock
    // needed that is required for every test, this is the place to do this.
    DataRetriever.default.mockClear();
  });
});

Is this making sense? Let's go over it again. This is the test file for DataDisplayer, which uses the DataRetriever. We import the DataRetriever into the test, just like DataDisplayer does. But after importing it we replace the default export of the component mocked method. And to be sure that all tests run in isolation, so no test is bothered by any mocking stuff another test did, we clear the mock before each test. But can we predict and control what the mock does yet? No, we can't yet, but we have prepared the tools to do so now. Let's write our first test.

// In this test we will mock the DataRetriever in a way that it will create a promise for us
  // which it will resolve with "fakeTitle" as argument. This simulates that the API has
  // given us a result with { title: "fakeTitle" } in it. We make the test asynchronous, since
  // we want to be able to use await in the code to wait for a promise to resolve.
  it("Should show the data, When retrieved", async () => {
    // We are going to set up the mock value that DataRetriever resolves to, we tell it when the 
    // code uses DataRetiever instead of actually calling it and fetching data from the API. It
    // instantly resolves to a value 'fakeTitle'.
    DataRetriever.default.mockResolvedValue('fakeTitle');

    // We shallow mount the component through enzyme. This renders the component with a fake DOM 
    // making us able to see the result that would be rendered. We specifically use the shallow
    // mount in this case. Not only is this the preferred render for unit tests, since it isolates
    // the component completely when rendering, we also use it because we don't want to trigger
    // the lifecycle methods. Since our lifecycle method handles code asynchronously, we want
    // to be able to wait for that code to complete, this requires manually calling this method.
    var wrapper = shallow(<DataDisplayer />);
    // We need to get the actual instance from the virtual DOM, so we can call any method that 
    // is available on it.
    const instance = wrapper.instance();
    // Now we call the componentDidMount event, telling the component that it mounted. But because
    // we called it manually we are able to await for it to resolve. This makes sure the promise
    // for the method is completed before going on with the code.
    await instance.componentDidMount();
    // Since we fake a result coming back from the retriever, we expect the text to actually show
    // the word "fakeTitle" in the component.
    expect(wrapper.text()).toContain("fakeTitle");
  });

Looks pretty much like any unit test right? Let's walk through it. First of we mock the resolve value. That means that any call to the DataRetriever method results in a promise, which resolves into the string 'fakeTitle'. Next we shallow mount the component. Why shallow? We need the lifecycle methods right? Yes we do, but we want to call them manually, you'll see why. On the next line we take the instance of the component from enzyme so we can interact with it. Preparing us for the next line, where the magic happens. We manually call the lifecycle method componentDidMount, but we use await so it waits for the promises to resolve. Awesome right? Now we know that state is at the point we want it to be, so we can just look and see if we can find the 'fakeTitle' being rendered in the component.

Now there is another scenario we want to test, what if the API would fail? Or it returns no title? Let's try it.

// In this test we will mock the DataRetriever in a way that it will return a different promise
  // which resolves without value. This simulates the API returning unexpected data.
  // We make the test asynchronous, since we want to be able to use await in the code to wait 
  // for a promise to resolve.
  it("Should show not available, When data has not been retrieved", async () => {
    // We are going to set up the mock value that DataRetriever resolves to, we tell it when the 
    // code uses DataRetiever instead of actually calling it and fetching data from the API. It
    // instantly resolves to an undefined value, so we can handle nothing coming back from the API.
    DataRetriever.default.mockResolvedValue(undefined);

    // We are shallow mounting the component again, using its instance, calling the 
    // componentDidMount and waiting for it to resolve. Only this time it will resolve to a value
    // of undefined.
    var wrapper = shallow(<DataDisplayer />);
    const instance = wrapper.instance();
    await instance.componentDidMount();
    // Since we fake no result coming back from the retriever we don't expect any title appearing
    // on the page, but instead we expect to see the text "not available"
    expect(wrapper.text()).toContain("not available");

So we do all the same magic again. We prepare the resolve value of the mock, shallow the component, get the instance and manually trigger the componentDidMount and wait for it to complete. Now we now it returned nothing, so we still expect the text 'not available' to be visible. Not that hard, right?

Resources

While the material on jest, enzyme, async and unit testing is widely available on the internet. This solution was actually pretty scarce to find. We found a lot of pretty complicated ways to get this working, which all didn't really work out. In the end we stumbled upon a stackoverflow post which got us to this solution. https://stackoverflow.com/questions/51895198/jest-enzyme-mock-async-function-in-lifecycle-method. And of course you can check the demo app with the source from this article, you can find it on https://github.com/JHotterbeekx/demo-jest-testing-with-async-lifecycle. Small tip, the final result is the async / await syntax, but if you check the history you can also find the promise implementation.

Wrapping up

We do more and more asynchronously, but it does give us some extra challenges. Is this the only way to do it? Nope. Is this the best way to do it? Probably not. Take the things you read, make them your own and when you've done that try teaching your solution to others. Remember, you learn by reading, you learn by doing and you make it stick by sharing.

Mocking with callbacks in Jest

John Hotterbeekx — Sat, 23 Feb 2019 18:38:27 +0000

Today I was helping one of my colleagues creating unit tests for their components. The application is built with react, running on parcel and the combination of jest and enzyme for testing. Everything was going nice and easy until we reached a component that used a callback method passed through to another component. The result in the first component would change once the callback was called. Up to that point I was able to reach into my own knowledge on how to build the unit tests, what to look for and how to handle mocking, but this situation didn't feel that straight forward. While digging through examples on the internet we figured it out and now I'd like to share the result of it with you.

Before I start with the actual code, this article expects you to have basic knowledge of react, jest and enzyme. I'll explain some basic parts of it but not the whole base, since that would be a pretty big area to cover. I started building a demo in https://codesandbox.io/, but ran into an issue with jest.mock not being supported yet. So I've created the demo locally and placed it on github, you can find it at https://github.com/JHotterbeekx/jest-mock-with-callback-demo.

Callbacks

Let's start of with a brief explanation of what callbacks are. I assume you are familiar with methods so let's just take this basic method as an example.

function doSomething(argument) {
  console.log(argument);
}
doSomething("I'm awesome")

What does this do? It writes the string "I'm awesome" to the console. Assuming that the argument you pass in is a string, it will write this string to the console. In fact you can pass anything into it and it will write it to the console. But what if we pass it a method? Lets try.

function doSomething(argument) {
  console.log(argument);
}

function doAnotherThing() {
  console.log("I'm awesome");
}

doSomething(doAnotherThing);

What does the console tell you now?

function doAnotherThing() {
  console.log("I'm awesome");
}

Okay, seems to make sense. You passed a method into the other method, so naturally logging this argument would show you this method. But wait, what if I were to call this method?

function doSomething(argument) {
  argument();
}

function doAnotherThing() {
  console.log("I'm awesome");
}

doSomething(doAnotherThing);

What does the console tell you now?

I'm awesome

What? How awesome is that? We passed a method to another method, which in turn calls the method we passed. Can we make it even more awesome? Yes we can, watch this.

function doSomething(argument, whatYouAre) {
  argument(whatYouAre);
}

function doAnotherThing(whatIAm) {
  console.log("I'm " + whatIAm);
}

doSomething(doAnotherThing, "even more awesome");

What does the console tell you now?

I'm even more awesome

We made it even more awesome! You see what we did there? We did not only pass the method, but also an extra argument that is later passed on to the method. You've just seen the basic concept of callback methods. I hear you thinking, "But this does not make any sense to me! Why would you do this?!?". Well the example above is setup to keep it easy to read, but it might not seem to make a lot of sense yet. Let my try and give you a more real-life example. Imagine that doSomething does an call to an API for you, when that call is done it parses the result and now calls the callback method with the result. Now the component that passed in the callback method and defined it, will handle it through the content of this method. Are you able to follow? It's always easier when you can see it, let's work it out.

Real use case

Okay let's assume we have a application that has two components. First of a DataDisplayer, this displays the result that it retrieves from a DataRetriever. However this retriever works asynchronously, so it can't just pass the result back. There are several ways to do this, but in this case we will use callback method. I've added comments to the code to try and explain what we do, let's look at DataDisplayer.

import React from "react";
import DataRetriever from "./DataRetriever";

export default class DataDisplayer extends React.Component {
  constructor(props) {
    super(props);

    // We initialize the state with a property that contains a boolean telling us if data is
    // available, which will be set to 'true' once the callback method is called. And a data
    // property which will be filled on callback containing a string with a title.
    this.state = {
      dataAvailable: false,
      data: null
    };
  }

  // We use the componentDidMount to trigger the retrieval of the data once the component is
  // mounted. Which means the component first mounts with its default state and than triggers
  // this method so data is retrieved.
  componentDidMount() {
    // We create a new instance of data retriever and call the retrieve method. In this
    // retrieve method we pass a so-called callback method as a parameter. This method will
    // be called inside the retrieve method. As you can see the method expects a title parameter
    // which it will set on the data property in the state and also setting the dataAvailable
    // property to true;
    new DataRetriever().Retrieve(title => {
      this.setState({
        dataAvailable: true,
        data: title
      });
    });
  }

  // This render method will initially render the text 'Data not available', because in the 
  // initial state the property dataAvailable is false. Once data is retrieved and the callback
  // method has been called the state will update, which triggers a re-render, so the render
  // is executed again. Now the dataAvailable will be true and the content in data will be shown.
  render() {
    if (!this.state.dataAvailable) return <div>Data not available</div>;
    return (
      <div>
        Data value: <strong>{this.state.data}</strong>
      </div>
    );
  }
}

Okay lets look at the basic functionality of the page. It renders the page with 'Data not available'. On the mount of the component it triggers a call to the retriever, passing a callback method. When called this callback method gets the result of the retriever, placing it in the state and re-rendering the component showing the retrieved title.

Now let's take a look at the DataRetriever, this is where the callback method is passed into.

export default class DataRetriever {

  // This demo method calls an open API, then translates the response to JSON. Once that is done
  // it calls the passed in callbackMethod with the title property as parameter. So when the API
  // gives us { title: 'myTitle' }, the code will perform callbackMethod('myTitle')
  Retrieve(callbackMethod) {
    fetch("https://jsonplaceholder.typicode.com/todos/1")
      .then(response => {
        return response.json();
      })
      .then(responseJson => callbackMethod(responseJson.title));
  }
}

So this does an API call to a website call jsonplaceholder. It parses this result as a JSON object and then calls the callback method with the title of the object as argument. Starting to make sense now right? Great, but how are you supposed to test all this? Let's dive deep into that.

Testing with callback mocking

Why would we even want to mock? We are writing unit tests, and the thing with unit tests is that you want them to test one unit. In this case one component. If you just call DataDisplayer it will also use DataRetriever, but that component is probably already tested on it's own. You actually want to be able to predict what your DataRetriever is going to do, control it from your other component. Another reason you want this isolation is that when you break DataRetriever, you only want the tests of that component to break not every component that might use it in one way or another. Imagine having to change dozens of tests when you change something in the logic of DataRetriever, you don't want that.

I mentioned you want to predict what the other component, in this case DataRetriever, does. We do that through mocking. Mocking allows us to replace the DataRetriever component with a fake (or mocked) component that does exactly what we want. Let's start by building the base scafolding inside the test file.

import React from "react";
import { mount } from "enzyme";
import DataDisplayer from "./DataDisplayer";
// We want to test DataDisplayer in an isolated state, but DataDisplayer uses DataRetriever.
// To keep the isolation we will need to mock out the DataRetriever. This way we control 
// what this component does and we can predict the outcome. To do this we need to do a manual
// mock, we can do this by importing the component we want to mock, and then defining a mock
// om that import.
import DataRetriever from "./DataRetriever";
jest.mock("./DataRetriever");


describe("DataDisplayer", () => {
  // Before each test we want to reset the state of the mocked component, so each test can
  // mock the component in the way it needs to be mocked. Should you have any default mock
  // needed that is required for every test, this is the place to do this.
  beforeEach(() => {
    DataRetriever.mockClear();
  });
});

Is this making sense? Let's go over it again. This is the test file for DataDisplayer, which uses the DataRetriever. We import the DataRetriever into the test, just like DataDisplayer does. But after importing it we replace this component by a mocked component. And to be sure that all tests run in isolation, so no test is bothered by any mocking stuff another test did, we clear the mock before each test. But can we predict and control what the mock does yet? No, we can't yet, but we have prepared to tools to do so now. Let's write our first test.

// In this test we will mock the DataRetriever in a way that it will call the callback method
// we pass to it, and call it with "fakeTitle" as argument. This simulates that the API has
// given us a result with { title: "fakeTitle" } in it.
it("Should show the data, When retrieved", () => {
  // We are going to set up a mock implementation on the DataRetriever, we tell it when the code
  // uses DataRetiever instead of the original code it will receive a mocked object. This mocked
  // object has one method call "Retrieve".
  DataRetriever.mockImplementation(() => {
    return {
      // The retrieve method is defined as a method with is own logic. It's a method that gets 
      // another method as argument, the so-called callback method. And the only thing it does
      // is call this method with the argument "fakeTitle". This means that when the code will
      // create a new instance of DataRetriever and calls Retrieve(callback) that the method
      // callback is instantly called with the argument "fakeTitle". Simulating the API returning
      // this result.
      Retrieve: (callback) => callback("fakeTitle")
    }
  });

  // We mount the compont through enzyme. This renders the component with a fake DOM making us
  // able to see the result that would be rendered. Usually in unit tests I'd prefer the shallow
  // mount which doesn't execute lifecycle methods, but in this case part of the logic of our
  // component is in the componentDidMount lifecycle method, so we need mount to make sure this
  // lifecycle is triggerd.
  var wrapper = mount(<DataDisplayer />);
  // Since we fake a result coming back from the retriever, we expect the text to actually show
  // the word "fakeTitle" in the component.
  expect(wrapper.text()).toContain("fakeTitle");
});

It isn't that hard right? It looks like most tests you'll come across in jest, the only weird part might by the mockImplementation part. That's where the key lays for mocking this callback. See, by implementing the mock we tell the code that when running this test, any instance of DataRetriever won't be the actual component instead we return a defined object, which also has a Retrieve method. So the code can just call this method. But this retrieve method is something we implemented, and we tell it to just call the callback with a string containing "fakeTitle". So as soon as the actual code calls Retrieve(callback) callback is instantly called, like callback("fakeTitle"). It might take some getting used to, but just try it, it does make sense.

Now there is another scenario we want to test, what if the API would fail? Or for whatever reason the callback is not called (yet). Let's write a test for this.

// In this test we will mock the DataRetriever in a way that it will not call the callback
// method we pass to it. This simulates tha API not being finished or returning an error.
it("Should show not available, When data has not been retrieved", () => {
  // We are setting up a new mock implementation on the DataRetriever again.
  DataRetriever.mockImplementation(() => {
    return {
      // This is where we made it a little different. Instead of passing a method which does
      // an instant call to the callback we pass an empty method that doesn't do anything. So
      // when the code will create a new instance of DataRetriever and calls Retrieve(callback)
      // nothing is done with this callback. To make it more clear you could also read this line
      // as: Retriever: (callback) => { /* Do Nothing */ }
      Retrieve: () => {}
    }
  });

  //We mount the component again, since we need to use the lifecycle methods.
  var wrapper = mount(<DataDisplayer />);
  // Since we fake no result coming back from the retriever we don't expect any title appearing
  // on the page, but instead we expect to see the text "not available"
  expect(wrapper.text()).toContain("not available");
});

The only 'big' change we did was swapping out the implementation of Retrieve(). Instead of calling the callback method directly, we do nothing. So when the code calls Retrieve() from this test, the actual callback is never triggered. Cool right?

Resources

The internet is full of resources, although on this subject you've got to take care not using the ones that are to out of date. For jest and mocking a great place to start is their documentation on https://jestjs.io/docs/en/getting-started, especially the part on mocking ES6 classes https://jestjs.io/docs/en/es6-class-mocks. Their documentation can be overwhelming, especially the many ways to mock things, but it's a very good documentation so just try it. Besides that, just google what you want, we were able to figure it out this way. You'll come across enough articles and stack overflow questions to help you out, just watch the date of the articles and try and look for more recent ones.

Promises and async / await

While this example uses a callback method, nowadays these are mostly replaced by using promises or the newer async / await structure. That doesn't mean you can't use callbacks anymore. Just use what makes sense to your code.

Wrapping up

Your head might be spinning at this moment, don't worry it will stop. Or you might be wondering, is this all? Then I'd say, good for you! As with most concept in developing software you'll have to see them somewhere, and start doing them yourself. Then doing them again, and again, and again. Finally making them stick, improving them and teaching them to other people. So I'm looking forward to that article somebody is going to write on how to do this in his or her way! You learn by reading, you learn by doing and you make it stick by sharing.

Building Evolutionary Architectures - Book review

John Hotterbeekx — Mon, 04 Feb 2019 18:45:54 +0000

This is a book review for Building Evolutionary Architectures: Support Constant Change.

A while ago I attended a basic talk about micro-services. While covering most of the basics of this architecture, somewhere near the end the speaker mentioned the term 'Evolutionary architecture'. Unfortunately he was only able to mention that he thought this was an important topic, but there was no time to talk about the actual content of it that day. A little bumped out by this, I was still really exited about the subject. I've never been a big fan of the traditional architect role. In my opinion architectures should be formed, evolved if you will, by a collaboration of the teams working on it. You see where my excitement came from? You see that little word in there, staring right into your soul? Evolved... And now hearing someone saying the term evolutionary architecture, this has to be the same thing right? Right? I'm so ready for this idea of biblical proportions now, so I decided to read the book "Building Evolutionary Architectures".

Conclusion

Just like my previous reviews and probably the ones to follow, I start of with the review. If I've sparked your interest, please keep reading on into my deep dive. If not, at least you know what I think of the book.

After closing the book I had some mixed feelings. The first half of the book kind of dragged on, but it got somewhat more practical in the second part. It didn't align with my vision, well maybe it did. The book has been written with the traditional architect role in mind and focuses on the way he or she can and should evolve their architecture. The most important part to accomplish this is defining the key characteristics (or -ilities) of your product and architecture. Make sure you have, preferably automated, ways of validating your system against these characteristics and adjusting them when needed. The first half of the book covers this in a deep theoretic way, talking about choosing you characteristics, considering evolvability as one of them and how to use them. The second part of the book goes more into practical things. Things like monoliths, micro-services, incremental changes and agile are all talked about. This part felt somewhat more useful to me. Throughout they use a widget company as example, which makes it a lot easier to comprehend in practice. Lastly it was a pretty short book, coming in at almost 170 pages, it's a lot shorter then most books I read. All in all I give this book a 7 out of 10, because the core of their evolutionary architecture view is a pretty good one.

Ready for a deep dive? I'll walk through the main chapters and talk about what stuck with me the most.

Chapter 1. Evolutionary architecture

It's common nowadays to hear that you shouldn't make any 5 year plans when creating software, the markets and industries are just too fast for this. While it worked, or at least most of it did it this way, in the past, now we benefit more in working in an agile way. This translates to software architecture as well, it's insane to think that you'll think of an architecture today that will suffice you for the next 3-5 or even more years. You should think a lot more short term, but make sure you can evolve your architecture. Changes to the technical landscapes will happen during the course of your product, and you should be able to handle them. Although architecture is defined a lot of the times as the "hard to change" parts of the software, this doesn't have to be the case as long as you make evolvability one of your key characteristics. This gives us evolutionary architecture: "An evolutionary architecture supports guided, incremental changes across multiple dimensions".

Guided is one of the most important parts of this definition. We need a way to safe guard the important characteristics of the architecture over time. And keep in mind, as the system grows, the number of characteristics that are important usually grow with it. The other key in the definition is incremental, you should be able to make changes to your architecture.

Chapter 2. Fitness functions

An architectural fitness functions allow you to assess if your system still meets the characteristics you defined as important. These means give you guidance, this is what separates evolutionary architecture from reactionary architecture. Reactionary would be responding to problems, like performance issues, not being able to scale or having a security leak when they are an actual problem. Evolutionary architecture makes you define them up front and safeguard them. One interesting development method for this mentioned in this chapter is Monitoring-drive development. Using monitors in production environment to asses both technical and business health. There is an obvious preference to automate the fitness functions, so it's easy to run them. A risk that is talked about is when you don't define the architecture characteristics up front, you will eventually have the responsibilities scattered throughout your whole code base, making a any impact analysis to understand a change a challenging operation.

Chapter 3. Engineering incremental change

Any reasonable architecture plan must include evolutionary change, without it you are doomed to get stuck. The definition and maintenance of fitness functions is a shared responsibility between developers and architects. Many things about architecture are testable, and there are a lot of tools for this (e.g. JDepend and NDepend). Other supporting libraries mentioned are Chaos Monkey, Similan Army and Conformity Monkey as ways to create holistic fitness tests. Chaos monkey for example randomly takes services down and assess the systems ability to deal with it. And Conformity Monkey assesses best practices on RESTful APIs.

Chapter 4. Architectural coupling

Discussions about architecture frequently boil down to coupling: how the pieces of the architecture connect and rely on one another. Many architects decry coupling as a necessary evil, but it’s difficult to build complex software without relying on (and coupling with) other components. Evolutionary architecture focuses on appropriate coupling — how to identify which dimensions of the architecture should be coupled to provide maximum benefit with minimal overhead and cost. Bounded context are a very useful way to prevent tight coupling and micro-services is an architecture that is very suitable to apply this. This architecture is designed to allow incremental changes, making it very suitable for evolvability. Regardless of which architecture pattern you chose, defining the quantum size is advised. Or in layman's terms, what goes into a part of the system. Small quanta implies faster changes. Several architectures are discussed, among them:

Layered architecture monolith

Developers find it easy to make changes in this architecture, although cross-layered changes require some coordination. Writing fitness function is considered easy as well, because the structure in this monolith is more transparent and the coupling is pretty easy to overview and maintain.

Modular monoliths

Many of the benefit architects see in micro-services - isolation, independence, small unit of change - can be achieved in monolithic architectures. If developers are very disciplined about coupling. Which can me guarded with fitness functions. It's as easy to change and write fitness functions for as in layered architecture, with the benefits of having appropriate coupling.

Both monolithic architectures are a common choice when starting a project because it's easy to understand. Eventually you might need to evolve into another architecture, which nowadays usually will be into micro-services. But keep in mind, if a development team has a hard time building one of the simplest architectures, how will moving to a more complex architecture solve their problems?

Micro-services

If you can successfully pull of micro-services, it is the most suitable architecture for evolvability. You can evolve per service, as long as you keep the communication between services intact.

Service-based architectures

A more commonly used architectural style for migration is a service-based architecture, which is similar to but differs from microservices in three important ways: service granularity, database scope, and integration middleware. Service-based architectures are still domain-centric but address some challenges developers face when restructuring existing applications toward more evolutionary architectures. For example the database is usually monolithic, since for many applications it isn't feasible or possible to restructure years of database schemas.

Chapter 5. Evolutionary Data

How can architects build systems that support evolution but still use traditional tools like relational databases? The key to evolving database design lies in evolving schemas alongside code. Continuous Delivery addresses the problem of how to fit the traditional data silo into the continuous feedback loop of modern software projects. Developers must treat changes to database structure the same way they treat source code: tested,
versioned, and incremental. The level of granularity is also important. Architects sometimes try to build an architecture with a smaller level of granularity than is natural for the business. For example, micro-services architectures aren’t particularly well suited for heavily transactional systems because the goal service quantum is so small. Service-based architectures tend to work better because of less strict quantum size requirements. Also we should consider the coupling characteristics of our application: classes, package/namespace, library and framework, data schemas, and transactional contexts. Ignoring any of these dimensions (or their interactions) creates problems when trying to evolve an architecture.

While systems often cannot avoid transactions, we should try to limit transactional contexts as much as possible because they form a tight coupling knot, hampering the ability to change components or services without affecting others. More importantly, we should take aspects like transactional boundaries into account when thinking about architectural changes. When migrating a monolithic architectural style to a more granular one, start with a small number of larger services first. When building a greenfield micro-services architecture, developers should be diligent about restricting the size of service and data contexts. However, don’t take the name micro-services too literally — each service doesn’t have to be small, but rather capture a useful bounded context.

Chapter 6. Building evolvable architectures

We can use these techniques for building an evolutionary architecture in three steps:

Identify Dimensions
We must identify which dimensions of the architecture we want to protect as it evolves. This always includes technical architecture, and usually things like data design, security, scalability, and the other “-ilities” we have deemed important. This must involve other interested teams within the organization, including business, operations, security, and other affected parties.
Define Fitness Function(s) for Each Dimension
A single dimension often contains numerous fitness functions. For example, we commonly wire a collection of code metrics into the deployment pipeline to ensure architectural characteristics of the code base, such as preventing component dependency cycles.
Use Deployment Pipelines to Automate Fitness Functions
Lastly, we must encourage incremental change on the project, defining stages in a deployment pipeline to apply fitness functions and managing deployment practices like machine provisioning, testing, and other DevOps concerns. Incremental change is the engine of evolutionary architecture, allowing aggressive verification of fitness functions via deployment pipelines and a high degree of automation to make mundane tasks like deployment invisible. Part of the responsibility of developers on projects that support evolutionary architecture is to maintain good cycle time. Cycle time is an important aspect of incremental change because many other metrics derive from it. For example, the velocity of new generations appearing in an architecture is proportional to its cycle time. In other words, if a project’s cycle time lengthens, it slows down how fast the project can deliver new generations, which affects evolvability.

While this advice seems obvious, we constantly see teams that have chosen the shiniest new architectural pattern rather than the most appropriate one suffer. Part of choosing an architecture lies in understanding where the problem and physical architecture come together. We must understand why we want to perform a migration, and it must be a better reason than “it’s the current trend.” Splitting the architecture into domains, along with better team structure and operational isolation, allows for easier incremental change, one of the building blocks of evolutionary architecture, because the focus of work matches the physical work artifacts. When decomposing a monolithic architecture, finding the correct service granularity is key. Creating large services alleviates problems like transactional contexts and orchestration, but does little to break the monolith into smaller pieces. Too-fine-grained components lead to too much orchestration, communication overhead, and interdependency between components.

They dive a lot deeper into developing a evolutionary architecture, talking about making you decisions reversible, be aware that you don't know what the future will bring, be aware of the risk of data corruption and stay away from the pitfall of shoving everything into technical debt. Last but not least, do not underestimate the power of prototyping and do not be afraid to throw it away afterwards!

Chapter 7. Evolutionary Architecture Pitfalls and Anti-patterns

This chapter dives into several anti-patterns and pitfalls. I won't go into them all but to name a few that caught my attention:

Code reuse abuse
Ironically, the more effort developers put into making code reusable the harder it is to use. Making code reusable involves adding additional options and decision points to accommodate the different uses. The more developers add hooks to enable re-usability the more they harm the basic usability of the code. In other words, ease of code use is often inversely proportional to how reusable that code is. When developers build code to be reusable, they must add features to accommodate the myriad ways developers will will eventually use the code. All that future-proofing makes it more difficult for developers to use the code for a single purpose. That doesn't mean you should avoid reuse of code, but you should continually evaluate if you still need it and not be afraid to break the links when it's better.

Planning Horizon
More and more effort put into the assumptions, even if they turn out to be false in six months, leads to a strong attachment to them. Put simply, the more someone invests time or effort into something, the harder it becomes to abandon it. In software, this is seen in the form of the irrational artifact attachment.

Chapter 8. Putting Evolutionary Architecture into Practice

Teams structured around domains rather than technical capabilities have several advantages when it comes to evolutionary architecture and exhibit some common characteristics. Domain-centric teams tend to be cross-functional, meaning every project role is covered by someone on the project. The goal of a domain-centric team is to eliminate operational friction. In other words, the team has all the roles needed to design, implement, and deploy their service, including traditionally separate roles like operations.

By modeling architecture and teams around the domain, the common unit of change is now handled within the same team, reducing artificial friction. A domain-centric architecture may still use layered architecture for its other benefits, such as separation of concerns. For example, the implementation of a particular micro-service
might depend on a framework that implements the layered architecture, allowing that team to easily swap out a technical layer. Micro-services encapsulate the technical architecture inside the domain, inverting the traditional relationship.

Another useful procedure is shifting from a project to a product mentality, as it really increases involvement. Product teams take ownership responsibility for the long-term quality of their product. Thus, developers take ownership of quality metrics and pay more attention to defects. This perspective also helps provide a long-term vision to the team. Also keep you teams small, big teams tend to have more problems and don't work together as well.

Try and apply the three strikes and refactor rule. The first time you do something, just do it. The second time, wince at the duplication but do it anyway. The third time you do something similar, refactor. And don't be afraid to experiment, try out new ideas so you keep growing.

Final Thoughts

They talk about a lot of stuff. Most of it isn't new, it was just applied to architecture and connected into the concept that is called "Evolutionary architecture". These are good concepts, and as developers we should really be aware of this stuff when developing and designing our projects. In the end it all boils down to creating awareness in these areas, don't just chase the new shiny diamonds of which there are a lot in our industry, but make a conscious and well considered choice when building your software.

Keep learning, should you invest your own time?

John Hotterbeekx — Mon, 03 Dec 2018 20:55:04 +0000

Every sprint our team dedicates some time to learning together, today we watched a video by the beloved 'Uncle Bob'. This time we watched an episode on productivity. It was a good episode, but in between him doing weird stuff with his dog and the many strange characters he portraits there was a piece that got all of our attention. He talked about learning, and at the moment I heard him say out loud "Next to your job you should spend 15-20 hours a week learning, you owe this to your employer." I noticed everybody looking like they just saw a ghost. How are you supposed to do that?!?

Uncle Bob's Learning Vision
It's been a while since I've read 'The Clean Coder', to which this video series belongs. A series dedicated to being a professional software engineer. While being fairly short, his stances on learning are pretty clear:

Spend 15-20 hours a week on your career
Learn at least one new language per year
You must at least know the Design Patterns, Design Principles, Methods, Disciplines and Artifacts by heart.
You should keep up with trends and movements, while also knowing and learning the past

And he has some pretty good reasons for thinking like this, I think a quote from his book sums it up pretty good.

The frenetic rate of change in our industry means that software developers must continue to learn copious quantities just to keep up. Woe to the architects who stop coding—they will rapidly find themselves irrelevant. Woe to the programmers who stop learning new languages—they will watch as the industry passes them by. Woe to the developers who fail to learn new disciplines and techniques—their peers will excel as they decline.

Would you visit a doctor who did not keep current with medical journals? Would you hire a tax lawyer who did not keep current with the tax laws and precedents? Why should employers hire developers who don’t keep current?

And his opinion on being professional is also pretty clear.

Perhaps you don’t want to make that kind of commitment. That’s fine, but you should not then think of yourself as a professional. Professionals spend time caring for their profession.

His comparison to lawyers and doctors
For a while I've been saying, our industry is not like others, it's insane to think other jobs require this much learning. Then I saw his example, the doctor. Hmm, I indeed expect my doctor to keep up with trends. It would be nice for him to be able to diagnose me using new techniques researched in the last few years. But in my mind I kept doubting about the amount of time they would need to invest in this learning. So I decided to ask doctor Google. While not being as entertaining as doctor House, it still got me some good answers. Some quick research told me that depending on which state you live in you need to earn between 20-60 CME credits per year to keep your license, thus staying a professional. There are various items that give you these credits, attending conferences, certain meetings, reading journals, etc. And one credit takes one hour of time. That means an average doctor needs to spend the same amount of time on learning per year, as what Bob expect a professional developer to spend in 1-3 weeks. Now the comparison starts to fade for me. Some doctors may be really dedicated, but even increasing this 10 times, just matches half the time of a professional software engineer at best.

What about his tax lawyer then? The same story as doctors, there is a required number of credits per state, which is somewhere between 20-40 credits per year. And the same as with doctors, one credit takes one hour of work. Some a professional tax lawyer, or accountant, needs to spend the same amount of learning per year, as what Bob expects a professional developer to spend in 1-2 weeks.

These are two jobs, which are known to be high maintenance, high responsibility, but also highly rewarding. They require a lot of working hours and a lot of learning hours to keep your license. Although they seem nothing compared to the expectations for a professional software engineer we just read.

Being influenced
While I'm sure Uncle Bob's vision was not meant to be as black and white as I'm using it in this article, there are some dangers to such statements. His work is very well known in the industry, a lot of us will consider him a guru of some form. Thus when he says stuff like that, a lot of people will believe him. It doesn't matter if you are an experience developer, or somebody that is still learning to code. A lot of us have some form of insecurity which is completely natural for a human and widely talked about in the many, many, you would almost say too many, articles about 'Imposter Syndrome'. And those of us that already are somewhat insecure could easily read something like this and start thinking that this is just something every software engineer should do.

Be aware
Why is it so dangerous then? We'll exactly like it states in his book that it isn't. I know, confusing right? In his book he states:

Perhaps you think this is a recipe for burnout. On the contrary, it is a recipe to avoid burnout. Presumably you became a software developer because you are passionate about software and your desire to be a professional is motivated by that passion. During that 20 hours you should be doing those things that reinforce that passion. Those 20 hours should be fun!

So what, you are saying that this is a recipe for burn out? Yes it is! But it is not the number of hours you are putting in, it is why you are putting them in. If your passion for code just flows through your veins and you live and breath to write it, this will probably not be that big of an issue for you. Although you should still be aware, because even doing that much of a thing you love can burn you out, or make you hate it. It gets a lot more dangerous when you are doing this because you think this is expected of you. Although he says it should be fun, it won't be all the time, it won't be a lot of the time, learning can't be fun all the time. And even when it is fun, don't you think that will reflect on your job? If it's really that fun to do and you've just written 4 hours of code after working the full day, you will start you next day after having basically worked for 12 hours. Which accumulates to 60 hours on Friday. There is no way you can still function as good as when you would work for 40 hours. Even when the hours don't wear you down, when the half a day you work at home is that much fun, and you have to spend the next 8 hours on something that isn't that much fun, day in and day out, it will eventually burn you out at your job.

What do actual developers think?
Some highly regarded people advice you to spend a lot of time learning. Besides Uncle Bob I was only able to find the same opinion by John Sonmez, who are both well known for investing time in their career. While others like Andrew Hunt and David Thomas do describe what to do, but not how much time to invest. The thing that I noticed the most is that these people fortunately seem to be the exception, most writers seem to steer clear of advising concrete hours to put in. When you start searching for the opinion of other software engineers, that's exactly what you will find. Opinions, and a lot of them, and a lot of them are different. Although most of them aren't as extreme as the ones above. A lot of people think you do not need to do any coding outside of your job, this thread gives a great vision on this opinion. While others think you do need to spend some time on this outside your job, as discussed in this thread. There are a lot of threads on this subject, showing the insecurity we have about this subject, but I was unable to find people who had agreed on the 15-20 hours a week. Most opinions are somewhere between just coding at work, learning should happen at the job, to a few hours a week at home are necessary.

What should I do?
Great question, I'm glad you asked it! To put it short; Whatever you want! You should try and figure out what you find important in your life, what you want to reach in life and divide you personal time to do those things and reach those goals. For a lot of us, most of us, being a lot of other things will be more important than coding. Think of being a partner, a parent, a friend, having fun, traveling, volunteering, or any other thing you may find important. You probably don't want the last vision you're having on your death bed being that of a factory pattern being injected in your decoupled class, right? Coding, or having a great career may be on the list somewhere, it may not be. Do you want to do everything you can to have the best career as a software engineer, and having that be the most important thing to you, than invest that time! As long as your spare time matches your goals in life, there is no going wrong!

Getting speed back, my Webpack debugging journey

John Hotterbeekx — Sun, 11 Nov 2018 21:55:00 +0000

I want to take you on a journey, through a land of slowness. Where everything lost its motivation to go fast, where hope seems lost. I want to take you through debugging the speed issues we were having with Webpack. Usually I'll try and write articles about processes, giving people advice and tips and get them to think about certain subject. For this article I'm trying a different direction, my goal is to share the process I went through while working on this problem, describing my thought process, but without any judgement. Just what I did, and why. Hoping to inspire and remove the illusion that experienced developers know everything. I'd love to hear your thoughts about the format.

Background

I've been working with React for around two years now. While our project uses a lot of libraries the one I'll be looking into today is Webpack. Webpack is a package we use to bundle our project, it takes all the javascript and typescript code we've written, transpiles, minimizes, and optimizes it into a file that browsers can use, while also processing stuff like CSS, HTML, images and other files we are using. For more information you can find a lot of information on their website.

Webpack is a widely used package, very widely, as in 4+ million downloads per week. While it has gotten somewhat easier, every time I have to work on it, it feels like going on a jungle expedition...on an island where some old guy managed to recreate some dinosaurs... which manage to escape... We've tried alternatives, but out of all of them Webpack matches our needs the most. The documentation is pretty good, there are a lot of tutorials and examples online, but at the same time a lot of them are outdated and no longer work. But for me the biggest issue may be that debugging can be pretty hard, fighting a lion with your hands tied behind your back hard.

The problem

When we finish a user story we first deploy our solution to our test environment, so our automated test suite can do it's magic. We had a story where we happened to find some issues while testing on this environment, resulting in having to do this process several times. But I started to notice that I took pretty long, longer than usual. When looking on Team City to confirm this feeling, I noticed our 'Run unit test & build' took up to 15 minutes, while this usually is around 7 / 8 max. When diving into the logs I found that our build step took between 8 and 10 minutes, at some moments even going up to 12. Our solution is not that big, it should not take this long. Even better, it didn't use to take that long. Something took its mojo!

Where to start?

My first instinct was to check the history on team city. It saves the information of the last couple of builds, so if the problem was introduced in one of the last few stories, I should be able to see a big increase in time here. But all of them were slow, so this wasn't caused recently.

What next? It could be a problem of our CI environment, so I want to check if it's also slow on my own machine. This was quite easy to confirm, building on my own machine takes 8+ minutes.

Could it be an update? We update all our packages at the beginning of our sprints. We do two week sprints, so I decided to take a look at the release notes of Webpack. There was no change in the last sprint that should have caused this. To be sure I tried downgrading it to a version of a few weeks ago, but still it stayed as slow as it was. I quickly checked out the change logs of other major packages we used, but there were no mentions of anything that made me suspect them. Since it was quite some work to revert all the packages to a earlier version I decided to skip this, and follow my feeling that this is probably some issue in Webpack itself.

Getting to the core

The next tool I was going to use was Google. First trying to find somebody with the same issue, looking up terms like "Webpack slow", "Webpack performance problem" and "Webpack speed". I found a lot of results, really a lot. Okay so now I know that a lot of people find Webpack slow, still that doesn't explain a sudden big drop in performance. I looked through some results, most of them being guides to increase performance through caching and pre-building, which wasn't what I was looking for. Since I didn't want a general performance guide, but a way to find the reason of our sudden bad performance I decided to change my search direction, going for "Webpack debug performance".

Debugging

The first suggestion that took my attention was a plugin that would show you the build times of the different parts of the process. "Just what I need!" is what I thought. So I installed the plugin, and started building again. When it was finally done I hopefully scrolled up through the build result looking for the times. And what did it tell me, nothing... The total build time was shown as a little over 8 minutes, it showed a few build steps, but their combined time didn't even touch the 3 minutes. I went to google to check for some examples, maybe I used it wrong, but came up empty handed. Going back to my earlier searched for performance debugging help I found a discussion, somewhere on github where the person was showing the slowness by using the '--progress' flag in webpack. So I decided to give that I try and see what it does.

I'm not alone

When running webpack with the progress flag I got real-time information about what it was doing and instantly noticed I stayed at "92% Chunk asset optimization" for almost 90% of the build time. This has to be the problem! So I opened up Google again and searched for "webpack 92% chunk asset optimization". There were a lot of people having the same issue, finally!

Trial and error

I found massive discussions on github projects about this issue, some spanning more then a year and some with the last message in the last few days. So this issue has been in there for a long time, and it may be still not solved. But there were a lot of suggestions in the replies, people saying "This fixes it for me", and a lot of different things fixed the same issue for different people. My next instinct was to use trial and error by just trying out the solutions posted there, starting of with the most mentioned solutions. The most mentioned solutions were disabling source maps and disabling compression (and some other options) in UglifyJS. So I went back to basics with build mode, disabling source maps and installed a separate version of UglifyJS so I could disable the compression. My next build, was 80 seconds!

The first sign of hope

This build time got me fired up, it was fixable! But in the ideal world I wanted to have the source maps we had, because we use them to get stack traces when errors happen on production. So I tried enabling source maps again, and BAM! an error happened. An error? How is that possible? Panic, confusion, frustration, thoughts about leaving software engineering to become a farmer, they all pasted my mind. I first tried to change the different settings for the source maps, but I got errors every time. So I decided to take a look at the errors, they were syntax errors on type script parts. That is weird, this is valid javascript! Let's google this error, so I searched for "typescript includes does not exist on type" and low and behold I found people with the same issue again.

Transpiling

We recently added typescript to our solution, which was working fine. But the suggestions were to add some parts to the typescript config, so I did just that. Build again, and now...different errors. Hmm, so it impacted something, let's check the new errors out. They were syntax errors again, but it looked like something messed up the type script translation to javascript. Now the pieces were falling into place. We are already using babel to transpile our javascript, can't we just let babel do the typescript as well? I quickly Googled how to do this, set the typescript configuration to just sent out the latest syntax version and updated the webpack config to send the typescript code to babel. Excitedly running the build command again, and...it....worked! I now had a build time of around 120 seconds, with source maps.

Tweaking and confirming

I was very close now, it worked. My next step was figuring out what was causing the slowness. Since the thing I read most was the compression flag on UglifyJS, I started by turning that off. The build time instantly jumped back to 8 minutes. Okay, so this causes the slowness, but why wasn't it an issue before? I think it just slowly got slower while the project grew, adding typescript as well eventually impacted it even more and slowly it reached this point. If it wasn't for having to deploy the same story multiple times to our test environment we probably wouldn't even have noticed this issue until it was even worse.

Can we make it even better?

Now that everything is working it's time to tweak it even more, see if we can get it to go even faster! First I slowly started to change settings back to what they were, so I ended up with only the settings on the UglifyJS part where the compression was disabled being the only real change made. Once this was confirmed to be working, I was at a build time of around 120 seconds. Remember the performance guides I found in my first searches? The ones I didn't need? Well, I was glad I found them now. Skimming back through these guides I saw one easy step that made a lot of sense to me. When using loaders the most efficient way is to use include paths, to make sure it only processes the files it needs to process. With this in mind I looked at the webpack configuration and saw that js / jsx and ts / tsx files were not limited to a folder. That could mean that every module file is put through the loader. I updated the settings to make sure paths were defined where possible. Time for one more build, and our new build time is...79 seconds!

Conclusion

Looking back at this process you'll notice a lot of different techniques. I like to follow my gut feeling, sometimes I just start trying solutions that I find even if I don't understand them. But I always try to understand the actual problem and solution in the end. I use Google, a lot, it would be insane, even impossible to figure this stuff out without using it. Other times I'll take a more structured approach, this all depends on the problem. No two issues are the same, no two developers work the same, but this was my way. I wrote this article to inspire, but also to see if people find these real world though processes on paper interesting, helpful or any thing else. I'd love to hear your thoughts about it.

My solution is the best!

John Hotterbeekx — Thu, 27 Sep 2018 19:16:43 +0000

A few days ago I was watching a keynote speaker talk about architecture. It was an interesting talk, he had a good concept and the idea definitely made a lot of sense. I didn't really like the presenter, which can happen, but that usually doesn't remove any credit from the content. Only this time it did, at least for me.

What happened?

Most of the presentation was pretty good, there were great examples in there and he really knew his stuff. But where he went completely wrong for me is the way he talked about other solutions and techniques. Calling them crappy platforms, telling people they should probably quit if they still use this, calling certain ways of doing things for hipsters, calling methodologies that are being used for years 'big mistakes'. You probably catch my drift by now, there were quite a lot of examples of these things during the presentation. So while the content was good, the way he treated other techniques actually made me lose complete respect of it. While this example is a little extreme, it got me thinking, why makes a lot of us treat our solution as the best, while talking bad about other solutions?

What causes this behavior?

We have a huge choice in techniques we can use for our work, and most of us will probably recognize the feeling of thinking the one we chose is the best. This feeling is completely natural, a part of human nature and it reflects the passion we have for the subject or the choice. While right after you chose a certain technique you might feel somewhat insecure about it, once you are familiar with it, this feeling usually gets replaced by the feeling of passion. If you talk with others and pay attention to yourself, you'll notice that you will be defending this choice quite passionately. So you go from having doubts to thinking it's the best you have ever seen, seems a little weird right? Don't worry, there is absolutely nothing wrong with you, you are just human.

Open yourself up

Who hasn't landed in a discussion about Windows being better than Linux, IOS being better than Android, React being better then Angular? We all did, we all still do and probably will do for a long time. While I'm not saying to just give up on these discussions, try and open yourself up a little. Try and place yourself in the shoes of other people, see how this might work for them and try and accept that we don't know everything and other solutions might work just as good and maybe even better. It seems very easy for us to judge something without actually having worked with it, and I think this all comes from that passionate human nature we have. But try and be aware of this nature and maybe next time try something or hear somebody out first before judging. The thought that I find very helpful for this is; "If a lot of people are using it, it has to be good in some way".

There is no best solution

When talking about this subject, one thing that tends to come up is that each language, framework or other technical solution has a purpose for a different situation. I don't think that is true. There is no "best" solution for a situation, at best there are some good solutions. Our possibilities in software engineering are just too big, solutions are too widely used, this makes the situations where there is one real best solution almost extinct. I even think that the more you learn about different technologies, the more you discover that they are a lot more similar at the core then you might think.

What can we change?

Now looking back at the presentation, what could he have done differently? It's actually very simple, he could've just not said anything about these things. They added zero value to the presentation. And if his goal was making it funny, just try and add a joke, or at least say something that isn't at the expense of others. Imaging the presentation this way actually sparks a light of enthusiasm and inspiration about the actual subject he was presenting, which I think would be the goal he'd want to reach with it.

When looking at our day to day work,you could start by trying to be aware of this, since awareness is key to bettering yourself. Like I said before, don't be judgmental on techniques and solutions but try and look at it from a more logical or rational standpoint. You'll notice that if you are more accepting to the choices of others and are able to acknowledge your lack of knowledge about the subject, others will tend to open up as well and you'll actually learn a lot more from it.

I'd love to end this article on a positive note and with that I'd like to ask you to try and treat others with respect, you don't need to put others down to add value to your own idea. Your vision, your idea, your opinion is worth sharing and is strong enough to stand on it's own!

Soft Skills: The software developer's life manual - Book review

John Hotterbeekx — Mon, 17 Sep 2018 19:57:30 +0000

This is a book review for Soft Skills: The software developer's life manual.

Everybody has read a decent amount of books about technical subjects, like programming languages, design patterns, best practices, etc. This is no different for me, which eventually led to a feeling which I can best describe as hitting some sort of ceiling. I've been a software engineer for about 15 years now and have been focusing on getting better at programming for most of this time. Don't get me wrong, I don't know everything about the languages I use, I don't know all the principles or patterns by heart, but I usually know how to find the things I need. I think this all got me thinking that if I want to grow a little in this area I'd have to deep dive some advanced areas, which means investing a lot of time. Sort of like the remaining part of the 80/20 rule.

So I was pleasantly surprised when this book crossed my path on a list of books that were suggested by a fellow software engineer. It pictured me the image that I'd probably liked coding and had that area covered, but what about the remaining part of the life as a software engineer? I say the remaining part of the life with a reason, since a lot of books focus on areas like talking to clients or job interviews, but this book promises to look at a whole facet of things in the life of a software engineer, besides coding. So I've started reading last month and now I'd love to share with you my opinion about this book.

Conclusion

Starting with the conclusion? Am I insane? Not at all, I'm going to give you my honest opinion about the book, so you can decide for yourself if it's interesting enough. If you still want more, than you can read my deep dive through the chapters. I'd rather you spent the time reading a good book than a review, so just decide for yourself how far you'll read.

When you read the book you'll feel that John Sonmez's intent is to share his years of experience in life with you. He looks at things like dealing with clients, peers and managers, but also at finding love, finances, productivity and staying in shape. The chapters are divided into 7 major sections which I will handle further on. The writing style is easy to read, the chapters are really small which allows for quickly reading one when you have 15 minutes or half an hour to spare. It's full of practical examples, personal stories and assignments to help you get started to bettering your life. When you've read more books about the same subjects, you'll notice a lot of similarity between popular techniques, but in a more simple and practical way. A lot of it feels very familiar, while some of the area's feel only unusable unless you live in the United States or a country where culture and finances are similar. And some chapters depend on where you are in your career for them to be useful, but again, most parts are really good, I think about 70% of the books is in some way applicable to most software engineers. My conclusion, this is a must read for any software developer, I'd even put it in line with books like Clean Code and Design Patterns, this stuff is too important in your life not to think about. My rating 4.5 out of 5 stars.

Section 1: Career

John focuses on your career. A must read for not just beginning software engineers, but any engineer in the business. It looks at subject like figuring out your goal in life and career, people skills and job interviews. But also things like employment choices, have you ever though about starting your own business? Or how about looking to get higher up the ladder? He looks at practical things like working remotely and fighting boredom. The two things that I love the most about this section is his chapter on "Fake it till you make it", the counter balance to the ever more appearing impostor syndrome, which I'll probably write a blog on at a later moment. The other chapter I really love was his closing one called "Don't get religious about technology". The title says it all. Stop arguing that Mac is better then windows, or C# beats C++ any day of the week, embrace technology and don't judge what you don't know, if it's widely used it's probably good if not great. There is only one thing I can say about this, and that's amen!

Section 2: Marketing yourself

This section covers your own branding. See yourself more as offering a service and not as a employee. You'll read about building your brand, why this is important, and mostly on how to build this. He covers subject like writing a blog, which actually is what got me started writing on dev.to. But not only writing, he goes on to creating a successful blog, adding value, using social networks. And the more advanced stuff in speaking, presenting, giving training and writing books and articles. For me the best chapter in this section is again his last one, he really loves to close with the good one. In this case it's called "Don't be afraid to look like an idiot", looking at confidence growing one uncomfortable step at a time.

Section 3: Learning

I can be really short about this one. John developed a 10 step learning program for himself that works really good. It looks at the big picture, but also helps you taking steps, getting progress and preventing the feeling of getting overwhelmed. The most inspiring part of the book was in this section for me, "Teaching: Learn you want? Teach you must." The last step of his 10 step program is teaching to others, a logical and clear step that never sunk in until I actually read it. Maybe it was the picture of Yoda he created, or he just had a great vision on this, either way this section got me exited and without hesitation I joined the dev.to mentorship program to teach, and learn from it.

Section 4: Productivity

There are millions of articles about this subject and to be honest the things John writes about are nothing new. However, they are easy to understand and applicable, which really makes a difference. Think about keeping focus, the Pomodoro technique and multitasking. Be he also talks about the dangers of burning out, which is a big issue in our industry. Lastly he handles subjects like wasting time, building habits and how to get and stay going. This is not my favorite section of the book, must of the stuff was already in the back of my head, but still it was great to look at it from his practical view and personal experience.

Section 5: Financial

Here he looks with you at your earnings, which he tries to help you increase and at your spending. There are really personal stories about investing in here and more advanced stuff if you are interested. He talks about different kind of investments and the dangers of debts and ends with a great personal life story of his retirement at the age of 33. This was my least favorite chapter, mostly because the way his career took of when he was young and the investments he made are just not realistic in the country where I live. It's pretty limited how much you can invest in real estate the way he did and also the starting salaries like silicon valley are just now available around here. Although I really love his openness in this section.

Section 6: Fitness

How many software engineers really pay good attention to their health? I know it's a stereotype that we live on coffee and pizza, but there is some truth in there. This section focuses on getting you healthy. John used to be a body builder when he was young and still is in very good shape. This makes it relatively easy for him to give you good advice on eating healthy, exercise and even the gadgets that will help you in getting there. There is not much I can say about this, he is right and we need to read about this.

Section 7: Spirit

The closing section of the book handles the spirit, without becoming to spiritual. Which I love, he keeps his practical view and still talks about the things in our mind, spirit and soul. Think about the influence of your mind, your mental health and your self-image. Even the subject of finding true love is handled as is failure and the books that helped John gain his success. A great way to end the book, although I think there could have been a little bit more in this section.

Hold on, there is more

I'm not talking the assignments, which you should really do. I'm talking about simple programmer. John not only talks about branding and building a successful blog, he talks from experience, so he really has one. His simple programmer blog is a really great addition to this book and made my daily must read list. Besides these blogs, he has a YouTube channel which I can really recommend, covering a lot of sections from the book in greater details as well as things that aren't in there. Last but not least, he even offers paid services helping you excel at the subjects handled in the book. I have no experience with the latter, but looking at his free stuff, this is probably great as well!

Livable code, embrace the practical mess

John Hotterbeekx — Tue, 04 Sep 2018 20:54:50 +0000

Let me start off with the fact that I've been interested in the clean code principles for quite some time, I think it's very important to keep these principles in mind while developing software. Other people should be able to read and understand your code without a decipher ring and you should not have to move mountains to add a simple feature. With this I've embraced the words from our Uncle Bob and tried to bring his words into practice. Until my last personal project where I found myself wondering, what can I do to clean this code even more, it felt like cleanliness itself had become somewhat of a burden, taking the enjoyment out of just writing the code, I felt it had to be as clean as possible. Was I taking it too far?

The first extreme, the hoarder

In our career we will all come across the two extremes of cleanliness in the code bases we will work with. Let's start of with the most common of all, the hoarder code base. You know what I mean, that one project, where even by speaking it's name you'll see a tear running across the cheek of your fellow developers. Where you open a file, just to add a simple input field to a page and three hours later it feels like the code is swallowing you whole. It's a mess, there are classes of more then a thousand lines, changing one property in page one breaks pages five, six and eight and you have no idea why. Your feature or fix will be slapped in the code, maybe some duct tape is placed over it, just to keep it in place and even though you know you've probably made the code even worse, you close the file as fast as you can and deploy it. Hoping to never have to touch it again. How do you think this happens? Most people will probably say laziness. But it's not, at least in most cases. This cluttering happens one small decision at a time. You start of with a nice clean class, and just add a small thing here. It doesn't really belong here, but it's easy this way. The next person sees this and changes another small thing, there already was something in the class that didn't really belong there, and now there are several things in there. You see where this is going?

The second extreme, the showroom

This is the other extreme, not everybody will recognize this extreme as fast as they will the other one, but it is there. I like to call this code, the showroom code base. If you dig into your memory you'll find them, those projects where you kept cleaning and cleaning, making the code better. Adding some abstractions, a pattern here, another there, some more layers. Do I see a duplicate line there? Let's create a base class. More then 20 lines in the class? That has to be split up! This feels great when you are starting out, and you'll be able to keep this up for quite a while but then a new guy has to work on the project, or you have to switch to another project for a while. You come back and want to add that same simple input field. Knowing that the code is cleanliness itself you open the class, to find that you should go a layer deeper since it's not really the responsibility of that class. In the next one you go deeper as well, and deeper, and deeper, oh a factory. You just have to figure out which direction you should go and two hours later you think you have found it. But you can't just add this here, because it will appear in several places, you need to abstract it further only to find when creating the abstraction, other implementations need even more abstraction. What has happened here? The code got too clean, it's just like a showroom home, like in one of those catalogs ready to sell. It looks great, but if you look closely, it's not practical to use. There is no place to put your coat, not place to put your drink, no place to put the TV and is that even a full-size couch? But it does look great. This is mainly caused by software design books, articles and blogs that show us unrealistically clean code and give us the wrong goal. Code that follows all the rules and is perfectly consistent and abstracted is unattainable unless there is nobody living there.

Whats in between? Livable code

When you take the analogies I used, between a hoarded living space and a showroom living space, there is the living space that's actually being lived in, the livable space. The beautiful decorated room, with some mess. It's might not be as pretty as the showroom, but at least there is room for your TV with a console, and a couch that sits great. The same thing goes for code, in between these two extremes is the livable code. There is some duplicate code in there, some classes are a little big, but their names tell you just what they do and it's pretty easy to find your way in the code. There may be some places where it's still a mess, but slowly but surely we are changing that mess to make it better. This is the code base we should embrace. This is the code base that feels like home. This is the code base we love to work in!

How to get there

Although most project tend to go more to the hoarder extreme then the showroom extreme, the steps to get the code base to the place you want for them is the same. Follow the following steps to slowly but surely move your code base to a healthy livable place.

1. Don't make it worse!

When you are touching bad code, don't take the easy way out and make it even worse. Don't fix the whole file, but at least make sure it doesn't get worse. This can go both ways, to make the clutter worse, but of you are leaning more against a code base that is too clean, don't just add another layer to it.

2. Improvement over consistency

Five books on a pile and one on the shelf is better then all six in a pile. Every time the code is touched someone can move one of the books to the shelf. If you wait to move all at once, it's not going to happen. This seems hard for a lot of people because a lot of clean coding articles value consistency very much. But remember, the picture that these articles give you, is that of showroom code. While that beautiful code looks like a picture, we need some clutter to feel comfortable. If you look at it practically, having five classes that are a mess and one is like it should be, is better than having six classes that are a mess.

3. Inline everything

Stop putting on stories for refactoring, or taking a week of to fix that one file. It's great that you do this, but it creates other big bangs. You should incorporate this in the way you work every day. Follow the scout rule; Leave the code cleaner then you found it.

4. Be transparent

You need to be able to communicate with everybody what you are doing all the time. So don't hide the stuff that you are doing, like making a file a little better while touching it. Refactoring and cleaning up should be part of everything you do and be open about it. However keep in mind that you should not ask permission to do this! It's part of your job, just but be upfront about what you are doing. And also, don't ask for forgiveness. Again, it's not something you need permission for. You will make mistakes, you will clutter, you will overdo your refactors a lot of the times. It's actually important that you make these mistakes, so you know what to watch out for, you'll learn a lot from them.

Is it all in the code base?

You can probably guess the answer, no it's not. The most important part about software is not the people, it's not the code, it's the system. What I mean with this is that the code base and the team go hand in hand. If the code base is a mess, the team is a mess as well. This would mean that for any problem you have writing software you have two directions you can approach it. You can make changes in your team which will drive changes in your code base, or you can make changes in your code base which will drive changes in your team. For example if your code contains three implementations of "User login", you could consider this just tech debt, but it also represents an issue in communication within your team. Those different implementations should not have been there. So everything you read until now can also be changed by changing the team? Yes it can! Just start by making everybody in the team aware about this principle and start communicating about this topic in reflection to your code base on a regular base and see what happens.

Another important aspect to look at is what software actually is. It moved on from actually building a product and being done a long time ago. Applications keep evolving, we use continuous deployment, the products are never done. Software development should be seen as an experience, involving developers, managers, product owners and the clients, a group creative effort. More like a group of theater people putting together a play, then a group of workers fabricating something physical. While a lot of developers don't think of what they do as creative, the way we work in software nowadays actually makes a lot more sense in a model of creative people collaborating then it does for a manufacturing model. The more we can think about software as an interconnected part of code and people the closer we will get to a code base we're excited to work in. And you can do this, if you build trust with your team members, if you strengthen your connection, the code base will follow.

Architecture and big refactors

So now we know where we want our code to be, what the relation between our code base and the team is and how to get to the code base that we want. One thing I haven't mentioned until now is are solutions a lot of developers yearn for, this big refactors. You see yourself struggling with the awful code day in and day out dreaming of a rewrite. If we could only build it again, we would build it a lot better. Write that jQuery project in Ember, rewrite that Monolith into micro-services. And sometimes it works for a little while, but if you don't change the habits within the team, the clutter will sneak in again. You'll just end up with a tangled network of micro-services just like you had a tangled network of class definitions in your monolith. Or you'll go into the other extreme and over-engineer the whole project and create such a big mess of layers that nobody can't find anything without a full search and rescue crew with them. What is actually more effective is, changing it little by little. And let the people that live there do it. Teams should reach these insights for themselves, there should not be an outside architect role that tells them how to build things. Architects always seemed a great way to keep structure clean, but like I said before, the team that lives in the code base is connected to it, the architect is not. That is like a stylist coming to your house and telling you everything should look. Don't get me wrong, you should really ask help from someone that knows a lot about architecture should you need it, but it should never be the other way around.

Where did this all come from

I didn't think of the principle of livable code myself and don't want to take credit for it. It just made a lot of things fall into place for me and I really wanted to share it with the world, make people think about this. The first place I came in contact with it was on a blog post from Uncle Bob, called Too Clean? I found it a really interesting read and love his conclusion;

We should not be ashamed if our code looks a little bit lived in. On the other hand, we need to be diligent about cleaning up after ourselves; and not let the mess spin out of control.

The article from uncle Bob starts off with the mention of a talk by Sarah Mei, called Livable code, which was my real inspiration. A lot of the things you read above is my vision for software development after hearing her keynote. Should you have 45 minutes to spare, you should really watch it and hopefully, it will inspire you as much as it did me!