DEV Community: James Hood

Beating the Quality Death March with BDD

James Hood — Mon, 15 Oct 2018 17:59:36 +0000

I have yet to meet a software developer or manager who would disagree with the statement, "We should build quality software." However, ensuring quality can feel like an uphill battle. I used to have this Despair.com poster in a frame on my desk, but it unfortunately got lost during one of the many building moves that happen when you work at Amazon. 😂

I've come a long way in my career when it comes to software quality. At my first job, we didn't even write automated tests at all. Every change was tested manually at the time you developed it *shudder*. I think it's because I've seen that side of software development that I've become such a huge quality advocate. Now, I've led multiple teams at Amazon to develop critical services and web UIs where each commit goes through a full continuous deployment pipeline with automated unit and integration tests and other safeguards to ensure quality all the way to production, so I have a lot to say on the subject.

What I wanted to cover today is a common developer question I get: How do you ensure integration tests get written for your software?

At this point, most teams/companies are on board with writing unit tests. It's relatively easy to enforce a rule on your team that all functional changes must include unit test changes in the same commit/PR. While code coverage is not a perfect metric, it is a decent measurable indicator that this policy is being enforced.

However, unit tests alone are just not enough. Ensuring integration tests are written can be tricky though. The key problem is you usually can't test a service or website until you've built enough of it to process an end-to-end request. If you don't think carefully about how you're going to structure the delivery of the features of your product, you can end up saving all integration tests until the very end of the project where it's common for them to get deprioritized due to deadlines.

I wanted to share a solution to this problem that I've tried on multiple teams and have found to be extremely effective in solving this problem: Behavior-Driven Development (BDD).

What is BDD?

BDD is a development methodology derived from Test-Driven Development (TDD) that focuses on defining feature requirements in terms of concrete scenarios before developing the solution, then turning those scenarios into executable tests that verify the solution meets the requirements.

Let me give you an example. Let's say you're writing a simple banking service that allows you to open accounts and make deposits and withdrawals. You might receive a bullet-list of requirements for this service, e.g.,

Users can open and close a checking or savings account with an initial balance.
Users can deposit or withdraw money from an open account as long as they don't overdraw the account.
Savings accounts have a minimum balance of $100.
...

These high-level requirements are a starting point, but when you use BDD, the next step is to break up this project into features and then write detailed scenarios for each feature using simple language that can be read and understood by both developers and key stakeholders. For example, the above requirements would be described in the following feature file:

Feature: Basic Account operations
  As a customer
  I want to be able to open checking and savings accounts, and credit and debit my accounts
  So I can buy stuff online^H^H^H^H^H save my money

Scenario: Customer creates a checking account
  When the customer opens a checking account with a $0 balance
  Then a new checking account should be created
  And the checking account should have a balance of $0

Scenario: Customer creates a savings account with the minimum balance
  When the customer opens a savings account with a $100 balance
  Then a new savings account should be created
  And the savings account should have a balance of $100

Scenario: Customer creates a savings account with less than the minimum balance
  When the customer opens a savings account with a $50 balance
  Then the call should fail

Scenario: Customer credits their account
  Given the customer has a checking account with a $0 balance
  When the customer credits their checking account $10
  Then the checking account should have a balance of $10

Scenario: Customer debits their account
  Given the customer has a checking account with a $50 balance
  When the customer debits their checking account $10
  Then the checking account should have a balance of $40
...

The format for these scenarios is very simple. The only real structure is that steps starting with the word "Given" are preconditions or assumptions, steps starting with "When" are the action that's taken, and steps starting with "Then" are assertions on side effects of the actions taken. "And" can be used to chain multiple steps together.

Once the feature files are reviewed and approved, you build the feature. After that, tools like Cucumber allow you to map each step in the feature file to a function in code and run the feature file as an executable test. Put a step in your CI/CD pipeline to execute the feature file and you now have automated acceptance tests for your feature that are run everytime you make a change to your codebase!

Why BDD works

Like I said, I've used BDD on multiple teams now and it's been incredibly effective. Here's what I like about it:

First, writing the feature files increases collaboration, ensuring developers, management, and key stakeholders are all on the same page about what the expected behavior of the system should be before it's built. The feature file example I gave above may seem simple to write, but you'd be surprised how many questions come up while trying to write one. Developers are awesome at thinking of all kinds of hairy edge case scenarios, and having to write them out drives requirements clarification questions back to management/stakeholders. This removes a lot of wasted time and effort lost due to requirement misunderstandings that are very expensive to fix later in the process. I find after management has been through a few cycles of this, they love it, because they have confidence that their team is building the right thing and it will work (and continue to work!) once it's built.

Second, it gives developers a systematic way to ensure automated integration tests are actually written. With unit tests, you set a rule that tests should be written for each commit/PR. With integration tests, you set a rule that tests should be written for each feature. Break your project into small, incremental features, and now you are ensuring integration tests are written incrementally as the software is developed.

With both management and developers seeing benefits, you don't get in situations where you deprioritize integration tests, because priority discussions happen at the feature level and writing integration tests is incremental rather than being this giant, insurmountable task at the very end of the whole project when you're feeling pressure to launch.

BDD Pitfalls

Hopefully at this point, I've made the benefits of BDD pretty clear. Maybe you're interested in trying it on your own team. Great! I highly recommend it. However, I've also used this in practice on several teams and wanted to cover some common pitfalls to watch out for when going down this path.

Pitfall 1: Features are too big

Since integration tests are written for each feature, in order to develop integration tests incrementally, you have to break down projects into multiple features. This practice has many other benefits as well, so I recommend it regardless of whether you're using BDD or not, but for BDD specifically, if your "feature" is the whole product or project, then you're back in the same boat of saving writing integration tests until the very end where they're likely to get cut due to time pressure.

So if you go to write the feature file and it feels like a daunting, never ending task, your feature is probably too big and needs to be further broken down. If you're practicing Scrum, think of a feature as something that should fit inside of a sprint. If you're not practicing Scrum, my team's sprints are 2 weeks, so you can use that amount of time as a guideline. 😊

Pitfall 2: Using BDD for unit tests

The first book I read on BDD was BDD in Action. It does a great job of explaining the benefits I describe above in more detail. However, it also advocates for not only using BDD at the acceptance test level as I describe above but at the unit test level as well. There are many things I enjoyed about that book, but I think using BDD at the unit test level is a complete waste of time.

When you think about it, what you're doing with BDD is maintaining a mapping between a human-readable description of how your software should work and the code that actually ensures it works this way. There is overhead in maintaining this mapping, but it's worth it as long as someone other than a developer reads the human-readable description. Why would a manager/stakeholder waste their time reading a feature file at the unit test level? They shouldn't, since they should be more concerned that the system behavior meets the requirements and less concerned about the low-level implementation details. I've also never seen developers read feature files at the unit test level, because it's usually easier for them to read the test code itself. So these mappings are being maintained for absolutely no benefit. Just don't do it.

Pitfall 3: Having non-developers write feature files

Many BDD books advocate for this idealized world where product managers or stakeholders write the feature files directly. I think the idea is it will force them to actually think through the detailed requirements before handing them to developers. I've never seen this actually work in practice. While the feature file language may seem very open-ended and expressive to a developer, it is still a pretty constraining format for a product manager to use, and they tend to be much more comfortable writing actual prose in a document editor like Word.

Also, as I'll cover below, since scenario steps have to map to executable code, writing feature files becomes a bit of an art where you not only specify requirements, but do it in such a way that the underlying step implementations are maintainable. A product manager will quickly become frustrated when developers code review their feature file and have all kinds of comments related to implementation of the feature scenario steps and fewer comments about the actual scenario content.

Finally, I think having developers translate requirements documents into feature files really helps them internalize the requirements of the feature. It also empowers them to ask many questions about different edge case scenarios that the product team may not have considered. If a developer was just handed a feature file, they may not scrutinize it or think through it the same way they do when they write it themselves.

Pitfall 4: Scenarios contain too many low-level details

When developers write feature files, they're also thinking about the underlying step implementations. Tools like Cucumber provide rich features for parameterizing steps, even allowing tables of data to be passed into a step implementation function. It's very tempting as a developer to try to minimize duplication in the step implementation functions by pushing more details into parameters which show up in your feature file.

I recommend using the step parameters feature of Cucumber very sparingly and to never use more advanced features like data tables. You should optimize for readability of the feature files by non-developers, which usually means pushing lower level, superfluous details about the scenario down into the step implementation. Do yourself (and your management/stakeholders) a favor and DON'T write scenarios like this:

# Don't do this!
Scenario: Customer creates a checking account
  When the customer opens an account with the following data
  | First Name | Last Name | Type     | InitialBalance |
  | This       | Sucks     | CHECKING | 0.00           |
  Then the new account should have the following data
  | First Name | Last Name | Type     | Balance |
  | This       | Sucks     | CHECKING | 0.00    |

Hopefully it's clear that all this additional detail takes away from the readability of the scenario. While precise, your product manager and stakeholders' eyes will glaze over as they try to parse the meaning out of the unnecessary details.

Pitfall 5: Step implementations can turn into spaghetti code

Good patterns for step implementations is probably worth a separate blog post, but I'll try to condense this down to the essentials. BDD tools allow each step in a scenario to map to a function call. However, for a given scenario, it's very common that the steps need data that was collected in a previous step. This means your functions will have to share data between them. Ok so maybe you can make a class to hold your step definition functions and use instance variables to share the data between functions. However, you usually want to group your step definitions into separate logical groups so you don't have a single monster class full of step definitions, and you'll inevitably end up writing scenarios that use steps spanning more than one class. So basically, you need some global data that's shared between step implementation functions. If you don't have an organized way to manage this global data, and instead set it somewhat randomly as you implement steps, you're going to end up with a spaghetti mess that's very difficult to maintain.

The solution I've found to work well is to have a single class that manages writing the global data. It can be read from within any step implementation function, but it's only written in one specific place. For example, when using BDD to test service APIs, I create a proxy class that wraps calls to that service. The proxy class saves any relevant information that may be needed by step implementation functions. Here's an example of what that might look like in Java:

public class BankingServiceProxy {
    private final BankingService service;

    // ...

    public void openAccount(OpenAccountRequest request) {
        BankAccountTestData.clearLastException();
        BankAccountTestData.setOpenAccountRequest(request);
        try {
            OpenAccountResponse response = service.openAccount(request);
            BankAccountTestData.setAccountId(response.getAccountId());
            BankAccountTestData.setOpenAccountResponse(response);
        } catch (Exception e) {
            BankAccountTestData.setLastException(e);
            // suppress exceptions. Step implementations will check last exception to assert on call failures
        }
    }

    // ...
}

Then your step implementation for When the customer opens a checking account with a $0 balance looks something like this (annotations are specific to Cucumber-JVM):

public class AccountSteps {
    private final BankingServiceProxy proxy;

    // ...

    @When("the customer opens a (.*) account with a \\$(.*) balance$")
    public void the_customer_opens_an_account(String accountType, int initialBalance) {
        OpenAccountRequest request = new OpenAccountRequest()
            .withAccountType(accountType)
            .withInitialBalance(initialBalance);
        proxy.openAccount(request);
    }
}

Note that the step implementation leaves saving global data to the proxy class. Then later steps can perform verifications on the saved data:

    @Then("a new (.*) account should be created")
    public void a_new_account_should_be_created(String accountType) {
        assertThat(BankAccountTestData.getLastException()).isNull();
        assertThat(BankAccountTestData.getOpenAccountResponse()).isNotNull();
        assertThat(BankAccountTestData.getOpenAccountResponse().getStatus()).isEqualTo(BankAccount.Status.OPEN);
        assertThat(BankAccountTestData.getOpenAccountResponse().getType()).isEqualTo(AccountType.valueOf(accountType.toUpperCase()));
        // ...
    }

    @Then("the call should fail")
    public void the_call_should_fail() {
        assertThat(BankAccountTestData.getLastException()).isNotNull();
    }

Pitfall 6: Test slowness

As your suite of feature files grows, test run times can increase quite a bit. Frequently, the tests are doing similar setup/teardown operations, which can take time. I honestly haven't found a silver bullet solution to this problem, because at this point, the benefits of BDD still far outweigh issues of long-running acceptance tests enough that we haven't really dug into a good solution to this.

However, here are a few tips:

You can chain multiple When/Then groups into a single scenario to save on redundant test setup steps. Don't go so overboard with this that your feature file becomes one long scenario. I tend to favor separate scenarios, but sometimes it makes sense to combine them. Use judgement to make sure you're not compromising the readability of the feature file to non-developers.
Make your step implementation global data class thread safe and use unique test data for each scenario so you can run your scenarios in parallel. I haven't found a parallel test runner for Cucumber that I can just plug into our current build setup without much effort so I don't have a good recommendation there. If others do, please comment! A quick and dirty solution is to setup your automated CI/CD pipeline to run each feature file as a separate test run so they can be run in parallel.

Conclusion

When used correctly, BDD is a very effective way to increase team collaboration, remove waste from the development process, and ensure integration tests are built incrementally along with every feature of a product. Hopefully this post sparks your interest in giving it a try!

Sprint Demo/Retrospective Best Practices

James Hood — Sun, 27 Aug 2017 21:03:58 +0000

This post was originally posted on my blog

Scrum is my personal development methodology of choice. A key reason I like Scrum so much is because it allows you to frequently practice estimating software and evaluating how well you did after the sprint is complete through retrospective. I frequently call out late software as the most common first step down the path of the Vicious Cycle of Software Development, so I'm passionate about helping teams get this right.

Often, teams I've spent time with aren't sure what to talk about during retrospective meetings. They may discuss what went well and what didn't in the sprint in a freeform way. I've seen cases where they just play up accomplishments and minimize failures. Or when discussing failures, they may use the time as a venting session and fail to exit with any concrete plans for preventing making the same mistakes again next time.

Here's the structure/format I've found that has worked well across multiple teams. On the last day of the sprint, have a combined meeting for demo and retrospective. I generally schedule 2 hours for this and then we can always finish early if we run out of things to talk about, but adjust as needed for your team.

Demo

Sprint Demo is where you get to share the accomplishments from the sprint with your team. A common pitfall I see is for developers to be reluctant to demo their accomplishments, because they think the accomplishments are not significant enough to impress management or stakeholders. Don't think of sprint demos as being for management and/or stakeholders. I always coach devs to give demos to their teammates, not management. This helps them feel less nervous and removes the pressure of everything having to go perfectly. Sprint demos should not be thought of as a vehicle to impress. I've found they are much more valuable as a knowledge sharing, and even bug finding tool.

You may not realize it, but there are probably people on your team who have no idea what you're actually working on. This can be surprising to some because everyone participates in estimating all sprint stories during planning. But the level of understanding a person needs to contribute to a story-level estimate is different than having a deep understanding of the feature. A demo can really solidify the concept for other devs on the team. This is helpful in a few ways. It increases everyone's sense of ownership over all of the team's software, and in teams that practice devops, it provides some of the cross-training necessary to support the team's software in production.

Demos can also be great for finding bugs. If you remove the expectation that the demo has to impress people, you'll start getting a lot of questions like, "What happens if you click on X, or enter Y input?" Ideally everything works correctly and the team just learns about how the software is expected to behave in those circumstances, however sometimes the software doesn't work correctly and can be fixed before causing any customer-facing issues.

Remember, it's important to celebrate accomplishments, no matter how small they may seem relative to the entire project. Integration tests and backend APIs may seem like they're not worth demoing, but demo them anyway.

Retrospective

Retrospective is, in my opinion, the most critical part of practicing Scrum. It's the key opportunity to improve your development process for future sprints. I've found the best way to start the discussion is with objective data. The scrum master should prepare the following data prior to Retrospective (ideally through an automated script) and lead off by presenting it to the team:

Number of stories complete/incomplete. Be very strict about this. If it's not fully complete, do not count it as complete.
Point completion % = (sum of estimated points on completed stories / sum of estimated points on all committed stories * 100%). It's important to note I'm only paying attention to the estimated points here. This gives better visibility into how well your estimate ended up matching reality.
Avg velocity = (sum of estimated points on completed stories / number of available dev days in the sprint * 100%). This is your measured velocity per person, per available day. It's important to note that number of available dev days is computed as part of sprint capacity planning. So this is the number of days in the sprint * number of devs minus any planned days out of the sprint (vacation, holidays, oncall, etc). For our team, we also subtracted the last day of the sprint for all devs to account for time spent on sprint overhead (planning/demo/retrospective).
Trending Avg velocity = mean avg of the avg velocity values for the last 6 (or whatever you choose) sprints. This value should be used to set your capacity for next sprint.
Emergent stories (stories created after planning). Not a metric, but these should be discussed during retrospective. Why weren't these accounted for during planning? What can we do to minimize emergent work and foresee this kind of work in future sprints?
Ballooning stories (stories that took longer than estimated). These should be discussed. Why did this take longer than expected? What can we do to improve our estimates for this kind of work in future plannings?

Gathering this data forces your team to take an objective look at how successful the sprint was. The goal, of course is all stories complete and a completion % of 100%. On the teams I've been on, the completion rate has always started out much lower than this. This starts a great discussion into what happened. It's important to understand that retrospective should not be a blaming/shaming session. If management and team members have put time and effort into creating a healthy, psychologically safe team environment, chances are good that everyone's doing their best. Focus on finding the root causes of the problem and then determine concrete mechanisms you can put in place to prevent the mistakes from being repeated.

Examples of actions taken in the past are to add items to existing checklists or create new checklists. I'm a strong advocate of checklists as a simple, lightweight mechanism for ensuring consistency. For example, a story may have ballooned because you completely forgot to call out integration tests during planning, resulting in underestimation. A feature checklist can help your team consistently consider creating subtasks for all aspects of a feature. Another learning from the past was if we are adopting new technologies/frameworks, we may need to lower capacity (either at the individual or team level) to account for ramp up time. When we ran into this on one team, our retrospective action item was to add an item to our capacity planning checklist to ask this question in the future, prior to planning.

Another best practice is to review action items from past retrospectives and determine if the changes you tried are actually helping or not. It's very important to understand that modifying your process does not always mean adding something new. Sometimes tweaking something or removing it completely is best. That way your process stays lean and doesn't start to become the dreaded bureaucracy.

If you don't feel you're getting much value out of your demo/retrospective meetings or worse, you don't even have them, I encourage you to try out this format and see if it helps. If you have any other tips on demos/retrospective, please add them to the comments!

Plan All of the Things!

James Hood — Wed, 16 Aug 2017 15:06:12 +0000

This post was originally posted on my blog

In my previous post, Bad Habits We Learn in School, I touched on the fact that software is notoriously difficult to estimate accurately, and when we are wrong, we tend to be wrong on the side of underestimation (sometimes by as much as 4-6 times!). This is hugely problematic, because severe underestimation is the first step towards late software, which puts you right onto the path of the Vicious Cycle of Software Development. So learning how to increase our accuracy in estimating software is a vitally important skill to develop.

In this post, I want to tackle one common root cause of underestimation: simply forgetting to account for critical work while planning out features or a project. It sounds silly, but it happens far more often than you think.

Feature-level Planning

First, let's start with feature-level planning. When planning the implementation of a feature, it's important to explicitly take into account all possible aspects. Here's how I picture the anatomy of a feature:

Not every feature requires all of the above pieces depending on its size and scope. However, I've seen teams repeatedly forget to plan for many of the quality measures listed above. Over time this compounds, leading to huge gaps in quality and technical debt that is a hard sell to the business to go back and address after the fact. So planning for quality up front saves you a lot of pain down the road.

Let's take a look at these pieces in more detail.

Core Implementation

The first major category of a feature is the core implementation, made up of functional code, unit tests, and integration tests.

Functional code is the code that implements the desired feature behavior. Unit tests test your functional code in isolation. Dependencies are mocked allowing for several edge cases to be covered. Unit tests are your first line of quality defense and should never be skipped, because they're so easy to write. Some teams push for using Test-Driven Development (TDD), which ensures unit tests are always included. I've found some developers can be really resistant to TDD, so I side-step that argument by instead having a rule that functional code and unit tests must always be included together in the same commit. You can write the functional code first and then your unit tests or the other way around. I really don't care as long as they're both there.

Integration tests are more complex tests where you test your functional code against other system dependencies. There are many flavors of integration tests, such as functional tests, system tests, acceptance tests, etc. For the sake of this post, I'm just using the term integration tests to cover any and all of these kinds of tests. Unlike unit tests, it's not usually practical to write integration tests in the same commit as your functional code changes. As a result, it's embarrassingly common to forget to account for time spent writing integration tests when planning your feature.

In recent years, I've become a fan of Behavior-Driven Development (BDD) as a means to address this, at least at the acceptance (user-facing) test level. BDD feature files are a nice way to document expected behavior of the feature as a whole, and ensure what you plan to build is what the business is expecting. Since you write feature files up front during requirements clarification, it's easier to remember to plan for actually implementing all or a subset of them as automated acceptance tests later on.

Ops/Launch Support

The second major category of a feature is operations and launch support, made up of metrics, alarms, ops dashboards, and feature toggles.

Metrics give you visibility into how your code is working in production. I have this as a separate component of a feature, however generally, metrics are something I try to think about and insert into the functional code as I'm building a feature. Automated alarms are used to notify you if your code is behaving outside of normal error or latency thresholds. Ops Dashboards give you an at-a-glance view of the health of your system by displaying key metrics. For major feature launches, it can be very useful to have a dedicated dashboard of key metrics to ensure the feature is working correctly. Smaller features may not require a dedicated dashboard, but you should consider whether this feature warrants adding new graphs to any existing dashboards.

Feature toggles are a way to decouple deployment of code changes from release of a feature. These are flags that are embedded in your functional code, allowing you to silently deploy your feature into production. Then when you're ready to actually make the feature go live, you toggle the feature on (usually by updating a toggle value in a database). Feature toggles are powerful, because they allow you to nearly instantaneously launch your feature, and--equally important--unlaunch your feature if things go wrong, which is far faster than deploying new code to your production systems. Since you have to write your functional code such that it can work one way with the toggle off and another with the toggle on, it's important to consider whether this is necessary for your feature and plan for it in advance.

Don't forget!

One thing I've found to help in planning a feature is to use a literal checklist of the above items during backlog grooming/sprint planning to ensure you're considering which ones are necessary and then explicitly create stories/tasks for them. Similarly, I've seen teams who leverage templates in their sprint/issue tracking system include these items in their story templates.

Project-level Planning

Like I said before, routinely forgetting things at the feature-level compounds over time causing huge quality problems. However, there are also similar misses that can happen during project-level planning. At larger companies, project-level planning is often important for trying to determine rough counts of how many people will be needed to deliver a project by a certain date. These kinds of estimates are far less accurate due to the huge amount of ambiguity involved when trying to estimate a project that will take several months. The general strategy is to try to enumerate as many big pieces as you can think of and then give them very rough sizings.

Brainstorming these big pieces of a project is another opportunity to forget to account for major pieces of work that are required. In this case, I find teams generally do a good job of thinking of the features that make up the core customer-facing functionality of their project. It's the non-customer-facing features that tend to be forgotten. Here are the ones I see get forgotten on a fairly regular basis:

Disaster planning - For projects with high availability requirements, it's important to go through a disaster planning exercise where you walk through key dependencies and exposures in your architecture and brainstorm mitigations. These mitigations often involve building some set of tools to help in disaster recovery scenarios.
Data Backup/Restore - Setting up backup/restore procedures for your datastores is easy to forget, but very painful if you need them and they aren't there...
Administrative tools - You may need to build special tooling for product owners or customer service associates to use to handle problem scenarios (process returns/refunds, ban/block misbehaving/fraudulent users, etc).

Again, the above are not always required depending on the size and scope of the project, but it's extremely frustrating when you need them and realize you forgot to account for them at all and must now scramble with your existing resources to fill the gaps.

Accurate estimation of software is tough for many reasons, but hopefully these tips will help you fight underestimation by at least considering all aspects of features and projects. Do you have anything to add to my lists? Feel free to share in the comments!

There is No U in CRUD

James Hood — Sat, 29 Jul 2017 02:59:34 +0000

This post was originally posted on my blog

REST is built around the concept of resources, represented as URIs. Making an HTTP call specifying an HTTP verb and a resource URI takes an action on the specified resource. Most REST frameworks provide generators where you specify a resource name, and the framework generates scaffolding around it. Unfortunately, many of these generators use the CRUD model (Create, Read, Update, Delete) as the default starting point. The resource is defined as a bag of attributes, using something like JSONSchema or a language-specific data object definition and then method stubs are generated to create, read, update and delete that resource.

While it's good to give developers a starting point to work from, I have a big problem with using CRUD as that starting point for an API. The U in CRUD is my least favorite, although I'm not fond of the other letters, depending on the use case. But let's talk about U. Generic update methods allow the client to update any field of the resource and then overwrite the existing version with the new version. However, if you allow clients to do this, your service API provides little value on top of whatever underlying datastore it's using. One of the key value-adds of a service layer is enforcing business constraints on top of the underlying data, and resources always end up having business constraints.

But can't we add business constraints to our update method? Let's use a simple bank account resource as an example and see what happens. First, clients shouldn't be able to call an API and just update their account balance to whatever they want. There might be a minimum balance for the account. Ok, so you add some checks to the update method such that if the account balance value is changed, it must be within a specified range. Problem solved? Well, no, not really. Any balance adjustment should be recorded as some kind of transaction, right? Was this a credit? A debit? A transfer? What if the client tries to change the account number? Is this allowed at all? Could that break other data relationships? It's not hard to see how your update method implementation can quickly turn to spaghetti code the more questions we ask. I've seen teams go down this path where their code tries to infer what the intent of the client is from which fields are changed, and the code ends up a total mess.

So what's the alternative? Personally, I'm a big fan of Domain-Driven Design (DDD) for designing any kind of API. DDD is based around the idea that software should be modeled after the real-world problem being solved. It creates a language for describing software in terms of key business objects called Entities or Aggregates. It also defines terms such as Services, Value Objects, and Repositories, which work together to solve problems in a particular business domain, or Bounded Context in DDD terms. You don't have to use REST to use DDD, however, I find it works especially naturally with REST APIs, because REST resources map very well to DDD entities.

So what does all this mean? It means your API should be centered around domain objects and the business operations they provide. Business operations are the key alternative to the generic update method and all of its pitfalls. Let's use the banking example from earlier to illustrate.

For a banking API, an obvious domain object (or entity in DDD terms) is an account, which models a bank account. Rather than following the CRUD model for accounts, we should define specific business operations that make sense to perform on a bank account. Here's a good starter set of write operations:

Open - open a new account.
Close - close an existing account.
Debit - remove money from an account.
Credit - add money to an account.

These operations are specific and can enforce certain business constraints. For example, we may not want to allow crediting a closed account, and we can enforce our minimum balance check as part of the debit operation. On the read side, we can also provide specific queries that match our client use cases:

Load - load single account by its account id.
Transaction history - list transaction history for an account.
Customer accounts - list accounts for the given customer id.

Now that we know what our business operations are, here's an example of mapping them to a REST API:

POST /account - open a new account.
PUT /account/<accountId>/close - close an existing account.
PUT /account/<accountId>/debit - remove money from an account.
PUT /account/<accountId>/credit - add money to an account.
GET /account/<acountId> - load single account by its account id
GET /account/<accountId>/transactions - list transaction history for an account.
GET /accounts/query/customerId/<customerId> - list accounts for the given customer id.

This looks a lot different than a basic CRUD API, but the key is that the operations allowed are specific and well-defined. This results in a better experience, both for the service implementor, as well as the client. The service implementation no longer has to guess what business operation is implied based on which attributes are updated. Instead, the business operation is explicit, which leads to simpler, more maintainable code. On the client side, it's much clearer exactly what operations can and cannot be performed. If the API is documented well, for example, using a Swagger definition, it will also be very clear what the constraints of each API are.

Defining your APIs this way requires more up front thinking than a simple CRUD generator, but I think this is a very good thing. If you're planning on exposing your API as a public endpoint, you are going to have to support that API for a very, very long time. Basically think of it as forever by software standards. I always encourage teams to take time up front on the things that are hard to change later, and APIs are the first example I give.

So resist the urge to follow the CRUD model for service APIs (REST or otherwise). Instead, use DDD to define your API in terms of domain objects and the business operations that can be performed against them.

If you want to see more examples of defining APIs in terms of domain objects, I recommend checking out the Amazon Web Services APIs. Look up the developer guide for any service and they should start with a section labeled "Key Concepts" or something similar. There, they describe the conceptual domain objects of the service. For example, S3 defines objects like Buckets, Objects, and Permissions. Kinesis has streams and shards. Once you understand the domain objects of a service, look at the API reference and skim the list of APIs for that service. What you'll notice is the API is built around these domain objects, making it more intuitive to understand and use.

Hope this helps!

Productivity Tips

James Hood — Mon, 17 Jul 2017 01:25:43 +0000

This post was originally posted on my blog

One of the new developer tips I gave in my previous post, Tips for New Software Developers, was

Tip 3: Set boundaries on work hours from the very beginning...but work hard and efficiently during those work hours.

I talked about how I've worked hard to become more and more efficient but didn't provide any additional details. In this post, I'll cover the key productivity tips that work well for me. But first, it's important to establish the core principle that underlies all of these productivity tips:

The key to high efficiency is to **consistently* maximize the amount of consecutive hours you spend working on the things that matter most*.

This is by far the most important takeaway from this post. All of the tips I'll give are just ways I've found to achieve this. At first this may sound simple, however doing all 3 of the bolded parts takes a lot of discipline and practice.

Productivity Tip 1: The TODO List

I've already written about the way I manage my TODO list so I won't spend too much time on it. However, looking at the core principle of productivity above, you can see that it ensures you're thinking about priority on a daily basis, which covers the "consistently" and "the things that matter most" parts of the principle.

One more note on this before I move on: In order to be highly productive, you have to exercise ruthless prioritization. I like that phrase, because it gets across how aggressively you need to prioritize. One analogy I often use is to picture a ladder. You can expend a lot of energy climbing the ladder as quickly as you can. But if it's leaning against the wrong building, all that effort is waste. Your time is a resource. Spend it wisely.

Productivity Tip 2: Minimize Interruptions

The TODO list helps me know what I need to work on. The final piece of the core principle is to spend "consecutive hours" on these important tasks. In the office, interruptions are the primary obstacle to achieving this. Here are the ways I minimize interruptions:

We have an open office floorplan at work, which can make the office area noisy and full of interruptions. If I have a lot of focused coding work to do, I'll work from home when I can. Otherwise, I have a pair of noise-cancelling headphones that I use. I really like to listen to musicForProgramming.
I turn off Outlook notifications and I close the program. I only open it at specific times of the day and check my email in batches (I try to carve out a 30-45 minute block of time to check and respond to email). Some people try to act like email is a realtime communication channel. Don't be fooled. It's not one.
I block out large sections of my calendar at the times when I'm most productive (for me it's in the morning when I first get to work). During that time, I'm careful not to start by checking email. That's an easy way to squander your most productive time. Instead, I immediately jump into my prioritized list that requires the most brain power and focus. I save checking email for later in the day.
Since I'm in a senior position, often many different people might approach me with questions, which can be a terrible source of interruptions. What's worked well for me is to schedule explicit times when I'm available, e.g., office hours. If people still come to my desk I'll politely remind them about office hours and ask that they bring it to the next session. Of course sometimes things can't wait or it's a really quick answer, but this helps reduce the number of random questions.

Productivity Tip 3: Meetings

Ah meetings. No developer I know likes them, but they can be a necessary evil of the job. Here are my tips for meetings:

Do not be afraid to decline meetings. If it's not a good use of your time and you don't have something significant to contribute, skip it.
For the meetings that are necessary, do your best to schedule them in batches. Nothing kills productivity like a swiss cheese calendar day (30 minute free blocks scattered throughout the day).
Bias towards email as a substitute for meetings. Sometimes this ends up being more time-efficient.
When you call a meeting, make sure it is run efficiently by doing the following:
1. State a clear objective ahead of time (it should go into the meeting invite). What are we trying to accomplish? What specific topics need to be discussed?
2. Start on time and keep the group on task. If the objective is reached before the end of the allotted time, end early. There's no need to stretch the meeting out just because you scheduled a full hour.
3. Ask people to close their laptops so they're actually present for the discussion. It can feel a bit uncomfortable to ask this, but people only think they can multitask. If their laptop's open, they're missing most of the conversation and are probably not being very effective at what they're trying to do on their laptop. I tell people, either attend the meeting or don't. They're not doing anyone (including themself) any favors by being half present.
4. Take notes during the meeting and email them out afterward with key decisions that were made and action items if any. Action items need an owner and a date or they're not going to happen.

Productivity Tip 4: Learn a scripting language and automate

Early on while I was in college, I learned shell-scripting, and it has been an invaluable skill throughout my career. Being good at scripting gives you the power to automate mundane tasks away very quickly. Because I've used shell-scripting for so long, I always find it surprising when I see developers manually performing repetitive tasks that could easily be scripted. This happens much more frequently than I'd like, especially when working on "non-coding tasks," like design documents, updating bug trackers, etc.

I've found myself saying to people, "Don't forget you're a developer! Be a developer!" What I mean by that is developers should not restrict their coding skills just to production code. Computers enhance productivity tremendously, and being a developer means you have the ability to make computers solve customized problems for you. Obviously, there's a bit of quick cost/benefit judgement that should be employed. Don't spend an hour writing a script when you only have to do the task once and it takes 5 minutes to do manually. However be aware that there's also a long-term benefit in building your scripting skills to the point where you can whip out a script in 10-15 minutes that saves you hours.

Productivity Tip 5: Utilize your commute to learn

I have a pretty long commute to and from work, and it turns out to be the perfect time to listen to tech talks, audiobooks, and podcasts. This turns a normally tedious part of my day into an opportunity to learn about various tech topics.

Productivity Tip 6: Exercise and get enough sleep

Last and certainly not least is to exercise and get enough sleep. It can be extremely tempting to work late into the night. You feel like you'll squeeze a few more hours out of the day and get ahead. But what I've learned and relearned (and relearned) in my career is that it always ends up evening out or setting you back. You may get a few hours of work in in the middle of the night, but then you can hardly function for the entire next day. Also, when your brain is exhausted, you take longer to do tasks that would normally be pretty easy. This can have psychological consequences since it can trigger Imposter Syndrome anxiety. Finally, when you're exhausted, you tend to make mistakes that can create more work for yourself. It's just not worth it.

This tends to be a very difficult one for developers to follow (me included!). I think it's because of programmer stereotypes (the image of the lone coder working through the night), but I can't stress enough how important this one is!

Hope this helps! Again, these are just the tips that have worked well for me, but anything that helps you achieve the core principle I described at the beginning of this post works. Give them a try and let me know what you think. Do you have additional productivity tips? Let me know in the comments.

Don't Worry About Failure...Embrace It!

James Hood — Sat, 08 Jul 2017 04:25:46 +0000

This post was originally posted on my blog

A member of a team I've been mentoring recently confided in me that he had decided it was time to move on and try working on another team. Amazon has a very friendly culture towards transferring internally between teams, so of course this wasn't a problem. He had identified two potential teams and wanted my help choosing between them.

Team 1 is in a high-profile area, working on cutting edge technology that is pushing the field forward. It would be challenging and he would have a lot of responsibility and big projects to lead. Team 2 is a team in a well-established space with a more incremental charter. Team 2's manager didn't have a specific big project in mind for him and he would start out with smaller tasks and go from there.

He asked which team he should switch to. I said I knew which team I would choose, but it wasn't my decision to make. After a bit more back and forth, it finally came out that he was more excited about Team 1, but he was worried that he might fail. He said, "What if I build a solution for a huge project and it doesn't work?" He thought maybe he should go with Team 2 even though he was not as excited about it so he could gain more experience and see more examples of production software before working on a larger project.

I told him he had done great work for our team over the last few years and has shown the ability to learn quickly and grow. So if the only thing holding him back from going to Team 1 was a fear of failure, then he should absolutely go to Team 1! The challenge of Team 1 would lead to much faster personal growth and I wasn't worried about his ability to tackle such a challenge.

I also showed him this wonderful quote, which he liked a lot:

"Every great developer you know got there by solving problems they were unqualified to solve until they actually did it."
-Patrick McKenzie

In the end, he decided to go with Team 1. I was happy for him and wished him luck.

I think it's so important not to let fear of failure hold you back. One of the most important and rewarding aspects of life is pursuing and achieving personal growth in whatever interests you. However this only comes from challenging your limits. When you're forced beyond your comfort zone, you are guaranteed to struggle at times, but that's ok! In fact, it's more than ok. It's a perfectly natural and normal part of learning.

A long time ago, a coach taught me that there are 4 stages to learning something new:

You're doing it wrong and you don't know it.
You're doing it wrong and you know it
You're doing it right and you know it
You're doing it right and you don't know it

In these stages, "it" can be any kind of personal growth or learning, be it a sport, a profession or being a good partner in a relationship. It also isn't just something big like learning to be a good developer. That is a career path, which is a journey filled with many smaller "its" that need to be learned with these same stages.

The thing I love about these stages is that they're simple, yet quite profound. In stage 1, you're blissfully ignorant that there is something you need to improve upon. In stage 2, you become aware of your shortcoming, but cannot figure out how to overcome it. With enough thoughtful persistence, you'll reach stage 3, where you finally figure it out, but it takes effort to do it each time. Then, at some point, you'll realize you're at stage 4 where you now do it without even consciously thinking about it.

To me, the critical stage is stage 2. If you're pushing your limits, stage 2 is easily the most frustrating stage. When you're making an honest effort to do something and it's not working, it's easy to look at those attempts as failures. It's during this stage that you will start to get negative, defeatist thoughts in your head like, "I'm terrible at this!" or "I should have gotten this by now!" or "I'll never get this!" or "Everyone else gets this except me!"

If you give too much weight to these thoughts, you'll feel a strong urge to quit. However to me, the only true failure happens if you give in to these thoughts and decide to give up. Instead, be kind to yourself. Remind yourself that it's ok to feel frustrated and this is a perfectly natural way to feel at this stage. Treat each failed attempt as a learning experience. What went wrong that time? What can I do differently next time? Being kind to yourself and treating setbacks as learning opportunities is key to getting through stage 2 and onto a path of positive personal growth. When you finally get to stage 3, acknowledge it and celebrate your accomplishment! Then keep going until you get to stage 4.

Hope this helps! What are your tips or tricks to getting through frustrating times in your career or personal life? Do you have a story of a time when you persisted and made it through something that seemed impossible at first?

Bad Habits We Learn in School

James Hood — Wed, 31 May 2017 21:35:18 +0000

This post was originally posted on my blog

One of the most pervasive problems in the tech industry is poor work-life balance. Long hours and "death marches" can seem like the norm, especially when you're fresh out of college and hungry to prove yourself and jumpstart your career. The industry as a whole has a long way to go to address this common problem. However, in my experience, I've found that it is far more common to see individuals or teams unknowingly make poor decisions that quickly accelerate them towards burnout.

So what are these poor decisions and what makes them so common? I've noticed a pattern that seems to disproportionately affect recent college hires, but experienced engineers are by no means immune. My hypothesis is that these patterns or habits are actually formed while we're in school. Don't get me wrong, school does a lot to set you up for success in life, but believe it or not, it also teaches you a lot of bad habits that may work well in school, but set you up for failure in industry.

So I want you to picture yourself back in school and let's take a look at some examples:

In School...

Deadlines: In school, deadlines come in the form of due dates. Due dates are set by your professor and are generally fixed and non-negotiable. While they may be challenging to meet, generally speaking, due dates are reasonable and achievable.
Individual vs Collaborative: In school, the vast majority of work is done individually. Sure, we may get the occasional group project, however that's generally the exception. In these group projects, it's not uncommon for one or two individuals to end up doing most of the work while the rest of the team coasts.
Overtime Work: In school, pulling all-nighters studying for a test or working on a big project is looked at as a normal, socially encouraged occurrence.
Assignment Complete: Once the due date comes, you turn in your work or take your test, receive a grade and--let's be honest--you never look at it again.

In Industry...

Ok so you've spent years in school and eventually it leads to your first job in the tech industry! Congratulations! Ready to change the world? Ok!

You are given your first big project to work on, and you're given a deadline for the first deliverable. You may not even question it. In school you don't negotiate due dates. You just go on faith that they're generally reasonable and achievable. You get to work. Things move relatively quickly at first. Early reports to management are confident versions of "it's practically done!"

As time goes on, you keep discovering additional complexity that you hadn't realized was there. When you said you were almost done, maybe you weren't thinking about the fact that you'd have to write tests too. The deliverable date is approaching and you start to get a little panicky. No worries, you can just work extra hours to make up for it. Maybe pull a few all-nighters like in school...

Turns out the amount of work was much more than you initially thought and a few all-nighters don't seem to be getting you much closer. The deliverable date passes and the software is not done yet. At this point, you and maybe your teammates are working crazy hours (possibly mandated by management at this point), which is demoralizing, but you may not even question it. This is just how software development works right?

Management is pretty frustrated at this point, and you're seeing your career aspirations slip away one day at a time, so you make a decision: You decide to cut corners to make things go faster. Maybe you can save those tests for later, after launch. Maybe you can just hack a few things in here and there...

Eventually, the product is finally launched! Phew! You're exhausted, but at least you don't have to worry about it anymore right? In school you just throw old assignments away. Actually, you do. Unlike school, you have to maintain the things you build and deliver. If you cut corners on quality or worse, architecture, there will be many bugs. This leads to customer pain, which means operational pain for you.

Of course while this operational pain is going on, you're trying to work on the next deliverable for the project. At this point, the next deliverable is more likely to also be late, not only because some of the mistakes made during the first deliverable may be repeated, but now the development team also has the added burden of randomization due to ongoing operational problems. Add to that technical debt and increased risk of regressions due to cutting corners on the first deliverable and the cycle not only repeats itself, it actually gets worse each time around...

The Vicious Cycle of Software Development

Congratulations! You've been burned by what I like to call the Vicious Cycle of Software Development. The basic cycle looks like this:

As software gets later and later, it can turn into a date-driven death march, which leads to compromised quality, which leads to operational pain, which makes the next feature late, and so on and so on. If this cycle continues for a prolonged period of time, people will get burnt out and leave the team, which adds these additional consequences to the cycle:

People get burnt out and eventually leave. Their knowledge of how the software works goes with them, making it even harder to develop new features in a timely manner.

Admittedly, the scenario I paint above is a worst-case scenario. However in my years as a software developer, I've seen teams fall into this cycle time and time again. I've described this vicious cycle in several talks and always see knowing smiles in the audience and people nodding their heads. It's surprisingly easy to get onto this path regardless of your experience level.

So what can we do? Here are some tips for avoiding or reversing the vicious cycle.

Tip 1: Deadlines should be based on estimates from multiple people and treated as goals/forecasts rather than hard, immovable barriers.

Remember the bad habit we learned in school that deadlines are generally reasonable and non-negotiable? That's not true in software development! Very few deadlines are hard, immovable deadlines. Most are really goals to get teams focused on delivery. Dates can be aggressive, but you should not treat them as if they are sacred. The benefit of aggressive deadlines is it forces priority and requirements decisions to reduce the deliverable to the minimum viable product that solves the customer's needs. Development teams should treat software deadlines as goals/forecasts. Throughout the project there should be ongoing, open communication about how actual progress compares to the goal.

Note that software estimation is very difficult and when we're wrong, we tend to be wrong on the side of underestimation. So combat underestimation by coming up with a system for estimation that involves multiple people to reduce bias.

Tip 2: Learn how to communicate status effectively, and handle decisions around missed expectations as a team.

In the vicious cycle scenario, as the deadline approached and there was still a large amount of work to do, two key mistakes were made: Status was not reported in a useful way ("It's almost done!") and the developer unilaterally decided to work extra hours and pull all-nighters when they thought the project was behind schedule. Again, this behavior lines up well with our school experience. In school, we never had to deal with reporting status. The due date came and you handed in your assignment or took a test. Also, since most of the work was done individually, you are used to making all of your decisions by yourself, especially when it comes to pulling all-nighters.

I have a whole post I've written regarding communicating effective status, so I'll defer to that for the first problem. For the second problem, if reality is not matching expected timelines, it's critically important to have an open conversation with your team, including management as early as possible. The earlier you surface problems, the more options you have to handle them. The discussion should consist of (1) root causing why things are taking longer than expected and (2) deciding on a path forward.

When deciding a path forward, working extra hours is an option, however it should be the last option. There are other much better options to consider such as

Resetting expectations - management may be fine with it being later than planned.
Challenging requirements/scope of a feature - sometimes there's a smarter/smaller/easier way to get the customer what they really need.
Drop lower priority features

You should work to build a team culture where there is open communication and handle decisions around missed expectations together as a team. When adopting this model, newer developers are frequently surprised by how often management is ok with dropping features or punting things forward to future iterations. Without that conversation, you might have worked yourself to exhaustion trying to deliver something that didn't actually matter that much to the business.

Tip 3: Treat quality as an inseparable part of a feature that cannot be deprioritized.

If you or team members are ever considering deprioritizing quality, ask yourself this question: Which would you rather choose: (1) A known, fixed upfront cost, or (2) an unknown, variable cost that can come up anytime, while you're trying to deliver other features? I hope you picked option 1. Option 1 is prioritizing quality up front. Deprioritizing quality leads you right into the operational pain stage of the vicious cycle, which causes future software to be late.

For me, the phrase "deprioritizing quality" doesn't even make sense. Prioritization happens at the feature level and quality is a non-severable part of a feature that cannot be deprioritized. When I discuss priorities with management, I always list things out at the feature level. That way there isn't even a discussion about deprioritizing quality. Although to their credit, I find most managers understand this and it is more common for developers to deprioritize quality and cut corners when trying to deliver faster. Don't fall into that temptation! As a developer, you own delivering quality software. Put in the work now to invest in quality. It will pay itself back many times over in the long term.

Tip 4: Always strive to build a team culture of trust and open communication.

Communication and collaboration are key to succeeding in industry. The scope and size of various projects and software at many companies is so vast, it's not possible for a single person to understand it all. In order to succeed, you have to learn to work well with others and communicate, especially when you really don't want to, like when things are not progressing as planned. One of the reasons I like Scrum so much is that you get to practice having these discussions constantly, during daily standups and retrospectives. Also, the frequent iteration cycle (in theory) prevents you from going too far off the rails without some kind of checkpoint to catch it. However I've also seen teams just go through the motions of Scrum without actually practicing it.

If you struggle with confrontation or difficult conversations, I highly recommend reading Crucial Confrontations. I didn't have a good model for healthy communication growing up, and that book completely changed the way I view and deal with confrontation, both in my professional and personal life.

Wow, this post ended up being a lot longer than I had intended, but I hope you find it helpful! ðŸ˜… Have you experienced the vicious cycle of software development? Do you have additional tips for reversing it or avoiding it altogether?

How to Set the Technical Direction for Your Team

James Hood — Sat, 13 May 2017 13:52:38 +0000

This post was originally posted on my blog

I've found that one of the keys to a productive and happy development team is when every member of the team understands (1) the technical direction of the team, and (2) how the work they're currently doing contributes to that direction. Because of this, one of my key roles as a senior developer is helping teams establish a clear technical direction, as well as a plan for how to get there.

After several years working at Amazon, one thing that continually stands out to me is the genius of Amazon's Leadership Principles. At some other large companies where I've worked, their version of leadership principles or corporate values were just a set of vague statements or buzzwords printed on posters and hung somewhere in the office. Most people ignored them and carried on with their daily work. However, at Amazon, we live and breathe the leadership principles and refer to them frequently in daily conversation. Collectively, they setup a framework for thinking that pushes everyone who works there to become fast-moving, innovative leaders.

When it comes to establishing technical direction for a team, two leadership principles come into play: Think Big and Bias for Action.

Think Big encourages you to "create and communicate a bold direction that inspires results," while Bias for Action pushes you to move quickly and not get lost in the dreaded analysis paralysis. A good technical direction incorporates both of these principles. As a result, I coach teams that they must establish, both a long-term, directional vision and short-term, incremental milestones that lead in the direction of the long-term vision.

Long-Term Vision

The long-term technical vision is where the Think Big leadership principle comes into play. Take the business problem, put pesky constraints like time and resources out of your mind and try to come up with an idealized architecture. Be liberal with unicorns and rainbows in your block diagrams. ðŸ˜‰

Seriously though, long-term visions are tricky. They have to be clear enough that they are believable and everyone on the team can understand and align to them. They have to be relatively stable over time. Finally, they also have to be abstract enough that individuals feel empowered to forge creative paths to get there. This is a tall order and a key challenge of effective leadership.

Here are some tips that help me when coming up with a long-term technical vision:

A picture is worth 1000 words. Architectural block diagrams can add a lot of clarity around key components and their relationship to one another.
Don't try to fit your entire vision into a single picture. Use multiple pictures at different levels of abstraction to provide clarity. I love Simon Brown's concept of thinking of architecture diagrams as maps of your codebase.
Stay abstract. Avoid choosing specific technologies. Instead, stick to defining components and keep concepts like databases, queues, streams, etc generic, rather than naming a particular solution.
Walk through key business use cases and show how the components interact.
Enumerate the key business and technical wins of the architecture.
It's all about boundaries. It's critical to define the key subsystems and components of the architecture and their responsibilities. A great way to solidify understanding of responsibilities is to focus on boundaries when walking through use cases. Specifically call out what each side is responsible (and not responsible) for.
Don't try to design to the last detail. You don't need to go deeper than component level, i.e., no class diagrams. Remember, the important thing for long-term vision is defining key components and their responsibilities.

Short-term Incremental Milestones

Long-term visions are important, but if you are really exercising Think Big, you will end up with the outline of a system that will take several years to build. If you just turn around and build that entire system from scratch, you might run out of funding before ever launching. Even worse, you might get to the finish line and realize what you've built wasn't actually the right solution. Businesses shift quickly and you need to be able to adapt to that change. This is where the Bias for Action leadership principle comes into play to balance out Think Big.

Once you've established your long-term vision, it's time to come back to the real world and come up with an incremental plan to get there. That plan needs to involve launching something real to customers quickly so you can get rapid feedback in case you need to course correct in the spirit of Agile and The Lean Startup.

I like to use the Impact vs Effort Matrix as a way to think about this:

Building your long-term vision from scratch should be firmly in the high effort, high impact quadrant. However, to get started, look for tasks in the low effort, medium-high impact area that are in the direction of your long-term vision. For example, your long-term vision may be to build a complex, feature-rich realtime business analytics platform for your business. However a short-term milestone in that direction would be to stand up a Kafka cluster and attach a connector to a replica of a few of your DBs and push updates into an ElasticSearch cluster. Then use Kibana to create a few dashboards. This is something that could be completed in a few months and will give you customer feedback that helps you determine next steps. It can also give you datapoints to help determine if you're headed in the right long-term direction or if your long-term vision needs to be tweaked.

Here are some tips that help me when coming up with short-term, incremental plans:

Look for milestones that are achievable within a 3-6 month (or shorter, depending on the size and scope of the project) timeframe. Ideally milestones are customer-facing, but don't stretch your milestones out to a year or 2 years just to make them customer-facing. If they're that big, there's probably a scrappier solution that will give you the feedback you need.
Be concrete. Unlike your long-term vision, short-term solutions should be well-defined. Choose specific technologies and justify your choices.
Enumerate the key business and/or technical wins of each milestone. This ensures you know why you're doing each one.
Use the long-term vision as your guide post. Your milestones should be in-line with the end goal, although they can veer off track a bit if there are good reasons for it. However, avoid lateral or backward movement.
Don't try to define every single milestone all the way to the end goal. The long-term vision is directional and it may (will) change and evolve as you learn more from each milestone. Trying to define every step at the outset is difficult and wastes time.

I can't stress that last tip enough. I've worked with people who try to push for detailed plans that span multiple years, and it just ends up being overwhelming and demoralizing to the team. The thing is, if you start with lower effort work and get some quick wins, it builds momentum and confidence. I also find that as you make progress, milestones in the high effort/high impact quadrant start to shift left. In other words, something you've built or learned in a previous milestone suddenly makes another milestone much more achievable. They can become the next milestones on your roadmap, further increasing confidence and momentum. I much prefer this model, because it's adaptive and decentralized, which empowers teams.

Conclusion

Having a clear long-term and short-term technical direction is critical to leadership in software development. Hopefully this helps give you an idea of how to approach setting technical directions, and how Amazon's leadership principles can help. If you're interested in learning more about the leadership principles, I recommend reading The Amazon Way.

Sick Day

James Hood — Fri, 12 May 2017 18:03:02 +0000

This post was originally posted on my blog

I've been out sick most of this week due to an annoyingly persistent headcold. When I was a kid and I got sick enough to stay home from school, I'd lay in bed until I was feeling a bit better, and then I'd watch cartoons on TV for the rest of the day.

As an adult, the cartoons have been replaced with TED Talks. I love TED Talks because they contain such a wealth of experiences and insights from a wonderfully diverse group of individuals. One talk that stood out to me was Dan Pink: The puzzle of motivation.

In this talk, Dan explains how external motivators such as promotions, bonuses and awards, commonly used by businesses to motivate employees, actually decrease performance if the work being done requires creativity. This resonated with me because I mentor many developers, and a common, ongoing topic of conversation is how to get promoted to the next level.

I always agree to help guide developers in the direction of getting promoted, however I first give them the following disclaimer: In my entire career, I have never made getting promoted or receiving good ratings the primary focus of my work. In fact, the few times I've felt myself starting to worry about how my actions might be perceived by others in the context of annual reviews or ratings, I've felt a drop in performance and general job satisfaction.

This may sound a little cheesy, but I honestly get a lot of personal satisfaction from knowing that customers are using my software and are having a good experience. The beauty of this is that it results in a strong alignment between my personal motivations and Amazon's core principles. I honestly want to deliver quality software that meets the customer's needs as quickly as possible, because I find that satisfying in and of itself. Promotions, good ratings and bonuses are a happy side effect of genuinely wanting to do "the right thing."

So my "big secret" to getting promoted or rated well is to stop worrying about it so much. This also leads to a generally happier work experience, because I'm not basing my personal feelings of self worth on something that may or may not come, and if it does, only comes once or twice a year. Instead, I get some level of satisfaction from every change I push to production, especially when operating in a full Continuous Deployment environment.

Ok, for you skeptics out there, I will admit that as I've gotten higher and higher in level, I have had to start thinking about promotions a bit more. Currently, my next promotion would be to Principal Engineer (which is a mind-blowing thought!), and I do have to take a bit more of a strategic approach to it. I make sure I'm picking up opportunities that align with getting to the next level, however again, the key point to stress is that promotion is never my primary focus. I don't just volunteer for any opportunity if it fits the profile of a Principal Engineer. It has to make sense, given my skill set and what I'm personally passionate about.

For example, a few years ago, I took ownership over a process for my org that helps more teams get to full continuous deployment. Since it spans the org, it definitely "looks good" in terms of showing my ability to operate at a level larger than an individual team, which is a requirement for Principal Engineer. I was conscious of this fact, however I volunteered for the role primarily because (1) I'm very passionate about full CD and (2) I saw some ways that I could really improve on the process from the previous year. If those two components hadn't been there, I would not have volunteered, no matter how good it would look for my career.

It's not that I don't care about my career. But I care much more about feeling passionate and motivated about the work I'm doing every day. I care much more about the developers I mentor, about forming strong teams and delivering great software. High ratings and promotions should be considered natural side effects, not the core goal.

Tips for New Software Developers

James Hood — Fri, 05 May 2017 18:25:44 +0000

This post was originally posted on my blog

Amazon is a fast-paced, decentralized and constantly changing environment. It can be a bit of a shock to new people coming into the company. Because of this, mentoring developers is a critical (and very enjoyable) part of my role. I mentor many fresh-out-of-college developers entering the industry for the first time, as well as other more experienced developers coming in from different companies. Some of the college grads ask me for tips on how to get a good start on their career.

Here are some of the tips I give that are non Amazon-specific and generally useful for any software developer.

Tip 1: Treat your career like a marathon, not a sprint.

I remember being in college talking to one of my first Computer Science professors. When he found out I was a freshman, he said, "Oh you're just right at the beginning!" He didn't mean it in a negative way, but I remember feeling so frustrated, because although he was absolutely right, I desperately wanted to prove myself and earn a level of respect and acceptance of my peers (and myself really) as quickly as I could. However some things just take time, and time wasn't something I could control.

I definitely see some of those same feelings reflected in the recent college grads beginning their careers, especially at a company like Amazon where expectations are so high. Here's what I tell them:

First off, you're at the beginning of what I hope will be a long and very fulfilling career. That's a very exciting place to be! The first thing you need to understand is that your career is going to span decades. I want you to really try to think about and internalize that. Decades. That's a long time. So you can't approach your career like it's a sprint. It's a marathon and you need to approach it accordingly. You need to pace yourself. Developing consistent, healthy habits will pay much larger dividends in the long term than short, sporadic bursts of effort that leave you exhausted.

Here's an example: For me, reading technical and personal development books has benefited me tremendously in my career. Your first instinct is going to be to binge read a 350-400 page book over a weekend. But in practice, what I've found is this isn't sustainable. You end up binge reading maybe a book every few months at best, but it usually lengthens out to become less frequent over time.

Alternatively, you can build 20 or 30 minutes a day into your schedule as reading time. That doesn't seem like a lot of time at first, but let's say you read 10 pages in that time, which is a pretty conservative estimate (but can be realistic for very dense technical books). If you do that every day, that ends up being about 300 pages a month. At that rate, you can get through a book in a little over a month. You won't be exhausted, and you'll have time for other things so after that book is complete, you can move on to your next book. Now fast-forward to a year from now. With binge reading, you've read maybe 2-4 books. With 30 minutes a day, you've probably read 10-12. That's a big difference and that's only one year. Remember: Decades. A year is 3.3% of a 30 year career. ðŸ˜³ Starting to feel some perspective?

This is the kind of thinking you need to apply to your career. Building regular time into your schedule is much more effective than trying to binge whenever you think about it or are feeling frustrated and stagnant. Of course those opportunities do come up. Maybe you're traveling and read your book for an entire plane ride. Maybe you're so interested in this book, you read it for 2 hours one day. But that should be bonus, and you should naturally fall back to your regular scheduled time as the default. Also, don't beat yourself up if you fall off the wagon and miss a few days or a week. You're not a robot and life happens. Just accept it and come back to your schedule.

Tip 2: Focus on your strengths, interests and the little wins over artificial rewards like promotions and bonuses.

I've written an entire post about this, so I won't spend too long on this here. My personal philosophy is that things like promotions and bonuses should never be the central focus, but rather happy side effects of being passionate about what you do and doing the right thing for the customer and the business. If promotions are something that are emphasized at your company, then become aware of the kinds of things that are being looked at for promotion. However, start with your interests and strengths and then look for ways to leverage those skills to make a contribution that's in line with getting to that next level.

However don't make a promotion or positive performance review your primary personal success metric. Maybe I'm just impatient, but that's too long for me to wait! It also relies on someone else, and I'd rather be my own source of motivation. I much prefer the satisfaction of things like finding the root cause of a bug and pushing the fix to production or having a great technical discussion with someone that yields new insights or having a good mentoring session with someone and feeling like I really helped them. Train yourself to see the little daily wins and you'll feel a lot more satisfied and motivated at work. Tracking my accomplishments in my TODO list is my mechanism for making sure I do that.

Tip 3: Set boundaries on work hours from the very beginning...but work hard and efficiently during those work hours.

It's very common for college grads to be single and have some number of years where they have a lot of extra time. Frequently they choose to work very long hours in the hopes that it will give their career a jumpstart. I had a very different experience. I actually met my wife when we were 19 and got married straight out of college, and we had kids relatively soon after. So I never had this kind of extra time to burn at work. However it turned out to be a blessing in disguise as it forced me to limit my working hours from the very start. What I found was that by limiting my working hours, it forced me to find ways to be more efficient in the time I did have. New developers fresh out of college are frequently amazed at how much I can get done in a very short amount of time, and it all comes from personal productivity skills I've worked hard to develop throughout my career.

Sometimes people are still skeptical of this, so one analogy I like to give is from Computer Science. Say you have an algorithm that runs in quadratic time, O(nÂ²), and it's too slow for your use case. One solution is to throw more hardware at the problem, another is to find a more efficient algorithm like one that runs in linear time, O(n). If you ask a college grad which one they'd do, they'll usually laugh and say find a more efficient algorithm of course! To me, adding more work hours is like throwing hardware at a slow algorithm. So how come with code, you'd rather go for efficiency, but with yourself, you'd rather throw hardware at the problem? ðŸ™ƒ

Again, your career will span decades (have I mentioned this already?), so putting the time into making your own personal productivity algorithms more efficient is going to pay itself back many, many times over. It will also make you far more valuable which will make it easier to move around between teams or companies in the future. So stop throwing more hardware at the problem! Limit your work hours, but during those hours, work hard and efficiently. Your output is the critical success metric, not the number of hours you spent working on it. In fact, less hours spent is better, right?

Tip 4: Accept that you will not know every technology and framework and that's ok...but never stop learning!

New developers frequently feel insecure because they know there's a vast array of technologies that they know nothing about. It doesn't help reading comments on some tech blogs where people try to present an image as if they have deep knowledge of all different technologies. Trust me, they don't. The tech field is so vast and constantly changing that there's no way for any one person to know every technology and framework out there. You need to accept that this is ok and you don't need to know everything to be effective.

You want to develop a healthy balance of having very deep knowledge in some focused areas, but build shallow knowledge in a wide variety of areas. Also, this shouldn't be a one-time acquisition of knowledge, but a continuous stream of incoming and growing knowledge so you can keep up with new advances. Sound difficult? It can be, but here's my strategy:

I let my current projects drive the areas of deep knowledge. It intuitively makes sense that you should develop deep knowledge of the technologies and frameworks needed for your current work. As you move from project to project, you'll get more opportunities to add to your areas of expertise. One of the factors you may want to consider in picking a new project or team might be the technologies being used. If it's one you're unfamiliar with, it could be a great learning opportunity. When evaluating technologies for a new project, you might want to pick one you aren't as familiar with, but seems like a decent fit so you can learn it. In a team setting you'll need to get some buy-in from your teammates since they'll need to support it as well, but this can be a nice strategy to broaden your deep knowledge areas.

So that covers the deep knowledge. However you also don't want to develop tunnel vision and lose track of what's happening in the wider industry. So you need a way to keep up with what's going on outside of the area you're working in. My primary mechanism for this is Twitter. If you follow the right accounts, your newsfeed delivers various software development articles, talks and project updates directly to you. I definitely don't stress about reading everything, but I do skim lots of articles on various topics. If I find something that interests me, I might do a couple searches to learn more, but my overall goal is to cast a wide net, but not go too deep unless it's relevant to what I'm working on directly.

This becomes really helpful for general knowledge. However, it's also useful specifically when I'm at a phase in a project where I am choosing technology options. At that point, I'll remember having read something about some other alternative technology that might be a good fit for the project. At that point I can research it further and decide if it's a good option. If so, then I'll be on the path to becoming deeply familiar with that technology.

Twitter doesn't have to be the only source of articles. Email digests can be another good source. I find DZone's email digests to be pretty decent. I also subscribe to Amazon Web Service's announcement email list, since I use many AWS services at work.

One of my favorite things is helping people feel empowered to drive their own career path based on their strengths and interests, rather than feeling forced to climb some ladder of artificial incentives that's been placed in front of them. I've seen these tips help start some people down that healthy path. I hope you find these tips helpful, whether you're at the beginning of your career or have been at it for a while. If you have additional tips to add or things you wish you had known at the beginning, please add them in the comments!

My Favorite Microwave

James Hood — Mon, 01 May 2017 20:27:18 +0000

This post was originally posted on my blog

I recently read this blog post, which describes how microwaves have gotten caught in the trap of adding features that no one needs while usability for the core use cases suffers. Not only do I agree, but it reminded me of my absolute favorite microwave I've ever used...

I was fresh out of college working at IBM, and they had microwaves in the break area for heating up lunches. Most of them were fairly standard microwaves, however one microwave stood out from all the rest. In addition to the usual buttons, it had a monochrome LCD display with 8 soft keys around it.

When you first plugged the microwave in, it would boot up and a little black and white animation of a chef would appear on the LCD display and bow to you. Then you would have to step through a welcome wizard introducing features of the microwave. The usual things like defrosting/cooking different types of meat were there, but it also had features I'd never seen on a microwave. For example, it came with recipes stored in it that you could search through. Once out of the welcome wizard, the LCD display was used for a complex menu system with all kinds of options.

The best part? The physical start button on the microwave was broken.

But not to worry, there was a workaround! You could also start the microwave by stepping through just a few menu screens using the soft keys.

I loved this microwave and thought it should be installed in every break area used by engineers to serve as the perfect example of overengineering the crap out of something. It seems so ridiculous when you look at the end result, but it can be so easy to overengineer a solution, especially in software, where requirements are often unclear and fluid.

I'm definitely not immune to overengineering, however the primary tool I've found that helps prevent it is domain-driven design. Most of my up front design time is spent breaking a problem down into a set of domain objects with clearly defined responsibilities. If you do this well, you can both solve the immediate problem at hand, while having the flexibility to handle future requirements, either by slightly modifying existing domain objects or adding new ones. While this is a simple concept, it is definitely not easy to do consistently and requires a lot of practice.

Do you have any overengineering stories? What are your tips for preventing it?

"It's Done!" "Great! What does that mean?"

James Hood — Sun, 23 Apr 2017 12:34:57 +0000

This post was originally posted on my blog

One of the unique challenges of software construction, versus other forms of construction, is that what software developers build is largely intangible and invisible. Measuring and tracking progress of a software project is a very difficult task. It's critical for software developers to realize that one of our key roles is to be able to clearly communicate task status to stakeholders. One of the cardinal sins of software development is communicating that a task or project is farther along than it really is.

With that in mind, I'd like to share a phrase that has become a huge pet peeve of mine:

"It's done!"

I've heard this phrase uttered countless times by developers, and upon further questioning, it almost always turns out that there is still work remaining. I'm not sure why developers have such a strong tendency to say "it's done" when it's not actually true, however I can say, it's generally not out of malice or the desire to misinform. Usually, what I find is that they mean they are done with some piece of the task. Maybe they've completed the functional changes (but there are still unit tests to write). Or maybe functional and unit test changes are finished and they've submitted it for code review (but no one's reviewed it yet, which could lead to more work). And so on.

This is such a common problem, that the term "done-done-done", has become popular throughout Amazon as a way to basically say "no really, it's completely done." I've tried using "done-done-done," but I find that after a while, you still end up running into the same problems.

I've come to the conclusion that "It's done" tells me absolutely nothing about the progress of a software task. Even worse, if most of the time when developers say "It's done," it's actually not done, then that means most of the time you're lying to stakeholders. So "It's done" is worse than worthless, it actually erodes trust between developers and stakeholders almost every time you use it!

To combat this problem, here are the phrases I've found useful for communicating status of a task:

Functional changes are in progress.
Functional changes are complete, unit test changes are in progress.
I've submitted a code review for the first draft of functional and unit test changes, and no one's looked at it yet.
The code review is currently on revision X (in which case the reviewer is a better person to ask for ETA on completion of the code review).
The changes are past code review and on their way to production.
The changes are deployed to production.

5 and 6 assume you're practicing Continuous Deployment. If not, you may need to break down release status further to better clarify progress.

Note, the word "done" at the overall task level has been completely removed from the conversation. I'm very anal about this on my teams. Anytime a developer says "It's done" during daily standup, it immediately triggers a game of 20 questions where I try to determine which of the above phrases best represents the current status. Most of the developers on the team have picked up on this and have learned to forego this game and just use one of the above phrases.

From a developer's point of view, this may seem like a minor miscommunication, but erosion of trust between stakeholders and developers is a very serious problem. When stakeholders don't trust their developers, they're more likely to interfere with the team's development process, for example, forcing dates, which leads to mandated overtime work, reducing morale and productivity, as well as leading to cut corners and overall reduced software quality. And to add insult to injury, when you actually are finished with a feature and inform your stakeholders, you can expect to get a skeptical response at best. So no pats on the back for all your hard work. But honestly, you kind of deserve skepticism if you've been constantly lying to them right?

I highly recommend teams practice reporting status in this way and hold each other accountable when status is communicated in a way that's unclear or misleading. It's actually a pretty easy change to make once you get used to it, and the benefits of increased trust are well worth it.

Ok this blog post is done. ðŸ˜œ