DEV Community: Rob Waller

How to Handle the Emergence of Complexity in Software

Rob Waller — Mon, 20 Jul 2020 10:02:31 +0000

All complex systems have what is called emergent properties. For instance water has emergent properties like damp and wet. When a surface has 10 water molecules spread across it we don't classify it as wet, but at some point when there are enough water molecules on a surface it will become wet. The property of wetness emerges from a collection of water molecules interacting with an object.

The difficulty with emergence is defining the boundaries, for instance between dry, damp and wet. Emergence is also situational, can a granite work top become damp in the same way as a t-shirt? Solid surfaces are generally defined as wet or dry where as permeable surfaces can become damp.

In software development complexity itself is an emergent property of code. At some point in the development process software crosses the boundary between simple and complex. Software goes from readable and easy to understand to unreadable and hard to understand. The emergence of this complexity can depend on a number of factors, how the code is written, how much code is written, how difficult the problem is, etc.

As software developers one of our primary aims is to minimise complexity, and there are strong incentives to do this. One obvious one is financial, as software becomes more complex it becomes harder and more costly to maintain. You require more developers to keep the lights on and get things done. The second is developer well-being, it is not fun to work on code which is too complicated. Instead of adding new features which generate business value and make users happy developers often feel like their only purpose is to ensure the tower of cards does not come crashing down.

What is Software Complexity?

When we refer to complexity in software it is important to define precisely what we mean. By its nature software is complex and most developers only deal with a small part of this complexity at any given time. JavaScript only has meaning because there are numerous other layers of software written in other languages which allow it to work. This complexity is not what is of interest to us because no developer has to consider the complexity of software in its entirety. If they tried they'd fail and probably go mad.

When we talk about complexity in software what we mean is how understandable or readable it is? For example if you sat a new developer in front of an existing codebase could they tell you what it does and how easily could they make a change to it? If complexity is low and the code is understandable then they will be able to tell you what the code does and easily make a change. If not you likely have a complexity problem.

How to Spot Complexity

So what can you do to minimise the emergence of complexity in a codebase? The first step is to learn to spot complexity. Luckily there are tools and metrics which can help with this.

Three important complexity metrics are:

Cyclomatic Complexity: How many control structures does the code have?
NPath Complexity: How many paths are there through the code?
CRAP: Given the complexity of the code are there enough tests?

Of these metrics Cyclomatic Complexity is the easiest to understand and begin using. It looks at a unit of code, usually a method, and checks to see how many control structures or decision points there are. For instance if, switch, foreach, etc. The more decision points there are in a method the more outcomes that method can have, which increases the complexity. Ideally you want code to have a Cyclomatic Complexity score below 5 and definitely below 10. If a codebase has a lot of methods with complexity above 10 there is likely an issue.

There are also plenty of tools like PHPMD and ESLint which allow you to run and automate complexity checks. You can add them to your Continuous Integration pipeline, set some thresholds and if some new code breaches the threshold you can review it and fix it. This process alone will help you keep a lid on a lot of complexity.

But of course the topic of complexity is not so simple. You also need to be able to read code and spot when complexity is creeping into the design.

For instance the PHP code below has a Cyclomatic Complexity score of four, which is good.

public function childrenAboveFiveFeet(array $parents): array
{
    $children = [];

    foreach ($parents as $parent) {
        foreach ($parent->getChildren() as $child) {
            $heightInFeet = $child->getHeight() / 30.48;

            if ($heightInFeet > 5) {
                $children[] = $child;
            }
        }
    }

    return $children;
}

On the surface this code doesn't seem too complicated, it is short and easy enough to understand, but there are problems. The main issue is the business logic is not isolated, it is hidden within nested foreach loops.

/** The Business Logic **/
$heightInFeet = $child->getHeight() / 30.48;

if ($heightInFeet > 5) {
    $children[] = $child;
}

The business logic is the code we really care about, it is the code which makes decisions and we need to ensure it works. But because it is nested in two foreach loops the business logic is much harder to test than it should be.

To test the business logic in detail requires us to spin up a collection of people and children objects each time we write a test. This will quickly become tiresome when we just need to ensure our centimeter to feet conversion is correct, so we can accurately calculate whether a child is above five feet tall. Ideally we should isolate this business logic into separate methods so it can be tested more easily.

To ensure code doesn't become too complex we need to be able to analyse it manually and highlight when the code design could be better. As the example above shows tools and metrics aren't enough on their own.

Code Isolation

This is where the principle of Code Isolation comes in, which is one of the main ways we handle and minimise complexity. There are two basic rules to Code Isolation:

A should not know about B or C or anything else.

If A uses B it should not know where B came from or where B is going.

In real terms these rules may look something like this:

Business Logic should not know about Databases or Filesystems or anything else.

If Business Logic uses Data it should not know where Data came from or where Data is going.

Code Isolation is the guiding principle behind Clean Architecture, but there is not much point in learning Clean Architecture unless you have an understanding of Code Isolation.

In basic terms Code Isolation means we separate decision making, also referred to as business logic or domain logic, from Input / Output. So in our code we don't muddle calls to the database or filesystem with making decisions.

In this Deno / TypeScript code example the retrieval of data from a JSON file is muddled with making a decision about the data.

export function overEighteens(): object {
  /** Filesystem Call **/
  const file = fromFileUrl(new URL("../../assets/people.json", import.meta.url));
  const json = readJsonSync(file);

  if (json instanceof Array) {
    return json.filter((person: any) => {
      if (person.age !== undefined) {
        /** Decision Point **/
        return person.age >= 18
      }
      return false;
    });
  }

  return {};
}

As a standalone method the above code is basically fine, and if this was the only function in a microservice then there wouldn't be an issue as complexity would be low regardless. But merging I/O and decision making in this way creates problems.

The code is harder to test because it is tightly coupled to the filesystem. So we either have to mock the filesystem somehow or ensure the filesystem is working correctly for us to test the code. Debugging the code is also more difficult, does the issue lie with the retrieval of the data, or does it lie with the age check? Does the issue relate to the I/O or the business logic? In this code it will be less clear.

But the main issue is if this approach to code is repeated across a codebase complexity will emerge quickly. The point where the code is difficult to understand, hard to test, debug and change will be reached far sooner than in a codebase which follows the Code Isolation principle.

It is also important to note the code isolation principle has nothing to do with the WET or DRY principles. They all relate to abstraction, but abstraction does not guarantee isolation. A developer can easily abstract tightly coupled code. If the aim of a developer is to minimise complexity they need to follow the Code Isolation principle. Following WET or DRY principles does not guarantee isolation nor will it guarantee minimal complexity. It's not to say WET or DRY don't offer a useful guide for practical development, but do not confuse them with the Code Isolation principle.

A Code Isolation Example

So how might we use the Code Isolation principle to improve the code example above? Well we break the code down into its component parts. The part which retrieves the data goes in one method and the part which makes a decision about the data goes in another.

interface Person {
  id: number,
  name: string,
  age: number,
}

export function overEighteens(): Person[] {
  return retrievePeople().filter(person => overEighteen(person));
}

/** Filesystem Call **/
function retrievePeople(): Person[] {
  const file = fromFileUrl(new URL("../../assets/people.json", import.meta.url));
  const json = readJsonSync(file)

  if (json instanceof Array) {
    return json.filter((person): person is Person => {
      return (
        person instanceof Object &&
        person.hasOwnProperty("id") && 
        person.hasOwnProperty("name") &&
        person.hasOwnProperty("age")
      );
    });
  }

  return [];
}

/** Decision Point **/
function overEighteen(person: Person): boolean {
  return person.age >= 18;
}

The above code is not production ready nor easily testable in JavaScript / TypeScript. But the changes highlight the principle of isolation and the code is now more robust. Data retrieval exists in one place and we ensure it returns a correct collection of data. And our age check exists in another place and expects a Person object.

The code can be improved further and made more testable by abstracting the code into separate modules. The age check can then be tested with a unit test and the data retrieval with an integration test. We've achieved isolation in this code because the age check overEighteen() method no longer knows where the Person data came from, or the purpose of the boolean it returns.

As I hope the example highlights, in medium to large codebases the Code Isolation principle helps keep code simpler, more robust, and testable. And this will minimise the complexity of the codebase making it easier to understand and more maintainable.

Overview

Minimising the emergence of complexity in software is difficult, as software by it's very nature is complex. There is also no one size fits all solution to the problem. How you handle complexity will depend on the problem you need to solve and the scale of it.

There are though strategies which can help developers with this problem. The first is metrics and tooling, and I would encourage all developers to impose Cyclomatic Complexity checks in their CI pipelines. If this is applied to an existing codebase start with a threshold of 20 and lower it as your code improves with the aim of getting below 10. If it is a new project be brave, start with a threshold of five or six and see how you get on.

Also begin to consider the Code Isolation principle and how it can be used to improve your codebase. Analyse where your business logic can be better isolated so it is easier to test and becomes more robust. And as part of this begin to look at Clean Architecture principles and the various implementations, you may find one which suits your use case.

And finally write some documentation as it is one of the best ways to tackle code complexity. It forces you to explain what your code does and what its purpose is. This will help you spot and fix some of the flaws in your code. But most importantly it will help other developers understand why your code exists and what it does which will make it easier for them to contribute.

It is unlikely you can stop any complexity emerging in the software you produce, but by applying some of the tools and ideas above you can hopefully minimise many of its negative effects.

How to Write Tests in Deno

Rob Waller — Mon, 15 Jun 2020 13:24:29 +0000

Deno is a new JavaScript / Typescript runtime which aims to supercede Node.js, and with the the release of Deno 1.0 it is gaining more traction with developers. Personally I'm a big fan of Deno, I think it's a step forward for the JavaScript community, and I hope it succeeds.

I'm also a big fan of tests and Test Driven Development principles, as such I am pleased to see Deno like Rust and other languages has tests built in. With this post I aim to show you how to begin to write tests in Deno, and provide an honest appraisal of Deno's test tools so you are aware of the current pitfalls.

To begin you need to install Deno. Once this is done just run deno test in the command line and any test files you have will execute. You can also run specific test files by referencing them after the deno test command.

deno test tests/example.test.ts

And you can run individual tests or groups of tests by using the --filter option which will run tests where the test name matches the filter pattern.

deno test --filter "test name" tests/

Take a look at the documentation to learn more on how to run tests in Deno.

Another great feature of Deno is it comes with TypeScript built in, so you get type checking out of the box. This also means Deno files can just be TypeScript files, and we can create test files by appending files with .test.ts. For example, person.test.ts will contain tests for our person module person.ts.

To write a test in Deno import the assertion you wish to use from the asserts module and then write the test in the required Deno format.

import {
  assert,
} from "https://deno.land/std/testing/asserts.ts";

Deno.test("Test Assert", () => {
  assert("Hello");
});

The assertions will throw an error if they fail, this will be caught and the error message will be outputted to the CLI.

There is no concept of NPM, Node Modules or package.json in Deno, which is a good thing. Instead you import modules directly from local or remote locations within files. In this case we import the assert() method from the remote asserts module located at https://deno.land/std/testing/asserts.ts.

This is where things become a little hazy, and developers need to be careful. All of Deno's core features, including the asserts module, are held under the std namespace. And this namespace is currently unstable, as of writing we're on version 0.57. This means the test modules like the asserts module are not feature complete, will have bugs and are subject to change at short notice. My personal view is it is possible to write stable tests in Deno but there is more work to do in this area and developers should consider the lack stability before they write their tests.

The asserts module makes nine assertion methods available for use:

assert(expr: unknown, msg = ""): asserts expr
assertEquals(actual: unknown, expected: unknown, msg?: string): void
assertNotEquals(actual: unknown, expected: unknown, msg?: string): void
assertStrictEquals(actual: unknown, expected: unknown, msg?: string): void *
assertStringContains(actual: string, expected: string, msg?: string): void *
assertArrayContains(actual: unknown[], expected: unknown[], msg?: string): void
assertMatch(actual: string, expected: RegExp, msg?: string): void
assertThrows<T = void>(fn: () => T, ErrorClass?: Constructor, msgIncludes = "", msg?: string): Error *
assertThrowsAsync<T = void>(fn: () => Promise<T>, ErrorClass?: Constructor, msgIncludes = "", msg?: string): Promise<Error> *

Assert

The assert method is a simple 'truthy' assertion and it's value is limited because numerous values will assert. This is not great if you're trying to write explicit and precise tests, but should be fine if you're writing something simple and generic.

Deno.test("Test Assert", () => {
  assert(1);
  assert("Hello");
  assert(true);
});

Assert Equal vs Strict Equal

There are three equality assertions available, assertEquals(), assertNotEquals() and assertStrictEquals().

The assertEquals() and assertNotEquals() methods are based on an internal equal method. This is a complicated method, it's cyclomatic complexity is 30, and it tries to provide a general equality check for JavaScript, which is no mean feat.

As the examples show it will assert numerous types, including objects.

Deno.test("Test Assert Equals", () => {
  assertEquals(1, 1);
  assertEquals("Hello", "Hello");
  assertEquals(true, true);
  assertEquals(undefined, undefined);
  assertEquals(null, null);
  assertEquals(new Date(), new Date());
  assertEquals(new RegExp("abc"), new RegExp("abc"));

  class Foo{};
  const foo1 = new Foo();
  const foo2 = new Foo();

  assertEquals(foo1, foo2);
});

Deno.test("Test Assert Not Equals", () => {
  assertNotEquals(1, 2);
  assertNotEquals("Hello", "World");
  assertNotEquals(true, false);
  assertNotEquals(undefined, "");
  assertNotEquals(new Date(), Date.now());
  assertNotEquals(new RegExp("abc"), new RegExp("def"));
});

The difference to assertStrictEquals() is the strict equality check will not assert two instances of identical objects as they won't be referentially the same.

Under the hood strict equals does a simple === check, there is no reference to the equal() method. This limits the scope of what assertStrictEquals() defines as equal, which is simpler, and makes things more precise and stable.

Deno.test("Test Assert Strict Equals", () => {
  assertStrictEquals(1, 1);
  assertStrictEquals("Hello", "Hello");
  assertStrictEquals(true, true);
  assertStrictEquals(undefined, undefined);
});

It should be noted neither assertion will handle the JavaScript float problem, both of these assertions will fail:

Deno.test("Test Floats", () => {
  assertEquals(0.1 + 0.2, 0.3);
});

Deno.test("Test Floats Strict", () => {
  assertStrictEquals(0.1 + 0.2, 0.3);
});

My overall feeling is the assertEquals() and assertNotEquals() assertions are focussed towards integration and functional tests. Also they need to be handled with care as they are based on complicated functionality, which has the potential to be buggy. If you are writing unit tests focused on pure business logic I would stick with assertStrictEquals() as it is a more precise and stable assertion.

Assert Contains

There are two methods available to assert a thing contains a thing in Deno, assertStringContains() and assertArrayContains().

The assertStringContains() assertion does what it says on the tin. It does a simple includes check on a string to see if it contains the expected string. It's not complicated and will be stable and useable.

Deno.test("Test Assert String Contains", () => {
  assertStrContains("Hello World", "Hello");
});

The assertArrayContains() assertion again does what you'd expect, it finds a value in an array. But in contrast to the assertStringContains() assertion it is quite complicated and contains nested loops which is concerning. I've noticed some bugs in the assertion, so you may experience unexpected behavior with this assertion.

Deno.test("Test Assert Array Contains", () => {
  assertArrayContains([1, 2, 3], [1]);
  assertArrayContains([1, 2, 3], [1, 2]);
  assertArrayContains(Array.from("Hello World"), Array.from("Hello"));
});

Assert Regex

You can assert regular expressions in Deno tests using the assertMatch() assertion. It is a simple assertion which does a basic RegExp test on a string. It's not complicated and does what you'd expect it to, so it will be stable and usable.

Deno.test("Test Assert Match", () => {
  assertMatch("abcdefghi", new RegExp("def"));

  const basicUrl = new RegExp("^https?:\/\/[a-z\.]+\.com$");
  assertMatch("https://www.google.com", basicUrl);
  assertMatch("http://facebook.com", basicUrl);
});

Assert Throws

There are two ways to assert whether something throws an error in Deno, assertThrows() and assertAsyncThrows(). Both assertions allow you to check an error has been thrown, the type of error thrown and what the message was. This is pretty standard functionality available in most assertion libraries.

The difference between the two assertions is assertThrows() accepts a standard function and assertAsyncThrows() accepts a function which returns a Promise.

The assertThrows() assertion is relatively straight forward, the only complexity exists around how it handles the message checks.

Deno.test("Test Assert Throws", () => {
  assertThrows((): void => {
    assertStrictEquals(2, 3);
  });

  // assertStrictEquals will throw an AssertionError with the message "Values Don't Match!"
  assertThrows((): void => {
    assertStrictEquals(2, 3, "Values Don't Match!");
  }, AssertionError, "Values Don't Match!");
});

The assertAsyncThrows() is a little more complicated, mainly because it deals with Promises. But basically it will catch thrown errors or rejections in Promises. And again most of the complexity exists around the message checks.

Deno.test("Test Assert Throws Async", () => {
  assertThrowsAsync((): Promise<void> => {
    return new Promise((): void => {
      throw new Error("Panic! Threw Error");
    });
  }, Error, "Panic! Threw Error");

  assertThrowsAsync((): Promise<void> => {
    return new Promise((resolve, reject): void => {
      reject(new Error("Panic! Promise Rejected"));
    });
  }, Error, "Panic! Promise Rejected");
});

It should be noted recent changes have been made to both these assertions to genericise them which will make them more usable but may result in some stability issues in the near future.

Custom Messages

Each of Deno's built in assertions allow you to overwrite the standard CLI message if you want to. For instance this example will output "Values Don't Match!" rather than the standard CLI message.

Deno.test("Test Assert Equal Fail Custom Message", () => {
  assertEquals(1, 2, "Values Don't Match!");
});

Overview

Overall Deno tests are relatively straight forward to set up and begin using, which is a massive benefit compared to the config hell of Node and NPM test libraries.

There is though some work to be done in this area for Deno. It's fair to say some of the assertions are complicated and stability may be an issue in the near future. But overall it is a great start and a big step forward for testing within the JavaScript community.

My advice is if you are writing unit tests stick to the following assertions as they are precise and stable:

assertStrictEquals(actual: unknown, expected: unknown, msg?: string): void
assertStringContains(actual: string, expected: string, msg?: string): void
assertMatch(actual: string, expected: RegExp, msg?: string): void
assertThrows<T = void>(fn: () => T, ErrorClass?: Constructor, msgIncludes = "", msg?: string): Error

Use the remaining assertions if you are writing more general tests like integration and functional tests.

Beyond Deno Core Assertions

If you want more than the Deno standard assertions module has to offer I have begun work on an assertion library called explicitly. The library extends the standard Deno assertions with a collection of simple but explicit assertions. These assertions are geared towards developers who wish to write clear and precise unit tests in Deno.

assertTrue(actual: unknown): void
assertFalse(actual: unknown): void
assertSame(actual: unknown, expected: unknown): void
assertGreater(actual: unknown, expected: unknown): void
assertGreaterOrEqual(actual: unknown, expected: unknown): void
assertLess(actual: unknown, expected: unknown): void
assertLessOrEqual(actual: unknown, expected: unknown): void
assertInstanceOf(actual: unknown, expected: any): void
assertTypeOf(actual: unknown, expected: string): void
assertDate(actual: Date, expected: Date | string): void
assertDateTime(actual: Date, expected: Date | string): void

* Recent changes have been made to these Deno assertion methods in terms of naming and implementation, please see the version history for more details.

The Code Maintainability Test

Rob Waller — Fri, 27 Mar 2020 09:33:59 +0000

Code maintainability is a fundamental part of good software development. It is important because as code becomes less maintainable the cost to maintain the code increases. There are more bugs to fix and it's more difficult to add new features. Also new developers find less maintainable code more difficult to learn and it takes them longer to contribute to a project, driving up costs further.

If we were to plot cost against code maintainability it would look something like this:

As maintainability increases costs fall. Maintenance costs never fall to zero but they do fall dramatically as a codebase improves.

One problem with code maintainability though is it's difficult to quantify. What does maintainable code mean or look like? There are tools like Code Climate which attempt to define it, but they don't offer a lot of detail on what it means. Also it's not always possible for a development team to use a cloud based tool like Code Climate.

To help I have created a quick ten point Code Maintainability Test. It is similar in intent to the Joel Test, it is a bunch of simple, binary questions. For a positive answer you score a point, and the closer your score is to ten the more maintainable your code is.

Of course this test is not meant to be comprehensive. It is only meant to provide a rough guide to how maintainable code is. Like sticking a licked finger in the air to test the direction of the wind. It does not capture all the nuance of code maintainability.

The test is broken down into three parts:

Usability: how easy is your code to learn and contribute to?
Bug Density: how many bugs do you have to fix?
Code Complexity: how easy is your code to understand and extend?

The Test

Usability

Usable code is easy to learn and fast to contribute to. Well formatted code with good documentation is usable code.

Do you have a well written README which explains installation, setup and usage? (+1 Point)
Do you have clear code comments which explain the existence and intent of the code? (+1 Point)
Do you run code lint tools? (+1 Point)

Bug Density

Strong types, static analysers and code tests will all reduce bug density. The lower your bug density the fewer bugs you'll have to fix in production.

Do you have a code coverage score greater than 60%? (+1 Point)
Do you have a code coverage score greater than 90%? (+1 Point)
Do you run mutation test or fault injection tools? (+1 Point)
Do you use a strongly typed language or run a static analyser? (+1 Point)

Code Complexity

Code with low complexity is easier to read and understand, easier to test, easier to debug, and easier to extend.

Do you run mess detection tools? (+1 Point)
Do you have an average code complexity below 10? (+1 Point)
Do you have an average code complexity below 5? (+1 Point)

Scores

0 - 3 Points: Your codebase is difficult to maintain. It will have a high bug density; It will be difficult to add new features; New developers will struggle to learn the codebase and contribute; Costs will be high; A long and difficult rebuild is likely.

4 - 7 Points: Your codebase will have maintenance issues. It will have a moderate to high bug density; Some features will be difficult to add; New developers will struggle with aspects of your codebase; Costs will be moderate to high. Improvements can be made.

8 - 10 Points: Your codebase is easy to maintain. It will have a low bug density; You can easily add new features; New developers will learn the codebase quickly; Costs will be minimised. Keep up the good work.

Tools

Below is a list of tools which can help with testing and code analysis which will improve code maintainability. It is not a comprehensive list, there are loads more tools out there across many languages.

Testing Tools

Jest (JavaScript)
Mocha (JavaScript)
Chai (JavaScript)
PHP Unit (PHP)
Codeception (PHP)
Mockery (PHP)
Tarpaulin (Rust)

Mutation Test Tools

Stryker (JavaScript)
Infection (PHP)

Code Analysis / Quality Tools

ESLint (JavaScript)
PHP MD (PHP)
Code Sniffer (PHP)
PHP Loc (PHP)
Radon (Python)
Pylint (Python)
Rust FMT (Rust)
Clippy (Rust)

Static Analysis Tools

PHPStan (PHP)
Psalm (PHP)

I do hope this is useful, and if you have any questions, thoughts or criticisms drop me a message on Twitter @RobDWaller.

How to Add a GitHub Actions Badge to Your Project

Rob Waller — Fri, 20 Mar 2020 13:09:11 +0000

I've been playing with GitHub Actions for a couple of months now. I had been an ardent Travis loyalist, but given how impressive GitHub Actions are it feels inevitable I will make the switch.

One thing which tripped me up though was how to add GitHub Actions badges to my projects. This is so I can provide some observability and prove my projects build and the tests pass.

It turns out it is really simple, but it doesn't seem to be well documented anywhere. So I thought I'd make a note of it and share it.

To add a GitHub Actions badge to your project just use the following markdown. Obviously fill in the relevant {user}, {repo} and {action} information:

[![Actions Status](https://github.com/{user}/{repo}/workflows/{action}/badge.svg)](https://github.com/{user}/{repo}/actions)

One thing which can be a little confusing is what the {action} name should be. This should reference the name property in the yaml action config file within your ./.github/workflows directory.

For a Rust project I recently worked on this is Build and Test.

name: Build and Test

on: [push]

jobs:
  build:
  ...

The only change required for the badge URL is to replace the spaces with %20, so the URL {action} reference becomes Build%20and%20Test.

Here is an example from my Rust project which should make this clearer:

[![Actions Status](https://github.com/RobDWaller/csp-generator/workflows/Build%20and%20Test/badge.svg)](https://github.com/RobDWaller/csp-generator/actions)

And you can see the working badge here:

I hope this info helps, have fun with GitHub Actions, and if you have any questions drop me a message @RobDWaller.

How to Easily Add JWTs to Slim PHP

Rob Waller — Mon, 16 Mar 2020 12:42:47 +0000

Slim PHP is one of my favourite web frameworks, it's simple to learn, lightweight, and great for building small websites and applications.

One of the best use cases for Slim PHP is when you need to build a small API with a handful of endpoints. In this scenario you may need to add a layer of security to authorise resource requests. A great way to do this is with JSON Web Tokens.

JWTs allow you to provide users with access to API endpoints and their resources in a granular manner. Access can be time limited, restricted to certain user groups and more.

The easiest way to add JWT authorisation to Slim PHP is via the library PSR-JWT. It is a PSR 7 / 15 compliant JWT creation and validation library, which works perfectly with Slim PHP as it is also PSR 7 / 15 compliant.

PSR-JWT is built on top of ReallySimpleJWT and it exposes authorisation middleware which can easily be added to Slim PHP's routing system.

Here's an example of how to add the middleware to a Slim PHP route:

require '../../vendor/autoload.php';

$app->get('/route/example', function (Request $request, Response $response) {
    $response->getBody()->write("JSON Web Token is Valid!");

    return $response;
})->add(\PsrJwt\Factory\JwtMiddleware::json('Secret123!456$', 'jwt', 'Authorisation Failed'));

It's literally a few lines of code, you just pass the JwtMiddleware::json() method a token secret, a request key and a response message. If the JSON Web Token passed with the request is invalid you'll see the response message and if it is valid the route will load as expected.

PSR-JWT is also completely customisable, you can even use your own handlers to define how authorisation works and what the response should be. You can also use the library to generate JSON web tokens.

require 'vendor/autoload.php';

$factory = new \PsrJwt\Factory\Jwt();

$builder = $factory->builder();

$token = $builder->setSecret('!secReT$123*')
    ->setPayloadClaim('uid', 12)
    ->build();

echo $token->getToken();

Have a read of the documentation to find out more about all the features available in PSR-JWT. Also if you want to understand JSON Web Tokens in more detail I suggest you give RFC 7519 and RFC 6750 a read. If you have any questions feel free to drop me a message on Twitter @RobDWaller.

What I've Learnt In 10 Years as a Developer

Rob Waller — Wed, 08 May 2019 11:38:12 +0000

At the end of last month I turned 35 years old and I realised I have been a developer for about ten years. I began coding when I was 21 and I got my first coding job, building email templates, when I was about 23. But it wasn't until I turned 25 that I got my first development job, writing Visual Basic and SQL for an email marketing company.

So I feel like it is an appropriate moment for me to reflect and share some of my learnings from the past ten years of work. And I hope this will help new developers, and even old developers like me, progress with their careers.

Don't Give Up

It is hard to learn to code, so hard you never cease to learn nor face challenges. This can intimidate new developers, but the best piece of advice I can provide is, don't give up, you will get there in the end. When I began to write Object Oriented software I found it really hard. I learnt HTML and CSS quite quickly, it wasn't very complicated 15 years ago. But I had no background in computer science, I have a degree in history, so OOP and SQL were difficult to learn and there were points when I thought I should give up.

I didn't though and this was in part due to a great piece of advice I received from a Senior SQL Developer, a guy called Owen. The advice was simple, continue to read, listen and practice, and eventually it will click. One day in the office Owen called me over to his desk and said to me, "Rob, I want you to sit with me and I'm going to talk to you about SQL. You probably won't understand most of it, but if you just listen it will go in your head and one day in the future some of this stuff will begin to click."

Owen talked to me for an hour about topics like joins, inner queries, indexes and all sorts of stuff, which at the time went over my head. But he was completely right, about six months later I faced a SQL issue and some of the stuff Owen mentioned began to make sense and I solved a problem I never thought I could. Owen's point was it takes time to learn, but if you keep at it eventually you will get it. So don't give up and look for great mentors like Owen to help you along the way.

Give It a Try

I've done a lot of different things in my career, but the most important was my decision to start a business. I was 27 and I decided social media marketing was going to be a big thing, and I would try to do something with social media analytics.

When I began my business I did everything wrong. I quit my job, I had no capital or savings to begin with, I was paying rent I could not afford, and I had no real plan. I wasn't even that good at coding. But what I did was start working, really hard. I worked very long hours and scraped by for quite some time.

After about 18 months one of my old work contacts put in touch with a company who were interested in what I was doing. The initial idea was for them to invest in my business and become part owners. Sadly once again I had no idea what I was doing and in the end we could not agree an investment strategy that would work. So after two years of struggle I decided I'd just sell them the social media marketing tool I'd built.

By the time I was 29, I was nearly bankrupt, the relationship I'd been in had fallen apart, mainly due to me, and I had no business. At this point you may be asking, why on earth did you do this? The answer is simple, it was one of the best experiences of my life, and fast forwarded my development career. I learnt more than any classroom or book could teach me about the business of technology and I became a much better software developer.

There were also some fun parts, I wrote a Twitter analytics algorithm that got me into Mashable and on to CNN News. And I got a free trip to LA to speak at a marketing conference about social media analytics. And overall nothing really bad happened, I tried something, it didn't work out, I went back to what I was doing, but I was earning a lot more money.

If there is something you want to try or do, just give it go, see what happens. It might not be for you in the end, but at least you'll know and you can easily go back to what you were doing previously. And it doesn't have to be as mad as starting a business, it could be to try free lance work, or remote work, or a role in management, or a contribution to an open source project. Just give it a go, you've got nothing to lose and you won't regret it.

You are Ignorant

Over the years I've learnt a great deal about myself, I can be arrogant, blunt, loud, dismissive, aggressive, single minded, difficult, selfish and worse. And I have to try and control these traits as much as I can. But the most important thing I've learnt is I am ignorant. As I've learnt more and gained experience I've come to the realisation I don't know much. This isn't a new concept I've stumbled upon, Socrates spoke about man's ignorance 2,500 years ago.

But it is a really important life lesson to understand your ignorance, or what you actually know. For instance I can code in numerous languages, but I only know PHP well. But even within PHP my areas of expertise are limited to Test Driven Development, Code Quality and JSON Web Tokens. I don't know a great deal about any of the PHP frameworks, I'm certainly not a Laravel Expert.

My advice to all developers is accept your ignorance because it is not a bad thing, it is natural. If you know a little about something it doesn't mean you understand it well or are an expert in it, so don't pretend you are. It's ok to say, "I don't know, I'll have to look into that." In fact, it's the mature and sensible thing to do. And if someone reprimands you for saying it then they are an idiot.

It can be difficult to work with those who have not come to this realisation. But one tactic which can help is the Socratic Method, which means ask lots of questions, all the time. Your aim should be to minimise confusion among team members, as confusion is one of the horsemen of the software development apocalypse. It should be avoided at all costs.

And finally, love your QA! A developer who accepts their own ignorance understands it is far more likely there is a fault in their code than their QA is wrong. So don't argue with them, work with them to understand and solve the issues they raise.

Avoid Stress

This is simple, avoid stress, like it were the plague, but like a worse plague, where people's heads explode. Stress adds nothing to your life, it's not a motivator, it doesn't make you more focused, it's just a life negative. And the consequences of stress can be significant, it can ruin your life.

The solution to stress is relatively straightforward, stop doing the thing that causes you stress. This is easier said than done and requires you to have a good understanding of yourself; who you are as a person; what your strengths and weaknesses are; and what you like and what you don't like.

I've realised I'm a bad employee but a good contactor / consultant. My main skill is analysis, which I believe comes from my degree in History. This means I am very good at spotting problems and defining solutions. I'm a bit like a problem seeking missile, I almost can't help myself, I just have to look under the rock in the dark corner.

The issue is, once I've found the problem I want to solve the problem, and the reality is many companies are not interested in solving problems. Their attitude is, "Has anyone died yet? No? Well carry on then..." When I've been an employee of companies like this it causes me enormous frustration and therefore stress, because I can't do the thing I enjoy and I hate doing things badly. Whereas when I'm a contractor or consultant the relationship is more like, "Here's a list of your problems, would you like to fix them? No? Ok, pay me, see you soon." I can detach myself from the issues emotionally, because it's their problem not mine, and this reduces my stress levels dramatically.

The point I'm making is stress is a unique experience, I know developers who cannot fathom the concept of contracting. They can't do it because it would just stress them out, "Where's my next pay check coming from?!" Learn who you are, and create a work life situation that minimises your stress levels.

Be Careful With Alcohol

Alcohol is fun, there is no doubt about that, and I've had lots of great times on alcohol. But I've also learnt I probably have a drinking problem. Not a, I need a drink to function kind of problem, but a drinking problem nonetheless. I'm not going to write about it in detail now, I'll do that another time.

But the point is you should be very careful with alcohol as it can have serious consequences. I've helped fire people because they drank too much and became violent. I've also seen even worse things happen as a result of alcohol, and all in a work setting. I also know, from my own experience that consuming lots of alcohol on a regular basis can seriously effect your work performance and your home life.

The combination of stress and alcohol is even worse, an explosive mix. I for a long time self medicated stress with alcohol, and it's not a good thing as alcohol seriously effects your judgement, behaviour and often brings out your worse traits, see above.

You should also be careful of companies who have drinking cultures. I used to think drinking cultures were fun, and I played my part in them, but they usually exist to hide a company's demons. Which more often than not is people are under too much pressure and stress levels are high.

I'm not saying don't drink or don't enjoy a drink with your work colleagues, just be careful. If you're drinking to help with stress; or you drink large amounts often, once a week or more; or you drink more often with work colleagues than friends and loved ones; you probably need to get some help, or make some significant work life changes. Don't let the demon drink take a hold, as my mother likes to say.

Fin

To finish on a positive note, despite the many challenges a career in development can present, I truly believe it's a great job. And I wouldn't change what I do for all the Burger Kings in America... ;). I enjoy learning new things every day and there is no feeling like writing good code. I hope the above notes are helpful and no matter where you are on your career path keep plugging away, it will all be worth it.

Picture: "No Hassle" by goehler.mike is licensed under CC BY-ND 2.0

The Law of Minimum Cost

Rob Waller — Tue, 19 Mar 2019 09:59:51 +0000

The Law of Minimum Cost in software development relates to the financial cost, including time and materials, to complete a software development task.

The law states:

Every software development task has a minimum cost which covers the creation of code that functions well and is bug free.

The Law of Minimum Cost has a number of principles:

It is hard to achieve the minimum cost.
It is impossible to complete a task for less than the minimum cost.
Costs will grow until the software functions well and is bug free.
If defects and maintenance are ignored costs will grow via side-effects.

Good tech teams and companies aim to get as close as possible to the minimum cost of development. This can take time and will require good planning, code analysis, review, and excellent testing, all executed before code is pushed to production. These steps will reduce defect density and improve quality, which means a team will get closer to the minimum cost.

By contrast if a team rush tasks, cut corners and release code too early costs will spiral as they will have to fix bugs and faulty functionality which are in production. It will force the team to divert resources to maintenance and bug fixing which will reduce their efficiency. This will cause the company to hire more developers or wait longer for new functionality, and both will cost them financially.

There are also untracked financial side-effects caused by poor functionality and bugs in production. For example, they may result in fewer users interacting with the software, leading to fewer conversions. Or they may drive inefficiencies in other company departments which are reliant on the software. Overall the poorer the work and the higher the defect density the more costs will diverge from the minimum cost.

My advice generally is you cannot save money on a development task, you can only postpone the point at which you pay for it. I encourage all those involved in the tech industry to consider this before they plan their next development task or project.

Article Note: This 'law' has been working its way around my head for a couple of years. It comes from my experience as a developer and manager working with companies and clients who often try to cut corners in the mistaken belief it will save them money. My aim is to arm developers with language and concepts which will help them better communicate this and other fallacies to the business and commercial people they work with. I'd really appreciate your feedback on this as a concept.

Why Tech Projects Fail: Management, Planning, Process

Rob Waller — Mon, 06 Aug 2018 12:18:28 +0000

Tech projects fail all the time, or they produce less than is desired. For instance, the product works but the code is awful. As developers we've all worked on projects like this, or had to tidy them up, and it can be frustrating. So it's important we understand why tech projects fail so we can reduce their occurrence. And hopefully make development a more enjoyable activity and job.

As developers when we discuss project failure we're generally referring to code quality. Low quality code has four features:

Poorly written
No documentation
Buggy
Fails to meet business requirements

Many developers believe code quality is related to principles and tests. This is logical as developers who fail to follow principles or write tests produce poorly written, buggy code. But code principles and tests, while important, do not cause project failure or directly reduce code quality. They are symptoms rather than the root cause. Their absence highlights project failure but does not explain it.

Projects fail for three reasons:

MPP

Management
Planning
Process

Management

Management and managers are the basis of project failure. Everything that can go wrong with a project derives from management.

Bad code is produced, as mentioned above, because developers don't write tests or follow code principles. But the reason developers don't follow code principles or write tests is due to experience and or time.

It is managers who define both experience and time. Managers are the ones who hire developers and if they only hire inexperienced developers things will go wrong, quickly. Inexperienced developers will often be able to make a product work, but they won't always appreciate the importance of principles, tests and documentation. And often they will struggle with push back, telling management they're wrong. Code quality can only suffer in an inexperienced environment, and this ultimately will lead to project failure.

There is nothing wrong with inexperienced developers, I was one for a long time. Inexperienced developers though require support and must be assigned suitable work. They should not be given too much responsibility and they must be mentored. Ideally by someone who can explain the value of tests, principles, documentation, and attention to detail.

Time, the other issue managers struggle with, is a result of complexity. The more complicated a project the more time is required. Managers often fail to understand project complexity and therefore set unrealistic or ill-informed timelines. This applies too much pressure and developers under pressure will often skip tests and principles. This will always result in bad code and errors.

The actions of managers have a strong link to code quality and project failure. If managers make bad decisions code quality falls and projects fail. If managers want to avoid project failure they need to hire the right team for their project. They also need to understand their project's complexity to define realistic timelines and this requires planning.

Planning

Planning has four aims:

Understand project complexity
Highlight known unknowns
Minimise unknown unknowns
Define priority.

Many organisations fail to plan adequately, and some even refuse to plan. They don't see its value or believe their project is simple and it's possible to 'just get on with the work'. In my career I've only worked at one organisation who did proper planning. This is extraordinary given I've been coding professionally for over a decade. Also, it means my career is pot marked with failure and low quality code.

It is essential to plan, every project and task, no matter how small, requires planning. As a manager and a developer it is sensible to assume a project or task is more complicated than it seems. Most tasks, once you consider testing, coding, documentation, review and deployment, take at least two to three days to complete. If you're timelining in hours you're probably not considering complexity properly.

So what does planning look like? It can take many forms, but its main aim, as mentioned, is to understand complexity.

This can be achieved by sitting with a client and writing some user stories. Define what they want their users to achieve and why. Alternatively you can hire business analysts and UX specialists to learn about and document a client's business and user needs. Note a UX specialist is someone who researches user behaviour, not someone who just draws up sitemaps and wireframes.

There is also Behaviour Driven Development. Ciaran McNulty has been pushing BDD principles in the PHP community for a number of years. In his latest talk at PHP London he highlighted the benefit of example mapping which aims to understand project complexity by writing requirements, rules and examples.

An example requirement may be:

Shop A has a user loyalty scheme.

For each dollar spent by a user the user receives one loyalty point.

On the surface this requirement seems simple. One dollar equals one loyalty point. But there are questions to answer or rules to define, for instance, what about tax?

This is where an example can help:

User 1 purchases Product A.

Product A costs $10 and there is 20% tax.

Product A's total cost is $12.

User 1 receives 10 loyalty points.

With an example the complexity of the loyalty scheme is clearer. Users only receive loyalty points for the cost of the product not the tax. Now imagine there are hundreds of examples for a big project. Suddenly the project is easier to understand, it is clearer what has to be done, and it's more testable. Also the unknown unknowns are reduced dramatically.

If you haven't read about Behaviour Driven Development I suggest you do. Even if you are unable to use it the underlying principles will help you become a better developer and or manager.

Project priority must also be clear before coding begins. If data entry is the most important aspect of a system, don't begin by building a notifications system.

Priority must be based on evidence, not whim. Just because the CEO or manager thinks something is important doesn't mean it is. Priority should be based on user feedback, user research or financials. If you're losing 20% of purchases due to a convoluted checkout process fix this before you implement the CEO's rebrand initiative.

Priority also changes over time and based on new information. So it must be tracked and fed into the planning of each feature. If you haven't got a clear priority list based on evidence then you will fail to deliver what you need to.

Planning is essential to project success. Developers need absolute clarity on what they need to produce. They cannot fulfil all the business requirements if there isn't a clear plan with well defined priorities and requirements. Code quality will also suffer without a plan. If developers have to constantly amend their code as new requirements or priorities are discovered 'hacking' will begin. As a result tests and principles will go out the window, quality will fall and the project will begin to fail.

Process

Process is the final aspect of a good tech project. It is like the cherry atop the cake. When discussing process we must accept code is only one aspect of the process, and not the most important. If your process looks like 'write code', 'deploy code' something is horribly wrong.

A tech process has three aims:

Ensure quality
Minimise bugs
Avoid delays

And a simple tech process may look like:

Sprint planning
Code
Code review
Bug fix
Quality assurance
Bug fix
Deployment
Sanity check

You could add more steps, such as documentation and automated testing. But the main point is 'code' is only one part of a multi-step process. The most important parts are testing and review. You have to assume there will be bugs, no matter how experienced the developer or how good the planning. So there must be steps to catch and fix bugs.

Another aspect of good process is communication. There must be a clear means for your team to communicate with each other. It is particularly important for them to be able quickly communicate problems or blockers. For me this is a five to ten minute daily stand up, for others it may be a Slack channel. Problems need to be exposed quickly as this will minimise delays as it allows resources to be reallocated and priorities amended.

Finally, teamwork is essential. Developers should never work in a silo on their own. Every piece of work should be looked at by multiple developers. A code review will cover some of this but it's important that developers plan and discuss their work together. This will help solve problems faster. It will also reduce pressure as no developer will feel solely responsible for a piece of work.

If you haven't got a clearly defined process a good place to start is the Joel Test. It was written by Joel Spolsky in 2000, and it defines 12 aspects of a good development team and process. You may not agree with all of it but a lot of it is valuable advice and still relevant almost 20 years later. Use it to assess where your team is today and then let it guide your progress. It will be difficult to achieve a perfect score, but if you improve in just two to three areas you'll feel the benefit.

Without a process code quality will suffer. Developers will work independently, produce code of varying quality and lots of bugs will slip into production. The knock on effects of this can be enormous. They may even result in complete project failure, so it is essential to implement a well structured process.

The Budgets Myth

Some in the tech community will argue it's not possible to get MPP right. Often they will argue there isn't enough budget and therefore time to do proper management, planning and process.

"We just need to get on with the work..."

This is one of the biggest myths in software development. Also those who use it are displaying their ignorance of 'the business of tech'.

The truth is every tech project has a 'full cost' relative to the team employed to deliver it. It will take a set amount of developer days to fully deliver a project. Fully deliver means all business requirements are met and the project is bug free.

The full cost is always met, eventually. The only question is whether the full cost is met pre or post release and by whom. Bugs released into production will have to be fixed and the owner, client or supplier will need to meet these costs.

Shortcuts can be taken pre-release but costs will begin to spiral post-release and drive additional costs. Bugs are significantly more expensive to fix post-release. Usually because they are not budgeted for, may require structural changes, slow down other work and will require enhanced testing. A lack of documentation means developers take longer to understand existing code. This will decrease velocity and drive up the cost of bug fixes and system extension. Business requirements not fully met may reduce efficiency in other teams driving up costs in other parts of the business. This final point can have significant untracked financial implications for businesses.

As an example, I designed a data warehouse and visualisation system for a client. I predicted it would take a team of three developers and a part time project manager about six months to deliver version one. The client were not happy with this quote and gave my design to a company who said they would deliver the project in eight weeks.

Six months after the project began it still wasn't fully delivered. Also to save time the developers involved decided to connect the visualisation tool directly to MongoDB. In my plan MongoDB was part of the data transformation pipeline. A temporary data store, the data visualisation tool was never meant to connect to it. This shortcut saved time but it meant the data analysts didn't have a standardised and well structured dataset to consume in the visualisation tool. The warehouse was almost useless to them, they couldn't do their job and they began looking for alternatives.

The client decided in this case to go with a cheap quote rather than a realistic one. As a result the project overshot and it drove business inefficiencies in the data team. The very team the project was designed to help. Eventually the full cost of this project will be met, hopefully by the supplier, and my original quote will likely be exceeded dramatically. It is even possible the project will be shelved, resulting in an enormous waste of money.

The reality is if a project costs $100,000 to complete you will pay $100,000 to complete it. If you try to cut corners and only spend $50,000 you'll most likely end up spending more than $100,000 as costs accrue over time. The only way to reduce a budget is to cut the number of requirements until the work can be fully delivered for your budget.

Management, planning and process are essential to the good running of any tech project. Without them code quality will suffer and projects will fail. They are also closely linked to the financials of a project and if they are not right or ignored costs will spiral. As developers we need to focus on MPP and make sure it's right before we worry about code. Without good management, planning and process it is simply not possible to write good code and produce great products which is what all developers want to do.

How PHP Type Declarations Actually Work

Rob Waller — Thu, 19 Apr 2018 11:49:37 +0000

Type declarations are a simple programming concept that lots of developers use on a daily basis. In the code example below we see a basic method that uses type declarations. The method places a string type hint on the $foo parameter and a string return type on the method itself.

function bar(string $foo): string
{
    return $foo;
}

$result = bar('Hello');

var_dump($result);

// Output: string(5) "Hello"

These type declarations tell us two things about the method. It accepts one string argument and it will return a string. In a strictly typed language like Go if the bar() method is called and an integer argument is passed in, eg 123, an error will occur. The code will also error if the method returns something other than a string for some reason.

Type declarations serve two basic purposes in code:

Low-level data / code integrity: it is more difficult to misuse methods and type checking is reduced.
Code readability: it is clearer what a method accepts and what it will return.

PHP has historically been a weakly typed language. Type declarations were only introduced in PHP 5, and in a limited form. Full type declarations and strict types did not appear until PHP 7's release in 2015.

PHP 7 was an exciting step forward for those of us who like the concept of type declarations and strictly typed code. There are though two important caveats that PHP developers should be aware of.

PHP 7 is a Weakly Typed Language

By default PHP 7 remains a weakly typed language. This may seem counterintuitive given the introduction of full type declarations and strict types. But it was decided to aid backwards compatibility that type declarations would not be forced on PHP code. Developers can turn strict types on by placing the declare(strict_types=1); method at the top of a PHP file.

This implementation means that PHP will 'ignore' type hints and return types unless the declare(strict_types=1); statement appears at the top of the file. This makes some sense as there is a lot of legacy PHP code still in use. And developers should be able to upgrade to the latest version of PHP without rewriting their entire codebase.

This approach does have some consequences though. If a developer defines type declarations but does not add declare(strict_types=1); PHP will use type coercion to make things work. Type coercion basically means a value of one type will be cast to a value of another type when required. This is less than ideal when using type declarations as methods may not function as expected.

As an example, if we edit our 'foo bar' code from above we'll see the type coercion at work. The bar method will now have an integer type hint on the $foo parameter and a string return type on the method. When we call the bar method with an integer argument of 123 the method will return 123 as a string. This is because type coercion has converted the integer value to a string value to comply with the defined return type.

function bar(int $foo): string
{
    return $foo;
}

$result = bar(123);

var_dump($result);

// Output: string(3) "123"

It's only when we add declare(strict_types=1); to the top of the PHP file that the type declarations are imposed.

declare(strict_types=1);

function bar(int $foo): string
{
    return $foo;
}

bar(123);

Now when we call the bar method with an integer argument of 123 we get an error. This is because the method expects to return a string but it is attempting to return an integer. Strict types have been imposed on the method and the following error is outputted.

FATAL ERROR Uncaught TypeError: Return value of bar() must be of the type string, integer returned

Strict Types Are Imposed on a File by File Basis

The other caveat to be aware of is strict types have to be imposed on a file by file basis, they cannot be applied globally. This means you have to place declare(strict_types=1); in every file where you want strict types to be imposed. Again this has been done to help backwards compatibility, but it means you cannot easily impose strict types across an application.

To show this in action let's look at another example. Imagine we have a small application that pulls in two functions from two separate files, foo.php and bar.php. We'll add a declare(strict_types=1); to the top of the application file just to prove it has no effect on the files pulled in. Also we'll place the methods we call inside a try catch to handle any errors that are thrown.

/**
 * This declares strict types but has no effect on the code.
 */
declare(strict_types=1);

/**
 * Call in the foo.php and bar.php files.
 */
require __DIR__ . '/../src/foo.php';
require __DIR__ . '/../src/bar.php';

try {
    /**
     * This method will output a string 123.
     */
    var_dump(foo(123));

    /**
     * This method will throw an error.
     */
    var_dump(bar(123));

} catch (Error $e) {

    /**
     * Print out the error message thrown by the bar(123) method
     * "Return value of bar() must be of the type string, integer returned"
     */
    echo $e->getMessage() . PHP_EOL;
}

In the foo file we'll have a foo method that accepts an integer and returns a string. In this file there will be no declare(strict_types=1); statement.

function foo(int $bar): string
{
    return $bar;
}

In the bar file we'll have a bar method that also accepts an integer and returns a string. This time though we'll have a declare(strict_types=1); at the top of the file.

declare(strict_types=1);

function bar(int $foo): string
{
    return $foo;
}

When we call the foo(123) and bar(123) methods respectively we'll get some very different output. The foo method will return a string so we'll see an output of string(3) "123". By contrast the bar method will throw an error because strict types have been imposed. We'll see an error message outputted, Return value of bar() must be of the type string, integer returned. This example shows strict types can be both on and off in the same application.

The Consequences

I don't wish to get into the rights and wrongs of PHP's approach to type declarations, smarter people than I have designed and implemented them. But the consequences of this approach should be obvious. The most important is PHP's type system cannot be trusted. And this means low level integrity errors can slip into code unless you are meticulous with declare(strict_types=1); statements.

It also means you should be careful when testing code. For instance when unit testing with PHPUnit you should use assertSame() instead of assertEquals(). This is because assertSame() checks value and type where as assertEquals() only checks value. And if you want to ensure code integrity you need to check type as well as value.

Type declarations are a big step forward for PHP and I believe all PHP developers should use them whether declare(strict_types=1); is on or off. It is though important that PHP developers understand how they work as misunderstanding PHP's type system will lead to bugs and will have negative effects on applications.

For more information on PHP's type system I suggest you read the RFC on PHP scalar type declarations.

Five Tips for Aspiring Tech Team Managers

Rob Waller — Wed, 04 Apr 2018 14:02:16 +0000

To become a tech manager and learn how to be good at it is difficult. There isn't a great deal of 'documentation' available and developers don't discuss the topic a lot. Just take a look at your average code conference, there are very few talks on management. Sadly this means most new tech managers and lead developers learn on the job, which isn't always ideal.

I've been a tech manager for two years now and it's been a real learning curve. There are lots of thoughts and insights I could share, but there are five I believe important. They are useful for any aspiring manager or lead developer, and even those who haven't considered a move into management yet.

Great Developers Make Great Managers?

There is often an assumption the most skilled developers will make the best managers. And a developer should not go for a management role unless they are the best developer in their team. This is a fallacy and a lie, you don't have to be a great developer to be a great manager.

It is important you have experience and a good understanding of code and technical issues, but you don't have to be the best in your team. Management is about a lot more than your field of expertise so don't let your code capability put you off a management role, you may be great at it.

A good analogy for this point is football or 'soccer', which like development is a highly skilled profession. Let's begin with a question, what do the following football managers all have in common?

Alex Fergusson
Arsene Wenger
Pep Guardiola
Jurgen Klopp
Jose Mourinho

Answer: none of them were particularly good at football, at least not exceptional. They are though some of the most decorated managers in the game.

The theory goes average football players make better managers because they have higher empathy. They get more out of less capable players because they better understand their limitations and how best to use and deploy them. Great players by contrast often can't comprehend why average players can't do what they can, leading to frustration and arguments.

Average players may also focus on developing other skills such as communication and strategy that are useful for managers. There is a good article on some of the data behind football managers in the FT that covers some of these issues. It's worth a read for any aspiring manager.

Overall football and sport highlight that many of the best managers are not always the most technically skilled. So remember that technical weakness does not exclude you from a management role.

Mistakes Will Happen

This is simple, when you begin your career as a manager you will make mistakes. If it's your first management role you won't know how to respond to every situation correctly. In the two years I've been a manager I've made numerous mistakes; I've hired the wrong people; I've overreacted and become angry unnecessarily; and I've been too ambitious which has led to failures.

The reality is you will make mistakes and it's ok, own up, apologise, note them down, change and move on.

Expect the Unexpected

No one can write a precise guide to management so expect and be prepared for the unexpected. You may have a lot of experience dealing with technical problems but when you become a manager you will have to deal with issues that sit outside your technical expertise.

You will be exposed to people's personal issues; their mental health problems; their love lives; their arguments; a wide range of issues. Expect this and be prepared to deal with these issues in as measured and sensitive manner as possible.

Learn to Compromise

This is important, you are not always going to get your own way and you are going to have to make the best of bad situations. Tech teams sit within businesses, so technical issues and consequences are not always going to drive business decisions. As a manager you are going to have to negotiate and compromise with other departments and managers that have different motives and aims.

Sometimes these aims are going to conflict with technical best practice and may effect your team negatively. You can't always respond in a combative or belligerent manner. You are going to have to compromise and come up with solutions to deal with these issues.

One way to do this is to define what your 'lines in the sand' are and then compromise on everything else. This is a two step process that focuses on what is understandable to the non-technical and what may have a negative impact on your organisation. Usually in financial or reputational terms.

Code principles for example fail to match these criteria as they're not understandable to the non-technical. You can't for instance turn around in a meeting and say, "I simply refuse to release the product this week because Dave's code doesn't comply with the Single Responsibility principle..."

// Dave's Code
// Convert every integer in the array so it is divisible by five.

public function everyValueDivisibleByFive($array)
{
    $newArray = [];

    if (!count($array)) {
        $result = 'error';
    }
    else {
        foreach ($array as $row) {
            if ($row % 5 != 0) {
                $remainder = $row % 5;

                echo $remainder;

                if (5 - $remainder < 3) {
                    $addition = 5 - $remainder;

                    $newArray[] = $row + $addition;
                }
                else {
                    $newArray[] = $row - $remainder;
                }
            }
            else {
                $newArray[] = $row;
            }
        }

        $result = $newArray;
    }

    return $result;
}

Dave's code is terrible but it works. This is all the business cares about and nobody in any other department is going to care about its quality. This is not to say you shouldn't encourage your team to follow code principles. But for various reasons you're not always going to be able to follow them so there is no point being belligerent about them.

A better 'line in sand' will focus on an issue like security. This a more understandable topic for the non-technical and can have a direct impact on an organisation. As an example you may outlaw certain tools such as WebMin or PHPMyAdmin. These tools can help developers and improve efficiency but they also expose servers and databases across the web so can be a security hazard and a business threat.

Other good 'lines in the sand' include:

Testing: demand that there is time to QA and test before release to reduce the number of bugs found in production.
Budgets / Timelines: demand that budgets and timelines match what can be delivered so developers face less pressure and produce fewer bugs.
Deployments: demand that code isn't deployed on a Friday so fewer problems occur over weekends when they're difficult to fix.

All of these issues are understandable and easily explained to the non-technical. They will also stand as solid principles to guide your team and provide them with the time and space to develop bug free and secure software. Overall this will be good for your business, reputation and profit margins.

Keep Coding

There is no reason to stop coding when you become a manager. I rarely code at work now, but I continue to work on personal projects and regularly attend tech conferences. It's important to keep your toe in the water and there is no reason you shouldn't.

Get involved in an open source project, or write and maintain your own library. It only has to be a few hours a week. But if you continue to code it will help you stay in touch with what your team members are doing; what is happening in the wider tech community; and maintain your passion for the subject. This will increase your team's trust in your decisions and help you better advise your organisation. Overall this can only help you as a manager, so keep coding.

The transition from developer to manager can be difficult. But I believe the above tips and advice highlight you can learn to become a better manager in the same way you learn to become a better coder. It just takes time, patience and persistence. I also hope that this post encourages more of you to consider a management role. More managers with technical experience can only be good for our industry, our clients, our work, and our fellow developers.

Four Security Principles That Software Developers Should Follow

Rob Waller — Thu, 22 Feb 2018 13:08:26 +0000

Security is a topic that is often poorly understood by developers because many of them focus on the technical side of security rather than the wider topic which involves people, money, risk and business priorities. As a result we often see poor decision making, unnecessary complication and wasted resource.

It's important that developers when building or choosing security solutions pick the correct one for their business or organisational situation. And it's particularly important that junior developers understand the wider context within which security decisions should be made.

1. Avoid dogma and absolutism

In a recent dev.to article a contributor shared the following advice on the topic of JSON Web Tokens and local storage.

The biggest security offenders I see today are those of us who store JWTs (session data) in local storage. Many people don't realize that JWTs are essentially the same thing as a username/password. If an attacker can get a copy of your JWT, they can make requests to the website on your behalf and you will never know. Treat your JWTs like you would a credit card number or password: don't ever store them in local storage.

The post which this advice comes from is good, it's definitely worth a read, and covers many important issues relating to JavaScript local storage. Sadly though this statement on JWTs and security is misguided or at least lacks the nuance that developers need to understand.

The position taken on JWTs and local storage is an absolute one, "Don't do it!!" But where you store a JWT is not really of great importance, and storing it somewhere 'safe' doesn't guarantee security. The important questions to ask are, what are you storing in the JWT? And, what are you using the JWT to do or access?

If the answer to those questions doesn't include any Personal Identifiable Information, or includes minimal PII, then you can probably do as you wish with those JWTs. If by contrast your answer to the above questions is, "All their credit card information!!" Then you should probably consider an alternative technology to JWTs.

As an example, if you were to implement a content paywall as many online news publications now do JWTs stored in local storage will be a perfectly acceptable security solution. The content you are protecting is of low value, no PII, so the likelihood that a hacker will be interested in hacking this content is very low. JWTs though will stop your average 'run of the mill' web user from accessing the content without paying for it. A simple solution to a security requirement.

You'll note that this approach to solving a security problem is less dogmatic and absolutist. There is a tendency among talented developers to become dogmatic and absolutist, possibly because everything they see is 'bad' or at least less than perfect. A little like when Plato looked upon Athens in the 5th century BC, but like Plato this approach can lead to poor solutions and bad answers. And it can be unhelpful for those attempting to understand a topic, particularly if they are junior.

It is sensible when dealing with security to avoid dogma, absolutism and one size fits all statements. There isn't an equivalent to the moral absolute "Do not murder" as security involves more nuance.

2. There is no such thing as security

There is a great irony at the heart of security, which is that it doesn't exist. Recently Google Chrome announced on Twitter that they will be marking all sites using HTTP as "Not Secure". They already mark HTTPS sites as "Secure" in the URL bar.

🔐⚠️ The moment we've all been waiting for! Chrome will mark all HTTP sites as "Not secure" in July 2018. 🔐⚠️https://t.co/2eV4GuEa2y
— emily schechter (@emschec) February 8, 2018

This is bizarre as HTTPS, or HTTP via TLS, is a very useful security enhancement but does not guarantee security. It is perfectly possible to build a site and serve it via HTTP that is more secure than a site served over HTTPS.

Google's actions here are surprisingly irresponsible as they could encourage average web users to feel safe when they are not and so be less cautious in their actions and behaviours online. And this is without covering the topic of how the HTTPS connection is implemented, see CloudFlare flexible SSL. A more sensible approach may be to describe the connection as "Private" or "Public", but "Secure" or "Not Secure" is misleading.

There has never been anything that has been completely secure, and even with all the technical advancements we've made there still isn't. Security has always been relative to what is being protected. People have spent millennia building walls of one type or another, but no-one has ever succeeded in building an impregnable dome.

If you don't believe me just ask the Iranians. In 2009 the Americans, definitely with the help of the Israelis and probably the British, hacked into an Irainian Nuclear facility called Natanz. You'll probably remember reading about the Stuxnet virus which was the likely culprit. What is extraordinary about the hack is that the Natanz facility was air gapped and probably one of the most secure facilities in the world. This though did not stop the Americans from getting a virus into the facility and disrupting Iranian nuclear production processes.

If you're interested in this topic and stories like this I suggest you read Gordon Corera's book Intercept: The Secret History of Computers and Spies. It's a wonderful book that will contextualise the topics of security and hacking for you.

Good security involves building a wall that is higher than the value of the assets you're protecting. That is it will cost a hacker more to hack your system than they will gain from hacking it. This also means your security should be proportionate to what you are protecting. Don't for example air gap a server to protect a few email addresses you've collected from a web sign up form, that would be an extraordinary waste of money.

3. Understand the threat

When building your wall it is important to understand the threat you face. Security threats can be broken down into four basic groups:

Kiddy Scripters and Automated Threats: See most WordPress / Joomla hacks.
Skilled Hackers and Hacker Organisations: Anonymous, LulzSec
Organised Crime and Minor State Actors: The Mafia, North Korea
Major State Actors: USA, China, Russia, Israel, UK

The majority of developers will rarely deal with anything above level one. The reason for this is two fold, you have to be doing something of significant financial value and or of significant political value to move above level one. Examples of this would be sensitive government work, aspects of financial work and aspects of major corporate work.

Also the threats are diverse and won't necessarily relate to hacking data. For example your organisation may be much more vulnerable to a DDOS Attack than a data breach. As a developer it's important you consider how your organisation might be vulnerable, and the primary vulnerability may not always be financial or PII focused, it could be reputational. Embarrassing your organisation by taking your website offline may be of far greater value to hackers than stealing the PII you have on your servers.

The Fappening in 2014 is a good example of an organisation failing to understand a threat properly. In this case Apple failed to properly value the content they had on their iCloud system and therefore did not implement security features that could have limited the damage. For example sending out emails when a new device or strange IP connected to an account. The Fappening was an edge case as no-one had really considered the value of celebrity data before, but it does highlight that the threats organisation face may not always be logical.

4. Implement a proportionate solution

If you implement generic security solutions without properly considering the threat you face you may be no safer than if you implemented no security at all.

As a developer you must seriously consider the threats you face before you implement any security solutions. This is so that you can implement proportionate security measures. Proportionate does not simply relate to the security threat though, it also relates to how much money you have to spend. A poor nation cannot build the Great Wall of China, but it can defend itself if it understands the threat and deploys its resources sensibly.

Much of the world enjoys laughing at North Korea, Kim Jong Un and his crazy nuclear plans. However behind the madness there may be some logic. There is a theory that North Korea has recognised that America and the West has happily carried out regime change against Afghanistan, Iraq and Libya, but has been much more cautious in relation to Iran and Pakistan. The theory goes that this is because Iran may have nuclear weapons and Pakistan does have nuclear weapons. North Korea has recognised this and is rushing to build a bomb and missile so it doesn't become the next Iraq. It has assessed the threat and is responding logically given its limited means, and it may succeed in defending the North Korean regime from America's bombs. Nuclear bombs may not protect them from their own people in the end though.

I'm not advising you follow North Korea's example, however you should consider what is a proportionate security response given your organisation's resources. This is particularly important given very few of us work at Apple, Google or Facebook. Most organisations have limited resources and they cannot justify spending large sums of money on security features that only produce marginal gains. When considering your security response you should begin by asking the following three questions.

What am I defending and how valuable is it?
Who is it that poses me a threat?
How much resource does my organisation have to defend itself with?

If the answer to these questions are low value, limited threat and low resources, then the basics will suffice. For example securely encrypt passwords, don't store too much PII, implement CSRF policies, etc, etc. If the answers are at the opposite end of the scale then you will have to consider much more advanced security features. And if the answers are mixed you may have to compromise in certain areas.

The overarching point here is that there is no 'one size fits all' security solution. So dogma and absolutism simply don't apply to security. When thinking about security consider what you're protecting carefully and then do an analysis of likely threats. After this build your wall with the resources you have available.

How to Create a JSON Web Token Using PHP

Rob Waller — Wed, 31 Jan 2018 13:42:30 +0000

I have spent the last year intermittently working on a PHP JSON Web Token library called ReallySimpleJWT, and this week I released version 1.0.0. The code is accessible via GitHub and Packagist.

For those of you who have not used JSON Web Tokens before they are a URL friendly, token based, authentication system. And they allow you to easily transfer information via an encoded JSON payload.

The core benefits of JSON Web Tokens are twofold: you don't need to use sessions or cookies to maintain authentication between states; and you don't have to constantly call the database for user information as this can be stored in the token payload.

Each token is broken down into three parts and each part is separated by a dot.

Header: This contains information on the token type, usually JWT, and the hashing algorithm used, eg HMAC SHA256 or RSA.
Payload: This contains any information you wish to transfer about the user, eg the user identifier.
Signature: This secures the token and is a hash of the encoded header and payload, along with a secret.

// Token structure
header.payload.signature

// A real world token
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWV9.TJVA95OrM7E2cBab30RMHrHDcEfxjoYZgeFONFh7HgQ

JWT security is achieved via the signature which is created by hashing the encoded header and payload and securing this with a secret only known to the author.

When receiving a token from a user the author will then be able to validate the signature by re-hashing the received header and payload with the known secret and checking it matches the received signature. If anyone were to tamper with the header or payload the signatures would not match and authentication would fail.

If you wish to get started quickly with JWTs the ReallySimpleJWT library offers an easy to use interface for generating and validating JSON Web Tokens.

use ReallySimpleJWT\Token;

// Generate a token
$token = Token::getToken('userIdentifier', 'secret', 'tokenExpiryDateTimeString', 'issuerIdentifier');

// Validate the token
$result = Token::validate($token, 'secret');

It's perfect if you need to quickly implement user authentication on a simple API. The library also offers more advanced usage and functionality if you'd like to read the documentation.

How to Build a JSON Web Token in PHP

If you'd like to build your own JWT generator or just learn a little bit more about them the following guide will help. While the examples below are written using PHP the concepts apply to any language so all developers should find them useful. The full script is at the bottom of this guide.

Create the Header and Payload

To begin we need to create header and payload JSON strings. We'll do this based on two simple arrays each asserting a number of claims about the token. You can read more about claims in the associated RFC. For the header we define the type typ and the algorithm alg claims which are RFC standard claims; for the payload we'll create our own claim user_id.

// Create token header as a JSON string
$header = json_encode(['typ' => 'JWT', 'alg' => 'HS256']);

// Create token payload as a JSON string
$payload = json_encode(['user_id' => 123]);

Create Base64Url Header and Payload Strings

Next we encode our $header and $payload JSON strings as Base64Url strings. This is slightly different to a standard Base64 string and there is no built in PHP Base64Url method yet. So we have to do a bit of string replace magic which will replace + with -, / with _ and = with ''. This is so that the Base64 string is passed within URLs without any URL encoding.

// Encode Header to Base64Url String
$base64UrlHeader = str_replace(['+', '/', '='], ['-', '_', ''], base64_encode($header));

// Encode Payload to Base64Url String
$base64UrlPayload = str_replace(['+', '/', '='], ['-', '_', ''], base64_encode($payload));

Create the Signature

To create the signature we need to use the hash_hmac() method available in PHP and use the sha256 algorithm. We pass in a concatenated string of the Base64Url encoded header and payload $base64UrlHeader . "." . $base64UrlPayload. It's important to note we have to include the dot . between the two strings. We add a secret, ideally a strong one that is longer than twelve characters. The ReallySimpleJWT library enforces this principle, but for our example we don't need to worry. Finally we force the hash_hmac() method to return the output as binary data.

// Create Signature Hash
$signature = hash_hmac('sha256', $base64UrlHeader . "." . $base64UrlPayload, 'abC123!', true);

Base64Url Encode the Signature

Once we have created the signature we simply need to Base64Url encode it as we did with the header and payload.

// Encode Signature to Base64Url String
$base64UrlSignature = str_replace(['+', '/', '='], ['-', '_', ''], base64_encode($signature));

Create the JSON Web Token

Finally we create the JWT by concatenating the header $base64UrlHeader, payload $base64UrlPayload and signature $base64UrlSignature. Each part of the JWT is separated by a dot.

// Create JWT
$jwt = $base64UrlHeader . "." . $base64UrlPayload . "." . $base64UrlSignature;

// Output
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxMjN9.NYlecdiqVuRg0XkWvjFvpLvglmfR1ZT7f8HeDDEoSx8

And that's it, really easy. You can test the JWT that this code produces on the JWT.io website. The code is below in full and I'd suggest you read the relevant documentation on the JWT site along with the RFC.

You can of course use the ReallySimpleJWT Library if you wish and I will produce a post on validating JWTs in the next week or two. If you have any thoughts or have noticed any mistakes please message me @RobDWaller on Twitter.

The Script

// Create token header as a JSON string
$header = json_encode(['typ' => 'JWT', 'alg' => 'HS256']);

// Create token payload as a JSON string
$payload = json_encode(['user_id' => 123]);

// Encode Header to Base64Url String
$base64UrlHeader = str_replace(['+', '/', '='], ['-', '_', ''], base64_encode($header));

// Encode Payload to Base64Url String
$base64UrlPayload = str_replace(['+', '/', '='], ['-', '_', ''], base64_encode($payload));

// Create Signature Hash
$signature = hash_hmac('sha256', $base64UrlHeader . "." . $base64UrlPayload, 'abC123!', true);

// Encode Signature to Base64Url String
$base64UrlSignature = str_replace(['+', '/', '='], ['-', '_', ''], base64_encode($signature));

// Create JWT
$jwt = $base64UrlHeader . "." . $base64UrlPayload . "." . $base64UrlSignature;

echo $jwt;