DEV Community: Jack Marchant

Beyond Autocomplete: A practical guide to AI-Assisted Development

Jack Marchant — Wed, 08 Oct 2025 23:05:36 +0000

To truly leverage the power of AI in software engineering, we need to move beyond simple code completion. In this post, I want to explore some practical techniques for using AI as a development partner, from pair programming to bug hunting, and even look at what the future might hold for AI other parts of software development.

The New Pair Programmer: Your AI Sounding Board

We've all been there: stuck on a problem, talking it through with a rubber duck on our desk. The act of articulating the problem often illuminates the solution. AI assistants have become the ultimate interactive rubber duck. Instead of just stating the problem, you can have a dialogue.

By asking the AI questions like, "Can you explain this block of code to me in simple terms?" or "What are the potential edge cases for this function?", you are forced to structure your own thoughts. The AI's response, whether it's perfectly correct or slightly off, provides a new perspective that can break you out of a mental block. This interactive process is a powerful evolution of the classic debugging technique, turning a monologue into a productive conversation.

AI-Driven Test Generation: A Second Opinion on Functionality

Writing comprehensive tests is crucial, but it can be tedious. This is an area where AI can shine. Instead of just writing tests yourself, you can present a function to an AI model and ask it to generate a suite of test cases.

This approach offers two key benefits. First, it accelerates the process of writing boilerplate test code. Second, and more importantly, the AI might interpret the function's purpose differently than you intended. The tests it generates can reveal ambiguities in your code or highlight edge cases you hadn't considered. It acts as an impartial reviewer, testing what the code actually seems to do, not just what you intended it to do.

This is typically where most engineers start experimenting with AI, beyond simple code completion and moving towards code generation.

Accelerating Feature Development with Contextual Prompts

One of the most effective ways to use AI is for brownfield projects where established patterns already exist. Instead of a generic prompt like "write a function to fetch user data," you can provide the AI with specific context.

For example, you could provide an existing API endpoint function and prompt it with: "Following this example, create a new endpoint to handle product data, including validation for the 'price' and 'stock' fields." By giving the AI a clear template, you guide it to produce code that aligns with your project's existing structure, conventions, and style. This makes adding new features faster and helps maintain a consistent codebase.

Increasing Confidence by Detecting Bugs (or Proving the AI Wrong)

AI can be an excellent "second pair of eyes" for catching subtle bugs. You can paste a piece of code and ask the model to review it for potential issues, race conditions, or security vulnerabilities. It's surprisingly effective at spotting common mistakes.

Interestingly, even when the AI is wrong, it provides value. If the model flags a piece of code as buggy and you investigate and prove it's correct, you've just engaged in a deep-dive review of your own logic. This process of validating your code against the AI's critique significantly increases your confidence in its correctness.

What's Next? AI in System Design

What's next for AI? Can models gain enough context and direction to assist with architecture and system design? This is the next frontier, but it comes with significant challenges. A good system design is all about making trade-offs on constraints such as:

cost vs. performance
consistency vs. availability
scalability vs. complexity

For an AI to make meaningful contributions here, it needs a vast amount of context. It would need to understand business goals, budget constraints, team skill sets, and existing infrastructure. Simply asking it to "design a scalable microservices architecture" will likely result in a textbook answer boiling the ocean with "best practices" that aren't practical for your specific situation.

The future of AI in system design will likely involve a highly interactive process, where architects use AI to explore different design patterns, model performance trade-offs, and generate diagrams, but the final strategic decisions will still rest on human experience and deep contextual understanding.

My Take on the Current State of AI for software engineering

In my view, the real power of AI in its current form is not as an autonomous coder, but as a thought partner. Using an LLM as a sounding board or a way to fact-check your own assumptions gives a necessary structure to the development process. It forces you to articulate your problem clearly and highlights potential issues much faster, allowing you to focus your attention on double-checking the most critical parts.

This is why I use these tools both inside and outside of the code editor. Limiting myself to simple autocompletion is like using a smartphone only for making calls. The true value comes from a more holistic integration: brainstorming solutions, drafting documentation, and debugging complex logic in a conversational interface. The AI isn't replacing the engineer; it's augmenting the development workflow and becoming an indispensable part of an engineer's toolkit.

Query optimisation using database indexes

Jack Marchant — Thu, 29 Feb 2024 10:03:56 +0000

Originally published on jackmarchant.com

A common question in software engineering interviews is how can you speed up a slow query? In this post I want to explain one answer to this question, which is: to add an index to the table the query is performed on.

What is an index in a relational database?

An index in a relational database is a key-value mapping for one or many columns where the key is the data in the column and the value is the primary ID of the row that contains the data.
A primary index also exists in every database table so querying by ID is always fast. A custom index is a reverse-lookup to that primary index.

How does an index speed up database queries?

An index tells the database which rows contain specific values, without having to scan each row individually.

A common way to understand it is the index of a phone book.
If I was trying to find someone with the last name "Martin" in a phone book, I would skip to the back pages to the index, find names starting with M and start looking from the referenced page number.

A database does the same lookup with an index.

Let's take a look at a more concrete example. Suppose we create a new table:

CREATE TABLE `users` (
  `id`          bigint          NOT NULL AUTO_INCREMENT PRIMARY KEY,
  `name`        varchar(255)    NOT NULL,
  `status`      int             NOT NULL,
  `joined_on`   datetime        NOT NULL
);

A query to find the users where status is 1 would result in a full table scan.

explain select * from users where status = 1;
> ... | Extra
        Using where

If we add an index to the status column because we know it's a common access pattern for our application:

ALTER TABLE users ADD INDEX status(status);

When we run the explain again, we can see it

explain select * from users where status = 1;
> ... | key     | .. | Extra
        status  | .. | Using index

When the database performs the operations for this query it will use the index instead of scanning every row, which starts making a big difference when there are millions of rows to scan.

Handling complex queries with a composite index

Continuing with this example, let's assume we have another access pattern which is to find all the users with a specific status who joined after a certain date ordered from most to least recent.

explain select * from users where status = 1 and joined_on >= '2024-02-24' order by joined_on desc;
> ... | key     | .. | Extra
        status  | .. | Using where; Using filesort

Without an index on joined_on column the query could still benefit from the index we added on status. It may not be the best performance, however, with the addition of the joined_on filter and the sort, which would result in a filesort operation which could make overall performance worse.

We could go ahead and create an index for joined_on but the database may still choose the status index and perform a filesort.

What would have better performance is a composite index with both status and joined_on.

ALTER TABLE users ADD INDEX status_joined_on(status, joined_on);

After adding the index, this is what the explain looks like:

explain select * from users where status = 1 and joined_on >= '2024-02-24' order by joined_on desc;
> ... | key                 | .. | Extra
          status_joined_on  | .. | Using index condition; Backward index scan

An index can be stored in either ascending or descending order depending on the definition. We see Backward index scan because we need the reverse order (descending) to sort results for the query above.

If we were to create the index where joined_on column is sorted in descending order we would see the Backward index scan removed:

status_joined_on(status, joined_on DESC)

Now we can run the explain again:

explain select * from users where status = 1 and joined_on >= '2024-02-24' order by joined_on desc;

This is an ideal index for this type of query.

Summary

We explored creating indexes on relational databases and evaluated performance at each step. What did we learn along the way?

An index in a relational database is a key-value mapping for one or many columns to tell the database which rows contain what values without having to scan each row.
Indexes can speed up query performance at the cost of write performance, though the former typically outweighs the latter.
For complex queries, it's possible to create a multi-column index. Ordering the columns is an important factor in its performance.
A descending index can help with searches for most recent data.

Refactoring for Performance

Jack Marchant — Sat, 24 Feb 2024 08:54:45 +0000

I spend most of my time thinking about performance improvements. Refactoring is tricky work, even more so when you’re unfamiliar with the feature or part of the codebase.

Some refactoring might be simple, but in this post I’ll attempt to dissect my approach to solving performance issues in the hopes it’ll provide value for others.

Where do we start?

Before we can design a solution to a performance issue we must understand the problem. For example, is a page not loading or is it very slow? Are there more queries than necessary to get data? Can we see a slow part in the process? How do we know it’s slow? Answering these questions first is a must.

Once we can see the slow part over and over again, if code is the culprit, I start by taking that piece out and seeing how fast it could be without it even though it may break or be incomplete. This helps me to see what the maximum amount of improvement we’ll get through performance optimisation – as if code didn’t run at all.

This is the incentive. If I know how much performance improvement is possible, it’s worth investing time into figuring out a solution. If I see marginal or little to no improvement, I’m either in the wrong place or it wasn’t as slow as I thought - time to move on.

The solution to the performance problem could be as simple as adding an index and as complicated as a complete rebuild. Code optimisation will naturally take longer than query optimisation because the behaviour of the code will generally change. If the problem is not that the query is slow but that the query runs thousands of times in a single request - those are two different problems to solve.

Going from prototype to production

The easiest way I get from identifying something slow to being able to fix the problem is to prototype the way I think it should work to be fast. Creating a prototype gives me the confidence the solution works at a high level, without addressing all of the edge cases. At minimum, I try to identify blockers standing in the way.

Once I’ve proven the solution works, I can invest more time to understand the product behaviour and the experience. How does the user actually use this feature? What are they trying to accomplish?

To be clear: this is the hardest point and often where the solution can fall over. If I misunderstand requirements or forget to include some parts, however minor they may seem, it undermines the performance optimisation and deflates any confidence in it when it comes time to release it.

Confidence is a fickle thing - it can be gone in an instant and hard to get back quickly. Customers are never going to applaud performance improvements - maybe it should have been fast to begin with - but many performance improvements add up to a better experience.

Testing builds confidence

Testing a performance improvement is like any other test of a change with the addition of a specific metric that you want to improve. For example if the goal of the refactor was to reduce page load time, compare the previous and current page load speed. If reducing the number of queries was the goal, show that the number of queries has gone down. I often start with manual tests to confirm impact on the user experience supported by some quantifiable metric. Screenshots, videos or links to observability metrics all support the fact that the refactor does what was intended.

Once I’ve covered the performance gains, the next thing to verify is correctness. To do this, I start with a few manual scenarios and compare the result of using the feature with and without my change. The most comprehensive way to do this is through a test spreadsheet which marks pass or failure for some scenarios. A user clicks a few buttons and assert the result is the same. Using a spreadsheet helps maintain regression tests and add test cases over time. Some features won’t be big enough that you’d need it, but even if you never share the results with anyone and use it for your own testing - it beats remembering all cases every time you test.

One day you could even turn those manual tests into automated tests, if that’s not readily possible now. At least creating automated tests for any new code is a task worth doing.

How do performance improvements differ from features? Feature development creates new functionality where it didn’t exist before, so there’s often time to assess its effectiveness and test with customers who might be more forgiving if something is not working. To break an existing feature that may be slow is to take it away. We must have extra care when dealing with something that is working today for some, even if it’s slow.

A performance improvement must be:

Cheaper or faster
At least equal, ideally better behaviour

It’s an unforgiving task, but rewarding when you can quantify performance improvements with a better experience for customers. Monitoring the outcome after release is a good place to start, even in the short term to verify the improvement was a success.

The hardest question, which will remain unanswered, is how can we know when performance optimisations are done?

Exploring Async PHP

Jack Marchant — Wed, 31 May 2023 03:59:37 +0000

Asynchronous programming is a foundational building block for scaling web applications due to the increasing need to do more in each web request. A typical example of this is sending an email as part of a request.

In many web applications, when something is processed on the server we want to notify people via email and it's common for this to be a separate HTTP request to a third-party service such as SendGrid, Mailchimp etc.

This becomes a more than trivial example when you need to send a lot of emails at once. In PHP, if you want to send an email and the HTTP process takes 100ms to complete, you'd quickly increase the total time for the request by sending tens or hundreds of emails.

Of course, any good third-party email service would provide a bulk endpoint to negate this, but for the sake of the example - let's say you want to send 100 emails and each has to be processed individually.

So, we need to make a decision: how can we move the processing of the emails into a separate process so that it doesn't block the original web request?
That is what we'll explore in this post, particularly all the different ways this can be solved in PHP with or without new infrastructure.

Using exec()

exec() is a native function in PHP that can be used to execute an external program and returns the result. In our case, it could be a script that sends emails. This function uses the operating system to spawn a completely new (blank, nothing copied or shared) process and you can pass any state you need to it.

Let's take a look at an example.

<?php
// handle a web request

// record the start time of the web request
$start = microtime(true);
$path = __DIR__ . '/send_email.php';

// output to /dev/null & so we don't block to wait for the result
$command = 'php ' . $path . ' --email=%s > /dev/null &';
$emails = ['joe@blogs.com', 'jack@test.com'];

// for each of the emails, call exec to start a new script
foreach ($emails as $email) {
    // Execute the command
    exec(sprintf($command, $email));
}

// record the finish time of the web request
$finish = microtime(true);
$duration = round($finish - $start, 4);

// output duration of web request
echo "finished web request in $duration\n";

send_email.php

<?php

$email = explode('--email=', $argv[1])[1];
// this blocking sleep won't affect the web request duration
// (illustrative purposes only)
sleep(5);

// here we can send the email
echo "sending email to $email\n";

Output

$ php src/exec.php

finished web request in 0.0184

The above scripts show the web request still finishes in milliseconds, even though there is a blocking sleep function call in the send_email.php script.

The reason it doesn't block is because we've told exec with the inclusion of > /dev/null & in the command that we don't want to wait for exec command to finish so we can get the result, meaning it can happen in the background and the web request can continue.

In this way, the web request script is simply responsible for running the script, not for monitoring its execution and/or failure.

This is an inherent downside of this solution, as the monitoring of the process falls to the process itself and it cannot be restarted. However, this is an easy way to get asynchronous behaviour into a PHP application without much effort.

exec runs a command on a server so you have to be careful about how the script is executed, particularly if it involves user input. It can be hard to manage using exec particularly as you manage scaling the application, as the script is likely running on the exact same box that is processing external web requests, so you could end up exhausing CPU and memory if many hundreds or thousands of new processes are spawned via exec.

pcntl_fork

pcntl_fork is a low-level function which requires PCNTL extension to be enabled and is a powerful, yet potentially error prone method for writing asynchronous code in PHP.

pcntl_fork will fork or clone the current process and split it into a parent and a number of child processes (depending on how many times it is called). By detecting the Process ID or PID we can run different code when in the context of a parent process or a child process.

The parent process will be responsibile for spawning child processes and waiting until the spawned processes have completed before it can complete.

In this case, we can have more control over how the processes exit and can easily write some logic to handle retries in case of failure in the child process.

Now, on to the example code for our use case to send emails in a non-blocking way.

<?php

function sendEmail($to, $subject, $message)
{
    // Code to send email (replace with your email sending logic)
    // This is just a mock implementation for demonstration purposes
    sleep(3); // Simulating sending email by sleeping for 3 seconds
    echo "Email sent to: $to\n";
}

$emails = [
    [
        'to' => 'john@example.com',
        'subject' => 'Hello John',
        'message' => 'This is a test email for John.',
    ],
    [
        'to' => 'jane@example.com',
        'subject' => 'Hello Jane',
        'message' => 'This is a test email for Jane.',
    ],
    // Add more email entries as needed
];

$children = [];

foreach ($emails as $email) {
    $pid = pcntl_fork();

    if ($pid == -1) {
        // Fork failed
        die('Error: Unable to fork process.');
    } elseif ($pid == 0) {
        // Child process
        sendEmail($email['to'], $email['subject'], $email['message']);
        exit(); // Exit the child process
    } else {
        // Parent process
        $children[] = $pid;
    }
}

echo "running some other things in parent process\n";
sleep(3);

// Parent process waits for each child process to finish
foreach ($children as $pid) {
    pcntl_waitpid($pid, $status);
    $status = pcntl_wexitstatus($status);
    echo "Child process $pid exited with status: $status\n";
}

echo 'All emails sent.';

In the above example using pcntl_fork we can fork the current process, which copies the parent process into new child processes and wait for the execution to complete. Additionally, after forking the child processses to send emails, the parent process can continue doing other things, before ultimately ensuring the child processes have finished.

This is a step above using exec where we were pretty limited in what is possible because the scripts are completely separate contexts so monitoring is not possible from an overall perspective.

We also gain process isolation as each child process runs in a separate memory space and does not affect other processes.
By tracking the process IDs we can effectively monitor and manage execution flow.

A downside in forking requests in this way, directly from the web request (parent process) is that by waiting for the child processes to finish, there's no benefit to the response time of the original request in doing it this way.

Fortunately, there is a solution to this and it's to combine both exec and pcntl_fork to get the best of both worlds, which looks like this:

Web request uses exec() to spawn a new PHP process
The spawned process is passed a list of emails as a batch
The spawned process becomes the parent as it forks to send each email individually

This can all happen in the background, rather than blocking the original request.

Let's take a look at making this work:

<?php

$start = microtime(true);
$path = __DIR__ . '/pcntl_fork_send_email.php';
$emails = implode(',', ['joe@blogs.com', 'jack@test.com']);
$command = 'php ' . $path . ' --emails=%s > /dev/null &';

// Execute the command
echo "running exec\n";
exec(sprintf($command, $emails));
$finish = microtime(true);

$duration = round($finish - $start, 4);
echo "finished web request in $duration\n";

pctnl_fork_send_email.php

<?php

$param = explode('--emails=', $argv[1])[1];
$emails = explode(',', $param);

function sendEmail($to)
{
    sleep(3); // Simulating sending email by sleeping for 3 seconds
    echo "Email sent to: $to\n";
}

$children = [];

foreach ($emails as $email) {
    $pid = pcntl_fork();

    if ($pid == -1) {
        // Fork failed
        die('Error: Unable to fork process.');
    } elseif ($pid == 0) {
        // Child process
        sendEmail($email);
        exit(); // Exit the child process
    } else {
        // Parent process
        $children[] = $pid;
    }
}

echo "running some other things in parent process\n";
sleep(3);

// Parent process waits for each child process to finish
foreach ($children as $pid) {
    pcntl_waitpid($pid, $status);
    $status = pcntl_wexitstatus($status);
    echo "Child process $pid exited with status: $status\n";
}

echo "All emails sent.\n";

The beauty of this solution, albeit more complicated, is that you can set up a separate process all together whose responsibility it is to run and monitor forked processes for the purpose of doing work asynchronously.

AMPHP

amphp (Asynchronous Multi-tasking PHP) is a collection of libraries that allow you to build fast, concurrent applications with PHP.

The release of PHP 8.1 in November 2021 shipped support for Fibers which implement a lightweight cooperative concurrency model.

Now we know a little bit about how amphp works and why it's exciting for the future of PHP programs, let's take look at an example:

<?php

require __DIR__ . '/../vendor/autoload.php'; // Include the autoload file for the amphp/amp library

use function Amp\delay;
use function Amp\async;

function sendEmail($to, $subject, $message)
{
    delay(3000)->onResolve(function () use ($to) {
        echo "Email sent to: $to\n";
    });
}

$emails = [
    [
        'to' => 'john@example.com',
        'subject' => 'Hello John',
        'message' => 'This is a test email for John.',
    ],
    [
        'to' => 'jane@example.com',
        'subject' => 'Hello Jane',
        'message' => 'This is a test email for Jane.',
    ],
    // Add more email entries as needed
];

foreach ($emails as $email) {
    $future = async(static function () use ($email) {
        $to = $email['to'];
        $subject = $email['subject'];
        $message = $email['message'];
        sendEmail($to, $subject, $message);
    });

    // block current process by running $future->await();
}

echo "All emails sent.\n";

The above script is a very simple version of running things asynchronously. It will create a new fiber asynchronously using the given closure, returning a Future (object).

This is a much simpler version than rolling your own and does the heavy lifting for you, which is key for building an application as you don't need to worry about how the work is queued internally - you just know it happens asynchronously.

Queues and Workers

A solution to this problem also exists outside of PHP and prior to PHP 8.1 it could be considered the gold standard because it's language independent and highly scalable.

The use of queues such as Amazon SQS, RabbitMQ or Apache Kafka has been a widely accepted solution for some time.

Queues are pieces of infrastructure to run workers indepdenent to your application for the processing of any work asynchronously. This is not without risk or downside either, but tried and tested over time.

Let's get into an example:

Sender, in this example, is typically your existsing web application.

sender.php

<?php

require 'vendor/autoload.php';

use Aws\Sqs\SqsClient;

// Initialize the SQS client
$client = new SqsClient([
    'region' => 'us-east-1',
    'version' => 'latest',
    'credentials' => [
        'key' => 'YOUR_AWS_ACCESS_KEY',
        'secret' => 'YOUR_AWS_SECRET_ACCESS_KEY',
    ],
]);

// Define the message details
$message = [
    'to' => 'john@example.com',
    'subject' => 'Hello John',
    'message' => 'This is a test email for John.',
];

// Send the message to SQS
$result = $client->sendMessage([
    'QueueUrl' => 'YOUR_SQS_QUEUE_URL',
    'MessageBody' => json_encode($message),
]);

echo "Message sent to SQS with MessageId: " . $result['MessageId'] . "\n";

Workers are an additional deployment of running code to process jobs.

worker.php

<?php

require 'vendor/autoload.php';

use Aws\Sqs\SqsClient;

// Initialize the SQS client
$client = new SqsClient([
    'region' => 'us-east-1',
    'version' => 'latest',
    'credentials' => [
        'key' => 'YOUR_AWS_ACCESS_KEY',
        'secret' => 'YOUR_AWS_SECRET_ACCESS_KEY',
    ],
]);

// Receive and process messages from SQS
while (true) {
    $result = $client->receiveMessage([
        'QueueUrl' => 'YOUR_SQS_QUEUE_URL',
        'MaxNumberOfMessages' => 1,
        'WaitTimeSeconds' => 20,
    ]);

    if (!empty($result['Messages'])) {
        foreach ($result['Messages'] as $message) {
            $body = json_decode($message['Body'], true);

            // Process the message (send email in this case)
            sendEmail($body['to'], $body['subject'], $body['message']);

            // Delete the message from SQS
            $client->deleteMessage([
                'QueueUrl' => 'YOUR_SQS_QUEUE_URL',
                'ReceiptHandle' => $message['ReceiptHandle'],
            ]);
        }
    }
}

function sendEmail($to, $subject, $message)
{
    sleep(3); // Simulating sending email by sleeping for 3 seconds
    echo "Email sent to: $to\n";
}

This solution is comprised of two parts:

Sender (pushes a message to an SQS queue)
Worker (receives a message from a queue and sends an email)

It can be scaled through increasing the number of workers relative to the number of messages that get sent by any number of senders.

By using a queue, the worker is completely independent from the sender and can be written in any language as the communication between sender and worker is through JSON messages.

Which solution is best?

It's almost impossible to say out of all of the solutions we've explored above, which would be the best for your application because although they all aim at solving the problem of running asynchronous code with PHP the implementations are quite different and have different benefits and drawbacks.

To summarise each option in a few points:

exec()

Perhaps the simplest and most effective way to run PHP scripts async
Fraught with potential security implications particularly around user input
Nothing is shared can be both a blessing and a curse
May cause increase in existing server resources (CPU/Memory)

pcntl_fork()

Allows management of parent/child processes to customise behaviour
Can be abstracted away in a simpler API for your application
Cloning the current process may cause other downstream issues

AMPHP

Requires PHP 8.1 for the user of Fibers
Library has abstracted away the "hard parts" of running async code
Steep learning curve over other more traditional methods (understanding event loop and multi-tasking in PHP)

Queues and Workers

Language independent, flexible for any use case
Introduces a distributed system (can be a good or bad thing in the long run)
Many solutions around and different queue providers to make it easy

Conclusion

The main reason I wanted to dive a bit deeper into all the different possibilities of async code in PHP is to understand how (if at all) the introduction of Fibers in PHP 8.1 changes how we can write async programs in the future.

There are many solutions available without requiring PHP 8.1 that have been battle tested, but it's interesting to see the direction the PHP language is going in to compete with the likes of Golang and Elixir, both of which support async programming and have done for years.

Ultimately, I would probably still reach for a Queue/Worker approach given the scalability and cross-platform/cross-language support - however I think over time we might see libraries such as AMPHP become more feature rich and make this problem easier to solve without introducing new infrastructure.

To see the code samples used in this blog post, you can find them on GitHub.

Maintaining feature flags in a product engineering team

Jack Marchant — Fri, 01 Apr 2022 01:09:33 +0000

I have mixed feelings about feature flags. They are part of the product development workflow and you would be hard pressed to find a product engineering team that doesn’t use them. Gone are the days of either shipping and hoping the code will work first time or testing the life out of a feature so much that it delays the project.

The benefits of using feature flags certainly outweigh the bad, but it doesn’t stop teams from cursing them every time a major bug is reported or an incident occurs as a result of enabled (or disabled) feature flags.

In this post I will discuss the benefits and some drawbacks of using feature flags, to help you learn from some of the lessons I’ve personally learned, in the hopes that you can avoid the mistakes.

First, let’s understand what a feature flag does, and why it’s there:

A feature flag is (at its simplest - there are more advanced controls you can use) an on/off switch to release some new functionality to your product through the code base.

Teams can safely ship code knowing the feature can be enabled for small groups of users at a time, and released to more customers as confidence in a feature or behaviour grows.

Before feature flags, you only had one shot to ship code to production and make sure it works, which meant a longer build up to releasing code for the first time, or complicated infrastructure to support canary releases.

The main problem with feature flags is what happens when you have too many and they start conflicting with each other or you have so many different flows to test that the team spends much longer on a feature than they should.

This leads me to the first lesson:

Lesson #1

The number of feature flags you maintain will spiral out of control

Every time you create a feature flag, you’re introducing a different behaviour for your code that may only be initially released to parts of your user base, meaning you now support two different behaviours (feature flag on and feature flag off).

This is empowering for a growing product and engineering team. As the number of feature flags in use in production grows, so too does the frustration of testing all of those different cases and receiving bug reports where your first instinct is to check which feature flags are enabled or disabled.

Keeping track of feature flags means making them attributed to your team so there’s ownership of the feature flags and tracking the rollout status, including ensuring rollout continues to happen, or the flag is retired.

When feature flags have already spiralled and are out of control, the best thing to do is pause development of new features and clean up any flags that are rolled out or no longer required.

Getting feature flags back under control should be a priority, given the impact on development and testing time for related features.

This is a hard lesson to learn because there’s only one way out. Clean up the flags!

Lesson #2

Clean up feature flags regularly, make it part of the development cycle

After some time in production, feature flags can become stale and turn into technical debt, which must be paid back at some point, or risk cumulating over time and being at the point of no return. In some ways you will always have to live with feature flags, and they become part of the work you do.

The difference between a feature flag that is new and part of the feature actively being worked on, versus a flag that has lost purpose is enormous.

To avoid this dysfunctional reality, we must clean up feature flags after the feature has been rolled out or no longer needed. Sometimes this will be a minor piece of work that involves only one person making the code change and testing it and other times it can require re-testing an entire feature.

In my experience, how well you have built up your automated testing around the feature will impact whether it’s a minor change or a major one.

Recently in my team, we had built up a lot of feature flags for a variety of reasons, whether it was changing teams, forgotten features or slow rollouts and so we had to take a week or two to clean up around 10-12 feature flags in a short period of time.

We ended up doing the work for these in a short time frame, then merging and releasing the changes incrementally and over a longer period just in case anything went wrong.

This proved to be successful in the end and the team swarmed on the work to get it done.

We’re working on keeping track of feature flags more closely now to make it part of our development cycle to clean up feature flags rather than waiting for the eventual build up.

When we release a new feature, with a corresponding flag, we document it and ensure it keeps rolling out and create a ticket for the future to clean it up.

Progressing with the rollout usually means opening up the feature to more customers and this brings me to the final lesson.

Lesson #3

Always be rolling out

Feature flags should be temporary and are meant to increase the velocity of your team by allowing you to ship quickly and get real feedback from smaller groups of users in a safe way.

Flags, therefore, should be intended to be rolled out completely to your whole user base at some point. It’s normal to start with a small group and then build up incrementally to larger groups, but this should always be on a timeline.

Once you forget about it or move on to the next thing and leave the flag there, it will become stale and your team falls into the trap of having to maintain it and test both states of the flag should a change to that area of the code base be required.

There’s no one right time frame for a feature flag to exist, it will always depend on the feature and the group of customers using it to give you feedback directly or indirectly, through usage.

That’s why keeping track of the current state of feature flags in your control is important including managing the continual rollout to more users.

As you find bugs you can pause the rollout until the bugs are fixed, but if you haven’t hit any road blocks it’s critical to keep forging ahead so removing the feature flag becomes possible once it has been made available for all of your users.

Feature flags are both a blessing and a curse, it’s probably no secret to most engineering teams. What’s missing in my opinion is a framework to manage feature flags over time and throughout engineering organisations.

They help keep the product working and make it easy to rollback changes without code deployment, and give on-call engineers peace of mind when they can safely turn off a flag that has caused an incident.

If left unchecked, feature flags can slow engineering teams down to a crawl so create each feature flag with caution and a plan for its eventual removal.

Feature flags have given product teams the confidence to move fast, with a plan to rollback at the click of a button, but with great power and flexibility, comes a cost which should not be underestimated.

What I’ve learned doing technical interviews

Jack Marchant — Fri, 18 Mar 2022 20:44:08 +0000

When I first started interviewing candidates for engineering roles, I was very nervous. The process can be quite daunting as both an interviewer and interviewee. The goal for the interviewer is to assess the candidate for their technical capabilities and make a judgement on whether you think they should move to the next round (there’s always a next round). Making a judgement on someone after an hour, sometimes a bit longer, is hard and error prone.

There are a lot of ways to assess someone for an interview for an engineering role. Depending on the role itself there may be certain requirements such as a senior role needing more focus on system design and manager roles more focussed on the team dynamics rather than their ability to write code.

In all types of interviews there is a lot you can learn from doing interviews, whether you’re a candidate or interviewer but for this article I’m going to talk about some of the things I’ve learned while doing many technical interviews over the past few years.

Disclaimer: I am not pretending to be an expert on interviewing nor have the perfect process, but these practices are what I wish I knew before I started interviewing more regularly.

Having a clear, repeatable structure helps you and other interviewers ask the right questions

When I started interviewing there was some process about what questions to ask, how the interview was structured and how it changed depending on seniority, but it was often not relevant and I always found it odd to roll off 20-questions one after the other, especially when the questions probably weren’t relevant for the candidate.
I also found stating the structure of the interview at the start allows the candidate to understand what’s coming and what time we should finish so we know to progress if one section is taking too long.

While you do want to have some canned questions you ask in case you can’t think of any, I try to think more in terms of the topics or capabilities I want to assess rather than direct questions.
For example, rather than asking a specific question about React or VueJS, I would ask what technologies have you used to solve writing reusable components on the frontend? This broader question allows the candidate to answer in a way that’s relevant to them.
If we want to asses whether they have experience with observability, we can ask about how they would know if something fails or is working correctly in production?

Setting these expectations up front helps you (the interviewer), any other interviewers and the candidate ask any questions before progressing and know what to expect during the interview.

You don’t need to know everything to interview someone

I used to dread interviewing someone who clearly had more experience than I did. They would probably say something I didn’t understand or ask a clarifying question I couldn’t answer thus prompting the question - why am I the interviewer?
While it doesn’t happen as much anymore, it’s not because I learned everything, it’s because I stopped trying to know the answer to every question. It’s ok to say you haven’t used that technology before or are unsure how something works, even as an interviewer.

Instead, ask a follow-up question if the candidate knows about it and you will learn something new.
Better yet, this will demonstrate the candidate’s ability to teach something to a co-worker, which is a very valuable skill to have as an engineer.

Your experiences are more likely to be different to every engineer you interview, so it’s better to acknowledge this rather than try to combat it or be worried about being stumped on a question.

Win-win.

When to help the interviewee along by giving hints

Sometimes you’ll interview someone who has just started their career and hasn’t had much experience yet. They might need some extra pointers to get to where you’re trying to get them to go. This isn’t a sign of weakness in an interview, but an opportunity for you to show how they can learn more from the company they’re interviewing for.
They may also just have different ideas to you and the other interviewers, which is also perfectly fine, albeit encouraged.

System Design is often not as simple as choosing a tool that can do the job and being done with it. More often there are a set of trade-offs that have to be considered, so offering the candidate a chance to understand the intent behind the question could be one way of aligning on what you’re getting at without specifically telling them what you’re looking for.

Some people will just need more time to think through their answers, as these whiteboard interviews can often be a source of anxiety for many engineers who don’t do it on a regular basis. Remember the desired outcome is not to trip up the candidate into saying the wrong thing, but rather providing them a chance to show you what they know and how they arrived at that conclusion.

Asking the right questions helps form the basis for your judgement about hiring/not hiring

It’s hard to know what questions to ask if everyone’s experiences are different, they haven’t used the same technologies as you, so we can instead ask questions that start a discussion about a broader topic, so the candidate can tell you what they know about that topic rather than trying to ask pointed questions.

Typically, I won’t have a prepared list of questions to ask in the interview, but will instead know what topics to discussed based on previous experience or knowing what we’re trying to assess from the candidate. It’s less about specific technologies and more about their problem solving ability, critical thinking and ability to make trade-offs and explain them clearly.

Sometimes engineers will have to make decisions that aren’t ideal, but are based on a set of tradeoffs (maybe they don’t agree with those either), but are part of business goals or customer requests. Understanding that this is where the tradeoffs come from and have a variety of reasons for being considered is more valuable to me than any specific technology we could learn more about.

Asking a question you don’t know the answer to but think the candidate will is a good way to build trust with the candidate that you’re not trying to trip them up on a question and are genuinely wanting to learn from them - remember they are interviewing you and your company too!

Practice is the only way to get better

Fortunately (or not) the only way to get better at interviewing is to do more of it. Avoiding awkward silences, especially when an audio delay happens over a video call, comes with practice and doesn’t come naturally to everyone, myself included!

Eventually, I got better at either stalling by talking about something else until I could think of the next think to ask about or knowing what question I want to ask next in advance.
Being curious about the candidate’s experience helps broaden your own perspectives and helps them relax by talking about things they understand well.

You want the candidate to do well, you’re not trying to trip them up on a trick question

Above almost everything else, you should want the candidate to do well. It’s your responsibility as the interviewer to set them up for success, focusing on their strengths and letting them show you what they know rather than asking a specific set list of questions about topics they might not be familiar with.

This will result in a better interview experience for everyone involved, and even if they don’t progress to the next round or eventually get the job, they could still come back again or leave with a good interview experience to learn from.

Evaluating a candidate for an engineering role is a bit different to other roles and has its own intricacies due to the technical nature of what you’re trying to asses, but more often than not I have found technical skills can be learned but attitude and how you treat other people is much harder to change. So, even though we’re assessing technical ability, it’s not an excuse to ignore basic decency of giving respect to everyone on the call. Thankfully, there have not been too many occasions where this is has impacted an interview.

Interviewing is a tricky subject and most of my interviews recently have been done over hangouts, which has its own challenges. Having a good experience in an interview should be a given, even if it doesn’t work out in the end.

It’s just as much an interview of the company you’re representing as it is of the candidate. It’s the first chance someone has to experience your company and leaving a bad impression will generally last forever. I’ve learned a lot over the interviews I’ve done so far, and hopefully I’ll learn more in the interviews to come!

Using a Dependency Injection (DI) Container to decouple your code

Jack Marchant — Wed, 03 Jun 2020 07:30:27 +0000

Dependency Injection is the method of passing objects to another (usually during instantiation) to invert the dependency created when you use an object. A Container is often used as a collection of the objects used in your system, to achieve separation between usage and instantiation.

What is Dependency Injection

Take for example, the repository pattern whereby you use a separate class to handle database access so that you can separate that functionality from your application's business logic. For example, you might instantiate a new Repository in a Service:

class BookService
{
    public function getSomeBooks()
    {
        $repository = new BookRepository();
        return $repository->getAll();
    }
}

It doesn't really matter at this point what the getAll function does in BookRepository, but the main point is that by instantiating dependencies in the same place as where they are used creates an implicit dependency on the BookRepository leading to tightly coupled and hard to change code. In the example above, we're no longer able to test it without having a database connected, nor can we switch it out at runtime or setup, meaning less overall flexibility.

Instead, we could declare a private member variable on the BookService class, and assign it during instantiation of the BookService class itself, and ensure whatever is passed in, implements a specific interface, so that functions you call on the repository are guarunteed to have been implemented.

class BookService
{
    private $repository;

    public function __construct(RepositoryInterface $repository)
    {
        $this->repository = $repository;
    }

    public function getSomeBooks()
    {
        return $this->repository->getAll();
    }
}

Now, BookService has no idea what specific repository will be passed in, and it doesn't need to care because it knows it implements the RepositoryInterface. This is an example of Dependency Inversion Principle and is critical to understanding why Dependency Injection (DI) in PHP (and most other languages) is an important concept and to understand why a DI container exists.

How a DI Container makes this a lot easier

We've seen how dependency injection can make testing easier, along with decoupling your code so that it can easily change over time, you might also consider what your codebase might look like if you had more complex objects than the BookService and you had to use it all over your codebase.
Everywhere you need to get some books, you need to instantiate both the BookService and it's dependency BookRepository, so that it can be passed into the constructor.

$bookRepository = new BookRepository();
$bookService = new BookService($bookRepository);

This is a great first step forward, but there's more that can be done to control how a BookService is instantiated and with what repository, since now it can be switched out with relative ease.
This is where a container comes in. If you've ever used Slim Framework, you might have noticed you can set up a DI container for your app.

$container = new \Slim\Container;
$app = new \Slim\App($container);

// Add a service to Slim container:
$container = $app->getContainer();
$container['BookService'] = function ($container) {
    $bookRepository = new BookRepository();
    return new BookService($bookRepository);
};

Wherever in your code you need a new BookService, you can simply use the container to build a new object for you with the repository.

// Use your service
$bookService = $container->get('BookService');
$bookService->getSomeBooks();

This makes any code that uses the BookService independent of the service as well, so by using the container, we're inverting the dependency on the BookService.

As you might have realised, the function we defined as the value for the BookService key in the container, will be passed the container as an argument, meaning you can pull off any other dependencies that exist in the container already, such as the repository itself:

$container['BookService'] = function ($container) {
    return new BookService($container->get('BookRepository'));
};

There are endless possibilities for how you can use a Dependency Injection (DI) Container to your advantage, to decouple related objects and remove implicit dependencies so that your software can grow over time with boundaries in place.

Originally published on jackmarchant.com

3 simple tips to get better at working from home

Jack Marchant — Thu, 16 Apr 2020 22:56:41 +0000

Working from home has been thrust upon those lucky enough to still have a job. Many aren’t sure how to cope, some are trying to find ways to help them through the day. Make no mistake, this is not a normal remote working environment we find ourselves in, but nonetheless we should find ways to embrace it.

These three tips will help you get the most out of yourself in productivity, while maintaining a healthy work-life balance (in whatever way possible in our current circumstances).

I have worked from home regularly for a number of years, however it’s only since it became a full-time situation that I put these tips into practice and it has helped maintain my sanity, if nothing else.

Set a start and end time

There’s no better feeling than clocking off at the end of a work day and while we probably don't have much more than a bit of exercise to look forward to, having a set time that you finish up work and check out for the day means you brain can adjust and get into the habit of switching off.

Especially now as we’re confined to our homes, it’s important to distinguish a time when you’re not “at work”.

Equally, starting work at a predictable time gives your brain a chance to clock on to work mode and focus. I have fallen into the trap of not having set a time to start, only to watch the clock fly by without getting anything meaningful done.

It’s easy to say, much harder to do and continue doing, but just like anything else, practice makes perfect. I set specific times that I must start and finish by, and I try to be realistic with those times.

Have a break (particularly lunch)

Just as important as setting times to start and end your day, it is worth your time to take a short break from your work to eat and drink.
Sometimes when you’re in the zone you forget about taking a break and you realise 3 or 4 hours have gone by without you leaving your seat.

A lunch break is like rebooting yourself (much like your computer). Your mind has probably been running around all morning and has a thousand things to think through. Taking time to step away from your workspace gives you the control to regain focus on what can be done in the afternoon.

I find that taking short, frequent breaks is beneficial to my own productivity and helps me to finish tasks more easily. It’s difficult for me to switch between different things, but when I finish something, I walk away and come back ready to go on the next thing.

Dedicate one room to being your workspace

A workspace has to feel comfortable and suited to your individual needs. If you can, it’s ideal to have a single room you can make your own so that when you’re in this room, it’s work time and perhaps more importantly, when you’re not in this room, work time is over.

Being able to leave a work environment makes it easy to separate work from the rest of the day. It also comes in handy when you’re on a video call and need to shut off the rest of your home from whatever noises may occur. This "home-office" also gives the people you live with the indication that you're working, meaning you may not want to be interrupted.

These little tips can make a big difference in your own wellbeing and productivity while working from home. I have become stricter with these boundaries as I’ve made the transition to doing it full-time and I’m so glad that I did because it has allowed me to get the best out of myself at work, while still being myself at home.

There are loads of great resources out there to help you with working from home, here are a few that I have found useful:

Originally published at jackmarchant.com

Making Software - a three step process

Jack Marchant — Tue, 14 Apr 2020 00:01:13 +0000

One of the most useful tips that has guided much of my decision over the years has been this simple principle: three steps, executed in sequential order;

Make it work
Make it right
Make it fast

These steps outline the process through which software should be made. You should refer back to these steps and discover for yourself which step you are currently in while creating software. This will also help you identify whether you need to or indeed can move to the next step.
An important distinction to make up-front is that you don't necessarily need to complete all steps in order to ship software to users. Let me explain.

Make it work

The first step should be the most obvious, but the key to (and the most difficult) is acknowledging you are at step 1. Making software work is about glueing all of the pieces together until the thing you're trying to build actually works.
Imagine you're building a skateboard and you take a plank of wood and screw 4 wheels to the bottom. There should be criteria to allow you to recognise when it's finished and can move to the next step, for example:

Can you stand on it?
Does it move forward when you push off? If the answer to both of these questions is yes, then congratulations you've got a working skateboard.

Keen readers will poke holes in the analogy - the skateboard will break easily because the wheels aren't correctly fastened to the plank and will buckle after the first ride. While this may be true, the first step is making it work and we may not be expecting to produce thousands of these skateboards and allow customers to purchase them at this stage.

Bringing this back to software, if you catch yourself on step 1 and you're already thinking about how to optimise for performance or you've got the best abstraction idea that is extensible to the nth degree, then the battle may very well be lost because if you can't make it work, nothing else matters.
This step is more about what you don't do straight away, as opposed to what you do.

Make it right

Continuing the skateboard analogy, this next step provides you with the time built into the process to take a step (pardon the pun) back and look at the big picture. Such as, the wheels need to be fitted correctly with the proper materials and safety considerations to withstand the rough and tumble expected in the riding of a skateboard.
This is no different to the wear and tear of software - without the proper guards in place such as tests, abstractions and extensibility in place the software will likely buckle under the pressure of real users.

This step is the right time to take the working thing and build it properly, regardless of whether you start it from scratch. The first step is more discovery than anything else and provides you with the confidence and knowledge of how to build the thing, so that in this step you can build the thing right.
Making software right ensures you have the correct checks and balances in place, such as applying principles to common problems and giving it the best chance of long-term sustainability.

At each step, a decision needs to be made about whether to progress to the next - for example there might be times in building software when making something work is more important than making it right. The key difference between this and pure negligence is the fact that this trade-off is a conscious choice, which makes it hard to see in hindsight unless well documented. It is often seen as tech debt, accrued for a purpose, but don't be fooled into thinking it's at all similar to financial debt (a common misconception, that may be for another time).

While I wouldn't recommend stopping before making it right, there are times where making something is better than not making anything at all. The hardest part is accepting the snowball effect this will have later down the track and whether you are actually prepared for the true cost.

Making it right, while technically a choice (hence it being a different step) is probably where most early software needs to land to be effective. Going further than this may become detrimental unless you have a reason to make it fast.

Make it fast

The final step is where all guns are blazing, software is often in production and you need the last piece of the puzzle to ensure the software works as intended (i.e. fast enough for the user). This step is all about optimising what you have done in the previous two. To keep the skateboard analogy going, at this step we would want to focus on the speed of the rider ensuring they maximise speed.

In software on the other hand, this may only happen once it's actually in the hands of users, so you know where the bottlenecks are. Sure, you could throw more hardware at it temporarily but eventually you have to make the software fast enough to scale correctly and appropriately.

In each step there are trade-offs but in this step the smallest decision can have the biggest impact, so you should always make data-driven decisions based on real usage rather than making educated guesses and hoping for the best. In time, experience will often tell you where the bottlenecks are, but this isn't always the case and depends on the system. Making it fast requires skill and practice and sometimes you don't even need this step until the software becomes popular enough that you have a reason to focus on speed. This is why it's best to analyse the data before embarking on this step.

One step at a time

In each step there is particular nuance to doing it right and doing it well. It's not like a recipe where the instructions tell you exactly what's required. All you can do is take it one step at a time and figure out where you can draw the lines around it. At times it may even be harder than building the software on it's own, but being able to do this consistently will help you in the long-run.

I have found great focus from following these three steps when building any software professionally.

Help me, help you - Code Review

Jack Marchant — Thu, 24 Oct 2019 07:08:36 +0000

Code Reviews are one of the easiest ways to help your team-mates. There are a number of benefits for both the reviewer and pull request author:

Product knowledge sharing through anecdotes or code samples
Sharing techniques for writing maintainable code
Differing perspectives collaborating on a single solution

I find it helpful to consider the intent of each party participating in a code review. Let’s think about what’s important to both the reviewer and author and how each can contribute to the success of the other.

As an Author, I want to submit my code for peer review, so that I can gather feedback and iterate on my solution

When you submit a new pull request, depending on your previous experiences, you might feel nervous about the response from your peers. It’s normal to be attached to your own code if you’ve spent a long time writing it, however this attachment can be a detriment to your willingness to accept feedback. While it’s not easy not to take feedback personally, it’s better to assume that the person providing feedback has the same good intentions you had when writing the code.

The time it takes from when you open a pull request, to when it is merged varies widely based on it’s content, risk and testability (among other things). When specific people or teams are the best people to review your code, it helps to reach out to those people and let them know you’d like them to review it. It is your responsibility to follow up with them and get your code reviewed.

It can make a huge difference if the pull request has a clear (and sometimes, thorough) description, breaking down context for the change, what the effect of the change is, and how it will be tested. In an ideal world, we’d be submitting automated tests along with our code, but this is not always possible. We should strive as a team to prove that our code works as expected, and including tests (manual or automated). This will aid the reviewer in understanding the reason for your change, so that they can in turn help you, by giving feedback.

Pull requests that are matched to tickets are often seen as a 1-1 relationship. I don’t subscribe to this idea, and instead I’d prefer to see one large code change broken down into multiple chunks (1 ticket, many pull requests) that can be easily pieced together (if necessary) and follow a progression to a releasable version of your code. Too often we try to build the whole solution and release it to users all at once, often forgetting the risk involved with doing so. In many cases, hiding new code paths with feature flags can make your code releasable, without necessarily running in production straight away. Breaking code down into manageable chunks, makes the review process easier for both you and the reviewer.

TLDR;

Make the pull request clear and concise.
Follow-up reviewers to look at your pull request
Provide comprehensive reasoning, where possible for the change
Break down large code changes into smaller chunks

As a Reviewer, I want to review code to ensure quality and accuracy and provide relevant feedback to help the Author move forward

Reviewing code can be difficult at times - when there are a large number of lines or not enough context in the description, it can make reviewing painful. As the reviewer, you want to ensure the correctness of the code in terms of the objective, rather than aesthetic - however the quality of the code organisation and formatting can have an impact on the maintainability of the code in the long term. In a large team with many contributors, or an open source codebase, this is particularly important. Thankfully, there are a number of automated formatting and code linters available in many different languages to help code authors meet this standard more easily.

When reviewing code, it’s easy to look at the code at face value and forget about the intent, requirements and restrictions under which the author wrote said code. Without knowing all of this, the reviewer should make an attempt not to assume the worst of the author and instead work with them to provide feedback in a polite and professional manner.

As a reviewer, you can only provide feedback from the information you’ve been given, whether that’s code, a description of the problem or a diagram of the solution. However, sometimes that is not enough, and you need to also seek further information from the author in order to provide great feedback. Your approach may vary depending on the size of the pull request, as with anything else in software development - it’s not a one size fits all situation, however it’s up to you as a reviewer to provide feedback that helps move the process forward.

TLDR;

Automate as much of the formatting concerns as possible
Discover the intent behind the code, not just the code itself
Seek further information, if something is not clear

Before your next pull request

You can only get out of the pull request process as you put in, both as a reviewer and an author. On both sides of the equation, there are many things to consider before code gets shipped to production, many of which I haven’t discussed today. Before you review or open your next pull request, understand how you can make the process easier for you and your colleagues - help them, help you.

Originally published on jackmarchant.com

A practical guide to Test Driven Development

Jack Marchant — Thu, 12 Sep 2019 00:08:52 +0000

It’s been a while since I last wrote about why testing is important, but in this post I thought I would expand on that and talk about why not only unit testing is important, but how a full spectrum of automated tests can improve productivity, increase confidence pushing code and help keep users happy.

Why do we need to test code?

Code gets tested every time users interact with your software, whether it’s through an application or part of an API. The unfortunate reality is that by the time your code is in the hands of users, it’s too late to find out it doesn’t work.

To reduce the chances of this occurring, we test code during development, after development (sometimes called Quality Assurance testing), right before releasing the code to users and even right after releasing.

At each step, it’s possible we could find a defect in the code and need to revert or write a fix to remedy the situation. The later the defect is found, the larger the impact and slower the turnaround to getting it fixed.

It is for these reasons that we test code at each step, building up confidence that the code does what we expect so that it may progress to the next stage in the development and release process.

Definitions

The following are definitions of terms I use throughout this post, and serve as a description of how I think about each type of test (these aren't necessarily textbook definitions).

Unit Test: A test where the subject is an isolated block of code, typically a single function with no dependencies.

Integration Test: A test where the subject could be a function with dependencies, or multiple functions/classes tested simultaneously.

User Acceptance Test: The closest test to how a user will interact with your software, sometimes referred to as a Functional Test.

Tests are a crucial - regardless of when they happen

It is always in your best interests as a developer writing code to find bugs as early as possible. The ideal scenario being that you find it as you’re working on the code itself, by making a change and then running automated unit tests. This way, you can identify the problem, fix it by writing a test case for that scenario and moving on.
Not all bugs are created equally however and by the nature of software development, some code is harder to test than others. This is why we introduce other forms of testing later in the development cycle, such as integration testing and user acceptance testing.

These three forms of testing: Unit, Integration and User Acceptance, build on top of each other to create a test pyramid. The general idea being that unit tests should be easy to create and run as they are without external dependencies. Integration tests allow you to see how different modules, when hooked up together, respond to certain inputs. Finally, User Acceptance tests may place an entire vertical slice (incorporating many parts of your software which may be slow or brittle) under test. As you go from most (unit), many (integration) to some (user acceptance), confidence in the overall system to be working correctly should increase.

Having tests doesn’t make bugs disappear completely, but it does reduce the frequency of them, along with ensuring that changes you make don’t have unintended side effects.

Now that we’ve discussed some of the terminology and theory behind testing practices in software development, it’s much easier said than done. So, let’s talk about some ways you can incorporate testing into your development workflow.

In a large codebase, it’s worth having a few strategies for testing depending on the code needing to be tested, for example:

New code integrating with existing code:

In this scenario it makes sense to unit test any new code you write, as much as possible. The point at which you integrate the new code into an existing code path, you may not be able to easily test, but because you have confidence from the unit tests, you can try either an integration or user acceptance test. The former will likely be running the existing code path and making sure the new code is being run, while simultaneously ensuring the existing code runs successfully. The latter, may require manual or automated testing of the entire feature, during which time your code is run. This has a slower feedback cycle, but equally an important step nonetheless.

Fixing a bug in existing code:

When you find a bug in your code, whether it’s during development or reported by a user, the best way to fix it is to write a test (any type will do) and then fix the code, ensuring that the test passes.

This will have a short term and long term effect:

It will ensure you have actually fixed the bug.
And, allows the test to be run again in the future, making sure further changes haven’t caused a regression.

Approaching the Test Pyramid from scratch:

Without any tests or very little, often the code is hard to test so it can be worthwhile starting from the top of the pyramid and working downwards. User Acceptance testing can be a good way to get started because you can mimic how a user interacts with the software. Then, as more tests are added, confidence that overall features are working might enable engineers to start building integration and unit tests with a bit of refactoring along the way.

Having tests doesn’t make bugs disappear completely, but it does reduce the frequency of them, along with ensuring that changes you make don’t have unintended side effects.

The effects of Testing over time

Improving the maintainability of a codebase through increasing test coverage over time has a dramatic affect on teams, individuals and businesses. There are a number of fallacies surrounding testing that exist in software development teams in regards to testing that hinder their collective ability to be productive.

A system that’s hard to test becomes a black box for developers, because it’s impossible to say with any certainty how something works. That being said, it is possible to open up the box and take parts out to figure out how they work. The best way I’ve found to learn a system is by introducing new tests.

There’s a common belief that Test Driven Development can only be practiced successfully through writing tests as if they are requirements, then writing code to satisfy the requirements. I would suggest that in reality, this is not how much of the software in the world is created - because it’s hard to do.

Instead, Test Driven Development to me, is the practice of incorporating any kind of testing into your development cycle, meaning you’re not always writing tests first - it could be after you’re done or midway through - the important part is to use automated and manual testing together to drive a faster feedback loop between writing code and knowing whether or not it works.

In practice there are trade offs, just as in any other engineering decision, which need to be considered when adding tests to your development workflow. Let's stop debating about whether TDD means red-green-refactor, all it does is discourage people from actually writing tests, for fear they're not doing it right.

There are always going to be tests, but which ones, and how will they be run? In answering these questions and developing with tests, you’ll find it increases your own productivity writing code and in the end it will improve the reliability of your software for your users.

Originally published on jackmarchant.com

The Facade Pattern

Jack Marchant — Fri, 05 Jul 2019 13:03:29 +0000

Design Patterns allow you to create abstractions that decouple sections of a codebase with the purpose of making a change to the code later a much easier process.
They are a set of blueprints for solving specific sets of problems, and hopefully don’t over-complicate.

There’s nothing worse than seeing an abstraction in a codebase that actually makes it harder to understand than without the abstraction.
Of course, it’s a trade off but often times an easy way to see when you should create an abstraction is when you start to see a pattern or repetition in the behaviours in your code - not necessarily just duplicated code.

I’ve been digging in to some design patterns lately, and one that I had to research again was the Facade Pattern.
If you don’t know what it is, you have probably already seen or used it many times before, but after reading this article, hopefully you’ll be able to identify the Facade Pattern in your own code.

Facade literally means a deceptive outward appearance, and that’s potentially the wrong angle for thinking about solving a problem with software.
When you create a new function, it’s unlikely you’ll name it anything other than exactly what the function does. Naming things is hard in itself but that should at least be the aim.

The Facade Pattern used in your code should be a simple interface for doing something more complicated. It should group related things together to make it easier to use.

If you’ve ever integrated a third party library into your application you may have subconsciously used this pattern without realising. Say you’re building an app where users can purchase things, you might want to create a new customer account, charge the customers credit card and send an invoice email.

Rather than having to think about each of these requirements whenever a customer makes a purchase, we can wrap this functionality in a specific class created for the purpose of making a purchase, then the construction of the internal objects in the application are centralised and consistent regardless of the type of purchase.
This type of abstraction hides away some of the complicated parts of the process behind a friendly interface that can be used throughout the application with relative ease.
It is this interface that can be described as a Facade.

class Customer {
  public function __construct(array $details) {}
}
class PaymentService implements PaymentGateway {
  public function createCustomer(Customer $customer) {}
  public function createCharge(Customer $customer) {}
}
class Mailer {
  public static function send(string $to) {}
}


class PaymentFacade 
{
   public static function purchase(array $customerDetails, Item $item)
   {
      $customer = new Customer($customerDetails);
      $service = new PaymentService;
      $result = $service->createCustomer($customer)->createCharge($customer, $item->price);

      if ($result) {
        Mailer::send($customer->email);
      }

      return $customer;
   }
}

For me, the Facade Pattern was a bit confusing so I took some time to figure out exactly why and when it was used. To really assist in learning about design patterns in software, I would recommend reading popular projects source code so you can see how certain patterns are used - then you’ll be able to identify it in your own code.

Originally published on jackmarchant.com