DEV Community: James Ford

On the establishment of Engineering Guidelines

James Ford — Wed, 28 Sep 2022 06:42:15 +0000

A shared vision

One of the most challenging aspects of leading software engineers is achieving a consensus. This is true everywhere I've had experience, and with any scale and any cohort and on any subject.

A consensus is important for practical things such as team harmony, code consistency and quality of output but it is especially important for ephemeral topics such as "strategy" or "best practices."

The trouble is that much of the "wisdom" and "knowledge" held, internalised and enshrined by engineers is accumulated not by a teachable linear path but by an inefficient process of osmosis from the industry zeitgeist making it quite impossible to replicate. Being impossible to replicate makes it difficult to teach, as it robs you of some of the resources you'd use to convert and convince others.

This experience you have - the lessons learnt or the things that have read which 'resonated' with you during your time as software engineer - are the treasures amidst the strata of your career, and the things that further influence your thought processes and decision making. They are the things which make you a leader, instead of just a good coder, and it's those things that we want to extract, distil, critically evaluate and then disseminate.

There's a problem with knowledge transfer that can be alleviated by better identifying what to share, but transferring the wisdom - the opinions and interpretations of an individual - is another challenge of its own.

Codified Wisdom

Enter then, our Engineering Guidelines. These are the output of a concerted effort to codify and document the wisdom of individuals, identify common themes, name them and prioritise them in accordance with our business strategy.

The goal of our Engineering Guidelines is to persist and achieve consistency over a longer time period, and scale better and reach further than any individual could on their own.

The source of truth for our guidelines is their written form - a collection of documents in our Wiki - but we can remix and reformat the content as appropriate for different scenarios, whether that be a workshop, presentation or release checklist.

The most important part of these guidelines is that they're documented, explained and evidenced. This means that they can be tracked, challenged, evolved and extended as requirements, technologies and trends also evolve over the course of time.

Better Code Reviews.

James Ford — Tue, 03 Sep 2019 06:40:11 +0000

Reducing fear and friction in the review process.

Code reviews are an essential part of developing software. And potentially one of the most challenging parts, because it involves exercising skills that aren’t necessary to write code, like diplomacy, tact and judgement.

A problem that can be solved with code might only have one correct solution, but that solution can be written in many different ways. And many of those different ways satisfy other unwritten criteria, such as execution time, capacity for reuse, simplicity, comprehensibility, portability and consistency.

The Test-Driven-Development (TDD) process makes checking technical solutions to problems simpler because it puts an emphasis on identifying problems to be solved and the ways of verifying the technical accuracy of a solution in advance of actually writing any code. But when we perform a code review, we’re looking for more than just the solution to a problem, because that verification is a boolean — the tests either pass, or they fail.

Code reviews are less about the solution and more about the shades of grey, and applying the unwritten rules of software development.

Unwritten rules

Having unwritten rules of software development is a bad thing.

Technical solutions are nice because they’re exact, precise and all aspects of them can be quantified. People solutions are complicated because they’re everything else.

Code reviews are the point at which another person comes and looks at the code you’ve written, and picks out its flaws. Maybe the code you’ve written is too complex, maybe it’s too simple. Maybe your variable names aren’t consistent with the rest of the codebase, or it’s not Object-Oriented enough or it’s not Functional enough. Maybe, maybe, maybe.

That’s the trouble with code reviews — they can be subjective, inconsistent in their scrutiny or based on unpredictable criteria.

Don’t stop reviewing!

We shouldn’t do away with code reviews. Code reviews are an excellent tool for maintaining consistency and a shared understanding in a team. Code reviews are an essential part of software development, and they should be done in all directions — juniors reviewing seniors and vice versa.

But code reviews can easily feel like personal attacks, which is why we have things like the Prime Directive:

Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand. — Norm Kerth

We can’t control humans. We can encourage better behaviours and be more mindful about the feedback that’s delivered in a review process, but it’s always going to be subjective, for the reviewer and reviewee.

Part of the answer to making code reviews better might well lie in removing the unknown criteria from the process — either by restricting review comments to only criteria specified in the original task, or by broadening scope of the task definition to include all possible ways in which a solution could be evaluated.

Restricting feedback

Restricting feedback during a review isn’t an approach that’s likely to result in long term happiness and team contentment. Reviews are an opportunity for engineers to express their thoughts, and censoring feedback in one place is only likely to make it emerge again in another channel. Isolating discussions to specific topics tends to lead to an endless series of disconnected conversations that distract from the original problem that spawned them.

Broader acceptance criteria

Broader criteria can reduce the surprise elements of a code review, but it can quickly become overly prescriptive about the way a problem is solved rather than defining what an acceptable solution looks like. Not only that, broadening the criteria for a task to include all possible measures of acceptable code runs the risk of creating impossible tasks for engineers to take on — solving the problem becomes too daunting, the feasibility of achieving certain criteria might be unknown at the beginning or impossible to satisfy, or the proposed solution is over-engineered and needlessly complex when a simpler solution would suffice.

The best solution to the challenge of making code reviews better likely has nothing to do with what policy or process actually gets put into place to facilitate them. The technical solution is binary — it works, or it doesn’t — but the opinions and experiences of engineers vary, leading to unpredictable results and quality standards in reviews.

A code review has to be organic, as it’s an opportunity for all members of the development team to have oversight and give input on each others’ code prior to it being finalised. It’s an opportunity for everyone to learn and for the team to make collective decisions about the future direction of other tasks.

The Problem with Code Reviews

The problem with having an organic approach to code reviews is that it seems to encourage purely functional commentary, which focuses only on the things that need to change, and not the elements which are good.

If a code review is only about highlighting the negatives, the overall result is negative. It should have some positives as well. Focussing exclusively on the bad elements, and where changes are perceived to be needed only serves to highlight and exacerbate any underlying divisions in the team. And that’s bad. However well-meaning a review might be, it can be received as a personal attack. We’re human and we respond emotionally to attacks, and negative reviews given improperly sow dissent and can reinforce any underlying differences in age, experience, gender, seniority, organisational structure, culture or personality.

I’m fortunate in that I’m a middle aged white male working in the tech industry, I have years of development experience, I’m the Technical Lead of my team and I’m familiar with my code base. I shouldn’t have a reason to feel attacked when someone reviews my code, and I have no underlying feelings of imposter syndrome or voices that tell me this industry isn’t for me. But not everyone has these luxuries of their background, and even I feel an emotional when my code is reviewed negatively.

Be kind.

The bottom line to this is, be kind. Be respectful, be diplomatic and give feedback about the positives as well as the negatives — it doesn’t all have to be an actionable list of tasks to complete, and treat it as a review and not a nitpicking exercise. As a team you’re working to create a shared understanding and a unified codebase. Treat your code reviews accordingly.

Code Review Checklist

I don’t believe that better code reviews can be mandated by any change in processes or ways of working. Code Reviews are a human-driven part of development and as such rely on the behaviours of humans. The best way to encourage behaviours is to lead with them, and to help me lead with them I follow an (informal) checklist which I’ve put together for myself, which looks as follows:

Has this person contributed code before?
Are they contributing in the format expected by the team?
Are the automated tests passing?
Have new automated tests been added?
What’s bad about this code?
What’s good about this code?
What’s non-critical but worth considering as an extension?
Is this code production ready?
What else would I say if we were face-to-face?

And that’s it. No magic formulas I’m afraid, just be a better human.

Illustrations courtesy of unDraw

Checking Availability in Office 365

James Ford — Wed, 13 Feb 2019 12:27:27 +0000

How to check for meeting room availability using the (beta) Office 365 Graph API.

Strange as it may seem this isn’t something that’s obvious to figure out, so allow me to save you half an hour of searching and just give you the answer.

Photo by rawpixel on Unsplash

Background

At the office we have a range of virtual and physical meeting rooms, but as good as off-the-shelf tools are, it’s still a laborious task to find an available room at a moment’s notice. It would be nice to tap into some APIs and be able to build our own custom tooling to make the availability more… visible.

The Answer; aka. How to do it:

First, get an API Access Token.
Make a Graph API request with a JSON payload that outlines the target email addresses and the time period to check.
Parse the response into usable data.

Getting an Access Token

Before starting you need an Access Token, which you can either get by following the steps in the official documentation, or to make things simpler you can try out requests using the Graph Explorer tool:

Graph Explorer - Microsoft Graph

Making a Graph API request

Make a POST request to the /me/calendar/getschedule Graph API endpoint: https://graph.microsoft.com/beta/me/calendar/getschedule, — which is currently only available in beta — using a JSON request body that looks something like this:

This request body includes the accounts for availability to be checked; Schedules, the start and end of the time period to check and the level of detail to look at; the availaibilityViewInterval — all of which affects the level of detail that comes back in the response.

By way of example, we can make this request using CURL, like this:

Or to make this same request with Javascript, like this:

Or just use the Graph Explorer:

Using the Graph Explorer UI

Parse the response

The response that comes back from the API will look something like this:

This response contains a object for each Schedule requested, a representation of the schedule availability, and the available details of each item in the schedule.

I find the availabilityView portion of the API response interesting — it’s a string that’s generated dynamically and represents the high-level availability for segments of time, beginning at the start time and split based on the initial availabilityViewInterval property of the POST request. The values in this string correspond with the following availability:

0 = free
1 = tentative
2 = busy
3 = out of office
4 = working elsewhere

This response can save you recursively parsing the scheduleItems, and you only have to split the string on each character to calculate availability.

Alternatives / the Explanation:

Checking calendars — whether it be for Meeting Rooms or for attendees — seems like it should be trivial task but it’s historically been quite complicated, because the APIs that expose the data needed to power this functionality usually requires read-level permissions from the calendar owner.

When I first attempted to build this functionality, I assumed that I could simply check the public calendar of the 20-or-so meeting rooms and collate the data myself, but I quickly discovered that the API doesn’t work that way, and each of the meeting rooms would require me to have been granted explicit permissions to their calendar, which wasn’t going to happen.

Fortunately, the getSchedule endpoint of the API exists under/me/calendar , so it just works, without the need for additional permissions. Counter-intuitive? Maybe. But it works, so we’ll overlook that.

Official Documentation

Here’s a link to the official documentation for this API endpoint:

Get free/busy schedule of users and resources (preview) - Microsoft Graph

That’s all for now — so let’s go check schedules!

Inspired by Open Source.

James Ford — Tue, 05 Feb 2019 11:10:18 +0000

Inner-sourcing at Compare the Market.

The Open Source community is an inspiration for us at ComparetheMarket — how it operates, the quality of their output and the ways they manage and engage with the community — all of it raises our expectations and standards, and I love that. 😍

Introduction

Inner Sourcing?

We’re unlikely to ever really be able to take our internal code and make it open source, but inspired by the open-source community and in particular by Paypal’s inner-sourcing initiative we’ve created our own ‘inner-sourcing’ initiative, to take our internal collaboration to “the next level.”

InnerSource Commons

Next Level, here we come…

Embracing GitHub

As a business we’ve traditionally used JIRA to track projects, but the GitHub UI feels more suited to open source style (collaborative) projects. We made the decision early on to embrace GitHub, and as much as possible stick to using only a single platform. We’ve incrementally adopted more and more features available through GitHub, and although other platforms are available, we always tried to work with the platforms we had available than introduce anything new.

Without further ado, here’s a few of things that our team has adopted, inspired by open source…

Documentation

Yeah, everyone has documentation. But what format, what kind, what quality and for which audience is the documentation produced? Projects like React set the gold standard here, with a broad coverage of different kinds of docs tailored to different audiences: starting with introductory concepts, tutorials, and moving through to highly detailed technical specifications.

Our existing Storybook — based component documentation

We already had Storybook for documenting our project, but now we’ve introduced our own hand-authored web-based documentation pages built with the aid of Docusaurus.

Docusaurus · Easy to Maintain Open Source Documentation Websites

Our Docusaurus — powered documentation pages.

Of course, the technology that drives a system isn’t important, it’s the content and how it’s written that matters most. We channel all of our contributors to a single website for our documentation, and for everything we produce we try to include:

High-level conceptual overviews,
Contributor guidelines,
Manually-authored documentation,
Auto-generated API documentation,
‘Quick start’ instructions,
Detailed step-by-step tutorials and a few bits more.

What’s really neat is Docusaurus’s own integration for editing pages via GitHub Pull Requests, making it really easy for readers to make their own contributions to the docs and expand or correct things as they work.

Semantic Versioning

Adopting semantic versioning was almost a no-brainer. Almost every open source library uses it, so it’s already an accepted part of the developer consciousness, and it’s a useful framework for the team because it really makes us reflect on the importance and severity of changes that we make when we release new versions of our code.

Semantic Versioning 2.0.0

It took us a little while to properly understand how to apply Semantic Versioning to our library and we made a few mistakes at the start, but we wouldn’t be without it now. We took a lot of ideas and attitudes towards versioning from the React team and embedded them into our workflow, and have established our own set of scripts to give us an element of automation and a set of workspace pre-release checks before we publish, to avoid repeating any common errors when publishing versions.

Monorepos

When we begin planning our new library we started off with multiple projects, each with their own separate GitHub repository, but that quickly started to become difficult and tedious to manage.

Before long we were in a situation where feature requests, issue reports and pull requests were being raised against multiple repositories, or the wrong repository, or would have knock-on effects on others and the administrative overhead of keeping everything compatible and up-to-date became overwhelming. What we needed was to establish a ‘single source of truth’ and find a way of collating feedback and disseminating updates.

Fortunately others like Babel 🐠 have already encountered this challenge and solved it with the help of Monorepos — single repositories with multiple output packages — so it made a lot of sense for us to explore adopting the same approach for our own project. We did that with a project called Lerna.

Lerna · A tool for managing JavaScript projects with multiple packages.

Pull Requests

Pull Requests are our strategy for contributing to the codebase, from people both internal and external to the core dev team. They serve as the nexus for our discussions about code, motivations for a change, making our work as transparent as possible.

Pull Request comments image, from the GitHub docs

The core team remain the only ones able to merge code into the main branch and to publish releases (more on that in a moment) but Pull Requests allow anyone in the company to propose a code change, at any point, without going through other formal processes.

In addition, we work in a tightly regulated industry and so have legal and compliance considerations for most things we do, and Pull Requests give us the ability to easily peer review and ‘sign off’ on technical changes before adopting them and allow for reviews and refinement before incorporating anything into the main development branch.

On the practical side of things we use branch-level permissions, required status checks, automated testing and commit squashing to keep things manageable. We keep as many of these checks as possible automated and they come from sources like the GitHub Marketplace (more on that in a moment.) Branch-level permissions and required status checks give us a balance of flexibility — the ability to restrict anyone from pushing to the master branch without approval from another human, for example.

About required status checks - User Documentation

(GitHub) Webhooks

Manual reviews of code are one thing, but we also have a series of automated checks and triggers that come from Pull Requests and commits.

To make our reviews easier we automate as much as we can, and as such we’ve adopted GitHub webhook integrations with systems like GoCD, Jenkins and our own homemade tools.

We use these tools to run automated unit tests, code linting checks, avoid committing work-in-progress code, ensure that change log files are updated and integrate with our JIRA systems.

Required status checks image, from the GitHub docs

For the future we’ve got our sights set on adding things like automated visual regression and accessibility testing, cross-browser and integration tests, code styles and linting, just to make it that little bit easier to keep our standards high.

At the moment, we’re currently using changelog, WIP and Auto-comment, as well as our own GoCD server integration and Github-JIRA-integration plugins:

In the future, we’d love to get involved with something like CodeCov and/or GitHub Actions. As Yoda once said, “Always in motion, the future is…”

Releases (on GitHub)

Hot on the heels of Semantic Versioning is GitHub releases. Arguably this seems like a small thing, but let me tell you why it has a significant impact…

As we build our library of code we periodically publish releases. We use semantic versioning and we publish releases to an internal NPM Registry, and then we communicate this to dependent project teams. So we have our changelog, our documentation, and our semantic versioning. On top of this there’s also our internal NPM Registry and Slack channel where we publicise our releases. All of these channels also have a lot of other things also happening in them, so there’s a lot of noise to filter through if you only want to discover the latest releases.

With a little bit of post-publish scripting we’ve tied together the GitHub Releases interface with our actual release process. Now whenever we publish to NPM we auto-update the release and release notes on GitHub, and this becomes a good long-term record of releases that’s slightly easier to work with than the changelog alone.

Screenshot from the GitHub releases interface

And finally…

Contributor Swag

All the best projects have stickers and contributor swag as incentives and rewards, and everyone* loves to display their allegiance via laptop stickers.

Pictured: One cool laptop.

We want to encourage contributions from as many people as possible in our inner sourcing strategy, but sometimes just saying ‘thanks’ doesn’t seem enough. So as an extra surprise for our contributors, we’ve arranged for some limited-edition contributor swag to be produced and distributed to everyone that has contributed to our project 😄

Recognition and Rewards

James Ford — Tue, 04 Dec 2018 13:44:59 +0000

Incentivising gratitude in the workplace

Being recognised and rewarded for your efforts in the workplace is important, as is taking the time to recognise and reward others. Taking the time to deliver a direct, verbal expression of gratitude can mean a lot to its recipient, but adding a tangible reward makes it even better.

Common sense says gratitude and appreciation for their efforts makes people feel happier. Science says happier people are more resilient, more motivated and more productive. Being included in a happy environment makes people happy.

Photo by rawpixel on Unsplash

Taking time to express gratitude to others is a mindset that doesn’t seem very common, that is to say it’s not a habit that many people naturally have. Perhaps it’s a cultural thing, but as gratitude is a cornerstone of happiness, it makes sense for a business to encourage a culture of gratitude and peer recognition in the workplace.

https://medium.com/media/526111d3b15ff7014c33ad92ba8137c9/href https://medium.com/media/bda31c71366506b91fa69defedd0e63f/href

Real world examples

I’ve worked at a few companies which support this idea of giving thanks, and here is a bit of information on the systems they have in place, followed by a little analysis of them.

Magic Moments

At MMT Digital, we had ‘Magic Moments’ – a scheme where anyone could nominate anyone else for a reward. Nominations are made by means of a message to HR or line managers, and had three tiers (small, medium and large.) When you made the nomination you would detail what exactly had been done to earn a ‘Magic Moment’, and you could indicate what you’d like to give the recipient as a reward.

The process was manually operated, with a little oversight and approval from line managers. The awarding process was fairly low-key and private, being between yourself, the recipient and your respective line managers.

HeyTaco!

At Deeson, we used an app called HeyTaco! to express gratitude, with a reward up for grabs each week for the top Taco recipients.

HeyTaco!

HeyTaco! is a Slack integration, and each day everyone in the company had a budget of five Tacos (🌮) which they could elect to award to someone else in the company.

There was no rigid criteria for awarding Tacos, it could be anything from having a birthday to smashing out some awesome work. Everyone had the same amount of Tacos, and all awarding of Tacos was public. You could award single or multiple tacos to individuals throughout the day or award them to multiple people at once.

At the end of the week a leaderboard of Taco recipients would collated and on a Monday morning the Top 5 from the leaderboard would be selected, a die would be rolled (recorded on video for the remote staff) and on a result of 1 or 6 those people would win money from the pool. On anything else, the value of the unused pool would be rolled over to next weeks die roll. You could win up to £50 each, and if the pool was larger the remaining value went to a local charity.

Competition was fierce, but I managed to get lucky and be in the Top Five one week and win with a lucky roll of the die. I spent my portion of the award money on some frivolous socks to wear at the office, because why not?

Instants

At CompareTheMarket we have ‘Instants’, which is a bespoke system for rewarding colleagues. Through the intranet we can visit the Instants page, select an employee, enter a reason for awarding them an Instant, select a company value which they’ve demonstrated and then pick a gift from the available pool. The reward is then emailed to the recipient and your respective line managers. No waiting, no approval needed, hence the name.

The rewards vary depending on availability and the office they’re located in, but can be anything from a free drink, chocolate or a shopping voucher to a half day holiday. The company has a generous but shared pool of rewards available for a specific time period, so to an extent you have to be lucky enough to get the reward you might want, but there’s always something available.

Taking a bit of inspiration from HeyTaco!, the team has recently built in a Slack integration for Instants, which lists the top receivers for the previous week, and I’m sure there’s more functionality they’re working on as we speak.

Analysis

Each of these systems is different, and arguably each flawed or open to exploitation, but any recognition systems can be exploited or manipulated if that’s your intention. Here’s some of the major pros and cons – without meaning to criticise too harshly. For all of these, my default position is that any system is better than no system at all.

Magic Moments

Pros: allows you to give personalised, graded rewards to anyone, and it’s a conscious coordinated affair to arrange a reward.

Cons: What I really felt was lacking from Magic Moments was the low-key nature of the rewards. Often they were a private thing, and while genuinely open to everyone, they weren’t promoted as much as they could have been – had these rewards been more publicly announced, it might have encouraged wider participation in the scheme.

HeyTaco!

Pros: as a Slack plugin it ties in very well with remote teams, and giving everyone a budget of 5 Tacos a day encourages very active, public and frequent positive feedback loop.

HeyTaco!

Cons: Our usage of HeyTaco! felt very much like a popularity contest. The interface was very slick and the app was good, but ultimately it felt that those that repeatedly won the weekly competition were those that worked across multiple teams – designers, managers, Agile coaches – and not people working away on a single team.

The inevitable end result being that it’s a numbers game, which drowned out the achievements and recognition of everyone when it comes to the weekly competition. The dice roll element added some fun, but after multiple consecutive weeks being pipped to the leaderboard by the same people, it didn’t quite feel very inclusive.

Instants

Pros: Instants is an automated system for issuing rewards. What I particularly like is that it feels more direct – there’s no need to coordinate with line managers or others to arrange a reward, and it arrives and can be claimed immediately.

Cons: If there’s any criticisms to be levied, it’s that the rewards come from a shared pool across the company. Thus, if you want to give someone a certain reward and there’s none left in the pool you either have to give them an alternative reward or wait until the pool is refilled!

Conclusion

Expressing your appreciation for colleagues does wonders for your working relationship and builds a tighter bond. Similarly a harmonious, appreciative working environment does wonders for productivity and employee retention. With these benefits it makes sense for the company to invest in a mechanism to promote a better, more thankful, working culture.

It’s better to have a system – even if it is flawed or imperfect – as part of a commitment to improving your company culture, than it is to have none at all.

Personally I like the visibility and interaction that HeyTaco! provides, but ultimately I prefer the ease and immediacy of the Instants system. Feedback should be direct and easy to give, and that’s what a successful system will do.

Putting our Component library on a diet.

James Ford — Tue, 06 Nov 2018 16:49:11 +0000

Performance matters, and “Houston, we have a problem.”

Photo by Bernard Hermant on Unsplash

A shared set of components is a great idea, so long as using those components doesn’t bloat your client applications. This is a brief tale of how we got it wrong, what the cause of our issues was, and how we fixed it.

Introduction

At comparethemarket.com we produce an internal library of React Components which are used across different portions of our website, which is part of a strategy that we refer to as ‘inner sourcing.’ It’s a great idea for getting consistency across our sites, but we soon realised that we had a bit of a problem with performance, page weight and download times.

A (cropped) screenshot of the network chart from WebPageTest.org

A (cropped) screenshot from the Chrome DevTools

The screenshots (above) shows some stats from one of our pages, which was using an early version of our component library. From this we could see that the Javascript file needed to load our page was around 300 KB in size, and uncompressed on the customer’s browser to a meaty 1.2 MB of raw code. 😱

The indicative performance impact of this single file — as shown in the WebPageTest.org snapshot above — is approximately half a second. That’s half a second of the browser doing literally nothing other than downloading, parsing and executing this file, during which it displays a nothing to the user.

Generally speaking, the majority of website performance issues are Front-End problems, and not hardware or network issues.

the Performance Golden Rule

This single file is actually a React application which renders the whole page. It is render-blocking, meaning that it must be downloaded and parsed by the browser before anything on the page can display, and thus this one single file is one big problem for performance.

Page Size Matters

Everything about Page Speed and Performance is all intermingled, but Javascript file size is a good indicator of how long the file will take to download, be parsed by the browser, and then execute.

There’s a certain amount of file size we can’t avoid with our Javascript. We need libraries like React for the experience we’re developing, and a shared library of components is a sensible idea, but something somewhere has gone horribly wrong if our output bundle size is so huge. — In some cases, we were seeing reports of +500Kb file size being introduced on the addition of a single button, so something wasn’t correct.

Experiment Driven Development

To take a closer look at what’s going on, we created a simple React application, looked at the application size, added a single (styled Button) component, and looked at the application size again.

The create-react-app tool outputs some pretty decent metrics as part of its default build process, so this became our quick-and-easy, side-effect free environment for testing the impact our library has on build sizes. (There could quite conceivably be issues elsewhere in the code of those making these reports, so it’s important to have a reliable benchmark.)

On the left: The out-of-the-box app. On the right: With a single component added.

It’s not quite +500 KB gzipped, but 75 KB is still an excessive amount of code for a single styled Button. Looking at the output bundles, it soon became clear that what was happening is that when we include any component, we’re also getting the source code of a dozen other components whose source code is also included in our library, but isn’t actually being used by our sample application. Not good.

Photo by Markus Spiske on Unsplash

Tree Shaking / Dead code elimination

All of this additional code for unused components is increasing our bundle size, but I had expected Webpack — with its native support for dead code elimination — to automagically remove that unused library code and deliver a more optimised bundle.

Reduce JavaScript Payloads with Tree Shaking | Web Fundamentals | Google Developers

So, why didn’t it?

The answer to this question is complex and fairly specific to our project, but it involves our Babel configuration, our CommonJS output format and our use of CSS modules. All of these elements combine to create library bundles which fundamentally cannot be tree-shaken by Webpack.

So, how to we build a tree-shakable library bundle?

A better bundler… with rollup.js

After a lot of experimenting with different options and configurations for Webpack, we eventually realised that it was ‘the wrong tool for the job’ when it came to building our bundle of library components. As a replacement we picked rollup.js, which amongst other perks is better at producing library bundles that supports dead code elimination.

rollup.js

Our consumer applications still use Webpack to package our final React applications, but using rollup.js for our library bundles now means that when Webpack compiles its output, it’s able to tree shake our library code and eliminate the components that aren’t used.

And the final results?

On our example project, switching to Rollup-bundled libraries immediately saves an extra 22 KB out of 75 KB (almost 30%) with literally no changes to the example application itself.

On the left: Create-react-app using our library packaged with Webpack, on the right: using the library packaged with rollup.

Even more exciting is the effect when we applied this same updated library to a real-world example, which yielded optimisations closer to 60% of the total file size:

File sizes after gzip:

 287.02 KB (-430.7 KB) build/static/js/main.5978ff57.js

430.7 KB saved from our output bundle, thanks to using a tree-shakable React Component library, bundled with properly-configured Rollup instead of our out-of-the-box Webpack configuration.

Summary

Our React Component library was part of a natural evolution from several projects, but we took our eye off the performance impact of the code that we were building and it quickly snowballed into something that had detrimental effect on the projects that included it.

The root cause of this problem was our bundling strategy, and the fact that it couldn’t determine which code was necessary to render the components, so it always included everything.

The solution was to shift our bundling strategy into Rollup and also to change our Babel configuration to one that better supported dead-code elimination.

Webpack isn’t a bad tool, but it has to be configured properly and used for the correct tasks, and because of the gradual nature of our project evolution, our setup wasn’t suitable. Pausing, evaluating, and spending some time focussing on our build processes has allowed us to make these gains.

Key Takeaways

Javascript syntax patterns, such as CommonJS, have different side effects when it comes to dead-code elimination in your code bundler.
Webpack is good for bundling projects, but Rollup is better for libraries.
You’re still going to need Babel, regardless of whether you’re working with Webpack or Rollup. It’s the Babel configuration which has the biggest effect on the outcome.
Test, test and test again, and do it in an environment that is representative of your anticipated customers’ workspace.

Bonus: Create-React-App 2 & Webpack 4

Since originally penning this article, we’ve had clients update to using create-react-app version 2 and Webpack 4, which delivers even more benefits to our bundle size and performance, with no additional code changes.

Repeating the tests with version 2 yields these results in our example application:

On the left: Create-react-app 2, on the right: with our library added

And here’s a real world example of the bundle size output on create-react-app version 1.1.4:

File sizes after gzip:

188.49 KB build/static/js/main.53169928.js
 112.37 KB build/static/js/BasicQuestions.9168ec4c.chunk.js
 110.43 KB build/static/js/ComplexQuestions.5706b399.chunk.js
 110.4 KB build/static/js/SubQuestions.194d1d33.chunk.js
 110.3 KB build/static/js/CustomQuestions.736521d5.chunk.js

File sizes after gzip:

255.75 KB build/static/js/main.b68bc851.js
 846 B build/static/css/main.a86b8cd7.css

And the same real-world project again, this time using create-react-app version 2:

File sizes after gzip:

112.58 KB build/static/js/7.58c36855.chunk.js
 66.56 KB build/static/js/0.72b13458.chunk.js
 65.62 KB (-122.87 KB) build/static/js/main.20b9faa3.chunk.js
 40.56 KB build/static/js/1.ac53a9a6.chunk.js
 3.32 KB (-109.06 KB) build/static/js/BasicQuestions.6df1fce5.chunk.js
 1.47 KB (-108.96 KB) build/static/js/ComplexQuestions.61562335.chunk.js
 1.44 KB (-108.97 KB) build/static/js/SubQuestions.d5066422.chunk.js
 1.24 KB (-109.06 KB) build/static/js/CustomQuestions.b2e5a8af.chunk.js
 1.24 KB build/static/js/runtime~main.6da60b22.js

File sizes after gzip:

156.02 KB build/static/js/1.c932bec8.chunk.js
 89.89 KB (-165.86 KB) build/static/js/main.2f744215.chunk.js
 837 B (-9 B) build/static/css/main.8ee9f0fa.chunk.css
 763 B build/static/js/runtime~main.229c360f.js

A ‘simple’ upgrade that implements cleverer bundle splitting for shared code libraries, and we’ve automatically saved even more 100’s of KB across the site, which is pretty neat. 👍

ESLint + Visual Studio Code: Editor integration For The Win

James Ford — Thu, 18 Oct 2018 09:31:36 +0000

Coding is complicated — there are loads of best practices to remember, guidelines to follow and ‘known issues’ to avoid — so it helps to have a ‘helping hand’ while you work.

ESLint — https://eslint.org/

ESLint - Pluggable JavaScript linter

ESLint is an extendible linter — or code checking tool — that can be set up as a standalone tool or integrated into the Visual Studio Code editor.

‘A linter or lint refers to tools that analyze source code to flag programming errors, bugs, stylistic errors, and suspicious constructs.’ [Wikipedia]

With ESLint properly configured in your editor, your development experience can be almost as-if you’re continually pair-programming with (an insufferably polite and knowledgeable) know-it-all. And as daunting as that might be, it’ll help you produce better quality code and teach you as you work.

Configuring Visual Studio Code with ESLint

First, a preamble: I’m going to start by setting up a React project using create-react-app, which gives me a package.json at the root of the project and manages dependencies through NPM.

To get a new project set up, run the following commands:

facebook/create-react-app

First, install create-react-app:

npm i -g create-react-app

Then, to create a new project:

create-react-app eslint-integration-example

And when we want to, we can run the project like so:

cd eslint-integration-example
npm start

Setting up ESLint

ESLint could be installed globally, but for better portability and resilience let’s add it, and some plugins and settings, to the project, with the following command:

npm install --save-dev \
eslint \
babel-eslint \
eslint-config-airbnb \
eslint-config-babel \
eslint-config-prettier \
eslint-plugin-import \
eslint-plugin-react

With all of those plugins installed, my package.json file looks like this:

The package.json file with ESLint configs and plugins

Strictly speaking, the necessary plugins for adding ESLint is only actually eslint, but I’ve added the other plugins in this example because they’re the ones I’m currently using in my active projects.

Alongside these npm packages, we’ll also add a configuration file to the root of the project, called .eslintrc, which looks like this:

My .eslintrc file

This tells ESLint to import the airbnb , prettier and jsx-a11y configurations (that we installed in the previous command) as a starting point, and gives us a springboard from which to extend the configuration in the future.

To run eslint against our code we can use the following command:

npx eslint src/App.js

Which gives the following output in the terminal:

Test output from the terminal

To make things a little easier to run, we can also add this command to the scripts section of the package.json file, making it look like this:

The scripts section of the package.json file

It’s already pretty neat to be able to run ESLint and highlight issues with our code like this, but what really takes it to the next level is editor integration.

Integrating with Visual Studio Code

The ESLint plugin can be installed from the Visual Studio Marketplace, and gives easy integration with Visual Studio Code.

ESLint

What does it actually do though? It adds inline code error highlighting, which transforms your editing experience (as illustrated below)

On the left: Without ESLint integration, On the right: With ESLint integration

With the plugin set up, rather than having the ESLint as a task that needs to be invoked manually, it runs automatically, displays the results inline with your code, and updates as you type so you get the kind of user experience you’d expect from a spell-checker but with the complexity of your ESLint rules.

It’s not all about inline validation though, there’s also some sweet integration with the rest of the editor, such as the ‘Problems’ panel:

(above) ESLint integration with the ‘Problems’ panel

Extension Suggestions

Seeing this editor integration does rely on having the correct plugin installed though, which can be a bit of a tedious task to maintain if you’re on a development team. To assist with this, we can add a VSCode extensions configuration file, which will prompt Visual Studio Code to suggest the installation of extensions when appropriate.

Create a file called extensions.json in a directory named .vscode in the root of the project, with the following content:

{
 "recommendations": [
 "dbaeumer.vscode-eslint",
 "esbenp.prettier-vscode",
 "dzannotti.vscode-babel-coloring"
 ]
}

The official documentation for this feature can be found here:

Managing Extensions in Visual Studio Code

Why ESLint?

What makes ESLint cool is that it’s something that can be extended to check for more than just logical code errors, it can check for things like accessibility issues or inconsistent code formatting, which is why I’ve added react-jsx-a11y and the prettier plugins to plugins list.

And with all of that setup, you can experience what it’s like to Pair Program with a stickler for consistency and best practices, without actually having to be pairing.

Photo by Mimi Thian on Unsplash

Building a humanised command-line interface with Node.js

James Ford — Mon, 13 Aug 2018 08:31:01 +0000

Photo by Rock’n Roll Monkey on Unsplash

Because config management sucks.

Background

As a by-product of my day-to-day software development work I have created a lot of specific-task automation scripts. If there’s a task that I need to perform on a fairly regular basis that involves multiple steps and which could be automated, I will (eventually) write myself a script to automate it.

I do this because — generally speaking — I enjoy writing the scripts and because abstracting mundane tasks into these scripts reduces the cognitive overhead and frees my time up for thinking about other aspects of my day-to-day job.

Configurable code

It’s well established that hard-coding values into code instantly makes it less reusable, and that extracting those values into code variables is logical because it then becomes easier to reliably make broad changes to the code.

To make a script reusable without actually editing source code each time there are two common strategies. One of them is passing configuration as arguments on the command line, and the other is creating and editing a config file which stores the options in a more long-lived format. Command line arguments are good for one-off commands and config files are great for repeated reuse with the same settings.

Problem

Both command line arguments and configuration files require knowledge of the available options and a measure of preparation — neither is quite as easy as a GUI (Graphical User Interface) could make the experience, and neither of them typically have an intuitive interface.

I always assume that in a few months’ time I will have completely forgotten everything about how to configure my scripts, and with no hope of finding assistance on StackOverflow or Google I need to ensure that I make them straightforward and operator-friendly.

A humanised experience 🤖

Essentially I want my command-line interfaces to be a little more human in their interactions with me and behave more like an artificial assistant like Alexa or Siri and less like a computer. That’s a humanized experience.

Without further ado, let’s make a simple script more human.

Core code

(Below) I have a Node.js script which communicates with the GitHub API to retrieve stats about a repository. It’s currently set up to take a configuration file in JSON format and also returns the output as JSON.

getstats.json

What the code (above) is doing is using the node-fetch module to make a request to the GitHub GraphQL API, and then it’s taking the response from that API and doing a bit of processing on the responses before spitting them back out as a string.

The information that’s absent from the source code and which is supplied by the config file is the repository owner , the repository name , a start date and an end date , all of which are used to identify the target repository and filter the response data.

For completeness, the GraphQL query used looks like this (below):

query.gql

Now we’ve got the code, how do we actually execute it and fetch our data?

Executing scripts with command line options

Without a humanised journey, I’d use a command like this (below) to execute my script return the repository stats:

node index.js psyked psyked.github.io 2018-08-08 2018-08-12

Executing scripts with config files

If I was taking the config file approach, I’d create a .json file with those settings in, save it as psyked-config.json in the same directory and then use the command (below) to run it:

node index.js ./psyked-config.json

It’s a little less to type on each execution and does ensure that the options would be the same each time, but it’s still not user friendly enough.

Humanised command line experiences

The humanised approach starts with only the simplest command (below):

node index.js

From then on it switches over to an interactive experience, which looks like this (below) — asking questions, hinting at responses and finally mapping inputs into the same data structure that would exist in a proper config file.

This experience is all facilitated by a module called Inquirer , which adds a really simple layer of code that supports several common types of question to the command line.

SBoudrias/Inquirer.js

Inquirer can prompt for several types of data, chain questions together, validate the responses and more. To achieve the questions used in the screen capture above I’m using the following code (below):

index.js

… and that’s about it. Humanised command-line interfaces with Node.js.

The big advantage that this approach has over other options is that playing around with the data feels so much easier. Putting all the values on a single line feels like an arduous process, especially if you make a mistake. Creating and editing multiple config files is much the same, but following an old-school wizard approach to running scripts? That’s fun. 🎩

Conclusion?

The Inquirer.js module makes it easy to make command line interfaces more operator friendly. It’s simple to author the questions and map the results into our code, and the experience at the outcome far outweighs the effort spent configuring the module.