DEV Community: Harriet

What You Need to Know About Optimistic Concurrency

Harriet — Wed, 27 May 2020 18:39:16 +0000

What is Optimistic Concurrency? First, a scenario...

Imagine you are a humble bookseller and you have signed into your shop's stock management system. A customer has just called you to tell you they want to add 2 more copies of Normal People to their order. You have their customer record open and you can see their order. They've already got a copy on order, but they've decided to give copies to their mother-in-law and their brother, too. It's just such a good book. While they're on the phone with you, they ask you to change the delivery address to their office, which you do, as well.

However, whilst you're updating the order, your colleague has just got to work and has opened her inbox to see an email from the same customer that they sent over the weekend, before they decided it would just be easier to phone. And also at that point they were only thinking about their mother-in-law so they only asked for 1 extra copy. Your colleague actions the email, opening the customer record, and sees that 1 copy of Normal People has been ordered, so she updates the total to 2. The email makes no mention of changing the delivery address so she doesn't touch that.

Both you and your colleague have the order record open, and you have both made different changes. One of you is going to save the updated record first, and then the second person is going to save the record. Whose changes are going to "win"? Will the customer end up with 2 or 3 copies of Normal People and will their books get sent to their home or office address?

This scenario, where two or more people have got a copy of a record and are are about to overwrite the other person's changes, is pretty common. It is often called The Lost Update Problem or a Mid-Air Collision.

We could simply say that the last person to submit the update "wins", but that wouldn't be great because we'd potentially loose a really important update that somebody else had attempted to make. In the above example, the humans involved know that the correct number of books is 3, because that was the most recent thing the customer said. But if your colleague made their update after yours, the order would end up with 2 books on it.

There are two solutions to this problem, and Optimistic Concurrency is one of them. Pessimistic Concurrency is the other one.

Optimistic Vs Pessimistic Concurrency

Concurrency is the idea that two processes (in this case, updates) might be going on simultaneously.

Both Optimistic and Pessimistic Concurrency offer solutions to deal with the conflicts that this concurrency might cause. They are both concepts and don't prescribe specific technical implementations.

Optimistic concurrency is the idea that these kinds of conflicts won't happen very often (hence it's "optimistic"), but when they do happen, the humans involved should be given the opportunity to react and decide what to do. In the above example, that means that if your colleague saved the record first, then when you attempted to save the record afterwards you would be warned somehow, and would have to decide whether to overwrite the other changes or not.

Pessimistic Concurrency assumes that these kinds of conflicts might occur regularly, and that it would be a pain to have to keep asking a human what to do, or perhaps not even feasible. A solution that employed pessimistic concurrency would need to ensure the system was protected from these kinds of conflicts ever happening. That generally means applying a "lock" to a record, so that if someone else is in the process of updating it, another person or system wouldn't be able to. In the above example, that means that your colleague wouldn't have been able to open the record while you were busy with it, thus preventing any possible conflicts.

Optimistic Concurrency in Web Applications

Web applications are one environment where pessimistic concurrency isn't really possible. This is because to maintain a "lock" on data in a database, the record would have to be kept in a "locked" state throughout the whole process of:

You opening the record when the customer first calls you
All the time you are updating the record

The lock would only be released when you are done with your call and you click the button to submit the Update request to the server. A fair amount of time could have passed without anyone else being able to update the record, which could easily cause a problem. And you might just get distracted and walk away, and then the lock would just stay in place until it expired at some point...

In web applications, the app doesn't maintain a constant connection with the server or the database, so it's impossible to know whether you're still actually editing the record and the lock needs to remain, or whether you've gone off to do something else. In other kinds of applications, where the app is directly connected to the database, locking can be a solution, but not in web apps.

Therefore we have to rely on Optimistic Concurrency, which means that we need a mechanism for identifying when these conflicting update requests happen, and ideally we need to bounce the problem back to the user for them to verify.

Isn't this just a merge conflict?

If you've worked with Git and have experienced merge conflicts, then you might be thinking that this whole idea of multiple conflicting updates sounds pretty familiar - and you'd be right.

Merge conflicts are a great example of Optimistic Concurrency in action. Because Git by its nature is distributed there's no way a single user could maintain a lock on a file they're editing, so the only solution is to allow multiple people to concurrently edit files, but when it comes to submitting those edits, if there are conflicts (to Git, this means lines of code which have been changed by two people) then the user submitting the most recent update is left with the oh-so-pleasant task of sorting out those conflicts and deciding which is the "right" version.

How is Optimistic Concurrency actually implemented?

Technically, optimistic concurrency is usually implemented by using version numbers. Every record in a database will be given not only a unique id but also a version number or timestamp. This means that every time a record is updated, logic can be executed that compares the version number of the record that its trying to save against the version that's stored in the database.

In the above example, that means:

You would have loaded version 1 of the record from the database
Your colleague would have also loaded version 1
You press "Save" first, making a request to update the record. The application submits the version number (1) along with the edits you want to make.
The backend accepts the request and updates the record. This is now version 2 of the record.
Your colleague presses "Save", making another request to update the record. The application submits the version number (1) along with the edits they want to make.
The backend rejects the request, because it can see that version 1 is not the most up-to-date version.
Your colleague is informed about the conflict and may be presented with an option to overwrite your changes, or resolve them.

Optimistic Concurrency in action

Here are a few examples of how various frameworks/technologies implement Optimistic Concurrency:

DynamoDB uses the mechanism of Conditional Updating, where you can declare that the item should only be updated if certain values match what you expect. You could check a version number, or you could check that a property hasn't been changed by someone else.
Rails uses version numbers, which it calls Optimistic Locking.
Azure's CosmosDB gives every document an _etag property, which it updates automatically when the item is saved.
In fact, plain HTTP requests can implement etag response headers, which can also be used like version numbers to check whether a resource has been changed or not since it was last loaded. Frameworks like Java's Spring provide support for generating HTTP etags for resources available at URLs.

Optimistic Concurrency on the frontend

On the frontend it's really up to you how you handle optimistic concurrency!

Assuming you receive a server error that you can identify as a concurrency error when trying to update an outdated record, it's up to you what you want to do.

You might decide to tell the user that the record they're trying to update has been modified by someone else, and ask them to reload the page.

Or you might provide a more interactive way for them to resolve the conflict, like some live document editing apps do - or think how Github allows you to resolve merge conflicts.

Not be be confused with.... Optimistic Updating

Okay, one more thing! Optimistic concurrency isn't the same as optimistic updating.

Optimistic updating is specifically a front-end concept and it means updating the UI before you've actually got confirmation from the server that the change was written to the database.

In our humble bookseller example it would mean that after pressing "Save", the request is sent to the server but rather than wait the 500ms or whatever for a response, the frontend application updates the UI to show the changes as though everything was successful. As you can see, this is optimistic!

Optimistic updating allows for a more fluid user experience. A great example is when making quick inline edits to a record, like editing a comment or upvoting an article. The edit/action results in a request to update the database record... but if you had to wait for confirmation before you saw your little "thumbs up" appear, the experience would feel very laggy.

I hope this has been helpful, follow me on Twitter and let me know if you have any comments 🤗

NB. This was originally published at harrietryder.co.uk

What did your onboarding documentation look like when you started your job?

Harriet — Sun, 29 Mar 2020 18:14:29 +0000

I'd love to know what onboarding materials were provided to you when you started your job.

What does good documentation look like for getting started with a codebase? For getting started building features?

It's common to receive documentation around getting your dev environment set up, and the basics of building/running/testing/deploying the software you'll be working on.

But has anyone been provided with anything further? I'm imagining some "this is how we build stuff" documentation, detailing preferred patterns, or giving an introduction to the codebase, useful shared components, etc.

How about "this is how we work" documentation - i.e. who approves PRs, how to ask for review, how long is acceptable to wait, how work gets assigned, etc. Has anyone ever been issued useful documentation around process?

If so, I'd love to hear

What the project was (language, framework, basic function)
What general areas the documentation covered
How useful the documentation was
If you have been involved in maintaining the documentation, how you have found that.

Hiding Functionality Behind Feature Flags with Launch Darkly

Harriet — Sun, 29 Mar 2020 17:56:50 +0000

Originally published on my personal blog, harrietryder.co.uk

What are feature flags?

In programming, a "flag" means a marker of some kind. Usually it's a boolean value - true or false which identifies or flags some piece of data as being in a certain state.

A feature flag is a marker that identifies whether a specific feature should be available to the user of a piece software. If you want to enable a feature of your application for only specific users, you can employ the concept of a feature flag to indicate whether the feature should be enabled or not. This feature flag will either be true or false for a specific user, and your application logic should check this value before rendering any UI or performing any functionality related to the specified feature.

What are the use cases for feature flags?

Hiding features when not applicable to certain users:

Imagine your application is a platform for collecting donations from donors around the world for numerous different charities. In the UK, there's a concept of Gift Aid, which allows charities to claim back tax on donations made from UK tax payers. So for a charity operating in the UK, it makes sense for the application to ask the donor if they are UK taxpayers. But for a charity operating in France, that option makes no sense and is just confusing. You could use a feature flag to decide whether, for this particular charity, you want to enable the Gift Aid feature.

Safety when rolling out new features:

Another very common use case of feature flags is to provide some protection when rolling out or testing new features. It can be hard and slow to roll back a lot of changes relating to the implementation of a new feature, but if you can control a simple boolean value and easily switch it from true to false or vice versa, you have a really quick way of turning off a feature if it looks like it's going pear shaped or you want to pause it for a moment.

The key is, your code needs to check the feature flag wherever there is some functionality relating to a given feature, so that turning the flag to false will seem to remove the functionality. All the code and logic for the feature remains - you're not reverting any changes - it's just not accessible any more:

function Header() {
  return (
    <Container>
      {flags.darkMode && <DarkModeSelector />}
      <Navigation />
    </Container>
  );
}

A/B testing and percentage rollouts:

Another use case is when you want to slowly expose your users to a new feature to begin gaining feedback/metrics before unleashing it on the whole world. For this, you might select an arbitrary percentage of users for whom to enable the given feature, and begin to up the % as you gain confidence. Or you might want to test 2 different variations of a feature (A/B testing) on two different segments, so you'd need to manage which users belonged to which segment through some kind of feature flag management.

What is Launch Darkly

Launch Darkly is a piece of software that enables you to manage feature flags in one place.

You can set a list of flags through the Launch Darkly web application, obtain a client token, and using an SDK for your programming language of choice, access these flags anywhere in your frontend or backend code.

One great feature about Launch Darkly is that its SDKs receive real-time updates whenever a flag changes. This means that even in a SPA, the end user will receive flag updates as soon as you change them on the Launch Darkly app, so you can turn features on/off immediately. You don't have to wait until a user does a hard page reload and fetches the flags again for the feature to turn off. When I first tried this out and saw how smoothly it worked, how the features just "popped off the page", it felt like magic!

Launch Darkly users

Not only can you turn featues on/off for your whole application, but you can manage features for specific users (i.e. the UK charities vs French charities example).

To this end, Launch Darkly offers the concept of "users". You can create a Launch Darkly user (you'll want to create this user when a user signs up to your application). This Launch Darkly user is identifiable as that specific user in your application (most likely by ID), and you can also store other information against this user such as Country of Operation, age bracket, or any other custom piece of data that might help you evaluate whether they should be shown a feature or not. These users don't need to be (and probably shouldn't) be exact copies of the users you store in your own database - just store the info you need and don't store PII unless you absolutely need to.

From the Launch Darkly dashboard, you can very easily apply feature flags to a selection of users based on the attribute you have stored about them (e.g. by country of operation), or by an abritrary percentage.

Managing Environments

Launch Darkly makes it super easy to manage environments - you get a different client key for each environment, so it's a piece of cake to have features turned on/off in different environments. For example, you want your Dark Mode feature turned on for all users in QA as you're testing it at the moment, but you can keep it turned off in Production for now until you're happy with it.

You are probably loading different environment variables in different environments anyway so it should be simple to load a different Launch Darkly API key too.

Integration in a React App

Just so you can get a picture of how this looks from the front end, here are some contrived code examples.

Imagine your backend has already done something like this upon signup to create a Launch Darkly user:

const user = {
  "key": "aa0ceb",
  "country": "GB",
  "custom": {
    "groups": ["Google", "Microsoft"]
  }
};

ldClient.identify(user);

Through the Launch Darkly dashboard you've enabled the "Gift Aid" feature for all users in GB.

Your frontend React Application is wrapped in a Launch Darkly provider at the root:

import { withLDProvider } from 'launchdarkly-react-client-sdk';

export default withLDProvider({ 
  clientSideID: 'your-client-side-id'
})(App);

When a users signs into your frontend application, you need to match them with the Launch Darkly user and retrieve their flags. In your sign-in success flow (within your wrapped App component) you'll do something like this, to identify your current signed-in user with Launch Darkly:

import React from 'react';
import { useLDClient } from 'launchdarkly-react-client-sdk';

const App = () => {
  const ldClient = useLDClient();

  const onLoginSuccessful = (user) => ldClient.identify({ key: user.id });

  return (
    <div>
      <MainApp />
    </div>
  );
};
export default App;

If you don't have users, and you just have flags for your entire application, that's fine too, you can skip the identify step above.

Anywhere in your wrapped application you can use the useFlags hook to obtain the flags for the current identified user or for the application as a whole. NB there is also a higher order component pattern that you can use - but in my view hooks are easier and nicer to use.

import React from 'react';

import { useFlags } from 'launchdarkly-react-client-sdk';

const Header = () => {
  const { darkMode } = useFlags();

  return (
    <Container>
      {darkMode && <DarkModeSelector />}
      <Navigation />
    </Container>
  );
};
export default Header;

There are various options for working with HOCs, async components or hooks in React, so you can use whichever pattern you prefer. There are also client-side and server-side SDKs for a large range of languages, see the docs for more information 🙂

The Three Month Rule of a New Job

Harriet — Sat, 07 Mar 2020 19:55:11 +0000

Originally published on my personal blog.

I've always maintained that it takes around 3 months, minimum, to begin to feel like you're really acing it in a new job. I'm speaking from the point of view of a software engineer, but perhaps it also holds true for any kind of more complex knowledge work, especially work that requires acquiring domain knowledge and close collaborative teamwork to do well.

The figure of 3 months came from my own experiences, and from talking to many others. I admit, this figure is probably more biased towards people at the start of their careers, since I've worked with lots of people who've been getting their first, second or third jobs in tech and listened to their experiences. Maybe with more experience, in more leadership-style roles, you can expect to feel like you're doing your best work sooner. Maybe a consultant, brought in to help with a specific problem or help with their particular area of expertise, has a different experience. I'd really be interested to know.

But for me, the figure of 3 months based mainly on:

The need to learn the problem domain. Unless you jump from one place to a competitor, solving a similar problem in a similar space, there will be a whole load of domain knowledge you need to pick up. For example, when I worked in analytics, I needed to learn about how digital marketing worked. Now I build a product for charities, I'm learning about how the charity sector works.
Figuring out the team dynamics. Software engineering is so collaborative, you can't do your best job until you know how to best work with your team. You need to figure out who is good at what, who takes responsibility for various areas, how to get the best out of various people and just generally how to fit in. I've never joined a company where all this stuff is completely obvious, or documented in some workflow system - it always seems to require a mix of intuition, attention, experience to absorb.
Learning the codebase. Depending on the kind of work you need to do, the kind of tickets you get to work on, and the size of the project (or your area of it) this might be more or less challenging, but it's definitely still something that takes time. You can get a lot done in a codebase you don't fully understand but it won't be your best work. Even when you ask about things you don't understand, there will be a whole load of unknown unknowns, such as what trade-offs were made at certain times, the fact that there is a component tucked away in an obscure project which does exactly what you're about to build, the fact that the tech lead has a handy bash script saved to parse those logs if you need it. You only pick these things up with time.
Learning new tools or technologies. It's unlikely you'll jump from one place to another which uses the exact same tech stack in the exact same way, with the exact same tooling thrown in. So there's likely to be some ramp-up as you pick up these new things.
Feeling confident in new responsibilities. If you took a new role partly for career progression, then your responsibilities and job role might have changed, and then of course you might be partly learning on the fly how to do a whole new job!

I always found the figure of 3 months helpful, especially when talking to entry level developers, because I think it sets realistic expectations about how productive and useful you might feel at the beginning, and how much each role itself is different to other roles, and requires its own up-front learning before it can be done well.

I'm not saying that after 3 months, you should feel like you can do the job with your eyes closed. But by that point, I think people are often beginning to feel more confident that they are valuable to their employer, know what they are capable of, are more aware of the areas they are less experienced in and might need help with, and are able to work more effectively with their team.

I've changed jobs a few times during my career as as a software developer and only once did I feel like I could already do the job well on my first day. In retrospect, that was great for my employer and great for my self confidence, but not great for my personal/career growth.

On the other hand, I've moved into roles where I very acutely felt that sense of "omg they're gonna fire me before I even hit the 3 month mark", which was challenging for my self-confidence but amazingly I never got fired, and grew a tonne instead.

What you can do

I think on the whole, companies understand that there is a ramp-up period involved in on-boarding any employee, at any level of seniority. It would be expected that a more senior developer needs less support to get started, as they will have seen more patterns by which they can make sense of this particular project, and this particular way of doing things, worked with more different teams and hopefully be pretty good at picking up new tech. However, even so, all the above points still apply and you still won't be doing your best work immediately.

On the other hand, I hear from a lot of developers earlier in their careers (and I include myself in this, 4 years in!) who find starting new jobs challenging, and beat themselves up over all the things they don't know straight away. That's why I find the 3-month rule helpful. If you keep the 3-month rule in your head, try to defer judgement on whether you're doing a good job or not until you pass that threshold. Then look back and ask yourself:

What did I learn in the last 3 months?
Am I doing a better job now, than 3 months ago?
What do I want to get better at over the next 3 months?
What was something that was really challenging 3 months ago? How much better/faster could I do that task now?
What tickets have I closed/features have I worked on/successes have I had so far?

You still might be feeling out of your depth, but hopefully reflecting on how far you have already come, and knowing that it's totally normal and expected to feel like this, can help.

And in another 3 months time, you can expect to ask yourself the same questions, and once again see that compared to 3 months ago, you've come leaps and bounds.

You should also remember that we're not paid to just come in and bash out code. We're paid to solve the business problem, and that involves asking questions, being inquisitive, and thinking. The period we spend learning about a new company/problem/product is part of the job. We're also paid to keep up to date in our industry and make sure we're using the tools/technologies that can best solve the business problem, so it's totally normal that we're paid to be learning.

What employers can do

Obviously employers have a lot to answer to here as well, in terms of developing a positive onboarding experience that helps new hires do the best work they can do, at that point in time, and supports them in their ramp-up period.

Larger companies might be able to provide specific training programmes for new employees, whereas smaller companies are more likely to implement pair programming as one of the principal methods of knowledge transfer.

But it's also important to make sure that the work given to team members is suitable for someone who is learning a new codebase. Sometimes this work isn't the most valuable work but the net overall effect of easing someone into a new codebase/project in a sensible way is still greater than dumping a complicated, important problem on them.

I'd love to hear examples around really positive onboarding experiences, as I'm lacking in anecdotal experience here 😂

Isn't this all a bit negative?

Sometimes I feel like this 3-month rule can come across as a bit negative. Shouldn't we all be preaching positivity and a can-do attitude? You won't be awesome until you believe it yourself, right?

But I think the alternative is more harmful, especially for people early in their careers or making career changes. There are enough reasons to second-guess yourself already, without unrealistic expectations about how quickly you can be doing your best work being thrown into the mix. If you're lucky and get a super supportive team who take great care to curate your work for you so that you're always pushed exactly the right amount, and make you feel valuable from day 1, then you're one of the lucky ones. You can carry on feeling unexpectedly competent and confident and no harm done.

But if you're like most people, working in an imperfect team in an imperfect world, you're likely to feel out of your depth at first, and if you're not prepared it can be a real challenge. You might think you're just not cut out for this work, or that you don't belong here, when actually you're just experiencing something totally normal.

I'd really love to hear other experiences around this - especially from people who've never had this feeling, or people at other points in their career on whether a similar timeframe still holds true, and how you navigate it.

The introduction to servers I wish I'd had

Harriet — Sat, 09 Nov 2019 15:47:50 +0000

When I first learned to code, I had no idea what a server was, what one looked like or how I would ever end up using one.

In this guide, I'll try and provide the intro that I wish I'd had, and hopefully you'll end up with a better idea of what a server really is and how you can use one. These instructions should work whether you are on a Mac, Linux or Windows 10 machine.

So, what is a server?

You might read all sorts of technical explanations of what a server is, but I like to describe it as a computer that's always connected to the internet. This is because it usually performs tasks that need an internet connection, such as providing the files that make up a website. That's the scenario we'll work with for this article.

When someone uses their web browser to request a website, a request is made to a specific server for the files that make up that website, and that server is always connected to the internet and is always ready to provide those files.

There is nothing special about that server apart from

It's always connected to the internet
It's always on
It's been configured to "listen" for requests and provide the right files

Your own computer could be a server, if you really wanted.

In reality, when we need a server we tend to rent one, or part of one, that's stored with many others in a big warehouse.

Servers don't have screens because humans don't need to interact with them through a graphical user interface. So you tend to see a load of them all stored efficiently together in racks.

So how do you get a server and configure it?

If you want a server, these days you tend to go to a provider such as Amazon Web Services, Google Cloud Platform or Digital Ocean and "provision" yourself a server.

Let's do just that!

Most of these platforms have free tiers for new users, or generous allowances for low-use customers.

AWS allows new users a year of use of a small server, but even if you're not in your first year, we can create a server now and use it for a very short amount of time and it will only cost a few pennies.

Sign up with AWS

If you haven't signed up for AWS before, create an account now! You will need an active credit/debit card to sign up but we won't be doing anything that costs money.

Provision an instance

AWS calls servers "instances" because technically you're not getting a server of your own, you're renting "compute space" in the cloud, in other words, a virtual server which is just a small slice of space on a single piece of hardware.

But for all intents and purposes it feels like your own server. And it's much cheaper than buying a server just for yourself.

First, find the service called EC2:

Then, choose the option to launch an instance:

There are various types of instance you can choose, but only some are available in the free tier. Select the Amazon Linux 2 AMI type which is an instance running the latest version of Amazon's Linux operating system. Most servers run a version of the Linux Operating System.

On the next page, ensure the correct AMI type is still selected and press "Review and Launch":

When you are asked whether to use an existing Key Pair or create a new one, create a new one and download it.

Your instance will then take a moment to provision, and when it does you'll be able to go to the EC2 dashboard and see a list of your running instances, probably only with 1 instance in it unless you already had some others running from before.

Connect to your instance

We mentioned that instances don't have screens, so how can we operate them? We can remotely access and operate them from the command line, by connecting to them with a command called ssh. SSH (Secure Shell) is a protocol for remotely and securely connecting to and operating computers across a network.

When we use SSH to connect to our instance we will need to prove we actually have permission to take control of our instance. This is vital, or anyone could connect to our server and wreak havoc, especially since servers are usually doing business-critical, sensitive tasks such working with user data or payment details.

When creating your instance you will have downloaded a private key. This key matches a public key, stored by AWS. Only the owner of the private key will be able to connect to the instance. This is called Public Key Authentication and is a standard method for authenticating with remote servers.

First, maybe move this key file out of your Downloads directory to somewhere you'll know where to find it in the future, if you ever use it again.

Then, we need to change the permissions on the file to make them more secure:

chmod 400 my-first-key-pair.pem

This means that it can only be read by the owner of the file.

Finally, go to the AWS EC2 dashboard and find the public IP of the instance:

Then, use the IP to construct the below command, making sure you are providing the correct path to your .pem certificate file which you just modified, and substituting the IP address "34.244.187.6" for your instance's:

ssh -i my-first-key-pair.pem ec2-user@34.244.187.6

When asked whether you want to continue, type "yes" and hit enter.

You are now connected to your EC2 instance!

If you are using a Linux machine and you don't already have the ssh command enabled, you might need to run the following commands to enable it first:

sudo apt update

sudo apt install openssh-server

Now this command should work:

ssh -i my-first-key-pair.pem ec2-user@34.244.187.6

Look around! Make some files!

Your terminal is now behaving as though it is the terminal connected to your EC2 instance. If you ls and pwd you will see that it is not your computer any more. Feel free to make some files, explore the file system.

There are no files in your home directory as it's a completely new instance but you will see plenty of standard linux files if you move to the root directory cd /.

Making our instance serve a website

If we want to configure this instance to serve a website we have a do a few things:

Allow HTTP connections, as browsers request websites using the HTTP protocol and currently our instance only allows SSH connections
Configure some server software that will respond to HTTP requests by serving files
Create these files (or get them onto the machine somehow)

Let's do these three things...

1. Allow web traffic to your instance

To allow web traffic we need to configure the "security group" settings of the instance. Click on the link to "launch wizard" next to the "Security Groups" heading on the EC2 dashboard.

On the Security Groups page, ensure your security group is highlighted and then select the Inbound tab, and press Edit:

Add a new HTTP rule and leave the default settings that are provided. This will allow HTTP traffic into the instance from any IP address:

Save this new rule.

2. Install Apache

Back in the terminal which has an SSH session established with your instance, run the following command to install a piece of software called Apache. Apache is a simple server software, and its default configuration allows it to serve static files for a single website when the instance receives HTTP requests on port 80, i.e. requests from browsers or other web clients.

We can use the yum package manager to install httpd which is the package name for Apache, weirdly.

sudo yum install -y httpd

3. Create some files

By default Apache will serve files that are created in the /var/www/html/ directory.

cd into that directory and create an index.html file:

cd /var/www/html

sudo touch index.html

To edit this file we will have to use a command-line editor such as Vim, because we have no graphical user interface, therefore no VSCode etc!

sudo vi index.html

If you haven't used Vim before, you need to toggle insert/command mode using the key i to enter Insert mode and the Escape key to enter command mode.

Once in insert mode, create your HTML file, and when done, toggle to command mode and type :wq to save and quit.

One final step: Start the server

We must start the Apache server software by running:

sudo service httpd start

Visit the website

Finally, we should be able to go to our instance's public IP address in the browser and the website should be served!

There are many more ways a server can be configured, and servers can do much more than serve a static web application - most of the time they run an application that performs some logic alongside serving static files, and often they are not used to serve an web application at all.

However, I hope this has been a useful insight into just one way that a server can be configured.

Disconnect from your instance

To end your SSH session, hit they key combination ctrl + D in the terminal. You should be back on your own computer now.

Terminate your instance

To avoid charges, don't forget to terminate your instance when you're done!

You can do this from AWS's EC2 dashboard. Terminating completely destroys the instance, or if you want to come back to it later you can "Stop" it and restart it again in the future. A stopped instance does not accrue charges.

Maybe I Should Have Just Used create-react-app

Harriet — Wed, 08 May 2019 07:57:06 +0000

I've recently been working on improving the performance of a personal side project I've been working on, Conju-gator.com, a little app for practicing verb conjugations in Spanish.

The app is built in React with webpack as the bundling tool, the static files are served from Amazon's S3 with CloudFront content delivery network in front. When developing with React it is common to end up bundling all your source files and dependencies into one single JavaScript file called a bundle. The amount of JavaScript you serve is known to be a probable cause of performance issues, since the bundle can grow quickly as you pull in more and more dependencies.

Originally I didn't give much thought to performance as it seemed like a tiny project, with very few source files and not many dependencies, and I thought performance would be something that I wouldn't need to worry about until later.

However, the site does rely on a fairly large quantity of verb data to produce the training questions, which initially was a JSON file that I imported and bundled along with the source code, and thus could potentially cause performance problems at some point.

I decided to run Chrome's Lighthouse performance audit (a brilliant tool) and get a benchmark for how my site was doing and to my horror it scored 0% on the audit!

What I was doing wrong

The audit highlighted a few key areas where I could make improvements:

Code was not minified
JavaScript payload was excessive
Non-essential CSS was not being deferred
Files were not served with an efficient cache policy
Files were not zipped with gzip or equivalent before serving

The final two points were issues that I needed to fix at the S3/CloudFront level since they are server settings. The solution involved adding metadata to the objects I uploaded to S3 to ensure they were served with a max-age Cache Control header, and that they could be served gzipped. With these two fixes my audit improved about 50%.

The issue of non-essential CSS being loaded too early when it could be deferred I ended up solving with Google Web Font Loader although I also came across other approaches to loading async CSS which may also have been useful. The CSS changes didn't make a big difference in the audit.

Webpack Improvements

The first two issues, however, are the ones I want to talk about as they have to do with bundle size. I was serving a 3000kb JavaScript bundle, and when you think that the recommended size is < 250kb, you can see how off the mark I was.

Firstly, my code was not minified, which was an easy mistake to fix as there is a webpack plugin that'll do the job for you, or if you're using webpack in production mode then minification comes by default!

That's another issue - I wasn't using production mode when building my bundle for production. A single line: mode: "production" in my webpack configuration solved so many problems - it brought the bundle size down considerably by only including the parts of libraries that were needed for production, and also gave me minification for free. Webpack's guide to bundling for production is extremely clear and helpful and I wish I'd read it earlier!

Off the back of more research, I also decided to remove source mapping in production (the webpack guide suggests to keep it, for debugging purposes, but to use a lightweight version). Source mapping maintains a map from your bundled code to your original source files so that line numbers & file names in the console refer to your original files and not the bundle. However I wanted to cut down my bundle as much as possible so removed it completely and will bring it back if needed.

By using Webpack Bundle Analyser I was able to watch as my bundle size decreased, and see where its size was coming from.

When the analyzer showed me that my node_modules were now taking up a reasonable amount of space compared to my source code, and my whole bundle size in production was under 250kb, I was pretty happy.

Finally, I decided to remove the verb data from the bundle and fetch it asynchronously, although I'd already got to about 98% on the audit at this point and although it reduced my bundle size further it didn't give me any Lighthouse performance audit improvements.

Reflections

Looking back, the changes I made were quite straightforward and I feel foolish for not realising how bloated my bundle was in the first place. However, at the time it took me some solid hours of work to work through all my problems, and figure out the best solutions.

At one point, I thought "I wish I'd just used create-react-app in the first place!" since CRA will provide default webpack configurations which would have surely been optimised for production and included all the things I'd originally omitted, plus more.

However, the CRA webpack configuration is about 400+ lines long, which is one of the reasons I always shy away from using CRA in the first place. I like to know what my configuration is doing and be able to change it if I need to, and I've traditionally found the configuration of CRA apps hard to debug and maintain.

What are your thoughts? Do you prefer an opinionated/optimised configuration at the expense of customisability?

Have you had an experience of optimising performance on a single page React app?

Am I missing any more performance wins?

🙌

From JavaScript to Ruby: A few of my favourite features

Harriet — Fri, 01 Feb 2019 12:47:22 +0000

Coming from JavaScript to Ruby, I was excited to learn the language that promised to be "friendly to developers" and "designed for developer happiness". I discovered a language which, like JS, is dynamic, object oriented and general purpose. Like JS, it also offers lots of ways to do the same thing, enabling the developer to have a fair amount of stylistic autonomy (if you like that kind of thing).

Ruby has a fairly low learning curve, since it seems to have been designed with lots of sensibly-named, consistent and easy to use methods, and it also has no concept of asynchronicity out of the box, making code easier to reason about than JavaScript.

Here are a few things I've noticed which I think are pretty neat about Ruby, in comparison to JavaScript!

Being able to check memory location

Unlike in JavaScript, Ruby lets you inspect the location in memory of a value with the object_id method:

For example, if we look at the object ID of 2 hashes:

a = {name: 'Harriet'}
b = {name: 'Heather'}
puts a.object_id # 6478367
puts b.object_id # 6471222

Those numbers are memory addresses. The actual addresses aren't that useful but it might help to see when you're dealing with two references to the same location in memory, or references to separate locations in memory.

I've never used this in a practical sense, but it was helpful when I wanted to explore the difference in how Strings and Symbols work in Ruby (see next section). There's no way in JavaScript to inspect where items live in memory, which has been annoying when I've been trying to demonstrate how JavaScript passes objects by reference, and primitives by value.

Symbols

In JavaScript, you have a few of ways of creating a string, the first two here being the most common:

let a = 'Hello world'
let b = "Hello world" // Functionally no different to using single quotes
let b = new String('Hello world') // Creates a String object
let c = `Hello world` // ES6 String Literal

In Ruby, there are also a few options:

a = 'Hello world'
b = "Hello world" # Double quotes allow for string interpolation & escape characters
c = String.new('Hello world')
d = String('Hello world')

In Ruby, by default, all Strings are types of String Objects, and as Objects they occupy different places in memory, even if the contents of two or more strings is the same. Potentially a bit wasteful, storing the exact same information twice over!

You can check this by looking at the object ID of 2 identical strings:

a = 'Hello world'
b = 'Hello world'
puts a.object_id # 6478367
puts b.object_id # 6471222

That’s where Symbols come in. A Symbol is created with a : at the beginning and means that any time the Symbol is used, it will reference the same value.

a = :hello
b = :hello
puts a.object_id # 1111708
puts b.object_id # 1111708

This works great for single words, but you can even turn a longer string into a Symbol and increase efficiency with the .to_sym method:

a = 'Hello world'.to_sym
b = 'Hello world'.to_sym
puts a.object_id # 92880
puts b.object_id # 92880

I use symbols over strings wherever I can, not just when I know a value will be used again in a program! Why not, when it’s easy to do and there’s nothing to lose?

Simple loops ➰

In JavaScript, sometimes you just want to loop a set number of times. You don't care about the start point or the end point, as long as your code executes n times. However, you're forced to explicitly construct the conditions for iteration yourself, starting with i = 0 and defining when you want the loop to end:

for (let i = 0; i < 10; i++) {
 // do stuff
}

In Ruby, you can simply do:

10.times do
  # do stuff
end

It's a simple, less imperative way of executing code a set number of times.

Functions are stricter about arguments

I like that in Ruby, you get an error if you give a function the wrong number of arguments. It just speeds up the process of debugging your code.

def greet(name)
  puts "Hello, #{name}"
end

greet()

# wrong number of arguments (given 0, expected 1)

You can also name your parameters, and if they're not passed, or you pass something unexpected, you'll get an error:

def greet(name:, city:)
  puts "Hello, #{name} from #{city}"
end

greet(name: 'Harriet', city: 'Manchester')

greet(name: 'Harriet') # missing keyword: city

greet(name: 'Harriet', city: 'Mancheseter', age: 27) # unknown keyword: age

No function call parentheses

In JavaScript, you must use parentheses when calling a function, for example add(1, 2).

In Ruby, parentheses are generally optional, which can sometimes lead to Ruby that looks very natural language-y and easy to read. For example, a testing library can provide a to method which, when used without parentheses, reads like this:

expect(result).to be_null

Although it can get a bit confusing if you've got multiple arguments. For example, is 5 the second argument to bar, or the second argument to foo? Without brackets it's not clear:

def foo(a, b)
  puts "in foo #{a}, #{b}"
end

def bar(a)
 12 + a
end

foo bar 55, 5 # wrong number of arguments (given 2, expected 1)

foo bar(55), 5 # Correct - 5 is the second argument to foo

Calling a function without parentheses also means we can do something like this:

def greet(name = 'Harriet')
  puts "Hello, #{name}"
end

greet

See how just referring to the greet method actually invokes it with no arguments? This is how Ruby implements getter methods on objects. When you call person.name for example, name is actually a method on that object, which retrieves the name instance varaible. It's not simply an object property like in JavaScript.

One effect of parentheses-less method calls means we can't pass methods around as values, like we can in JavaScript. In JavaScript, we can do this:

function greet(name) {
  console.log(`Hello, ${name}`);
}

const welcomer = greet;

welcomer('Harriet');

But in Ruby, trying to pass a reference to the method actually invokes it! So we end up with:

def greet(name = 'Harriet')
  puts "Hello, #{name}"
end

welcome = greet # This line actually executes the greet function

welcome "Paula" # undefined method `welcome' for main:Object

Just one way to create Classes

In JavaScript there is not really a concept of true classes, at least not in the way people from truly Object Oriented languages would expect them to be. Instead we have a the prototype chain, and at least 4 different ways of creating objects with shared methods and behaviour. This is super confusing, so I really like that Ruby just offers the one way to do it!

Creating class methods, class variables, instance methods and instance variables is much more straightforward in Ruby:

class Person
  attr_reader :name, :title

  # Class variables
  @@legs = 2
  @@arms = 2
  @@diet = 'omnivore'

  def initialize(name, title)
    # @name and @title are instance variables
    @name = name
    @title = title
  end

  # Instance method
  def greet
    puts "Good day, #{title} #{name}!"
  end

  # Class method
  def self.describe
    puts "A person is a #{@@legs}-legged, #{@@arms}-armed #{@@diet}"
  end
end

jerry = Person.new('Jerry Jones', 'Mr')

jerry.greet

Person.describe

Implicit returns

In Ruby, the return statement is optional, or can be used to return early from a function. If you omit it, the function will return the last evaluation.

def double(nums)
  nums.map{ |n| n * 2 }
end

Metaprogramming

This one's quite a big a topic and I don't know it that well so I'm only going to touch on it briefly. Metaprogramming means a program being able to modify itself at runtime, based on the state of the program at that time.

Rails uses Metaprogramming to allow us to do something like this:

Book.find_by_ISBN("19742084746")

You defined the Book class when you set up your Models, but nowhere did you defined the find_by_ISBN method. Defining a find_by_x for all your columns would be really tiresome; it's no wonder the Rails framework doesn't want to make you go to all that work. But Rails itself didn't add that method for you, either. How would Rails magically know what your Book instances needed a find_by_ISBN method?

Instead, when Rails sees you trying to use the find_by_ISBN method it will extract the ISBN part and attempt to match it to a column in the database, and if successful, will execute some code to find your item based on the ISBN column, responding as though find_by_ISBN were an actual method that had been defined on Book instances.

This is just one example of what Metaprogramming can do!

Personally I think it's pretty cool, and once you know it exists, you begin seeing it "in the wild" all over the place in Ruby. It's the foundation stone for being able to create DSLs (Domain Specific Languages) like Rails and makes Ruby extremely flexible.