DEV Community: Andrew Davis

What is an HMAC?

Andrew Davis — Fri, 17 Jan 2020 13:40:00 +0000

I recently was asked to explain some code that used an HMAC signature for authenticating/verifying a message. While I understood that HMACs are used for message authenticity, I did not fully grasp the implementation. Here is the information I have discovered through more research.

MAC stands for "message authentication code". Essentially, it is a way to generate a code to be submitted with a message that verifies the sender of the message and also verifies the message has not been modified. HMAC stands for "hash-based message authentication code". It uses a cryptographic hashing algorithm to generate the MAC. An HMAC algorithm works by hashing a message along with a secret key. The resulting hash is called a signature or digest. The message is then transferred to a recipient along with the signature. If the recipient has the secret key, they can hash the message with the same algorithm and verify the resulting signature matches the one sent with the message. As a result, the message is simultaneously authenticated and its integrity verified.

We can try to better understand an HMAC through an example. Let us imagine that you have a doctor's office application that needs to send an HTTP request to a secure API that contains medical history for patients. The message you want to send to the API is a JSON packet containing the results of the latest visit to the doctor by the patient.

{
    "first_name": "Andrew",
    "last_name": "Davis",
    "visit": "2020-01-12 09:56:12",
    "symptoms": [
        "sore throat",
        "runny nose",
        "sneezing",
        "headache"
    ],
    "diagnosis": "Common Cold",
    "suggested_treatment": [
        "drink water",
        "rest",
        "consume ibuprofen twice a day"
    ]
}

The API has provided you with a secret key that might look something like this 13441c78e4440683a9e32aa7e4ee39fd3544b87968cc5a30f621f7e32029b2f5 and says you must use it to create a SHA-256 HMAC signature to pass along in the Authorization HTTP header for requests.

Our programming language of choice is PHP. Thankfully, PHP already has functions for generating HMAC signatures. To see all the supported algorithms, you can run hash_hmac_algos() and PHP will return an array of all the hash types supported. Here is the list for PHP 7.4 on my machine.

>>> hash_hmac_algos();
=> [              
  "md2",       
  "md4",       
  "md5",       
  "sha1",      
  "sha224",    
  "sha256",    
  "sha384",    
  "sha512/224",
  "sha512/256",
  "sha512",    
  "sha3-224",  
  "sha3-256",  
  "sha3-384",  
  "sha3-512",  
  "ripemd128", 
  "ripemd160", 
  "ripemd256", 
  "ripemd320", 
  "whirlpool", 
  "tiger128,3",
  "tiger160,3",
  "tiger192,3",
  "tiger128,4",
  "tiger160,4",
  "tiger192,4",
  "snefru",    
  "snefru256", 
  "gost",      
  "gost-crypto"
  "haval128,3",
  "haval160,3",
  "haval192,3",
  "haval224,3",
  "haval256,3",
  "haval128,4",
  "haval160,4",
  "haval192,4",
  "haval224,4",
  "haval256,4",
  "haval128,5",
  "haval160,5",
  "haval192,5",
  "haval224,5",
  "haval256,5",
]

We see that SHA-256 is supported by PHP. SHA-256 is a common hashing algorithm created by the Unites States' NSA which is often used with HMAC algorithms.

Now, we need to create the actual HMAC signature of our message. To do so, we can use PHP's hash_hmac function. The first parameter is the algorithm, the second parameter is the message and the third parameter is the secret key.

>>> hash_hmac('sha256',\
... '{"first_name": "Andrew","last_name": "Davis","visit": "2020-01-12 09:56:12","symptoms": ["sore throat","runny nose","sneezing","headache"],"diagnosis": "Common Cold","suggested_treatment": ["drink water","rest","consume ibuprofen twice a day"]}',\
... '13441c78e4440683a9e32aa7e4ee39fd3544b87968cc5a30f621f7e32029b2f5');
=> "0def47f4878406abf864da0d34f9e3627653317e7c77da8230e0f65b52c09479"

PHP responds with a message digest/signature: 0def47f4878406abf864da0d34f9e3627653317e7c77da8230e0f65b52c09479. We can submit the JSON message to the API along with the signature. The API will then take the message and it will create an HMAC signature using SHA-256 and its secret key. If its resulting HMAC signature matches the HMAC you provided, then it accepts the message for processing.

The API exchange highlights the two main benefits of HMAC.

First, since an HMAC requires a secret key, both the sender and recipient have to have the same secret key for HMAC verification to work. As a result, the HMAC acts similarly to a password. It is very important to keep the secret key safe. Since it is used like a password, it needs to be saved in encrypted storage. It is also important to use a sufficiently long secret key that cannot be brute forced by password breakers.

Second, an HMAC is based not only on the secret key, but also on the message itself. If two HMACs are created with the same secret key, but different messages, the resulting codes will be different. When the API sees that its HMAC matches the HMAC in the request, it knows the request was not modified during transport. Since hackers will try to modify messages in between clients and servers, HMAC signatures can be a powerful tool to verify authenticity of requests.

Hopefully this explanation has shed some light on HMAC signatures. Unfortunately, I do not have the math and cryptography experience to explain how the HMAC algorithm works to create the signature. Maybe someone else in the DEV Community can provide a layman's explanation of the computation.

What is your experience with HMAC and where have you seen it used it effectively?

Let's Talk Windows vs macOS

Andrew Davis — Fri, 27 Dec 2019 14:38:55 +0000

When I started writing websites professionally, my work computer ran Windows so I had to learn to set up PHP and Laravel using IIS (an unpleasant experience). However, my personal computer was a 2015 MacBook Pro and I really liked the Bash terminal, Homebrew and Valet. When I moved to my current employer, I had the option to have an HP laptop or a MacBook so I naturally chose the MacBook. I became a full Apple convert. It was really easy to work on web development because macOS is so similar to Linux. Homebrew makes package installation really easy. PHP runs really well on macOS because of the Unix environment. I was ready to ride in the sunset as a satisfied Apple customer.

Well guess what? I am writing this blog post on Windows right now.

You do not hear many stories about users going back to Windows after switching to Mac. So what happened?

Well first off, I am not a total Microsoft convert. I still use a MacBook for work and am generally happy with it. I recently got an upgrade to a 2019 MacBook from my employer and it works well. I want my escape key back (my pinky shudders on the cold Touch Bar every time I exit Vim), but overall it is a solid laptop.

At home, my personal computer is a custom assembled PC running Windows. This year my 2015 MacBook started showing its age. I also was running out of space on the hard drive. 256GB does not go very far these days. I found that I was not using the laptop part of my MacBook very much. Generally, I just used it on a desk and it never left the house. When I looked at the prices of a new Mac, an equivalent replacement to what I had would have cost me $2400 (USD) which was not worth it to me because I use my personal computer mainly for blogging, streaming and small side projects. So it was then that I decided to try my hand at building a PC. I ended up with a desktop PC that has better specs than a 15" MacBook and only cost me $1300.

My custom PC has a six core Ryzen 5 processor with 16GB of RAM and a Nvidia RTX video card running off of two SSDs. It is very fast, apps load immediately and basic things like web browsing feel incredibly snappy.

I also love that I can upgrade the hardware on my PC whenever I want. Do I want to upgrade to 32GB of RAM? Easy. Add another hard drive? Piece of cake. MacBooks will never offer this level of customization.

Unfortunately, there are always downsides. The number one for me is the Windows filesystem. I really miss using a POSIX operating system. Microsoft is working on it with the Windows Subsystem for Linux which I have been using some. I have run into a lot of bugs with it though and Microsoft is having to rewrite it for WSL 2 which supposedly will offer more stability. I am hopeful that WSL 2 will solve my complaints. We will see when it comes out of beta in 2020.

I also am struggling to find equivalent apps on Windows for what I use on Mac. There is no Alfred or iTerm or Sequel Pro on Windows. While I have found some alternative options on Windows, they are not quite as good as the Mac apps.

If you asked me if you should get a Mac or PC, I would say:

On a PC, you can have faster hardware that can be upgraded for much cheaper than a Mac. However, web development is still easier on a Mac because of the Unix underbelly and app ecosystem. I am crossing my fingers that Windows will improve with projects like WSL 2 and the new Windows Terminal, but only time will tell if they become as good as what Macs offer.

Ultimately, I am happy with my PC right now. I like to play video games so that is another plus in the PC column. If I was having to use it full time for PHP development though, I doubt I would be as productive as I am on a Mac.

What do you like to use for web development? Have you recently switched to Windows from a Mac? What are your favorite apps?

P.S. I have tried to use a full Linux distro (Ubuntu), but had basic issues with it like audio not working, so left it out of the comparison. I may attempt to use a Linux distro again in the future.

Introduction to PHP 7.4 Preload

Andrew Davis — Sat, 10 Aug 2019 18:16:02 +0000

PHP 7.4 is the latest version of the PHP programming language that will launch later this year. PHP 7.4 recently went into beta, so I started exploring the new features. One feature showing a lot of promise is called Opcache Preload. Using preload you can set PHP to load certain PHP files into memory on startup. Doing so reduces the amount of files PHP has to compile when receiving an HTTP request which improves performance. Before exploring preload, let us look at how opcache works in the PHP runtime.

Opcache Is Your Friend

In PHP, the term PHP is used for the programming language. However, the runtime that executes PHP code is called the Zend Engine. When the Zend Engine receives a request for a PHP file, it actually compiles the PHP file into a list of operation codes, hence the term opcode. The opcache Zend Engine extension stores those opcodes in memory after a PHP file is compiled, so that the PHP file does not have to be compiled again in subsequent requests for the file.

The opcache.preload directive allows a PHP programmer to instruct the opcache to cache the opcodes for a set of PHP files when the Zend Engine is activated. The preload option should be pointed to a PHP file that runs the function opcache_compile_file on the files you want preloaded.

Enabling Preload

To test preloading, I set up an example project that uses Slim 3 for routing.

To use preloading, you have to install PHP 7.4. I am using Ubuntu WSL on Windows with the ondrej/php PPA activated. Since I'm using Ondrej's PPA, I can run sudo apt install php7.4-fpm in bash to get the latest 7.4 beta on my system. I use the FPM installation with Nginx as my web server and a host entry called preloading.local pointed at my project. After installation, I opened the FPM php.ini file (located at /etc/php/7.4/fpm/php.ini on my system) and changed the opcache.preload option to the path to my preload.php file in my project.

The preload.php file uses a modified preload function based on Dmitry Stogov's example in the Preload RFC. I have all the Slim dependencies loaded in a vendor folder using Composer. I then use the preload function in my preload.php on all the Composer source files I need to run the Slim application. After restarting PHP FPM (sudo service php7.4-fpm restart), I can open preloading.local in my browser and see my application run successfully.

My Thoughts on Preload

I like that I did not have to use the Composer autoloader for my framework. Since all of the framework code was preloaded, autoloading is unnecessary. I do hope that Composer will generate an automatic preload.php file, like the autoload.php file, so that vendor dependencies can be preloaded easily.
The performance gain is miniscule for a small project like this one, however, I can see preloading being a great way to speed up large frameworks like Symfony, Laravel and especially Magento.
Preloading will be tricky to implement if you have multiple PHP projects on the same server, since you can only set one file to be preloaded.
I had an issue where preloading would not load the functions.php file in the nikic/fast-route module because it was wrapped in an if statement. I had to set up a patch file to manually load those functions without the if statement. It is probably because 7.4 is still in beta or my preload.php needs to be improved.
I do not think many PHP developers will use preload in development environments. I generally am reluctant to use features in production that are unused in dev because of the potential for unseen issues to arise on production servers. However, the risk is pretty low for preloading, so it will still be worth it to have it enabled.

All in all, I'm really looking forward to the performance gains from preloading. PHP is already one of the fastest scripting languages on the web and I am happy that the core team continues to push PHP into faster territories.

Next, I want to try arrow functions, the foreign function interface and typed properties. There are a lot of great features being added to PHP soon!

How to Write a Magento 2 Console Command

Andrew Davis — Sun, 09 Jun 2019 20:54:32 +0000

In Magento 2, the platform introduced a new command line interface called bin/magento that is included in every installation. Most Magento developers are familiar with it because it makes it very easy to peform common tasks on the install like clear cache and install a database. What you might not know is that you can add custom commands to the Magento CLI to allow you to run your own code. I have used this feature to create custom cron jobs that can be executed by the OS cron instead of Magento's cron system. Magento's CLI is based on the Symfony Console package which is commonly used in the PHP ecosystem. Laravel bases their artisan command on Symfony Console, for example. In Magento 2, you can create custom Symfony Console commands and include them in Magento's CLI. To help other developers get started with console commands, I want to provide this quick tutorial.

I am assuming you have a Magento 2 installation ready to use for development. To learn how to set up Magento, check out Magento's devdocs post for installation instructions. I used Magento Community 2.3.1 for this tutorial.

1. Make a Module

We need to create our own custom module in app/code. I am calling my module Restoreddev_Console. For every module we need a registration.php and module.xml file.

app/code/Restoreddev/Console/registration.php

<?php

use \Magento\Framework\Component\ComponentRegistrar;

ComponentRegistrar::register(ComponentRegistrar::MODULE, 'Restoreddev_Console', __DIR__);

app/code/Restoreddev/Console/etc/module.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Module/etc/module.xsd">
    <module name="Restoreddev_Console" />
</config>

Now you can run php bin/magento setup:upgrade to install your custom module into Magento's database.

2. Create Console Command Class

Next, we can create a class that implements Symfony Command that will be run when our command is executed. I want to create a custom command that clears the generated directory, similar to how cache:flush works. Add this class to your module:

app/code/Restoreddev/Console/Console/Command/GeneratedFlushCommand.php

<?php

namespace Restoreddev\Console\Console\Command;

use Symfony\Component\Console\Command\Command;
use Magento\Framework\Filesystem\DirectoryList;
use Magento\Framework\Filesystem\Io\File as FileIo;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

class GeneratedFlushCommand extends Command
{
    protected $fileIo;
    protected $directoryList;

    public function __construct(
        FileIo $fileIo,
        DirectoryList $directoryList
    ) {
        parent::__construct();

        $this->fileIo = $fileIo;
        $this->directoryList = $directoryList;
    }

    protected function configure()
    {
        $this->setName('generated:flush');
        $this->setDescription('Deletes code in generated folder');

        parent::configure();
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        $path = $this->directoryList->getPath('generated');
        $dirs = glob("$path/*", GLOB_ONLYDIR);
        foreach ($dirs as $dir) {
            $this->fileIo->rmdir($dir, $recursive = true);
        }

        $output->writeln('Generated folder successfully flushed');
    }
}

There is a lot happening in this class, so we need to analyze it in sections.

You will see that the class extends Symfony\Component\Console\Command\Command. The Symfony parent class provides the interface for our class to be included in the command list.
In the __construct method, we can inject Magento dependencies like other Magento classes. However, you do have to run parent::__construct(); for the class to be implemented properly.
The configure method allows you to define the command name and description that appears in the CLI menu.
The execute method contains the code that will run when the CLI command is executed. The method receives two objects: $input (contains any console arguments) and $output (allows you to send text to the terminal). I am using Magento's classes to get the generated directory and to delete any directories inside of it. At the end, I am using $output->writeln() to send a success message to the user.

3. Register Command in Magento

Finally, we need to register the command in Magento's dependency injection system. Create the following di.xml file in your module.

app/code/Restoreddev/Console/etc/di.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
    <type name="Magento\Framework\Console\CommandListInterface">
        <arguments>
            <argument name="commands" xsi:type="array">
                <item name="generatedFlushCommand" xsi:type="object">Restoreddev\Console\Console\Command\GeneratedFlushCommand</item>
            </argument>
        </arguments>
    </type>
</config>

In the di.xml, we are targeting the CommandListInterface and adding an additional command to its arguments. Now you can flush Magento's cache (php bin/magento cache:flush) and run php bin/magento. You will see a new command called generated:flush. If you execute it php bin/magento generated:flush, then you will see all subdirectories in the generated folder have been deleted.

I hope this quick tutorial has helped you learn about a neat new feature in Magento 2. If you would like to see another specific Magento tutorial, let me know in the comments!

GraphQL Server Primer

Andrew Davis — Mon, 25 Mar 2019 12:59:17 +0000

GraphQL is one of the hottest topics in web development right now. I do a lot of work with Magento and they recently announced a whole new GraphQL API to be used for front end development. To learn GraphQL, I built an experimental Magento frontend using Next.js. After that project, I still did not understand how to build a GraphQL server. As a result, I decided to build an example GraphQL server using Node.js. The original GraphQL specification was written in JavaScript so it is a good language to use when learning GraphQL. I created a GitHub repository called graphql-nodejs-example if you want to view the whole project. In this post, I want to discuss a few concepts about GraphQL that really helped me understand how it works.

A GraphQL Server Has Only One Endpoint

In REST APIs, it is common to have multiple URLs for a single resource. You might have several endpoints for loading and creating data like /tasks, tasks/1, /tasks/create. In GraphQL, your server runs only a single endpoint, usually at the root / or at /graphql. When submitting a query to a GraphQL server, you explicitly set the resource you want in the request body, so the server can decide what values to return.

GraphQL Is About Types

In a GraphQL API, you define what resources you have using a type language. GraphQL supports five scalar types that you can use to compose more complex object types. The five scalar types are: Int, Float, String, Boolean and ID. To create a resource, you build an object type for it. I wanted to emulate a forum so I created three resources: User, Thread and Comment. In GraphQL types, those resources look like this:

type User {
  id: Int!
  userName: String!
  firstName: String
  lastName: String
}

type Thread {
  id: Int!
  name: String!
  description: String!
  user: User!
  comments: [Comment]
}

type Comment {
  id: Int!
  description: String!
  user: User!
}

You can see that you can create an object type using the type keyword followed by a name. In curly braces, you define the properties of the object by writing the name of the property followed by a colon and the type. An exclamation point ! after the property indicates that the value cannot be null.

You can also use custom types in other custom types. The Thread type has a user and comments property that reference the other two types I created. Brackets around the type name like [Comment] indicate that the property is an array.

When writing a server, where do you put those types? I put them all in a file called schema.graphql and used the Apollo Server helper gql to import that file into my server.

Requests Are Handled By a Query and Mutation Type

In GraphQL, there are two types of requests you can send to a GraphQL server: query and mutation. A query is used to retrieve data and a mutation is used to perform actions on data, like creating or updating. In your server schema, you define a query object type and a mutation object type, like so:

type Query {
  thread(id: Int!): Thread
  threads: [Thread]
}

type Mutation {
  createThread(name: String!, description: String!, userId: Int!): Thread
  createComment(userId: Int!, threadId: Int!, description: String!): Comment
}

You can see in my Query type, I define two ways to retrieve a thread. The threads property returns an array of all threads and the thread(id: ID!) returns a single thread. The parentheses denote arguments that can be passed in the query. Since I marked id as a non-nullable Int, to retrieve a single thread you have to pass in the id of a thread in your GraphQL request.

In the Mutation type, there are two properties for creating a thread and creating a comment. Each operation requires a set of values for creating the resource and each returns the newly created resource.

Resolving Your Queries and Mutations

After defining the schema, how do you implement the logic to load the resources from a data source? You use resolvers! Resolvers are similar to controllers in a REST API. For each Query and Mutation property, you create a JavaScript function that accepts arguments and performs the operation on the resource to load data or change it.

I used the Apollo Server library to build my GraphQL API. The library allows you to write your schema, import it and pass in a resolver object that will handle all of the requests.

My Apollo Server setup looks like this:

const fs = require('fs');
const { ApolloServer, gql } = require('apollo-server');
const schema = fs.readFileSync(__dirname.concat('/schema.graphql'), 'utf8');
const typeDefs = gql(schema);
const resolvers = require('./resolvers');

const server = new ApolloServer({ typeDefs, resolvers });
server.listen().then(({ url }) => {
  console.log(`Server ready at ${url}`)
});

All I need for my Apollo Server instance is to pass in my schema and resolvers and it will start a node server that I can query.

My resolvers file just exports a JavaScript object with a Query and Mutation property that holds references to functions for each property defined in my schema:

const threads = require('./threads');
const comments = require('./comments');

module.exports = {
  Query: {
    threads: threads.all,
    thread: threads.findOne,
  },
  Mutation: {
    createThread: threads.create,
    createComment: comments.create,
  }
};

The threads and comments imports each return an object of functions that can be passed into the resolver object.

So what does a resolver function look like? Here is a query resolver that returns all Thread types from a database:

exports.all = async function () {
  const threads = await db.Thread.query().eager('[comments.[user], user]');

  return threads;
};

The function queries a database for the data needed to resolve the Thread type and then Apollo Server pulls out the values it needs and returns it to the client that requested all the threads.

A mutation is very similar:

exports.create = async function (parent, args) {
  const thread = await db.Thread.query().eager('user').insertAndFetch({
    userId,
    name,
    description,
  } = args);
  thread.comments = [];

  return thread;
};

The second parameter a resolver function receives is all the arguments passed from the request. I use those arguments to create a new thread in the database and then return the data for Apollo Server to pass back to the client.

Querying the Server

There are many ways to test a GraphQL API. I like to use Insomnia. In development mode, Apollo Server will return your schema so that Insomnia can read it, allowing you to auto-complete queries for the API.

Here is an example query you can send to the server with the above schema:

query getThreads {
  threads {
    id
    name
    description
    user {
      id
      firstName
      lastName
      userName
    }
    comments {
      id
      description
      user {
        id
        userName
      }
    }
  }
}

In the query, I am requesting the threads property of the query object and passing in the attributes I want for each thread. Dynamic queries are what make GraphQL so good, because you can ask for as little or as much data as the API can provide. The following json represents what the API server returns to the client:

{
  "data": {
    "threads": [
      {
        "id": 1,
        "name": "Thread 1",
        "description": "This is the first thread",
        "user": {
          "id": 1,
          "firstName": "Test",
          "lastName": "User 1",
          "userName": "testuser1"
        },
        "comments": [
          {
            "id": 1,
            "description": "This is a comment on the first thread",
            "user": {
              "id": 2,
              "userName": "testuser2"
            }
          },
          {
            "id": 3,
            "description": "Another comment",
            "user": {
              "id": 1,
              "userName": "testuser1"
            }
          }
        ]
      },
      {
        "id": 2,
        "name": "Thread 2",
        "description": "This is the second thread",
        "user": {
          "id": 2,
          "firstName": "Test",
          "lastName": "User 2",
          "userName": "testuser2"
        },
        "comments": [
          {
            "id": 2,
            "description": "This is a comment on the second thread",
            "user": {
              "id": 1,
              "userName": "testuser1"
            }
          }
        ]
      }
    ]
  }
}

A mutation query for creating a thread looks like this:

mutation createThread {
  createThread(
    userId: 1,
    name: "A new thread",
    description: "This is a description"
  ) {
    id
    name
    description
    user {
      id
      firstName
      lastName
      userName
    }
  }
}

I am calling the createThread property of the mutation type and passing in the required arguments. It returns to me the resource that it just created:

{
  "data": {
    "createThread": {
      "id": 7,
      "name": "A new thread",
      "description": "This is a description",
      "user": {
        "id": 1,
        "firstName": "Test",
        "lastName": "User 1",
        "userName": "testuser1"
      }
    }
  }
}

Some General Tips and Tricks

Here are a few more general tips for starting a GraphQL server project:

If you use a database, use a NoSQL database like MongoDB or a SQL database with an ORM that supports eager loading. GraphQL types often use nested objects so it can be difficult to write plain SQL queries and map the data for your responses. I used the Objection.js ORM with sqlite and that made my database code much simpler.
GraphQL naturally validates the data types of any arguments passed into your API, but it only validates the type. By default, a string type can be empty or any length. I used the validation features of Objection.js to prevent the use of empty strings in mutations.
The ID scalar type converts ID values to a string. That will work great for some databases, but in my case I was using sqlite with numeric primary keys so I left my ID values as an Int.

Conclusion

I was surprised by how quickly you can build a GraphQL API, especially with the help of libraries like Apollo Server and Objection.js. I really like being able to define my API around types which become natural documentation for your available resources. Not having to set up URL routing or type validation saves a lot of time as well. The benefits of GraphQL for building API clients have been touted widely, but I think there are some real advantages for the server too.

I hope this article helped you understand GraphQL servers even better. Leave a comment if you have any questions or thoughts about this post!

Rewriting My Personal Site with Hugo

Andrew Davis — Mon, 04 Mar 2019 15:33:50 +0000

A couple months ago I created my personal website called andrewdavis.me. I built it as a custom server side app in Go with MySQL holding the content. I wrote about the experience in Building My Personal Site with Go and TailwindCSS. I enjoyed the project, however, I recently wanted to make some SEO improvements and decided maintaining a custom app was more work than I wanted. I started looking at different static site generators and found Hugo. It is a really fast site generator that uses Go templates for HTML and Markdown for content. Hugo is very flexible, but also easy to use so I was able to move my site to it in a single day! In fact, I open sourced the code so you can see it all here on GitHub. Now, my site is served from HTML files on GitHub Pages, making it much faster at rendering. Here are a few reasons for why I liked Hugo so much and why you may enjoy using it for your next project.

Easy to Install and Start a Project

Hugo is a simple binary that can be installed on any OS (I used Homebrew on MacOS brew install hugo). To start a new project, open a terminal and run hugo new site blog. Hugo will create a folder called blog and generate a project skeleton so you can start adding content. To start building templates, add a _default folder in the layouts directory. In there you can create a new file called baseof.html which will become the base template for all your pages. After that you can create an index.html, list.html and single.html template. The index template will be used for the homepage, the list template for any list pages and the single template for content pages.

Next, you can start adding Markdown in the content folder. You can create Markdown files like post.md or post/index.md which will create different paths: /post.html and /post/ respectively. Each markdown file must have frontmatter, but it can be written in YAML, TOML, or JSON. I wrote mine in TOML, which makes a page look like this:

+++
title = "My Blog Post"
date = "2019-03-01"
+++
The content of my blog post goes here.

To build the pages, you can run hugo server in your terminal and it will start a server for you to see the site at localhost:1313. The server will also automatically reload anytime you make changes to your content. Hugo takes the Markdown pages' content and builds them into individual HTML pages using the Go templates. In the template, the Markdown file will be injected using predefined variables so you can use {{ .Title}} and {{ .Content }} to indicate where the Markdown data should appear in the template.

Of course, there is a lot more to templating and content, which you can find in the docs. Hugo is very configurable and can do as little or as much as you need.

Automatic Code Highlighting

On the previous iteration of my site, I used the Prism.js JavaScript library to enable code highlighting. It worked well, but all the styles are added client side, which is unnecessary work for the browser. Hugo uses a Go syntax highlighter called Chroma that will add the CSS to the HTML in the build process. To have it run on Markdown codeblocks, just add pygmentsCodefences = true in the config.toml in the root of your project. The config.toml file is the main configuration file used by Hugo when building the site.

Integrated CSS Processing

I like to use PostCSS for my stylesheets and Hugo works with it easily. I just created a package.json with postcss-cli installed and my PostCSS plugins. Then, I created a postcss.config.js file (not required by Hugo, just what is necessary for PostCSS). Afterwards, I created a CSS file in the assets folder and then imported it into my baseof.html template using:

{{ $css := resources.Get "site.v01.css" }}
{{ $style := $css | resources.PostCSS }}

<link rel="stylesheet" type="text/css" href="{{ $style.Permalink }}">

Now, when I build with Hugo, it will automatically compile my stylesheet and link to it in my base template.

Easy Deployment

Hugo has multiple different ways to deploy a site. When you run hugo in the root of your project, it will build the files in a public folder. I just followed their guide for deploying to GitHub and created a publish.sh script that takes the contents of the public folder and commits it to a separate git branch. The deployment branch is then used by GitHub Pages to serve the files.

Conclusion

I really like that Hugo is straight forward and does not require writing JavaScript to build a static site, like many other generators. Plus, I can now easily update templates and add more features using frontmatter variables without having to write much code. Generated HTML files allow me to not maintain a server setup, decreasing the amount of maintenance time to keep my site going. All in all, I count it as a successful move and plan to use Hugo in the future!

Creating a Magento Theme with Next.js and GraphQL

Andrew Davis — Tue, 12 Feb 2019 13:43:38 +0000

At the end of 2018, Magento released the 2.3.0 version of their e-commerce framework with a new GraphQL API. Traditionally, Magento themes are created with PHP templates, similar to WordPress. However, with the new GraphQL API and PWA Studio, new themes can be created in JavaScript frontend frameworks like React and Vue. To experiment with the API, I have created an open source theme called Basil using React, Next.js and Apollo. Here is a preview of the theme with Magento’s sample catalog. The theme is very experimental so it needs code cleanup and responsive styling, but it allowed me to explore how a theme could be created this way. It does not use any PWA Studio components because I wanted to see what it would take to build a frontend from scratch. Here are my thoughts and impressions about the whole process.

Deciding on a Framework

There are a ton of options for frontend JavaScript frameworks so I tried to narrow down my choices using two requirements: it must have easy server side rendering for SEO (crucial for e-commerce) and it must support GraphQL. In the end, I tried three frameworks Nuxt.js (Vue), Ember with Fastboot and Next.js (React). Ember actually had my favorite development experience. The code generation CLI and easy templating were a big draw, however, its GraphQL support was wanting. The best Ember GraphQL plugin could not support loading states. Nuxt.js’s GraphQL integration was lacking as well. It was too difficult to get the GraphQL data to load when rendering on the server. In the end, I chose Next.js because of Apollo’s excellent React integration and because Next’s server rendering is top notch. Next will render the whole React component tree with the necessary GraphQL data before outputting the HTML on the server.

I like GraphQL

Basil was the first time I have seriously used GraphQL and the development experience was great. The queries are really easy to write and I like that you can store them in files for reuse. It really simplifies retrieving data because you can write queries in a GraphQL client like Insomnia and the fields will be auto-populated as you type based on the schema returned from Magento. In React, Apollo makes GraphQL easy by supplying a Query component that you can use to wrap your JSX and your template will be automatically supplied the query results. Here is an example:

<Query query={PRODUCT_QUERY}>
    {(response) => {
        if (response.error) return <div>Error</div>;
        if (response.loading) return <div>Loading...</div>;

        let product = response.data.product;
        return (
            <div class="product__name">{product.name}</div>
        );
    }}
</Query>

Routing was a Pain

Magento has a unique routing system because any path underneath the root could map to a category, product or content page. For example, opening /backpack on a Magento site could take you to a product named Backpack, a category for Backpacks or a content page about backpacks. It is all determined by how the catalog is configured in Magento’s database. To allow for this routing on the frontend client, I had to set up a special wildcard route that would accept any URL path and then query Magento’s urlResolver to figure out if the page should display a product, category or content component. Unfortunately, that means two queries have to be run for each catalog page. A framework like Gatsby might be a better option for a small catalog since it can generate a frontend page for each URL in a build step.

I Wish Unit Testing React Was Easier

One of the features of Ember I liked was that it came with a unit testing setup and a framework for performing integration tests. Next and React have plenty of examples of how to set up Jest (a test runner), but there is little documentation about good ways to integration test a component that renders HTTP results. I set up Enzyme to help with component rendering and analyzing, but still have not built out a way to mock the GraphQL server for more extensive tests.

Viability for Magento

Unfortunately, I do not think it is time to make production frontends this way for Magento. Magento’s GraphQL endpoints are lacking and will not be fleshed out until later this year. Currently, to build a cart and checkout, you would have to fallback on the old REST API. I think it would be better to wait until the GraphQL API has better frontend coverage. Even when it does cover the checkout, it will still be missing some of the smaller Magento features like related products and wish lists. Magento is still working on their frontend solution, PWA Studio, as well. When it is feature complete, PWA Studio will be another option as a basis for a JavaScript frontend.

Final Thoughts

The new GraphQL support is exciting because it will allow Magento developers to make modern and unique frontend experiences. However, there is a lot of development to be accomplished before it is production ready. While I will miss the simplicity of PHP templates, the performance and capabilities that JavaScript frameworks provide will make them popular for many merchants. Plus, it will make it much easier to introduce frontend developers to the Magento ecosystem.

Let me know if you have any questions about my experience with making Basil. Also, check out the GitHub repo and the online preview and let me know what you think!

Announcing PHP Apprentice!

Andrew Davis — Fri, 11 Jan 2019 13:42:58 +0000

I am excited to finally announce PHP Apprentice! It is an online, free and open-source book for learning the PHP programming language. The book is a side project I have been building for six months. My goal is to provide a contemporary resource for learning the basics of PHP with a focus on high quality code. PHP has been around for 25 years and has changed a lot since its inception. A lot of the resources for writing good PHP have not aged well or are inapplicable today. I wrote PHP Apprentice to remedy the lack of available websites for learning PHP.

Currently, PHP Apprentice has chapters on the basic building blocks of the language, but in the future I will be adding sections on generating webpages, connecting to databases and advanced techniques like using Composer and autoloading. I am looking for feedback so please leave a comment here or an issue on the repository if you have any thoughts or requests.

Each chapter in PHP Apprentice was written as a Markdown file. I then built an HTML generator in PHP that takes each file and parses it with Parsedown into an HTML chunk and loads them with a template into a full HTML page. The pages are then served by GitHub pages. The build process can be performed using a CLI command so I automated the deployment using Travis CI. Whenever any code is merged into master, Travis will pick it up, re-build all the webpages and push them to another branch on the repository which is used by GitHub Pages.

The frontend is all plain HTML, CSS and JavaScript. I used PostCSS to add file imports and variables. The code highlighting is performed by Prism.js. I wanted the focus of each page to be on the code so the styling is very minimal. The site is responsive, but has room to improve the experience on mobile. I focused on desktop since I recommend writing the code in the chapters while reading the book.

PHP Apprentice was inspired by Elixir School and Go by Example, two sites I admire for their clarity and quality. I hope that PHP Apprentice will become a similarly helpful site for the PHP ecosystem. It is a living book so I will be changing and improving the site over time.

Please check out PHP Apprentice and let me know what you think!

Building My Personal Site with Go and TailwindCSS

Andrew Davis — Mon, 17 Dec 2018 01:57:00 +0000

Recently, I found the domain andrewdavis.me available for purchase. I tried to buy it in the past, but it was already taken. I quickly purchased the domain, but then I had to decide what to put on it. I made up my mind to build a personal website that would showcase my work and writing. However, I wanted to use it as a learning opportunity and build the site in something unconventional. The two most popular solutions for building a personal site are WordPress or a static site generator. I wanted something in the middle, a site with a backend for post writing and a custom frontend with few dependencies. I built the site using Go with a couple frontend packages including TailwindCSS.

Backend in Go with Gin

I decided to use Go for the backend. In the last few years, Go has become very popular for server apps, particularly micro-services. I chose Gin as a web framework to help with routing and serving templates. Gin has a Sinatra-like syntax that allows you to express a route as a HTTP method and a function to be called for the route.

func main() {
    router := gin.Default()

    router.GET("/", func (c *gin.Context) {
      c.HTML(http.StatusOK, "index.html")
    })
}

I also pulled in Gorm for database access. Gorm allows you to write your tables as Go structs and it will build the columns from the struct fields. Then, you can easily query records based on your struct.

type Post struct {
    gorm.Model
    Name string
    Content string `gorm:"type:text;"`
}

var post Post
db.First(&post)

fmt.Println(post.Name)

All in all, I was successful using Go to build the backend. I like the safety you get with data types and compile checks. The beauty of Go is that if you can get the code to compile, then you have more confidence the app will work. It is also very fast. However, most of the request time is spent querying the database anyway so the real world speed boost is not a significant difference from PHP or Ruby. I had to piece together several packages to get all the features I need, but there are other Go frameworks like Buffalo that offer a more Rails-like experience.

Frontend with TailwindCSS, SimpleMDE, Stimulus and Prism

I used TailwindCSS as my styling framework. I have become a huge fan of Tailwind for all of my personal projects. It takes a little time to adapt to a utility framework, but I love the flexibility and features you get from it. It allows me to build a unique website without much custom CSS. Plus, it comes with some great default color schemes and font stacks.

I used SimpleMDE on the backend to give me a Markdown editor. It was very easy to setup and has worked with no problems.

Also on the backend, I used Stimulus to help me build in a little more interactivity. Stimulus is an excellent framework for server rendered HTML and has become a replacement replacement for jQuery when I need some JavaScript sprinkles. Stimulus is easy to learn and is also much easier to keep organized than jQuery.

Lastly, I used Prism on the frontend to highlight code blocks. It was also easy to set up and run. It allows you to pick which languages you need for highlighting and gives you the options to pick a theme (Tomorrow Night for the win!).

Conclusion

andrewdavis.me is up and running now so please check it out and let me know what you think! While my primary blogging platform is Dev.to, I enjoy having my own site to show my skills and writing. I plan to add a few more pages showing my current projects and past work. I think a personal site is a great way to experiment with new patterns and have some fun with new languages. What languages and libraries have you used for your personal sites?

Showing Kindness while Programming

Andrew Davis — Tue, 20 Nov 2018 14:34:14 +0000

Software development and programming is a difficult profession. It is very new and requires a lot of practice, learning and mental focus. It can get frustrating when you are working with others and mistakes are made. Programmers are also very demanding of those who provide our programming languages and frameworks, especially with bugs or problems that hinder our daily work. When we encounter these problems, it is very tempting to get angry and make a snide comment on social media or give a dirty look in the office.

An angry response is not what we need in development. Showing patience with new developers allows them room to grow without fear of retribution. Being kind to vendors allows them the mental energy to improve their product or framework. Forgiving mistakes improves the emotional health of your team and their ability to code incredible features. To encourage patience, here are my thoughts on how we can better show kindness in software development.

Be Patient with Junior Developers

Being a junior developer is very difficult. There is so much to learn when starting to develop software and it goes beyond just knowing how to use a programming language. A junior developer has to learn how to work on a team, understand software requirements, solve problems and communicate with non-technical stakeholders. It is very intimidating and takes several years for a junior developer to even begin to master these different areas, not to mention keeping up with programming trends. Therefore, it is key for more experienced developers and managers to show patience with junior developers as they learn. For a junior developer to find confidence in their work, they do not need to be afraid of failing. They need to be able to fail and be instructed, with kindness, about how to prevent the mistake again.

Do Not Mock Someone for their Chosen Language or Framework

We developers are very opinionated about what is the best programming language or framework to use. Having opinions and healthy debates is a good thing and allows the programming community to grow with even better features and tools. However, we need to stop mocking each other for building a product or tool in a given language or framework just because we do not like it. I see this a lot in the JavaScript and PHP communities. We criticize each other for using React or Vue or Angular or (heaven forbid) jQuery! PHP developers are mocked for their language’s inconsistencies. These criticisms are not constructive and just discourage developers from building. At the end of the day, it doesn’t matter if you like React or Ember when you are making successful software.

Be Considerate with Open Source Maintainers

Our profession is very unique in that it relies a lot on free frameworks and tools provided by volunteers. Since open source tools are built out in the open, we also have more access to the maintainers and creators behind them. It can get very tempting to send an angry comment to maintainers on Twitter or GitHub when their framework breaks. An open source developer is often unpaid in their contributions so they do not deserve our ire or disrespect. Even if they are being paid, showing anger does not help them fix a problem. Being kind and giving respectful feedback is key to the project being successful and the maintainers not getting burned out on their work.

Show Forgiveness when Something Breaks

When something goes wrong with your application, it can be really easy to start the blame game. Who pushed the code? What did they screw up? Why were they not more careful? These kinds of questions are unhelpful and hurtful to whoever receives the blame. Instead of blaming, we need to focus on what went wrong and how we can help each other not make that mistake again. Maybe we need a better unit testing setup? Maybe we need better documentation of this feature? Should we test more thoroughly after deploying code? Questions like these are constructive and help programmers recover more quickly from mistakes and stay confident in their work.

Celebrate Victories

When development is going well, it is easily taken for granted. As a result, we often do not display an emotional reaction until a bug happens or a customer complains. To increase programming morale, it is important to regularly celebrate accomplishments. Success is very satisfying and helps programmers stay motivated and avoid burnout. Next time you achieve a new feature, celebrate it! Even if the reward is just a box of cookies and some high fives.

Conclusion

Being kind and forgiving is a conscious choice that we need to encourage in our development teams. It allows for developers to learn more quickly, prevent mistakes and enjoy work. Kindness is a cure for burnout and exhaustion and encourages fresh ideas and unique features. Being patient with others is not easy, but essential for happy and successful teams. Next time you work with another programmer, I encourage you to think about how you can be more kind and patient with them. It will make your work more fulfilling in a way you may not have already experienced.

Writing Beautiful Code

Andrew Davis — Mon, 24 Sep 2018 14:28:54 +0000

Because programming is a technical profession, we often approach it without thought to design. Code is for getting a computer to do what we need, why should design matter? The majority of the programming languages we use were built for the programmer and are transformed into something that is optimized for the computer. As a result, we need to write our code, not just for the compiler or interpreter, but for humans to read. To make code readable, I approach it as an art. When you treat code as art, it becomes quickly digestible and easy to extend. Creative code also reduces bugs because it is easier to understand what it is executing.

The problem with artful coding is that it is not easy to do. Oftentimes, it is hard enough to get the code to work in the first place, much less be creative! However, just like any art form, there are some basic techniques we can follow to write well designed code. I am still exploring the best ways to write beautiful code, but these steps are what I follow to bring a little art to my programming.

Start with a Unit Test

Test Driven Development is not universally liked, but I find it to be a very good practice. Not only does it ensure all your code is tested, it also forces you to think about how your code will be used before writing it. In these examples, let’s imagine we are writing an email client in PHP and we want to be creative in the client’s design. Let’s start with a unit test for sending an email:

<?php

class EmailClientTest extends \PHPUnit\Framework\TestCase
{
    public function test_sending_an_email()
    {
        $client = new \Email\Client();
        $emailMessage = new \Email\Message('Hello world!');

        $result = $client->send('andrew@dev.to', $emailMessage);

        $this->assertTrue($result);
    }
}

Since we wrote the unit test first, we already know how we want the code to be shaped. We know we will have an Email namespace with two classes representing the Client and the Message. We also know that we will have a send method for the process of submitting the email message with the client. In the unit test, we also decided how the pieces of our code will be named, which brings us to my next point.

Use Grammar when Naming Variables and Methods

Programmers often joke that naming things is hard and it’s the truth! However, naming a piece of code is not just hard, it’s also very important. How code is named determines how quickly a developer can understand it, reducing bugs and development time. In the above example, we decided to put all of our code in the Email namespace which is very clear about what it contains. Next, we chose Client to represent the object that sends messages and Message for the email itself, both descriptive nouns. Finally, we call a send method which is a verb commonly associated with email. Now, we see a good pattern: use nouns for class names and verbs for methods.

Keep Your Functions Small

It is tempting to put all the code for an operation in a single function. However, it’s really important to break it up into small chunks for readability and testability. Let’s use the send method as an example.

<?php

namespace Email;

class Client
{
    public function send(string $emailAddress, Message $message): bool
    {
        // Validate parameters
        $validateEmail = filter_var($emailAddress, FILTER_VALIDATE_EMAIL);
        $validateMessage = strlen($message->text()) < 1000;
        if (!$validateEmail || !$validateMessage) {
            return false;
        }

        // Communicate with email server
        $curl = curl_init();
        curl_setopt($curl, CURLOPT_URL, 'https://mail.example.com');
        curl_setopt($curl, CURLOPT_POST, true);
        curl_setopt($curl, CURLOPT_POSTFIELDS, [
            'to' => $emailAddress,
            'text' => $message->text(),
        ]);

        $result = curl_exec($curl);
        curl_close($curl);

        return $result;
    }
}

This method is straightforward, but cluttered. It’s a little difficult to understand, so we added some comments so the reader knows what is happening. However, what if we created a new method instead of a comment?

<?php

namespace Email;

class Client
{
    public function send(string $emailAddress, Message $message): bool
    {
        if (!$this->validateParameters($emailAddress, $message)) {
            return false;
        }

        return $this->sendToMailServer([
            'to' => $emailAddress,
            'text' => $message->text(),
        ]);
    }

    private function validateParameters(string $emailAddress, Message $message): bool
    {
        $validateEmail = filter_var($emailAddress, FILTER_VALIDATE_EMAIL);
        $validateMessage = strlen($message->text()) < 1000;
        if (!$validateEmail || !$validateMessage) {
            return false;
        }

        return true;
    }

    private function sendToMailServer(array $params): bool
    {
        $curl = curl_init();
        curl_setopt($curl, CURLOPT_URL, 'https://mail.example.com');
        curl_setopt($curl, CURLOPT_POST, true);
        curl_setopt($curl, CURLOPT_POSTFIELDS, $params);

        $result = curl_exec($curl);
        curl_close($curl);

        return $result;
    }
}

Our send method is now much cleaner. The other functions perform a single task with less than 10 lines of code each, making them easy to read.

Use a Style Guide

Finally, I recommend using a style guide for your chosen programming language. For PHP, I like to use the PSR-2 FIG standard for my code. The FIG standard is widely recognized and used by popular PHP frameworks like Laravel and Symfony. Plus, the Symfony team has released a CLI utility that automagically reformats your code to follow PSR-2: PHP Coding Standards Fixer.

Conclusion

Designing code requires effort and practice, but reaps many rewards. It causes your code to be in smaller, more readable chunks and reduces bugs by bringing clarity and focus. To continue learning more, I recommend reading code that is well designed. The Laravel and Ruby on Rails codebases are both examples of clean code that is treated with an artful touch. Do some source diving in both to learn more!

What libraries do you like to use to write unit and integration tests for React Native? Any TDD advocates using React Native?

Andrew Davis — Thu, 13 Sep 2018 02:40:39 +0000

I am currently working on a React Native project for the first time. I used react-native init so I have Jest which I can use for snapshot testing. However, what do you like to use for integration testing, like filling out fields and submitting buttons? I would like to find a workflow that works well with TDD if possible.