DEV Community: Max

Prettier Code

Max — Mon, 18 Feb 2019 16:56:39 +0000

If you care about code formatting, you might want to take a look at Prettier. It changed the way I think about coding styles quite a bit.

So I used to spend a lot of time fiddling with code styles. From debating spaces vs. tabs to comparing Symfony's Coding Standards and Google's Styleguides. With JavaScript starting to be the language of choice for most new projects, I settled on the Airbnb JS Style Guide and with the matching linter rules the topic was settled for quite some time.

But half a year ago, we decided at work to use Prettier for a new project. And this has changed how I think about code styleguides in a pretty fundamental way: I just don't care anymore.

What prettier does: Instead of looking at the code style as it was written and applying rules on it, Prettier parses the code and prints it in its own format. So the leeway classic styleguides give every developer isn't a topic to ponder on anymore.

Like many linters, it automatically reformats the files on saving. At first it felt like a heavy intrusion in my work. After all I – at least pretended – to put some effort and pride into the styling of the code I wrote. But a few days later I almost completely stopped thinking about code formatting. Months later I'm on the other end: I write code and am always heavily confused, if it doesn't automatically get reformated in the now familiar Prettier-style.

So if you happen to start a fresh project, just give Prettier a try for a couple of days.

Makefiles to Rule Them All

Max — Tue, 17 Apr 2018 05:45:00 +0000

In my last blogpost you might have already stumbled over me using a Makefile to simplify a project task. My enthusiasm for Makefiles goes a bit further and I add one to most of my projects by now. Here is why I do this and how they make my everyday developer life a bit easier.

None of my projects is written in C or C++ which rely on a Makefile for compiling them. Instead my projects are written in JavaScript (Node), Ruby and PHP. I use the make command as a wrapper for the individual tools coming with each ecosystem to create a common interface. This way I can just run make test no matter whether the tests are in JavaScript and use Mocha or use PHPUnit.

Besides the different set of tools belonging to a platform (e.g. Node’s npm, PHP’s Composer and Ruby’s Bundler) there are a few more difference between projects that I created a shortcut for: To run a project locally it might be anything between npm run, a Docker container or a Vagrantbox. And on the plain level of a project’s structure there is also a wide variance: Everything might be in the root directory, or split in a src/ and docker/ directory. Every project is its own special snowflake in this regard.

After a short trial with shell scripts to do repetitive tasks always from the project root I quickly switched to use make instead. It’s widely available and make it easy to handle different commands. So I started to develop my strategy for Makefiles. It’s schema applies to most of my web-based projects perfectly:

make setup – Does everything which might be necessary on the first setup like installing the dependencies, like run npm install for a node-project.
make run – This will mostly default to docker-compose up. Or to bundle exec jekyll serve --drafts for this blog.
make test – You might have guessed it: This one executes the unit tests of the project.

Of course it doesn’t stop there. There are often a bunch more commands which might for example trigger an existing build system like ant. Or just as an alias for the npm scripts. Nonetheless it’s a great shortcut saving me quite some time when switching between projects.

A small example for a PHP project, which uses Composer and comes with a Docker-Compose setup. The tests are executed within the Docker container.

.PHONY: setup
setup:
    @cd src/ && \
    composer install

.PHONY: test
test:
    docker-compose run web /bin/bash -c 'composer test' && exit

.PHONY: run
run:
    docker-compose up -d

So if you also happen to often switch between diverse projects, maybe you also want to give Makefiles a try.

Adding a Microblog to Jekyll

Max — Sun, 15 Apr 2018 14:30:00 +0000

Since I learned about micro.blog back in January I wanted to give it a try on my blog. For the last few days I had a few attempts on it. And as you can see at xam.io/microblog and on micro.blog I was successful. Now I want to share how I implemented it.

A hat tip to the Creating a Microblog with Jekyll post which covers a lot of the groundwork. I picked up a lot of his ideas and will focus a bit more on the integration with my existing blog in this post.

A Collection for the Microblog

Since I wanted to keep the regular posts separate from the microblog posts, I decided to use a collection to do so. If you don’t care about this separation as much, using a dedicated category within the regular posts is probably the easier option.

So here the configuration for the collection:

collections:
  microblog:
    output: true
    permalink: microblog/:year/:month/:title/

I decided to keep the :year/:month/:title path for the microblog posts, but on level deeper at microblog — of course the permalink schema is totally up to you.

This configuration will pick up any files in the _microblog directory, interpret them as part of the microblog collection and create a rendered HTML page in the output directory.

Adding Microblog Posts to the Index Page

Items from collections aren’t part of the site.posts array that’s normally looped over on the index page, listing one item after the other. Instead they can be accessed by using site.microblog. To merge them together (and have them paginated), I replaced the (no longer maintained) jekyll-paginate with the jekyll-paginate-v2 gem.

In the front-matter of my index.html I now have this configuration to paginate over both, the posts and the microblog:

--------
layout: default
pagination:
  enabled: true
  collection: posts, microblog
  per_page: 10 # Number of entries per page
  permalink: "page/:num/" # Permalink structure
  sort_reverse: true # Newest first
--------

{% for post in paginator.posts %}
    {% if post.collection == 'microblog' %}
        {% include microblog.html %}
    {% else %}
        {% include post.html %}
    {% endif %}
{% endfor %}

The microblog.html is a reduced version of my post.html template that usually renders a blog post.

Creating Separate Pages for Posts and Microblog only

As you can see this blog also has Posts page (containing only regular posts) and a Microblog page containing only the microblog posts.

To do so I have copied the index.html as microblog.html (for the microblog) and as posts.html (for the regular posts). But changed the collection: frontmatter to only include microblog or posts.

The permalink structure of the pagination is relative to the source directory, together with the permalink configuration for posts and microblog, this is the current URL schema on this site:

/ -- Page with microblog and posts
/page/:num/ -- Pagination for both microblog and posts

/posts/ -- Page only with posts
/posts/page/:num/ -- Pagination for regular posts
/:year/:month/:title/ -- URL for single posts

/microblog/ -- Page only with microblog
/microblog/page/:num/ -- Pagination for microblog posts
/microblog/:year/:month/:title/ -- URL for a single microblog

Adding a Feed for the Microblog

For now I keep my regular feed (available at /atom.xml) limited to regular posts only. But I created a separate feed for the microblog (/microblog.xml).

Since also the regular feed doesn’t use any Jekyll plugin, I also opted for the vanilla approach for the microblog feed — my microblog.xml template:

--------
layout: null
--------
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>xam.io | Microblog</title>
    <description>Microblog of Max Kleucker</description>
    <link>{{ site.url }}</link>
    <atom:link href="{{ site.url }}/microblog.xml" rel="self" type="application/rss+xml" />
    {% for post in site.microblog reversed limit:50 %}
      <item>
        <description>{{ post.content | xml_escape }}{% if post.external-url %}{% endif %}</description>
        <pubDate>{{ post.date | date: "%a, %d %b %Y %H:%M:%S %z" }}</pubDate>
        <link>{{ site.url }}{{ post.url }}</link>
        <guid isPermaLink="true">{{ site.url }}{{ post.url }}</guid>
      </item>
    {% endfor %}
  </channel>
</rss>

This feed is the source for my micro.blog account as well.

Creating Microblog Posts

So publishing works, we can manually create a new file in the _microblog/ directory and it will be published on the index-page, the microblog-page and the microblog-feed. Yeah!

Creating a microblog post means to manually create a file at _microblog/<YEAR>-<MONTH>-<DAY>-<TITLE>.md. In particular the <TITLE> part is a bit unfortunate: Per @manton’s definition the a microblog doesn’t use titles for posts. Yet you can’t omit it, since otherwise you could only create one entry per day.

So I wanted to have something quick like the jekyll post "my-title" command. I decided to a add a Makefile that can create a new microblog-file from a template and open it in my editor of choice right away — with an autogenerated title, so I don’t have to care about it.

The template (at templates/_microblog):

--------
layout: microblog
date: %CURRENT_DATE%
title: %POST_TITLE%
--------

And the Makefile:

# Makefile
MICROBLOG_TEMPLATE := _templates/microblog
POST_DATE := $(shell date +%Y-%m-%d)
POST_TIME := $(shell date +%Y-%m-%d\ %T\ %z)
POST_TITLE := $(shell openssl rand 100000 | shasum | cut -c1-8)
POST_FILE := _microblog/$(POST_DATE)-$(POST_TITLE).md
.PHONY: new-microblog
new-microblog:
    @cat $(MICROBLOG_TEMPLATE) | \
    sed "s/%CURRENT_DATE%/$(POST_TIME)/g" | \
    sed "s/%POST_TITLE%/$(POST_TITLE)/g" > ${POST_FILE} && \
    subl ${POST_FILE}

Running make new-microblog on your shell will do the following and leave you with a new microblog post ready to publish:

Read the template from _templates/microblog
Replace the %CURRENT_DATE% in the template with the current date and time
Replace the %POST_TITLE% with a randomized 8-characters string
Save this at _microblog/<CURRENT_DATE>-<RANDOMIZED_TITLE>.md
Open this file in vim. You might change this to the editor of your choice.

Where To Go From Here

Setting everything up, I ended up learning a lot about the Micropub standard and Webmentions. The IndieWeb page is a great resource for this stuff. But in short, there is a whole set of specifications and standards defining the protocols to run your own micropub backend. And Webmentions are a neat way to let somebody when you referenced them.

Yet these protocols need a dynamic webservice to actively work. That might look a bit counterintuitive to running a static blog, but on the indieweb wiki there already are a few ways to do so. But that’s for another weekend.

PHP Integration Tests Using the Built-in Server

Max — Fri, 30 Mar 2018 15:42:17 +0000

Microservices have plopped up everywhere and I have written a few of them at work by now. Given the limited scope I find microservices a pleasure to write tests for compared to most (legacy) monolith. In this post I want to share an approach on how to test microservices written in PHP using it's built-in server and some pitfalls we encountered so far.

Let's assume we have a nice little microservice built with Slim, Lumen or which ever framework you prefer. We have some routes that return JSON responses and accept various JSON payloads.

We're able to run this application using PHP's built-in server by running for example:

ENVIRONMENT=production php -S 0.0.0.0:8080 -t public public/index.php

In this case the public/index.php is the entry file for any HTTP request, and the server is no reachable at http://localhost:8080.

Also you already have a PHPUnit setup available which covers unit tests. For the sake of brevity we will skip the whole PHPUnit setup part and jump directly into writing tests against the built-in server.

Basic Setup

The goal is to write a test case where we might have a body payload which we want to send to specific endpoint and then check if the response has the HTTP status 200 and the returned answer is a JSON containing a specific information. Like this:

<?php
namespace Tests\Integration;

final class ControllerTest extends BaseTest
{
    public function testCorrectCall()
    {
        $body = json_encode(
          [
            'field_one' => 'some data',
            'field_two' => [
              'apple',
              'banana',
            ],
          ]
        );

        /** @var \Psr\Http\Message\ResponseInterface $response */
        $response = $this->dispatch($body, '/api/myfunction', 'POST');

        $this->assertSame($response->getStatusCode(), 200);
        $parsedResponse = json_decode($response->getBody(), true);
        $this->assertArraySubset(['message' => 'success'], $parsedResponse);
    }
}

The interesting part is to resolve the $this->dispatch part. We hand in the endpoint, the HTTP method and the payload and expect to retrieve a reponse object. So the dispatch() function must actually send a HTTP request and return the response object. To extract this from the actual tests, a BaseTest class is created:

<?php
namespace Tests\Integration;

use PHPUnit\Framework\TestCase;
use Symfony\Component\Process\Process;
use GuzzleHttp\Client;

abstract class BaseTest extends TestCase
{

    protected static $process;

    const ENVIRONMENT = "production";
    const HOST = "0.0.0.0";
    const PORT = 9876; // Adjust this to a port you're sure is free

    public static function setUpBeforeClass()
    {
        // The command to spin up the server
        $command = sprintf(
          'ENVIRONMENT=%s php -S %s:%d -t %s %s',
          self::ENVIRONMENT,
          self::HOST,
          self::PORT,
          realpath(__DIR__.'/../../public'),
          realpath(__DIR__.'/../../public/index.php')
        );
        // Using Symfony/Process to get a handler for starting a new process
        self::$process = new Process($command);
        // Disabling the output, otherwise the process might hang after too much output
        self::$process->disableOutput();
        // Actually execute the command and start the process
        self::$process->start();
        // Let's give the server some leeway to fully start
        self::usleep(100000);
    }

    public static function tearDownAfterClass()
    {
        self::$process->stop();
    }

    protected function dispatch($data = null, $path = null, $method = 'POST'): ResponseInterface
    {
        // STEP 2: Using Guzzle we send a request to the server
        $params = [];
        if ($data) {
            $params['body'] = $data;
        }

        // Creating a Guzzle Client with the base_uri, so we can use a relative
        // path for the requests.
        $client = new Client(['base_uri' => 'http://127.0.0.1:' . self::PORT]);
        return $client->request($method, $path, $params)
    }
}

For this to work you have to add these packages to your development dependencies:

composer require --dev symfony/process
composer require --dev guzzlehttp/guzzle

What's Happening Here?

We're making use of PHPUnit's setUpBeforeClass() and tearDownAfterClass() function to control the lifetime of the local PHP server. The symfony/process package gives us a nice handler on executing the PHP command on the local machine. We don't want to recreate the server for every single test, using one server per TestCase-class is sufficient for our use case.

In the dispatch() function we're creating a request using guzzlehttp/guzzle. This is sent to server (using the same port as the local server) and it's Response object is returned to the caller.

Et voilà the testCorrectCall is executed against a local PHP server running your application.

Some Notes

We ran into weird issues where tests would be stuck forever, adding self::$process->disableOutput(); resolved this for us.
If you have multiple classes inheriting from BaseTest you might want to use different ports. We had issues where the ports were not yet released by the system. Swapping the cont $port for a static $port and either increasing or decreasing it, might help.

Combining Remote Tests with a Mocked Environment

As noted above, most PHP frameworks come with a way to test them in a mocked environment – without spinning up a local PHP server. Our tests don't make any use of that ability so far.

In an attempt to further enhance the test suite, we wanted to identify occasions where the application behaves differently when accessed directly to when it is accessed via HTTP. In fact, our initial trigger to use remote tests was a scenario where we only had test against a mocked environment and everything looked good in the tests, only to learn that our service would crash for the exact same calls when execute remotely (after being deployed of course).

So here the solution we're currently using to run tests against both the remote server as well as a mocked environment:

<?php

namespace Tests\Integration;

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;

abstract class BaseTest extends TestCase
{
    # Added to the part of BaseTest shown above

    protected function dispatch($data = null, $path = null, $method = 'POST', $headers = []): ResponseInterface
    {
        $localResponse = $this->dispatchLocal($data, $path, $method);
        $remoteResponse = $this->dispatchRemote($data, $path, $method);

        $this->ensureLocalAndHttpIdentical($localResponse, $remoteResponse);

        return $remoteResponse;
    }

    private function dispatchLocal($data = null, $path = null, $method = 'POST'): ResponseInterface
    {

        // TODO Mock your application as necessary, so it can be executed and returns an instance of the ResponseInterface
        $req = new Request($method, $uri, $headers, $cookies, $serverParams, $body);

        $this->app->getContainer()['request'] = $req;

        return $this->app->run(true);
    }

    private function dispatchRemote($data = null, $path = null, $method = 'POST'): ResponseInterface
    {
        $params = [];
        if ($data) {
            $params['body'] = $data;
        }

        $client = new Client(['base_uri' => 'http://127.0.0.1:' . self::PORT]);

        return $client->request($method, $path, $params);
    }

    protected function ensureLocalAndHttpIdentical(ResponseInterface $localResponse, ResponseInterface $remoteResponse)
    {
        $this->assertSame($localResponse->getStatusCode(), $remoteResponse->getStatusCode());
        $this->assertEquals((string)$localResponse->getBody(), (string)$remoteResponse->getBody());
    }
}

So when we now use the dispatch() function in any of our tests it will create two requests: one against the mocked application and an actual HTTP request against the local PHP server. As we don't want to test the properties of both requests individually, we compare the body and the response code int eh ensureLocalAndHttpIdentical() function. Any differences here would cause the tests to fail. Then the $remoteResponse is being returned to the actual test.

This doesn't remove the need for unit tests. But it added a lot to my ease of mind when refactoring controllers or adding new features. Next for me will be add Mutation Testing to my palette of test approaches (most likely with Infection).

Maintaining Different Git Identities

Max — Sun, 02 Jul 2017 15:08:10 +0000

I like to have separate "identities" for my private and work stuff when using Git: Commits at work should be authored with my work email and commits in private projects with my private account. Until now I would always configure this per repository as soon as I noticed a commit done by the wrong email.

As I was setting up my new computer and edited the global .gitconfig, I wondered if there is a better way to keep this separate. Turns out there is one: With the release of version 2.13, Git introduced "Conditional Includes".

With these includes I can set a specific gitconfig-file to be include for all repositories within a specific location. As I store all my work projects within the folder ~/Work, I set the default user-config to be my private one and include the work-specific configfile for all Git repositories within that location.

# ~/.gitconfig

[user]
    name = Firstname Lastname
    email = <private email address>

[includeIf "gitdir:~/Work/"]
    path = .gitconfig-work

# ~/.gitconfig-work

[user]
    name = Lastname, Firstname
    email = <work email address>

The risk of being annoyed by wrongly associated commits is vastly smaller now — until I start checking out work projects to ~/Desktop/tmp for minor fixes.

To check your configuration, make sure to be in a directory which is tracked by Git. Non-Git directories will always show the default configuration:

$ cd ~/dev/justcurious 
$ git config user.email
<private email address>

$ cd ~/Work/projectA
$ git config user.email
<work email address>

$ cd ~/Work/not-a-repo
$ git config user.email
<private email address>