DEV Community: John Braun

Getting started with Mutation Testing

John Braun — Thu, 11 Feb 2021 00:00:00 +0000

Introduction

How do you determine if the current test suite adds enough value? Do the tests cover all edge cases? If you solely rely on code coverage, you might be missing out.

In a recent talk at the Laracon EU Online conference, my colleague Jeroen Groenendijk highlighted the significance of Mutation Testing in achieving greater confidence in the test suite of your application.

In this blog I'd like to highlight the concepts of Mutation Testing, explain how to get started using Infection PHP by Maks Rafalko, show some practical examples and finally explain how to use Mutation Testing in a CI setup.

What is Mutation Testing

A mutation testing tool will manipulate (mutate) pieces of your source code and run the test suite against this piece of mutant source code. The mutated code should trigger a failing test, or the mutant escapes. Escaped mutants are a sign of weakly tested code.

As an illustration, take a look at the following example.

The add() method is mutated in three different ways:

The method returns null

The + operator changed to the - operator

Method visibility changed from public to protected

In the diagram above, we see that the tests fail in case null is returned and when the addition is replaced with a subtraction. The method's visibility however is likely too broad and mutating it to protected didn't yield a failing test. Therefore, it should be changed to protected or even private, adhering to the best practice of keeping the public API of a class to the minimum.

Getting started with Infection PHP

If you just want to play around, check the Infection Playground, where you can write the code, tests, and run Infection right from within your browser. Read on to learn more about setting up Infection PHP in your project.

Setting up Infection PHP

Infection requires PHP version 7.2 (or higher) and an enabled debugger of choice: Xdebug, phpdbg or pcov. Tip: make sure to check out LearnXDebug.com when you're setting up Xdebug in a Laravel oriented development environment.

In the steps below, I assume having xDebug installed.

While you have multiple options to install Infection, I would recommend installing it as a dev dependency using composer within your project.

Step 1. Install Infection PHP

composer require --dev infection/infection

Step 2. Run Infection

vendor/bin/infection

On the first run, Infection will ask for input:

Directories to include:

For Laravel projects this means your app directory.

For PHP packages this means the src directory.

Excluding directories from source directories:

Leave this blank, unless you have PHP code living in the source directory that shouldn't be mutated.

Where to store the text log file:

I would recommend saving the mutations to e.g. infection.log. All escaped mutants are saved here for later review.

Alternatively, you may use the --show-mutations option to log the mutations to the terminal output.

You'll now find infection.json.dist in the root of your project reflecting your input:

{
    "source": {
        "directories": [
            "src"
        ]
    },
    "logs": {
        "text": "infection.log"
    },
    "mutators": {
        "@default": true
    }
}

You can specify if you want to enable (or disable) a specific mutator under the "mutators" key. Make sure to check the complete list of available mutators.

Tip: allow the mutation tests to run in parallel by providing the --threads option and setting it greater than 1, for example by running vendor/bin/infection --threads=4. This will greatly speed up running the mutation tests. An overview of all command-line options can be found here.

Code Examples

Example 1: Calculating shipping cost

To demonstrate Mutation Testing in practice, I'll borrow the example from Jeroen's talk.

Imagine having a ShippingCalculator service class to determine if a passed in Product qualifies for free shipping. For simplicity’s sake, let's say the Product class accepts a $price integer through its constructor and provides public access to a $shipsForFree property.

The ShippingCalculator class determines that a Product receives free shipping when:

The price is equal to or greater than the threshold (set to an arbitrary value)
The product's $shipsForFree property is true (or truthy)

class ShippingCalculator
{
    const FREE_SHIPPING_THRESHOLD = 20;

    public static function hasFreeShipping(Product $product): bool
    {        
        if ($product->price >= self::FREE_SHIPPING_THRESHOLD) {
            return true;
        }

        if ($product->shipsForFree) {
            return true;
        }

        return false;
    }
}

Testing the code with PHPUnit

To make sure the ShippingCalculator::hasFreeShipping() method works properly, we can think of adding the following unit tests to ensure proper behavior:

When a product's $price exceeds the threshold, it should ship for free
When a product's $price does not exceed the threshold, it should not ship for free
When a product's $shipsforFree property set to true, it should ship for free

class ShippingCalculatorTest extends TestCase
{
    /** @test */
    function product_ships_for_free_when_price_is_above_treshold()
    {
        $product = new Product($price = ShippingCalculator::FREE_SHIPPING_THRESHOLD + 1);

        $this->assertTrue(ShippingCalculator::hasFreeShipping($product));
    }

    /** @test */
    function product_does_not_ship_for_free_when_price_is_below_treshold()
    {
        $product = new Product($price = ShippingCalculator::FREE_SHIPPING_THRESHOLD - 1);

        $this->assertFalse(ShippingCalculator::hasFreeShipping($product));
    }

    /** @test */
    function product_ships_for_free_when_ships_for_free_property_is_true()
    {
        $product = new Product(ShippingCalculator::FREE_SHIPPING_THRESHOLD - 1);
        $product->shipsForFree = true;

        $this->assertTrue(ShippingCalculator::hasFreeShipping($product));
    }
}

With these three tests, the code coverage report from PHPUnit (which you can generate using vendor/bin/phpunit --coverage-text) reveals a code coverage of 100% for both classes.

 Summary:
  Classes: 100.00% (2/2)
  Methods: 100.00% (2/2)
  Lines:   100.00% (7/7)

Running Infection PHP

Now, we'll run Infection PHP using vendor/bin/infection and see if any mutants escape.

.M....                                               (6 / 6)

6 mutations were generated:
       5 mutants were killed
       0 mutants were not covered by tests
       1 covered mutants were not detected
       ...

Metrics:
         Mutation Score Indicator (MSI): 83%
         Mutation Code Coverage: 100%
         Covered Code MSI: 83%

Oh no, a mutant has escaped!

Furthermore, our MSI is 83%, while the generated mutations covered 100% of the code. This means 5 out of 6 mutants were killed.

When we check our log file, we see the [M] GreaterThanOrEqualTo mutant escaped:

Escaped mutants:
================
1) ../src/ShippingCalculator.php:15    [M] GreaterThanOrEqualTo

-------- Original
+++ New
@@ @@
-        if ($product->price >= self::FREE_SHIPPING_THRESHOLD) {
+        if ($product->price > self::FREE_SHIPPING_THRESHOLD) {

The missing test

Now it becomes obvious that we are missing a crucial test: we did not assert what happens when the $price is equal to the free shipping threshold.

Infection PHP expected at least one of our tests to fail when mutating the conditional from a great-than-or-equals to a greater-than comparison. Since our test suite didn't fail, this mutation got away unnoticed.

Let's fix that by adding in the "forgotten" boundary test:

/** @test */
function product_ships_for_free_when_price_equals_threshold()
{
    $product = new Product(ShippingCalculator::FREE_SHIPPING_THRESHOLD);

    $this->assertTrue(ShippingCalculator::hasFreeShipping($product));
}

When we run Infection again, we get an MSI of 100%. No escaped mutants this time!

Example 2: Redirection

After using mutation testing in my Laravel projects, I learned that mutation testing drives out tests I would otherwise not have written.

When redirecting users to a specific page with a "date" parameter from the store() controller action.

class AppointmentController
{
    public function store()
    {
        // controller logic

        return redirect(route('appointments.index', ['date' => $date]));
    }
}

In my test for this controller action, I initially did not include a check that the user was redirected to a specific page. However, when running the Infection PHP mutation test suite I got the following escaped mutant:

3) .../app/Http/Controllers/AppointmentController.php:163    [M] ArrayItemRemoval

-------- Original
+++ New
@@ @@
-        return redirect(route('appointments.index', ['date' => $date]));
+        return redirect(route('appointments.index', []));
     }
 }

While this date parameter in the redirect is crucial for users of the application to land on the appropriate page, I didn't have a test for this behavior. Until now, not having a test for the specific redirect could go unnoticed. Thanks to mutation testing I've now discovered this gap and it has forced me to add a test covering this scenario.

Using Infection in CI

It is possible to run mutation tests in Continuous Integration (CI) against newly added code or before builds.

You can set a --min-msi score option to force a certain percentage of mutants to be killed and gradually increase this number to improve the project's test suite. If the MSI score is lower than required, the build will fail.

Small projects

For small projects, the simplest option is to first run the PHPUnit tests while saving the test coverage files in a build directory and feeding those directly to Infection PHP.

vendor/bin/phpunit --coverage-xml=build/coverage-xml --log-junit=build/phpunit.junit.xml
vendor/bin/infection --threads=2 --coverage=build --min-msi=70

GitHub Actions

Although Infection can be integrated with any CI setup, I like to highlight the newly added integration with GitHub Actions (in version 0.20), where it is possible to log escaped mutants in committed code directly within a PR.

Basically, you can add the following action to your GitHub Action workflow:

- name: Run Infection for added (A) and modified (M) files
  run: |
    git fetch --depth=1 origin $GITHUB_BASE_REF
    php vendor/bin/infection --threads=2 --git-diff-base=origin/$GITHUB_BASE_REF --git-diff-filter=AM --logger-github

Check out the release documentation for more details.

Larger projects

If you want to use Mutation Testing in CI for larger projects, make sure to check out this article by Alejandro Celaya.

In his blog post, he advises using phpdbg to run Infection which leads to a clear gain in performance.

Conclusion

I hope this blog post provides some insight and direction to get started with Mutation Testing. In summary, Mutation Testing creates mutants from your source code which run against your existing test suite. If none of these tests fail, it is a sign that this piece of the source code (the mutant) is weakly tested. This methodology helps to identify gaps in your current test suite as well as dead or unnecessary code (or e.g. unnecessary broad method visibility).

Although this blog post merely covers some examples, there is a lot more to explore in Mutation Testing: make sure to check out all available mutators and the how-to guide.

Drawbacks

While Mutation Testing has a lot of advantages, think of the following possible drawbacks and things to be aware of.

Running mutation tests is quite slow. You can speed up the process by allowing testing in parallel (set --threads > 1).

Not all mutants have to be killed. Trying to kill all mutants might lead to tight coupling between your code and your tests. Or lead to code that is too strict and no longer flexible.

More resources

If you want to learn more about Mutation Testing, make sure to check out these great resources on Mutation Testing (using Infection):

Articles:

Infection: Mutation Testing Framework - Maks Rafalko (Infection PHP creator)

Mutation testing in PHP - Moxio ( Arnout Boks)

Mutation Testing With Infection - Matthew Daly

Mutation testing with infection in big PHP projects - Alejandro Celaya

Talks:

Kill All Mutants! - Dave Aronson

Better Code By Making Bugs with Mutation Testing - Theo Fidry

How to use fake dates in Cypress end-to-end testing

John Braun — Mon, 07 Sep 2020 00:00:00 +0000

Introduction

When writing an end-to-end test (or so called browser test), it might be tricky to force the application to use a certain (fake) date. In this post, I share my approach using a custom middleware to be able to write Cypress tests while your application is in a specific (fake) date.

Recently, I wanted to write a Cypress acceptance ("end-to-end") test for a Laravel project where I wanted Carbon::now() to return a specific (fake) date, so I could act as if it were that specific date in my browser test.

Initially, I thought I could just set Carbon::setTestNow(), however this would only survive a single request and I wanted the custom date I specified to be applied globally in the application to make assertions about several endpoints within these tests.

After a few online searches, I decided to ask for help on Twitter and got a reply from @sasin91 which sparked a new idea: perhaps I could use a middleware 💡.

Eventually, I ended up using the following setup.

The middleware approach

Adding a new middleware

First, I created a new middleware called SetTestDate. We want to apply this middleware within the 'web' middleware group, but only when the environment is either "local" (the default dev environment) or "testing" (the environment in .env.cypress).

Let's first conditionally push our middleware to the 'web' group within the boot() method of the AppServiceProvider as shown below.

// app/Providers/AppServiceProvider.php

use App\Http\Middleware\SetTestDate;
use Illuminate\Contracts\Http\Kernel;
use Illuminate\Support\ServiceProvider;

class AppServiceProvider extends ServiceProvider
{
    public function boot()
    {
        if ($this->app->environment(['local', 'testing'])) {
            $kernel = $this->app->make(Kernel::class);
            $kernel->appendMiddlewareToGroup('web', SetTestDate::class);
        }
    }
}

Within the SetTestDate middleware, we want to check if a certain cookie ("set_test_date" in this example) is set containing the specified (fake) date. For clarity and consistency let's store the name of this cookie in a class constant TEST_DATE_COOKIE.

// app/Http/Middleware/SetTestDate.php

<?php

namespace App\Http\Middleware;

use Carbon\Carbon;
use Closure;

class SetTestDate
{
    const TEST_DATE_COOKIE = 'set_test_date';

    public function handle($request, Closure $next)
    {
        if ($this->wantsToSetTestDate($request)) {
            $this->setDateNow($request->cookie(self::TEST_DATE_COOKIE));
        }

        return $next($request);
    }

    private function setDateNow($date)
    {
        Carbon::setTestNow(Carbon::parse($date));
    }

    private function wantsToSetTestDate($request)
    {
        return $request->cookie(self::TEST_DATE_COOKIE) !== null;
    }
}

Setting a custom (fake) date in the Cypress test

Now that we have our middleware in place, we can use the request variable we defined in our middleware to visit a route which uses the fake date.

it('shows the current date', () => {
    const date = 'tuesday 1 september 2020'
    cy.setCookie('set_test_date', date);

    cy.visit("/");

    cy.contains('Tuesday, September 1st 2020')
})

Encrypted cookies

If you'd run the test at this stage, no cookie will be resolved from the request and null will be returned for $request->cookie('set_test_date').

Since Laravel encrypts the cookies by default due to the EncryptCookies middleware, we need to create an exception for the set_test_date cookie.

// app/Http/Middleware/EncryptCookies.php
class EncryptCookies extends Middleware
{
    protected $except = [
        'set_test_date'
    ];
}

Conclusion

It is possible to manipulate the current date in Laravel end-to-end tests using a middleware that accepts a fake date within a certain cookie and calling Carbon::setTestNow() for each request that has this cookie.

When using Cypress in a Laravel application, make sure to checkout out the laracasts/cypress helper package.

To learn more about Cypress in the context of Laravel applications, I can highly recommend this video series on Laracasts.

Refactoring toward reusable Vue components

John Braun — Sun, 26 Jan 2020 14:07:14 +0000

Introduction

In this post, I want to highlight possible refactoring strategies toward resuable Vue components: renderless components vs using Provide/Inject.

To illustrate these strategies, I'll use a Dragonball Z character selectioncomponent made using VueJS and Tailwind CSS .

For this project I've used Laravel as a backend and some things might be Laravel specific.

Possible approaches

This post is divided in three sections. Each section is accompanied by a CodeSandbox demo, illustrating the approach.

The straightforward approach: It makes sense to focus on the simplest implementation first, and just get it to work. Then, refactor later. In this section we'll build the selector from scratch and I'll remind you of some basic principles. View the straightforward approach on CodeSandbox
The Renderless approach: One way to make your components (more) reusable is taking advantage of Vue's render() function. In this section I'll show you how to take full control of the way your data is displayed (rendered) within the view. View the renderless approach on CodeSandbox
The Provide/Inject approach: Although Renderless components give more flexibility than standard components, a major downside is that all code in your view gets more complicated/verbose. The Provide / Inject strategy is somewhere in the middle between the other two strategies. View the provide/inject approach on CodeSandbox

What do we want to achieve?

Show 35 avatars of various Dragonball Z characters
Make selection by clicking
Undo the selection by clicking (again)
Store the id of the selected character in hidden form field

The Character model has an id, avatar and name column in the database. The avatar column holds the relative path to the image source.

Note: I will refer to blade directives , which I will not go into detail to in this post.

1. The straightforward approach

Creating the view

Given we have access to a $characters variable, which holds an array of characters, like so:

[
  {
    id: 1,
    name: "Goku",
    avatar: "goku.jpeg"
  },
   # and so on...
 ]

We might structure our view file as follows:

<form action="/fighters" method="POST">
  <!-- other fields --> 
  <character-selection
      :characters="{{ $characters }}" 
       previous-character="{{ old('character_id') }}" 
  /> 
  <!-- submit button -->
</form>

The :characters and previous-character props we're sending through will be available in our component. We leverage Laravel's old() helper to pass the previous selection to the component (on failed submission of the form) to make sure we remember the selected character.

Creating the Vue component

After you've created the CharacterSelection.vue file, register the component globally within resources/js/app.js.

Vue.component(
    'character-selection', 
    require('./components/CharacterSelection.vue').default
 );

I won't mention this step whenever creating new components in the rest of this post, but please keep in mind that all Vue components mentioned are registered globally.

Accepting the props

In the CharacterSelection component, we'll accept the previous-character and characters props in the <script> section.

<script>
export default {
    props: ['characters', 'previous-character'],
}
</script>

Iterating over all characters

Furthermore, we want to iterate over all characters and show an image in the <template> section of our component. From the <template> we can only return one root element and therefore have to wrap everything in a parent element, e.g. a <div>.

When iterating over items, due to its reactivity, Vue needs to be able to differentiate between DOM elements which is why we also pass a unique :key attribute.

To display the avatars in a grid, we employ flexbox by adding the tailwind CSS classes 'flex' and 'flex-wrap' to wrap as necessary.

The images are displayed at a predefined width and height (w-16 and h-12), also using Tailwind CSS classes.

<template>
  <div class="flex flex-wrap">
    <div 
       v-for="character in characters" 
       :key="character.id"
    >  
      <img
        class="w-16 h-12" 
         :src="`/images/fighters/${character.avatar}`" 
        :title="character.name"
        :alt="character.name" 
      > 
    </div>
  </div>
</template>

Adding reactivity

Although we can see the avatars now, there is no reactivity. To help us achieve that, we should employ a dedicated <single-character> Vue component representing a single avatar rather than a <img>. This child component will receive the character as a prop.

<template>
    <div class="flex flex-wrap justify-center">

      <single-character
        v-for="character in characters"
        :character="character"
        :key="character.id"
        />

    </div>
</template>

In our SingleCharacter child component, we first need to make sure to render all avatars properly. We accept the current character and show the image:

<template>
  <div 
    class="(omitted for clarity...)"
   >

    <img :src="avatar" />

  </div>
</template>

<script>
export default {
  props: ['character'],

  computed: {
    avatar() {
      return `/images/fighters/${this.character.avatar}`;
    },
  },
};
</script>

Now that the avatars are showing up, let's add some reactivity by adding a click handler (@click) and let the parent component know that we've made a choice by emitting an event (this.$emit) sending along the character's id.

<template>
  <div 
    class="(omitted for clarity...)"
    @click="selectCharacter"
   >
    <img :src="avatar" />
  </div>
</template>

<script>
export default {
  // ...
  methods: {
    selectCharacter() {
      this.$emit('character-selected', this.character.id);
    },
  },
  // ...
};
</script>

To make the parent component capable of listening and acting on this event, we'll need to make some adjustments.

Listening to the 'character-selected' event

First, we need to listen for an event called character-selected by specifying an attribute on our child component: @character-selected which will call a method that sets the internal property selectedCharacter of the parent component to the selected character's id.

This value is then bound using v-bind to the hidden input field using the :value attribute. If the selected id was already selected, we set the selectedCharacter property to null.

<template>
  <div>
    <div class="flex flex-wrap justify-center">
      <single-character
        @character-selected="selectCharacter"
        v-for="character in characters"
        :character="character"
        :key="character.id"
        :selected="selectedCharacter === character.id"
        />
    </div>

    <input 
      type="hidden" 
      name="character_id" 
      :value="selectedCharacter" />
  </div>
</template>

<script>
export default {
    props: ['characters', 'previous-character'],

    data() {
        return {
            selectedCharacter: parseInt(
                    this.previousCharacter
            )
        }
    },

    methods: {
        selectCharacter(id) {
            if (this.selectedCharacter === id) {
                return (this.selectedCharacter = null);
            }

            this.selectedCharacter = id;
        },
    },
}
</script>

Tell the child which character is currently selected

Lastly, our child component needs to know if it is currently selected. Therefore, we also pass the :selected attribute. Within the child component we can accept the value as a prop and let it determine the classes which need to be applied.

<template>
  <div
    @click="selectCharacter"
    class="(omitted for clarity...)"
    :class="classes"
  >

    <img :src="avatar" />

  </div>
</template>

<script>
export default {
  props: ['character', 'selected'],

  methods: {
    selectCharacter() {
      this.$emit('character-selected', this.character.id);
    },
  },

  computed: {
    avatar() {
      return `/images/fighters/${this.character.avatar}`;
    },

    classes() {
      return this.selected ? 
        `border-2 border-black shadow-lg opacity-100` :
        `${this.selected ? 'opacity-35' : 'opacity-85'}`;
    },
  },
};
</script>

This concludes our first approach and we have a basic component which does exactly what we want. However, the current implementation is tightly bound to our specific grid of 7 x 5 and contains terminology like 'character' and 'fighters'. What if we want to use this component in a quiz about animals?

2. Refactoring to a renderless component

Like we concluded from the previous section, the straightforward example works but is hard to reuse. All details regarding styles, layout and image paths of the avatars are hard coded in the components. If those change, we need to create a new component. That's not what we want.

Adam Wathan has a great post (and awesome videos !) that describe the goal of renderless components pretty well: "Separating Presentation and Behavior".

Ideally, we want to be able to configure (some) behaviour from our view directly as indicated in the HTML below (which will not work, yet). This way, the Vue component allows itself to be "decorated" from within the layout file.

The create.blade.php view file:

<form action="/fighters" method="POST"> 

  <image-selector 
       pre-selected="{{ old('character_id') }}"
      :images="{{ $characters }}"
      selected-classes="(omitted for clarity...)"
       once-selected-classes="(omitted for clarity...)"
  > 

      <div v-for="character in images" :key="character.id">
        <!-- render each image here --> 
      </div>  

  </image-selector>  

  <button type="submit">Submit the form</button>
</form>

Notice that the component's name and the name of the props are more generic. Instead of characters, we now have an images prop. Although we changed the name of the prop, we still want to pass our original source of $characters.

Looping over the images

From here on, I assume you already understand what Vue's slots are and how they work.

Since we try to iterate over a images variable which we don't have access to, the listed code above will not work. Here is where scoped slots come into play.

A scopedslot allows us to pass data from a parent component to a child. The child might then use that data in rendering the slot of the parent component.

Creating the renderless component

Since the templating part will be managed by the layout file and passed back via a single scoped slot, we don't need to provide a template for our component. By definition, renderless components don't have a <template> section, just a <script> section.

Here we can still accept any props and declare the render() function, which will pass any variables ('slot props') back to the child component.

For our images prop, that would look as follows, in a more generically named ImageSelector.vue component:

<script>
export default {
    props: ['images'],

    render() {
       return this.$scopedSlots.default({
          images: this.images
       });
    }
}
</script>

Using the slot props in our layout file

To loop over the images in our current layout file, we will need to extract the images variable from the slot scope. Note that the syntax has been updated in Vue 2.6.0+ from using slot-scope to v-slot (more info).

create.blade.php:

<form action="/fighters" method="POST"> 

  <image-selector 
       pre-selected="{{ old('character_id') }}"
      :images="{{ $characters }}"
      selected-classes="(omitted for clarity...)"
       once-selected-classes="(omitted for clarity...)"
  >

       <template v-slot="{ images }">
        <!-- we need to return a single root node -->
         <div class="flex">      

          <div v-for="character in images" :key="character.id">
            <!-- render each image here --> 
          </div>  

        </div> 
      </template> 

  </image-selector>  

  <button type="submit">Submit the form</button>
</form>

To bring the component to life, i.e. adding reactivity, let's add the props, data() and methods() from the SingleCharacter.vue and CharacterSelection.vue components (from the previous, 'straightforward' approach) and insert them above our render() function.

Sharing data via the render() function

To keep the component as generic as possible, we'll catch any events using Vue's v-on directive and proxy them through to the imageEvents() method, which registers handlers for specific actions. In our case, the click event. The props are passed through using imageProps().

Since we want to use v-bind on the input field, we'll also need to offer the selected image as a value through the inputProps() method.

ImageSelector.vue:

<script>
  export default {
    props: [
      'images', 
      'preSelected', 
      'selectedClasses',
      'onceSelectedClasses',
    ], 

    data() {
      return {
        selectedImage: parseInt(this.preSelected),
      }
    },

    methods: {
      selectImage(image) {
        if (this.selectedImage === image.id) {
          return this.selectedImage = null;
        }

        this.selectedImage = image.id;
      },

      classes(image) {
        return this.selectedImage === image.id ? 
          this.selectedClasses : 
          `${this.selectedImage ? this.onceSelectedClasses : '' }`
      },
    },

    render() {
      return this.$scopedSlots.default({
        images: this.images,

        inputProps: {
          value: this.selectedImage,
        },

        imageProps: (image) => ({
          selected: this.selectedImage,
          class: this.classes(image),
        }),

        imageEvents: image => ({
          click: () => {
            this.selectImage(image)
          }
        }),
      });
    }
  }
</script>

Use variables in the view

Now, from within our create.blade.php view file we can leverage destructuring to obtain each key.

<form action="/fighters" method="POST">

  <image-selector 
    :images="{{ $characters }}" 
    pre-selected="{{ old('character_id') }}" 
    selected-classes="border-2 border-black shadow-lg opacity-100"
    once-selected-classes="opacity-35"
  >
    <template 
      v-slot="{ 
        images, 
        imageProps, 
        imageEvents, 
        inputProps 
      }"
     >

       <div class="single-root-element">
         <div class="flex flex-wrap justify-center">

           <div 
             v-for="character in images" 
            :key="character.id"
            class="(classes for an image...)" 
            v-on="imageEvents(character)"
            v-bind="imageProps(character)"
           >

             <img 
               :src="`/images/fighters/${character.avatar}`"
            >
           </div>

         </div>

         <input 
           type="text" 
           name="character_id" 
           v-bind="inputProps"
         />
       </div>
    </template>
  </image-selector>

  <button type="submit">Submit form</button> 
</form>

We now have a customizable component which can be reused amongst other projects.

A downside to this approach is that it generates a lot of extra code and it results in a somewhat more complex layout file.

However, since we have a configurable, reusable component it is possible to generate pre-configured components which are composed by one or more renderless components.

3. Provide / Inject approach

Vue offers another solution, which falls somewhere in the middle between the "straightforward" and renderless approach. It gives more flexibility to configure the component in the view template, while generating a less verbose layout file.

In our create.blade.php view, we render a <single-character> component within a <character-selection> wrapper:

<character-selection 
  previous-character="{{ old('character_id') }}"
 >
    @foreach ($characters as $character)
        <!-- Looping over Single Character component -->
        <single-character
          class="(omitted for clarity...)" 
          :character="{{ $character }}"
          imageurl="/images/fighters/{{ $character->avatar }}"
          selected-classes="(omitted for clarity...)"
         />
    @endforeach
</character-selection>

At this stage, all looks very familiar to our first approach, except that we're taking care of looping over <single-character> in the view instead of in the <character-selection> Vue parent component.

Now, instead of emitting an event from our child component to the parent, state will be shared in a characterSelectionState property. This property will be provided by the parent component and injected into the child component.

The child component, however, can manipulate this shared (reactive) property

Here, we provide a shared variable characterSelectionState using the provide() method in CharacterSelection.vue:

<template>
   <div>
       <div class="flex flex-wrap justify-center">
           <slot></slot>
       </div>

       <input 
          type="hidden" 
          name="character_id" 
          :value="sharedState.selectedCharacter"
       >

   </div>
</template>

<script>
   export default {
       props: ['previous-character'],

       provide() {
           return {
               characterSelectionState: this.sharedState
           }
       },

       data() {
           return {
               sharedState: {
                   selectedCharacter: parseInt(
                      this.previousCharacter
                     ),
               }
           }
       },
   }
</script>

In SingleCharacter.vue we inject the characterSelectionState variable, making it available:

<template>
   <div @click="selectCharacter" :class="classes">
       <img :src="imageurl">
   </div>
</template>

<script>
   export default {
       props: ['character', 'imageUrl', 'selectedClasses'],

       inject: ['characterSelectionState'],

       methods: {
           selectCharacter() {
               this.characterSelectionState
                 .selectedCharacter = this.active ? 
                      null : this.character.id;
           }
       },

       computed: {
           active() {
               return this.characterSelectionState
                  .selectedCharacter === this.character.id;
           },

           classes() {
               return this.active ? 
                     this.selectedClasses : '';
           },
       }
   }
</script>

Conclusion

In conclusion, Vue offers a couple of nice possibilities which allows us to write reusable components:
Renderless components give full control over how they are rendered since the behaviour is completely decoupled.

However, in the end, you do create a more complex component and you end up with more verbose code in your view template.

Vue's Provide/Inject methods can be a useful intermediate solution if you don't need a full blown completely configurable renderless component, but still want some configurational flexibility.

Each of the three approaches mentioned in this post have their use. Each approach has their advantages and downsides.

The power of Renderless Components

Disclaimer: I would advise to always go with the simplestoption for your specific needs. Don't create a renderless component when all you need is a simple component that you'll only use once.

View the multiple images approach on CodeSandbox

Selecting Multiple Images, using our renderless component

So far, we can use our renderless component to display any set of images in a particular way.
However, what if we want to select multiple images?

With a little tweak to our renderless component, we can come up with the following solution in create.blade.php:

<multiple-image-selector 
 :images="{{ $characters }}"
 selected-classes="border-2 border-black shadow-lg opacity-100"
 once-selected-classes="opacity-35"
>
  <template v-slot="{ 
     images, 
    imageProps, 
    imageEvents, 
    inputProps 
   }"
  >

    <div class="single-root-element">

      <div class="flex flex-wrap justify-center">

        <div 
          v-for="character in images" 
          :key="character.id"
          class="(omitted for clarity...)" 
          v-on="imageEvents(character)"
          v-bind="imageProps(character)"
         >

           <img :src="`/images/fighters/${character.avatar}`">

         </div>

      </div>

      <input 
        type="text" 
        name="character_id" 
        v-bind="inputProps"
      >

    </div>

  </template>
</multiple-image-selector>

Then, in our renderless MultipleImageSelector.vue component:

<script>
  export default {
    props: [
      'images', 
      'selected-classes', 
      'once-selected-classes'
    ],

    data() {
      return {
        selectedImages: [],
      }
    },

    methods: {
      selectImage(image) {
        let index = this.selectedImages.indexOf(image.id);

        if (index > -1) {
          return this.selectedImages.splice(index, 1);
        }

        return this.selectedImages.push(image.id);
      },

      classes(image) {
        return 
          this.selectedImages.indexOf(image.id) > -1 ? 
            this.selectedClasses : 
              `${this.selectedImages.length > 0 ?
                 this.onceSelectedClasses : '' }`
      },
    },

    render() {
      return this.$scopedSlots.default({
        images: this.images,

        inputProps: {
          value: this.selectedImages,
        },

        imageProps: (image) => ({
          selected: this.selectedImage,
          class: this.classes(image),
        }),

        imageEvents: image => ({
          click: () => {
            this.selectImage(image)
          }
        }),
      });
    }
 }
</script>

OAuth2 authentication across Laravel projects

John Braun — Sun, 26 Jan 2020 13:24:21 +0000

Introduction

A question popping up every now and then is how to let users log in to separate (child) applications using a single account they own on a central application.

In this post I try to explain how to achieve this infrastructure, by creating a central application using Laravel Passport (example.com) where users register once and then use OAuth2 to grant access to their account to the other applications (app1.example.com, app2.example.com, etc.) using Laravel Socialite . In this way, users will be able to login to the child applications without creating a new, separate account.

Note: the applications do not necessarily have to use the same domain.

If you're not yet familiar with the OAuth2 protocol, I've included a section on the how and what below.

What is OAuth2?

Before jumping into Laravel Passport, it is important to understand the OAuth protocol it implements.

OAuth is an open standard, designed to provide API access delegation. Think of using a third party Twitter app which can tweet on your behalf to the Twitter platform.

I explicitly mention Twitter since development of this standard was (amongst others) driven by lead developer Blane Cook, in need of authorization of external parties.

After initial release of version 1.0 in 2010, the protocol matured over the course of 2 years after which version 2.0 was released. This improved protocol offers support for Bearer tokens and provides "specific authorization flows for web applications, desktop applications, mobile phones, and smart devices." (via Wikipedia)

Let's have a look at what is meant with the terms "Bearer token" and "authorization flow" .

Bearer tokens

The word "Bearer" means you're in possession of a certain (access) token. The bearer being the third party application, which possesses an access token issued by the identity provider.

This token provides immediate access to a resource, without requiring a username and password.

All necessary information is linked to this token, including user details and the scope of possible actions this third party may perform on behalf of this user.

The bearer token is usually included in the Headers of a GET or POST request to an API endpoint.

A concrete example of using the Bearer token is shown below, making a GET request to /api/user using the Guzzle HTTP library to a corresponding API endpoint sending along a Bearer Authorization header.

$response = $client->request('GET', '/api/user', [
    'headers' => [
        'Accept' => 'application/json',
        'Authorization' => 'Bearer ' . $accessToken,
    ],
]);

Authorization flow

Now that we know what a Bearer access token is ... how do we obtain one?
First, let's look at the formal roles in OAuth2:

Resource Owner: the user who wants to login and delegate access to their account details to a third party application
Resource Server: the server (API) where the user has an account
Client: the third party application that wants to access the account information on the resource server

The OAuth 2.0 protocol performs a standard communication flow between Client and Resource Server, where each step and given/required parameters are defined in advance.

Ultimately, the Client receives the Bearer access token from the Resource server.

This process is illustrated in the figure below (created using draw.io ).

Note: it is assumed that the Client (third party application) is registered with the Resource Server.

After this little "dance", the Client now possesses an access token which can either be long-lived or short-lived (more secure).

If the access token is short-lived, the Resource Server will also provide a refresh token which can be used - in combination with the client secret - to obtain a new access token in a way resembling Step 3 in the diagram above.

Note: the "state" parameter is used to prevent CSRF related attacks, by verifying the request is not forged.

Now, let's see how Laravel Passport implements this protocol.

Laravel Passport

In case of our example, we want the identity provider (example.com) to make use of Laravel Passport.

Laravel Passport is an OAuth2 server, built upon the League OAuth2 server . It provides an easy implementation for existing Laravel applications by requiring the composer package.

Resource server

To set up a resource server, follow the clear installation instructions and/or watch this explanation on Laracasts by Taylor Otwell. Don't forget to add the hasApiTokens trait to your User model.

After installation, you now have the possibility to add new Clientshaving a callback url and an automatically generated secret. For each of the "child" applications (app1.example.com, app2.example.com, etc.) you need to create a new Clientwith their own callback, for now you might choose https://app1.example.com/login/callback for example (we will come back to this).

Laravel Passport will take care of the authorization dialog, providing an authorization code, verifying the client secret in combination with the authorization code and lastly provide a User object and (by default) a long-lived access token. The lifespan of the access and refresh tokens are configurable .

Laravel Socialite

Now that we have a Resource Server (identity provider), we need to take care of the Clientside of things using Laravel's package called Laravel Socialite .

Out of the box, it allows authentication with the services of Facebook, Twitter, LinkedIn, Google, GitHub, GitLab and Bitbucket. However, there is a collection of additional providers , amongst which an adapter supporting Laravel Passport.

To achieve a shared login system across multiple Laravel applications, my proposed solution involves making use of the Socialite provider for Laravel Passport.

Looking at the installation instructions of the Socialite provider, there are quite a number of steps, to make sure a client application is able to identify users using the Laravel Passport identity provider.

Install Laravel Socialite
Install Laravel Passport provider for Socialite
Add methods to LoginController
Copy the 'laravelpassport' config to config/services.php
Add SocialiteWasCalled event and listener to EventServiceProvider

Now, this seems like a lot of steps to repeat for every Client application we might already have and want to couple to our Resource Server. That's why I created a support package, as described in the next section.

Socialite-Passport support package

To simplify the process, I made a support package ) that combines Laravel Socialite with the Passport driver and can be configured in a more efficient way.

Installation

In a Client Laravel application that you want to couple to the Passport*Resource Server*, first install the package:

composer require jhnbrn90/socialite-passport

Next, publish the configuration file:

php artisan vendor:publish --provider="JhnBrn90\SocialitePassport\SocialitePassportServiceProvider" --tag="config"

This will publish a file socialite-passport.php in the config directory, in which you can define which Controller (defaults to LoginController that ships with Laravel) and which method should be called when the login route (also configurable) is called.

return [
    'controller' => [
        'class' => \App\Http\Controllers\Auth\LoginController::class,
        'method' => 'loginWithPassport',
    ],

    'route' => [
        'name' => 'login',
        'uri' => '/login',
    ],
];

Assuming the defaults in the above configuration, after Authorization is granted by the Resource Owner at the Resource Server (Laravel Passport), the method loginWithPassport() will be called and a User object will be injected.

This method needs to be implemented in the defined controller (in our example, the LoginController):

class LoginController extends Controller 
{
    public function loginWithPassport($user) 
    {
        // example:
        User::firstOrCreate(['name' => $user['name'], 'email' => ...]);
    }
}

To be able to provide the Resource Server with a client_id, callback_uri, etc. you must add these variables to your .env file:

LARAVELPASSPORT_CLIENT_ID=
LARAVELPASSPORT_CLIENT_SECRET=
LARAVELPASSPORT_REDIRECT_URI=/login/callback
LARAVELPASSPORT_HOST=https://example.com

Remember, LARAVELPASSPORT_CLIENT_ID and LARAVELPASSPORT_CLIENT_SECRET come from the Laravel Passport Resource Server, where the Client needs to be created first.

The package will make sure to match the route you provide as the LARAVELPASSPORT_REDIRECT_URI environment variable and proxies the request through the corresponding method and controller you've configured in the config file.

Final remarks

That is all it takes to implement a basic functional shared login system.

I hope this post could shed some light on the OAuth2 protocol and how it can be used with the mentioned tools to achieve a shared authentication system amongst different Laravel projects.

It might be a good idea to also store the access token (and the resource token) on the user model to be able to update information whenever a user changes its profile on the Resource Server. Or to collect other data of that user from the Resource Server of course.

Summary

Laravel Passport implements a fully functional OAuth2 server (Resource Server)
Laravel Socialite implements a way to authenticate with an OAuth2 server (Client)
The Socialite provider for Laravel Passport allows authentication with Laravel Passport
Clients can be prepared using this package combining both Socialite with the Passport adapter

Show waiting users what's happening in the background

John Braun — Sun, 26 Jan 2020 13:03:11 +0000

Recently, Miguel Piedrafita (https://dev.to/m1guelpf) tweeted about the importance of including some form of progress indicators whenever your application needs to perform one or more slow (background) tasks/processes.

This inspired me to write this blog post. In his example, the users' website needs to be saved to a database, added to a deployment platform and dispatched for first deploy.

Like Miguel mentions, by adding small indicators to each of these running processes your users are reassured something is happening and they just need to be patient.

After I reading this tip, I wondered how to achieve these progress indicators. In this post I want to share my approach, using VueJS.

I do not claim this to be the best available option and I'm open to alternative solutions and improvements.

The code is available on CodeSandbox.

My Approach

Since we'll need to update the progress in realtime, I like to defer to VueJS, my javascript framework of choice.

Ultimately we want to display a list of tasks, which are processed sequentially. To this extent we'll leverage javascript's async/await functionality.

Additionally, the tasks should indicate whenever they're finished and show an epic spinner (by Epicmax ) in the meantime. An example of our desired end result is shown below:

Object representation of a single process

To achieve this, I was thinking of the following object representation of a single process: we specify a name, the work that needs to be done with a callback (returning a Promise) and lastly keep track of its state through a finished boolean.

{
    name: 'Collecting credentials',
    work: () => new Promise((resolve) => {
        // perform the task

        // resolve the promise
        resolve()
    }),
    finished: false,
}

Note: we're only passing a resolve argument to the Promise object for now, ignoring potential failures. Make sure to check out the "not-so-happy-path" section in the Conclusion on managing (potential) errors.

Building the Vue component

With this approach and end goal in mind, we can shape our basic Vue component, in which we'll register three processes: 'Collecting credentials', 'Saving to database' and 'Finishing up registration'. For this demo, let's simulate performing work by a setTimeout function, waiting for 2 seconds (2000 ms):

<script>
// if we want to use the epic spinner, let's import it here
import { LoopingRhombusesSpinner } from 'epic-spinners';

export default {
  // and declare the epic spinner component here
  components: {
    LoopingRhombusesSpinner
  }, 

  data() {
    return {
      processes: [
        {
          name: 'Collecting credentials',
          work: () => new Promise(resolve => {
            setTimeout(() => resolve(), 2000);
          }),
          finished: false, 
        },

        {
          name: 'Collecting credentials',
          work: () => new Promise(...),
          finished: false, 
        },

        {
          name: 'Collecting credentials',
          work: () => new Promise(...),
          finished: false, 
        },
      ]
    }
  }
} 
</script>

Now we have access to our loading spinner and the processes property, we can generate a list of processes in the template:

<template>
    <ul>
      <li :key="process.name" v-for="process in processes">
        {{ process.name }}

        <span v-if="process.finished">&check;</span>

        <looping-rhombuses-spinner v-else
          style="display: inline-block;"
          :animation-duration="2500"
          :rhombus-size="6"
          color="#ff1d5e"
        />
      </li>
    </ul>
</template>

With the template in place, we need to make sure that our processes start whenever the page is loaded. We can do so by hooking in to Vue's mounted() lifecycle hook.

<script>
export default {
  components: {
  // ...
  },

  data() {
    //...
  },

  mounted() {
    this.initialize();
  },

  methods: {
    async initialize() {
      for await (const process of this.processes) {
        await process.work();
        process.finished = true;
      }
    }
  }
} 
</script>

In the initialize() method (which is called when the component was created) we want to loop over the processes and perform the work of each process in sequential order by handling the promises one by one using await before continuing to the next task. This requires that we declare the method as async initialize().

Since we are working with Promises, we can not simply use a forEach loop to iterate over the processes. Instead, we use a for/of loop ( MDN reference ) which allows us to iterate over interable objects (in our case the asynchronous processes).

After the work has been performed, we'll mark a process as finished which dynamically updates the loading spinner to a checkmark.

Conclusion

We've made a basic process indicator Vue component, in which we can define multiple tasks by declaring a name, (initial) state and a callback.

The tasks are then executed sequentially and their "finished" state is updated in realtime.

For our demo purposes we've simulated the workload with a setTimeout(), however in real life this would probably be an AJAX call which could look as follows (using axios):

{ 
  work: () => new Promise(resolve => {
    axios.get('https://www.johnbraun.blog').then((response) => {
    // (optional) do something with the response ...
    resolve();
    })
  }),
}

💡 You might want to add additional tasks at runtime, which you can easily do by adding the following method to your component:

<script>
  // ...
   methods: {
    addProcess(name, callback) {
        this.processes.push({
            name: name,
            work: callback,
            finished: false,
        });
        return this;
    }
  }
</script>

Handling queued processes in the backend

Now, there might be situations where the processes are queued on your backend. In that scenario, the above discussed frontend solution doesn't suffice and I would advise to defer to a WebSocket implementation.

WebSockets allow realtime communication from the backend to the frontend. You might have a look at my post on using WebSockets in Laravel , which explains how to communicate changes in a queued job back to the frontend.

The not-so-happy path

So far, we only discussed the happy path, but let's be realistic: what if a process fails?

Javascript's Promise() object accepts - in addition to 'resolve' - another argument 'reject' to indicate failure.

In this regard, we should dissect the code for a single "process" in our component into a part that resolves the promise when succesful, and rejects the promise upon failure.

{
  name: 'Saving to database',
  work: () => new Promise((resolve, reject) => {

    axios.get('https://www.johnbraun.blog')
        .then((response) => resolve(response))
        .catch((error) => reject(error));

    }),
  finished: false,
},

Additionally, you might want to halt all other sequential processes since we're dealing with a failure. One way to go about it is to delete all remaining processes from the array, making sure the .work() method is not called. We also should store the process which failed to show a big red cross.

To accommodate for potential failure, one way to go about it is to let the user know which process failed (with a big red cross) and delete all remaining, unexecuted processes from the array preventing the remaining .work() methods being called. Additionally, we store the process in an 'errors' property so we can show the user which process failed.

These changes are summarized in the code block below, and also available on the Codesandbox page .

<template>
<div>
  <ul>
    <li :key="process.name" v-for="process in processes">
      {{ process.name }}

      <span v-if="process.finished" style="color: green;">&check;</span>
      <span v-if="errors.includes(process)" style="color: red;">&cross;</span>

      <looping-rhombuses-spinner v-if="!process.finished && !errors.includes(process)"
        style="display: inline-block;"
        :animation-duration="2500"
        :rhombus-size="6"
        color="#ff1d5e"
      />
    </li>
  </ul>

  <p v-if="errors.length > 0" style="color:red;">
    Something went wrong, so we bailed...
  </p>
</div>
 </template>

<script>
export default {
  data() {
    return {
      // keep track which process(es) failed
      errors: []
    }
  },

  methods: {
    async initialize() {
      for await (const process of this.processes) {
        await process.work()
          .then(() => {
            this.markFinished(process);
          })
          .catch((reject) => {
            this.errors.push(process);

            this.haltIteration(process);
      });
    },

    haltIteration(process) {
      // get the current item's index
      let index = this.processes.indexOf(process);

      // determine how many processes are left
      let items = this.processes.length;

      // remove other processes from being executed.
      this.processes.splice(index + 1, items - 1);
    }

  }
} 
</script>

Handle incoming mail in Laravel

John Braun — Sun, 26 Jan 2020 12:41:11 +0000

Introduction

Sending e-mails with laravel couldn't be simpler using the Mail facade, but what if you want to act on incoming e-mails instead?

For example, if you want to build a helpdesk system, where users can respond to their tickets directly via e-mail. Or maybe you want to publish posts to your blog via e-mail.

That's where the 📬 Laravel Mailbox package by Marcel Pociot comes in!

In this post, I want to guide you through the steps to create an application which listens to incoming e-mails and perform an action based on its sender (from), recipient address (to) or subject. Alternatively, you can also catch all incoming mail.

Requirements

Domain name and server: You'll need to own a domain name with access to its DNS management, a server to host the application and a mail service that listens to incoming e-mail and forwards them to the application.
Mailgun account: Laravel Mailbox offers support for the mail services Mailgun , Sendgrid and Postmark out of the box. For this article, I chose to use Mailgun since I am most familiar with their service and I think they offer a reasonably generous free plan.
PHPUnit: Since I'll demonstrate driving out the desired functionality using TDD, you will need to have PHPUnit working on your system. If you are new to this, I'd recommend to check out this free video on Laracasts.

Note: references to "example.com" or "johnbraun.blog" should be replaced by your own domain.

Setting up Mailgun

Plans

Mailgun offers a Free plan which allows you to send 10 000 e-mails / month, limited to a sandbox domain. If you provide your payment details (credit card) you'll be upgraded to the Concept plan, which is still free but unlocks the ability to add custom domains. Be aware that you'll be charged if you send more than the 10 000 free e-mails. You're allowed to receive an unlimited number of e-mails.

Adding a domain

Visit the domains page in the Sending menu section and choose Add your domain. Mailgun will suggest you to use a subdomain like mg.example.com. However, if we want to be able to receive e-mails on info@example.com (instead of info@mg.example.com) we'll need to ignore the red warning and register example.com. Choose your region (EU / US) and confirm.

Setting up DNS records

After adding a new domain, you'll be asked to add some DNS records:

TXT records (2x)
MX records (2x)
CNAME record (1x)

Those are (most probably) managed by the company you've registered your domain name with.
As long as your registrar supports manual configuration of the DNS records it doesn't matter which company you choose. Personally, I use Hover for registration of my domains and DigitalOcean to mange the DNS settings (by directing the nameservers to DigitalOcean) since my server lives there as well.

Check out these guides for specific configuring of DNS records at GoDaddy , NameCheap , Network Solutions , Rackspace Email & Apps , Rackspace Cloud DNS , Amazon Route 53 and Digital Ocean .

TXT records

To be able to send e-mail with Mailgun, you'll need to configure two TXT records.

First, using the root hostname (using an "@" in DigitalOcean) with the value "v=spf1 include:eu.mailgun.org ~all".

The second TXT record indicates your domain key using the provided subdomain prefix, "mailo._domainkey" in the example below. Note: The specific subdomain you'll need to create will vary.

MX records

Next, to be able to receive e-mails, we'll need to add two MX records.

Since we're not using a subdomain, we'll add those MX records on our root hostname, which is shown below (for DigitalOcean). Important: if you do use a subdomain, you'll need to configure the MX records for the subdomain.

Repeat this step for the other (mxb.eu.mailgun.org) MX record.

CNAME record

Lastly, add a CNAME record for the "email" subdomain:

Verify DNS Settings

After you've configured the DNS records, hit 'verify DNS settings' on the Mailgun page after which a green tick should appear, indicating that everything was properly configured.
You're now ready to send and receive incoming mail on this domain.

Installing Laravel Mailbox

Pull in the Laravel Mailbox package in your Laravel app using composer.

composer require beyondcode/laravel-mailbox

Further more, following the installation instructions, at the documentation page:

Publish the migrations

php artisan vendor:publish --provider="BeyondCode\Mailbox\MailboxServiceProvider" --tag="migrations"

Run the migrations

php artisan migrate

Publish the config file

php artisan vendor:publish --provider="BeyondCode\Mailbox\MailboxServiceProvider" --tag="config"

Now that the package is ready, let's dive into driving out some functionality using TDD (Test Driven Development).

Testing

For this demo we will simply record a received e-mail in a received_mails table containing a 'sender', 'subject' and 'body' of the e-mail. We'll drive out this behaviour using a feature test.

php artisan make:test IncomingMailTest

Setting up our test

Environment file

To be able to test this package locally, instead of receiving a call to the Mailgun webhook, the Laravel Mailbox package can listen to e-mails sent to the log. Therefore, we can specify MAIL_DRIVER=log and MAILBOX_DRIVER=log in the .env file.

Using the log driver in our tests

In the setup method of our test, we have to explicitly tell PHPunit to use the log driver, otherwise it will default to the "array" driver.

class IncomingMailTest extends TestCase
{
   use RefreshDatabase;

   public function setUp(): void
   {
       parent::setUp();

       config(['mail.driver' => 'log']);
   } 
}

Writing a test by wishful thinking

Let's write a test that will verify incoming e-mails are saved to the database, so we can later act on them. Note: not all of the code described exists yet, we'll let our test tell us to implement / write the missing pieces.

<?php

namespace Tests\Feature;

use App\ReceivedMail;
use App\Mail\TestMail;
use Illuminate\Support\Facades\Mail;

class IncomingEmailTest extends TestCase
{

  + setUp() ... 

  /** @test **/  
  function incoming_mail_is_saved_to_the_mails_table()
  {
    // Given: we have an e-mail 
    $email = new TestMail(
            $sender = 'sender@example.com',
            $subject = 'Test E-mail',
            $body = 'Some example text in the body'
        );

    // When: we receive that e-mail
    Mail::to('incoming@johnbraun.blog')->send($email);

    // Then: we assert the e-mails (meta)data was stored
    $this->assertCount(1, ReceivedMail::all());

    tap(ReceivedMail::first(), function ($mail) use ($sender, $subject, $body) {
        $this->assertEquals($sender, $mail->sender);    
        $this->assertEquals($subject, $mail->subject);    
        $this->assertContains($body, $mail->body);    
    });
  }
}

Note: Want to know more about Laravel's tap() function? Check out Taylor Otwell's post on one of his favorite helpers.

Create a mailable

The first error we'll encounter when running the tests is that there is no IncomingTestMail mailable class, so let's create it:

php artisan make:mail TestMail --markdown="emails.tests.testmail"

Open up the mailable, and accept the sender's address, subject and body in the constructor and store them in the public properties.

class TestMail extends Mailable
{
    public $sender;
    public $subject;
    public $body;

    public function __construct($sender, $subject, $body)
    {
        $this->sender = $sender;
        $this->subject = $subject;
        $this->body = $body;
    }

    public function build()
    {
        return $this
            ->from($this->sender)
            ->subject($this->subject)
            ->markdown('tests.emails.testmail');
    }
}

In the markdown e-mail, echo out the body:

@component('mail::message')

{{ $body }}

Thanks,<br>
{{ config('app.name') }}
@endcomponent

Create a ReceivedMail model

If we run the test again at this stage, we're missing a ReceivedMail model, so let's create it, together with a migration.

php artisan make:model ReceivedMail -m

First, make all properties on our model fillable.

class ReceivedMail extends Model
{
    protected $guarded = [];
}

Next, add the 'sender', 'subject' and 'body' columns to our received_mails_table migration file.

public function up()
{
    Schema::create('received_mails', function (Blueprint $table) {
        $table->bigIncrements('id');
        $table->string('sender');
        $table->string('subject');
        $table->text('body');
        $table->timestamps();
    });
}

Run the migration with php artisan migrate.

Defining a Mailbox

Running the test again, shows the following error, as we don't act on incoming mail yet:

Failed asserting that actual size 0 matches expected size 1.

We can easily set up a listener in the AppServiceProvider, referring incoming e-mails addressed to a certain address (Mailbox::to()) or coming from a specific address (Mailbox::from()) to a callback or - as we'll use in this case - an invokable MailHandler class.

use App\MailHandler;
use BeyondCode\Mailbox\Facades\Mailbox;

class AppServiceProvider extends ServiceProvider
{
    public function boot()
    {
        Mailbox::to('incoming@johnbraun.blog', MailHandler::class);
    }
}

To learn all of the available options, check out this page in the documentation.

Note: you are able to use parameters (see docs) in the matching rules, which will be passed down to the handling class as additional arguments to the InboundEmail $email.

Create an invokable MailHandler class

In the app folder, let's create MailHandler.php and set up a magic __invoke( method, which gets passed in an InboundEmail object where we can call a from(), subject() and text() methods to obtain the corresponding data. All available methods are listed in the documentation.

<?php

namespace App;

use App\ReceivedMail;
use BeyondCode\Mailbox\InboundEmail;

class MailHandler
{
    public function __invoke(InboundEmail $email)
    {
        ReceivedMail::create([
            'sender'    => $email->from(),
            'subject'   => $email->subject(),
            'body'      => $email->text(),
        ]);
    }
}

If we run our test now, it passes which means we can now listen for e-mails to 'incoming@our-domain.com', however we still first need to setup an endpoint for Mailgun, which we'll do in the next section.

Setting up for production

Installing Guzzle

Since Laravel requires GuzzleHTTP for working with the Mailgun driver, let's first install this dependency:

composer require guzzlehttp/guzzle

Configuring a webhook in Mailgun

We're almost ready to receive inbound e-mails on our domain, we just need to configure Mailgun to call our application whenever it receives mail. To do this, first push your application to the domain and make sure it is accessible (externally).

In Mailgun, hit up the "receiving" link in the menu and choose "create route". Here we can specify to either catch all e-mails, or a selection. Let's only catch e-mails sent to "incoming@johnbraun.blog".

Check "store and notify" and fill in the endpoint for the Mailgun driver (as described here ): https://[your-domain.com]/laravel-mailbox/mailgun/mime.

Give your route a description and hit the create button.

Update production environment variables

Lastly, we'll need to specify the Mailgun driver in the environment file. Add/update the following properties in your .env file. The Mailgun secret key is obtained from the "API security" page under the "settings" tab. You'll need to provide the private key.

MAILBOX_DRIVER=mailgun
MAILBOX_MAILGUN_KEY=......

MAIL_DRIVER=mailgun
MAILGUN_DOMAIN=johnbraun.blog
MAILGUN_SECRET=........
MAILGUN_ENDPOINT="api.eu.mailgun.net"

Ready for action

At this stage, you're able to send mails to your defined address (in our demo case 'incoming@johnbraun.blog') after which a new ReceivedMail model wil be saved to the database.

Don't forget, that there are a lot of other things you could do. To name a few: it can handle attachments, send an auto-reply, capture the HTML contents of the e-mail, etc.

Check the available methods in the Laravel Mailbox documentation .

I hope this post helped you to make a start if you're interested to start receiving and handling inbound e-mail.

WebSockets in Laravel

John Braun — Sun, 26 Jan 2020 12:08:25 +0000

Freek Van der Herten (from Spatie ) and Marcel Pociot (from BeyondCode ) have published a package called Laravel WebSockets which provides an alternative to services like Pusher .

In this post I want to explain how to get started with realtime broadcasting in Laravel using WebSockets. The stepwise guide will explain how to setup basic broadcasting. The first part covers using Pusher, the second part dives into switching to the Laravel WebSockets package.

When to use WebSockets?

WebSockets is a relatively young TCP protocol, finalized in 2011, which enables two-way communication between the server and a user's browser. They are nowadays commonly used in applications that want to communicate changes in realtime. Chatrooms are probably the most well known example along with online multiplayer games (quizzes), social media streams and sports tickers.

Our demo application

Let's assume our application needs to do some heavy lifting (which we will mimic using PHP's sleep() function). For example, imagine that a user can upload a video, which then needs to undergo a time intensive post-procesing task. Since we don't want to keep our user waiting, we'll put a job on the queue to execute that work behind the scenes. In the meantime, our user will see a "waiting to be processed" status until the job is processed. Within the job, we'll emit an event upon completion that should be broadcasted to the specific user only. That should in turn update the status for that user without requiring a "hard refresh" of the webpage.

Both implementations, Pusher as well as Laravel WebSockets, require the following prerequisites:

Laravel Echo: a javascript library that makes it a breeze to listen to broadcasted events from our front end. You can install Laravel Echo via npm: npm install --save laravel-echo pusher-js. After installation, uncomment the Laravel Echo specific code in your resources/js/bootstrap.js file:

import Echo from 'laravel-echo'

window.Pusher = require('pusher-js');

window.Echo = new Echo({
  broadcaster: 'pusher',
  key: process.env.MIX_PUSHER_APP_KEY,
  cluster: process.env.MIX_PUSHER_APP_CLUSTER,
  encrypted: true
 });

Redis: In this example we'll use Redis as our queue driver, which can be installed using composer: composer require predis/predis. Now, in the .env file set QUEUE_CONNECTION=redis.

Tips and tricks before starting

If you are new to broadcasting, from my experience some things might be a bit tricky at first. Therefore I wanted to dedicate a small section to hopefully clear things up.

Uncomment the BroadcastingServiceProvider

I've been tricked more than once, in debugging where things go wrong only to find out I forgot to uncomment this line in the providers array of config/app.php. Make this the first thing you do! (if you're like me).

// config/app.php:

  /*
  * Application Service Providers...
  */ 
  App\Providers\AppServiceProvider::class,
  App\Providers\AuthServiceProvider::class,
  App\Providers\BroadcastServiceProvider::class,
  App\Providers\EventServiceProvider::class,
  App\Providers\TelescopeServiceProvider::class,
  App\Providers\RouteServiceProvider::class,
],

Always recompile your assets

If you ever change the PUSHER_* credentials in the .env file, always recompile your assets using npm run dev (or watch / prod). This is necessary as Laravel Echo needs them and compiles them down from your environment variables, as you can see in the instantiation of Echo in resources/js/bootstrap.js:

({
   key: process.env.MIX_PUSHER_APP_KEY,
  cluster: process.env.MIX_PUSHER_APP_CLUSTER
});

Pusher

The easiest way to get started is using the Pusher implementation. We'll first require the pusher specific package using composer.

composer require pusher/pusher-php-server "~3.0"

Now, we need to get our credentials from Pusher.com . If you don't have an account yet, you first need to register. They offer a free account, which should suffice for small to medium applications. On their website, create a new app and select the cluster that's closest to you. Copy the credentials on the "App Keys" page to the .env file:

PUSHER_APP_ID=(app_id)
PUSHER_APP_KEY=(app_key)
PUSHER_APP_SECRET=(app_secret)
PUSHER_APP_CLUSTER=(cluster)

Building a demo application

In this section, let's walk through a minimum setup to get our basic demo application working.
First, we'll scaffold the default Laravel authentication and a Video model with a migration and a controller.

php artisan make:auth

php artisan make:model Video -mc

Add a user to the Videos table migration:

public function up()
{
  Schema::create('videos', function (Blueprint $table) {
    $table->bigIncrements('id');
    $table->bigInteger('user_id');
    $table->timestamps();
  });
}

Now, perform the migrations: php artisan migrate.

We have a post route to store a new video, and a show route which will let the user know the current status of his/her uploaded video. In routes/web.php:

Auth::routes();
Route::get('/videos/{video}', 'VideoController@show');
Route::post('/videos', 'VideoController@store');

In the VideoController, we'll add the corresponding methods:

// app/Http/Controllers/VideoController.php:

use App\Video;
use App\Jobs\ProcessVideo;
use Illuminate\Http\Request;

class VideoController extends Controller
{
   public function store(Request $request)
   {
       // here we would the uploaded video from $request
       // and store it along with its path  
       $video = Video::create([]);

       // Then, pass the heavy lifting to our job
       ProcessVideo::dispatch($video);

       return redirect('/videos/' . $video->id);
   }

   public function show(Video $video)
   {
       return view('videos.show', compact('video'));
   }
}

To reach the post route, we add a form to our welcome.blade.php:

welcome.blade.php:

<form action="/videos" method="POST">
  @csrf
  <!-- some upload and other form fields -->
  <button>Upload some video</button>
</form>

After posting this form, the user will hit the store() method of our controller and a ProcessVideo job will be dispatched before the user gets redirected to the video's show page. We haven't created any of these yet, so let's start by scaffolding the ProcessVideo job:

php artisan make:job ProcessVideo

Since we want to have a reference to the video that needs processing, accept the $video in the constructor. In the handle() method, the application wil sleep for 10 seconds and then emit an event called VideoWasProcessed. Don't forget to import the Video and VideoWasProcessed classes.

// app/Jobs/ProcessVideo.php:

use App\Video;
use App\Events\VideoWasProcessed;

class ProcessVideo implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public $video;

    public function __construct(Video $video)
    {
        $this->video = $video;
    }

    public function handle()
    {
        sleep(10);

        event(new VideoWasProcessed($this->video));
    }
}

Since the job implements the ShouldQueue interface, Laravel will always try to queue this job. Then, our Laravel queue workers (using the redis connection) in the background will make sure the job is executed. When completed (in our case after 10 seconds), the VideoWasProcessed event will be fired.

We haven't created this event yet, so run:

php artisan make:event VideoWasProcessed

In addition to accepting the reference to the video in the constructor, we also want to implement the ShouldBroadcast protocol. The event will not be broadcasted unless you implement the ShouldBroadcast interface. Also, don't forget to import the Video class.

// app/Events/VideoWasProcessed.php:

use App\Video;

class VideoWasProcessed implements ShouldBroadcast
{
    use Dispatchable, InteractsWithSockets, SerializesModels;

    public $video;

    public function __construct(Video $video)
    {
        $this->video = $video;
    }

    public function broadcastOn()
    {
        return new PrivateChannel(
          "videos.{$this->video->id}"
        );
    }
}

The broadcastOn() method specifies which channel to broadcast on. Since we don't want everyone to get an update on the processing status of this video we broadcast on a PrivateChannel. Who can access to this channel is configured in the routes/channels.php file. If the closure as defined below returns true, the user has access. If the closure returns false, the user fails authorization. We add a constraint that only allows the owner of the video to see updates in the routes/channels.php file:

// routes/channels.php:

Broadcast::channel('videos.{id}', function ($user, $id) {
  return $user->id === Video::find($id)->user_id;
});

Reflecting updates on the front end

The magic revolves around the show page of our video. For the javascript side of things, we'll levarage Vue JS. In resources/views/videos/show.blade.php:

<html>
  <head>
    <title>Video's page</title>
    <!-- always include your CSRF token -->
    <meta name="csrf-token" content="{{ csrf_token() }}">
  </head>

  <body>
    <div id="app">
      <!-- our Vue component --> 
      <video-progress :video="{{ $video }}" />
    </div>

    <!-- include our compiled javascript -->
    <script src="{{ asset('js/app.js') }}"></script>
  </body>
</html>

The VideoProgress.vue component consists of a data variable processing which holds information on the processing status of the video. In the created() lifecycle hook of Vue, we defined a private listener which passes through the video's ID and listens for a VideoWasProcessed event. When it gets that event, the closure is executed and in this case the processing data variable will be set to false, reactively updating the <span> in our template.

resources/js/components/VideoProgress.vue:

<template>
    <div>
        Your video with ID {{ video.id }} is currently:
        <span style="color: red;" v-if="processing">
            Processing...
        </span>
        <span style="color:green" v-else>
           Finished!
        </span>
    </div>
</template>

<script>
    export default {
        props: ['video'],

        data() {
            return {
                processing: true,
            }
        },

        created() {
            window.Echo.private(`videos.${this.video.id}`)
                .listen("VideoWasProcessed", (e) => {
                    this.processing = false;
            });
        },
    }
</script>

Lastly, we need to register our Vue component in resources/js/app.js:

Vue.component(
  'video-progress', 
  require('./components/VideoProgress.vue').default
 );

Now, after recompilation of all assets using npm run dev(or watch / prod), start a queue worker (php artisan queue:work) and visit the demo app in the website.

In the welcome view, click the submit button and (if all went well) you should see the status updating from 'running' to 'finished' automatically, after the job was processed.

Switching to Laravel WebSockets

The Laravel WebSockets package makes it easy to smoothly transition from using Pusher to using your own WebSockets server.

Installation

We first need to require the package via composer

composer require beyondcode/laravel-websockets

Then, publish the migrations:

php artisan vendor:publish --provider="BeyondCode\LaravelWebSockets\WebSocketsServiceProvider" --tag="migrations"`

* Publish the configuration file:

php artisan vendor:publish --provider="BeyondCode\LaravelWebSockets\WebSocketsServiceProvider" --tag="config"

Configuration

The package will use the pusher driver, but we don't actually want to use Pusher. Therefore we add our own host and port configuration to 'pusher' section in config/broadcasting.php:

'pusher' => [
    'driver' => 'pusher',
    'key' => env('PUSHER_APP_KEY'),
    'secret' => env('PUSHER_APP_SECRET'),
    'app_id' => env('PUSHER_APP_ID'),
    'options' => [
        'cluster' => env('PUSHER_APP_CLUSTER'),
        'encrypted' => true,
        'host' => '127.0.0.1',
        'port' => 6001,
        'scheme' => 'http'
    ],
],

In our bootstrap.js file we need to tell Laravel Echo to use the alternative host and port (wsHost, wsPort):

import Echo from 'laravel-echo'

window.Pusher = require('pusher-js');

window.Echo = new Echo({
   broadcaster: 'pusher',
   key: process.env.MIX_PUSHER_APP_KEY,
   wsHost: window.location.hostname,
   wsPort: 6001,
   disableStats: true,
});

Although we are not using Pusher, we still need to supply some Pusher configuration in our .env file:

PUSHER_APP_ID=testapp
PUSHER_APP_KEY=websocketkey
PUSHER_APP_SECRET=somethingsecret

Compiling assets

We need to recompile bootstrap.js, as discussed in the Tips and Tricks section, by running npm run dev (or watch / prod).

Starting the WebSockets server

Finally, boot up the WebSockets server and start a queue worker (in a separate terminal window):

php artisan queue:work

php artisan websockets:serve

Follow the documentation to learn which other options are available to you.
Now, you should see that our demo application still works but now handling the WebSockets locally.

Running the server in the background

Just like queue workers, we don't want to run the WebSockets server ourself on our real server. The solution is the same as for our queue workers: let supervisord take care of it. To keep this post focused, please review the docs:

Dashboard

The Laravel WebSockets package even comes with a dashboard, comparable to Pusher's Debug Console. The default location of the dashboard is /laravel-websockets from your root path of the demo application and is automagically available. Follow the documentation to learn more about the dashboard.

Conclusion

I hope that I've convinced you that it's easier than ever to get started with realtime broadcasting. Two of the many possible implementations were discussed in this post.

Pusher is a great option to get up and running fast and easy. They offer a free account which offers a maximum of 100 simultaneous connections, 200 000 messages a day and unlimited channels. For small applications, it's a no brainer and I would always choose for the convenience of Pusher.

However, if your application needs a bit more room to breathe the Laravel WebSockets package offers a great alternative which let you handle all WebSocket connections yourself and even comes with a dashboard! On top of that, switching over from Pusher to Laravel WebSockets is a breeze.

Happy broadcasting!