DEV Community: Grant

Complex Filtering in Laravel Scout with Typesense

Grant — Sun, 03 Aug 2025 23:49:26 +0000

tl;dr Passing an array as the second parameter in the query builder's where function allows you to use different comparison operators for your Typesense filters as [$operator, $value].

Laravel Scout is the first-party solution to adding a variety of full-text search tools, such as Algolia, Meilisearch or Typesense. Since it's a plug-and-play solution, they have to make the query builder API a one-size-fits-all solution for each driver.

In this post I'll talk about how I added support for more complex filtering for Typesense that was undocumented in Scout using the built-in where query builder helper.

I'll be talking in the context of Laravel Scout with Typesense only. These principles or ideas won't necessarily apply to the other search tools.

You probably don't need Scout

First, a bit of a warning.

In my experience, you probably don't actually need Scout. Scout adds technical burden (read: debt) in a few ways, and unless you absolutely need full-text searching capabilities, with built-in typo support or other unique features, a simple query will do the trick most often than not.

Configuration takes time. Setting up the fields, applying the configuration to each model, and ensuring the right data types can be time consuming. There are some caveats, such as you cannot use null for a string field.
You have to maintain the schema for the collection in Typesense. This is a deployment burden if/when the schema for your model changes.
- If it changes often enough, then you'll be doing a lot of scout:flush and scout:import calls. If you're not careful, exceptions will be thrown before you're able to update it in production.
Keeping the collection up-to-date is sometimes tricky. Scout does its best to keep the data in Typesense current, but getting out of sync is still easy.

When you need it

When you do actually need the capabilities of Typesense, then it's absolutely powerful. If you can overcome the annoyances I've described above and are certain you need to leverage Typesense, you have a lot of power at your fingertips.

Reading all the search options available in Typesense, it's easy to see they do all the heavy lifting for filtering and searching collections. It's fast and efficient, as advertised.

There are things to keep in mind in a Laravel app with Scout, though.

You have to really understand the Scout query builder. Again, it's a unified query builder API for each vendor. This means it's simplified compared to what each vendor actually offers.
Laravel is searching in Typesense, then converting that back to your model instances using your own database. You have to have a good mental model of the Typesense collection schema and the one that it actually references.
Relationships aren't a thing in Typesense. You have to be creative in the ways that build your Typesense collection schema so that you can leverage filtering based on relationships.

The Setup

grantholle / scout-typesense-example

An example of using Typesense with Scout using undocumented filtering

Laravel Scout with Typesense

Clone the repo.
Run composer install.
Copy .env.example to .env.
Set your Typesense server URL and API key in the .env file.
php artisan key:generate
php artisan migrate --seed
php artisan scout:import "App\Models\Post"

View on GitHub

I've created a sample repo that shows how this works.

It's a simple configuration with a Post model that has many Comment relationships. The Scout configuration for Typesense looks like this:

\App\Models\Post::class => [
    'collection-schema' => [
        'fields' => [
            [
                'name' => 'id',
                'type' => 'string',
            ],
            [
                'name' => 'title',
                'type' => 'string',
            ],
            [
                'name' => 'content',
                'type' => 'string',
            ],
            [
                'name' => 'comments',
                'type' => 'string[]',
            ],
            [
                'name' => 'created_at',
                'type' => 'int64',
            ],
        ],
        'default_sorting_field' => 'created_at',
    ],
    'search-parameters' => [
        'query_by' => 'title,content,comments',
    ],
],

When we call Post::search(), by default we want to search across the title, content and comment contents. We can adjust this if we wanted to, but this will allow us to leverage Typesense in a way that is harder than a typical Eloquent query. Again, the uses cases are less common than you might think for reaching for Scout. A blog search is a good example, though.

Here's how we're indexing our actual data:

public function makeSearchableUsing(Collection $models): Collection
{
    return $models->load('comments');
}

public function toSearchableArray(): array
{
    return [
        'id' => (string) $this->id,
        'title' => $this->title,
        'content' => $this->content,
        'comments' => $this->comments->pluck('content')->toArray(),
        'created_at' => $this->created_at->timestamp,
    ];
}

First, we're making the search query a bit more optimized by eager loading comments in makeSearchableUsing(). Then we're populating the fields we configured in scout.php.

We're wanting to search by comments as well, so we're creating an array of each comment content, which is supported by the field's string[] type. Now, Typesense knows and can search based on the comments. Note: since we're indexing the Post model, it's not going to reindex when a comments change. We're going to have to re-trigger the index manually on the comment's post when comments change.

One caveat I'll point out here: if title was nullable in our database, we'd want to update toSearchableArray() in the title field to be 'title' => $this->title ?? '',. If we had null as a title value, indexing would throw an error: "Error importing document: Field title must be a string."

Searching

When we visit the endpoint, there's a simple controller to perform the searches for us.

public function __invoke(Request $request)
{
    $posts = Post::search($request->input('q', '*'))
        ->when(is_numeric($request->input('created')), function (Builder $query) use ($request) {
            $query->where('created_at', ['>=', now()->subDays($request->integer('created'))->timestamp]);
        })
        ->when(! empty($request->input('exclude')), function (Builder $query) use ($request) {
            $query->whereNotIn('id', explode(',', $request->input('exclude')));
        })
        ->when($request->input('not_title'), function (Builder $query, string $title) {
            $query->where('title', ['!', $title]);
        })
        ->get();

    return $posts->load('comments');
}

Here's the breakdown of the search options:

If we pass in a q query string variable, this is the whole point of Scout. This performs the "search" in Typesense leveraging all of its magic. It's searching across the fields we've set up to query_by. It's typo-tolerant, etc..
Sending a created value will search for posts that have been created after X days ago. More on this later.
We can exclude post ID's by comma separating them in exclude
Lastly, with not_title we can exclude posts where titles contain the given values.

So, what's special?

Behind the scenes, Scout is using the query builder to retrieve results from Typesense and taking those results to subsequently query the actual database to retrieve the model values.

In this example, we could have just leveraged Scout for its searching capabilities and called it a day. However, that's not really a real-world example. Usually we're going to have multiple filtering options like I've implemented.

Since Scout is a one-size-fits-all and each driver will implement filtering differently, it's basic by nature out of the box. As the docs state:

Since a search index is not a relational database, more advanced "where" clauses are not currently supported.

We can hook into the query that's used to retrieve our models after performing the search in Typesense using the query hook. There's also a catch there:

Since this callback is invoked after the relevant models have already been retrieved from your application's search engine, the query method should not be used for "filtering" results. Instead, you should use Scout where clauses.

Here's an example. Let's say we're just performing our search and paginating on those results with page sizes of 5. If we use the query hook to filter further, we're only going to filter on those 5 results that were returned. Each page would have inconsistent number of results, or even no results if the filter doesn't apply to any of those 5.

Scout supports simple where's out of the box, but it's always strict comparison. When it's static, like give me posts with a given tag ID (tag = 1), then it's straightforward. But what about other types of comparisons?

Spoiler: Typesense allows us to configure the comparison operators.

Behind the scenes

In our example, we're retrieving posts that were created with in the last X days (created_at > 10 days ago). By default, Scout doesn't support this. The method signature for where is the following:

public function where($field, $value)
{
    $this->wheres[$field] = $value;

    return $this;
}

It only accepts a field key and its value. Unlike Eloquent's query builder, which accepts an $operator parameter, Scout's is simplified.

The good news is, since each engine (Algolia, Typesense, Meilisearch) implements the filtering themselves, we can dive into Typesense's engine to see what's possible.

The beginning of the hunt for the functionality comes from buildSearchParameters(). The key we're after for filtering is the filter_by key, which is set by the filters() function, which ultimately builds the filter clauses in parseWhereFilter():

protected function parseWhereFilter(array|string $value, string $key): string
{
    return is_array($value)
        ? sprintf('%s:%s', $key, implode('', $value))
        : sprintf('%s:=%s', $key, $value);
}

If the $value we pass in is an array, it's constructing the comparison using the two parameters. Otherwise, it's a strict "exactly equals" filter (:=).

This is what explains our filter of created_at in the controller:

$query->where('created_at', ['>=', now()->subDays($request->integer('created'))->timestamp]);

We can do numeric filtering now (greater than, less than, etc.) by passing an array as the value.

$query->where($field, [$comparison, $value]);

Now we can retrieve consistent results for pagination without having to rely on Eloquent to filter further for us which will be inconsistent behavior. The same goes for the not_title query variable. We can use the :! Typesense operator (not exact contains) to exclude posts with a title that contains a given value.

Conclusion

Thankfully, the Scout Typesense engine was built to be flexible enough to support these filters out of the box. This allows us to leverage Typesense to its full potential. By passing a tuple of [$operator, $value], we can use Typesense's built-in filtering much simpler, providing an excellent all-in-one search/filter solution for our data.

Extra credit: the Typesense also allows us to define a typesenseSearchParameters() function on the search model to include or overwrite other parameters that it supports when searching.

Exploring Middleware in Laravel 11

Grant — Thu, 04 Jan 2024 05:00:19 +0000

Laravel 11 is set to release in "Q1" of 2024, which may be as soon as next month.

I am starting a new project, and because it's so close to the release date I figured I would take a look at what's going to be different with the new major release. I remember reading a Laravel News article 6 months ago about how the Http Kernel was going away and didn't think much of it.

When I created the project using laravel new project --dev, I was quite surprised at how much smaller the project size was. It was very surprising to see an empty config folder (you can publish the config files with php artisan config:publish)!

And sure enough, there is no Http Kernel. So… how do you add or change middleware? Prior to Laravel 11, the Http Kernel (found in app/Http/Kernel.php) is where all the configuration for middleware was found. Also prior to Laravel 11, you would typically never touch bootstrap/app.php. However, that is no longer the case in this new version.

To be honest, as a Laravel user since v4.2, this is quite the paradigm shift. Instead of an easy-to-read Kernel.php file, there is a lot more implicit knowledge you need before adding middleware.

The new starting point for bootstrap/app.php is as follows:

return Application::configure()
    ->withProviders()
    ->withRouting(
        web: __DIR__.'/../routes/web.php',
        // api: __DIR__.'/../routes/api.php',
        commands: __DIR__.'/../routes/console.php',
        // channels: __DIR__.'/../routes/channels.php',
    )
    ->withMiddleware(function (Middleware $middleware) {
        //
    })
    ->withExceptions(function (Exceptions $exceptions) {
        //
    })->create();

I am just exploring middleware in this post, but as you can see this is quite a different approach than we've seen historically. I sat there scratching my head, "How do I set up my own middleware? How do I change the defaults?" I had to explore the Illuminate\Foundation\Configuration\Middleware class to find out.

Calling Application::configure() returns an instance of Illuminate\Foundation\Configuration\ApplicationBuilder, where it then calls the various functions, withProviders(), withRouting() and the withMiddleware() functions. The withMiddleware() function takes a callable.

Using the boilerplate, we can add new middleware aliases by calling alias():

function (Middleware $middleware) {
    $middleware->alias([
        'some_key' => \App\Http\Middleware\MyMiddleware::class,
    ]);
}

Once this some_key is added, we can assign it to individual routes and route groups. If we want to add middleware to every request, we can use the append() or prepend() functions to add global middleware.

function (Middleware $middleware) {
    // Using a string
    $middleware->append(\App\Http\Middleware\MyMiddleware::class);

    // Or adding multiple
    $middleware->append([
        \App\Http\Middleware\MyMiddleware::class,
        \App\Http\Middleware\MyOtherMiddleware::class,
    ]);
}

We can remove middleware that's there by default by calling the remove() function.

function (Middleware $middleware) {
    // Using a string
    $middleware->remove(\Illuminate\Http\Middleware\ValidatePostSize::class);

    // Or removing multiple default middleware
    $middleware->remove([
        \Illuminate\Http\Middleware\TrustProxies::class,
        \Illuminate\Http\Middleware\HandleCors::class,
    ]);
}

We can add or remove middleware to specific groups, i.e. the web group using the appendToGroup()/prependToGroup() and removeFromGroup() functions.

function (Middleware $middleware) {
    $middleware->appendToGroup('web', \App\Http\Middleware\MyMiddleware::class);
}

If/when this bootstrap/app.php file gets a bit messy (which I imagine it will), we can clean it up by moving this stuff to an invokable class.

I've created a class in app/Http called AppMiddleware.php. As old habits die hard, you might call this Kernel.php…

<?php

namespace App\Http;

use Illuminate\Foundation\Configuration\Middleware;

class AppMiddleware
{
    public function __invoke(Middleware $middleware)
    {
        $middleware->appendToGroup('web', \App\Http\Middleware\MyMiddleware::class);
    }
}

Now in bootstrap/app.php, replace the closure with a new instance of our invokable class.

return Application::configure()
    ->withProviders()
    ->withRouting(
        web: __DIR__.'/../routes/web.php',
        // api: __DIR__.'/../routes/api.php',
        commands: __DIR__.'/../routes/console.php',
        // channels: __DIR__.'/../routes/channels.php',
    )
    ->withMiddleware(new \App\Http\AppMiddleware())
    ->withExceptions(function (Exceptions $exceptions) {
        //
    })->create();

Like all new things, change is hard. I have a lingering question with this change. We have all the same tooling and ability, but it's a bit more difficult, at least at first, to understand how to manage it all.

I feel like this change will require a lot more implicit knowledge of the built-in middleware. Does it matter? Probably not. But in some cases, you might need to remove or replace a default middleware. This requires you actually know what is there by default. Not just in the global middleware, but in groups and aliased middleware. Personally I feel like this change increases the learning curve. Even I forget what's there by default or what the aliases are and reference the app/Http/Kernel.php file regularly.

I question whether the lack of verbosity and smaller file footprint is worth it?

What are your thoughts with this new change?

Does anyone else experience FOMO?

Grant — Wed, 03 Nov 2021 00:40:42 +0000

Cover Photo by Heather Zabriskie on Unsplash

FO·MO | ˈfōmō |
noun informal
anxiety that an exciting or interesting event may currently be happening elsewhere, often aroused by posts seen on social media.

FOMO, or fear of missing out, can be paralyzing.

I'm starting to feel paralyzed.

Languages I don't know but want to learn, JavaScript frameworks, blockchain and Web3.0, new CSS features, Jamstack and serverless tooling, conferences, books, career development, "hustling"...

How do you manage it all?

Laravel 🔥 tip: Generating migrations easier

Grant — Fri, 15 Oct 2021 01:31:15 +0000

Generating database migrations in Laravel is already really easy to do with the php artisan make:migration. There are some nice ergonomics that aren't documented that will make it even easier.

I find myself struggling to write the name in snake case (replacing spaces with the _ character). I was thinking, wouldn't it be nice if I could just write it normally and it do that for me? It already does!

If we wrap the name of our migration in quotes, we're able to write it much quicker.

php artisan make:migration "my migration name"

This will generate the correct name of the migration for us by automatically running the name through the Str::snake() helper. This means we could also have weird casing, and it will all be ok.

Auto table detections

If you want to generate the migration with the Schema::create() or Schema::table() scaffolding already there for you, you can pass a --create= or --table= option respectively.

php artisan make:migration "my migration name" --table=users

However, the command also tries to guess your table name based on the name of your migration if you don't explicitly tell it a name.

The below example will automatically create a migration with the Schema::table('users'...) scaffolding.

php artisan make:migration "add column to users"

We didn't specify the table option, but it detected the table based on how we named our migration.

public function up()
{
    Schema::table('users', function (Blueprint $table) {
        //
    });
}

public function down()
{
    Schema::table('users', function (Blueprint $table) {
        //
    });
}

The up() and down() functions automatically included our table. Let's check out how.

In the MigrateMakeCommand command class, there's a call to TableGuesser::guess($name). In that guesser class, it's checking if our name contains a certain pattern:

const CREATE_PATTERNS = [
    '/^create_(\w+)_table$/',
    '/^create_(\w+)$/',
];

const CHANGE_PATTERNS = [
    '/_(to|from|in)_(\w+)_table$/',
    '/_(to|from|in)_(\w+)$/',
];

What this means is that if our migration name is simply create_[table name]_table or just create_[table name] it will assume you're creating a table name whatever value [table name] is.

php artisan make:migration "create planets table"

# or

php artisan make:migration "create planets"

Our migration will include the same scaffolding had we done the command this way:

php artisan make:migration --create=planets "create planets"

Likewise for the table migrations, we can just include a few keywords along with the table name at the end of the migration name.

php artisan make:migration "add favorite color to users table"

# or 

php artisan make:migration "add favorite color to users"

The migration will include the table details for us. You just need to include any one of the prepositions "to/from/in" followed by your table name at the end of your migration name and it will detect your table name for you automatically.

Here are some examples of good migration names that will automatically detect the users table for us:

"make name nullable in users"
"add favorite color to users table"
"remove ssn from users"

This behavior "rewards" us when we write meaningful migration names in the form of less typing... win!

Better CSRF refreshing in Laravel and axios

Grant — Sun, 10 Oct 2021 09:17:02 +0000

This article is written in the context of using Inertia.js, but the concept and code snippets work anytime you're using the axios package in your Laravel project.

I'm a huge fan of Inertia.js when building applications with Laravel. If you're unfamiliar with it, go to their site and read up on it.

A common issue when doing an SPA-like application, like when using Inertia, is that you'll run in to CSRF mismatch exceptions (read more about the what and why of CSRF here). Inertia mentions a way to make the user aware in a friendly way, but I still find that lacking as far as a good user experience.

The user has no concept of what a CSRF token is or that it's needed to make some requests in the application. If they come back to their session after it being idle, the token will have expired. If they're in the middle of doing something and attempt to save, telling them that the "page has expired" will be frustrating, and it's not very intuitive that they will need to refresh the page to solve the problem.

I've found that by adding a simple axios interceptor and an endpoint to create a fresh token, it can be seamless for the user and still serve its security purpose.

Adding the endpoint

First, let's add the endpoint for getting a fresh token.

php artisan make:controller RefreshCsrfTokenController --invokable

This command will generate an invokable (single-action) controller.

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;

class RefreshCsrfTokenController extends Controller
{
    /**
     * Handle the incoming request.
     *
     * @param  \Illuminate\Http\Request $request
     * @return \Illuminate\Http\Response
     */
    public function __invoke(Request $request)
    {
        //
    }
}

The functionality we need to generate a fresh token is only one line!

$request->session()->regenerateToken();

That creates a new token and stores it in our session for us. After regenerating a new token, we can return a simple json response. The completed controller function looks like this:

public function __invoke(Request $request)
{
    $request->session()->regenerateToken();

    return response()->json();
}

Finally, we need to add the endpoint to routes/web.php.

Route::get('/csrf-token', \App\Http\Controllers\RefreshCsrfTokenController::class);

Now that the backend is ready, let's handle the frontend.

Adding an axios interceptor

Axios, which is the popular library that Inertia uses to make requests, has a feature called interceptors which allows us to "intercept" the response immediately after it is made and do some custom handling before Inertia takes over the response.

I like to organize this behavior in a plugins directory in my Vue application and include it in the main app.js file. Let's create a file in resources/js/plugins called https.js.

We need to import the axios library and set up the interceptor.

// resources/js/plugins/http.js
import axios from 'axios'

axios.interceptors.response.use()

// Add this line in resources/js/app.js
import './plugins/http'

The axios.interceptors.response.use function accepts two arguments, a successful (2xx status code) response handler and an error (non-2xx status code) handler. For the successful response, we don't need to do anything.

axios.interceptors.response.use(response => response)

This just returns the response normally without any processing.

For our error handler, we need to check for a status code of 419, which is what Laravel sends when it throws an Illuminate\Session\TokenMismatchException. I'm going to be using lodash/get (lodash comes with the default Laravel project) for a clean way to get a variable from an object that may not have all the properties we're wanting.

import axios from 'axios'
import get from 'lodash/get'

axios.interceptors.response.use(response => response, err => {
  const status = get(err, 'response.status')

  if (status === 419) {
    // Do something
  }

  return Promise.reject(err)
})

If the status isn't 419, we're going to reject the promise as usual.

In the error response, axios gives us the configuration that was used to make the initial request. Basically, we just need to do a couple things at this point:

Set a new token using the /csrf-token endpoint that we made at the beginning.
Retry the original request having the regenerated token.

I'm going to make the callback async so we can call our /csrf-token endpoint using await. Here's the full implementation.

import axios from 'axios'
import get from 'lodash/get'

axios.interceptors.response.use(response => response, async err => {
  const status = get(err, 'response.status')

  if (status === 419) {
    // Refresh our session token
    await axios.get('/csrf-token')

    // Return a new request using the original request's configuration
    return axios(err.response.config)
  }

  return Promise.reject(err)
})

At this point, we actually only needed to add a couple lines of code to make a seamless experience for the end user.

Testing

Let's create a simple post endpoint to try the functionality using a closure. In routes/web.php

Route::post('/test', fn () => response()->json(['status' => 'ok']));

Now we need to add a little disruption to our app/Http/Middleware/VerifyCsrfToken.php middleware to simulate an expired token. By default, this middleware extends Laravel's Illuminate\Foundation\Http\Middleware\VerifyCsrfToken class that already includes the functionality for the handle function. We can cause a little mayhem by regenerating our token before it's checked.

public function handle($request, Closure $next)
{
    if (random_int(0, 1)) {
        $request->session()->regenerateToken();
    }

    return parent::handle($request, $next);
}

The random_int(0, 1) generates a random number, either 0 or 1, and if it's 1 then it will generate a new token in our session. Here's how it breaks down:

When the middleware gets handled, it will check if the token that was passed with the request matches the one we sent.
The snippet we added will randomly regenerate the token. If it gets regenerated, the tokens won't match.
When they don't match, it will throw the TokenMismatchException, aka a 419 status code.
Our interceptor will then call /csrf-token, which then once again regenerates a token.
Eventually, the middleware will not preemptively change the token and the request will resolve correctly.

Since I'm using Vue 3 with Inertia, I'm going to create a component to make this request.

<template>
  <button type="button" @click.prevent="sendTest">Test</button>
</template>

<script>
import { defineComponent } from 'vue'
import axios from 'axios'

export default defineComponent({
  setup () {
    const sendTest = async () => {
      const { data } = await axios.post('/test')
      console.log(data)
    }

    return {
      sendTest
    }
  }
})
</script>

I've just got a simple button that when clicked sends a post request to our /test endpoint and logs the results, which in this case should be { "status": "ok" } as returned from our /test endpoint.

Clicking on my button shows this in the console:

The first time it posts to /test, it fails with a 419, which means that it threw the TokenMismatchException.
It then sends a request to /csrf-token, which means that it has been caught by our interceptor.
The original /test request is retried, this time successful (the middleware didn't regenerate the token before handling the request).

If we click the button several times, sometimes it doesn't fail, while other times it will fail multiple times before finally being successful. This is because we're simulating a random token mismatch.

Cleaning up

Once we're done testing, we can delete the middleware's handle function entirely since we want Laravel manage this particular middleware. We'll also delete the /test route from routes/web.php.

Summary

Overall, the impact on our codebase is quite minimal, yet the user experience is greatly improved. The user can leave their session and allow their CSRF token to expire. Coming back and doing a request won't interrupt their desired workflow, which in my opinion is more desired than telling them that the "page has expired" and making them refresh the page.

Speed up migrations when running Laravel tests

Grant — Wed, 25 Aug 2021 07:15:46 +0000

I'm currently doing maintenance on a two year old application. Not that old, right? Eh, sort of.

It was created using Laravel 7. I was able to upgrade to Laravel 8 and PHP 8.0, so that's great! The thing is, I'm now going test-by-test and optimizing and house cleaning. I'm also mad at Past Grant for being lazy.

The application is... big. One of my biggest I've ever built. Like 126-migration-files big. When I went to my first test, I noticed that it took forever for it to actually run the test (it failed if you were wondering). When I mean forever, I mean over 11 seconds, which might has well been 11 minutes when I'm staring at the console waiting for it to start.

I remembered when watching the keynote on Laravel 8, Taylor introduced the new feature of squashing migrations. Let's see what impact it had on my tests.

php artisan schema:dump --prune

This command generates a file in database/schema and deletes all my migration files. Before squashing the migrations, it was taking 11.031 seconds according to well placed Ray call in the RefreshDatabase trait:

protected function refreshTestDatabase()
{
    ray()->measure('migrating');
    if (! RefreshDatabaseState::$migrated) {
        $this->artisan('migrate:fresh', $this->migrateFreshUsing());

        $this->app[Kernel::class]->setArtisan(null);

        RefreshDatabaseState::$migrated = true;
    }
    ray()->measure('migrating');

    $this->beginDatabaseTransaction();
}

After squashing, it took only 1.396 seconds. That's almost a 90% decrease in time! Now I can spend the next several days fixing all my terrible tests, but they'll at least run faster.

Implementing Laravel's built-in token authentication

Grant — Thu, 04 Mar 2021 02:07:55 +0000

grantholle / laravel-token-auth-example

This is a basic implementation of Laravel's default "token" auth driver.

One of the great things about Laravel is its mission to provide developers with the tools they need out of the box, as easily as possible.

More often than not when developing an application you're going to need some mechanism of authentication. Up until recently, Laravel shipped with a complete authentication toolbox: controllers, routes and views. Recently Laravel migrated a lot of its backend authentication functionality into Laravel Fortify and provided a frontend simple implementation using Breeze. There's also a more opinionated auth setup using JetStream which combines Fortify and other currently-popular frontend tools Livewire and (my personal favorite) Inertiajs.

But what about API's? Laravel provides two solutions, Sanctum and Passport. Both of these are quite fully featured; they come with all the tools for users to generate tokens for themselves to interact with your application.

But recently I needed to implement very simple API authentication for interacting with an application from a different application. Think "machine to machine" interaction. Passport ships with an entire OAuth implementation, which includes client credentials grant tokens. The gist is that you provide a token to authenticate without any of the traditional OAuth flow.

While Passport is a great tool, it includes way more than I needed. Lucky for you and me, Laravel already ships with a token-based authentication mechanism. Unfortunately it's actually quite undocumented.

Begin!

I'm starting with a clean project using PHP 8 (version isn't important, just being thorough) for this tutorial, but I added it to my existing project using the same logic. What's really great is that there are no packages we need to install. Laravel ships with all the tools we need.

The important thing to note here is that for this example, I needed to perform some actions at the admin level, not anything based on a particular user account. I actually don't care who is doing this API action (i.e. what the value of $request->user() is), just that I needed to do something remotely that is triggered by a different system, sort of like a microservice.

Create the sample project

Create the new project.

laravel new token-auth-example

You'll need to set up your own database and know how to configure it to get it going. I'm not going to cover that here.

Since we don't care about users right now, we can ignore that entirely.

Create the token table

Next, let's create a new migration for a table called api_clients.

 php artisan make:migration --create=api_clients create_api_clients_table

I'll modify it to have just a column, api_token. This is the column that Laravel will use to look up the token:

Schema::create('api_clients', function (Blueprint $table) {
    $table->id();
    $table->string('api_token')->unique();
});

Now we'll run the migrations.

php artisan migrate

Configure the api guard and provider

We need to make the necessary changes to config/auth.php to configure our token authentication. This was confusing for me for a while to understand what the different keys mean, but I'll try myself to explain it.

The guards are what protect parts of your application. Laravel allows you to create separate guards so theoretically you could have different user models that use different parts of your application. I've done this and it can get a little messy, but it's nice that Laravel is flexible enough to allow this. By default it has two generic guards, "web" and "api". The web guard is for authenticating into your web app via the browser and the api guard is for, you guessed it, api access. There are separate guards because a browser and api are two different mediums of interacting with your application.

The default configuration is pretty close to what we need.

'guards' => [
    'web' => [
        'driver' => 'session',
        'provider' => 'users',
    ],

    'api' => [
        'driver' => 'token',
        'provider' => 'api_clients', // was users
        'hash' => true, // was false
    ],
],

We've modified the provider key to be api_clients (more on that next) and then set hash to be true. The hash option means that we're going to store the token hashed (i.e. not in plain text)

Next we'll need to configure its provider. The provider "provides" who the user is that is being authenticated. Your users (in our case clients) are stored in the database, and we need to tell Laravel how to retrieve the user to verify they're valid.

The default users provider is that it's your App\Models\User Eloquent model. For our api client, we could use Eloquent too, but again, we don't care who it is. We don't need Eloquent's bells and whistles. We just need a table, which we've already created.

'providers' => [
    'users' => [
        'driver' => 'eloquent',
        'model' => App\Models\User::class,
    ],

    'api_clients' => [ // <- Add the api_clients that was configured in our `api` guard
        'driver' => 'database', // We don't need eloquent
        'table' => 'api_clients', // Change to be our table name, which happens to be the same as our provider name
    ],
],

Believe it or not, that's all we need for Laravel to automatically do token authentication in our api routes. But... how do we create clients?

Creating API clients

For our simple use-case, we don't need routes to manage tokens and all of that jazz. We need one token to exist at a time. In times like this, I reach for an Artisan command.

php artisan make:command GenerateAuthToken

This will give us all the functionality we need: remove old tokens and create a new one, providing the new token to me to use. In app/Console/Commands/GenerateAuthToken.php we can add this in a few lines.

Change the signature and description to something more meaningful:

/**
 * The name and signature of the console command.
 *
 * @var string
 */
protected $signature = 'make:token';

/**
 * The console command description.
 *
 * @var string
 */
protected $description = 'Generates a new api client auth token to consume our api';

Remove the code for the constructor because we don't need it. In handle() we need to create a token, remove old clients and create the new client with the new token.

use Illuminate\Support\Facades\DB;
use Illuminate\Support\Str;

public function handle()
{
    // Generate a new long token
    // Be sure to import Illuminate\Support\Str at the top
    $token = Str::random(60);

    // Delete existing tokens, maybe we lost the token
    // so we don't want existing ones floating around somewhere
    DB::table('api_clients')->whereNotnull('api_token')->delete();

    // Create a new entry with the hashed token value
    // so we don't store the token in plain text
    DB::table('api_clients')->insert([
        'api_token' => hash('sha256', $token),
    ]);

    // Spit out the token so we can use it
    $this->info($token);

    return 0;
}

We can try it out and generate a token:

➜ php artisan make:token
YHEt7CNf1SBmQs6JbTPf7qMK8FgnynI5SiPmyJELrbAO61heKy0eKuiXrxBJ

If we check out the database, the value for api_token is hashed as 277b302457eeccc1607b024b16f6c50bfaf37d7db028f9bf1daf4add58282a57. When we make a request, Laravel will hash our token to match it against what's in the database, rather than comparing the raw token. This also means that you better put that token somewhere safe, as that's the only time you'll be able to see it.

Implement a route

Head on over to routes/api.php and change the route to something else:

Route::middleware('auth:api')->get('/test', function (Request $request) {
    return 'Authenticated!';
});

For this simple example, we are using a Closure. You would probably use a real controller, such as a single action controller.

There are a few ways we can use our token to access this route. The main ways are either passing a variable in the query string of the request or using an Authentication header Bearer token. I'll just use the browser with a query string. I'm going to use Laravel's php artisan serve to test it out quickly.

I can now go to http://localhost:8000/api/test?api_token=YHEt7CNf1SBmQs6JbTPf7qMK8FgnynI5SiPmyJELrbAO61heKy0eKuiXrxBJ to see if it worked.

I do in fact see "Authenticated!". Awesome! I'm going to modify the token, or even remove it entirely, to see what happens.

You're going to see a "Route [login] not defined" error. This just means that we weren't authenticated and it tried to redirect us to a login page, which doesn't exist.

Conclusion

For my use case of performing an admin task in my application from a different application (server to server/machine-to-machine), this token authentication is just what I needed. The best part was that I didn't need any additional package as Laravel already had the token driver I needed. An Artisan command gave me all the functionality I needed to manage clients and tokens. We're all set!

Laravel Inertia and API Resources

Grant — Wed, 29 Apr 2020 02:27:08 +0000

If you haven't checked out Inertia, it's worth a look. It's sort of like a hybrid SPA. I've built a few apps with it and don't see myself not using it for the next while.

Let's take a look at a nice server side tip that I find really useful.

Say we're working on our /users/{user} route. Assume we have a show method in our UserController:

public function show(User $user)
{
    // This is just a helper for Inertia\Inertia::render()
    return inertia('User', [
        'user' => $user,
    ]);
}

This will be totally valid and fine. When your user variable gets sent to the front end, it will be converted to an array with the $user->toArray() method. You can define that on App\User and all will be good.

But what happens when you have relationships that you'd like included or you'd like your app to be more flexible? Then you would reach for API Resources. Resources allow you to define the shape of your objects when they get sent to API's. Since Inertia is sort of like using an API, we can leverage API Resources to make our objects be sent to our views more reliably and consistently.

php artisan make:resource UserResource

This generates app/Http/Resources/UserResource.php.

<?php

namespace App\Http\Resources;

use Illuminate\Http\Resources\Json\JsonResource;

class UserResource extends JsonResource
{
    /**
     * Transform the resource into an array.
     *
     * @param  \Illuminate\Http\Request  $request
     * @return array
     */
    public function toArray($request)
    {
        return [
            'id' => $this->id,
            'name' => $this->name,
            'email' => $this->email,
        ];
    }
}

We can refactor our controller to return our resource instead:

public function show(User $user)
{
    return inertia('User', [
        'user' => new \App\Http\Resources\UserResource($user),
    ]);
}

Behind the scenes, Inertia will convert this to its "response" version as if you were returning it directly from your controller, but allows you to return a resource for each variable you want to return to your views.

We can go one step further to make things "cleaner" and easier to use when you have multiple controllers returning the same resource. I'm lazy so I like this.

In our App\User model, let's add a method called toResource and return that resource.

public function toResource()
{
    return new \App\Http\Resources\UserResource($this);
}

The final state of our controller looks like this:

public function show(User $user)
{
    return inertia('User', [
        'user' => $user->toResource(),
    ]);
}

I love API Resources. Coupled with Inertia, it's a really great developer experience on top of an already amazing experience with Laravel. Make sure to check out Inertia and API Resources. The real power with API Resources comes with relationships!

Multi or single-tenancy?

Grant — Wed, 18 Mar 2020 03:20:13 +0000

I have an idea I'm excited about for a Human Resources (HRIS/HRM) SaaS project. I'm doing some light planning and prepping.

The question I can't quite answer is how should I manage the tenants. I'm leaning single-tenancy. In my mind here's how it would go...

When someone signs up, it would automate creating a server instance and install/configure/deploy the application. They are isolated from other tenants in every sense, given a server geographically beneficial to them, [insert other benefits of being single tenant]. They get their own subdomain (also possible with multi-tenancy) or could point their own domain to that server (branding to an extent is also possible with multi-tenancy).

There would be a central server to manage the subscriptions/accounts, but it could also be managed from their own server talking to the said "central" server.

Does anyone have anecdotes of doing single vs multi-tenancy? Is this too much work? A silly idea? Too ambitious?