DEV Community: atymic

Running Laravel on DigitalOcean's App Platform

atymic — Sun, 11 Oct 2020 09:59:37 +0000

Today DigitalOcean App Platform entered general availability and they offer support for PHP/Laravel apps out of the box.
Read on for a deep dive and explore the pros and cons of DigitalOcean's new app platform with me.

If you've used Heroku in the past you'll be right at home on the App Platform. It actually uses heroku buildbacks under the hood, making it super easy to get an app running with no or minimal configuration (or, so they say).

Just want the Review / TL;DR? Skip the Deep Dive and read the Review.

Deploying your first App

Head over the Apps tab on the DigitalOcean dashboard. You'll need to link your GitHub account (they don't support gitlab or any other providers currently). I like that you can give access to only the repos actually required, instead of granting account wide permissions like most other services.

Once you'd configured the GitHub integration you can start deploying your project. For the purposes of this demo, I scaffolded a Laravel 8 application with Intertia Jetstream, as it encompasses all of the moving parts in a normal Laravel app (Backend, plus building the javascript with Mix). I selected my demo app and hit Create.

Next up, we need to give the app a name and select the region. I've kept the default name (repo name) and selected New York. Currently, they offer 3 regions being New York, Amsterdam and Frankfurt.

This isn't ideal for me as these locations are all very far away from Australia, but I'm sure they'll launch more regions in the future. I've also selected the Auto Deploy Code Changes option for continuous deployment.

On the next screen we can configure the service. It seems that we can't actually pick the buildpack here (was guessed from the repo code). We can also set environment variables (what you'd put in your .env normally).

I've also added a development managed database (Postgres is the only option). It's a bit pricey at $7/m for a 256mb dev instance and 1GB of disk. You can also link an existing Managed Database (supports both pgsql and mysql, from $15/m).

Now we've got the app configured we're presented with the plan selection page. I've picked the $5/m 512MB option here. See the "Review" section below for my thoughts on the pricing. Total cost for our dev setup is $12/m.

After hitting Launch Basic App the platform gets started building right away. For this app, it took about 10 minutes to provision the app containers and the database. You can watch a live tail of the build logs under the Deployments tab.

Once the initial deployment finished I jumped on over to the automatically provisioned URL. My first deployment was 500ing because I forgot to set the database driver to pgsql, but after updating the environment variables (which automatically redeploys the app) the app was able to connect to the DB.

After that was finished, I jumped into the Console tab to run the migrations. This was super easy, so props to them for
the awesome console.

Whoohoo! We're live, and I can see the splash page. Bad news is we've got an issue. The build pack didn't build the javascript, so our InertiaJS app doesn't work.

I had a look through the docs, but it isn't immediately clear how to get the platform to build the javascript. I tried setting the Build Command to yarn install && yarn prod, ~~but that didn't work because yarn isn't installed~~.

Update: Turns out I didn't commit the yarn.lock file (which is used to detect what build packs to load). It worked fine once I committed it. It would be good if the documentation around this was a bit better (someone from the DO team did reach out to me via twitter which was awesome but I still couldn't find it in the docs).

I guess this is also where we'd do stuff like running migrations, so it would probably make sense to extract these steps into a bash script and execute that instead.

Onto the next issue! Our javascript and CSS is now built, but Laravel thinks we're running on http, so it's generating http urls, which our browser is refusing to load because the app is being served over https. I tried setting the APP_URL to be prefixed with https://, but this didn't solve the issue. I'm guessing this is due to the internal load balancer configuration.

To fix this, I change the AppServiceProvider to force HTTPS in production, which fixed the issue.

public function boot()
    {
        if ($this->app->environment('production')) {
            \URL::forceScheme('https');
        }
    }

Adding a Queue Worker

Another pretty common task is running a queue worker (either using queue:work or horizon). To do this, we need another separate worker container. Unfortunately this means an extra $5-12/m (depending on if you're on the basic or pro plans), since there's no way to reuse the existing resources.

From the Components tab, hit Create Component and select the Worker option. You'll need to select the git repository and pick a branch (this will be the same for Laravel apps) as well.

Next, you need to set the run command that will be executed. In this case, it's php artisan queue:work.

It's worth noting that if one of your containers is on the Pro plan, every future container needs to be as well. This is pretty annoying since you may be happy with a small, cheap queue worker.

Success, we're live 🎉

All in all, it took about an hour for this first exploratory deployment. If I was doing it again, it would probably take me around 15 minutes, which is super quick to get a fully featured app running in production (with horizontal scaling!).

On top of that, we also get Zero Downtime Deployments out of the box as well.

Review + Pros/Cons

At a high level, I really like DigitalOcean's App Platform, and it's awesome to have a Heroku alternative in the space. It's way, way cheaper than Heroku ($5/m vs $25/m for the base plan, more savings for the higher plans). It's easy to approach for beginners, but still suitable for advanced users.

Here's what I'd say an average service would require:

2x 1GB Web Containers - $12m * 2 = $24/m
1x 1GB Managed Database - $15/m
1x 1GB Queue Worker Container - $12/m

At ~ $50/month, it's not super cheap but consider the savings in time (no need for devops, managing servers, etc) I think it's a pretty great deal.

Pros

Much cheaper than Heroku
Zero downtime deployments
Built in CI/CD
Built in Horizontal scaling (no autoscaling, but it's coming)
Awesome console for managing the app
Built in managed databases (though, a bit expensive)
Built in Cloudflare CDN

Cons

Tiny included outbound bandwidth (40gb on the Basic plans, 100GB on Professional plans)
Super expensive bandwidth overage (10c per GiB, that's more than AWS 😬, and they offer 1TB bandwith on $5/m droplets)
Documentation is a bit lacking (though, it's a very new offering)
No VPCs for isolating apps
No wildcard subdomains

Wrapping up

It's awesome to see competition, and nothing is perfect on launch. I'm excited to see the DigitalOcean App Platform improve, and hopefully they address some of the cons/concerns (which seem to be shared by a lot of people).

I hope you enjoyed this post and helped you to evaluate the App Platform for your projects.

If you have any questions, leave a comment below & I'll do my best to respond to them all.

MySQL Performance Tuning - Speeding up slow inserts

atymic — Wed, 10 Jun 2020 11:17:45 +0000

If you're running a write heavy application with a MySQL database you might notice a fairly significant slowdown as the write load on the server increases. In our case, we're running an event sourcing based service which is writing incoming events to read tables. Under heavy write load, simple inserts and updates with small payloads were pushing the 250ms mark. Here's how you can speed up those slow writes.

Use a server backed by performant SSD storage

MySQL (with default settings) writes the log buffer to disk after every transaction. Under high load, this causes a ton of small writes which are particularly slow on traditional platter based hard drives.

In our case, we had undersized the Amazon RDS instance disk size, which has a direct impact on write performance (higher capacity means more provisioned IOPS/write bandwidth). Increasing the disk size dropped out write average down about 50ms.

TL;DR: Make sure the disk for your server is as fast as possible

Increase the log file size limit

The default innodb_log_file_size limit is set to just 128M, which isn't great for insert heavy environments. Increasing this to something larger, like 500M will reduce log flushes (which are slow, as you're writing to the disk). This is particularly important if you're inserting large payloads.

In our case, bumping the log file size didn't actually help us much, since we had many smaller reads (not enough to overflow the log file frequently).

Defer disk writes by flushing the log less frequently

By default MySQL will write the log to disk after every single transaction. In our case, it wasn't possible to wrap batches of inserts in transactions, so every single insert query was causing a disk write.

You can instead write the log to disk at an interval by setting innodb_flush_log_at_trx_commit to 2. The main issue with setting this away from the default of 1 is possible data loss. While you're still protected from MySQL crashes causing data loss, the entire server loosing power would mean potentially losing data. For example, if you have the flush interval set to the default of 1, you could lose a seconds worth of writes upon failure of the server.

By setting innodb_flush_log_at_trx_commit to 2, we were able to drop the insert time average from over 200ms to under 50ms. While this is a pretty massive improvement, it's possible to squeeze even more speed if you are willing to risk a few more seconds of potential data loss.

You can increase the flush interval by setting innodb_flush_log_at_timeout to your desired interval (in seconds). In our case, we went with 5 seconds, which resulted in the insert time average dropping under 5ms! A massive difference from the original 250ms, and in our case worth the risk of 5 seconds of potential data loss.

Reverse engineering Google Reminders for fun and profit

atymic — Sun, 07 Jun 2020 08:52:55 +0000

If you've got a Google device you've probably used google reminders before. It's built into a lot of google services including Calendar, Keep, Inbox (💀), Home and others. It's pretty simple, but like many other things in the Google ecosystem, it's absurdly inconsistent. You'll get different options and see different reminders in a completely different UI even though they are all "Google Reminders".

I use Todoist which is an awesome task management service. Naturally, I'd expected them to have some sort of integration with Google Reminders, but when I checked with their support, they said it wasn't possible as Google Reminders didn't have a public API.

I decided that I'd build my own tool to sync reminders from Google to Todoist (todo-sync.atymic.dev for those interested). First step, figure out how to query, update & delete google reminders.

Reverse Engineering Google's Reminders API

I jumped into Google Calendar with Chrome's dev tools to figure out how the API worked. Usually it's pretty easy to figure out how they work, even without access to the code. I was completely surprised when I saw what looked like a gibberish request and response format. For example, this is a "List" request:

{
  "1": {
    "4": "WRP / /WebCalendar/calendar_200531.18_p0"
  },
  "2": [{"1":3},{"1":16},{"1":1},{"1":8},{"1":11},{"1":5},{"1":6},{"1":13},{"1":4},{"1":12},{"1":7},{"1":17}],
  "5": 1,
  "6": 100,
  "24": "1599314400000",
  "25": "1583157600000"
}

Checking the Content-Type header (application/json+protobuf) tell us this is a protobuf request. Protobuf is typed data transfer protocol, used internally by lots of teams at google. Unfortunately, this made it a lot hard to figure out how we can interact with their APIs.

Finding the Protobuf Definitions

Jumping into the Sources tab in dev tools, I started looking through the Javascript files for code related to the reminder APIs. This was made more difficult by minified javascript but I managed to find the definitions burried in the code for part of the API Client. Here's what one of the looked like:

 FSc.Ra = FSc.prototype.Ra;
    TQ.prototype.Ra = function() {
        var a = ISc;
        a || (ISc = a = UH(TQ, {
            0: {
                name: "ListTasksResponse",
                fullName: "caribou.tasks.service.ListTasksResponse"
            },
            1: {
                name: "task",
                Ud: !0,
                Ia: 11,
                type: zQ
            },
            2: {
                name: "continuation_token",
                Ia: 9,
                type: String
            },
            4: {
                name: "continuation",
                Ia: 11,
                type: HQ
            },
            3: {
                name: "storage_version",
                Ia: 4,
                type: String
            },
            5: {
                name: "response_header",
                Ia: 11,
                type: KQ
            },
            6: {
                name: "skipped_storage_read",
                Ia: 8,
                type: Boolean
            }
        }));
        return a
    }
    ;

A mess, but clearly we're looking in the right place. I also found some other attempts to re-implement the API which were helpful in figuring the format out. By referencing all the objects in the google calendar javascript we can figure out the format of all the requests.

For example, when listing reminders in TodoSync, we make the following query:

{
  "5": 1, // include_archived = true
  "6": 500, // limit = 500
  "13": { // RecurrenceListOptions
      "1": 1 // collapse_mode = true
    }
}

I've also created a repository in github documenting how the API works for those interested.

Building TodoSync

The concept is pretty simple. Poll Google Reminders, and sync those across to Todoist. Naturally, I leaned on Laravel's awesome tooling to quickly get the tool up and running.

I used Laravel Socialite to quick add Oauth authentication for both Google and Todoist. Once you've logged in and set up the sync, a minutely scheduled task runs to sync each user's reminders. To make this scale with more than a few users, we trigger a background job for each user so they can run asynchronously. Laravel Horzion manages the workers and provides alerts if there's any issues.

The code is all open source on Github if you'd like to see how an application like this works, or want to host it yourself!

Feel free to hit me up with any questions you have and i'll do my best to answer them.

Serverless Cobol

atymic — Tue, 19 May 2020 08:32:25 +0000

I'm currently in the process of writing a post covering how to run Laravel Serverless on Google Cloud. There I was, sitting there hashing out the post and I wrote this:

This pretty much means you can run anything that responds to HTTP request, hell you could even run Cobol if you wanted!

Well, I can't really claim that without actually trying it, can I? Well, an hour or so later and were running serverless cobol!

https://serverless-cobol-max7gzuovq-uc.a.run.app/cobol+dev.to=%3C3

It was actually surprisingly easy to build a docker container to run cobol, despite it being a 60+ year old language. There's a modern re-implementation called GnuCobol which support modern OSes like Ubuntu.

Here's the dockerfile which compiles the cobol source code & sets up apache (which interfaces with the cobol program as a CGI script).

FROM ubuntu:bionic

RUN apt-get update
RUN apt-get install software-properties-common -y
RUN add-apt-repository ppa:lud-janvier/gnucobol
RUN apt-get update
RUN apt-get install gnucobol apache2 -y

COPY . /var/www/public
WORKDIR /var/www/public

RUN cobc -x -free serverless.cbl -o the.app
RUN chmod +x ./the.app

COPY docker/apache.conf /etc/apache2/sites-available/000-default.conf

EXPOSE 8080


RUN echo "Listen 8080" >> /etc/apache2/ports.conf && \
    chown -R www-data:www-data /var/www/ && \
    a2enmod rewrite && \
    a2enmod cgid

CMD ["/usr/sbin/apache2ctl", "-D", "FOREGROUND"]

Building the container & getting the cobol code to run was the hard part, deploying to Cloud Run is incredibly easy - two command and a few minutes later and we're running cobol in the cloud!

Is it useful? Definitely not. Was in fun? Sure was 🙃

Source code on Github.
Now to do some actual work 😬

Building Calndr, a free AddEvent alternative

atymic — Sun, 10 May 2020 02:37:29 +0000

Coronavirus has impacted pretty much every business large and small, and has resulted in a massive surge in online events. Companies that previous provided solely physical services are switching to webinars and online courses, so the demand for tech in this area has spiked.

In the last few months, I've had multiple different clients come to me with requests to help them schedule events online. There's some existing services that allow you to create add to calendar links out there, such as AddEvent, but they are expensive for what they do - essentially just generating a few links in a certain format (for different calendar software).

After integrating a simple version into one of my client's applications, I decided to create Calndr.link. It's a super simple service that allows you to generate calendar links for all the major providers in a couple of clicks. Enter the event details (title, location/meeting link, date, etc) and hit generate and you'll be provided with some HTML to copy/paste directly into your newsletter, website, email signature, etc. You can also copy/paste the direct links as well, if you prefer.

The Tech

I decided to have some fun when building this, using Interia.js (totally overkill for moment, but fun!). On the back end, it's running Laravel 7.

It's deployed on Google Cloud using Cloud Run, so it's running completely serverless. I love the flexibility and ease of use, just whip up a simple docker container, push it to the image registry and hit deploy!

Since you're only charged for requests that actually hit the server, it's extremely cheap. It's basically Lambda, but you can run anything (since it runs custom docker containers).

I'm planning on writing a guide on how to deploy Laravel on GCR serverless, so keep an eye out for that!

Feel free to leave a comment if you've got any questions or suggestions for Calndr.link!

Using Custom Eloquent Casts in Laravel 7

atymic — Sun, 01 Mar 2020 05:20:48 +0000

Laravel 7 is pretty much upon us (scheduled for release on 3rd March) and brings a bunch of awesome new features & improvements. One of these ones I'm most looking forward to is Custom Eloquent Casts.

Historically you've been limited to the default set of casts provided by Laravel, which cover basic language type plus dates. While there are some existing packages out there implementing custom casting they have a major drawback. Because they override the setAttribute and getAttribute methods through a trait, they can't be used with any other
packages that also override those methods.

Now that Laravel 7 supports these casts natively there won't be compatibility issues between libraries!

How custom eloquent casts work

Any object implementing the new CastsAttributes contract provided by Laravel can now be used in the $casts property on your model. When accessing properties on your model, Eloquent will first check if there's a custom cast to transform the value with before returning the value to you.

Keep in mind that your cast will be called on every single get and set
operation on the model, so consider caching intensive operations.

Creating Custom Casts

A popular feature suggestion for Laravel has been allowing selective encryption of model properties, and with custom
casts this is super simple to implement.

use Illuminate\Contracts\Database\Eloquent\CastsAttributes;

class EncryptCast implements CastsAttributes
{
    public function get($model, $key, $value, $attributes)
    {
        return encrypt($value);
    }

    public function set($model, $key, $value, $attributes)
    {
        return decrypt($value);
    }
}

In your model, you can then assign a property to the cast we just created.

class TestModel extends Model
{
    /**
     * The attributes that should be cast to native types.
     *
     * @var array
     */
    protected $casts = [
        'secret' => EncryptCast::class,
    ];
}

Now that we've set it up, let's test it out! Because the encrypt/decrypt functions serialize input, you can store
pretty much anything (but you should probably stick to the simple built-in types).

$model = new TestModel();

$model->secret = 'Hello World'; // Plain text value

// Encrypted value (which will be saved to the database)
// Raw Value: eyJpdiI6InV4Q25ZN0FZUW5YSEZkRCtZSGlVXC9BPT0iLCJ2Y...
$model->getAttributes()['secret'];

$model->save(); // Save & reload the model from the DB
$model->fresh();

echo $model->secret; // Hello World

Because custom casts are just objects, they can be as simple or complex as required. There's a couple of additional features in the implementation, including "Inbound" casts (that only cast set values) and the ability to accept config from the deceleration in the model's $casts.

For example, we could limit strings to a configurable length, but only when setting values (therefore any existing values would remain the same length).

class LimitCaster implements CastsInboundAttributes
{
    public function __construct($length = 25)
    {
        $this->length = $length;
    }

    public function set($model, $key, $value, $attributes)
    {
        return [$key => Str::limit((string) $value, $this->length)];
    }
}

In the model, separate the class name of the cast and the parameters with a colon.

class TestModel extends Model
{
    protected $casts = [
        'limited_str' => LimitCaster::class . ':10', // Limit to 10 characters
    ];
}

Wrapping Up

I'm really excited to see some of the awesome casts the community comes up with once Laravel 7 is out. It's super easy to publish custom casts as in packages as well, so hopefully we'll see a bunch of packages proving custom casts in the future.

One of the ones I've missed on previous projects is a DateInterval or CarbonInterval cast, so I've published a package
in case anyone else has come across the same problem.

Got any questions? Feel free to comment below and I'll do my best to answer them!

Submitting sitemaps on deploy with Netlify

atymic — Thu, 16 Jan 2020 09:15:49 +0000

There's plenty of static site generators out there, and most support generating sitemaps. To make sure google indexes
your pages as quickly as possible, it's a good idea to ping them with your sitemap when it's updated.

Netlify provides some hooks which can be used to run functions once the deploy is complete. This allows us to ping
google when production is deployed.

Github Repo with Code

Submitting Site Maps Automatically

Copy the functions/deploy-succeeded.js file to your project. Keep in mind you can't change the name, otherwise
Netlify won't detect it as a hook.

Next, update your netlify.toml to define your functions folder. This is where the compiled functions will be stored, and can
be named anything (in this example, we use lambda). These files should be committed to your VCS.
Alternatively, you can update your build script to build them in CI (in which case, you can gitignore them),

[build]
  publish = "dist"
  command = "yarn build"
  functions = "lambda" # netlify-lambda reads this

You'll also want to install netlify-lambda and axios and compile the function (this bundles the dependencies & code into a single file):

yarn add netlify-lambda
./node_modules/.bin/netlify-lambda build functions

Next, configure the SITEMAP_URL environment variable inside Netlify's build settings, like so:

Make sure your static site generator is updating the sitemap on each build.

If all went well, on your next deploy Netlify will run the hook, and alert google that your sitemap has changed

10:14:21 AM: 2020-01-14T23:14:21.146Z 864f3fb2-1c2d-41a5-aa4f-0379008aa81c INFO Sending sitemap ping to google for https://sitemap-submit.netlify.com/sitemap.xml

10:14:21 AM: Duration: 59.46 ms Memory Usage: 75 MB Init Duration: 137.25 ms

Editing DNS records for ZEIT Now

atymic — Thu, 16 Jan 2020 09:14:37 +0000

ZEIT Now is awesome, but one of the features that's missing is a GUI interface to edit and configure DNS records. It's actually
not even clear this is possible in the dashboard, but luckily there's a set of commands you can use with the CLI.

First off, if you haven't already install and authenticated in the now cli, you'll want to do that:

npm install -g now && now # Will prompt to login

Once you're logged in, you can use the commands in the now dns namespace.

Adding Records

now dns add <domain> <name> <record type> <value> [mx_priority]

<domain> is the address owned by the user and previously registered withzeit.world by using the commands now domain add zeit.rocks or now alias
<name> is the subdomain that will be prefixed to <domain> (@ as a <name> refers to the domain without any prefix)
<record type> contains one of the supported record types shown above
<value> indicates the target of the record (like an IP address or a hostname)
[mx_priority] sets the priority of a certain MX record and can therefore only be used in conjunction with this record type

Examples

Here's a few examples of commands that might be helpful.

Add a record for a subdomain

  $ now dns add <DOMAIN> <SUBDOMAIN> <A | AAAA | ALIAS | CNAME | TXT>  <VALUE>
  $ now dns add zeit.rocks api A 198.51.100.100

Add a MX record (@ as a name refers to the domain)

  $ now dns add <DOMAIN> '@' MX <RECORD VALUE> <PRIORITY>
  $ now dns add domain.com '@' MX mx1.improvmx.com 10

Add a SRV record

  $ now dns add <DOMAIN> <NAME> SRV <PRIORITY> <WEIGHT> <PORT> <TARGET>
  $ now dns add zeit.rocks '@' SRV 10 0 389 zeit.party

Remove a MX record

  $ now dns ls # Show the records, and copy the ID you wish to remove

  $ now dns rm rec_92bd1b1e4311ffd93e4a2cae # ID from previous step

API

If you're building a tool that integrates directly with Zeit Now, you could also manage the DNS records directly through the
API. You can find the endpoint documentation here.

Zero Downtime Laravel Deployment with Github Actions

atymic — Fri, 03 Jan 2020 00:27:39 +0000

A few months ago I wrote a post on setting up zero downtime continuous deployment with Gitlab's free CI offering.
Now that Github actions is out of beta I've moved most of my CI/CD pipelines over.

In my experience Github Actions is a bit faster, but the it's not as user friendly in terms of actually building the pipelines.
The definite advantage for me is simply the fact it's built into Github, meaning I only need to use a single service. The free tier is pretty generous as well.

Just want to see the code and configuration? The example repo for this post is here.
I've tried to make it pretty generic, so you should be able to copy/paste the workflow & deploy.php into your own project with minimal changes.

Setting up Gitlab Actions

For the purposes of this post, I'm going to start from scratch with a fresh Laravel installation and guide you through the process of setting up the actions which we'll use for testing, building and deploying our code to production.

Github actions is configured using yaml files, placed in .github/workflows. They do provide an editor, but in my experience it's much easier to create and edit the files manually.
To create a new workflow you just need to create a new yaml file inside the aforementioned folder (for example, deploy.yml).

Inside this file, you can define your build stages, their dependencies, caching and many other features. For our application, we'll just use a simple three step build (github calls these jobs).

Scaffolding the workflow

In .github/workflows, create a new workflow, deploy.yml. We'll need to add some basic configuration to instruct actions to run our workflow on commits to the repo, and define a name.
Under the jobs sections, we'll need to define each job of the build/deploy process. For the moment, we'll just set out the basic scaffold and we'll populate it as set up the three jobs.

name: CI-CD # Give your workflow a name

on: push # Run the workflow for each push event (i.e commit)

jobs:
  build-js:
    name: Build Js/Css
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
        # Build JS
  test-php:
      name: Test/Lint PHP
      runs-on: ubuntu-latest
      steps:
      - uses: actions/checkout@v1
        # Test and Lint PHP code
  deploy:
    name: Deploy to Production
    runs-on: ubuntu-latest
    # Note that this needs both of the other steps to have completed successfully
    needs: [build-js, test-php] 
    steps:
    - uses: actions/checkout@v1
      # Deploy Code

Building Javascript and CSS assets

Laravel comes bundled with Mix, which provides an easy interface to webpack to build and compile your front end assets (Javascript and CSS).

It's worth noting here that we are using the upload-artifact action. This is required so that our deployment job can deploy the compiled files, as each job operates on a fresh version of the source code.

jobs:
  build-js:
    name: Build Js/Css
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1 # Download the source code
      - name: Yarn Build
        run: |
          yarn install
          yarn prod
          cat public/mix-manifest.json # See asset versions in log
      - name: Upload build files
        uses: actions/upload-artifact@v1
        with:
          name: assets
          path: public

Running PHP linting and tests

Most applications will have some sort of automated linting and (hopefully) tests. Ensuring that the code is valid and works before deploying to production is a good idea, for obvious reasons.
If the CI build finds issues such as a syntax error or failing test, it won't proceed to deployment.

In our action, we'll define a test-php job. Laravel has some sample tests built in, which we'll run. You could also run limiting and static analysis tools in this stage.
You can use the setup-php action to use a specific PHP version, or add additional extensions.

test-php:
  name: Test/Lint PHP
  runs-on: ubuntu-latest
  needs: build-js
  steps:
    - uses: actions/checkout@v1
    - name: Setup PHP
      uses: shivammathur/setup-php@master
      with:
        php-version: 7.3 # Use your PHP version
        extension-csv: mbstring, bcmath # Setup any required extensions for tests
    - name: Composer install
      run: composer install
    - name: Run Tests
      run: ./vendor/bin/phpunit

Deploying the code

Now that we've made sure that our code works & have built our assets, we can move on to the actual deployment.

Head over to your repository settings, and select the Secrets option in the sidebar.

My biggest gripe with Github actions is the secret management. For some reason, you can't edit secrets, so to update anything you need to delete and then re-create the entire secret. Hopefully they this fix this at some point.

We'll need to add a private key with access to the server so that deployer can SSH in.
It's a good idea to generate a new key specifically for deployment (ideally on a specific deployment user with minimal permissions). You'll want to add it under the variable SSH_PRIVATE_KEY.

You'll also need to add your Laravel .env file as DOT_ENV, so it can be deployed along with the code (you should never store secrets in git).

Since every CI build/deployment starts from a fresh slate, the containers ~/.ssh/known_hosts file won't be populated.
To ensure there aren't any MITM attacks, we need to our server's SSH fingerprint as a variable, SSH_KNOWN_HOSTS.

You can find you server's host fingerprint by running ssh-keyscan rsa -t <server IP>.

Notice that we have defined both of the other jobs in the needs section. Both build-js and test-php should run asynchronously (this has been inconsistent for me, often they run one at a time), and once both complete the deployment will commence.

We've also added a conditional if rule to ensure that only the master branch is deployed (we want tests/linting to run for all branches).

The job first downloads the compiled javascript/css from the build-js and applies that to current working tree.

Before we can deploy, we need to set up ssh (start ssh-agent with the provided private key, update the known_hosts) and install deployer. To make this nice and simple, I've created an action to handle everything
for you, but you could run all of the shell commands manually instead.

atymic/deployer-php-action

The job finally runs dep deploy, which uses deployer to carry out the zero downtime deployment (we'll set it up in the next section).

deploy:
  name: Deploy to Production
  runs-on: ubuntu-latest
  needs: [build-js, test-php]
  if: github.ref == 'refs/heads/master'
  steps:
  - uses: actions/checkout@v1
  - name: Download build assets
    uses: actions/download-artifact@v1
    with:
      name: assets
      path: public
  - name: Setup PHP
    uses: shivammathur/setup-php@master
    with:
      php-version: 7.3
      extension-csv: mbstring, bcmath
  - name: Composer install
    run: composer install
  - name: Setup Deployer
    uses: atymic/deployer-php-action@master
    with:
      ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
      ssh-known-hosts: ${{ secrets.SSH_KNOWN_HOSTS }}
  - name: Deploy to Prod
    env:
      DOT_ENV: ${{ secrets.DOT_ENV }}
    run: dep deploy production --tag=${{ env.GITHUB_REF }} -vvv

Putting it all together

Now that we've got all of our jobs configured your workflow should look something like the one below.
Feel free to copy and paste this into your own project.

name: CI-CD

on:
  push:
    branches: master

jobs:
  build-js:
    name: Build Js/Css
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v1
      - name: Yarn Build
        run: |
          yarn install
          yarn prod
          cat public/mix-manifest.json # See asset versions in log
      - name: Upload build files
        uses: actions/upload-artifact@v1
        with:
          name: assets
          path: public
  test-php:
    name: Test/Lint PHP
    runs-on: ubuntu-latest
    needs: build-js
    steps:
      - uses: actions/checkout@v1
      - name: Setup PHP
        uses: shivammathur/setup-php@master
        with:
          php-version: 7.3 # Use your PHP version
          extension-csv: mbstring, bcmath # Setup any required extensions for tests
      - name: Composer install
        run: composer install
      - name: Run Tests
        run: ./vendor/bin/phpunit
  deploy:
    name: Deploy to Production
    runs-on: ubuntu-latest
    needs: [build-js, test-php]
    if: github.ref == 'refs/heads/master'
    steps:
      - uses: actions/checkout@v1
      - name: Download build assets
        uses: actions/download-artifact@v1
        with:
          name: assets
          path: public
      - name: Setup PHP
        uses: shivammathur/setup-php@master
        with:
          php-version: 7.3
          extension-csv: mbstring, bcmath
      - name: Composer install
        run: composer install
      - name: Setup Deployer
        uses: atymic/deployer-php-action@master
        with:
          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
          ssh-known-hosts: ${{ secrets.SSH_KNOWN_HOSTS }}
      - name: Deploy to Prod
        env:
          DOT_ENV: ${{ secrets.DOT_ENV }}
        run: dep deploy production --tag=${{ env.GITHUB_REF }} -vvv

Setting up Deployer

Now that we've got the CI set up, it's time to install deployer in our project and configure it.

composer require deployer/deployer deployer/recipes

Once you've got it installed, run ./vendor/bin/dep init and follow the prompts, selecting Laravel as your framework. A deploy.php config file will be generated and placed in the root of your project.

By default, deployer uses GIT for deployments (by SSHing into the server, running git pull and executing your build steps) however since we are running it as part of a CI/CD pipeline (our project has been built and is ready for deployment) we'll use rsync to copy the files directly to the server instead.

Open your deploy.php in a code editor. Copy and paste the code below into your deploy.php above the hosts section.

<?php

namespace Deployer;

// Include the Laravel & rsync recipes
require 'recipe/laravel.php';
require 'recipe/rsync.php';

set('application', 'dep-demo');
set('ssh_multiplexing', true); // Speed up deployment

set('rsync_src', function () {
    return __DIR__; // If your project isn't in the root, you'll need to change this.
});

// Configuring the rsync exclusions. 
// You'll want to exclude anything that you don't want on the production server.  
add('rsync', [
    'exclude' => [
        '.git',
        '/.env',
        '/storage/',
        '/vendor/',
        '/node_modules/',
        '.github',
        'deploy.php',
    ],
]);

// Set up a deployer task to copy secrets to the server. 
// Grabs the dotenv file from the github secret
task('deploy:secrets', function () {
    file_put_contents(__DIR__ . '/.env', getenv('DOT_ENV'));
    upload('.env', get('deploy_path') . '/shared');
});

Next, we need to set up our hosts. In this example, we'll only use a single host but deployer supports as many as required. Copy the code below into your deploy.php, replacing the existing hosts block. You'll need to customise it for your specific server configuration.

// Hosts
host('production.app.com') // Name of the server
    ->hostname('178.128.84.15') // Hostname or IP address
    ->stage('production') // Deployment stage (production, staging, etc)
    ->user('deploy') // SSH user
    ->set('deploy_path', '/var/www'); // Deploy path

Next, we'll set up the steps that deployer will execute as part of our deployment. These can be customised to your needs, for example you could use the artisan:horizon:terminate task to restart your horizon queues. Copy the block below into your deploy.php, replacing the tasks section.

after('deploy:failed', 'deploy:unlock'); // Unlock after failed deploy

desc('Deploy the application');
task('deploy', [
    'deploy:info',
    'deploy:prepare',
    'deploy:lock',
    'deploy:release',
    'rsync', // Deploy code & built assets
    'deploy:secrets', // Deploy secrets
    'deploy:shared',
    'deploy:vendors',
    'deploy:writable',
    'artisan:storage:link', // |
    'artisan:view:cache',   // |
    'artisan:config:cache', // | Laravel specific steps 
    'artisan:optimize',     // |
    'artisan:migrate',      // |
    'deploy:symlink',
    'deploy:unlock',
    'cleanup',
]);

Configuring your Server

We'll also need to make a few changes the nginx or apache configuration files on the server.

You want to set your web server root to deploy_path (set in your deploy.php) + /current/public. For example, in our case this is /var/www/current/public.

You'll also need to make sure that your deploy user has the correct permissions to write to the deployment path. Deployer's deploy:writable task will ensure that the folders have the correct permissions so your web server user can write them.

First deployment 😎

Now that everything's set up, it's time to test our pipelines!

Commit your workflow anddeploy.php as well as your composer json/lock and push! If all goes well, you can head over to the Actions tab on your Github repo and watch your build/deployment progress.

If anything goes wrong, check the CI logs. The logs from github actions can sometimes be a bit opaque, but as long as your yaml if structured correctly it's usually fairly obvious as to what went wrong (missing SSH keys/fingerprints, invalid server config, etc).

If everything went well, you'll have the latest version of your project deployed. Any new commits will be built and deployed without any interruption for you users.

Here's the first deployment of my test repo, and one with a migration.

Wrapping up

This post outlines a basic CI/CD pipeline, but there's plenty of improvements and additions that could be made, such as adding staging environments, release notifications, multi server deployment (for load balanced server groups).

I hope you enjoyed this post and it helped you improve your builds & deployments, or migrate existing ones to github actions.

If you have any questions, leave a comment below & i'll do my best to respond to them all.

DEV Community: atymic

Running Laravel on DigitalOcean's App Platform

Deploying your first App

Adding a Queue Worker

Success, we're live 🎉

Review + Pros/Cons

Pros

Cons

Wrapping up

Further reading

MySQL Performance Tuning - Speeding up slow inserts

Use a server backed by performant SSD storage

Increase the log file size limit

Defer disk writes by flushing the log less frequently

Reverse engineering Google Reminders for fun and profit

Reverse Engineering Google's Reminders API

Finding the Protobuf Definitions

Building TodoSync

Serverless Cobol

Building Calndr, a free AddEvent alternative

The Tech

Using Custom Eloquent Casts in Laravel 7

How custom eloquent casts work

Creating Custom Casts

Wrapping Up

Further Reading

Submitting sitemaps on deploy with Netlify

Submitting Site Maps Automatically

Editing DNS records for ZEIT Now

Adding Records

Examples

API

Zero Downtime Laravel Deployment with Github Actions

Setting up Gitlab Actions

Scaffolding the workflow

Building Javascript and CSS assets

Running PHP linting and tests

Deploying the code

Putting it all together

Setting up Deployer

Configuring your Server

First deployment 😎

Wrapping up

Further reading