DEV Community: Luc Shelton

Homebrew and Anaconda: Configuring for Apple M1 and Intel Silicon

Luc Shelton — Tue, 17 May 2022 10:24:26 +0000

This is a quick article that will serve as a brain dump for how I've recently configured my /bin/bash profile for using Homebrew Package Manager with both M1 and Intel Silicon (with x86 emulated using Rosetta).

Installation Locations

Depending on which CPU architecture you are installing Homebrew Package Manager with will determine where brew is installed. By default, brew will be installed to the following locations.

M1 Silicon Installation Path

If you are using brew on Apple M1 silicon, then the default installation script will install Homebrew Package Manager to the following paths.

#!/bin/bash
/opt/homebrew/bin
/opt/homebrew/Caskroom

Intel Silicon Installation Path

If you are using Homebrew Package Manager on Intel silicon, or through Rosetta emulation, then you will notice that Homebrew is installed to the following paths when you use the default installation script that is available from the official homepage.

#!/bin/bash
/usr/local/bin/brew
/usr/local/Caskroom

Configuring Bash

Now that we know where brew is installed, we can configure bash to automatically load the relevant shell environment depending on which architecture is being used. This is useful if you have multiple copies of iTerm2 (for example), whereby one copy may be configured to launch with Rosetta emulation. If your Terminal instance (including iTerm2) is launched with Rosetta emulation, it will automatically attempt to use Homebrew at the listed installation path above (under "Intel Silicon"). However, because both installations are at different paths, we need some logic to detect which architecture (emulated or not) is being actively used, so we know which shell environment to load.

Refer to the logic below. It uses a command called "uname" to detect which CPU architecture is actively in use. Based on the resulting value, it will then load the relevant shell environment. x86_64 is considered to be Intel Silicon, and arm64 is considered to be Apple M1.

Homebrew Shell Environment

The snippet below describes the logic for automatically loading Homebrew shell environment depending on the architecture.

#!/bin/bash
if [["$(uname -m)" == "x86_64"]]; then
    echo "Loading: Homebrew (x86)"
    eval "$(/usr/local/bin/brew shellenv)"
    CONDA_BREW_PATH=/usr/local/Caskroom/miniconda
else
    echo "Loading: Homebrew (ARM)"
    eval "$(/opt/homebrew/bin/brew shellenv)"
    CONDA_BREW_PATH=/opt/homebrew/Caskroom/miniconda
fi

Anaconda Shell Environment

The snippet below describes the logic (typically generated by Anaconda) for loading the relevant shell environment.

#!/bin/bash
if [["$(uname -m)" == "x86_64"]]; then
    # >>> conda initialize >>>
    # !! Contents within this block are managed by 'conda init' !!
    __conda_setup="$('/usr/local/Caskroom/miniforge/base/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
    if [$? -eq 0]; then
        eval "$__conda_setup"
    else
        if [-f "/usr/local/Caskroom/miniforge/base/etc/profile.d/conda.sh"]; then
            . "/usr/local/Caskroom/miniforge/base/etc/profile.d/conda.sh" # commented out by conda initialize
        else
            export PATH="/usr/local/Caskroom/miniforge/base/bin:$PATH" # commented out by conda initialize
        fi
    fi
    unset __conda_setup
    # <<< conda initialize <<<
elif [["$(uname -m)" == "arm64"]]; then
    # >>> conda initialize >>>
    # !! Contents within this block are managed by 'conda init' !!
    __conda_setup="$('/opt/homebrew/Caskroom/miniforge/base/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
    if [$? -eq 0]; then
        eval "$__conda_setup"
    else
        if [-f "/opt/homebrew/Caskroom/miniforge/base/etc/profile.d/conda.sh"]; then
            . "/opt/homebrew/Caskroom/miniforge/base/etc/profile.d/conda.sh" # commented out by conda initialize
        else
            export PATH="/opt/homebrew/Caskroom/miniforge/base/bin:$PATH" # commented out by conda initialize
        fi
    fi
    unset __conda_setup
    # <<< conda initialize <<<
fi

Configuring iTerm to Launch with Rosetta Emulation

If you are running on Apple M1 silicon, you can still run the same applications with x86 emulation by using Rosetta. You can install Rosetta from the command-line by running the following command.

#!/bin/bash
/usr/sbin/softwareupdate --install-rosetta --agree-to-license

And now you can create a copy of your iTerm installation, and modify the copy to launch with Rosetta emulation. Refer to the screenshots below for greater reference.

A picture of "Applications" in macOS finder, with a duplicate or copy of iTerm.

With your duplicate instance of iTerm, you can then modify the properties so that it launches with Rosetta emulation.

After you've duplicated your iTerm installation and renamed it appropriately, you will then have to modify the properties so that it launches with Rosetta.

Additionally, if you want to be able to interchangeably use either Homebrew installations while using one kind of architecture over the other, then you can make use of the following aliases that can be added to your ~/.profile file. It should be noted that this snippet includes support for pyenv, a tool that is used for managing multiple installations of Python on the same machine.

#!/bin/bash
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
alias ibrew='arch -x86_64 /usr/local/bin/brew'
alias mbrew='arch -arm64e /opt/homebrew/bin/brew'

There you have it. You should be able to use Homebrew with both architectures from the same Apple M1 machine.

Perforce: Installing and Configuring on Ubuntu

Luc Shelton — Sun, 08 May 2022 15:05:41 +0000

In this article I will outline in clear simple steps what is required for installing command-line tooling for Perforce on Ubuntu. These instructions are for those who are wish to install it using APT (apt install etc.), instead of manually installing it themselves by downloading and copying the binaries from the official website to your system's /usr/bin/local directory. This method guarantees that you will always receive updates for as long as you have your apt configured to source the Perforce repository.

#!/bin/bash
apt update && apt upgrade -y

Configuration

This section outlines some essential steps that must be done before installing Perforce using apt.

Add Trusted Package Repository Keys

The command below does the following.

Performs a HTTP GET request with no additional verbose logging.
Outputs the contents of the response to standard output.
Pipes the content of standard output to gpg, a tool used for encryption, and decodes the file which is typically encoded in base64 (making it transmittable as text).
Stores the contents of the base64 decoded file into a file path that can then be read by apt when searching for packages from the Perforce repository (read next).

#!/bin/bash
wget -qO - https://package.perforce.com/perforce.pubkey | sudo gpg --dearmor -o /usr/share/keyrings/perforce-archive-keyring.gpg

Add Repository Source

This section outlines steps required for adding the Perforce repository as an installable source.

The following commands need to be executed so that apt will be able to pull package information from the official Perforce package repository. You'll note that it references the public key that we downloaded in the previous section; this is used for verifying that packages installed from this source are signed with this key-pair.

#!/bin/bash
echo "deb [signed-by=/usr/share/keyrings/perforce-archive-keyring.gpg] http://package.perforce.com/apt/ubuntu $(lsb_release -c -s) release" | sudo tee /etc/apt/sources.list.d/perforce.list
sudo apt update

At this point, helix-cli should now be available as an installable package from the Perforce repository. You can check whether the new package repository source is available by running the following command.

Resulting output from running the command "apt search", providing that you have followed the steps correctly.

Installation

This section outlines how to install Helix Core CLI (p4), assuming that you have successfully completed the steps above.

Installing helix-cli is as simple as running the following command.

#!/bin/bash
apt install helix-cli

If everything installed correctly, you should now be able to use the p4 command from the terminal. You can check if it is correctly installed simply by running the command "p4", or by running the following command instead.

#!/bin/bash
command -v p4

The resulting output should be the absolute path to where the command is made available from in the $PATH environment variable.

If you wish to be even more pedantic, you can run the following command which should display the status of the newly installed package.

#!/bin/bash
dpkg -l helix-cli

That's it.

SilverStripe: Generating URL Segments for DataObjects

Luc Shelton — Sat, 23 Apr 2022 18:23:27 +0000

As documented on the official SilverStripe website, you can render DataObjects as individual pages using controller actions. This is fine if you're comfortable in using the ID of the DataObject as part of the page URL, but if you're looking to create and use human-friendly URLs similar to that of SiteTree pages, then you will want to implement something that generates hyphenated and sanitized URL segments instead.

You can consider a "URL segment" to be a sanitized, human-friendly readable string generated from the Title field of the DataObject it is intended for. It doesn't contain any special URL encoding characters, and strictly only uses alpha-numeric characters. Anything that isn't alpha numeric is replaced by a hyphen.

By default, these are automatically generated for pages that derive from the SiteTree type. Unfortunately this same kind of behaviour does not exist for DataObjects, but this can be easily remedied by introducing a new data extension to your project and applying it to your DataObject type's list of extensions.

<?php

use SilverStripe\ORM\DataExtension;
use SilverStripe\View\Parsers\URLSegmentFilter;

class URLSegmentExtension extends DataExtension
{
    private static $db = [
        'URLSegment' => 'Varchar(255)'
    ];

    public function onBeforeWrite()
    {
        parent::onBeforeWrite();

        if ($this->owner->hasField('URLSegment')) {
            if (!$this->owner->URLSegment) {
                $this->owner->URLSegment = $this->generateURLSegment($this->owner->Title);
            }

            if (!$this->owner->isInDB() || $this->owner->isChanged('URLSegment')) {
                $this->owner->URLSegment = $this->generateURLSegment($this->owner->URLSegment);
                $this->makeURLSegmentUnique();
            }
        }
    }

    public function IsURLSegmentInUse($URLSegment)
    {
        $class = $this->owner;
        $items = $class::get()->filter('URLSegment', $URLSegment);

        if ($this->owner->ID > 0) {
            $items = $items->exclude('ID', $this->owner->ID);
        }

        return $items->exists();
    }

    public function makeURLSegmentUnique()
    {
        $count = 2;
        $currentURLSegment = $this->owner->URLSegment;

        while ($this->IsURLSegmentInUse($currentURLSegment)) {
            $currentURLSegment = preg_replace('/-[0-9]+$/', '', $currentURLSegment) . '-' . $count;
            ++$count;
        }

        $this->owner->URLSegment = $currentURLSegment;
    }

    public function generateURLSegment($title)
    {
        $filter = URLSegmentFilter::create();
        $filteredTitle = $filter->filter($title);

        $ownerClassName = $this->owner->ClassName;
        $ownerClassName = strtolower($ownerClassName);

        if (!$filteredTitle || $filteredTitle == '-' || $filteredTitle == '-1') {
            $filteredTitle = "$ownerClassName-$this->ID";
        }

        return $filteredTitle;
    }
}

Breakdown

There's a bit to digest here, but the summary of what's going on in this extension is the following.

This extension registers the field "URLSegment" to whichever DataObject that this extension type is appended to through the .yml configuration file.
Each time the DataObject is published or written through from the Object-Relational Mapping system, it will automatically generate a new URL segment string if none has been generated yet.
It achieves this by checking the URLSegment field, and if it has not been changed or updated, then it will attempt to use the Title field of the DataObject to generate a string suitable for the URLSegment database field.
It checks recursively to see if there is an existing DataObject in the database that has the generated URLSegment already.
If there is an existing DataObject in the database with the generated URLSegment, it will instead increment an integer, append it to the string, and recursively check again to see if the newly generated URLSegment has already been assigned.
Once it has determined that the generated string is not in use, it will then apply it to the DataObject's field and continue with committing the data to the database.

Usage

Now that we have an applicable URLSegment string for our DataObject, we can put it into use in a similar manner as what has already been described on these pages. Except, in this instance we will be making use of the URLSegment field for filtering (based on the $Action value) instead of the ID of the DataObject.

The code for achieving this would look something like this.

if (is_numeric($params['ID'])) {
    $eventImages = EventImage::get()->filter([
        'EventID' => $this->dataRecord->ID,
        'ID' => intval($params['ID'])
    ]);
} else {
    $eventImages = EventImage::get()->filter([
        'EventID' => $this->dataRecord->ID,
        'URLSegment' => $params['ID']
    ]);
}

Ensure that you have configured your project to use this data extension. In the example configuration file below, I am applying the extension that I've written above to the DataObject that I want the URL segments to be generated for.

Once the configuration file is saved, simply navigate to /dev/build?flush=all on your project, and this should now automatically generate the URL segments each time you publish modifications to your DataObjects.

---
Name: urlsegments
---
Portfolio\Models\TechnologyTag:
  extensions:
    - Portfolio\Extensions\URLSegmentExtension

And that's all there is to it. You could make further modifications if you like, including ensuring that the URLSegment field for a DataObject uses the appropriate form field when modifying it from the CMS admin.

CSS: Loading Custom Fonts

Luc Shelton — Sat, 23 Apr 2022 13:16:25 +0000

There are a number of methods you can adopt for optimizing overall page performance of your website. An area of concern for optimization can often be custom fonts. Modern websites often make use of custom fonts as part of their overall style of design, however there unfortunately comes the performance overhead in loading them. There are number of tricks you can use to ensure that you are loading custom fonts correctly using CSS.

My website uses two custom fonts that were heavily inspired by the likes of Hashnode - a blogging platform for developers. Those fonts are Manrope and Inter, which I personally think make for a great font pairing.

Minification with Font Awesome

Static font assets (such as WOFF2 and TTF) can't be "minified", however if you are using a icon library such as Font Awesome that ships with a custom font for rendering, then it's important to ensure that you are using minification for the CSS asset file that it also comes with (typically fontawesome.css). For example, the CSS file that it ships with has classes for every single icon that is available as part of the font glyph. This is what makes each icon capable of being rendered as part of text (using the <i></i> tag).

Depending on what your tech stack is will ultimately determine what kind of tooling you ought to use for minification. You may wish to consider using one of the following.

JavaScript and Single-Page Application Frameworks (SPAs)
PHP and SilverStripe

Preloading JavaScript Libraries

Font Awesome ships with a supporting JavaScript library that will automatically load the correct icon as a fallback, if the user's browser does not support any of the underlying static assets (including .ttf, .eot, and .woff2). If you want to begin loading the supporting JavaScript library asynchronously, you can use the following code snippet.

<link rel="preload" href="/js/fontawesome.min.js" as="script">
<!-- ... -->
<script>
  var usedLaterScript = document.createElement('script');
  usedLaterScript.src = '/js/fontawesome.min.js';
  document.body.appendChild(usedLaterScript);
</script>

Use WOFF2

Typically most systems make use of the True-Type Format (TTF) for font glyphs that are used system-wide. This is perfectly fine for local usage on each system, but unfortunately comes with added baggage of irrelevant meta data used solely for system installation, when loading static assets over the web. WOFF2 was developed to address this by ensure that compression was taken into account, and that additional metadata originally intended for local system installation was removed.

Blocking Resources

When developing web applications, you want to ensure that you defer loading non-critical resources otherwise this will heavily impact your First-Contentful Paint load times. "First-Contentful Paint" is the metric used by most web benchmarks for indicating how long it takes for the browser to load all critical static assets, in order to begin rendering elements on screen.

Ideally you want to ensure that the browser is rendering your website in as little time possible, with perhaps the caveat that some styling or CSS classes will be loaded after elements have begun to render to the page.

Prioritise Loading Custom Fonts (Asynchronously)

Custom font assets should typically ahead of most static assets on your website. To ensure that this happens, you must declare a <link> tag within the head of your page with the following attributes included.

<link rel="preload" as="font" type="font/woff2" href="/resources/themes/portfolio-devlog-theme/fonts/manrope/woff2/manrope.woff2" crossorigin>

Given that the <link> tag can be used for various static assets (including .css and .js files), it's important to identify the asset's type by using the following attributes as="font" type="font/woff2".

Prevent Invisible Text

If you choose to load your custom font asynchronously, you still run the risk of displaying text that will appear as invisible on your page. The reasoning for is simple - the font has not loaded yet, so the browser has nothing to render the text with! Fortunately there's a convenient way around this, and all it requires is ensure that your @font-family declaration in your static CSS file asset is properly declared.

@font-face {
    font-family: 'Manrope';
    font-display: swap;
    unicode-range: U+000-5FF;
    src: local('Manrope'), url('/resources/themes/portfolio-devlog-theme/fonts/manrope/woff2/manrope.woff2') format('woff2');
}

The above snippet can be declared in the head of a page as part of an inline <style> declaration, or included as part of a static .css file asset.

I'm using my own website here as an example. Pay close attention to the CSS attribute font-display. We're setting the value to swap simply so that placeholder text (and fallback font) is used while the custom font is still being loaded (providing that it can be loaded).

Quoting the exact description from the W3C draft:

In other words, the browser draws the text immediately with a fallback if the font face isn’t loaded, but swaps the font face in as soon as it loads.

Quoted from official W3C draft.

We're also doing three other other additional things...

Stating that the browser should avoid loading the custom font if it's already installed locally.
Load only a certain character range of glyphs from the font (e.g. latin glyphs).
Load the WOFF2 version of the font (where possible), otherwise use WOFF and TTF version of the font as fallbacks if the browser does not support it.

Automatically Serve WebP Images with Silverstripe and NGINX

Luc Shelton — Sat, 12 Mar 2022 19:47:42 +0000

Overview

Fast page load times are vital for any modern website with heavy traffic. To achieve quick response times, it often this means having to compress, minify, and cache static page assets that are served regularly on page load; including JavaScript, plain HTML, or images in particular. However, the problem with heavy compression often comes with a loss of quality in image assets, which can compromise aesthetics or styling, and detract from overall user experience.

WebP

Fortunately there's a solution to serving mostly lossless images with small payloads. Quoting from WebP's homepage.

WebP is a modern image format that provides superior lossless and lossy compression for images on the web. Using WebP, webmasters and web developers can create smaller, richer images that make the web faster.
The WebP image file format is not anything terribly new and has been for a while now, however many websites have yet to implement steps for generating .webp images from their existing image assets.

Integrating with Silverstripe

As a case study for this blog, I've decided to talk about how to integrate the usage of this image file format with Silverstripe. What is Silverstripe? For those of you who don't already know...

Silverstripe CMS is a free and open source content management system and framework for creating and maintaining websites and web applications. It provides an out of the box web-based administration panel that enables users to make modifications to parts of the website, which includes a WYSIWYG website editor.

You can otherwise consider it as an alternative CMS to WordPress, although arguably better and less bloated. 😁 Assuming that you already are working on a Silverstripe project, you can install a convenient module that I've developed that will automatically convert any image asset that has been rasterized, resized, or scaled from your Silverstripe project (through using Silverstripe's image templating syntax).

You can install the module with the following command.

composer require loveduckie/silverstripe-webp-image

After installing this with the Composer package manager, simply run /dev/build?flush=all. Without having done anything else, your Silverstripe project is now automatically generating and serving .webp assets on page load! Woah! Let's take a look at how it's achieving this.

If we take a look at the repository for the module, you will see that we are using a LibGD function (made available as part of PHP) for converting PNG images when intercepting requests for serving assets. This line in particular.

...

public function createWebPImage($path, $filename, $hash, $variant = false)
{
    if (function_exists('imagewebp') && function_exists('imagecreatefromjpeg') && function_exists('imagecreatefrompng') && function_exists('imagecreatefromgif')) {
        $orgpath = './'.$this->getAsURL($filename, $hash, $variant);

        list($width, $height, $type, $attr) = getimagesize($path);

        switch ($type) {
            case 2:
                $img = imagecreatefromjpeg($path);
                imagewebp($img, $this->createWebPName($orgpath), $this->webp_quality);
                break;
            case 3:
                $img = imagecreatefrompng($path);
                imagesavealpha($img, true); // save alphablending setting (important)
                imagewebp($img, $this->createWebPName($orgpath), $this->webp_quality);
                break;
        }
        imagedestroy($img);
    }
}

...

After the conversion is made in memory (using imagewebp), it then gets served to disk using the file-path generated by createWebPName($orgPath). This path is something along the lines of this.

/silverstripe-project-root/public/assets/Upload/your-image-filename.png.webp

The way that the file is named is important to note for later on.

Serving Generated WebP Images from NGINX

You can use the following NGINX configuration for serving WebP images automatically from your Silverstripe project's public assets directory. You can compare this configuration to the default one that Silverstripe has on their documentation pages for NGINX connectivity with PHP-FPM.

map $http_accept $webp_suffix 
{
  default   "";
  "~*webp"  ".webp";
}

server 
{
  listen 80;
  listen [::]:80;
  server_name ${WEBSITE_DOMAIN_NAMES};
  server_tokens off;

  include /etc/nginx/mime.types;

  ...

  location / {
      try_files $uri /index.php?$query_string;
  }

  location ~* /assets/.+\.(?<extension>jpe?g|png|gif|webp)$ 
  {
    gzip_static on;
    gzip_types image/png image/x-icon image/webp image/svg+xml image/jpeg image/gif;

    add_header Vary Accept;
    expires max;
    sendfile on;
    try_files "${request_uri}${webp_suffix}" $uri =404;
  }

 ...

  location /index.php {
      fastcgi_read_timeout 10000;

      fastcgi_buffers 4 65k;
      fastcgi_buffer_size 64k;
      fastcgi_busy_buffers_size 128k;
      fastcgi_keep_conn on;
      fastcgi_pass   your-php-fpm-server-goes-here:9000;
      fastcgi_index  index.php;
      fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
      include        /etc/nginx/fastcgi_params;
  }
}

From the snippet above, ${WEBSITE_DOMAIN_NAMES} is automatically replaced when the NGINX server starts by using envsubst, but you can replace it with your own domain names, too. I've simply taken this configuration from my own webserver, as I am using the Alpine Linux Docker container image for NGINX.

The most important part of this configuration is this section.

map $http_accept $webp_suffix 
{
  default   "";
  "~*webp"  ".webp";
}

You can think of it as a switch statement in most languages. Depending on what $http_accept resolves to when a request is served, will ultimately determine whether the resulting variable $webp_suffix will resolve to either .webp or nothing. This resulting variable will then be used when serving static files from the assets directory, which occurs in this part of the configuration.

  location ~* /assets/.+\.(?<extension>jpe?g|png|gif|webp)$ 
  {
    gzip_static on;
    gzip_types image/png image/x-icon image/webp image/svg+xml image/jpeg image/gif;

    add_header Vary Accept;
    expires max;
    sendfile on;
    try_files "${request_uri}${webp_suffix}" $uri =404;
  }

This configuration block basically instructs NGINX to respond to requests that end with the suffix of .jpeg, .png, .gif, or .webp. It will then attempt to locate and serve the .webp of the same requesting asset if the Accept header has been defined to include .webp.

You can see that happening on this line.

    try_files "${request_uri}${webp_suffix}" $uri =404;

Depending on what the incoming browser request has in their Accept header, will automatically determine if the $webp_suffix is populated, which will in-turn decide which version of the same asset is retrieved from your server's filesystem.

Once you've applied this configuration, you can then run the following command to test it.

nginx -t

Followed by this one to reload it, assuming that the previous one did not fail.

nginx -s reload

Now when you navigate to your running Silverstripe project, you should notice something interesting if you open up your Developer Console in Chrome.

You'll notice that even though your browser is loading an image asset with a .jpeg extension, the MIME type for the response is that of image/webp. This means that NGINX was able to locate the generated .webp image asset, and was able to serve it back. This is good news!

Conclusion

And there you have it. An end-to-end solution for integrating .webp image assets with your Silverstripe project, and how to serve them through a modern web-server such as NGINX. Using some clever NGINX configuration trickery, we can automatically pick and serve .webp should it be discoverable on the filesystem. These image assets are being automatically generated by Silverstripe whenever templates are being rendered that make use of the syntax for manipulating images.

Useful Links

NGINX configuration for Silverstripe with WebP image serving

Thanks for reading! Feel free to follow me on Twitter, or check out my website where I often blog about other topics.

SilverStripe: Integrating Memcached Caching with Docker

Luc Shelton — Fri, 04 Mar 2022 17:07:05 +0000

In this blog post, I am going to demonstrate how you can hook up an existing memcached server instance with SilverStripe. The purpose for this is so that you can in-memory based caching, instead of the default alternative that is available which is a file-based OpCache solution that SilverStripe uses as the fallback.

Configuration

There is already existing documentation on the SilverStripe pages for configuring memcached, but it makes use of the YAML-based configuration engine along with SilverStripe's type injector for instantiating the type. For some reason, this configuration sample doesn't work despite being a convincing solution.

Instead, for my solution, I will be making use of the _config.php file for writing the code explicitly for instantiating the memcached caching interface adapter. It looks something like this.

<?php
use SilverStripe\Core\Injector\Injector;
use SilverStripe\Core\Cache\MemcachedCacheFactory;

$cacheClient = new Memcached();
$cacheClient->addServer('your-memcached-server-hostname-goes-here', 11211);
$cacheFactory = new MemcachedCacheFactory($cacheClient);

Injector::inst()->registerService($cacheFactory, "SilverStripe\\Core\\Cache\\CacheFactory");
Injector::inst()->registerService($cacheFactory, "CacheFactory");
Injector::nest();

You can alternatively find the Gist snippet for this configuration here. Afterwards, all you need to do is simply navigate to /dev/build?flush=all, and it should automatically work providing that you have correctly configured your memcached server.

What does it do?

Basically this configuration updates the class manifest of the Injector service, to say that should a type be instantiated named "CacheFactory" or more specifically SilverStripe\Core\Cache\CacheFactory, you should the instantiate the MemcachedCacheFactory instead. All we're doing in the snippet above is instantiating the adapter, or configuration for memcached with the relevant properties, and then ensuring that it is used everywhere in the Silverstripe project.

The injector is globally present service that is responsible for instantiating objects of every detectable type in the project.

Refer to the section below if you need further information on running this with Docker.

Docker

If you're using Docker, you might also appreciating using this docker-compose configuration. This is how best to describe a memcached service if you are intending on using it with your SilverStripe instance.

version: '3.9'

services:
  memcached:
    container_name: '${SERVICE_NAME}-memcached'
    image: memcached:1.6.9-alpine
    restart: always
    ports:
      - "11211:11211"
    environment:
      MEMCACHED_MEMUSAGE: 256
      MEMCACHED_THREADS: 8

Using this docker-compose configuration is as simple as running the following command.

#!/bin/bash
docker-compose up memcached

And that's it. Leave a comment if you need any more information.

SilverStripe: JSON-LD Structured Data

Luc Shelton — Fri, 04 Feb 2022 11:51:17 +0000

This website is currently powered by SilverStripe which is an all encompassing Content Management System (CMS) and framework for building web applications using PHP. I chose to adopt this framework some time ago because I had yet to find anything that came close, using the web-development stack that I would otherwise enjoy using (TypeScript etc.).

SilverStripe fortunately comes out of the box with a lot of features (including blogging), but there are some things that are missing that could make it much more competitive when developing websites that are performant with search engine results.

JSON-LD

I've been spending some time trawling through the Google Developer pages trying to find better ways of optimising my website for search engine performance. I've already written a blog post here on best practices for optimising page performance and load times (which can impact search result rankings), but this blog post is more focused on exposing relevant metadata that search engines can use for recognising the type of content that your website has.

In particular, Google provides support for enhancing search results for pages that publishes information such as Recipes, Events, and News Articles.

Extending SilverStripe for JSON-LD Support

After reading through the Google Developer pages, I naturally was inclined to search SilverStripe's module repository to find out whether or not there was any thing already available that suited my needs. While there were some interesting projects that seemed almost relevant, they also still seemed largely incomplete, or only supported an older version of the framework.

Relevant SilverStripe module projects include:

I decided to take it upon myself to write a module extension that would suit my needs, and offer comprehensive support to the entirety of the available and supported schema supported by Google search engine crawlers.

GitHub Repository
- https://github.com/LoveDuckie/silverstripe-json-ld-structured-data
Packagist
- https://packagist.org/packages/loveduckie/silverstripe-json-ld-structured-data

You can install my module with the following command:

#!/bin/bash
composer require loveduckie/silverstripe-json-ld-structured-data

It shouldn't require any other additional configuration, but you will need to add the following code to your parent Page.ss template file.

<html>
<head>

...

$PageStructuredData().RAW

...

</head>
<body>

...

</body>
</html>

This will the invoke this piece of code that is responsible for generating breadcrumbs and invoking any extension functions available for the SilverStripe\CMS\Model\SiteTree type.

<?php

namespace LoveDuckie\SilverStripe\JsonLDStructuredData\Extensions;

use Exception;
use SilverStripe\ORM\DataExtension;
use SilverStripe\Control\Director;
use SilverStripe\Core\Config\Configurable;
use SilverStripe\Core\Config\Config;
use SilverStripe\SiteConfig\SiteConfig;
use SilverStripe\CMS\Controllers\ContentController;

class JsonLDStructuredDataExtension extends DataExtension
{

...

    public function PageStructuredData()
    {
        $structuredDataContainer = [];
        return $this->InjectedStructuredData($structuredDataContainer);
    }

...

}

You can view the rest of the source code for this module on my GitHub repository here.

NGINX: Default Server Configurations

Luc Shelton — Sun, 04 Jul 2021 15:28:27 +0000

I recently encountered a critical issue when configuring my NGINX server (that serves this website), when I had multiple (unrelated) domain names configured to point to the same virtual private server (VPS). The problem was that only one set were meant to be in use (such as loveduckie.*). Unfortunately, this then meant that the remaining domain names (the ones intended to be left unused) were erroneously pointing to my portfolio website when they should not have been. This is can be particularly problematic, because Google can severely relegate the search ranking for your website, if it deems it not to be the "canonical" version of it.

What this means exactly is that there could be two completely separate and unrelated domain names pointing to the same page or content, but because Google considers the wrong one to be the "one true source", it then defines it as the canonical version which is not our intention. I don't want an unrelated domain name to become the "canonical" source for my portfolio!

To fix this, I produced a NGINX configuration that ensured that any time the unused set of domains were visited, they would be redirected to a default error landing page (much like you would expect when navigating to a HTTP 404). This means that subsequent crawls from Google will be able to determine a difference between my portfolio's domain names, and the ones that are considered to be unrelated.

The error pages look a little something like this.

The default landing page that is presented to viewers when they navigate to the wrong domain name.

And of course, there are custom error pages depending on the HTTP status code that is being returned.

The error page that is served to the user when the HTTP 404 error code is returned.

Aside from the overkill templating of the error pages with Bootstrap, there's nothing particularly fancy about this so far.

NGINX Configuration

Configuring your NGINX server is pretty straight forward, and only relies on you needing to use a particular set of keywords that NGINX parses when reading your configuration files. To begin with, you are going to want to create a new server configuration file called default.conf. The name of the configuration file is largely irrelevant, as your NGINX server should be configured to read all configuration files under a certain directory. For instance, your default nginx.conf configuration file should contain a statement such as include /etc/nginx/conf.d/*.conf so that it can read all configuration files (that presumably have server blocks) and load your virtual servers accordingly.

server 
{
    listen 80 default_server;
    listen [::]:80 default_server;
    listen 443 ssl default_server;
    listen [::]:443 ssl default_server;
    server_name_in_redirect off;
    server_name default_server;
}

So far, so good. All this server block is ensuring that it is binding itself to both port 80 and 443, which are used for HTTP and HTTPS traffic. You'll also note the usage of "default_server", which basically tells NGINX that if the domain name does not have a server block configuration available for it on the server, then simply make use of this "default" server block configuration instead.

There's a few other things going on here as well.

server_name_in_redirect off; basically states that there doesn't need to be a match between the host name defined in the HTTP request Host header and the server_name configuration value in order for the our default configuration to be considered a valid match.
server_tokens off; is not strictly related to this article, but basically states that the HTTP response mustn't specify that this was served by NGINX (i.e. Server HTTP header).

Handling Specific HTTP Errors

In the instance that someone navigates to a page that does not exist or cannot be served by any of the "server block" configurations loaded by NGINX, you will likely want to redirect them to a 40x or 50x error status page. Configuring page redirects for both range of error codes is straight forward.

server 
{

    ...

    root /var/www/default;
    index index.html index.htm;

    location ~* ^.+ {
        try_files $uri $uri/ =404;
    }

    location / {
        try_files $uri $uri/ =404;
    }

    error_page 404 /404.html;
    error_page 403 /403.html;
    location = /404.html {
        root /var/www/default;
    }

    error_page 500 502 503 504 /500.html;
    location = /500.html {
        root /var/www/default;
    }

    ...

}

In the example above, I set the root directory to /var/www/default which is the path I am using for storing static page files for my error pages in my NGINX Docker container (as shown in the screenshots above). If you are building a NGINX service from a Docker image, you will want to make sure that the path exists, and that there are static files that you can serve from the path.

Handling SSL Traffic

Next, you are going to want to make sure that you have some kind of SSL certificate that you can use for serving HTTPS traffic. Unless you actually have a valid HTTPS certificate for the traffic that you are intending on redirecting, you will want to create your own self-signed one using the available SSL command-line tooling.

Installing Dependencies for SSL in Docker (Optional)

If you are using the Alpine Linux variant of the NGINX Docker image (nginx:stable-alpine for example), you must ensure that you've installed the required dependencies through the Alpine Linux package manager.

RUN apk add --no-cache openssl

And then you will want to generate your own self-signed certificate, and then store it somewhere appropriate in the filesystem for the Docker container.

RUN openssl req -new -x509 -nodes -days 365 -newkey rsa:4096 -extensions 'v3_req' \
        -keyout /etc/nginx/ssl-default/default-privkey.pem \
        -out /etc/nginx/ssl-default/default-fullchain.pem \
        -config /etc/nginx/openssl-gen.cnf > /dev/null 2>&1

You'll note that this command-line expression is referring to a configuration file that is located at /etc/nginx/openssl-gen.cnf. This is a custom configuration file that I've copied into the Docker image from a previous COPY statement. The path can be changed with wherever you decide to copy this configuration file to inside your Docker container. The configuration file looks little something like this...

[req]
default_bits = 4096
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no

[req_distinguished_name]
name = Your Name Goes Here
countryName= Your Country Name Goes Here
stateOrProvinceName = Your State or Province Name Goes Here
emailAddress = Your Email Address Goes Here
localityName = London
organizationalUnitName = Your Name Goes Here
commonName = localhost

[v3_req]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = localhost
DNS.2 = 127.0.0.1

Nothing too fancy, and it doesn't necessarily need to have the SAN (subject alternate names) definitions for the unsupported domain names that you intend on redirecting to your default landing pages. Of course, because it is a self-signed certificate (i.e. a certificate signed using your own created certificate authority), you should assume that this will throw HTTPS errors should people navigate to the domain through HTTPS.

Testing Configuration Changes

Ensure that you've tested your changes before restarting your Docker container, or reloading your configuration file.

#!/bin/bash
nginx -t

And then reload your configuration if the response is without errors.

#!/bin/bash
nginx -s reload

Alternatively, if you are running NGINX from a Docker container, you can do it from the command-line (outside of the container) using a command similar to this.

#!/bin/bash
docker exec -it your-nginx-container-name-goes-here nginx -s reload

Conclusion

Use a default configuration to prevent there being "search result collisions" between two unrelated domain names that target the same host.

I hope you found this useful. There is another approach to this, and that is to adjust the firewall configuration for your virtual private server, so that all traffic to that particular host (read: domain) name is rejected. This is largely contingent on what Linux operating system you are using, and is arguably not as convenient as managing it at container-level (i.e. from the NGINX instance itself).

You can find the complete NGINX configuration snippet for everything discussed in this article, in this Gist on GitHub.

Complete NGINX Configuration

server 
{
    listen 80 default_server;
    listen [::]:80 default_server;
    listen 443 ssl default_server;
    listen [::]:443 ssl default_server;
    server_name_in_redirect off;
    server_name default_server;
    server_tokens off;

    charset utf-8;

    access_log /var/log/nginx/host.access.log main;
    error_log /var/log/nginx/host.error.log warn;

    ssl_certificate /etc/nginx/ssl-default/default-fullchain.pem;
    ssl_certificate_key /etc/nginx/ssl-default/default-privkey.pem;

    root /var/www/default;
    index index.html index.htm;

    location ~* ^.+ 
    {
        try_files $uri $uri/ =404;
    }

    location / 
    {
        try_files $uri $uri/ =404;
    }

    error_page 404 /404.html;
    error_page 403 /403.html;
    location = /404.html 
    {
        root /var/www/default;
    }

    error_page 500 502 503 504 /500.html;
    location = /500.html 
    {
        root /var/www/default;
    }
}

Useful Reading

Find below some other useful links that I found when trying to troubleshoot my woes.

I hope you found this useful. Feel free to get in touch if you require any help!

Tips for Optimizing Page Speeds

Luc Shelton — Sat, 03 Jul 2021 14:30:55 +0000

I've been spending a lot of time optimising the page load performance of my website on both mobile and desktop devices. The bulk of my effort has been guided by the documentation listed by Google's PageSpeed Insights tool. I've documented some of the things that I've learned in the process.

Server

For the Docker container image, my biggest concern were largely to do with CPU usage and memory footprint. I am making use of a all-encompassing CMS framework called SilverStripe which (fortunately) ships with some additional features for caching to either a local filesystem, or to a distributed caching system such as Redis or Memcached. However, it does come with some added bloat, and often requires me to install a lot of additional extensions for PHP in order for it to work correctly.

NGINX

There are a few simple ways that you can overall improve page performance of your website by tweaking a few configuration options that are defined by default in NGINX.

Use GZip Compression Rules

By default NGINX only serves text/html as with gzip compression. You'll note that if you navigate to a file of any other type than a simple .html document, that the response headers won't specify the Accept-Encoding header with gzip as the corresponding value. This can be tweaked within the scope of a location configuration block.

For instance, within the location block for my SilverStripe installation's NGINX configuration, I've defined a wider range of assets that can be supported for compression.

Refer to the NGINX location block configuration below:

 location ~* /assets/.+\.(?<extension>js|css|xml|json)$
  {
    gzip on;
    gzip_static on;
    gunzip on;
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_min_length 1100;
    gzip_buffers 4 8k;
    gzip_proxied any;
    gzip_types
      text/plain
      text/xml
      text/css
      text/comma-separated-values
      image/png
      image/x-icon
      image/webp
      image/svg+xml
      image/jpeg
      image/gif
      text/javascript
      application/x-javascript
      application/javascript

}

The rule does two things:

Matches the applicable file extensions based on the incoming request, by using the regular expression pattern specified (/assets/.+\.(?<extension>js|css|xml|json)).
Depending MIME type recognised, it will then serve the request with the applicable gzip headers, ensuring that the static asset will be served as such.

Use Reverse Proxy Caching

If you are using NGINX as a reverse proxy - which is often the case for PHP-FPM servers - then you might want to consider adding rules for caching content to a local path on your NGINX instance.

Refer to the location configuration block below.

http {
    proxy_cache_path /data/nginx/cache levels=1:2 keys_zone=STATIC:10m
    inactive=24h max_size=1g;
    server {
        location / {
            proxy_pass http://1.2.3.4;
            proxy_set_header Host $host;
            proxy_buffering on;
            proxy_cache STATIC;
            proxy_cache_valid 200 1d;
            proxy_cache_use_stale error timeout invalid_header updating
                                   http_500 http_502 http_503 http_504;
        }
    }
}

Apply Cache Control Headers

I'm working on this section!

Automatically Serving WebP Optimized Image Assets

Depending on how your project is developed, it's possible to write a configuration for NGINX so that it automatically attempts to send the .webp version of an image to the requesting user, instead of the original image asset itself. The benefit of this approach is that if your server-sided software automatically converts your image assets to .webp once they are uploaded, or rendered by the server, the optimized version of the asset will be used by NGINX, and the original asset will be used as a fallback should there not be one.

map $http_accept $webp_suffix {
  default "";
  "~*webp" ".webp";
}

location ~* /assets/.+\.(?<extension>jpe?g|png|gif|webp)$ {

...

    # more_set_headers 'Content-Type: image/webp';
    add_header Vary Accept;
    sendfile on;
    try_files "${request_uri}${webp_suffix}" $uri =404;

...

}

Which translates to logic that can be otherwise described by this flowchart diagram.

A simple diagram illustrating the logic flow of requesting and retrieving a static image asset with SilverStripe using the NGINX configuration.

The above code snippet is an example of a NGINX "location block" that this website uses for serving publicly visible image assets to the requesting user. It's doing several things here, but most importantly the try_files statement at the bottom of the location block is attempting to do one task. If the .webp counterpart for the image that is being requested exists, then serve that back to the requesting user of the website. If it doesn't exist, then simply use the original version of the asset as a fallback.

Providing that you have integrated the NGINX location block correctly with your website, HTTP responses for image assets should be appearing in your developer tools as such.

A screenshot from Google Chrome's Developer Tools displaying a response from a HTTP request for an image asset on the website.

You'll notice that the Content-Type header is now resolving image/webp as opposed to the image assets actual MIME type which would typically be image/jpeg or image/png.

You might also find this project interesting: SilverStripe Automatic WebP Image Conversion

Website Performance

Improving the performance of your website itself is a significantly more complex subject, as it is of course largely contingent on which tools you use for developing your website with in the first place.

Minify Text-based Static Assets

Most modern Single Page Application frameworks such as Angular already minify and "bundle" JavaScript and CSS static assets, but if you're like me and not using a SPA framework such as Angular, then you will need to seek other tooling. "Minification" quite simply is the process of reducing the amount of unnecessary characters found in most JavaScript and CSS assets. Certain strategies in this approach include, but not limited to...

Reducing symbol length - reducing the length or size of syntactically recognized keywords such as variable names, numerical values, and operators (e.g. inline function statements converted to lambda declarations).
Removing white-space characters, and condensing all interpreted text - Applicable in both JavaScript or Cascading Style Sheet assets; this ensures there are fewer characters (and therefore bytes) saved in the file, and served to a user's browser.
Removing duplicates - Removing any potentially duplicate code that might be imported or loaded by libraries with shared dependencies.
Bundling - combining multiple JavaScript files into a single one so that it reduces the amount of HTTP requests to the server for individual JavaScript assets.
"Tree-shaking" - Removing "loose" or unused code that is otherwise found in third-party libraries. Typically this is only done in production environments.

Fortunately there's already an abundance of tooling available, and you don't need to waste your spare time on the weekend reinventing the wheel.

Tooling - NodeJS and JavaScript

Much to what I was alluding to previously about SPA frameworks, there's already a significant amount of JavaScript tooling available for minifying and bundling JavaScript and CSS assets. I personally recommend the following libraries, as they have been suitable for my projects...

Tooling - PHP

If you're like me, and you're using reluctantly using PHP because your website's backend is powered by it (ahem, SilverStripe), you will probably be seeking alternatives to what has been mentioned above.

Minify

Optimising Image Assets with WebP

As described on the official website, "WebP is a modern image format that provides superior lossless and lossy compression for images on the web". It's an low-memory image format that is widely compatible with most modern browsers and has a significantly reduced on-disk memory footprint with minimal reduction in overall image quality.

Depending on how your website is developed will ultimately determine how best to implement the usage of this image format, but the tooling available is widely compatible with most (popular) operating systems including Linux, OS X, and Windows. Furthermore, the tooling is largely compatible with other existing image formats such as JPEG, PNG, and GIF.

In general, if your website makes use of some form of backend for rendering pages (i.e. Server-Sided Rendering or SSR for short) or for serving an API for data retrieval, you should be attempting to automatically rasterize or convert image assets to WebP image file format where possible.

SilverStripe and Automatic WebP Image Conversion

SilverStripe doesn't have any official support for the WebP image format, but there are community members that have developed extensions for automatically generating .webp assets for popular formats including JPEG and PNG when SilverStripe attempts to render images at a reduced size. This largely depends on how you have developed the templates for your SilverStripe website, but arbitrary template statements used for resizing images such as $Image.ResizedImage(200, 300) and $Image.Fit(300,300) will trigger SilverStripe to rasterize or resize the image. This behaviour or functionality can be overridden, so that the resulting resized image can be saved or compressed to a WebP image at the same time that it is being rasterized or resized.

Find below a screenshot of how the resulting image asset appears in the filesystem once it has been resized on disk.

A screenshot of how the newly resized or rasterized images appear on disk once they are converted to the .webp image file format.

If you are using the SilverStripe framework for your website, then I strongly recommend that you use a plugin or add-on such as the one that was originally developed by a community member by "nomidi". It's a SilverStripe add-on that seamlessly integrates the ability to automatically generate .webp images if the WebP PHP extension is loaded and available for usage on the server.

If you are using PHP (or SilverStripe), you can make use of this simple PHP code snippet for testing whether the WebP GD image library extension is loaded and available to use.

<?php

if (function_exists('imagewebp'))
{
    echo 'webp support available';
}
else
{
    echo 'function not found';
}

echo phpinfo();

As it may be possible to infer from the code, it simply checks whether or not the function "imagewebp" is loaded and available globally.

I made a fork of this add-on that only does one change, and that's to save newly created *.webp images as *.jpeg.webp instead of *_webp.jpeg. The reasoning behind this modification is simply so that it's easier to produce an NGINX configuration that can construct a path to the .webp version of the image instead of the original. This constructed file path is then of course used for loading and serving the .webp image asset to the user (if it exists on disk).

You can find my fork for this add-on here.

I've since added support for

GIF Images
Silverstripe 4.10

WebP Image Format Compatibility with PHP

In the event that imagewebp functions are not available for usage with your PHP Docker container, can make use of the following snippet as part of your Dockerfile for building PHP FPM containers. These snippets assume that you are making use of the Alpine Linux variant of the PHP FPM container.

The first snippet ensures the system-wide dependencies for creating .webp images are available, and that the supporting PHP extension can create .webp images. The second snippet makes the function listed above available for usage within PHP, so that it can actually access the system-wide tooling for creating and converting assets to .webp.

Install packages and dependencies of WebP. This is _ only _ applicable if you are using both Docker, and the Alpine Linux image variant of PHP FPM.

RUN apk add --no-cache libwebp-dev libwebp

Next, the GD library has to be configured so that it points to the relevant paths for the PNG, JPEG, FreeType, and WebP system libraries.

RUN docker-php-ext-configure gd \
    --with-freetype-dir=/usr/include/ \
    --with-jpeg-dir=/usr/include/ \
    --with-png-dir=/usr/include/ --with-webp-dir=/usr/include/

Lastly, you will of course want to run your container with the aforementioned modifications, and test that the required modules are loaded and are accessible.

Improving "First Contentful Paint" Loading Speed

In summary, "First Contentful Paint" is the amount of time it takes for any DOM element to be rendered to the screen. This doesn't necessarily mean the entire page must be loaded, rather it simply means that a single element has begun rendering, and that the browser has acquired the all required assets to render the page with. In order for the browser to begin rendering, it must first download all required assets. These are typically defined in the <head> element of the page as <script> and <link> statements. Unless defined otherwise, these statements or HTML elements are considered to be "blocking", therefore meaning that the rest of the page cannot be loaded, or rendered to the screen until the assets defined in those tags have been downloaded by the browser. As you might imagine, this can potentially significantly impact the "First Contentful Paint" (FCP for short) speed.

Structured Usage of Async and Defer Statements

In some instances it could be the case that not all script or CSS assets need to be loaded in order for the page to be rendered to the user. Instead, they could be deferred or done asynchronously while the remainder of the website or page is being rendered or displayed to the user.

Lazy Load Blocking Resources

Static assets such as images, cascading style sheets and JavaScript files are considered to be blocking resources when they are loaded by default. Where possible, it's important to ensure that these resources are loaded after text content and other DOM elements have. There are a number of methods that you can adopt for reducing the time it takes for the "First Contentful" Paint (FCP) to complete, depending on the type of static assets you are optimizing. The idea of "lazy loading" is simple. Only load static, blocking assets when they are required such as when the window has focus, or if the DOM element is made visible to the user.

Images

Most images on a page can be "lazy loaded", particularly ones that are being rendered as part of a home page carousel (i.e. revolving gallery of images). For instance, not every image in a carousel is displayed to the user at once (this is by design), so therefore they don't need to be loaded immediately. However, by default, a web-browser will automatically load all images regardless of whether they are considered visible (i.e. display: none; or display: block;)

Fortunately there is a convenient way of achieving this kind of asynchronous loading behaviour for images, by using a third-party JavaScript library named "lazysizes".

It can be loaded as part of your website by simply declaring it as part of your header as such.

<head>
  <script src="lazysizes.min.js" async="true"></script>
</head>

Integrating it with existing templates and images is as simple as doing the following.

<!-- responsive example with automatic sizes calculation: -->
<img
    data-sizes="auto"
    data-src="image2.jpg"
    data-srcset="image1.jpg 300w,
    image2.jpg 600w,
    image3.jpg 900w" class="lazyload" />

The original image asset might look like this.

<!-- responsive example with automatic sizes calculation: -->
<img src="image2.jpg" class="lazyload" />

Fonts, Stylesheets, and JavaScript

No additional third-party library is required for loading static assets that are fonts, CSS stylesheets or JS assets. Instead, depending on the kind of behaviour you are seeking, you can use one of the following specifiers as part of <link/> or <script/> statements when loading static assets.

Google's Third-party Libraries

Lazily loading third-party libraries such as the ones that are used by Google Analytics engine, and Google AdSense are a little bit more tricky. It should be noted that in doing this, it can incur edge-case behaviour should you continue to use and rely on the data that is recorded in Google Analytics.

Google AdSense

Observe the following code snippet.

<script type="text/javascript">
    window.addEventListener('load', function() {
        var is_adsense_load = 0

        window.addEventListener('scroll', function() {
        if (is_adsense_load == 0) {
            is_adsense_load = 1;

            var ele = document.createElement('script');
            ele.async = true;
            ele.src = 'https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js'
            var sc = document.getElementsByTagName('script')[0]
            sc.parentNode.insertBefore(ele, sc);

            //google_ad_client: "ca-pub-xxxxxxxxx",
            (adsbygoogle = window.adsbygoogle || []).push({
            google_ad_client: "$AdSenseID",
            enable_page_level_ads: true
            });
        }
        })
    })
</script>

Typically such a code snippet would be placed in the

section of your page. If you look closely enough, you'll note that this script is in fact just a revised version of the one that Google asks you to integrate as part of your pages, but it ensures that the script is only loaded after the JavaScript event call back is fired when the window has loaded.

It constructs the <script> element manually, and appends it to the page's Document Object Model (DOM). This code will only get invoked once the page has loaded (and therefore invokes the callback).

This works, but the caveat with this is that it may affect the accuracy of your statistics, as it will likely discount users who are only visiting your page for less than a couple of seconds. However, you may also not consider this to be of importance, so this would be a significant improvement if this is the case.

Tools for Testing Performance

Once you have made the necessary modifications for optimizing your website for improved load times, you can make use of some of the following websites for testing page load times, and for identifying any other errors.

Miscellaneous Links

You may also find these other links useful.

Securing HTTP Headers with NGINX in Docker

Luc Shelton — Sat, 05 Jun 2021 17:46:57 +0000

Nginx, stylized as NGINX, nginx or NginX, is a web server that can also be used as a reverse proxy, load balancer, mail proxy and HTTP cache. More typically it is the web server of choice for most applications and APIs targeting the web. Like other popular web servers, NGINX tends to emit more data than necessary when serving HTTP requests when using the default configuration, which in turn can make it susceptible to (zero day) exploits.

An example of what is meant can be found in the image below.

HTTP headers from our page response using Google Chrome's developer tools.

This is the emitted response from navigating to any page on this website when using the default configuration. This is not ideal, as hackers or penetration testers can use automated tools for scanning and detecting websites that have potential vulnerabilities.

This article will cover some steps that you can take for securing the HTTP responses that your NGINX server emits when using the Alpine Linux version of the Docker image.

This article assumes that you have appropriately configured your development environment for usage with Docker's BuildKit features.

Extending NGINX

In the root of your project, create the following directory layout.

containers/
- nginx/
-- build/
--- Dockerfile
-- configuration/
--- templates/
docker-compose.yaml
.env

Your docker-compose.yaml should look something like this...

version: '3.9'

services:
  nginx:
    container_name: '${SERVICE_NAME}-nginx'
    image: '${REMOTE_REGISTRY_HOST}${SERVICE_NAME}/nginx:${BUILD_VERSION}'
    build: 
      context: '${BUILD_ROOT}' 
      dockerfile: '${CONTAINERS_ROOT}/nginx/build/Dockerfile'
      target: portfolio-nginx-build
      args:
        NGINX_VERSION: ${NGINX_VERSION}
        NGINX_HEADERS_MORE_VERSION: ${NGINX_HEADERS_MORE_VERSION}
    environment:
      NGINX_ENVSUBST_TEMPLATE_DIR: /etc/nginx/templates
      NGINX_ENVSUBST_OUTPUT_DIR: /etc/nginx/conf.d
      NGINX_ENVSUBST_TEMPLATE_SUFFIX: .template

Your .env file should look something like this...

SERVICE_NAME=your-service-name-goes-here
COMPOSE_PROJECT_NAME=${SERVICE_NAME}
BUILD_ROOT=${PWD}
PROJECT_ROOT=${PWD}
CONTAINERS_ROOT=${PROJECT_ROOT}/containers

So far, so good, and nothing out of the ordinary either.

Dockerfile

Keep in mind that this article assumes that you are intending on using the Alpine Linux flavour or variant of NGINX's Docker image. It's significantly smaller in size, as the operating system variant comes typically bundled with far less system utilities and running services. The Linux variant is typically used in embedded systems for the same reasoning.

The beginning of our Dockerfile is nothing out of the ordinary. It provides 3 arguments that are used for describing this particular flavour of the image, which are

CUSTOM_BUILD_VERSION
CUSTOM_BUILD_DATE
CUSTOM_BUILD_UID

These are ideal if you are generating the Docker images as part of your continuous integration or build pipeline. At the time of writing this article, we are using version 1.20.1 of NGINX, which is compatible with version 0.33 of the Headers More plugin.

In addition, our Dockerfile "parameterises" the version numbers used for retrieving the correct version of NGINX and Headers More.

ARG CUSTOM_BUILD_VERSION
ARG CUSTOM_BUILD_DATE
ARG CUSTOM_BUILD_UID

ARG NGINX_VERSION 1.20.1

FROM nginx:${NGINX_VERSION}-alpine AS builder

ARG CUSTOM_BUILD_VERSION
ARG CUSTOM_BUILD_DATE
ARG CUSTOM_BUILD_UID

ARG NGINX_VERSION 1.20.1
ARG NGINX_HEADERS_MORE_VERSION 0.33

Next, we are going to have to download the source files for both NGINX and the Headers More plugin.

RUN wget "http://nginx.org/download/nginx-${NGINX_VERSION}.tar.gz" -O nginx.tar.gz && \
    wget "https://github.com/openresty/headers-more-nginx-module/archive/v${NGINX_HEADERS_MORE_VERSION}.tar.gz" -O headers-more.tar.gz

This is necessary as we have to produce a local build of NGINX that integrates Headers More as a dynamic module. Therefore this means that we need to ensure that Alpine Linux has the GCC compiler toolchain available, along with make tool and some additional packages.

RUN apk add --no-cache --virtual .build-deps \
  git \
  gcc \
  libc-dev \
  make \
  openssl-dev \
  pcre-dev \
  zlib-dev \
  linux-headers \
  curl \
  gnupg \
  libxslt-dev \
  gd-dev \
  geoip-dev

This bit consists of preparing the compilation environment for NGINX, including adjusting environment variables used by the compilation toolchain.

RUN mkdir -p /usr/src

# Reuse same cli arguments as the nginx:alpine image used to build
RUN CONFARGS=$(nginx -V 2>&1 | sed -n -e 's/^.*arguments: //p') \
    tar -zxC /usr/src -f "nginx.tar.gz"

RUN tar -zxvC /usr/src -f "headers-more.tar.gz" 

RUN HEADERSMOREDIR="/usr/src/headers-more-nginx-module-0.33" && \
  cd /usr/src/nginx-$NGINX_VERSION && \
  ./configure --without-http_autoindex_module --with-compat $CONFARGS --add-dynamic-module=$HEADERSMOREDIR && \
  make && make install

This does the following:

Uses the tool sed to search and replace part of the string output from invoking nginx -V in the shell.
1. The output from nginx -V displays the switch parameters used for compiling NGINX and configuring make.
2. It makes use of the output generated, while appending our additional module (which is Headers More).
Extracts the contents of the Headers More plugin that we downloaded in our previous snippet.
Defines a variable pointing to the path where Headers More has been extracted to
Runs make.

In this next stage, we finally make use of the generated build output after downloading and building NGINX and Headers More together. We make use of a feature in Docker called "multi-stage" builds, which enables us to copy the contents from another stage of a build. This approach to generating Docker images is considered to be significantly more efficient, as it enables us to reuse cached stages when rebuilding the Docker image, or making iterative changes.

FROM nginx:${NGINX_VERSION}-alpine as your-service-name-goes-here-nginx

# Extract the dynamic module "headers more" from the builder image
COPY --from=builder /usr/local/nginx/modules/ngx_http_headers_more_filter_module.so /usr/local/nginx/modules/ngx_http_headers_more_filter_module.so

The rest of the Dockerfile definition is entirely up to you. Depending on the flavour of the build (i.e. development or production), you can choose to copy or include other files that you would find useful in that particular flavour of the build.

You can find the complete Dockerfile constructed in this article here.

Configuration

Now that you have a working Dockerfile definition for your NGINX image, we can now configure how the NGINX server will behave. The intent behind this configuration is to ensure that we are not sending more data that necessary in our HTTP responses. Using the Headers More extension that we downloaded, compiled, included as part of our NGINX Dockerfile, we can now prepare a NGINX server block configuration that can remove headers using the appropriate syntax.

The Headers More extension makes the more_clear_headers command available for us to use, meaning that we can now remove headers that are added by default by the NGINX web server.

more_clear_headers 'Server';
more_clear_headers 'X-Powered-By';

A more complete example would look like this. Find below an example root nginx.conf configuration file.

# Load in the headers more module
load_module /usr/local/nginx/modules/ngx_http_headers_more_filter_module.so;

user nginx;
worker_processes auto;

error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;

events {
    worker_connections 1024;
}

http {

    more_clear_headers 'Server';
    more_clear_headers 'X-Powered-By';

    server_tokens off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log /var/log/nginx/access.log main;

    sendfile on;
    #tcp_nopush on;

    keepalive_timeout 65;

    include /etc/nginx/conf.d/*.conf;
}

This configuration ensures that all HTTP responses served, regardless of which server blocks you have configured, will have the X-Powered-By and Server headers stripped from the HTTP response. Handy!

Now you can rest assured that visitors will be none-the-wiser about which kind of software you are using for serving your web applications, and thus mitigating the ability for a malicious visitor to find or target exploits in your software stack.

Games Development at Scale: Personal Builds

Luc Shelton — Fri, 04 Dec 2020 14:14:19 +0000

In this article I am going to breakdown some of my previous professional experiences in developing and using personal build systems, that enforced robust sanity checks on code submitted to mainline repositories of large multi-team projects

During my time at Splash Damage, I was involved in developing the "personal build pipeline" that has since been highlighted in a talk presented by Valentin Galea, former Technical Director at Splash Damage. You can check out the slides from that presentation here.

For most of my professional career, I've made use of a continuous integration system called TeamCity. It's a closed-source, flexible, user-friendly build and automation system developed by JetBrains. It comes shipped with a wide range of server-sided plugins, including a vendor-supported plugin for running personal builds.

Links to documentation about Personal Builds with TeamCity, and it's servers-sided Remote Run plugin can be found here:

We made use of these set of vendor supported tools for creating a personal build pipeline, whereby changes made locally on the user's machine, were tested on a remote build machine for verification purposes.

The "personal build pipeline" that I developed was contingent on there being a few tools available.

Perforce Changelist Tool
- A user-facing tool that would construct a valid changelist description that would include all relevant information about a change, including a link or build ID (i.e. a build that is succeeding with the changes found in the changelist) to a valid personal build, a valid link or ID (i.e. approved or greenlit) to a Helix Swarm code review.
Perforce Trigger Handler
- A Perforce daemon trigger (p4d trigger) that was responsible for verifying that the changelist could be submitted by checking the contents of the changelist and its description.
Personal Build Tool
- A tool that would aggregate local changes, and submit those changes directly to the build server for triggering a personal build.
- Otherwise known as the Content Validation Tool, as it was previously only ever meant to be used for testing changes made to packages and content assets.

You can read more about the personal build pipeline from the UnrealFest presentation here.

Goal

Enable developers to conveniently run "personal builds" containing changes that are stored locally on the user's machine.
Prevent developers from breaking the build by ensuring code compiles in a "live" build system environment.
Ensure that change list descriptions or commit messages contain relevant information that conform to a enforced standard.

Personal Build Tool

The Personal Build Tool is a desktop application responsible for aggregating local changes in a Perforce workspace, submitting those local changes directly to the TeamCity server, and subsequently triggering a new build on TeamCity with the submitted changes. The local changes would be applied on top of the head revision of the repository checkout from mainline.

API

TeamCity exposes some of its functionality through a HTTP based "RESTful" API. This REST API is used in conjunction with the Remote Run plugin to perform a few tasks including:

Checking the current status of the build, including whether or not it is queued, if it's running, and also retrieving the most recent build log data.
Starting or queueing a new build with the changes generated with the patch file generated by the Personal Build Tool.
Cancelling the build if there are any changes.
Updating the build description or information with a checksum generated from the local modifications on disk.
- This is used as part of the verification step with the Perforce Trigger Handler.

Information on the TeamCity REST API can be found here.

Decompiling Tools

Like most JetBrains products, the command-line runner was a server-sided plugin that was written in Java. This server-sided plugin was responsible for handling requests for personal builds from another command-line based tool that was also written in Java.

This command-line tool enables a user to specify one or more local files, a build type configuration ID, along with additional parameters (optional), and then submit those changes directly to the TeamCity server for starting a new personal build.

In order to achieve this, the command-line tool constructs a "patch file" which comprises of the user's local changes, along with additional metadata that enables the build system to understand where the files are to be placed in relation to the Version Control System (VCS). In this instance, the VCS was Perforce

Constructing the Patch File

The patch file is a simple binary file that consists of metadata, and the raw contents of local changes from the user's machine. The local changes included in this patch file is what the build system will use, on top of the existing head revision from the mainline of the code repository in the VCS (in this instance, Perforce). The goal of this entire system is to prevent the user from submitting local changes into the repository without first verifying whether or not those changes would compile successfully in a "live" build system environment (e.g. a machine not belonging to the user). This is the entire premise of the personal build system.

The structure of format specification for this patch file are unfortunately not documented anywhere, and it will require you to decompile the Java-based command-line remote run tool in order to understand how the patch file is constructed. The code itself is relatively simple enough to understand even without prior knowledge of Java.

The patch file consisted of information including the following:

Version control system information, and which branch or stream this was meant to use as the head revision.
Information about each locally changed file, including their names, total file size, and where they are meant to be placed on a build agent when building in a live environment.

With this knowledge, we were then able to develop a basic serialization mechanism as part of our Personal Build Tool, that was able to aggregate all local changes on the system, and produce a "patch" file that can be used for starting a personal build with.

This patch file would then be submitted directly to the build system, so that the developer can avoid submitting changes to the repository first. The changes found in the patch file would then be combined with the head revision of changes in the repository.

Perforce Changelist Tool

The Perforce Changelist Tool is a simple desktop application that produces a changelist description that conforms to a format or "specification" that the Perforce Trigger Handler expects. The goal of having a specific format or specification for changelist descriptions is to ensure that they are both informative, and consistent. The information found in a changelist description assists in tracking the completion of work.

The Perforce Changelist Tool, as documented in the UnrealFest presentation by Valentin Galea from Splash Damage.

Summary

The Perforce Changelist Tool is a simple desktop application that is launched from P4V, the GUI application "frontend" for Perforce.

Perforce Integration

When the tool is launched for the first-time from a context menu, the tool will automatically integrate itself with P4V by locating the configuration files that are available under the following path.

%USERPROFILE%\.p4qt\customtools.xml

This is an XML based file that gets loaded by P4V upon when the application is started. It provides definitions for custom tools that the client must use, including additional context menu actions that enable you to launch an external application with injected parameters (including the number of a changelist).

The changelist number then gets passed to the Perforce Changelist Tool through a command-line argument (i.e. --changelist 1234), which it then uses to load additional information about a changelist (e.g. files checked out, author, time created).

Generated Output

The purpose of the tool is to populate the description of a changelist with all required information in a format that is recognisable by the Perforce Trigger Handler, which is described later on in this article. The _Perforce Changelist Tool _makes use of a predetermined "specification", that was designed for ensuring that all subsequent changelists submitted into the repository had clear and concise information associated with them.

This "specification" ensured that the following fields were populated.

Helix Swarm Review URL and ID
- This only was required if the changes contained code-based changes.
Jira Issue ID
- If the change being submitted was associated with a Jira user story, then this was required.
Tags
- These were optional identifiers that simply made it easier to observe change lists at a glance when scrolling through changelist history in P4V.

Resources

Read more documentation here about integrating "custom tools" with P4V.

Implementation

The tool made use of C#, WPF, and various other third-party libraries for UI controls and styling. In addition, the tool made use of toast notifications for notifying the user when an operation completed, and presented progress information in the task bar.

Perforce Trigger Handler

The Perforce Trigger Handler is a lightweight console-based application that is responsible for validating submitted changelists, by performing a series of checks. If the changelist is considered invalid, the application will terminate with an exit code of 1 (or anything not 0), otherwise if deemed acceptable the application will terminate gracefully with an exit code of 0. The behavior of this application is largely dictated by the specification for Perforce triggers that is documented here.

As stated, the console application performs a series of checks including:

Validation Checks
- Does it contain all required information?
- Does it conform to the specified format for changelist descriptions?
Time of Day
- Is this changelist being submitted outside of work hours?
Build Health
- Is the latest build on TeamCity currently failing?
- If so, is the author of the changelist the one who broke the build?

Validation Checks

Each changelist description needed to conform to an enforced standard of what a changelist description should look like. The specification stipulates how a changelist should appear, and what information it should contain. This information is generally populated by the aforementioned Perforce Changelist Tool, that ingests all necessary data from external sources and then produces a conformant changelist description.

Build Health

Before another user can submit code into the repository, it's important that existing build breakages are resolved otherwise subsequent commits to the code repository could worsen the state of the build. If the user submitting the changelist is not the one considered to be responsible for breaking the build, then the user will be prevented from submitting their changelist until the build goes green.

This ensures that existing build breakages are urgently resolved, otherwise there will be a blockage in the pipeline.

Time of Day

This a simple check that ensures that there is no one outside of working hours that is submitting changes into the repository, and that there's no possibility that the build will fail before people arrive at the office the following day.

The message presented to users through P4V if their changelist was considered invalid. The output seen in the image is produced by the Perforce Trigger Handler, a server-sided console application responsible for performing validation checks against the changelist beign submitted.

Implementation

The decision process for submitting the changelist looked a little like this.

The Perforce Trigger Handler had to perform a series of checks before allow for a changelist to be submitted to the Perforce server. The Perforce Trigger Handler could ultimately prevent a changelist from being accepted.

The above diagram encapsulates what happens when a changelist is submitted to the Perforce server.

Conclusion

Personal build pipelines are an extremely powerful tool, but they likely will not suit everyone's needs.

Problems

Personal builds are a great means for ensuring that all changes submitted into the mainline stream of the code repository are vigorously tested, and are less likely to break the build once changes are submitted into the repository.

Of course, the biggest issue with using such a pipeline is that it significantly reduces the speed of iteration across the team. Now, the team must successfully complete multiple steps in order to be able to submit a change into the repository.

There are some instances where this is a hindrance more than a benefit.

Exceptions

With the usage of a specifically configured "magic" keyword, it's possible to enable for the Perforce Trigger Handler to ignore performing all checks before allow a changelist to be submitted to the repository.

There is of course the possibility that such a "magic" keyword could be abused by members of the team, which is why it was imperative for the Perforce Trigger Handler to send out email notifications to team leads each time this particular keyword was used. This was simply to ensure accountability, and so that changes that were submitted using this keyword that were considered problematic can be accounted for (and reverted if necessary).

Splash Damage makes use of a special "[BuildFix]" keyword found in the changelist description to get around the Perforce Trigger Handler's preventative measures.

Build System Resources

As stated in the presentation by Splash Damage, allowing for personal builds at any time of the day, and launched on request by any development team member, means that there is an significantly increased demand on the "build farm" (i.e. systems dedicated for doing nothing other than build the game). Therefore, it's important that there are sufficient build machines made available if a personal build system is adopted, and that those build machines are placed into separate agent pools from the ones dedicated to "mission critical" build type configurations (i.e. ones that must run on a regular basis, or at a particular time of day).

This then ensures that "mission critical" build type configurations will continue to run uninterrupted (and not placed in queues) at any time of day, or whichever time of day that they are configured to run.

Comments

Having a personal build system available to a large-scale development team is an invaluable asset when it is integrated correctly. The fundamental goal is to ensure that a development team can continue to work at a fast and regular cadence without hindrance, and without compromising on the quality of the "product" they are delivering.

Whether or not you wish to integrate a personal build system with your team depends on a few things.

How often are users committing code?
How big is the team?
Does your team have sufficient build system resources available (i.e. build agents)?
Do you have sufficient development resources for building the aforementioned components?

User Experience

It's important to note that if you do not provide adequate user-friendly tooling around this system, then you will very quickly frustrate and demoralize members of the team. This is understandable, as you are expecting them to adopt an otherwise contrived workflow for results that may not be immediately clear. If you expect users to perform additional steps before submitting their work, then be sure to do as much as you can for them, and with minimal interference.

Tooling

Here's what each of the tools in the pipeline did for the user to mitigate confusion, frustration, or the amount of time required to read documentation.

Perforce Changelist Tool

P4V Integration

On first-time launch, the tool made itself launchable through P4V by locating P4V's configuration file on disk, and injecting its own configuration into the file, so the tool was available as a context menu option upon right-clicking any changelist in P4V.
Automatically pulled information about the user and their superior by making use of LDAP or Active Directory, and made "smart" suggestions about who the reviewers were.
Automatically pulled information about code reviews associated with the changelist (from Helix Swarm).
Automatically detected which Perforce client the user was using by iterating over all locally available Perforce clients and determining which one the tool resided in (the tool was distributed through Perforce).

Typically custom tools configuration are typically stored on the user's machine at the following path.

%USERPROFILE%\.p4qt\customtools.xml

It's a simple XML file that any external tool can simply inject values into, providing that the changes made compliant with the XSD (XML Schema Definition).

Perforce Build Tool

P4V Integration

Similar to how Perforce Changelist Tool behaved, the tool integrated itself as part of P4V custom tools configuration file that is located at the path mentioned earlier on in this article.

Personal Builds Support

The tool made use of the official JetBrains server-sided plugin for performing personal builds, by constructing "patch files" that the TeamCity server instance would recognise. In order to achieve this, we decompiled the existing CLI tooling, and understood how it was constructing the patch files.
The tool automatically started new builds on the TeamCity server instance with local changes, and provided the user with regular feedback in the form of taskbar progress information or toast notifications through Windows 10.

Perforce Trigger Handler

Remotely Manageable and Configurable

The Perforce Trigger Handler is a command-line application that sits on the same server as the Perforce daemon. More often than not, developers on the team won't have (nor will they require) direct access to the system that powers the Perforce daemon. Therefore, being able to remotely control or alter the behaviour of the Perforce Trigger Handler on the server is important. In our case, the approach for enabling this was simple. We had the application run from a network share that was accessible by both developers, and the server that the Perforce daemon operated from.

Perforce: Formatting, Scripting, and the Command-line Interface

Luc Shelton — Wed, 10 Jul 2019 12:50:52 +0000

In this blog post I will discuss some of the convenient tricks that you can leverage when navigating Perforce and its command-line interface. A lot of these are well documented in a post that's available on Perforce's website, but this blog post will look at a few of those features with more detail.

Formatting Output

The Perforce command-line interface features support for a variety of useful global command-line arguments that can be used for altering the output and and operation of the command-line tool.

The output from the Perforce command-line interface can be formatted using a combination of the -ztag, -e, and -F command-line parameters.

Using -ztag parameter

The -ztag global command-line parameter displays the standard output from the Perforce command-line interface in a format that conforms to a specification.

The specification is denoted in that each "key" or "attribute" of an object, is prefixed with "... " and then proceeded by the name of the key. The data for that particular attribute or property is then displayed afterwards using a white-space character as the delimiter. Find below an example.

Usage

p4 -ztag info

Which produces an an output similar to the screenshot below.

The resulting output from executing "p4 -ztag info"

More information on the p4 info command can be found here.``

Using -e and -F parameters

These global command-line arguments are particularly useful for filtering and identifying "keys", "properties", or "attributes" from command outputs. It's possible to see which "keys" are available to filter or "format" our output with by using the -e parameter. The resulting output should look something like this.

Usage

` p4 -e info `

The -e in this command will now reveal all "formattable" data that is produced by the command info. The parameter "info" can be replaced with another command including "clients", which can display all clients that are owned by the currently logged in user.

` p4 -e clients `

The above command will display available parameters for each entry that is displayed the "clients" command. Look below at an example of a resulting entry.

` info: **REDACTED** code0 352327909 (sub 229 sys 6 gen 0 args 5 sev 1 uniq 6373) ... code0 352327909 ... fmt0 %domainType% %domainName% %updateDate% %'root'% %domainMount% '%description%' ... domainType Client ... domainName **REDACTED**... updateDate 2021/03/18 ... domainMount **REDACTED**... description Created by **REDACTED** `

Under the entry fmt0, we can see that there is a space delimited list of available formatting parameters that we can use. Let's try using the clients command again using one of these parameters.

` p4 -F %domainName% clients `

The above command should now output each entry that is emitted by the clients command, but only the name. This is because we have selectively chosen which entry to output using %domainName% as the selector.

The same command can be changed to another available formatting parameter such as %updateDate%, which should display the last time the respective client has been updated.

` p4 -F %updateDate% clients `

Scripting

Understanding how these global command-line parameters can be used is incredibly powerful. With this, we can now write applications that are capable of executing these commands and parsing the stdout stream from the commands we execute with minimal time spent on parsing.

DEV Community: Luc Shelton

Homebrew and Anaconda: Configuring for Apple M1 and Intel Silicon

Installation Locations

M1 Silicon Installation Path

Intel Silicon Installation Path

Configuring Bash

Homebrew Shell Environment

Anaconda Shell Environment

Configuring iTerm to Launch with Rosetta Emulation

Perforce: Installing and Configuring on Ubuntu

Configuration

Add Trusted Package Repository Keys

Add Repository Source

Installation

SilverStripe: Generating URL Segments for DataObjects

Breakdown

Usage

CSS: Loading Custom Fonts

Minification with Font Awesome

Preloading JavaScript Libraries

Use WOFF2

Blocking Resources

Prioritise Loading Custom Fonts (Asynchronously)

Prevent Invisible Text

Automatically Serve WebP Images with Silverstripe and NGINX

Overview

WebP

Integrating with Silverstripe

Serving Generated WebP Images from NGINX

Conclusion

Useful Links

SilverStripe: Integrating Memcached Caching with Docker

Configuration

What does it do?

Docker

SilverStripe: JSON-LD Structured Data

JSON-LD

Extending SilverStripe for JSON-LD Support

NGINX: Default Server Configurations

NGINX Configuration

Handling Specific HTTP Errors

Handling SSL Traffic

Installing Dependencies for SSL in Docker (Optional)

Testing Configuration Changes

Conclusion

Complete NGINX Configuration

Useful Reading

Tips for Optimizing Page Speeds

Server

NGINX

Use GZip Compression Rules

Use Reverse Proxy Caching

Apply Cache Control Headers

Automatically Serving WebP Optimized Image Assets

Website Performance

Minify Text-based Static Assets

Tooling - NodeJS and JavaScript

Tooling - PHP

Optimising Image Assets with WebP

SilverStripe and Automatic WebP Image Conversion

WebP Image Format Compatibility with PHP

Improving "First Contentful Paint" Loading Speed

Structured Usage of Async and Defer Statements

Lazy Load Blocking Resources

Images

Fonts, Stylesheets, and JavaScript

Google's Third-party Libraries

Google AdSense

Tools for Testing Performance

Miscellaneous Links

Securing HTTP Headers with NGINX in Docker

Extending NGINX

Dockerfile

Configuration

Further Reading

Games Development at Scale: Personal Builds

Goal

Personal Build Tool

API

Decompiling Tools