DEV Community: Scott

Performing an in-place Git Rebase

Scott — Wed, 16 Feb 2022 03:44:21 +0000

It's not common but sometimes we have uses for rewriting our git history. This post may (or may not) have been inspired by starting at a new job and not wanting my peers to know I completely misread one of my first programming tickets.

Many organisations approach their git commit standards differently. Some have strict adherence to commit message formats, while others are more lax. For traceability of past work it is often common to use a ticket (or some other reference) number in commit messages. That's fine, but what happens if you are new at a company, taking on a little too much information at once, and typo one of your first ticket numbers, only to realise as you go to submit a Pull Request?

You try and fix your mistake of course! Learning is always fun and git is surprisingly hard to break so let's see how we can fix the problem.

The Setup

Let's assume we have a repository that uses trunk based development and some sort of feature branching strategy. We might have something like below.

Working on a Feature Branch

We start working away while others continue to merge work into our trunk. For brevity I've commited directly to the main branch but I suggest in real world scenarios you have branch policies that prevent this behaviour. As a result we might be in a state similar to below.

In this example we have "Ticket Number 123" appended to the start of the commits in the form [T123].

The Problem

Whoops! We were actually working on "Ticket Number 234", not only do I look silly but we might also break integrations with other work tracking tools, we might want to try and fix this ASAP.

To fix this particular mistake we are going to take advantage of the rebase command. Normally I would suggest that best practice would be to just rebase the branch off your current parent as preserving the commit it branched from isn't super useful. However there might be cases (other than pride) where doing an in-place rebase is useful. So how can we rebase this branch in-place and also re-write all of our commit history?

Rebasing

We first want to find the parent commit for this branch, given we are attempting to do it in-place. There are a few ways to do this and I will show them in my preferred order.

The Merge-Base Command

The merge-base command attempts to find the base commit of the supplied branches. For a feature branching strategy where that should be one commit (and not some hypothetical commit based off a multi-branch merge) this yields the fastest results. In context of our example this is what it looks like.

Using a Visual Git Tool

Using a visual git tool provides you with a good overview of how your branches look in relation to the rest of the repository and can be easily scanned to find the parent commit. Naturally the longer your branch the harder this might be.

Using Git Log

Much like a visual git tool git log can be a little cumbersome if your feature branch or parent branch are long, however the following command gives us a very similar analogue to our visual tool.

Now that we have our commit hash we can start an interactive rebase that not only replays our commits it lets us choose how we want to handle each commit.

Start the rebase command.

We are now prompted with the history of the commits on our branch as well as some instructions on what we can do in interactive mode to change our previous commits.

You will notice that one of the options below is reword or r for short. We still want to use this commit but we definitely want to change that commit message. Below is what it looks like once we have changed this file. You will not change the commit message here just the command, changing the message comes later.

You can see that we have changed the word pick to reword for each commit as they all contain the wrong ticket number. Depending on your text editor the process for this might be different. I am using nano so I can save and close the file.

Once you close the initial rebase file you will be prompted with a new file for each commit on your branch. The longer the branch the longer this will take, sorry.

Very similarly to our initial rebase file we can now edit the commit message here, replace the incorrect ticket number.

We have changed [T123] to read [T234]. Save and close the file and repeat the process for every commit.

When done check your git log again and you should see updated commits with the correct messages, also referencing the original commit.

Assuming this branch is already in an upstream, which for me it was, you will need to force push over this branch to completely rewrite the history using git push -f.

Congratulations! You have now successfully updated all the commit messages on your feature branch.

Branches with Ticket Numbers

In my example my branch name didn't include a ticket number, sometimes they do. In this case you might want to also change the branch name, again if it makes sense. We can perform the following steps:

git checkout <your_branch>
git branch -m <new_branch_name>
git push origin -u <new_branch_name>
git push origin --delete <old_branch_name>
Profit.

A Note on Rewriting History

In general it is not great practice to rewrite history, the git log exists for a reason and rebasing in particular (which rewrites commits) might cause issues if other developers are relying on those commits. If you're working on a public repository, or are generally unsure about how to handle a situation like this, it is better to reach out to your repo owner for advice.

In my case it was a feature branch only I was going to touch and having correct commit messages on the branch was more useful for the team than trying to come up with a different solution that maintained the git history with the wrong commit messages preserved.

The same goes for branches too, I was comfortable to rename and delete the upstream branch for this particular piece of work but other repo owners might want to keep the wrongly named branch stale, or you might not have access to delete branches, so double check before making changes you might not be able to reverse.

Do you have any git success or horror stories? I would love to hear about them on Twitter.

From The Archives: De/Serializing Top Level Arrays in JSON with Unity

Scott — Thu, 08 Apr 2021 07:10:21 +0000

From the archives celebrates the good and bad of my journey in software development. It's a chance to dredge up hundreds of hours of older work. Some I am proud of, some not. But it's with self reflection that we get to understand how far we have come.

The Problem (& Solution) Circa. 2016

Well it has been a while since my last tutorial. I’ve been experimenting with video tutorials and working on my game and also doing some paid project work at the same time. At least I am not bored!

That being said I like to note something down when I find something new or tricky that I get stuck on and today’s is simple but effective. For those that don’t know Unity actually has a nifty built in tool to serialize and de-serialize objects through the JsonUtility class. This is super useful for creating more human-readable data quickly and then adding it into your game. There is only one tiny problem for me, it doesn’t handle top level arrays. By that I mean the following won’t work.

[{"myvar";"myvalue"},{"myvar":"myothervalue"}]

I’m not sure the reasoning but my searches online did reveal from Unity staff that they currently do not support this feature. The easiest way to fix this is just wrap the array in a top level object like below.

{[{"myvar";"myvalue"},{"myvar":"myothervalue"}]}

It is a simple fix but it is annoying if you want to serialize or deserialize straight to (or from) an array, which is what I needed to do. So let’s look at making a JsonHelper class that fixes this issue for us by wrapping up top level arrays in a Wrapper class.

using System;

public static class JsonHelper {

    public static T[] FromJson<T>(string json) {
        Wrapper<T> wrapper = UnityEngine.JsonUtility.FromJson<Wrapper<T>>(json);
        return wrapper.Items;
    }

    public static string ToJson<T>(T[] array) {
        Wrapper<T> wrapper = new Wrapper<T>();
        wrapper.Items = array;
        return UnityEngine.JsonUtility.ToJson(wrapper);
    }

    [Serializable]
    private class Wrapper<T> {
        public T[] Items;
    }
}

The above code provides a simple black box to allow you to serialize or de-serialize those top level arrays but how does it work? The Wrapper is a [Serializable] class that is essentially empty, in fact it only holds a generic array. The generic array is for all intents and purposes our top level array. The thing we actually want to work with.

When de-serializing an array from Json it simply loads the Wrapper class, ignores the wrapper itself and returns the array. Conversely when serializing into Json we create a wrapper so that we have something to hold our array in. We then assign the array passed in to our function as the wrapper’s local variable. Lastly we use the built in JsonUtility to return the string back for writing.

There is now only one slight change to our Json file, the array is actually named therefore we should tweak it to suit. In any situations where you want to use a top level array instead of using our first example instead wrap it in an object and label the array “Items” as shown below.

{"Items":[{"myvar";"myvalue"},{"myvar":"myothervalue"}]}

NOTE: This only applies if you are ever manually creating the json you want to de-serialize. If for some reason you’re only ever going to be loading objects you’ve previously saved the JsonUtility does this for you.

Easy? You bet, but that was the whole point!

This "From the Archives" was super interesting because I actually had to look this one for one of my own personal game "Project Zap" only this week. In the original article I mentioned that

I’m not sure the reasoning...(behind why it behaves this way)

I guess technically I still don't know why, obviously that is just their implementation but the Unity documentation probably listed this constraint all along. It's funny how something I tackled so long ago came to front of mind so quickly simply because I took the effort to write it down in the past.

I still don't love this solution, having to set a top level array to an arbitrary key feels fraught with danger to me. Though it has proven effective for me on the projects in which I've used it because I can easily control the data format on import. Also it made for a super simple copy & paste solution for my current project!

It's strange to me that this requirement still exists and a more feature rich solution doesn't exist out-of-the-box, though I assume Newtonsoft JSON or something else could be leveraged if you have more robust JSON parsing requirements.

Anyway, here is to remembering to write things down 👍

Locally Ignoring Git Files

Scott — Wed, 17 Mar 2021 06:47:10 +0000

Have you ever wanted to maintain some local files in your git repository? Have you had to try juggling not staging those files so as not to effect the project configuration settings? What if you could add those files, have them ignored, but not touch your .gitignore file? Let's take a quick dive into how (and why) we might do that!

Why Exclude Local Files

Let's start with the why? There are a number of reasons you might want to investigate ignoring local files in your repository that are specific to you but not the project as a whole.

My use case is using containers to run my development environments. They simplify dependency management, are easy to setup and even easier to completely remove (completely) and let me take advantage of a Linux based environment on my Windows PC.

The easiest way to do this in VS Code is to add a .devcontainer folder in the project root. You can find the documentation about development containers in the VS Code Documentation if this is something you might be interested in.

Our project however might not be set up for development containerization (for any number of reasons) and we certainly don't need to throw that requirement to the rest of the team if this workflow is something unique to me.

So now I have a .devcontainer folder (and its associated files) in VS Code but don't want to accidentally commit it, now what?

Git Handles Local Ignores

There is already a solution to handle this very problem in git which we can read more about in the Git Documentation.

All you need to do is open /path/to/project/.git/info/exclude. The file will already be there and follows the same syntax as the standard .gitignore, so editing it is a breeze.

Note: Windows users may need to show hidden files and folders if editing this via the Explorer.

This file exists for:

Patterns which are specific to a particular repository but which do not need to be shared with other related repositories (e.g., auxiliary files that live inside the repository but are specific to one user’s workflow)

For my use-case I have added .devcontainer to this file and now don't worry about having to check this one into the shared team repository. Nice!

Persisting UI Customizations in Strapi

Scott — Mon, 15 Mar 2021 23:01:57 +0000

Today we will be looking at a Headless CMS called Strapi. To keep this post on topic I am going to assume some prior knowledge on Strapi. If you're new to the technology and would like some intro guides, let me know and I will make a short series on this. Generally speaking though their documentation should be enough to get you up to speed quickly, I've leaned on them heavily to learn the platform.

What I would like to address today however is a multi-environment setup for this tool, specifically around UI customization in the admin panel and persisting these changes across each environment.

If you've ever run Strapi in a multi-environment setup you will notice that whenever you spin up a new instance of your site you lose a lot of the UI customization in your application. By this I mean the "list" and "edit" views in your admin panel for a particular content type are set back to their default structure. It would be tempting to just set these back up manually in each environment because "It only needs to be done once". While this is true (and honestly probably not a deal breaker) it does present a possibilty for configuration drift between environments and just generally adds more potential places for there to be an error or misconfiguration.

While there are no out-of-the-box solutions to persist this customization it can easily be achieved by using the bootstrap functionality baked into Strapi. Let's disect that one below.

The Initial Content Type

For the example today we are going to stick with the blog format where we create a Post content type. This type will have the basic building blocks that we need to render our posts on whatever website is consuming our Strapi API. A basic blog content type might look like below.

Post Content Type

Post List View

Post Edit View

Now we have the option to go in and manually configure some of these views, for example in the Post Edit View we can clearly see the button on the right marked "Configure the view".

Let's do that now so we can change the order of some of the items, perhaps move the featured image to the top (to mimic the actual blog post structure) and rename some labels or add descriptions.

Configured Post Edit View

As you can see we have changed Featured_image to read Featured Image and moved it's position. We have also supplied an arbitrary description to the Slug field. That's great and all but what happens when I spin up a new environment? I would have to do that all again manually. In this instance for two fields I would say that's probably OK, for a complicated data type that has additional logic or a hierarchy of information this isn't an overly acceptable solution.

Thankfully Strapi saves this configuration in the database and we can pull it out as JSON, let's do that now!

UI Customiztion via JSON

Connect to your local database instance. I am using Azure Data Studio with their PostgreSQL plugin. Feel free to use whatever tools you're comfortable with.
Select all entries from core_store table.
Look for a row labelled plugin_content_manager_configuration_content_types::application::{content_type}.{content_type}. In this instance plugin_content_manager_configuration_content_types::application::post.post
Copy the value field. It should look something like below.

The data structure should be fairly self explanatory, we can make some tweaks the the settings of this content type, whether it is filterable or searchable for example. Under metadatas we can change details about each value in the content type. For example you can see the change we have made in featured_image to change the label to Featured Image when in the edit view, though you can see that same change hasn't propogated over to the list view.

The layouts key shows the heirarchy of our edit view, I find that making these changes is easier in the UI and then exporting out the associated JSON however once you have the file the first time you can definitely move these around in code. Lastly the list key under layouts specifies which keys, and the order of the keys gets shown.

Tweaking the UI Customization

Now we have our JSON file and understand the structure we can go ahead and make some changes.

We are going to make sure that the created_at key says Created At in the list view as well as the edit view.
We are going to make the featured_image media picker take up the full width of the admin panel (a change you seem to only be able to make via JSON)
We are going to remove id and featured_image from the list view and replace it a created_at column instead.

The differences are minor but can be seen below.

In order to test these changes the simplest way to it do (for now) is to edit the database directly and replace the current string with your current data and restart the server.

The Final List View

As you can see our changes have worked, id and featured_image have been removed and created_at added in place with the correct label.

The Final Edit View

As you can see our image picker now takes up the full width of the admin panel even though we only had the option to move it in the editor.

Persisting the Changes

Now we know how to edit our UI customization file lets persist it.

Save that JSON file somewhere in your project so that it stays in source control and is easily referenced later.
Create a file under config/functions/bootstrap.js as per their docs on Functions.
In the bootstrap function call some code to update the corresponding database row with the file you have saved in your project. Here is an example utility function I have written that lets me pass in a json file location that corresponds to <contentType>.json.

As you can see I am using knex for this because I am familiar with it. That being said you could use the bookshelf or any other adaptor, or the suitable connector for your database type.

Closing Thoughts

Now whenever your application starts (or hot reload fires in your dev environment) your UI customization is updated in the database and now your UI is configured across multiple environments.

With some extra tweaks you can do the same with components that are within a more complex data type. The syntax is slightly different so I suggest a cool extension to this post, if you want to dig into it, is to work out how to persist UI Customization in components too. I do this in my current project and it's really handy.

For those wondering when spinning up an entirely fresh environment the bootstrap code gets run AFTER strapi does a bunch of set up. That means your content type, even if it is "brand new" will still be in the database and your SQL query will run so you shouldn't need to do any complicated checks for rows or do some weird double-deploy nonsense. I have seen some back and forth on the Strapi Issue Tracker as the whether bootstrap should run before or after Strapi initializes, howevever given it provides us access to the database connector (and makes sure our data types are set up correctly) I am all for after as the default, as it currently is.

Now you can worry less about your UI customization and more about the logic required to run your application. What's great is this method can be used whenever, so if you have to make changes through the UI you can easily pull out the updated files as required and persist them in your database. You could even find ways to automate that process, or pull your customization from a master database that it used for this purpose. Whatever your setup, by storing your UI customization in the database you have a wealth of opportunities to streamline your multi-environment customization.

My Experiences Jammin' The Stack

Scott — Thu, 11 Mar 2021 01:30:22 +0000

This is an open letter to the JAMStack community about my experiences with the technologies so far. What I love, and more importantly what I don't love about the experience. While I am likely to be critical here I don't know how else to share this in long form. If you've come across this on social media and have any thoughts please share them as I'd love to iterate on what I have experienced already.

Stop trying to put jam in there...

There are a number of things in this world I want to love. Either because I just think they are cool or because I think they fit the vision that I have of myself. Whenever I can't enjoy one of these things, particularly the ones that deviate from my own internal picture of myself, it makes me question if I really know what drives me. There are a number of these such things but a few that I keep trying and failing to fall in love with are:

Linux as a full time OS
Drawing
Being super into a franchise
Making physical things (with my soft hands that always get hurt because I only require enough dexterity to hit these keys in roughly the right order every day)

And lately I've been able to add the JAMStack to that list.

What is a JAMStack?

In case you have stumbled across this post and wonder why anyone would want to put jam anywhere near their tech stack let's get out of the way what this acronym is all about. JAMStack stands for a stack made up of Javascript, API's and Markup. Strictly speaking this typically means applications that have been developed by static site generators (or similar) that can be served out of a file storage location such as Amazon's S3.

The proposed benefits of the JAMStack are:

Better Performance
Higher Security
Cheaper, Easier Scaling
Better Development Experience

For more information I'd visit jamstack.org.

Is it really better?

I completely agree with the notion that serving static files from a Content Delivery Network (CDN) makes for better end user performance and provides a much cheaper way to scale website throughput.

I can even get behind the notion that the site is more secure, although I still firmly believe you can misconfigure anything. I wouldn't be using this solely for its security benefits and rather consider it as a defense-in-depth mechanism along with your other tooling and build processes.

What I am struggling to get behind in practice is the Better Development Experience. Naturally everyone creates content differently and for different reasons. I chose to create my own little corner of the internet and create blog content outside of the larger centralized blogging platforms. I risk a smaller reader base but I get to grow something that is my own and enjoy writing about things that make me happy and not necessary to watch the stats in my admin panel get better.

Because of that my experience running my website now is largely about content creation. While I can (and definitely will) re-do this site again at some point, the design is purely driven by intrinsic motivation and a need to spice things up. What is most important to me is being able to jump into my browser and create blog content either on a whim, when motivation strikes, or as a planned endeavour (like today). Despite there being tooling out there to fit this exact purpose I've had a really hard time adopting the JAMStack when all I want to do is regularly author content.

The following experiences aren't going to be presented in any sort of chronological or hierarchical format. They are just different things, that as the sum of their parts, put me off of JAMStack given my current workflow. I would love advice on how this could be changed, if for nothing more than my adding to my "not for me" list.

Markdown is fine, I guess.

Markdown is great. Let me just get that out of the way. I know that in the editor I am using right now it's all getting stored / parsed out as Markdown. However for publishing I still find a What You See Is What You Get (WYSIWYG) editor to be my preferred method of blog writing. It's the closest analogue I get the final result in my browser and it largely matches my experiences in other non-tech editor spaces such as Microsoft Word.

I also don't really want to work in VSCode if I can work in my browser. I write code in VSCode and I want separate tools for separate responsibilities.

So I had a look at Netlify CMS and am yet to try (but am eagerly looking forward to trying) Forestry. These do somewhat fill the gaps I am looking for, but I'd rather not be tied to an authentication provider to make these work if I am going to be moving to static site generation. I'm not sure this is an odd opinion to have, or one that even makes sense. The goal of moving to static site generation allows for flexibility of hosting options, from S3 to Azure, Google Cloud or even self hosting off a Raspberry Pi. Thus, it feels counter to me to lock myself into an auth provider for my server-less CMS. Perhaps this is my major hurdle to enlightenment, letting go and creating "yet another login™".

Oh, I have to make that now...

The benefit of using a CMS is (in my opinion) being able to leverage the combined efforts of a larger team. If I want tags, they exist. If I want to schedule a post my server can do that too. If I want multiple authors I can make that work as well.

Now I can do all those things myself too. But like the rest of us I have limited time resources and I like to use those doing a myriad of things. I don't want to spend 10's of hours writing new features for my site or debugging ones when they go wrong. I don't want over complicated GitHub Actions or serverless Lamba style functions that send dummy commits to my repository to simulate scheduling posts. It's fine but it all seems like a workaround, with far limited user (and developer) experience just to replace something that doesn't seem that bad to me.

If I want to preview my website before it goes live I don't want to jump into a GitHub Pull Request to see it, I want to just open the preview in a new tab. It's not just about when I want my content to come out but how I can best manage it while it is in draft also.

The JAMStack promises flexibility and no reliance on any particular technology. A blank canvas is daunting and easily leads to over-engineering of things that really should be intuitive and simple. I'd argue that for most personal blogging platforms the 1-Click style installers that live on most shared hosting solutions are far less a barrier to entry, even to developers, than using the JAMStack.

Static Sites & Headless CMS

I feel like there is some nice middle ground where I have a headless CMS, connected via webhooks to go and trigger a new static website build. It feels like a nice marriage between many of the perceived benefits of a JAMStack site with the added benefit of a better publishing user experience.

There is just one catch though, if I have a headless CMS that I am paying for I might as well use it to also serve my content. It's true that paying for a VPS is a relatively cheap exercise these days, particularly for low traffic blogs like mine. But they still do incur a cost so if I were to suddenly get famous (maybe off this very post) the cost for me to scale sufficiently would be problematic. I could, as an interim solution, split between serving my website statically but generating content on my headless CMS but now I am paying for two resources and not consolidating to one.

I know it's not all roses over here on the Headless CMS side. In some respects I feel locked into my current provider even though I self host. By writing content in a WYSIWYG Editor my posts are now not portable. If I do need to pick them up and move them tomorrow, or cross post them elsewhere I really don't have a good solution for that. I think in this respect the JAMStack shines and is one of my key motivators for moving.

Where to from here?

Dear JAMStack community. Where should I be going from here? My goal would is and 11ty based site running either Forestry or a competing serverless CMS. I could fairly easily port this website over the 11ty, in part because Ghost uses the same markup language. I could either port all my posts over and try Forestry, or I could continue to use Ghost with the 11ty starter.

For regular content authors what was your approach to moving to the JAMStack and was it worth the effort? I'd love your feedback.

Instantiating MonoBehaviours in Unity

Scott — Thu, 03 Dec 2020 00:11:41 +0000

This article is intended for novice programmers who are looking at using Unity in a repeatable and less tightly coupled way than they currently are. I assume a basic understanding of inheritance and generics as well as Unity functions such as AddComponent.

The backbone of Unity games are GameObjects and MonoBehaviours. One of the complications with using them is that we cannot utilise constructors when creating an instance of these objects. So there is a fine balancing act instantiating and seeding MonoBehaviours when they are being added to a scene dynamically via code.

This article will explore different design patterns that attempt to provide constructor-like functionality to MonoBehaviours to make them a little more user-friendly for programmers used to the Object-Oriented programming paradigm.

The Problem

For the purposes of this article we want to create a new Projectile which will be used by a Gun and when we create that projectile we want to give it an initial speed. Ideally, in an Object-Oriented paradigm we would pass this value in through the constructor like so.

However in Unity we are forbidden from using constructors on MonoBehaviours because they are utilised by the Editor before and after serialisation. Using them can lead to editor and engine breaking side effects when used improperly. In 2016 Unity introduced errors whenever trying to call Unity “main thread” API endpoints from constructors (and pretty much everything is a main thread API endpoint). If you’d like to read about it here is a short Official Unity Blog post.

The Unity way to create our projectile and set it’s speed would look something like the following.

Not only is there a lot more boilerplate code here but we have broken the encapsulation on the Projectile by making the speed float public. We also have no good way to make sure the projectile has a valid speed when it is created.

This is the crux of most of the Unity pattern. It is super flexible, values can be added in the Editor or via code, but because of that a lot code based implementations feel very tightly coupled. If we want to change the projectile to use a prefab instead of a new game object any class that makes a Projectile has to go and implement these changes. *EVERYWHERE.* The gun probably shouldn’t even know how a projectile goes together, it should just be able to make one. What are some ways we can get around this?

Init Method

The Init Method is the most simple implementation which, likely because of its simplicity, is also the most useful in most situations. Simply add an Init method to any class that needs to be set up with values as soon as it is created.

Pros

Flexible
Easy to implement

Cons

Unable to enforce use of the Init Method (we will discuss this later)
Unable to use the Awake Method to set up anything else that relies on speed (this will also be a common theme moving forward)

Static Create Method

I like the Static Create Method approach because it is like the Init method above wrapped in some syntactic sugar. For basic instantiation we often want to repeat the same code over and over again. By leveraging a common base class we can do a lot of that repeatable code in one place. We can even generic out the base class, perhaps like below.

We could then add our own custom Create implementation in the child class, which we were doing with our Init function anyway…

Except now we don’t have to worry too much about actually creating and naming our GameObjects which is nice. We have also preserved encapsulation in our Projectile because the static method can work on the private members of that instance when it creates it. Then lastly our gun can create the projectile.

Pros

Some nice syntactic sugar around Init
Core implementation changes can happen in one place
Obfuscates the need for Gun to know how to make a projectile

Cons

Unable to enforce use of the Create Method
Unable to use the Awake Method to set up anything else that relies on speed

Nested Private Class

This was about the weirdest and most restrictive use case I could come up with. It was derived from the idea that if we decide on a non-standard instantiation method for Monobehaviours it needs to be enforcable. My requirements for “enforcing” a code style (without git hooks or linting or anything crazy) was to make it so that you could not manually add the Component to a GameObject in the scene and also not randomly call AddComponent from anywhere in the game and improperly set up the MonoBehaviour.

Thus the Nested Private Class solution. How does this work, why does this work and should I use it?

How it works

This works by abusing encapsulation. Despite the speed float being public in the ProjectileBehaviour class, because that class itself is a private helper of the Projectile class, we can never get it’s values from outside of the parent.

Why it works

This boils down to how Unity handles MonoBehaviours. If a MonoBehaviour does not contain the same name as the file in which it resides (private or otherwise) then you cannot add it to a GameObject in the Editor.

Should I use it?

Probably not. One neat little trick though was to add some implicit operators so if I really wanted I could do something like this…

There are many cases where you might want to make a new Component but only care about it’s Transform, or some other general type. For example you may want to track it’s position, or see if it still exists in the scene. Although we created a new class of type Projectile we can implicitly operate on it as if it were a Transform.

Pros

Enforces creation via class constructors

Cons

Can’t get the actual MonoBehaviour type (in this case ProjectileBehaviour) back out due to encapsulation. You would need to wrap everything at the class level
Not easily repeatable, at least, I’ve not found a good way to generic this out
Implicit operators aren’t overly verbose and can make for hard to read code later
Unable to use the Awake Method to set up anything else that relies on speed

I wouldn’t really recommend this pattern, but it does work!

Making Your Own MonoBehaviour

If I was willing to nest a private MonoBehaviour class inside a normal class, could I not just wrap the entire GameObject class? Of course I could! I shouldn’t, and for the sake of stopping this article getting too long I won’t show you the whole thing. But if you were nuts about attempting to enforce this pattern everywhere you could do something crazy like this.

I don’t suggest you try this at home, but for the curious…..

Making my Projectile:

Pros

Able to enforce constructor based instantiation
It feels “GameObjecty” when coding in the Projectile class because it utilises the Unity Lifecycle methods
It is super over-engineered (and isn’t that what this is all about)

Cons

It is super over-engineered
You would need to expose and maintain all the functions that MonoBehaviours expose (which seems entirely pointless)
All those “magic” functions that Unity looks for, such as Update, will not be optimised out of your build and each one of your ScottoBehaviours will be creating unnecessary calls to empty functions
I didn’t performance check it but all those delegates would surely cause lots of issues down the track
I couldn’t get the Finalizer to fire on my ScottoBehaviour either?

Prefabs & Factory Method

Prefabs are pre-made GameObjects with preset values that can be reused over and over again like a template in your game. You can make them in the editor and then save them into the project hierarchy for re-use within your project.

If you wanted to implement something that was the MOST Unity I would suggest creating Prefabs and using a class to return an instance of that Prefab at run-time. The specific implementation is up to you. A singleton instance would allow you to drag and drop the Prefabs into your game at run-time. A static class would allow you to load Prefabs from either an Asset Bundle or the Resources folder or you could come up with your own serialisation and deserialisation methods. Perhaps look at combining with a pool for even better performance! As an example I have written a simple static class that loads assets from the Resources folder based on a string value. Please don’t use string values in production code, use an enum or something else that can’t easily be typed incorrectly. The code below is merely supposed to demonstrate the concept.

Our Gun would then create a Projectile in the following fashion.

Pros

Utilises all the Unity tools rather than trying to fight them
Allows for tweaks to be done in editor rather than always in the code base
No weird class wrapping
Gun doesn’t need to concern itself with the speed of a bullet

Cons

Can’t enforce creation through the factory method
Can’t use the Awake function in projectile

Final Thoughts

Most game developers that I talk to that work on larger projects seem to spend most of their time fighting with Unity rather than embracing it’s paradigms. Sometimes there is good reason to do that but often I think it’s to try and do things a “better” way. Which, aside from being totally subjective, takes a lot of time away from making the actual product.

In almost all my test cases aside from going to everyone I work with and saying “This is the pattern we use for instantiation on this object now” there isn’t a great way to enforce a “newable” pattern on MonoBehaviours. All the solutions above would work in different circumstances but none of them work well in all situations. I tried very hard to do run-time checks to make sure an Init or Create function were called but they are simply too easily overridden and any other service on top just makes these approaches too heavy to manage. I’d like to explore linting rules later around this problem to see if that is an enforcement option.

Interestingly in all the situations, with exception of the entirely wrapped GameObject class, calling AddComponent on any MonoBehaviour class basically renders the Awake method moot if it needs to use any of the values you want to set via a script. Whether you do that the good old fashioned way, or over-engineer some nonsense like above, Awake is called immediately. You just have to deal with the fact that your MonoBehaviour won’t be set up in time properly to utilise that method so stick with Start if your set up requires these values.

Debugging Meta Tags On Localhost

Scott — Wed, 25 Nov 2020 22:29:46 +0000

The Problem

Meta tags are useful machine parse-able tags that live in the header of your website. As their name might suggest these tags provide extra metadata about your site. That is, data about your data...

Their usefulness (aside from providing extra metadata about your site) is largely derived from their use on social media. Whenever you see a link shared with a nice preview image and description this information is getting pulled from the meta tags embedded on the site.

The only slightly annoying thing about meta tags is testing them on your local machine, which is imperative if you don't want to do live debugging in production.

The reason this is difficult is because your machine is not (typically) accessible to the internet, particularly from localhost, so the sites that you might want to check them against simply can't pull the relevant information to generate the previews you are after.

The Solution

We need a quick and easy solution to make the website we are hosting locally accessible on the internet without worrying about building it or hosting it elsewhere, or even port forwarding. To do that we are going to use a product called ngrok. This tool provides "...secure introspectable tunnels to localhost...". In short we can give our running localhost server a short lived, temporary url to test our meta tags against.

Make sure you have your meta tags set up on your website already. If you don't know where to start you can scroll down to the section "Altering (Or Making) Tags" and then come back.
Download ngrok and follow the setup instructions. (You will need an account but they have a free tier).
Spin up your local web server.
Start ngrok on the same port as the server you are running. For my local Ghost CMS that is port 2368.
Copy the URL from the command line and paste it into your web browser.
You should see the same web server you had running on localhost. 👍

Testing the Tags

There are a couple of ways to do this, and I used both of them, to be sure they were working as I intended.

Use a website that parses meta tags for you, I have had good luck with both metatags.io and heymeta.
You can DM yourself on Twitter, or even post your link to yourself in Slack to the Slackbot, and then unfurl the URL.

There are a few "gotchas" that I noticed with both of these methods.

I am not sure if the above sites cache results locally, but when I was consistently changing the default image for my meta tags I had to test in an incognito window. Otherwise I would often receive the same result over and over.
If, like me, you are checking your meta tags because a previous share left for a lackluster link preview, just note that in applications like Twitter (which preview the link before you send it) are likely to show your old preview. You are going to want to actually send it properly which seems to resolve whatever caching occurs on the link preview.
I wanted to use fully qualified asset links to be safe (e.g. https://site/path/to/image) rather than relative (e.g. ./path/to/image). Because of this quirk I made sure my static assets were available on my production domain before I did the test. If relative links work for you though please let me know so I can update this part of the post.

Altering (Or Making) Tags

If your meta tags aren't quite right, or you aren't even sure where to start the two sites mentioned above (metatags.io and heymeta) allow you to edit the tags directly in the browser and generate the result. This lets you quickly iterate on your design without having to worry too much about the code or missing important tags.

When you're done close down ngrok, push to production and pat yourself on the back.

I Contributed to Open Source (and it was hard...)

Scott — Mon, 23 Nov 2020 22:20:43 +0000

This is a look back at my first experience with Open Source prior to Hacktoberfest 2020.

With Hacktoberfest coming up in just over a month I thought now was a great time to share my experience contributing to my first Open Source project (that wasn't my own).

Why Open Source?

I don't want to spend too much time on this question from a social perspective because there is already a lot of writing on the value of Open Source tools within the technology community. What I do want to do though is share why I have wanted to contribute to Open Source for so long.

Just as my Honours year at university was winding down I applied for (and successfully received) a PhD scholarship, which for various reasons I ended up turning down. However, my initial desire to chase a life of academia was born out of a conversation I have replayed with a friend a few times over late nights.

He shared his passion for the academic system and particularly the PhD path. This idea that we all incrementally make progress by adapting and uplifting each others ideas and ultimately then give that knowledge back to the rest of the world to use in ways that empower and help each other at scale. While I am critical of whether the modern day university fills that romantic notion any more there was something elegant to being able to give back in some small way. That my actions would ultimately be greater than the sum of parts I had to put in to make it possible.

Open Source represents that same ideal to me. That I am able to make a series of small changes that ultimately, along with my peers, make for a better thing™ is a really rewarding and empowering idea. I think it captures the essence of what I was searching for in academia but seems to apply it in a more pragmatic and timely approach.

I lay out the above because for me contributing to Open Source is akin to volunteering. It requires my time to make something better (hopefully) and have others enjoy it. My time has inherent value and so when I chose I was ready to commit to Open Source it needed to be for a project (or projects) that I either used or wanted to support.

Getting Started

Here is the issue though, just because I use certain software doesn't mean I always know how it works, or in some cases, it just doesn't utilise the languages or skills I know. For some people this provides a fantastic place to start learning new skills or pick up a new challenge. But what if you're a relatively new coder, or at the very least, new to collaborating on Open Source projects? You, like me, probably want to tackle a language you know and hopefully a tool where the code base is at least representative of work you have done in the past.

With Hacktoberfest coming out soon I took some time out of a quiet work day to do some investigation. What repositories were out there that I knew of and thought were worth supporting? Of those which ones had languages I was comfortable with? Did those repositories have huge communities where issues were either snapped up really quickly (or the sheer volume was overwhelming)? Was there anything that looked small enough that I can conceivably grab and get a small win on in a day or so? There's a lot of things that can be a blocker to getting started that aren't just code related.

After a while I ran into the Ghost blogging platform (the very platform that powers my blog). It seemed perfect, it was in a language that I was familiar with and it was a product I used. By chance 9 hours earlier a small visual defect in the Admin Panel was raised as an issue which seemed like something I could pick off.

Setting Up My Development Environment

What I have always taken for granted on my development projects has either been that they are "green-fields" (brand new) or I've worked with someone that I am inheriting the project from so there is a wealth of knowledge on how to set up and maintain those projects. Additionally as most of those have been with similar team members the patterns are the same. Despite Ghost providing robust install from source documentation it was still frustrating to get it to run locally. For brevity here was roughly the journey:

Not wanting to run yarn side-by-side with my npm installs (I thought these were competing platforms, but really they can totally live side-by-side)
Windows being unfriendly with some Ghost related setup (like having to run Powershell in Admin mode?)
Trying to circumvent the whole issue by running it in a Docker container.
Realising that the Ghost install only listens on localhost and having to change default values to listen on all IP addresses so I can access it from my host machine, then worrying I will accidentally commit the file.
Giving up and just running yarn and npm next to each other...
Yarn commands not working unless I prepended them with yarn because npm was trying to take over!!!!

Making The Change

It is really cumbersome looking through a new code base to find where things hang together. There is a feeling of complete loss and confusion. Hacking in a fix is generally pretty easy. Is this the right place, is this where the maintainers would want me to put it? How thoroughly do I need to test the fix, it's three lines of code! What if I cause a regression? Has someone else already fixed it in the time it has taken me to track the fix down?

About 4 hours after deciding that I was now an Open Source Contributor I was feeling a little tired but glad I took the plunge, now all I had to do was get the PR up and wait...

Making the PR

Although I am starting to adopt Github for some of my public side projects I am largely familiar with Bitbucket and Azure DevOps and predominantly on single projects where merges happen internally on branches. So forking repositories and making merges from one repository to another and trying to work out if I can link issue tickets from the main repository to a sub-repository was all a little taxing. It's doable and it's not hard, and almost all of it is reversible but you still want to get it right. Having my first Open Source PR publicly rejected because I missed something silly would have been embarrassing for me. Thankfully the good folks over at Digital Ocean do have a good primer on this if you're new to the wacky world of Open Source Pull Requests.

PR Complete

Using my Changes

In less than 24 hours my change was approved, merged and included in a new build. The following morning I was able to update my blog to Version 3.31.1 (for those of you playing along at home) and my glorious css selector was added to the code editor widget here in the very Admin Panel I use to draft posts. Pretty cool!

Where To Next?

I always have a lot of side projects I am keen on and there is one I plan on starting around Hacktoberfest because it seems like a great way to get a free t-shirt and work on something that is both fun and maybe pointless! I'm definitely going to keep an eye on software I use (or think should continue to exist) and try and pick more of these smaller issues up. It was rewarding to give back to the software that I get to use for free.

With that said I do hope the Open Source maintainers try to find easier ways to get projects up and running for local development. If I could download a docker image, do some minimal set up to that and just connect to it via VSCode to do all my development work that would get me onboard so much faster. Having to stand things up and configure them for what was a three line change was definitely at some points a little frustrating. And if I don't come back and pick up anything soon it was definitely a lot of work for a small reward.

Perhaps for Hacktoberfest I will get lucky and find some other small PR's or even documentation fixes. If you're like me and looking for a place to start then I found that the "Awesome For Beginners" List was a good place to start.

With it all said and done it was great to finally break that seal of indecision and get in there and give it a go. It went better than I thought and I am now more confident to try it again in the future. Just don't take the easy tickets from me if you can help it 😉. Happy Open Sourcing!

Exploring the Myth: Calculating Square Root is Expensive

Scott — Fri, 20 Nov 2020 06:11:38 +0000

I know this curious fact about game development that says where possible never to use the magnitude of a vector (unless required) because it involves a costly square-root calculation. Even the Unity documentation affirms this notion. What’s interesting is that I’ve never officially learned this and I only really see it floating around the internet, most recently on Reddit. It seems to be one of those things aspiring game developers learn through osmosis.

It is one of those sayings that makes sense on the surface but I wanted to dig into how much slower the square root operation was and if it had any meaningful impact on performance in what I would deem, “normal” circumstances.

When programmers talk about the cost of an operation they typically mean how many instructions are required to perform the operation. For example a multiplication would typically take three instructions, two reads and one write. For more complex operations (such as division) it often takes many more steps to calculate an accurate representation of the result, thus, the expense in the operation. While square root may have once been an extremely costly exercise I have a hunch that it is now a much less relevant piece of advice than it used to be. Additionally it leads newer programmers to focus on changing the way they write code in order to optimize said code as they go. I am a big believer in writing clean, verbose code and optimizing only when it is absolutely necessary. That can be hard as a new programmer where you often want to write cleaner and more efficient code than your last attempt at solving the same problem. It gives you a sign that you are progressing and allows you to tackle bigger problems.

I devised some small tests to get some real world numbers on the time it took my machine to complete a large number of square root calculations and then compared them with the alternative.

The Experiment

Perform 1,000 loops of 1,000,000 calculations (yes for a total of 1,000,000,000 calculations). Record the minimum, maximum and average time it took to complete each one of these loops in “real world” time. Each loop consisted of either a square root calculation, multiplying a target variable by itself, or raising the same target variable to the power of 2.

I’m not overly concerned about how long any of these operations actually take. I don’t care about the fastest time, I care about the proportionate time between the operations. The likelihood here is that I may very well be able to get faster times given different set ups. As an example everything was being run in debug mode on a Windows machine, this is likely to effect the overall time it takes to complete each task. Take the actual values with a grain of salt, we can compare the interesting parts further down. To see the code I used to run the tests check out my gists here. If you want a brief overview of the code I was testing out it really all boils down to comparing the below.

The basic set up

If the prevailing opinion is that square root is slower than simply multiplying our target value by itself then it is obvious to pit those two calculations against each other. I chose to add the power function to my testing because it seems like a simple interchange to make. Instead of using square root I could instead square my target value by raising it to the power of two.

I’ve also added some Unity specific tests focusing on Vector3.magnitude vs Vector3.sqrMagnitude as another metric by which to judge, because quite frankly, that is more important to me as a predominantly Unity developer.

To make sure this myth is not language specific I tested in dotnet Core 2.1, Unity 2018.2.14f1 .NET 3.5 Equivalent, Node 8.9.0 and Python 2.7.15. For reference I am testing on Windows 10 on an i7 8750-H CPU.

Results

As mentioned above I’m testing that this myth exists across programming languages. However I don’t want to compare between programming languages because the speed of the language in general doesn’t bother me. Let’s see how each language performed.

Time per 1,000,00 calculations.

Time per 1,000,000 calculations. Notice the added calculations for Vector magnitude.

Time per 1,000,000 calculations.

These results show us that there is a small difference in the speed of calculating a square root when compared to simply multiplying our target value. In c# the power function was considerably slower on average than both the square root and multiplication approaches. We could easily write code that performs worse than just simply using our square root calculation to start with. The latter also happens to be easier to read code.

Ignoring the fact that performing Vector math is on average slower than float math, which I expected, checking the magnitude operation was not that much slower than checking the square magnitude.

In an attempt to put this altogether I tried to visualize how much faster, or slower, each approach was than using a square root.

Number of ‘times’ of performance.

We can see that in the case of the Unity it is significantly better, 2.5x in fact, to use multiplication over using a square root. The other languages however are all modestly different, if we use either approach for a reasonable amount of calculations we are unlikely to see a serious bottleneck in performance.

In the best case scenario, at 2.5x better performance for multiplication, what sorts of gains could we expect to see? Per operation, that is for a single square root, we could save a whopping 0.033173 microseconds. If we instead tried to be smart and raise our target value to the power of two we would make things considerably worse, however we would still only be adding 0.157795 microseconds. There is no doubt that performing Vector math will have overhead because of the two dimensions but performing a check on square magnitude instead of magnitude only nets a performance increase of 0.051819 microseconds.

Final Thoughts

The above is a classic case of micro-optimization. On paper it seems amazing to write code that is 2.5x faster. But it does come at the cost of some readability, and debug-ability, for fairly minimal performance gain. Technically square root is slower than multiplying our target value by itself, but practically I am not so sure, not for typical use cases anyway. If you are new to programming it is fine to learn these pieces of information and keep them tucked away. With that said you do not need to rush out use them when you could simplify your math. Checking against something such as magnitude will be easier for you, or your coworkers, to debug later.

If you are in the position where you need to calculate 1,000,000 square roots in one frame of a game loop then I’d argue you have a design problem. Look at alternative solutions such as separate threads or an async pattern instead of looking to try to optimize your square root function. I would also like to hope that by the time you reach a problem like this you are already well on your way to understanding the pitfalls of micro optimization.

As a final note I found the speeds coming out of Unity as a whole really interesting. Of all the languages I expected the Unity square root approach to be one of the fastest across the board. Given the language is designed for game development I expected a slightly less accurate float with the benefit of speed. That just didn’t seem to be the case here. My advice, get the feature in and optimize it once you know it is a problem.

Gittin' efficient with Git Push

Scott — Thu, 19 Nov 2020 05:47:16 +0000

At work we undertake two university placement rounds each year. It's always fun to get to teach people things you know and get exciting new ideas and energy into the team from fresh faces.

One of the other benefits of hosting juniors is that they often ask questions you've never thought about or just taken the answer at face value. One such question during onboard related to the semantics about git push. So let's take a look at the current behaviour, and then how we might change it.

The Problem

Let's say our student is being on-boarded to an existing project and they are being asked to work on an existing branch upstream. That's pretty easy, we get them to do a git pull on the relevant branch.

Now assuming that the developer does not have a local copy of this branch git goes ahead and sets all that up for us making an association between the upstream branch and the local copy it is currently creating.

What happens however if the student being on-boarded is asked to work on a new feature or bugfix branch and nothing exists in the upstream yet?

They create the new branch, perhaps with git flow, and start working. After their first commit they decide to push their changes upstream. But unlike when we pull a branch git doesn't automatically create an upstream version of our branch in the same way that it does with a local one if pulling.

No upstream set

When asked about this behaviour my response was the following:

Just copy that last command in the terminal with the set-upstream bit and it will do it for you.

While that is true and it does work it doesn't really answer the question and that bothered me, so why does that happen?

The Git Push Spec

push.default: Defines the action git push should take if no refspec is explicitly given. Different values are well-suited for specific workflows; for instance, in a purely central workflow (i.e. the fetch source is equal to the push destination), upstream is probably what you want. Possible values are:
nothing - do not push anything (error out) unless a refspec is explicitly given. This is primarily meant for people who want to avoid mistakes by always being explicit.
current - push the current branch to update a branch with the same name on the receiving end. Works in both central and non-central workflows.
upstream - push the current branch back to the branch whose changes are usually integrated into the current branch (which is called @{upstream}). This mode only makes sense if you are pushing to the same repository you would normally pull from (i.e. central workflow).
simple - in centralized workflow, work like upstream with an added safety to refuse to push if the upstream branch's name is different from the local one.

As this branch is new and no refspec has been given the default git implementation (which is simple) says that for added safety we refuse to push up the branch without an upstream. It allows beginners to make sure they truly understand what they are doing before they make their change. That's nice and all but after a long time with git it feels inefficient so how can we change this to be useful and what are the caveats?

The Solution

As usual the actual solution here depends very much on your work flow and how you like to use git. I propose a few of the following after testing how they work.

Change push.default

The simplest method here is to change the push.default config parameter to something other than simple. Both current and upstream work but there are a few gotchas to each.

git config push.default current

When using the above command git will happily create / update a branch upstream for us that matches the name of our current local branch. In cases where you do trunk based development, or your developers largely work on their own branches (like we do with git flow) then this is almost universally going to be a great choice.

What happens if we routinely rename our branches though? I work with Jakob, and while Jakob is a good guy (for the sake of this post) he creates terrible branch names. For example imagine the branch name below:

feature/scott-is-the-worst

As this branch is is an outright lie, when I work on this locally I want it to read something else. So on checking it out I set up tracking and change it's name.

git checkout -t -b feature/scott-is-the-best origin/feature/scott-is-the-worst

Now my branch is appropriately named I can work on it and finish whatever needs to be done. With my push.default set to current what happens when I push now? Git creates a new upstream branch notifying everyone that I am awesome, but leaves poor Jakob's branch telling everyone I am the worst to be stale. In this case I think he has every right to call me the worst, messing up our nice branching strategy.

Perhaps I could fix this with push.default set to upstream instead?

git config push.default upstream

If we replay the same steps above (for brevity I won't repeat them, expect for the Jakob being mean bit) then we expect I have a renamed branch on my local machine with changes ready to push upstream. What happens now? As the name might suggest we update Jakob's poorly named branch with my changes despite the naming being different. This happens because when we checked out the branch locally and changed it's name git set up tracking of the upstream so it knows which branch it needs to update. Nice!

Now I am done fixing Jakob's branch I need to do some new work. I go and create a new branch of my own to work on.

git flow feature start jakob-is-mostly-alright

I commit all my changes and want to push this upstream. So I do a push only to find that git doesn't know what to do. Like with push.default set to simple, upstream has no tracking set up between this local branch and our origin so it doesn't push up the code and asks me what I want to do.

Git Aliases

Depending on how you want to work you might find that keeping push.default set to simple or upstream provides the safest method for making sure the correct branch always receives the correct updates.

That's totally reasonable but it still doesn't help us with the initial push. Thankfully we can create git aliases which provides short hand on the command line to run multiple or verbose git commands. I tend to use a version floating around the internet called pushup.

git config alias.pushup \!"git push --set-upstream origin 'git symbolic-ref --short HEAD'"

What Should I Do?

My current work flow is to use push.default current because it suits my use case. If you're nervous about changing the defaults you could try push.default upstream with the alias set up for some quality of life on the command line. What's great about the latter is you'll always know whether your branch is upstream or not because it will cause an error the first time. If you're nervous about sending your code upstream until it's not embarrassing this might be a good method to double check that.

A Word on Git Aliases

In almost every online tutorial for git the writer will append the --global flag to their git config commands. As this might indicate the --global flag changes your configuration for all the git repositories on your machine. While that's great shorthand I like to configure each repository manually each time. It means I don't accidentally inherit a setting (like push.default current) which might cause issues on a new project I am part of.

The default git settings are extremely conservative so always be aware of overriding them at the global level. With that being said some things (like aliases) can be super handy in your global config.

Enjoy the power of typing less with aliases or a push configuration that better matches your workflow!

Serving a SPA with Express Server Router

Scott — Mon, 16 Nov 2020 05:42:36 +0000

For the longest time, largely because of the size and tech stacks of projects that I have worked on, I've treated the client and server of an application as strictly disparate entities. The client is it's own self contained application that often lives on a different host with it's own configuration, code repository and CI/CD pipelines. In a PaaS environment (like Azure) I think this pattern makes sense because spinning up and down new instances is easy and doesn't come with the same maintenance as perhaps IaaS where your containers and operating systems need to be maintained, patched and updated yourself.

Introducing the side project

I recently started working on a personal link service, like bit.ly or similar that allows me to better manage my links on social media. Mostly to allow me to retro-actively fix stale links or provide a more consistent link experience. I know there are plenty of open source variants that do this but I wanted to use the opportunity to roll my own. Mostly for fun, partly to learn.

As with all side projects I want this to be cheap when I deploy it, vanity domains and hosting add up really quickly when you don't monetize any of your work. It got me thinking. I already host my blog, I'm about to run another node server and it also needs some sort of admin panel so that I can enter and edit links. Perhaps I can consolidate all of these onto the one server, save costs and learn a little bit about hosting files in node.

You might have other motivations for serving a Single Page Application (SPA) from your Express Server though.

You might need to deploy to a more traditional server that requires patching and maintenance and you really want to minimize the amount of infrastructure that requires that level of up-keep. (I'm not going to lie, this is another motivation for me).

Alternatively you might want your spa to live at your-domain.tld/app rather than app.your-domain.tld. The former is trivial to do if it's being served by your API which we will step through now.

The Anatomy of a Basic Express Application

One of the reasons that I chose node (and Express) to run my API was how simple it is to get a web server up and running. (I also think I really like JavaScript now, but don't go telling my colleagues that, I have a reputation to uphold 🤐 )

A Simple Express Server

Here is a really simple Express Server, you can send a get request to the /ping endpoint and be returned the pong message to know the server is alive.

I even have an extremely flat folder structure for this demo as you can see below.

I'm going to assume some prior knowledge for node and Express here to keep this post short. In general though Express applications are built by a series of middlewares which execute against your request in order (and that order matters). The same is true for evaluating which endpoint actually receives the request, it matches the first route that satisfies the request even if a more specific one is defined later.

Adding Middlewares

The natural consequence of maintaining middleware order could be to continue to add all your routes to your main Express Server file (normally index.js) to try and preserve the order you want them evaluated in. That gets messy though so where possible you want to compose your major routes with the Express router. For example let's say that we want to add some "admin" routes for my new admin portal. I could do the following and tell my app to use a different file to manage any routes that start with /admin.

Telling Express to use another file to manage admin requests

Now I can separate my code out into logical units with more ease and also add and remove arbitrary admin routes without constantly having to go and refactor my main file that is largely just configuring the application. But how might we define a router in a new file and specify some routes?

A router definition for an Express application.

Hopefully the code here is fairly self documenting. We create a new Express Router. We create a new route definition and then handle the request like we were in our index.js file, except instead of appending the get operation on the app itself we append it to the router. Because we want to serve a SPA, and one of the defining features of a SPA is that it handles it's own routing, we want to create a rule that matches any sub-route of /admin and just return the index page of the SPA itself. In scenarios where the SPA is hosted by itself this would happen in your reverse proxy or your web server configuration, but now we can easily do it in from within Express itself. You notice that I am serving an index.html file from a folder called admin-client that should be in the current working directory of the application (typically the project root).

You will note that I don't have to prepend my routes with /admin in the controller because we specified where to attach the router in our index.js file. If we decide in a week that we prefer for the SPA to live at /app instead of /admin we can simply change the one line in our index.js file and the routes all work again, yet another reason to pick up the Express Router in your projects.

Assuming you have built your SPA and dropped it into the admin-client folder you should notice that running your application and hitting http://localhost:3000/admin in your browser nothing renders and you get lots of errors in the browser console. We're still missing one step, now any time we try to request any file at all (remember the *) we are returning our index.html page. Want your css stylesheet? Have our index.html! What about a favicon? You guessed it more index.html. While we have set up routing to deal with serving our pages we haven't added anything to serve our static content and Express provides that functionality out of the box as well.

Adding static file serving to an Express route.

Above our router definition we add another middleware definition, on the same route but instead using the express.static middleware. You might be able to guess what this is doing. When a request for a resource to /admin is made it first runs through the express.static middleware and attempts to find a file in the admin-client folder that matches the request file. If one is found it returns it, if one is not is falls through to our admin controller. Restarting your Express Server and refreshing your browser you should now see your SPA being rendered correctly.

To illustrate exactly how middleware operates if you were to swap the express.static and router implementations around you would end up with the same issue as when we hadn't specified the express.static middleware at all. All requests to /admin/* would get caught up by our router middleware and always returns index.html which is why we declared it the way we did above.

Some Thoughts

For an API first solution, or one where you want to save on costs this could be a really great solution. You would have to worry about issues of scalability long term (as opposed to say serving your content via a CDN), and the Express documentation says that production workloads should live behind a reverse proxy. Mostly so that the proxy can do things like handle caching of the index.html file and generally do what reverse proxies are good at doing (and Express is not).

One thing I have not done yet (as the project isn't quite as polished as I would like) is determine the best way to actually build and deploy this solution. When I started my repositories were separate, because I was treating the client and server separately. In reality because I need to build my SPA, drop it into my Express Server and then publish that perhaps I should be looking at a Monorepo set up or some other way to streamline the process.

I also think given I need to consolidate my blog into this server as well that there may be some containerization coming my way. Stay tuned as the service rolls out and I can offer more learnings from this experience.

Using Hot Module Replacement in Angular 11

Scott — Fri, 13 Nov 2020 06:21:50 +0000

Angular 11 was released today and with it comes a large number of performance upgrades, bugfixes (including the third most popular issue) and improved logging.

Not only was the upgrade from Angular V10 to V11 painless but I wanted to share the new Hot Module Replacement feature. This has been exposed on the CLI if you hadn't tried it in previous versions, so now is a great time to dive in and give it a go.

For those unaware this feature is not native to Angular, rather webpack. If you'd like to read more about it you can find their documentation here. In short you can think of Hot Module Replacement as a more advanced version of a file watcher, only replacing the modules that need replacement and (where possible) preserving state. So what does this look like in action? See the videos below (which are best viewed full screen).

Classic Angular Reloading

The above should look very common for front end developers. We are testing out our model binding in our form. We decide that it's working well but we want to change the label on our input. Doing so forces a refresh of the page and we lose our state.

Hot Module Replacement in Angular

With the Hot Module replacement flag used we see a slightly different story. The initial flow is the same. We test our model binding and again want to change the label on the input. This time the Hot Module Replacement preserves our state and our input still has our user generated value. Nice!

Adding Hot Module Replacement & Use Cases

In my current project I've nominated that we additively pick up this new feature. For most uses cases the standard watcher works just fine so retooling for a slightly better development experience for some might hurt others in the team. I propose you add a new script to your package.json while you're adopting this to see if it suits your needs. Perhaps something like "serve:hmr": "ng serve --hmr". Then your team can either run npm start for their normal experience or npm run serve:hmr. This gives you the best of both options and allows you to pick and choose which reload strategy based on the functionality you are building out.

Consider using Hot Module Replacement when:

You have a complex flow that you do not want to restart.
You are making cosmetic changes and want to test on non-standard input (such as long strings).
You are fixing / tweaking API calls and want to reuse the same form values.

What else do you like about the Angular 11 update? Let me know on Twitter or check out my content on my site.