DEV Community: Paul Mooney

Enhancing the Developer Experience of Testing Part 2

Paul Mooney — Sun, 08 Dec 2024 01:14:07 +0000

In my last post, I talked about writing tests with natural language descriptions, and leveraging structured testing to provide organized and comprehensible output. Here I will expand more on these concepts.

Structured Testing Part 2

Structured testing is a nesting of contexts, until eventually terminating in one or more tests. In NodeJS test frameworks, a context is a describe function. In JUnit a context is the test class itself or a @Nested inner class. Contexts, just like tests, can also be described (or named) with natural language and can appear in the test output's tree structure. Here are some of the ways we can use this contextual test structure for well described tests:

Given-When-Then Pattern

Borrowing from Behaviour Driven Development (BDD), one of the ways we can structure our tests is to use the Given-When-Then pattern. In this pattern we nest the Givens, Whens, and Thens in a hierarchical layout where the Givens and Whens are contexts and Thens are the tests (in most popular testing frameworks).

The Given context's clause describes some kind of preconditions. I like to include inputs here as well because it gives purpose to the Given clause even when the thing being tested does not require preconditions. I will talk more on this later, but some people may prefer to not include inputs here.

The When context's clause describes the execution. It can also be used to describe inputs if you prefer. Again, more on this later.

The Then clause for tests should describe our expectation.

Note: It's common to see synonymous conventions to these like "Arrange-Act-Assert", or the use of "it should ..." for tests instead of "Then...". This is all fine, as long as consistency is maintained.

There are generally at least two more kinds of contexts in this pattern that we will see:

The And context. This is used for providing additional preconditions or inputs to compliment the Given or When
contexts.
The class or file under test context. This is the highest level context and just describes the class or file containing the units to be tested. It could also just be a name for a group of tests in a particular test file. This comes implicitly in some test frameworks and may default to the name of the class or file containing the tests.

Here's an example all of this in action using JUnit:

@DisplayName("REST Endpoint tests")
class RestEndpointTest {

    @DisplayName("Given an `id` for an existing record")
    @Nested
    class GivenValidId {

        @DisplayName("When calling the `GET /api/record/{id}` endpoint")
        @Nested
        class WhenCallingRecordEndpoint {

            @DisplayName("Then it should return the record for that `id`")
            @Test
            void testRecordReturned() {
                // test code here
            }

            @DisplayName("Then it should return a 200 response code")
            @Test
            void test200Response() {
                // test code here
            }

            // ... other tests
        }

        @DisplayName("And the `Accept` header is `application/xml`")
        @Nested
        class AndAcceptHeaderIsXml {

            @DisplayName("When calling the `GET /api/record/{id}` endpoint")
            @Nested
            class WhenCallingRecordEndpoint {

                @DisplayName("Then it should return the record for that `id` in XML format")
                @Test
                void testRecordReturnedInXml() {
                    // test code here
                }
            }
        }
    }

    @DisplayName("Given an `id` which does not correspond to an existing record")
    @Nested
    class GivenInvalidId {
        @DisplayName("When calling the `GET /api/record/{id}` endpoint")
        @Nested
        class WhenCallingRecordEndpoint {

            @DisplayName("Then it should return a 404 response code")
            @Test
            void test404Response() {
                // test code here
            }
        }
    }
}

This pattern fits nicely with the "One assert per test" rule. Sometimes, from one execution (a When context) we have multiple expectations (Then tests). For example, if we were testing a REST endpoint then we might expect that the response body looks a particular way, but also separately we expect the response code to be a particular value. These are two conceptually different expectations to assert from the same single execution with all the same preconditions. And there could be more - maybe there were certain response headers expected, too. Breaking our expectations up into multiple Then clauses/test lets us describe each expectation separately.

UnitUnderTest-Given-Then Pattern

It can be helpful to group all contexts and tests by the unit being tested. From an organizational standpoint, this is where the Given-When-Then pattern needs an alteration because, in its current form, it lacks a way to organize all tests for a particular unit. This can be difficult from a code organization standpoint, and since test order is usually randomized, the tests for a particular unit become scattered amongst the contexts and tests for other sibling units.

To combat this, I like to organize my tests in a UnitUnderTest-Given-Then pattern. "UnitUnderTest" is the particular unit being tested, whether that be a method, function, REST endpoint, or other unit. Our structured test hierarchy would look like this:

ClassUnderTest
- Method One Under Test
  - Given some arguments and preconditions
    - It should return an expected result
  - Given some other arguments and preconditions
    - It should return a different expected result
- Method Two Under Test
  - Given ...
    - It should ...

Here's an example of this pattern in action using JUnit. Notice how the test scenarios for a particular unit (in this case units are methods) are grouped together under a context for that unit:

@DisplayName("PersonService")
public class PersonServiceTest {

    @DisplayName("findById") // <-- Unit under test: the `findById` method
    @Nested
    class FindById {

        @DisplayName("Given an `id` for an existing record")
        @Nested
        class GivenIdForExistingRecord {

            @DisplayName("It should return an `Optional` containing the record for that `id`")
            @Test
            void thenRecordIsReturned() {
                // test code here
            }
        }

        @DisplayName("Given an `id` which does not correspond to an existing record")
        @Nested
        class GivenIdForNoExistingRecords {

            @DisplayName("It should return an empty Optional")
            @Test
            void thenNullIsReturned() {
                // test code here
            }
        }
    }

    @DisplayName("deleteById") // <-- Unit under test: the `deleteById` method
    @Nested
    class DeleteById {

        @DisplayName("Given an `id` for an existing record")
        @Nested
        class GivenIdForExistingRecord {

            @DisplayName("It should delete the record for that `id`")
            @Test
            void thenRecordIsDeleted() {
                // test code here
            }
        }
    }
}

Note: This does put a bit of a demand on the reader to understand the convention going on here, as any sentence that can be inferred from the hierarchy starts deeper into the hierarchy than it does in the "Given-When-Then" pattern. I believe this is not a difficult expectation for readers to pick up on this pattern, especially given the organizational advantage it provides.

Alternative: UnitUnderTest-Given-When-Then Pattern

So far, I've omitted the When context because I state the inputs in the Given context. This is a personal preference and if you believe Givens should only state preconditions, then you can include a When context to describe the inputs. Both of these do not need to exist - sometimes you don't have preconditions, and sometimes you don't have inputs.

Example with both Given and When contexts, because there is both a preconditon and an input:

PersonService
- deleteById
  - Given there is an existing record
    - When called with the `id` for the existing record
      - It should delete the record for that `id`

Example with just Given context, because there is only a precondition and no input:

PersonService
- deleteAll
  - Given multiple records exist
    - It should delete all records

Example with just When context, because there is only an input and no precondition:

PersonService
- getFullName
  - When called with a `firstName` and `lastName`
    - It should return the `firstName` and `lastName` concatenated with a space in-between.

Some Dos and Don'ts

We've gone over the general framework for writing good tests, at least from a structure and output standpoint. Let's look at some more specific tidbits as Dos and Don'ts.

Do: Provide contexts which describe preconditions, inputs, and their relationship to each-other

Look at a test structure like

PersonService
- deleteById
  - Given an existing record
    - It should delete the record

This sounds wrong. We know from the description that a record exists and a record is deleted but was it the same record? How is it deciding which record to delete? Details are missing. Clearly there is an input which is not being described here and then there's no relationship described between the input and anything else. The improved version of this test would be:

PersonService
- deleteById
  - Given an `id` for an existing record
    - It should delete the record for that `id`

Now it's easy to understand what's being deleted and why.

Don't: Put explicit values in the description

Unless that exact value has relevance in the code. For example, if we had

PersonService
- deleteById
  - Given an `id` of `1234` for an existing record
    - It should delete the record for that `id`

Reading this, it seems like 1234 is important to the unit being tested. As if, there is an explicit check in the code for 1234, and if we had a test Like "Given an id of ABCDE for an existing record..." then we might expect a completely different behaviour. In reality though 1234 is probably not important, it's just a sample value, so it should be left out of the description to avoid confusing the reader. Another example of this might be where both the inputs and expectation are described with explicit values:

MathUtil
- hardToFullyDescribeMethodName
  - When called with a `radianValue` of `1`
    - It should return the `2.3974`

Now I'm left wondering what happens if the radianValue is 0 or 2? What does hardToFullyDescribeMethodName actually do? It's better to leave out explicit values and convey the preconditions, inputs, and expectations in more general terms:

MathUtil
- hardToFullyDescribeMethodName
  - When called with a `radianValue`
    - It should return the sum of tan and sin of the `radianValue`

It's fine to use explicit values in the actual test code, just not in the description. There are some exceptions to this rule, like when the value is some kind of enumeration, boolean, or actual special case value. Or when writing parameterized tests. Then it makes sense to state those values in the description.

Don't: Leave out important information

I've seen tests like

Should return a Person record with no bio

This leaves me asking so many questions. The author might as well have called this "test 1". I'm going to be generous and say that maybe they did try to describe the preconditions and inputs:

PersonService
- findById
  - Given an `id` for an existing record
    - It should return a Person record with no bio

Maybe this is fine, because that's the general expectation that this particular method always leaves the bio field empty. But if the reality is that there's a condition where the bio field should be empty then what is it? It's not described as it should be:

PersonService
- findById
  - Given an `id` for an existing record
    - And the record's `showBio` field is `false`
      - It should return a Person record with no bio

Do: Avoid overly vague terms

It's not uncommon to run into tests that have some overly vague terminology and say things like "it should return the appropriate value". What is the "appropriate value"? While we want to be general in our descriptions of our preconditions, inputs, and expectations, we still don't want to leave the reader guessing. This "appropriate value" gives no clue about what the unit being tested is supposed to do, just that whatever it is, it does it.

This can be a bit of a challenge, as you also don't want to be overly specific or too verbose. You need to find the phrasing that is just descriptive enough to give the reader a good idea of what's going on. Ideally we should fully understand the behaviour of the unit from reading its test's output, but there are times that's not possible or reasonable. So instead we need to find phrasing that invites the reader to dig into the test code if they need more specifics. It should be rare that they need to do this.

For example, with this test:

PersonService
- findById
  - Given an `id` for an existing record
    - It should return a Person record for that `id` with all fields populated from the database

This is a bit vague, I'm not really sure what "all fields populated from the database" exactly entails. Like is the field lastName populated from the database column SURNAME? I don't know. I'll have to dig into the test code to find out. While that's inconvenient, if there's 20 fields in a Person record, leaving it kind of vague makes it much easier to both write and read these tests rather than describing how every single field gets populated.

We also want to be careful here not to mask any behaviours. For example, if the Person returned has an age, which is computed based on the birthdate stored in the database, then that should have its own test somewhere and not just hidden amongst the assertions in the "It should return a Person record for that id with all fields populated from the database" test.

Conclusion

This post has all been about the organization and verbiage around tests. This is important because it not only helps the reader of these tests, but it can help organize your thoughts about how to actually write the test code. This advice is all derived from my own experiences; I hope you find it useful.

Enhancing the Developer Experience of Testing

Paul Mooney — Sun, 08 Dec 2024 01:13:24 +0000

One overlooked aspect of testing is the developer's experience when running tests. Particularly the next developer's experience, whether that be a different person, or future you who has just come back to some area of the code that you have not worked with in awhile. It may also be current you practicing Test Driven Development (TDD), and you need to organize your thoughts around what and how to test. Many times we run these tests, we can be confident that nothing broke but the test output is incomprehensible! We aren't really sure what's supposed to be happening, or if it's safe to change anything. Or worse, something breaks, and we're not sure what the test was supposed to be doing, to get back to a working state. Let's see if we can fix that!

Prioritizing the Test Output

One of the pillars of writing good tests is to make the test's output look good for human eyes. We want it to be self affirming documentation of the expected behaviours of the various units of the system, as well as the behaviours of the system in general. This may seem obvious, but the statement I'm trying to make here is that the test code itself is not good enough to explain what is happening. It's not good enough for a couple of reasons. The first being, that it significantly slows developers down having to read the test code. The second is that the test code does not express meaning and intentions. Looking at the sample inputs and outputs that the test uses is not enough to explain what's going on.

So how do we know if we have good output? We just look at it. Various IDEs and other test runners usually provide some kind of test output report, this is what we're aiming to make look good.

Example IntelliJ Test Run Output

Example VSCode Test Explorer Output

The rest of this post will cover tips for how to make this output look good.

Human Readable Test Descriptions

Depending on your test framework, this is may be a no-brainer. NodeJS testing frameworks, RSpec, and most styles of Kotest all have a required space for you to write real, human-readable sentences right into the test and their test contexts. This makes their output quite legible.

describe('MyClassUnderTest', () => {
  test('Given an `id` for an existing record, when `deleteById` is called, then the record is deleted from the DB', () => {
    // test code here
  });
});

The above results in nice output like this:

Example NodeJS Test output

Other method based test frameworks like JUnit and TestNG don't require this, and it's led to these awful conventions of writing out descriptions of tests as the method name with mixes of camel and snake case. Eg,

@Test
void givenAnIdForExistingRecord_whenDeleteByIdCalled_thenRecordIsDeletedFromDB() {
    // test code here
}

This is barely human-readable. It's hard to pick out the methods and inputs involved from the rest of the test verbiage. When viewing many of these test methods together, they're difficult to pick out from each other. It's an eyesore. Nowhere in life do we need to read sentences that look like this, so we shouldn't need to read tests like this either.

Fortunately these test frameworks usually have a way to add a description to the test. For example, In JUnit, you can use the @DisplayName annotation to write normal sentences. Or TestNG has @Test(name="...some name... "). If we're using this kind of testing framework then we should ALWAYS use these kinds of annotations. And then it doesn't really matter what the method name is.

An improved example of the test above:

@Test
@DisplayName("Given an `id` for an existing record, when `deleteById` is called, then the record is deleted from the DB")
void testDeleteById1() {
    // test code here
}

This will give us a nice human readable output in the test report, just like the NodeJS Test above.

Structured Testing

Another tool that can help make our test output look good is to take advantage of the test frameworks ability to group tests within a nesting of "contexts". This helps us create reusability in the test code, but more importantly in the test sentence structure. Imagine we have these 3 [flat] tests for a REST endpoint:

Given an `id` for an existing record, when calling the `/api/record/{id}` endpoint, it should return the record for that `id`
Given an `id` for an existing record, when calling the `/api/record/{id}` endpoint, it should return a 200 response code
Given an `id` for an existing record, and the `Accept` header is `application/xml`, when calling the `/api/record/{id}` endpoint, it should return the record for that `id` in XML format

In this example there are two problems: We're repeating the same setup over and over again, and the test descriptions are becoming verbose run-on sentences. Structured testing can help solve this problem. We can group these tests into nested contexts like so:

Given an `id` for an existing record
- When calling the `/api/record/{id}` endpoint
  - It should return the record for that `id`
  - It should return a 200 response code
- And the Accept header is application/xml
  - When calling the `/api/record/{id}` endpoint
    - It should return the record for that `id` in XML format

Resulting in the following output:

Notice the expand/collapse controls that can help narrow down on the specific cases you may be interested in

This definitely looks cleaner. One could argue that this deviates from a natural language sentence structure, or that there's extra cognitive load in having to piece together the full declaration of a single test from its setup, to execution, to expectation. This might be true, but in the first example with the flattened test structure we also have an arguably poor sentence structure given that they verge on run on sentences. And there may be just as much mental load grouping the tests together which share the same contexts. So in that sense it's a wash. But from a visualization, and organization standpoint, providing a structure like this is the clear winner.

In JUnit this can be accomplished using the @Nested annotation:

@DisplayName("REST Endpoint tests")
class RestEndpointTest {

  @DisplayName("Given a valid `id`")
  @Nested
  class GivenValidId {

    @DisplayName("When calling the `/api/record/{id}` endpoint")
    @Nested
    class WhenCallingRecordEndpoint {

      @DisplayName("It should return the record for that `id`")
      @Test
      void testRecordReturned() {
        // test code here
      }

      @DisplayName("It should return a 200 response code")
      @Test
      void test200Response() {
        // test code here
      }

      // ... other tests
    }
  }
}

Structured testing isn't always mandatory. Sometimes a unit is simple enough that its tests can be written out in a flat list. But more often then not, I find that for the sake of readability and organization, it's necessary to introduce some kind of structure.

Conclusion

By focusing on clear and comprehensible test output, we can ensure that both current and future developers can easily understand the purpose and results of tests. This not only aids in debugging and maintaining the code but also increases confidence in making changes and improvements. Well-written tests serve as valuable documentation, making the development process smoother and more reliable.

See more in my next post

Ditching Docker Compose for Kubernetes

Paul Mooney — Tue, 06 Apr 2021 22:05:49 +0000

When developing locally I usually incorporate Docker Compose into my local development workflow: Bringing up supporting containers needed to run databases, reverse proxies, other applications, or just to see how the container I'm developing works. Given that Docker Desktop comes with a single node Kubernetes (K8s) cluster and I usually end up deploying my containers to a Kubernetes cluster, I wanted to figure out if I can switch from Docker-Compose to Kubernetes for local development. It's also a good way to work the kinks out of Kubernetes manifests or Helm charts without disrupting any shared environments.

There are five things I need to be able to do in order to replace Docker-Compose with Kubernetes:

Build an image locally and run it on the Kubernetes.
Make changes to an app and redeploy on Kubernetes.
Make an easily accessible volume mount on a container in Kubernetes.
Have Kubernetes apps easily communicate with host OS apps.
Have host OS apps easily communicate with Kubernetes apps.

If you want to skip to how all of this works out here's the TL;DR otherwise keep reading.

Warning: The rest of this post assumes some familiarity with Docker and Kubernetes.

You can find sample applications that demonstrate all of this in this monorepo along with an explanation to get up and running.

Build an image locally and run it on the Kubernetes

With Docker Compose I can build an image and run it with just one simple command docker-compose up --build, assuming I have my docker-compose files setup. What's the analogue of this with Kubernetes? When I build an image, how can Kubernetes pull it? Do I need a local Docker Registry to push my image to?

The answer to that last question, luckily, is "No". When building an image locally using the standard docker build command docker build --tag my-image:local . the image is stored in docker's image cache. This is the same image cache Kubernetes will use because it's using the same docker instance. There are two things to note here:

The image name of a Kubernetes pod must exactly match the name given via the --tag parameter of the docker build command. In the example given it's my-image:local
The imagePullPolicy must be set to Never or IfNotPresent. It cannot be set to Always otherwise Kubernetes will attempt to pull the image from a remote registry like Docker Hub, and it would fail.

containers:
  - name: my-container
    image: "my-image:local"
    imagePullPolicy: Never

Container definitions would contain an `image` name that matches your build command and an `imagePullPolicy` that is not `Always`

That covers how to build and run an image locally.

Make Changes to an app and redeploy on Kubernetes

If I were making changes to the application or its image definition (ie, Dockerfile) and wanted to see it running in Docker Compose I would just run the command docker-compose up --build. For kubernetes we can rebuild the image docker build --tag my-image:local. That much is the same as the initial build but you will probably notice your changes aren't actually running in Kubernetes right away.

The problem is there's been no signal for Kubernetes to do anything after the image was built. The solution is to delete the pod the image was running in and recreate it. If you are running single unmanaged pod (which I think is unlikely) you would have to delete it and recreate it yourself from the pod definition yaml. If you're running a deployment or a statefulset you can either delete the pods and they will automatically be recreated for you, or you can scale down the replicas to 0 and then back up again:

Delete a pod: kubectl delete pod my-pod-xyz --force
Scale down kubectl scale deployment my-deployment --replicas=0 and then back up kubectl scale deployment my-deployment --replicas=3

Make an easily accessible volume mount on a container in Kubernetes

In Docker Compose, volumes can be fairly straightforward in that we can mount any file or subdirectory relative to the directory we are executing docker-compose from. That makes it easy to find, inspect and cleanup those files. But Kubernetes is not the same. It's not running from a project's folder like Docker Compose, it's already running on the Docker Desktop Virtual Machine somewhere. So if we defined a volume to mount into a container, where would the data for that volume live? It lives in the Docker Desktop Virtual Machine somewhere (unless we're running WSL 2). Luckily Docker Desktop has file sharing setup with the host OS so we can take advantage of this to do any inspection or cleanup of persistent data.

Going into the Docker Desktop dashboard under Settings/Preferences -> Resources -> File Sharing I can see and manage all of the file sharing that is available. Using this information I can create a hostPath Persistent Volume that my application can claim and use. In my example below I picked a path under /Users since that was already shared (on MacOS):

apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-volume
spec:
  storageClassName: my-volume-class
  accessModes:
  - ReadWriteMany
  capacity:
    storage: 1Gi
  hostPath:
    path: "/Users/Shared/my-volume"
    type: DirectoryOrCreate

This volume obviously differs from what you would use in your dev or prod Kubernetes clusters, so I recommend having a folder of "local" persistent volume definition yamls like this that can be reused by team mates (or your future self) to populate their Kubernetes with. Unfortunately you may have no choice but to have different persistent volume yamls for both Mac and Windows if your team uses a mix of those.

One last thing - if you ever delete the claim to this Persistent Volume, you must delete and recreate the Persistent Volume too, if you ever want to run your application again in the future. This is not unique to local Kubernetes development.

Have Kubernetes apps easily communicate with host OS apps

Often times I will be working on an application in the host OS. Most of my primary development is done here, as you get the advantages of automatic rebuilds and IDE tooling, etc. There will be other applications that I'd like to run in Kubernetes that can talk to this application on the host OS. For example, I may have a reverse proxy like nginx running in Kubernetes that needs to serve up my host OS application. This is super easy, and done exactly the same as we would do it with just Docker or Docker Compose: with the host.docker.internal DNS name.

An example of my nginx config running on kubernetes that reverse proxies my app running on the host port 4200:

server {
    listen       80;
    server_name  localhost;

    location / {
        proxy_pass http://host.docker.internal:4200;
    }

Have host OS apps easily communicate with Kubernetes apps

Whether I'm developing an application on the host OS that communicates with an application on Kubernetes, or if I want to access the application on Kubernetes in a web browser, or some kind of client, the application needs to be exposed. There are two ways to do this. The first, and not my recommended approach, is to use kubectl port-forwarding. I don't like this approach because you need to be re-run this command whenever you restart your cluster for every service that needs to be exposed. My preferred approach is to use a NodePort service.

A NodePort exposes a port on the kubernetes node that you can access your application through and in Docker Desktop that exposes the port on your host OS.

So I can create a service for my application like this:

apiVersion: v1
kind: Service
metadata:
  name: my-app-service
  labels:
    app.kubernetes.io/name: my-app-service
spec:
  type: NodePort
  ports:
    - port: 3000
      targetPort: http
      protocol: TCP
      name: http
      nodePort: 30001 
  selector:
    app.kubernetes.io/name: my-app

And I can access my application at localhost:30001!

I prefer to define my nodePort for predictability of the port, but you can leave it empty for Kubernetes to decide what it should be. Then there's less chance of a collision for an already occupied port.

Chances are the application's service might be a ClusterIP or LoadBalancer type when deployed to other Kubernetes clusters, or that the nodePort will have a different value in those clusters. You can get around this by templating your service definition in Helm, and having different service configurations for your local Kubernetes versus other Kubernetes clusters.

Helm

Without Helm, or similar tools, using a local Kubernetes cluster for development is pointless beyond just experimentation purposes. We want to use the local Kubernetes cluster so that our running applications will mirror shared environments, like production as closely as possible. Helm lets us accomplish this by allowing us to template out our kubernetes manifests, and abstract out only the necessary environmental differences into values files.

When you're using Helm you'll be creating values files for every environment. I recommend creating values files for local clusters as well that can be shared with the team. You can even create personal "overrides" values files that you can use to change some minor configurations for your own purposes (just be sure to .gitignore them). Helm lets you chain these files together, and gives precendence to the rightmost file. E.g., helm upgrade my-app ./my-app -f values-local.yaml -f .values-override.yaml.

Another benefit of Helm is in it's package management. If your application requires another team's application up and running, they can publish their Helm chart to a remote repository like a ChartMuseum. You can then install their application into your Kubernetes by naming that remote chart combined with a local values file. E.g., helm install other-teams-app https://charts.mycompany.com/other-teams-app-1.2.3.tgz -f values-other-teams-app.yaml. This is convenient because it means you don't have to checkout their project and dig through it for their helm charts to get up and running - all you need to supply is your own values file.

Scripting

Working with kubernetes, and then layering in extra tools like Helm, there are a lot of commands to get to know. Most of your team will probably need some kind of containerized apps running locally, but it can be a high bar to expect them to know all of the docker and kubectl and helm commands. You will also want to take the things that are done often and condense them into some simpler scripts for your own convenience. Things like:

Build and Install your app on the kubernetes cluster:

docker build --tag myimage:local \
&& kubectl apply -f my-volume.yaml \
&& helm install my-app ./my-app -f values-local.yaml

Build and restart your app:

docker build --tag myimage:local \
&& kubectl scale deployment my-app --replicas=0 \
&& kubectl scale deployment my-app --replicas=3

Update your configuration:

helm upgrade my-app ./my-app -f values-local.yaml

Install another team or organization's app:

helm install other-teams-app https://charts.mycompany.com/other-teams-app-1.2.3.tgz -f values-other-teams-app.yaml

Clean up

helm uninstall my-app \
&& kubectl delete -f my-volume.yaml
&& rm -Rf /path/to/my-volume

You can script this however you like, whether it be in bash, Makefile, npm scripts, Gradle tasks. Use whatever suits your team best.

Comparing to Docker Compose

Using Docker Compose for local development is undoubtedly more convenient than Kubernetes. For the most part you only need to be familiar with two commands to build, run, re-build and re-run, and shutdown your applications in docker: docker-compose up --build, and docker-compose down. For volumes, Docker Compose lets you mount a directory relative to where you execute docker-compose from and in a way that works across platforms. Docker Compose is also safer - there's no chance you're going to accidentally docker-compose up a mid-developed image into production!

Docker Compose has the disadvantage that it's a duplication of effort to recreate an analogue of your Kubernetes manifests into docker-compose files. Considering the extra configurations, volume definitions, and scripting that needs to be added for local Kubernetes development, this is probably a negligable difference.

Kubernetes, on the other hand, more accurately represents what you will be deploying into shared Kubernetes clusters or production. Using a tool like Helm gives us package manager-like features of installing externally developed manifest or dependencies without having to redefine them in your local repository.

Using Kubernetes requires a good familiarity with Kubernetes and its surrounding tools, or extra scripting to hide these details. These tools like kubectl and helm rely on a context which could be set to the wrong Kubernetes cluster, which would cause unwanted trouble! I recommend putting safeguards in place like setting up RBAC where possible in the shared or production Kubernetes clusters where possible. Or, work within a namespace locally that does not exist in other clusters.

Final Thoughts

It's possible to replace Docker Compose with Kubernetes for local development, but for the added complexity and trade-offs it may be worth using both. For most local development, Docker Compose is probably good enough, and much simpler. Using a local Kubernetes cluster is a step up in terms of complexity and effort so it is up to you if you want to take that on. It is definitely worth it for Helm Chart / Manifest development or situations where you absolutely must re-create a part of your deployment architecture.

TL;DR

Building and running an image on Kubernetes works because Kubernetes will pull from the same shared image cache you built from, just make sure your pull policy is not 'Always'.

To re-build an image and re-run, just delete the old pods running the old image. Newly created pods will come up with the new image.

Docker Deskop's file sharing locations can be found and configured in the Preferences/Settings. A Persistent Volume can be created with a hostPath to one of those locations.

Applications running on Kubernetes can access applications on the host OS via the host.docker.internal DNS name.

Applications running on Kubernetes can be accessed by setting up kubectl port forwarding and then accessed using localhost:{forwardedPort}. Or, even better, make the Application's service a nodePort service and access using localhost:{nodePort}.

Use Helm. Simplify the common tasks via scripting. Maybe don't ditch Docker Compose completely.

Baking Configuration into your Angular App Pie

Paul Mooney — Fri, 25 Oct 2019 14:50:35 +0000

In this post I'm going to talk about some of the best ways to get your configurations to your Angular app. Just note, this isn't a post about Angular framework level configurations, this is about how the features you're developing receive your configurable values.

Where's my Backend API Server?

Most SPAs need a backend API server, so when development starts there's the question of "how do I tell my app where my API server is?" The answer is that you don't. Your app should assume the API server is served from the same host as the app itself. It will only use relative URLs (in this case "relative" means no protocol, host, or port specified) to call the API server.

For example:

@Injectable({
  providedIn: 'root'
})
export class ServerTimeService {

  constructor(private httpClient: HttpClient) { }

  getTime(): Observable<string> {

    // Calls relative path `/api`. No host in the URL here.
    return this.httpClient.get('/api/servertime')
      .pipe(map((data: any) => data.servertime));
  }
}

This is nice and clean, and avoids CORS complications and issues.

How do we achieve this? With Reverse Proxies.

Let's look at the scenario where your backend API server sits at http://myinternalhost:8080/api and we want the app to be able to make requests only to paths starting with /api. Here's how you can configure reverse proxies for development and when deployed:

Proxy Server during Development

When a project is generated using Angular CLI it uses webpack (at least at the time of writing this) which includes a dev server that hosts the app and watches for changes when we run ng serve (or npm start if you're using the Angular CLI defaults). This server also includes a reverse proxy which can be configured via proxy.conf.js or proxy.conf.json file. You can read more about it in the Angular CLI repo. I prefer the 'js' version of the file since it gives us more flexibility.

Given our example scenario for getting requests from the relative path /api to the absolute path http://myinternalhost:8080/api, we can setup our proxy.conf.js in the root of our project folder like so:

const PROXY_CONFIG = {
  '/api': {
    'target': 'http://myinternalhost:8080',
    'secure': false,
    'logLevel': 'debug',
    'changeOrigin': true
  }
};

module.exports = PROXY_CONFIG;

And alter the "start" npm script to tell it to use the proxy.conf.js file:

"start":"ng serve --proxy-config proxy.conf.js"

Of course it would be better if the target value was not hardcoded to a specific server in a file that we're going to be checking into version control, so we can use an environment variable instead. Let's make the above snippet better:

const PROXY_CONFIG = {
  '/api': {
    'target': process.env.API_SERVER,
    'secure': false,
    'logLevel': 'debug',
    'changeOrigin': true
  }
};

module.exports = PROXY_CONFIG;

The environment variable can be passed via commandline API_SERVER=http://myinternalhost:8080 npm start.

Reverse Proxy when Deployed

When you're deploying your application, you won't have webpack's dev-server to use as a reverse proxy so you'll need a separate standalone one. Popular options for reverse proxies are webservers like NGINX or Apache HTTP Server. These serve other purposes as well such as handling HTTPS, load balancing, or if you're not using Server Side Rendering (https://angular.io/guide/universal) they can be used to serve your Angular app's static assets. So it's likely you'll need one of these anyways.

The key idea here is that the reverse proxy is the single point for traffic to and from the browser for both requests to your app, and requests to the API server.

Here's a snippet of nginx configuration that forwards traffic to your app, and to our http://myinternalhost:8080 API server:

server {
  listen       80;
  server_name  localhost;

  # Reverse proxy all traffic to the Angular app
  location / {
    proxy_pass http://localhost:4000;
  }

  # Reverse proxy all traffic starting with `/api` to the backend API server
  location /api {
    proxy_pass http://myinternalhost:8080;
  }
}

NGINX itself can be configured to use environment variables as mentioned on its Docker Hub page.

What about Server Side Rendering?

In server side rendering (SSR), your Angular app's code is running on the server similar to how it would run in the browser, complete with the API calls it needs to make but with a few exceptions. One of those exceptions is that relative URLs are meaningless on the server. Servers want absolute URLs. So it turns out that our app does need that absolute URL to the backend API afterall.

Luckily, when rendering on the server, we're not in a context where we need to worry about CORS, and we are in a context where your code can read environment variables. So our example HttpClient request can be altered to look like this:

@Injectable({
  providedIn: 'root'
})
export class ServerTimeService {

  constructor(private httpClient: HttpClient, @Inject(PLATFORM_ID) private platformId) { }

  getTime(): Observable<string> {

    const path = '/api/servertime';

    // Make URL absolute only if on the server
    const url = isPlatformServer(this.platformId) ? process.env.API_SERVER + path : path;

    return this.httpClient.get(url)
      .pipe(map((data: any) => data.servertime));
  }
}

This doesn't mean we can ditch the reverse proxy setup, we still need that when the app is running in the browser. This is just an extra consideration to make when leveraging SSR.

Note:
For this to compile, you will also need to install node types via npm i -D @types/node and then add "node" to the compilerOptions.types array of the the tsconfig.app.json file.

Environment Variables vs Environment.ts

Let's imagine another scenario where your Angular app has a typeahead search in it, and it needs a debounce time to decide when the user has stopped typing and it's safe to make an API call. Kind of like this article describes. We want to make the debounce time configurable.

It'd be tempting to use the Environment.ts and Environment.prod.ts as the configuration point for this debounce time, but you probably shouldn't. Actually, just don't. It's a violation of the third factor of The Twelve-Factor App. The short of it is that if you are using a version controlled file in your app to store configuration then your app has to be rebuilt and redeployed just to affect a configuration change. Sounds like hardcoding not configuration. This is fine for the world of Infrastructure as Code and GitOps but it is not ideal for applications.

In general you probably won't use the Environment.ts files much unless there are different modes your application needs to be built in. If you find yourself writing Environment.staging.ts or Environment.qa.ts files, you're doing it wrong.

So how do you configure this 'debounce' time in the app? With Environment Variables! How do we use environment variables in an app that mostly runs in the browser? Serve them via API server.

There are multiple ways to do this. We'll take the approach that we're using a purpose built "Config" REST endpoint just for this Angular app.

Sending environment variables during Development

A quick and easy way to create a Config REST endpoint to use during development is to leverage the webpack's proxy server. We can create a faux backend inside the proxy.conf.js file like so:

const PROXY_CONFIG = {
    '/config': {
        'bypass': function (req, res, proxyOptions) {
            switch (req.url) {
                case '/config':

                // Send an map of config values
                res.end(JSON.stringify({
                    DEBOUNCE_TIME: process.env.DEBOUNCE_TIME || 500 // Read from environment or default to 500
                    ... // Other config values here
                }));
                return true;
            }
        }
    }
    ... // Other proxy settings
};

export PROXY_CONFIG;

From there it's just a matter of making a call to this /config endpoint just like any other endpoint.

this.httpClient.get('/config');

You can start your development server with an environment variable like so DEBOUNCE_TIME=300 npm start

Sending environment variables when Deployed

For this, you'd probably just have to build a separate server, perhaps using something like Express. However, if you're leveraging server side rendering then you'll probably already have a server in the form of the server.ts file (likely generated by a schematic like @nguniversal/express-engine). This is a good place to add a little extra functionality to serve up configuration read from server side environment variables in a similar manner to how it's done in the proxy.conf.js example.

Add the following to the server.ts file used for SSR:

app.get('/config', (req, res) => {
  res.status(200).send({
    DEBOUNCE_TIME: process.env.DEBOUNCE_TIME || 500 // Read from environment or default to 500
    ... // Other config values here
  });
});

During server side rendering, when the code is executing on the server you won't necessarily need to call this API (though you could) since you can just directly access the environment variables from within code. To keep things simple, it's probably best to hide how all of your configuration values are retreived behind a single "Config" Angular service:

@Injectable({
  providedIn: 'root'
})
export class ConfigService {

  constructor(private httpClient: HttpClient, @Inject(PLATFORM_ID) private platformId) {}

  getConfig(): Observable<any> {

    // Direct, speedy access to environment variables when on server.
    if (isPlatformServer(this.platformId)) {
      return of({
        DEBOUNCE_TIME: process.env.DEBOUNCE_TIME
      });
    }

    // Otherwise from the brwoser call the `/config` API.
    return this.httpClient.get('/config');
  }
}

Avoid Depending on Transferstate to Transport your Configuration

When using server side rendering, It may be tempting to avoid setting up a "Config" REST service like the one above and just leverage transfer state to gather values from environment variables on the server and send them to the client. This may or may not work for you but if you're enabling Progressive Web App then a good deal of the time server side rendering won't even come into play since the app is rendered from javascript and other assets cached in the browser, bypassing SSR completely. Since there's no SSR happening in a PWA, there's no transferstate, so it's not a good idea to make it the sole medium for transporting configuration values.

The right time to call your Configuration API endpoint

There are different situations where you may need to call a configuration API in the lifecycle of your app. The earlier it's called the better, but it can also get more complex. These are some of the places where you could call the config API from:

On Demand, maybe leveraging a behaviour subject

This is like the title says, call it only when you need. This is ideal when you require configuration values for some of the views or components you're developing. You can call the config API from one of the lifecycle hooks of your components.

Perhaps use something like a Replay Subject to prevent multiple or competing calls going to the config API at once and to cache your config values.

From the Angular APP_INITIALIZER hook

An APP_INITIALIZER function gets called during Angular's startup. This is likely the place you want to execute your config retrieval if some of those configurations are central to the app. Like say, if they relate to how you might configure a global aspect of the app such as internationalization, or possibly affect some change in routing, or maybe if you prefer the app to just fail fast when there is an invalid configuration instead of finding out later when the config value is finally used.

You can read more about the APP_INITIALIZER.

Again, it's probably good to wrap the config API call in a Replay Subject just so that its results can be cached for later.

Before Angular Starts

This is the earliest time to retrieve configuration: before anything Angular begins to bootstrap. This is good for situations where you need these values even earlier than APP_INITIALIZER allows. Examples might be if you need them to configure a custom HttpInterceptor or if you have a special Error Handler that needs an API key to a logging service.

The place to make this call is in the main.ts file. On return, store the results in local storage so that they can be retrieved when needed. Note that angular service such as HttpClient won't be available so the browser basics like fetch or XMLHttpRequest will have to do.

Example main.ts file:

if (environment.production) {
  enableProdMode();
}

document.addEventListener('DOMContentLoaded', async () => {

  const response = await fetch('/config');
  if (response.status === 200) {
    const result = await response.text();
    localStorage.setItem('config', result);
    platformBrowserDynamic().bootstrapModule(AppModule)
    .catch(err => console.error(err));
  }
});

.env Files

One last bonus tidbit of information: It can be tedious to setup environment variables in the command line when developing. Especially if there's a lot of them. The answer to this problem is the .env file.

It's a simple file where each line is an environment variable assignment in the format VARIABLE_NAME=value. And it supports comments!

The .env file works out of the box in some runtimes, like for docker-compose, but doesn't work out of the box in node.js. You'll need to install the library dotenv as a dev dependency: npm i -D dotenv and then have it loaded up.

To load it in your proxy.conf.js, just add the following line to the top of the file.

require('dotenv').config();

To load it for SSR, alter the npm script called "serve:ssr" to the following:

"serve:ssr":"node -r dotenv/config dist/server"

Finally be sure .env file entry is added to your .gitignore file. This file is for your local development, it would be really annoying if your settings were regularly and unexpectedly clobbered by someone else's changes whenever you're pulling the latest.

Wrapping up

To summarize what we've learned here about getting configuration to your Angular app:

Use a reverse-proxy to "host" your Angular app and Backend APIs from the same server, don't try to configure where that backend API is in your Angular app.
You may have very frontend specific configurations that aren't appropriate to serve from your existing business oriented backend APIs. If so, create a simple config API by hijacking your webpack dev-server during development, and by hijacking your server.ts file if your're using SSR.
Environment Variables are a good medium to set config values from the server side.
You probably won't need Environment.ts files as much as you think.
There are various times to call your config API. Pick one.
Don't forget the .env files

Hope this was a good read. Not all of it will be appropriate for your project, but I'm sure some of it will be.

Sample project source, and this blog in the works, can be found here

Tips for Your Site's Iconography

Paul Mooney — Fri, 19 Apr 2019 22:22:47 +0000

Choosing your Iconography Approach

At the beginning of every web app's development there is the question of iconography. Which icon library do we use? There's lots to choose from: Font Awesome, Fort Awesome, The Noun Project, Material Design Icons, IcoMoon. Generally this choice is up to the designer (if you have one), but as a developer your choice is going to be between using SVG's as icons or Icon fonts. There are a number of pros and cons to SVGs vs Icon fonts that you can find better articles on such as this one. Here's my quick comparison:

SVG Pros:

SVGs are generally sharper than fonts.
You are able to specifically pick out and download only the SVGs you want.
Aligning SVGs is simpler and more predictable because they aren't susceptible to weird font rules such as line height which can add unwanted white space.

SVG Cons:

Coloring external SVGs can't be done in CSS in a way that's supported in all browsers.

Icon Font Pros:

Can be colored easily by setting the containing elements 'color' property.

Icon Font Cons:

While the font is loading up, or if it fails to load, you will see squares like this □ or the ligature of the icon. E.g., you will see the text 'get_app' where the icon should appear.
Fonts Icons can be trickier to align due to font rules applied to them (mentioned above).
More difficult to hide from screen readers.

With that brief comparison, it's up to you to decide but I'm going with SVG icons, at least for the rest of this article ;)

Inline SVGs vs External SVGs

Inline SVGs, where the SVG definition is part of the html document, versus External SVGs is the next choice to make.

Mostly this choice comes down to your school of thought on iconography: is iconography part of the style or theme of your web application, aimed to be fully controlled by CSS? Or is iconography a part of the web app's makeup, meaning your html has additional markup just to support icons?

If your school of thought is the former, then congratulations you've made the right choice! Or at least you have from a purists point of view, and definitely if you plan on treating the theme of your web application as a separated concern.

Inline SVGs are still great in the sense that you get the flexibility to use CSS to select and color their elements. Or that they can be animated, but a lot of iconography design on the web today is static and monochrome. If you don't need multicolored, animated icons then you don't need to pollute your HTML with extra inline SVG markup and blur the lines between theme and content structure.

Working with External SVG Icons

Generally the best way to work with SVG Icons is to load them in as background images. This way screen readers will avoid them and we don't need to add extra ARIA attributes to say that they're presentation only.

Why are icons presentation only you say? Well an icon itself doesn't have any meaning to a screenreader. It's something that's meant to convey [extra] meaning to a sighted user in a shorthand way without having to consume precious screen real estate with text. So it's useful to have on a button for example, but that button should already have ARIA attributes to describe its meaning.

SVG Icon as a Background Image

For things like buttons that we're adding SVG icons to, we'll start by setting up a reusable mixin that takes as an argument the path to the icon, and the size of the icon (assuming square dimensions). This mixin will cover all of the properties we'll typically apply to every icon:

@mixin createIconStyle($path-to-svg, $icon-size) {
  background-image: url('~/../src/#{$path-to-svg}');
  background-repeat: no-repeat;
  background-size: $icon-size auto;
  background-position: center;
  height: $icon-size;
  width: $icon-size;
}

Then we can setup a specific css class for an icon using that mixin:

.download-icon {
  @include createIconStyle('/assets/baseline-get_app-24px.svg', 2rem)
}

Note: My example is using webpack which is why it has the weird '~/..' prepended to the icon path.

Now we're free to add this style class to a download button:

<button class="btn btn-primary download-icon" aria-label="Download the thing"></button>

The download button with an icon. Note: I changed my icon to white for this example because the default black on blue looked bad.

Data URLs for Icons

Data URLs are a nice way to bundle your CSS and icons together into the same HTTP request, though there are some good and some vague reasons why this is bad. If handled properly, this can still be a good thing. Especially if you're not using HTTP/2 yet. The main concern is to avoid duplicate data URLs otherwise it can easily bloat your CSS assets fast.

With Sass we can avoid this duplication pretty easily with inheritance. Let's say we want to make a .download-icon-large which is twice as big as the original, then we just inherit the original and override its properties. It's possible to make a mixin of this if it's a common enough occurrence:

@mixin overrideIconSize($extends-class, $icon-size) {
  @extend .#{$extends-class};
  background-size: $icon-size auto;
  height: $icon-size;
  width: $icon-size;
}

.download-icon-large {
  @include overrideIconSize(download-icon, 3rem);
}

This will result in CSS which only includes that data URL once:

.download-icon, .download-icon-large {
  background-image: url('data:image/svg+xml;utf8,<svg ...> ... </svg>')
}

SVG Icon as a Side Image

A popular use for icons is to put them beside some text for a link or a button to give a bit more of a hint to what will happen when it's clicked. Some examples might include a down arrow beside a menu item to hint that it will expand into a submenu. Or a download icon beside the link to a file to hint that it will download instead of open.

These are situations where it's common for developers to add extra markup, usually in the form of an <i> tag or worse an <img> tag. Yuck! Let's keep it all in the CSS please! Remember icons are presentation only, no need for extra markup in our content. That's also not the purpose of an <i> tag and <img> tags download images regardless of visibility which is inefficient, and looks bad with its display of the browser's broken image icon if the download fails.

So without adding extra markup, the solution is to use a pseudo element. Either before or after will do. Which you choose depends on the position of the icon (before for icon on the left, after for icon on the right) but it doesn't matter since position can be controlled in other ways.

For this example, let's say we have a download link with the download icon to the right of it like this:

Expanding on the .download-icon example above, let's create a Sass mixin to create a pseudo element that inherits from the .download-icon:

@mixin existingIconAfter($extends-class, $icon-size) {
  display: inline-flex;
  align-items: center;
  &:after {
    @extend .#{$extends-class};
    vertical-align: middle;
    display: inline-block;
    background-size: $icon-size auto;
    height: $icon-size;
    width: $icon-size;
    content: '';
  }
}

We set the display property to inline-flex and align-items to center so that we can take advantage of Flexbox to vertically align the icon next to the text

Next we create a new class using this mixin and the existing .download-icon class:

.download-icon-after {
  @include existingIconAfter(download-icon, 1.5rem);
}

Then we can add setup our download link:

<a href="..." class="download-icon-after">
  <span>Download</span>
</a>

What's with this <span> tag wrapping the 'Download' text? In this example it's for positioning. On some browsers Flexbox won't recognize text as an item to apply positioning and alignment to. Doesn't this go against "don't add extra markup for icons" approach we're trying to achieve? Maybe, but regardless we'd still want a way to apply styling rules to the separate elements that ultimately make up this link anyways so we can use that span to target the text portion of the link. For example, where horizontal space is limited, we may want to apply rules such that our text will wrap without wrapping the icon too.

If we had a separate theme in our webapp we'd be able to have different looks for this link such as flanking the text with icons on both sides, or removing the icons completely without having to change the structure of the html.

Coloring External SVG Icons using CSS Mask

Applying color to External SVG Icons is unfortunately its weak point depending on your needs. If you don't need to support Internet Explorer, then there's a great solution here using CSS mask which we'll follow. If this doesn't work for you, then one option might be to just create different color variants of the icon SVG files by opening them up and changing their fill color and saving them, or some automated variant of this.

Back to the CSS mask option. The gist of it is that a mask decides the shape of an element, and the only visible parts of that element are inside that mask shape. So if we set an element's background color to blue, and apply a star shaped mask to it then we end up with a blue star. Unfortunately, because the rest of the element outside the star became invisible we lose the elements important details. This means we can't use an SVG mask as a direct replacement for a background image on say a button because the intended outline, background, hover, etc, whatever falls outside the mask shape is invisible. That's no good.

We can solve this using pseudo elements. If we want a star shape icon on a button, we can apply the mask to a pseudo element within the button instead of on the button itself.

Similar to how we have the createIconStyle mixin above, we can start out with a createMaskStyle

@mixin createMaskStyle($path-to-svg, $icon-size) {
  mask: url('~/../src/#{$path-to-svg}');
  mask-repeat: no-repeat;
  mask-size: $icon-size auto;
  mask-position: center;
  height: $icon-size;
  width: $icon-size;
}

And create a css class leveraging that mixin

.download-mask {
  @include createMaskStyle('/assets/baseline-get_app-24px.svg', 2rem)
}

But, like we talked about, we usually don't want to apply a style like this directly to an element. We want to apply it to a pseudo element. Similar to our createIconAfter mixin we will create another mixin to leverage this style class:

@mixin existingMaskAfter($extends-class, $icon-size) {
  display: inline-flex;
  align-items: center;
  &:after {
    @extend .#{$extends-class};
    vertical-align: middle;
    display: inline-block;
    mask-size: $icon-size auto;
    height: $icon-size;
    width: $icon-size;
    content: '';
  }
}

Then we can leverage that mixin to create style classes we can apply to any element we want to have apply this pseudo element to (and maybe readjust its size if we want):

.download-mask-after {
  @include existingMaskAfter(download-mask, 1.5rem);
}

Now you can add an icon to a button like so:

<button class="download-mask-after" aria-label="Download Button"></button>

But wait... I don't see an icon! That's because the pseudo element has nothing to show, so it looks invisible. If we give the pseudo element some color, the icon appears:

button.download-mask-after:after {
  background-color: red;
}

Now we're coloring SVG icons using CSS!

Hurray we have a red download icon to make our eyes sore!

SVG Sprites

In the Data URLs section above we talked about using data URLs as a way to minimize http requests, but that it has drawbacks such as needing to find ways to reduce data URL duplication. If you want to use the CSS Mask technique mentioned above to color your icons, and you have an autoprefixer, then chances are you're going get duplicate data URLs anyways since for every mask property with a data URL there will be at least a -webkit-mask property with the same data URL. An alternative that also helps minimize http requests is to use SVG Sprites.

SVG Sprites is just delivering all of your SVG icons in a single SVG file. Unlike traditional sprite files where you need to know the position of the image inside the file and do some magic to align background positioning over that image, with SVG sprites you can simply reference the image through the URL by its identifier.

For example, I've created a 'sprites.svg' file and inside of it I have, among other icons, my 'getapp' (download) icon which is identified by the name 'getapp' and referenced as a URL fragment like so:

.download-mask-sprite {
  @include createMaskStyle('/assets/sprites.svg#getapp', 2rem)
}

There is some trickery around getting the sprites.svg structured to work for this purpose. Here's an example:

<svg version="1.1"
  xmlns="http://www.w3.org/2000/svg"
  xmlns:xlink="http://www.w3.org/1999/xlink" class="sprites">

  <defs>
    <style>
    svg.sprites {
      display: inline;
    }
    svg {
      display: none;
    }
    svg:target {
      display: inline;
    }
    </style>
  </defs>

  <!-- Here I've pasted in the original SVG, and wrapped it's path elements in a g tag-->
  <svg viewBox="0 0 24 24" id="getapp">
    <g>
      <path d="..."></path>
      <path d="..." fill="none"></path>
    </g>
  </svg>

  <!-- Another SVG in the same file referenced via #setttings URL fragment -->
  <svg viewBox="0 0 20 20" id="settings">
    <g>
      <path fill="none" d="..."></path>
      <path d="..."></path>
    </g>
  </svg>
</svg>

You can read more about it in this article

From here it's up to you to decide how you want to incorporate this into your project. Do you build the sprite file yourself? Alternatively you can use a command line tool, like svg-sprite, to build the sprite file for you. Another option is you can try to make it a part of your project's build.

Coloring SVG Sprites

If the CSS Mask option is not for you then you can change the fill color of the icons inside the sprite file. We can gain an efficiency here where each icon (or rather its paths) is only defined once and then can be referenced multiple times taking advantage of SVG's <use> tag and applying different fill colors.

First all of the icon definitions (the <g> tags) can be moved into the <defs> section of the SVG and given identifiers. Then all of the different color variants of each icon are created using <svg> tags with <use> tags inside them and something to distinguish their color in a CSS selector like a CSS class name. Finally we can use CSS inside the <defs> section to set the fill color on the icons matching that selector.

Here's an iteration on the previous example:

<svg version="1.1"
  xmlns="http://www.w3.org/2000/svg"
  xmlns:xlink="http://www.w3.org/1999/xlink"
  class="sprites">

  <defs>
    <style>
    svg.sprites {
      display: inline;
    }
    svg {
      display: none;
    }
    svg:target {
      display: inline;
    }

    <!-- Make SVGs with 'blue' classnames blue -->
    svg.blue use {
      fill: blue;
    }

    </style>

    <!-- Icon definitions -->
    <g id="getapp_icon">
      <path d="..."></path>
      <path d="..." fill="none"></path>
    </g>

    <g id="settings_icon">
      <path fill="none" d="..."></path>
      <path d="..."></path>
    </g>

  </defs>

  <svg viewBox="0 0 24 24" id="getapp">
    <use xlink:href="#getapp_icon"/>
  </svg>

  <svg viewBox="0 0 20 20" id="settings">
    <use xlink:href="#settings_icon"/>
  </svg>

  <!-- Referenced via #getapp_blue. Will display a blue version of the #getapp icon -->
  <svg viewBox="0 0 24 24" class="blue" id="getapp_blue">
    <use xlink:href="#getapp_icon"/>
  </svg>

  <svg viewBox="0 0 20 20" class="blue" id="settings_blue">
    <use xlink:href="#settings_icon"/>
  </svg>

</svg>

Wrap up

There are a number of approaches to handling iconography in your web application, and even more reasons to choose different approaches (fonts or SVGs, inlined or external, data URLs or sprites, etc.). It all depends on your tastes, technical limitations and capabilities. The aim of this post is to guide you down a specific path of using SVG icons and outline reasons you might want to do it that way.

Tips for working with CSS Grids

Paul Mooney — Mon, 11 Mar 2019 03:03:43 +0000

Overview of Grids

A common practice in creating responsive layouts in web sites is to use what's referred to as a grid. Grids are tools used by designers and developers alike to help layout a page, align elements within the page and help define how much of the page those elements should take up for different screen sizes.

A very common implementation is the 12 column grid which is generally used to help line up elements horizontally along "columns". They are made up of three elements:

A grid container, which defines the left and right side boundaries (or the width) of your site.
A row, which lays out grid cells, and is responsible for wrapping the cells when they don't fit.
A cell, which takes up space in a row. Most of the time cells take up some multiple of one 12th of the width of the grid which is how we get the 12 column effect.

The magenta outline shows the borders of the grid container. The translucent bars represent the columns that cells will align to in the grid.

This is seen in some form or another in popular CSS frameworks such as Bootstrap or Foundation XY Grid, but can also be implemented in the native CSS-grid if your browser supports it. All of these systems are capable of creating other kinds of layouts beyond the 12 column grid, but it is still a popular choice to use them for this purpose.

Working with a grid system like this isn't all scotches and skittles, there are some things that are unintuitive or require some workarounds. Here are some tips for working with grids.

This post is going to focus on implementing tips in Bootstrap with Sass, though these concepts can apply to other systems.

Breakout Backgrounds

A common web site design feature is the have backgrounds that stretch out all the way to the left and right edges of the viewport while the main elements of your site remain constrained inside the grid. We'll refer to this as a "breakout background".

In this design, several sections of the page have backgrounds that break out beyond the grid container (outlined in magenta)

The naive solution to this is to create a full width div, give it a background styling, then put a grid container inside that div. Rinse and repeat this process for every section that requires a breakout background. This could work, but it's ugly. The structure of the html has been compromised to support styling, and it probably puts some tough limitations on how you code your site.

Our goal is that we want to have a single grid container for the whole site and still have these breakout backgrounds. We can achieve this purely in CSS without messing with our html structure.

The trick is that we use a pseudo element on the element we want to have a breakout background for and then assign some background styling to it. Then we stretch that pseudo element out to the edges of the viewport.

First we start with a class that has a pseudo element positioned behind the content of its parent on the Z-axis:

// Apply this class to elements which should have breakout backgrounds
.breakout-background {
  position: relative;
  z-index: 0; // establish stacking context for breakout

  // Breakout background pseudo element
  &:before {
    content:'';
    position: absolute;
    top:0;
    height: 100%;
    width: 100vw; // Takes up 100% of the viewport width
    z-index: -1; // pseudo element is behind its parent
    background-color: blue;
  }

The tricky part is positioning the element horizontally so that its left value is the far left side of the viewport. We want it to be moved left half the viewport width, but then since it's starting it's move from the left side of the grid container we have to take the size of the grid container into consideration. So after it's moved left half the viewport width it needs to move right again by half the grid container width:

    left: calc(-100vw / 2 + #{$container-max-width} / 2);

Now we have to figure out what the grid container width is. In Bootstrap there's a different width for every breakpoint, so we'll need our left positioning to be different for every breakpoint as well. We can accomplish this by looping over the available breakpoints Bootstrap provides:

@each $breakpoint, $container-max-width in $container-max-widths {
  @include media-breakpoint-up($breakpoint, $grid-breakpoints) {
    left: calc(-100vw / 2 + #{$container-max-width} / 2)
  }
}

Our final output looks like this:

@import '~bootstrap/scss/variables';
@import '~bootstrap/scss/bootstrap-grid';
@import '~bootstrap/scss/mixins';

// Apply this class to elements which should have breakout backgrounds
.breakout-background {
  position: relative; // establish breakout positioning ancestor
  z-index: 0; // establish stacking context for breakout

  // Breakout background pseudo element
  &:before {
    content:'';
    position: absolute;
    height: 100%;
    top:0;
    width: 100vw; // Takes up 100% of the viewport width
    z-index: -1; // pseudo element is behind its parent
    background-color: blue;

    // Different left position for each breakpoint
    @each $breakpoint, $container-max-width in $container-max-widths {
      @include media-breakpoint-up($breakpoint, $grid-breakpoints) {

        // Position -50% of viewport width, then readjust right again half the grid container width
        left: calc(-100vw / 2 + #{$container-max-width} / 2)
      }
    }
  }
}

Now we can add this class wherever we need a background breakout:

<div class="row breakout-background different-background">
  <div class="col">
    <h1>Here's a breakout background</h1>
  </div>
</div>

And even override the background style either using Sass inheritance or by adding a simple modifier class:

.different-background:before {
  background-image: url('./myImage.png');
  background-size: cover;
}

Nested Grids vs. Designers

When a designer creates mockups to a 12 column grid, not only is it used for the general layout, but it's also common to align more specific elements to these grid columns. This can be good because it cuts down on the extra work of creating custom widths for various elements, those elements can squeeze down in size in a consistent way with the rest of the grid, and maybe it's more visually appealing as well.

The problem is that those pesky designers don't have a clue how you need to structure your page (nor should they!). You're building your page in boxes that nest into each other, not line by line, which leads us to the problem of nested grids.

Nested grids are exactly what they sound like. A grid nested down somewhere deeper into the cell of an existing grid.

Let's take this example to illustrate the problem.

A designer creates a mockup like this:
This designer is a charity case. His name is Paul.

This is the same design with the designer's grid overlay to show the alignment of components to grid columns and separation of main area and sidebar.

The problem is that when it's time to implement the design, we might end up with something that looks like this:
Grid overlay added to illustrate misalignment of elements

What has happened is we've started a nested the grid in the main area, which itself is confined to a cell within a grid. Now there's no longer a set of CSS classes to help us size our elements properly. A column in a grid takes up 1/12th (or %8.3) of its parent. In this case we have a parent (the main area) that's already 10/12ths (or 83%) of the overall grid. Nesting a grid inside of that gives us columns that are about 6.9% of the overall grid container. They will never align to the designer's grid (ie, the overall grid).

The red bars show the columns in the nested grid. Where they overlap with the overall grid columns it's grey. Since they don't line up there's not a perfect overlap.

We need a new set of Bootstrap's col-* classes here for a nested grid that is 10/12ths of the overall grid. Luckily Bootstrap has some Sass mixins to make this easy. It comes with a mixin called make-grid-columns which takes as arguments the number of columns wide you want your grid to be, and then a map of "infix" names to breakpoints. Using the existing $grid-breakpoints map from Bootstrap's variables we can create our ten column grid like so:

@import '~bootstrap/scss/bootstrap-grid';
@import '~bootstrap/scss/mixins';
@import '~bootstrap/scss/variables';

$breakpointsCustomColumns: (
  xs10: map-get($grid-breakpoints, xs),
  sm10: map-get($grid-breakpoints, sm),
  md10: map-get($grid-breakpoints, md),
  lg10: map-get($grid-breakpoints, lg),
  xl10: map-get($grid-breakpoints, xl),
);

@include make-grid-columns($columns: 10, $breakpoints: $breakpointsCustomColumns )

This will produce classes col-xs10-1, col-xs10-2, up to col-xs10-10 for the xs breakpoint. Likewise for col-sm10-1, col-md10-1, col-md10-2, etc. for the remaining breakpoints. The keys from the breakpoints map were used as the "infix" value between the word col- and the column number at the end: col-${infix}-${columnwidth}. We can now create a grid at 1/10th widths. Using these new classes in a nested grid gives us perfect alignment with the overall grid.
Notice how the two grids column's perfectly overlap blue and red creating the grey bars.

Now we can take this one step further and create other col-* classes for other nested grid widths:

@mixin make-custom-grid-columns($columns) {
  $breakpointsCustomColumns: (
    xs#{$columns}: map-get($grid-breakpoints, xs),
    sm#{$columns}: map-get($grid-breakpoints, sm),
    md#{$columns}: map-get($grid-breakpoints, md),
    lg#{$columns}: map-get($grid-breakpoints, lg),
    xl#{$columns}: map-get($grid-breakpoints, xl),
  );

  @include make-grid-columns($columns: $columns, $breakpoints: $breakpointsCustomColumns )
}

$customColumnCounts: 5,7,8,9,10,11;

@each $customColumn in $customColumnCounts {
  @include make-custom-grid-columns($customColumn);
}

That's it! We now have other sized grid columns we can use to align sub grid columns to the overall grid the designer is aligning elements to. col-sm5-1, col-sm5-2, col-md7-5, etc. You may have noticed we skipped creating columns for grids that are 2, 3, 4, and 6 wide. That's because they all divide evenly into 12, so the default 12 column grid can be used on those situations.

Grids are for Parents. Not for Kids

This tip is more of a general guideline or a best practice for working with any kind of grid system and a component based application framework such as Angular, React, Vue, or Web Components. The tip is this: A component should not know about the grid it is being laid out into.

The reasons why are reusability, and just generally knowing where to draw the line between your parent component and your child component.

So what am I talking about? Well using the 12 column grid as an example, a parent component is probably setting a layout using rows. That much makes sense. But it should also fall on the responsibility of the parent to layout the cells. The child component should not have any styling on it that says "I take up 3 columns of ... something".

Here's a brief example of the bad practice:


<!-- the Parent component -->
<div>
  <h1>I'm the parent!</h1>
  <div class="row">
    <app-mychild></app-mychild>
  </div>
</div>

<!-- The child component 'app-mychild'. Assumes it needs to take up 6 columns -->
<div class="col-md-6 child-style">
  <h2>I'm the child!</h2>
  <!--  some other important child component stuff here-->
  ...
</div>

Here's an example of the good practice:


<!-- the Parent component. Decides how much space to give the child. -->
<div>
  <h1>I'm the parent!</h1>
  <div class="row">
    <div class="col-md-6">
      <app-mychild></app-mychild>
    </div>
  </div>
</div>

<!-- The child component 'app-mychild' -->
<div class="child-style">
  <h2>I'm the child!</h2>
  <!--  some other important child component stuff here-->
  ...
</div>

In the good example above, the child is not coupled to any specific grid layout. It can be reused in other places where it may need to take up more space, or less space. From a layout perspective, the child component's responsibility is to either take up 100% of the space given to it by it's parent, or only as much as it needs to display its content depending on scenario.

There may be some exceptions to the rule, like if the child component only ever exists with that parent component and is never reused anywhere else, but often its probably still a good idea to follow this rule just so we know where to draw the line on layout responsibility.

This rule doesn't mean a child component can't have a grid layout of it's own, to layout its own children. That's perfectly fine as long as the grid existing entirely within the child component doesn't bleed into its own children.