DEV Community: CompanyCam

Creating and Testing Apollo Client Links

Jason Gaare — Mon, 23 Oct 2023 15:29:59 +0000

When using Apollo Client, there might come a time when you want to inspect or make some changes to your GraphQL requests. This is where Apollo Link comes into play. The library sets up a chain of actions for each GraphQL operation the client executes.

What exactly is a link, and how would we create and test one? Let's get into it and see what's going on.

Understanding the Concept of a Link

In order to grasp the concept of a link in the context of Apollo Client, let's start with what the docs tell us:

The Apollo Link library helps you customize the flow of data between Apollo Client and your GraphQL server. You can define your client's network behavior as a chain of link objects that execute in a sequence. Each link should represent either a self-contained modification to a GraphQL operation or a side effect (such as logging). Source

When Apollo Client initiates a request, the request traverses through the chain of links until it reaches a terminating link. Typically, the terminating link is responsible for executing the actual fetch to the GraphQL server and managing the subsequent response. The response then travels back through the chain in reverse order.

// The `from` function combines an array of individual links into a link chain
const client = new ApolloClient({
  link: ApolloLink.from([errorLink, myCustomLink, httpLink]),
  ...
});

// alternatively, you can also use concat like so
const client = new ApolloClient({
  link: errorLink.concat(httpLink)
  ...
});

Creating a Link

There are two types of links: stateless and stateful. A link is an instance of the ApolloLink class (or a subclass of it) and each link requires the definition of a request handler: request(operation, forward).

Let's take a quick look at those two request handler params:

operation - this is all the juicy info about the current, well, operation - the entirety of the query and what is attached to it. Here we can easily access operation.query and operation.variables to understand information about the request, as well as the rest of the context for the request via operation.getContext()
forward - this is a function that takes the operation and forwards it to the next link, like so: forward(operation)

Handling Responses in the Link Chain

Now that we've successfully forwarded the operation along the link chain, it's time to handle the response as it journeys back through the chain. To accomplish this, we need to establish an observer and subscribe to the response.

Fortunately, this process is quite straightforward. Apollo Client provides the Observable class for precisely this purpose. You can employ this class to set up an observer subscription. Here's how it's done:

import { Observable } from '@apollo/client';

// the required request handler
request(operation, forward) {
  return new Observable((observer) => {
    const subscription = forward(operation).subscribe({
      next: (result) => { 
        observer.next(result);
        observer.complete();
      },
      error: (error) => observer.error(error),
      complete: () => observer.complete(),
    });

    return () => {
      subscription.unsubscribe();
    }
  });
}

In the code snippet above, we create a new Observable instance within the request function. This Observable instance facilitates the subscription to the response. We subscribe to the response by forwarding the operation and defining handlers: next, which receives the data from a successful operation attempt, error, and complete, which signals we are done with handling the response.

While the code above represents a straightforward way to set up a subscription, it really doesn't do anything yet. The real magic happens when we add custom logic into this process. For instance, if we set a startTime in our request handler before forwarding the operation, we can then utilize our observer logic to calculate total request time:

request(operation, forward) {
  // set the startTime before we forward the operation
  operation.setContext({
    startTime: Date.now(),
  });

  return new Observable((observer) => {
    const subscription = forward(operation).subscribe({
      next: (result) => { 
        const { operationName } = operation;
        const startTime = operation.getContext().startTime;

        if (operationName && startTime) {
          // calculate total request time in our observer
          logRequestTime(
            operationName, 
            Date.now() - startTime
          );
        }

        observer.next(result);         
        observer.complete();
      },
      error: (error) => observer.error(error),
      complete: () => observer.complete(),
    });

    return () => {
      subscription.unsubscribe();
    }
  });
}

Here, we perform additional logic by recording the request time before forwarding the result to the observer. This flexibility enables you to intercept and manipulate data seamlessly within Apollo, adding a powerful tool to your toolkit.

Testing Apollo Links

When testing an Apollo Link, it's useful to write a separate simple link which acts as a terminating link. Then, create a composed link with the link to test, with the terminating link being our mock link. This test link should be simple and respond in a predetermined the way in the scope of our testing.

Here's an example mockLink:

const mockLink = new ApolloLink((operation) => {
  const context = operation.getContext();
  const { mockError, mockResponse } = context || {};

  return new Observable((observer) => {
    if (mockError) {
      observer.error(mockError);
      return;
    }

    if (mockResponse) {
      observer.next({ data: mockResponse });
      observer.complete();
      return;
    }

    observer.error(
      new Error('mockLink: You must provide mockError or mockResponse')
    );
  });
});

Within the mockLink function, we inspect the operation's context to check for the presence of mockError or mockResponse. We construct an Observable which, depending on the context either emits a mock error or a mock response wrapped in the expected data structure. In case neither is provided, the mockLink generates an error to ensure that test cases explicitly define the expected behavior.

This link then sends data or an error back down the chain (in our case, back to the link we are testing).

A Simple Link to Test

Let's see this in action to make more sense of it. Imagine our link OperationHistoryLink does exactly that: keeps an array of our operations in order as we execute them. We can create a stateful link to handle this for us:

import { ApolloLink } from '@apollo/client';

// stateful link
class OperationHistoryLink extends ApolloLink {
  operationHistory = [];

  request(operation, forward) {
    this.operationHistory.push(operation);
    return forward(operation);
  }
}

To test this, we must execute a query, then assert the operation is added to the array when it succeeds.

import { execute } from '@apollo/client';

describe('OperationHistoryLink', () => {
  it('adds operation to the operationHistory array on success', () => {
    const operationHistoryLink = new OperationHistoryLink();
    const composedLink = operationHistoryLink.concat(mockLink);

    return new Promise((resolve) => {
      const operation = {
        query: 'some query',
        context: {
          mockResponse: { foo: 'bar' },
        },
      };

      execute(composedLink, operation).subscribe({
        next: ({ data }) => {
          expect(data.foo).toBe('bar'); // mockResponse is handled by mockLink
          expect(operationHistoryLink.operationHistory).toHaveLength(1);
          expect(operationHistoryLink.operationHistory[0].query).toEqual(
            operation.query,
          );
          resolve();
        },
        error: () => {
          // optionally reject or throw here, we shouldn't get an error
        },
        complete: () => {},
      });
    });
  });
});

A few things to note about our test:

To trigger our GraphQL operation, we use the execute function provided by Apollo Client. This function begins the process of sending the operation down the link chain, creating an observer subscription with the same signature as we used in our link itself (next, error, complete)
To ensure that our test has time to execute, we use a Promise. This allows us to await the resolution of the Promise, ensuring that our test assertions are carried out only after the GraphQL operation has completed its execution
Our assertions live inside our next subscription callback. Of course, when our data comes back from our mockLink, we can expect it to match what we sent in the context. However, the main thing we are asserting here is the operationHistory in our OperationHistoryLink has a length of 1. If that is true, we know our link is working as intended!

OperationHistoryLink
    ✓ adds operation to the operationHistory array on success (3 ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        2.141 s

Success! We have now successfully created a custom link, and written a test to assert its functionality.

Want to try it out yourself? You can view all the code here

Cover Photo by Aylin Çobanoğlu on Unsplash

Sidekiq Request ID Tracking

Chad Wilken — Fri, 08 Jul 2022 15:47:38 +0000

Sidekiq is the go to job system for most Ruby and Rails applications. It's blazing fast, well supported, trusted by thousands, and best of all it's free! However, I wish that it included a default method for tracking the Rails request_id that caused the job to be enqueued. This is useful if we want to see which controller action kicked off the job. No worries though, Sidekiq has the concept of middleware - similar to Rack - that allows us to implement functionality on the Sidekiq client and/or server.

Let's Get Started

Before we can implement the Sidekiq middleware we need a place to store request_id for the current request. In our case we use the Current model. Rails has some magic that resets the attributes defined on Current before and after each request - this makes it a great place to store thread-isolated data. That being said, let's go ahead and add the request_id attribute to our Current model.

class Current < ActiveSupport::CurrentAttributes
  attribute :request_id
end

We can use a before_action in our ApplicationController so that every request will store it's request_id.

class ApplicationController < ActionController::Base
  before_action do
    Current.request_id = request.request_id
  end
end

Now that we have global access to the request_id we need to create some client and server middleware for Sidekiq.

Client Middleware

Client middleware runs before the job is pushed to Redis - allowing us to modify the job. Knowing this, the middleware is pretty easy.

# You can put this in lib/request_id/sidekiq/client.rb

module RequestId
  module Sidekiq
    class Client
      def call(worker_klass, job, queue, redis_pool)
        # Stash the request_id in the job's metadata
        job['request_id'] = Current.request_id
        yield
      end
    end
  end
end

Server Middleware

The server middleware runs around the execution the job, allowing us to do something before and after the job's execution. Once again it's a class that responds to call.

# You can put this in lib/request_id/sidekiq/server.rb

module RequestId
  module Sidekiq
    class Server
      def call(worker, job, queue)
        Current.request_id = job['request_id']
        yield
      ensure
        Current.reset
      end
    end
  end
end

On the server we pull the request_id from the job's metadata and set the value on Current. We'll want to use ensure and call Current.reset so that all attributes are reset when the job completes - whether it's successful or not.

Hooking it Up

Now that we have our middleware let's tell Sidekiq to use it.

# config/initializers/sidekiq.rb

require 'lib/request_id/sidekiq/client'
require 'lib/request_id/sidekiq/server'

Sidekiq.configure_server do |config|
  if Sidekiq.server?
    config.server_middleware do |chain|
      chain.add RequestId::Sidekiq::Server
    end
  end
end

Sidekiq.configure_client do |config|
  config.client_middleware do |chain|
    chain.add RequestId::Sidekiq::Client
  end
end

Now in our job we can include the request_id that triggered the job in the logs.

class ImportUsersFromCsvWorker
  include Sidekiq::Worker

  def perform(csv)
    # …do stuff
    Rails.logger.info({ message: 'Imported Users', request_id: Current.request_id })
  end
end

Summary

Knowing what action triggered a job can be quite powerful while debugging. In a few lines of code we can implement middleware for Sidekiq allowing us to use the request_id of the request that caused our job to be enqueued. We could even go a step further and add the request_id to Sidekiq's context and include it in the default logs if we wanted to.

What Sidekiq middleware have you implemented or found useful in your app?

Building a Workflow for Async Searchkick Reindexing

Chad Wilken — Wed, 29 Jun 2022 20:36:17 +0000

We lean heavily on Elasticsearch here at CompanyCam, especially to serve our highly filterable project feed. It's incredibly fast, even when you apply multiple filters to your query while searching a largish data set. Our primary interface for interacting with Elasticsearch is using the Searchkick gem. Searchkick is a powerhouse with many features out of the box, but we bump up against the edges when trying to reindex a large collection.

Mo' Projects, Mo' Problems

CompanyCam houses just under 21 million projects with tens of thousands of new projects added daily. On occasion we change the fields that we want to be available for filtering. In order to make the new field(s) available, we have to reindex the entire collection of records. Reindexing a collection of this size - where each record additionally pulls in values from associated records - can be quite slow. If we run the reindex synchronously it takes about 10 hours, and that is with eager loading of the associations and other optimizations. Never fear though, Searchkick accounts for this and has the ability to use ActiveJob to reindex asynchronously. The one thing that isn't accounted for is how to promote that index once the indexing is complete. You can run the task like reindex(async: { wait: true }) which will run the indexing operation async and do a periodic pull waiting for indexing to complete and then promote the index. This almost works but can still take hours and I hate to sit on a server instance waiting for this to complete - what if I get disconnected or the instance terminates due to a deploy? We decided that it was time to build a small workflow around indexing large collections asynchronously that automatically promotes itself upon completion.

Enter Tooling

My first goal was to start a collection indexing operation from our internal admin tool, Dash, and monitor the progress. With those two simple goals in mind, this is what I came up with.

I needed two jobs - one job to enqueue the indexing operation and another to monitor the operation until completion. Once those are built, I can use Rails basics to wrap this in a minimalistic UI.

The Jobs

The first job needed to:

Accept a class name and start the reindex for the given class
Store the pending index name for later usage, optimally in Redis
Enqueue another job to monitor the progress of the indexing operation

This is what I came up with:

module Searchkick
  class PerformAsyncReindexWorker
    include Sidekiq::Worker

    sidekiq_options retry: 0

    def perform(klass)
      result = klass.constantize.reindex(async: true)
      index_name = result[:index_name]
      Searchkick::AsyncReindexStatus.new.currently_reindexing << index_name
      Searchkick::MonitorAsyncReindexWorker.perform_in(5.seconds, klass, index_name)
    end
  end
end

The second job, as you may have guessed, monitors the indexing progress until completion. The basic functionality needs to check the operation and:

If incomplete, re-enqueue itself to check again in 10 seconds
If complete, promote the index for the given class and removes the index name from the collection in Redis.

module Searchkick
  class MonitorAsyncReindexWorker
    include Sidekiq::Worker

    def perform(klass, index_name)
      status = Searchkick.reindex_status(index_name)
      if status[:completed]
        klass.constantize.search_index.promote(index_name)
        Searchkick::AsyncReindexStatus.new.currently_reindexing.delete(index_name)
      else
        self.class.perform_in(10.seconds, klass, index_name)
      end
    end
  end
end

By default the Searchkick::BulkReindexJob uses the same queue as regular async reindexing, blocking user generated content from being indexed while performing a full reindex. So I also patched the Searchkick::BulkReindexJob to use a custom queue we have just for performing full collection indexing operations. In an initializer I simply did:

class Searchkick::BulkReindexJob
  queue_as { 'searchkick_full_reindex' }
end

The Status Object

You may be wondering what Searchkick::AsyncReindexStatus is. It is a simple class that includes the Redis::Objects library so that we can store a list of currently reindexing collections. It looks like this:

module Searchkick
  class AsyncReindexStatus
    include Redis::Objects

    def id
      'searchkick-async-reindex-status'
    end

    list :currently_reindexing
  end
end

Note: I opted to use Redis::Objects since it was already in our codebase and it is a bit simpler than interacting with Redis directly using Searchkick.redis.

How to Kick off the Job

An indexing operation can be kicked off in one of two ways. You can start it via the command-line if you have access such as Searchkick::PerformAsyncReindexWorker.perform_async(model_class). Instead, we built a crude interface into our internal admin tool. The UI allows us to select a model and start the indexing operation and then track it's status until completion.

The Code

For the full code that we use you can look at this gist. Always happy to hear improvements that could be made as well!

Recap

Searchkick saves us serious time and energy developing features. By taking one of its existing features like async reindexing and wrapping it in a bit of workflow, we can get even more out of it. In the end, we were able to scratch our own itch for truly async indexing operations.

How We Sped up Our CI Pipeline by Over 4x

Jeff McFadden — Thu, 05 May 2022 21:07:23 +0000

The Problem

Here at CompanyCam, we rely on automated testing to ensure that our code is working as expected. Our backend is written in Ruby on Rails, and we leverage automated testing (via rspec) to help ensure that new code doesn’t break existing functionality. We have many thousands of these automated tests—over 3500 at the time of writing—and we’re always writing new tests alongside our new features, to be sure those don’t break in the future, either.

Because these tests are so important to our development process, and because we have many developers working on our codebase simultaneously, we execute our full test suite dozens of times each day.

For new features and maintenance work, we use a typical pull-request workflow, running tests at each step along the way. A normal day for a developer might look like this:

Write code.
Test that code locally.
Push this new code to our central repository (Github).
Run tests to ensure that our new code is working properly.
If changes are needed, go to step 1, otherwise proceed.
Merge in changes that other developers have written in the meantime.
Run tests again, ensuring everything still works.
If changes are needed, go to step 1, otherwise proceed.
Merge code into the main branch.

Because the run tests step is done so frequently, it’s important that it happens as quickly as possible.

This is where we had a problem.

It was taking 13 or more minutes to run our full test suite from start to finish. During that time there wasn’t much for a developer to do—if their tests failed, they were going to have to fix or redo some work. If they passed, they could carry on—but it wasn’t safe to carry on before knowing everything was working properly. With 13 minutes to kill, it was easy to get caught up in another conversation, go make another cup of coffee, play some Wordle, or get lost in any other distraction. Suddenly that 13 minutes was 20 minutes, or a half hour. Multiply this delay by all the times the tests had to be run each day (many dozens at least), and you can see that tests were a major slowdown for our team.

Setting a Goal

We knew we wanted the tests to run faster, but how much faster was enough? We set a goal to get the tests under 5 minutes. We had an intuitive sense this was possible, and 5 minutes seemed like a short enough time frame that it wouldn’t turn into a full on-distraction. I had an internal stretch goal of getting our tests down to 3 minutes, but at the time I had no solid reason to believe we could get there.

Step 1: Parallel Tests

Once we had a goal, we needed to figure out how to get there. The first thing we knew we had to do was get our tests to run in parallel. We figured that we could always throw more computing power at the tests, but unless we could run many tests at the same time, side by side, we’d never be able to make full use of any available computing power we might have.

Several members of our team worked together to get the parallel_tests gem working for our codebase. Most of our tests were fine running in parallel, but we found enough that made assumptions about the order they would be run in that we had to do some fixing and rewriting. We kept this work in a feature branch, and kept chipping away at it until all the tests would pass.

Once we had all the tests running in parallel, we pushed the branch up to our CI provider, and our tests were passing in about 8 minutes. That’s about a 40% time savings! A good start, but we knew we could do better.

Step 2: Evaluating Our CI Provider

The process of running tests automatically with each new bit of code is called Continuous Integration (CI). We use a CI service in the cloud to run these tests for us.

As we dug into the timeline reports for our running tests, we discovered that our tests were actually running pretty quickly—about 3 minutes—but the environment setup step was taking a full 5 minutes before the tests could even get started!

We spent some time reading through the documentation from our CI provider at the time, and made as many optimizations as we could. Still, we just couldn’t get the setup time down. It seemed that we were maxing out the capacity of their servers during our setup process.

We decided it was time to look around at other CI providers to see if we could find one that better fit our needs.

Step 3: Testing Other CI Providers

We set up trial accounts with a few providers and spent the time to get our testing environment working on each of them. During this process, we kept our existing CI provider integration working; it was imperative that we didn’t disrupt the regular workflow of our engineers while we endeavored to speed things up.

Right away we discovered that other CI providers had more capable platforms, and we were able to get our tests down in the 6 minute range—tantalizingly close to our 5 minute goal!

One provider stood out from the others: CircleCI. After testing we decided to put our optimization efforts into CircleCI and see what kind of results we could get.

We were not disappointed by this decision.

Step 4: Optimizing CircleCI

CircleCI offered more choices for machine size and parallelism than other providers we tested, and by using a few “large” instances we were able to see our tests run blazingly fast—under two minutes! However, our setup time —the time it takes before the tests can even begin, during which our test environment is initialized—was still in the 90-120 seconds range. After some investigation, we were pretty confident we could make that step much faster as well.

Caching Assets

In order to make that setup time faster, the first thing we needed to do was cache our gems; gem install was taking 30-45 seconds, and our gems rarely change. By leveraging CircleCI’s asset caching feature, we are able to store a copy of the installed gems, which we restore every run instead of re-installing. By using this cached copy we were able to cut the assets step of setup down to a 2 second check, saving half a minute per run. (When the gems do change, the first run takes a little longer. Subsequent runs use the new cached gems, and are fast again).

The Big Win: Using CircleCI’s Parallelism Feature

CircleCI has a really cool feature for running tests in parallel. The parallel feature is kind of like the parallel_test gem, but on steroids—it splits up the individual test files for you, to as many virtual test servers as you want (up to about 64).

Let’s say you have 16 tests that need to be run. By running your tests with the parallelism value set to 2, two test servers will be set up. The first will run 1-8, the second 9-16. If you set parallelism to 4, then you’d have four servers, running 1-4, 5-8, 9-12, and 13-16, respectively.

But the magic doesn’t stop there! After every run, CircleCI stores the amount of time each test took to run, and the next time the tests are run, it divides them up by estimated duration!

Here’s an example. Let’s say you have four tests, and the time to run each of them varies:

Test 1: 30 seconds
Test 2: 27 seconds
Test 3: 17 seconds
Test 4: 9 seconds

If you split up the tests in order (1&2, 3&4), then your total run time will be the time of the longest-running server. So, in this example, that would be 30+27, for a total of 57 seconds. The other server will have finished in 26 seconds. CircleCI will notice how long each test takes, then in the next run break them up to be the closest total per group possible. So the next run would be 1&4 and 2&3. Because the longest run in this case is just 44 second, you’d cut nearly 25% (13 seconds) off your total run time!

Because we have many thousands of tests, we’re able to split them up very evenly.

By leveraging CircleCI’s parallelism we were able to run our tests on as many as 32 instances at a time, cutting our test time down to less than 30 seconds! Still, because of setup, total run time was in the 2:30 range.

Aside: As another bonus, running the tests this way meant we didn’t have to use the parallel_tests gem. We didn’t have any problems with the parallel_test gem (it still makes sense to use it in local development, for example, where we aren’t using CircleCI’s magic) but not having to use it in CI made our test stack simpler and easier to maintain.

The Last Step: Docker Image Hosting and Caching

Now that the testing process itself was running crazy fast, we needed to optimize the last few pieces of the setup process.

The first step was to move our Docker images to Amazon Elastic Container Registry (ECR). ECR infrastructure is close (in the network sense) to CircleCI, and downloads have been very fast for us.

The second step was to ensure that we were using the CircleCI Convenience Images whenever possible. These Docker containers are heavily cached; in most cases they are instantly available for you.

The last step was a passive one: as more and more of our tests were being run on CircleCI’s environment, it meant we were more likely to get cache hits on our own Docker container images, cutting out the time it takes to download those images from ECR.

Altogether, these optimizations brought our total run time down into the low two minutes (2:15 average)! Our original goal had been under 5 minutes—we were under half that time!

Price/Performance Balance

While we were able to run our tests blazingly fast with a parallelism of 32, we were paying for 32x the setup costs of running in a single box. Over time, this can get expensive.

We ran a series of tests, changing the parallelism parameter in our CircleCI config, and testing the resulting total run time.

In the end, we settled on a parallelism of 12. That’s about 1/3 the total cost of 32, and our test suite completes in 2:45-3:00, which we find is an acceptable time range for our staff.

What’s Next

With testing, the work is never truly done. We still have some lingering flaky tests (tests that sometimes pass, sometimes don’t) that we’re working on cleaning out. We’re also in the middle of the process of identifying slowly running individual tests and optimizing them to speed up our suite a bit.

And then, of course, as we keep writing tests, we’re going to need to maintain our CI suite so that we don’t slow down over time. We regularly check our average test run times, and will adjust our parallelism factor as needed (and optimize individual slow tests) to keep things moving quickly.

Speeding up our test suite has been a huge win for our engineering team. It’s improved morale, removed a bunch of frustration, and opened up new avenues of possibility. Mostly, it’s saving us a ton of time every day. Time that we can spend Showing Up, Growing Up, and Doing Good.

Build an Embeddable Widget using Preact and the Shadow DOM

Coleman Imhoff — Wed, 13 Oct 2021 21:36:50 +0000

Our team at CompanyCam was tasked with building a widget that our users could embed on their websites. The widget needed to be easy to install, responsive, and provide a fullscreen application experience. This article introduces and explains the technical decisions made and how we got there.

Discovery

Before jumping into the code, I want to quickly discuss some things our team learned during discovery. Hopefully, this will assist you in making the right decisions for your project.

After learning about the details of the product, we found that the codebase had two requirements.

Encapsulation

Our team needed to prevent external CSS from cascading into our code. In addition, our styling needed to be scoped to our application. We explored wrapping the widget in an iFrame, which provides a nested browsing context. This offered the encapsulation we needed, but we found it difficult to control the iFrame in order to provide a quality fullscreen experience. The Fullscreen API was a potential solution, but it did not hold the required browser support. Using an iFrame to encapsulate a smaller product could be a great solution, but did not fit our use case.

We turned our attention to the Shadow DOM API. The Shadow DOM provides a way to attach a hidden DOM tree to any element. This creates encapsulation, but doesn't limit your ability to have control of the application. In addition, the Shadow DOM API has good browser support.

Small Bundle

It's imperative that the widget loads quickly. With the strategy the team had in place, it was clear that it was going to be difficult to code-split our application. At CompanyCam, engineers write user interfaces in React therefore it made sense to stick with that.

As we added 3rd party libraries, our bundle size grew. We found that Preact was a good solution to this problem. It provides all the same features as React, but in a much smaller package. You can compare the unpacked size of Preact to a combined React and React-DOM and see a significant difference!

Now, let's jump into some code! Feel free to clone this starter repo if a working example is helpful for you.

Mounting Your App with a Shadow DOM Layer

Preact is easy to integrate into an existing project. Mounting our Preact App component should look similar to React.



/* @jsx h */
import { h, render } from "preact";

import App from "./components/App.jsx";

const appRoot = document.querySelector("#app-root");

render(<App />, appRoot);

Now let's add a Shadow DOM layer.



/* @jsx h */
import { h, render } from "preact";

import App from "./components/App.jsx";

// app shadow root

const appRoot = document.querySelector("#app-root");
appRoot.attachShadow({
  mode: "open",
});

render(<App />, appRoot.shadowRoot);

We can attach a Shadow DOM layer to a regular DOM node, called a shadow host. We can do this by calling the attachShadow method, which takes options as a parameter. Passing mode with the value open allows the shadow DOM to be accessible through the shadowRoot property. The other value for mode is closed, which results in shadowRoot returning null.

To verify things are in working order, we can open our
browser's developer tools and and look at the DOM tree. Here, we can see our Shadow DOM layer.

Styling the Shadow DOM

Styles must be scoped inside the Shadow DOM in order to render.



/* @jsx h */
import { h, render } from "preact";

import App from "./components/App.jsx";

import styles from "./styles.css";

// app shadow root

const appRoot = document.querySelector("#app-root");
appRoot.attachShadow({
  mode: "open",
});

// inject styles

const styleTag = document.createElement("style");
styleTag.innerHTML = styles;
appRoot.appendChild(styleTag);

render(<App />, appRoot.shadowRoot);

If you're using webpack, keep in mind you will need css-loader in order for this approach to work. Create a style tag and set its innerHTML to an imported stylesheet.

As our application grew, managing our styles became cumbersome and our team wanted to find another solution. At CompanyCam, our designers enjoy designing our products with styled-components. With styled-components, a generated stylesheet is injected at the end of the head of the document. Due to our Shadow DOM layer, this won't work without some configuration.



/* @jsx h */
import { h } from "preact";
import styled, { StyleSheetManager } from "styled-components";

const Heading = styled.h3`
  color: #e155f5;
  font-family: sans-serif;
`;

const App = () => {
  return (
    <StyleSheetManager target={document.querySelector("#app-root").shadowRoot}>
        <Heading>Hey, Shadow DOM!</Heading>
    </StyleSheetManager>
  );
};

export default App;

The StyleSheetManager helper component allows us to modify how styles are processed. Wrap it around the App component's children and pass the shadowRoot of the shadow host as the value of target. This provides an alternate DOM node to inject styles into.

Just like the previous technique, we can see our styles scoped within the Shadow DOM.

Avoid Inheritance

The Shadow DOM will prevent outside CSS selectors from reaching any contained markup. But, it is possible for elements in Shadow DOM to inherit CSS values. We can reset properties to their default values by declaring the the property all to the value initial on the parent element of your application.



/* @jsx h */
import { h } from "preact";
import styled, { StyleSheetManager } from "styled-components";

const Heading = styled.h3`
  color: #e155f5;
  font-family: sans-serif;
`;

const WidgetContainer = styled.div`
  all: initial;
`;

const App = () => {
  return (
    <StyleSheetManager target={document.querySelector("#app-root").shadowRoot}>
      <WidgetContainer>
        <Heading>Hey, Shadow DOM!</Heading>
      </WidgetContainer>
    </StyleSheetManager>
  );
};

export default App;

Win the Stacking Order Battle with Portals

Whether it's Wordpress, Squarespace, Wix, or something from scratch, our widget needed to live on any website. Since stacking order depends on the DOM tree hierarchy, we immediately saw z-index issues in our fullscreen components.

Portals provide a way to render children into a DOM node which exists outside the context of the application. You can mount your Portal to any DOM node. In our case, we needed to render these fullscreen components as high in the DOM tree as possible. Therefore, we can append our Portal to the body of the document we are installing the widget on.

Let's create our Portal by starting at the root of our application.



// index.js

/* @jsx h */
import { h, render } from "preact";

import App from "./components/App.jsx";

// shadow portal root

const portalRoot = document.createElement("div");
portalRoot.setAttribute("id", "portal-root");
portalRoot.attachShadow({
  mode: "open",
});
document.body.appendChild(portalRoot);

// app shadow root

const appRoot = document.querySelector("#app-root");
appRoot.attachShadow({
  mode: "open",
});

render(<App />, appRoot.shadowRoot);

Create a shadow host for the Portal component and give it an id. Then, just like we did with appRoot, attach a new Shadow DOM layer.



/* @jsx h */

import { h, Fragment } from "preact";
import { useLayoutEffect, useRef } from "preact/hooks";
import { createPortal } from "preact/compat";
import styled, { StyleSheetManager } from "styled-components";

const Portal = ({ children, ...props }) => {
  const PortalContainer = styled.div`
    all: initial;
  `;

  const node = useRef();
  const portalRoot = document.querySelector("#portal-root");

  useLayoutEffect(() => {
    const { current } = node;
    if (current) {
      portalRoot.appendChild(current);
    }
  }, [node, portalRoot]);

  return (
    <Fragment ref={node} {...props}>
      {createPortal(
        <StyleSheetManager target={portalRoot.shadowRoot}>
          <PortalContainer>{children}</PortalContainer>
        </StyleSheetManager>,
        portalRoot.shadowRoot
      )}
    </Fragment>
  );
};

export default Portal;

Next, create the Portal component. Add an effect to append portalRoot to the parent element of the component. From there, pass children and portalRoot.shadowRoot to createPortal.

Remember to scope your styles to the Portal Shadow DOM layer using StyleSheetManager and reset child elements' styles to their default values.



/* @jsx h */
import { h } from "preact";
import styled, { StyleSheetManager } from "styled-components";

import Portal from "./Portal.jsx";

const Heading = styled.h3`
  color: #e155f5;
  font-family: sans-serif;
`;

const WidgetContainer = styled.div`
  all: initial;
`;

const App = () => {
  return (
    <StyleSheetManager target={document.querySelector("#app-root").shadowRoot}>
      <WidgetContainer>
        <Heading>Hey, Shadow DOM!</Heading>
        <Portal>
          <Heading>Hey, Shadow Portal!</Heading>
        </Portal>
      </WidgetContainer>
    </StyleSheetManager>
  );
};

export default App;

Now, we can wrap any fullscreen component within our Portal.

Conclusion

Recently, our team has released the widget to GA. The techniques outlined above have allowed us to build a rich application experience with a small codebase that is... mostly encapsulated. We still run into the occasional z-index issue or JavaScript event conflict provided by a website builder theme. Overall, widget installs have been a success.

Triggering Bitrise Builds via Pull Request Label

Jason Gaare — Wed, 01 Sep 2021 19:43:57 +0000

Bitrise makes mobile deployment a breeze, but the options they provide for automatic build triggers are very all-or-nothing. In this post, we'll go over how to easily trigger Bitrise builds, but only when we want to.

Triggering Bitrise Builds

Bitrise provides "fast, flexible, and scalable mobile CI/CD that just works." That's actually quite accurate! Creating and triggering build workflows is straightforward with Bitrise.

Out of the box, it easily integrates with your Git provider. After integration, Bitrise uses webhooks to provide the following automatic build triggers:

Trigger via push to a specific branch (or, ANY branch)
Trigger via pull request (can filter by target or source branch)
Trigger via specified git tag

These are super helpful for many workflows, however our ideal trigger wasn't quite covered...

Close But No Cigar

At CompanyCam, our ideal trigger would be closest to the "Trigger via pull request" method, but we really didn't want every single PR to trigger a build (we are paying for each of these builds, after all).

First off, certainly not all PRs would require our entire pipeline to run, especially if all that changed was the README! Other smaller changes could easily be tested on our nightly builds and wouldn't require an entire dedicated build.

Secondly, filtering and triggering builds via a branch naming convention could work, but would require developers to either:
(a) have prior knowledge when creating their branch that they would later want a Bitrise build for that branch, or
(b) rename/duplicate their branch as they encountered the need for a build (to match the triggering naming convention)

Neither of these felt ideal.

We were looking for a way to enable our developers to selectively choose whether their PR would or would not have a Bitrise build associated with it, at the moment they opened the PR (or even after!)

Ad-Hoc Automation

The solution we sought was simple. We were looking for a way to add "Trigger by PR Label" functionality to connect our repository and Bitrise. How could we achieve this ad-hoc automation?

Well, more on that in a second... but first, there's a hidden gem you should know before we go further (one that we'll use shortly):

While using the Bitrise web app, when you click "Start/Schedule a Build", navigate to the Advanced tab, and scroll to the bottom, you should find this handy-dandy, copy-paste-ready cURL request for triggering a build (with the specific options you added in the form above) - Slick!

Putting Our Labels to Work

Okay, now we can actually answer "How could we achieve this ad-hoc automation?"

Enter: GitHub Actions.

Along with our copy-paste-ready cURL request, and some simple logic to check for a label with the name "Build Bitrise" (code below), we have a short and simple GitHub action that effectively acts as a "Trigger Build by PR Label" for our repository.

✨ Now, any PR with any source or target or branch name can have an associated Bitrise build simply by slapping a label on it!

Here's the entire GitHub Action we use: (with some secrets removed).

Happy Triggering!

About CompanyCam

👋 Hi, we’re CompanyCam. We create simple-to-use, visual-first communication and accountability tools for contractors.

We are almost always hiring developers (we like React and Rails a lot). Check out our careers page to learn more.

Post Script

You might have noticed, there are some extra params we added to the copy-paste-ready cURL request:

"build_params":{
  "branch":"${{ github.head_ref }}",
  "branch_dest": "${{ github.event.pull_request.base.ref }}",
  "workflow_id":"deploy-internal",
  "commit_hash":"${{ github.event.pull_request.head.sha }}",
  "pull_request_id": ${{ steps.pr_number.outputs.number }}
},

These params help better identify this PR in Bitrise, and perhaps most importantly help with associating future builds from the same PR to maintain the rolling builds functionality provided by Bitrise 🎉

Credits

Cover Photo by Dan-Cristian Pădureț on Unsplash