Adding syntax highlighting to my blog with Torchlight

#beginners #javascript #webdev #tutorial

Background

A little while ago I read the following blog post, Web Component for a Code Block by Chris Coyier, it got me thinking about how I could enhance the code blocks on my website. This is a tech blog after all, and from time to time I want to share tips and tricks which involve code snippets. Code syntax highlighting is vital for blog posts because it helps to visually distinguish different elements of code, making it easier to read, understand, and spot errors. HTML has very handy <pre> and <code> elements which help with the markup. I previously had some styling that left me with this solution:

Note that all the text is just white, not at all what you would see in your IDE of choice.

Research & Implementation

All of the content on this blog is hosted on WordPress, and I've built a Nuxt module to fetch the content and transform it which I'll write another blog post on at a later date once I've finalised the API for it.

I initially extended my module to process all <pre> elements in a blog post. I wrote a loop over each element and which let me experiment with a few different syntax highlighting libraries.

Chris had used PrismJS in his web component, so I thought that was as best a place to start as any. It was relatively easy to install with Yarn.

I realised none of my code examples included a language, and Prism needed a language set for each block. I went down a rabbit hole of finding a language detector. Trying to Google that got a lot of results for detecting human languages, such as English or Spanish, rather than code languages, such as JavaScript or PHP. I did find a few detectors, including one based on what GitHub use, Linguist, but most of the auto-detection libraries rely on other contextual clues such as file extension, whereas I just wanted to pass a string to the detector.

I then found a blog from LogRocket, exploring the best syntax highlighting libraries, and looked into Highlight.js. They offer an auto-detection language feature which worked, but wasn't very accurate. Similar to PrismJS, it was very quick to get a working solution together.

I bit the bullet and went through all of my old blog posts and assigned each code block a language. It didn't take much longer than 5 minutes as I didn't have that many code blocks in the first place. I was just over-complicating things!

I stumbled upon another blog post talking about the Torchlight syntax highlighting library and realised that Highlight.js was not very good at Syntax highlighting at all. Torchlight works slightly differently from Highlight.js and PrismJS in that it is a HTTP API. You pass your content to it, and it returns it with syntax highlighting. The highlighter is closed-source, but it uses the engine behind VSCode, so highlighting is much more accurate. The code examples on my site could match exactly what I see in VSCode, perfect! Using Torchlight also meant I could pre-render the highlighting, massively reducing the weight of scripts I was making the end user download. I'm all about performance, so I was sold. There's a JavaScript client library but it had no docs, so I had to spend a lot of time in the source code figuring out how everything works.

To get started, you import the Torchlight constructor and call the init() method. The Torchlight constructor takes 3 arguments, a configuration object, a cache class, and a boolean of whether to perform a re-initialisation if Torchlight has already been set up. The library offers a makeConfig() helper function, which looks for a torchlight.config.js file in the root directory if it is not passed a configuration object. For my case, I was happy with the defaults Torchlight sets, but wanted to change the theme, so opted to pass this through the function rather than created a config file. I took an informed guess from the options that are available to the other libraries. Torchlight has various themes available which are documented here. Torchlight offers 2 caching mechanisms, a file-based cache, and an in-memory cache. As this site is statically generated, an in-memory cache wouldn't be any good for me, so I set up a file cache. Again, the library provides a handy helper, allowing you to call new FileCache or new MemoryCache. The FileCache constructor takes a configuration object allowing you to specify the directory in which to cache the files. For each request to the HTTP API, Torchlight stores the response in a JSON file in this directory. When making a request, Torchlight looks in the cache directory first, checking the expiration time, before making a new request to the HTTP API if needed. I omitted the force argument as this is set to false by default.

import { makeConfig, torchlight, FileCache } from '@torchlight-api/client'

torchlight.init(
  await makeConfig({
    theme: 'dark-plus'
  }),
  new FileCache({
    directory: './.torchlight-cache'
  })
)

I installed the netlify-plugin-cache package which allows you to specify directories that Netlify should cache. This means that the .torchlight-cache directory persists between builds. I added the following to my netlify.toml.

# netlify.toml
[[plugins]]
package = "netlify-plugin-cache"
  [plugins.inputs]
  paths = [
    ".torchlight-cache"
  ]

Now that Torchlight was set up, I actually needed to pass code to it to highlight. There's another class, block, which you create for every block of code you want to highlight. You push these into an array and then call torchlight.highlight() on it, which returns you an array of highlighted blocks. The block constructor takes 2 arguments, the code, and the language. I've added CSS classes to all my code blocks now, so can grab the language from there. I'm using the Cheerio library to parse the WordPress post content, so fetching a class is very simple. I also add the language as a data attribute to the code block, so I can display it with CSS. WordPress's post content is HTML encoded, so I use the he library to decode it before adding it to the block. This allows HTML inside code blocks to be formatted correctly. The block class generates a unique id, which we set on the <code> element to enable us to update its content once we have received the highlighted code.

import { load } from 'cheerio'
import he from 'he'
import { Block } from '@torchlight-api/client'

$('pre code').each((i, code) => {
  const $code = $(code)
  let language = $code.parent().attr('class').split(' ').find((className) => className.startsWith('language-')) || null
  if (language) {
    language = language.replace('language-', '')
    $code.parent().attr('data-language', language)
  }
  const torchlightBlock = new Block({
    code: he.decode($code.html()),
    language
  })
  torchlightBlocks.push(torchlightBlock)
  $code.attr('data-torchlight-id', torchlightBlock.id)
})

Now that we have created each block ready for highlighting, we can make the request to the Torchlight API. The JS library has some features to optimise requests, such as sending chunks, so we use the helper function torchlight.highlight() here. We then loop through each highlighted block and update the HTML with the highlighted version, notice how the ID comes in handy here for selecting the correct code block.

const highlightedBlocks = await torchlight.highlight(torchlightBlocks)

highlightedBlocks.forEach((highlightedBlock) => {
  const $code = $(`pre code[data-torchlight-id="${highlightedBlock.id}"]`)

  $code.parent().addClass(highlightedBlock.classes)

  $code.html(highlightedBlock.highlighted)
})

That's it, code snippets have been highlighted in a performant and accurate way. You can see them in action on this very post. I obviously have a lot of custom styling but this is personal preference. The nice thing about Torchlight is that all the styling for the highlighting is done inline, so no need to include any other stylesheets and worry about theming, just change the config property. I do like my implementation of the language identifier though, which you can see a snippet of CSS for below:

pre.torchlight {
  position: relative;
  &:before {
    position: absolute;
    top: .75rem;
    right: 1.25rem;
    z-index: 10;
    color: #858585;
    transition: color .25s ease-out;
    content: attr(data-language)
  }
}

Going back to the original blog post that introduced me to Torchlight, I did want to have a go at adding copy-all functionality to my code snippets. The blog's example was using Statamic and Alpine, so I had to adapt it to my Nuxt use case. The copyable Torchlight config option did nothing for me, but the original code was available in the response object under the code key. The basic idea is to add a container to each code block which contains text to be displayed when a user clicks the copy button, the copy button itself, and the raw code to copy. I could again use the he library to encode the code so that it would display correctly inside of the <code> element.

$(`
<div class="copy-button__container js-copy-to-clipboard-container">
  <div class="copy-button__text js-copy-to-clipboard-notification">Copied!</div>
  <button
    type="button"
    title="Copy to clipboard"
    class="copy-button__button js-copy-to-clipboard-button"
  >
  </button>
</div>
<span class="torchlight-copy-target js-copy-to-clipboard-target" style="display: none">${he.encode(highlightedBlock.code)}</span>
`).appendTo($code.parent())

I then wrote some JavaScript for the client side to handle clicking the copy button. I've been using classes for this type of thing a lot at Rareloop and it's really helped me write cleaner code. I'm passing the registerCopyToClipboardContainers() function to a Vue mixin, but it could be used quite easily in vanilla js.

class CopyToClipboard {
  constructor (container) {
    this.container = container
    this.notification = this.container.querySelector('.js-copy-to-clipboard-notification')
    this.button = this.container.querySelector('.js-copy-to-clipboard-button')
    this.textToBeCopied = this.container.parentElement.querySelector('.js-copy-to-clipboard-target')

    this.button.addEventListener('click', () => {
      navigator.clipboard.writeText(this.textToBeCopied.innerHTML)
      this.notification.classList.add('copied')
      setTimeout(() => {
        this.notification.classList.remove('copied')
      }, 2000)
    })
  }
}
function registerCopyToClipboardContainers () {
  document.querySelectorAll('.js-copy-to-clipboard-container').forEach((element) => {
    return new CopyToClipboard(element)
  })
}

The CSS is then as simple as below. We hide the raw code, and the copied notification, position the icon at the bottom right of our <pre> element, and show the copied notification when the .copied class gets applied.

.torchlight-copy-target {
  display: none
}
.copy-button__container {
  position: absolute;
  right: 1.25rem;
  bottom: .75rem;
  z-index: 10;
  display: flex;
  gap: .5rem;
  align-items: center;
  height: 2rem;
  color: #858585;
  transition: color .25s ease-out;
  &:hover {
    color: white;
    .copy-button__button {
      background-color: white
    }
  }
}
.copy-button__text {
  opacity: 0;
  transition: opacity .25s ease-out;
  &.copied {
    opacity: 1
  }
}
.copy-button__button {
  width: 2rem;
  height: 2rem;
  padding: 0;
  background-color: #858585;
  border: 0;
  cursor: pointer;
  mask-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' fill='none' viewBox='0 0 24 24' stroke-width='1.5' stroke='currentColor'%3E%3Cpath stroke-linecap='round' stroke-linejoin='round' d='M11.35 3.836c-.065.21-.1.433-.1.664 0 .414.336.75.75.75h4.5a.75.75 0 00.75-.75 2.25 2.25 0 00-.1-.664m-5.8 0A2.251 2.251 0 0113.5 2.25H15c1.012 0 1.867.668 2.15 1.586m-5.8 0c-.376.023-.75.05-1.124.08C9.095 4.01 8.25 4.973 8.25 6.108V8.25m8.9-4.414c.376.023.75.05 1.124.08 1.131.094 1.976 1.057 1.976 2.192V16.5A2.25 2.25 0 0118 18.75h-2.25m-7.5-10.5H4.875c-.621 0-1.125.504-1.125 1.125v11.25c0 .621.504 1.125 1.125 1.125h9.75c.621 0 1.125-.504 1.125-1.125V18.75m-7.5-10.5h6.375c.621 0 1.125.504 1.125 1.125v9.375m-8.25-3l1.5 1.5 3-3.75' /%3E%3C/svg%3E");
  mask-size: contain;
  mask-position: center;
  mask-repeat: no-repeat;
  transition: background-color .25s ease-out
}

The final code for highlighting my post content with Torchlight is below:

// modules/prepare-wordpress-content/convert-code.js
import { load } from 'cheerio'
import he from 'he'
import { makeConfig, Block, torchlight, FileCache } from '@torchlight-api/client'

export async function syntaxHighlightCodeWithTorchlight (postContent) {
  const regex = /<pre.*?>/g
  const matches = postContent.match(regex)
  if (!matches) {
    return postContent
  }
  const $ = load(postContent, null, false)
  $('pre').removeClass('wp-block-code')

  torchlight.init(
    await makeConfig({
      theme: 'dark-plus'
    }),
    new FileCache({
      directory: './.torchlight-cache'
    })
  )
  const torchlightBlocks = []

  $('pre code').each((i, code) => {
    const $code = $(code)
    let language = $code.parent().attr('class').split(' ').find((className) => className.startsWith('language-')) || null
    if (language) {
      language = language.replace('language-', '')
      $code.parent().attr('data-language', language)
    }
    const torchlightBlock = new Block({
      code: he.decode($code.html()),
      language
    })
    torchlightBlocks.push(torchlightBlock)
    $code.attr('data-torchlight-id', torchlightBlock.id)
  })
  const highlightedBlocks = await torchlight.highlight(torchlightBlocks)

  highlightedBlocks.forEach((highlightedBlock) => {
    const $code = $(`pre code[data-torchlight-id="${highlightedBlock.id}"]`)

    $code.parent().addClass(highlightedBlock.classes)

    $code.html(highlightedBlock.highlighted)

    $(`
    <div class="copy-button__container js-copy-to-clipboard-container">
      <div class="copy-button__text js-copy-to-clipboard-notification">Copied!</div>
      <button
        type="button"
        title="Copy to clipboard"
        class="copy-button__button js-copy-to-clipboard-button"
      >
      </button>
    </div>
    <span class="torchlight-copy-target js-copy-to-clipboard-target" style="display: none">${he.encode(highlightedBlock.code)}</span>
    `).appendTo($code.parent())
  })
  return $.html()
}