DEV Community: Jonathan Yeong

Setting up dev containers in Rails

Jonathan Yeong — Sun, 30 Jun 2024 00:00:00 +0000

Development containers, aka dev containers, have been around since 2019. Made by Microsoft, they allow you to use a container as a development environment. If you've used GitHub Codespaces, you've loaded a dev container. In Rails 7.2, there will be a new devcontainer flag that will generate a dev container folder for you. Ever since I wrote about Rails dev containers, I've been eager to try them. If you also want to be an early adopter, here are a few issues I ran into.

Installing libraries

Before I can get dev containers working locally, I need to install a few libraries and extensions.

VSCode dev container extension, which will run my application as a dev container inside VSCode.

Colima, which is my container runtime.

brew install colima

Docker to run my containers.

brew install docker

Docker compose, which is needed to run the dev container compose.yaml file.

brew install docker-compose

I also updated my ~/.docker/config.json to add the plugin

"cliPluginsExtraDirs": [
    "/opt/homebrew/lib/docker/cli-plugins"
]

Setting up a Rails dev container

First off, I created a new Rails API app with the devcontainers flag:

rails new dev-containers-app --main --database=mysql --devcontainer

After opening the project in VSCode, you should get a popup asking if you want to Reopen in container. If you don't, you can use command palette (cmd+shift+p for mac) and type dev container: Reopen in Container.

Issues with dev container

When I first ran the dev container, the bin/setup step erroredr:

ActiveRecord::ConnectionNotEstablished: Can't connect to local server through socket '/tmp/mysql.sock'

The issue is that config/database.yaml is trying to read a mysql.sock file that doesn't exist in the container. The easiest fix is to connect to the DB through the network. Modify your config/database.yaml file with:

default: &default
  adapter: mysql2
  encoding: utf8mb4
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  username: root
  password:
  host: mysql
  port: 3306

After this change, I was able to get my dev container up and running. However, when I tried to do Git things like git status, I got an error:

fatal: detected dubious ownership in repository at '/workspaces/dev-containers-app'
To add an exception for this directory, call:
    git config --global --add safe.directory /workspaces/dev-containers-app

To resolve this issue, I updated the devcontainer.json file. Adding a postStartCommand.

{
    ... // Rest of config
    "postStartCommand": "git config --global --add safe.directory ${containerWorkspaceFolder}"
}

Initial thoughts

Here's what I loved:

Modifying files in the container also modified the files on my local machine. And vice versa! It's a fast development workflow
Git just works! It pulled my gitconfig from my machine, and I could push verified commits from the container.
Since the app runs in a container, I don't need to worry about dependency management any more. If I need a new dependency, I can pull it into my docker image.
It works out of the box with VSCode dev container extension.

Here's what I didn't like:

You can't run --skip-bundle and generate a .devcontainer folder. Maybe someone can correct me if I'm wrong. I would like to not have to install anything locally if possible!
The best dev experience is with VSCode. If you don't use that editor, your workflow would be more involved. You would want to use the devcontainers/cli.
If you have an underpowered machine, dev containers might not be for you. If your container run time isn't beefy enough with RAM & CPU a-plenty, then your dev container might be sluggish.

GitHub repo for reference.

Dynamic Nested Forms with Rails and Stimulus

Jonathan Yeong — Wed, 19 Jun 2024 00:00:00 +0000

I've been away from Ruby on Rails for a couple of years. And in that time, Rails introduced Hotwire, and Stimulus. In this post, I extend my Rails nested from tutorial by adding a button to dynamically add new nested form elements using a Stimulus Nested Form component.

We're going all in on Rails, so I'll be using the default importmap JS package manager.

Set up

Let's first install the Stimulus nested form component

bin/importmap pin @stimulus-components/rails-nested-form

Update app/javascript/controllers/application.js to register the stimulus component.

import { Application } from "@hotwired/stimulus" // Should exist already
import RailsNestedForm from '@stimulus-components/rails-nested-form'

const application = Application.start() // Should exist already
application.register('nested-form', RailsNestedForm)
...

Let's modify our previous form by moving our nested fields into a partial. First create the partial app/views/training_sessions/_training_step_fields.html.erb and add the following code:

# app/views/training_sessions/_training_step_fields.html.erb 
<%= f.label :description %>
<%= f.text_field :description %>

After we create our partial, we can modify the form in new.html.erb:

# app/views/training_sessions/new.html.erb
<%= form_with model: @training_session do |form| %>
  Session Steps:

  <%= form.fields_for :training_steps do |training_steps_form| %>
    <%= render partial: "training_step_fields", locals: { f: training_steps_form } %>
  <% end %>

  <%= form.submit "Create New Session" %>
<% end %>

At this point, when we go to training_sessions/new we should see the same behaviour as before.

Implementing Stimulus Nested Form Component

Let's implement the Stimulus component. Modify your new.html.erb to add a Stimulus target:

<%= form_with model: @training_session do |form| %>
  Session Steps:
  <template data-nested-form-target="template">
    <%= form.fields_for :training_steps, TrainingStep.new, child_index: 'NEW_RECORD' do |training_steps_form| %>
      <%= render partial: "training_step_fields", locals: { f: training_steps_form }%>
    <% end %>
  </template>

  <%= form.fields_for :training_steps do |training_steps_form| %>
    <%= render partial: "training_step_fields", locals: { f: training_steps_form } %>
  <% end %>

  <div data-nested-form-target="target"></div>

  <button type="button" data-action="nested-form#add">Add Training Step</button>

  <%= form.submit "Create New Session" %>
<% end %>

Now, if we reload our app, there's a button that lets you add more training steps 🎉.

Breaking down the changes

Let's break down the changes.

<template data-nested-form-target="template">
...
</template>

<div data-nested-form-target="target"></div>

We register two targets with Stimulus, template and target. The template target, which in Stimulus, gets referred to as this.templateTarget is inserted into this.targetTarget via insertAdjacentHTML. The template is defined similar to our original definition of the training_step fields. But it has two additional params: TrainingSteps.new and child_index: 'NEW_RECORD'

<%= form.fields_for :training_steps, TrainingStep.new, child_index: 'NEW_RECORD' do |training_steps_form| %>
  ...
<% end %>

Rails requires the index of nested fields to be unique. It does enforce uniqueness via a child_index. See the Rails docs. We must keep the NEW_RECORD value. Stimulus nested form component will replace the term NEW_RECORD with Date.getTime().toString().

TrainingStep.new is used to build a new instance of the Training Step model. Without this model, we'll see some odd behaviour where we add two fields every time we press the "Add Training Step" button. Note: the two comes from our controller: 2.times { @training_session.training_steps.build }.

Fun fact : even though two form elements get added, only one of those elements gets saved! Can you guess why? That's right, it's because the child index for those elements are the same. Rails will only save the last of the two values.

For reference, here's the component source code.

At the time of writing:

Rails 7.1
Ruby 3.3.2
Stimulus Nested Form Component 5.0.0
Stimulus 3.2.2

Tools I use daily as a solo dev

Jonathan Yeong — Tue, 11 Jun 2024 00:00:00 +0000

I was inspired by Nick Taylor's post "Tools that keep me productive", to document tools I use daily. These tools help me build and showcase projects, stay organised, and keep me productive. Most of these tools are cross-platform, at least between macOS and Linux.

Editor + Terminal

VSCode is my main editor. I've been playing around with Zed, but it doesn't have the plugin ecosystem (yet) for me to switch over. I'm using the Dracula theme, and SF Mono Powerline font.

Bonus tip: if you're livestreaming, you can enable screencast mode in VSCode to have it show keystrokes. If you want to turn off all keystrokes, go to settings, search for keyboardOptions, and uncheck show raw keys. After that setting is turned off, it will only display modifier keys.

iTerm2 is my terminal of choice. I use zsh as my shell, with oh-my-zsh installed to manage plugins for zsh. Starship is my prompt, it shows valuable information on a project out of the box. Finally, I added z to jump around to different folders from anywhere.

asdf is my plugin manager. It can manage versions of multiple different languages.

Screen capturing

CleanShotX (paid) is what I use to capture screenshots, videos, and GIFs. It's a new tool in my workflow, so I haven't had a chance to fully use it yet.

QuickTime player was what I was using to screen record before CleanShotX. It's a free tool that comes with macOS. The screen recording feature displays mouse clicks and allows you to select a window or area to record.

Design

Figma is my go-to design tool. It has a free plan that gives you a limited number of projects and pages. It also has powerful features like auto-layout, and components which make building designs faster. The only downside is that you have to pay monthly for multiple projects.

Penpot is a free, open-source Figma alternative that I've been playing around with. I don't have the same muscle memory as Figma, and it's browser only. But it seems like a promising free alternative.

Excalidraw is my whiteboarding / brainstorming tool of choice. I love Excalidraw. I have a whole post about it! Figma is a too heavy for rough wireframing or brainstorming. That's where Excalidraw shines.

Note-Taking

Logseq for learning new concepts, or taking quick notes. I dump all my knowledge into Logseq and let references between those notes form over time. There's no real concept of folder structures, which means I don't need to spend a lot of time figuring out where notes should go. It also comes with a built-in query language, which can help you find information. My favourite part about Logseq is its unlinked references. Meaning you don't need to explicitly add references to things, Logseq can do it for you.

Obsidian for long form writing, like blog posts. I have a blog post workflow in Obsidian. It's also where I write journal entries or do my goal setting. Both Obsidian and Logseq store notes locally, so I can write offline.

Notion for documentation that needs some kind of UI (like a table of contents), that I won't need if I'm offline. It's where my brag doc lives.

Project management

GitHub Projects is how I manage all my projects. I've set up a single GitHub Project that pulls in issues from various GitHub repos. It gives me an overview of all the things that I want to do, and then lets me drill down into what I should do next.

Slow productivity framework

Jonathan Yeong — Mon, 10 Jun 2024 00:00:00 +0000

Years ago, I was fully hooked to the world of productivity. At the time, I was getting up at the crack of dawn to get as much done. I used time blocking to try to maximise every hour of my day. Working a full-time job, then spending 4–5 hours outside the job working on various side projects. I would cut out any activity that I deemed "unproductive". And when something unexpected came along that interrupted my time block, it caused anxiety.

Only recently did I realise that this productivity mindset was toxic. Yes, it took me a pandemic and also burning out to realise that toxic productivity exists. Gaslighting myself is my forte. To this day, I'm still battling with the feeling of not doing enough. That there aren't enough hours in the day to be "productive".

To have a healthier mindset, I needed a new "productivity" framework. I wanted a framework that would help me prioritise what to do outside my 9-5 job. Something that would let me roll with the punches when the unexpected came up. And would help me feel good about what I chose to spend time on. So I present to you my "slow productivity" framework.

When I'm trying to decide what to do. I have to answer three questions:

Will I have fun?
Will I learn something?
Will I regret it if I didn't do it?

If I say yes to any one of these questions, I should do the thing and not feel guilty about doing it. Some activities are more worth doing than others, I can tell if it's a yes to multiple questions. But if it's a no to all questions, then I won't do the activity.

Here's an example of how I applied the framework this morning.

I woke up feeling refreshed, which rarely happens. I wanted to immediately start working on a side project before work. It's something that I would have fun with and learn something. Ticking two of the three questions. This task is something that I would consider productive.

Now I sit at my desk, turn on my laptop, and get ready to code. But my cat jumps up on my table and wanted pets. She was far too cute, and demanding, to ignore. There went my productive morning time. In the past, I would've been annoyed and felt guilty that I was doing something unproductive. But today, I ran through the three questions. Am I having fun? Yes. Will I regret it if I didn't do it? 100% yes. Always pet the cat if it offers.

Sometimes unexpected events happen that interrupt your productive task. With this framework, I feel less anxiety and guilt around these events. It might not work for everyone, but I hope it helps you feel better about doing the "unproductive" activities.

Advice for Effective Developers

Jonathan Yeong — Fri, 07 Jun 2024 01:24:52 +0000

Syntax FM released a podcast episode 11 habits of highly effective developers yesterday, it's filled with anecdotes, and great advice. If you haven't already, give it a listen! Scott Tolinksi and Wes Bos shared these 11 habits:

Understand stakeholders and business goals
Have an open mind about new tech
Help others with problems
Understand work-life balance
Pay attention to details
Be curious and always learning
Ask for help when needed
Have fun with development
Be empathetic to users and coworkers
Be part of the developer community

While listening, I made a mental note to keep practising these habits, I loved them! Especially, having fun with development. Usually, I see programming as a way to solve business problems. I forget that it can also be used to make fun things!

After the podcast ended, I thought about what advice would I have given my past self to be a more effective developer? So I made a list! I'll add to it over time, and I hope you'll find the advice useful.

Naming is hard. Understanding the domain of your application can help. If possible, err on being specific with your name rather than generic.

Build your pattern matching muscle. Not regex (although that can be useful), but patterns with solving problems. Watch how others solve problems, debug, and write code. Go through the same process yourself over and over again.

Pair frequently and with different people. The benefits of pairing greatly outweigh working solo on complex issues.

"For each desired change, make the change easy (warning: this may be hard), then make the easy change" - Kent Beck

Find the most straightforward path to solve a problem. If this path doesn't exist, can you make it happen? Only when you've exhausted your options should you look at more complicated solutions. I find this advice the hardest to follow.

Introspect often and stay away from autopilot mode. After finishing a tutorial, take the time to understand what you've learnt and how you could use that knowledge in other ways.

Pursue what is interesting to you. Drown out the hubbub, the trends, the absolutists. If you don't know what fascinates you yet; experiment.

Go broad on a topic before going deep. This advice ties in with the one above. Experiment until you find something, then go deep on it. Going broad diversifies your mental models. Going deep strengthens your models.

Have a career document aka brag document. Use it to show your impact at your role. Save all the good stuff people say about you or things that you're proud of. Look at it whenever you're feeling down or feeling like an imposter.

Practice saying no. Overcommitting is stressful. Burnout is real, take the time to care for yourself.

You don't need to write blog posts, produce content, or commit to open source to be a successful developer. But it can be a nice way to give back to the community.

Making a custom kebab case method for pretty URLs

Jonathan Yeong — Sat, 01 Jun 2024 00:00:00 +0000

Recently, I was working on a Rails app where I wanted to make some pretty URLs for it. I have a route where you can view a Dog resource GET /dog/1. Having an ID in the URL is not what I want. People could increment the ID to potentially see other dogs. But more importantly, it's not personal! I wanted the URL to show your dog name instead, like /dog/albie or /dog/st-george-bernard-the-3rd. I needed a way to make pretty kebab case URLs.

Pretty kebab case URLs

To get a URL that's not the default id, I added the to_param method to my model. Where slug was a column on my Model:

def to_param
  slug
end

Since I wanted the slug to be auto-generated, I had to transform the name column to kebab case (see below). Much to my surprise, there wasn't a Ruby / Rails kebabcase method:

class Dog < ApplicationRecord
  before_create ->(dog) { dog.slug = dog.name.kebabcase } # .kebabcase doesn't exist

  def to_param
    slug
  end
end

Looking at the docs, I found .underscore and .dasherize. Unfortunately, they only work for camel case names:

"DogName".underscore.dasherize
# => "dog-name"

But who names their dog in camel case? Maybe people do, but I'm assuming most people use spaces. Unfortunately, .underscore only works on camel case strings.

"Dog Name".underscore.dasherize
# => "dog name"

Searching for a Gem

If Ruby and Rails didn't have a kebab case method. I'm sure there's a gem out there that does. In comes Strings::Case to the rescue:

strings.kebabcase("PostgreSQL adapter")
# => "postgre-sql-adapter"

I could've called it a day here, but looking at the gem, it has so many other methods that I wouldn't use. Honestly, it seems like overkill for my tiny side project. It's also another library I would have to maintain and make security updates to. In the end, I decided to roll a custom kebab case method.

Rolling a custom kebab case method

Before jumping into a solution, I wanted some inspiration from Strings::Case.

Under the hood, the gem scans through a string, building words determined by whether there is a specific delimiter or if there's an uppercase. "DogName" would be two words "Dog" and "Name" since there's an upper case in the middle. And "Dog Name" would be two words, since there's a space in the middle. These words get put into an array, and after the string has been scanned, a .join method is called with a separator. In kebab case, the separator would be -. I'm oversimplifying this approach. If you're curious, about the source code, you can view it here.

For the gem, this process makes total sense. It handles many use cases as well as includes multiple string manipulation functions. Since I only wanted kebab case, I decided to go with a regex approach:

dog.name.downcase.gsub(/[^\w\d\s]/, "").gsub(/\s/u, "-")

Let's break down what this is doing:

.downcase - A built-in function that makes the string lowercase.
.gsub(/[^\w\d\s]/, "") - Regex that matches anything that's not a character, digit or space, i.e. any special characters and removes it.
.gsub(/\s/u, "-") - Regex that matches a space and replaces with a dash

"St. George Bernard the 3rd".downcase.gsub(/[^\w\d\s]/, "").gsub(/\s/u, "-")
# => st-george-bernard-the-3rd

The biggest limitation with my approach is the regex. I make plenty of assumptions with what I think names should be, and it's focused primarily on English. But what if someone uses a character language like Japanese?

"餅".downcase.gsub(/[^\w\d\s]/, "").gsub(/\s/u, "-")
# => ""

If I wanted to make this more robust, I should handle non-English characters properly. If I can, I could transliterate it. Like é becomes e. If it's in a character language, I could leave it because non-English characters can be used in URLs.

However, for a small side project, what I have is enough. And I didn't need to pull in a new gem to achieve it.

Using Ruby 3.2.0 and Rails 7.1.3.3 at time of writing

What I missed about Ruby

Jonathan Yeong — Wed, 29 May 2024 00:00:00 +0000

I'm back in Ruby on Rails land after two years using only Node/TS. And I'm thrilled to be back! I feel like I missed so much progress with Rails 7 over the last 2 years. But now that I'm back, I can finally use all the things I missed. Here's all the things that I took for granted before my break:

Convention is enforced in Rails. Organization is much easier in a rails project, and I'm able to hop into any Rails project and find my way around.
Some magic is nice. Such as auto-loading files!
Rails console!!! Omg being able to run code snippets is amazing. I downloaded RunJS to be able to achieve the same thing. But it's not a replacement for being able to play around with the data along with your code.
Mocking with RSpec is way easier than Jest. In Node/TS, we used dependency injection to be able to mock classes and methods. But it was annoying to use dependency injection when it was only used in testing. Please let me know if there's a better way to do it.
Having Rails routes to get an overview of all the entry points to the app is so underrated.

One thing that I took away from the last two years, is typing. I'm not going to lie. I like it. It's nice to be able to hover over an object and know what fields exist in it. I would love with Ruby to be able to hover over a hash and see all the keys that exist. I added Sorbet and RBS to my things to learn list.

Adding Topics (aka tags) in Astro

Jonathan Yeong — Tue, 11 Jul 2023 00:00:00 +0000

This month's Virtual Coffee challenge is Build in Public. And I wanted to kick things off with some website updates. I've been meaning to add tags to my site for a long time. After poking around at other people's sites, I decided to switch from the concept of tagging to the concept of topics. While the naming is different the process to adding topics was the same. In this post I want to share how I added topics, what I learnt, and what's next.

How I added topics to my site

I mainly followed the Astro guide on building a tag index page.

Firstly, I created this structure in my project:

pages/
    topics/
        [topic].astro
        index.astro

index.astro is rendered when you go to /topics (example link).

[topic].astro is rendered when you go to /topics/:topic. Where :topic is something I define in my blog posts. Example, topics/api%20design/ (example link).

I then added the topics field to my front matter yml in my blog posts:

---
pubDate: 2023-04-05
updatedDate: 2023-04-05
title: "template"
description: "A template"
draft: true
topics: []
---

Added topics key in my config.ts file (source code)

topics: z
    .array(z.string())
    .optional()
    .default([])

Finally, to display the topics on my blog post, I made a component that is pulled into the Blog Post Layout:

BlogPost.astro

...
{topics && topics.length > 0 && <TopicList topics={topics} />}
...

TopicList.astro

---
const { topics } = Astro.props;
---  

<p class="flex space-x-3">
{topics.map((topic: string) => (
    <a href={`/topics/${topic}`}>{topic}</a>
))}
</p>

Deep dive (aka what did I learn)

Digging into the index.astro and [topic].astro files a bit more.

index.astro

The way we get a list of topics to display on the /topic page is through this code (source code):

const posts = (await getCollection('blog'))
const topics = [...new Set(posts.flatMap((post) => post.data.topics))];

This line will get all my posts from my blog folder, pull out all of the topics, and return an array of topics with no duplicates.

[topic].astro

To generate the corresponding topic endpoints like /topic/api%20design/ I needed to modify the getStaticPaths() function in the [topic].astro. (source code). I forgot to modify it originally, and I was very confused when I kept getting a 404 not found error when trying to navigate to a topic page 🤦🏻!

export async function getStaticPaths() {
    const posts = (await getCollection('blog'))
    const topics = [...new Set(posts.flatMap((post) => post.data.topics))];
    return topics.map((topic) => ({
        params: { topic }
    }));
}

getStaticPaths() is an Astro function that Astro uses to generate each page at build time. It returns an array that looks something like this:

[
    { params: {topic: "API Design"} },
    { params: {topic: "Productivity"} }
]

What's next

I'm not super happy with how my topics are styled. But I'm also not sure how I would style them - I'm open to suggestions! Next up for me is writing an About me page. I don't know why but I find it incredibly hard to write! I'll be chugging away at it while trying to come up with a better styling in the meantime.

The Art of Determining the Right Amount of Testing

Jonathan Yeong — Sun, 11 Jun 2023 00:00:00 +0000

What is the right amount of testing?

Over the years my answer to this question has changed drastically. From fully embracing integration testing and requiring code coverage numbers on Pull Requests (PRs). To writing only unit tests covering the critical paths of the application. Leaving much of code untested.

Of course there's no definitive answer for the right amount of testing. If there was, everyone would be doing it. Instead, I'd like to share what I consider a good default amount of testing. But before diving into that, let's discuss the problem we're trying to solve.

The primary goal of testing is to prevent bugs from reaching production. Too little testing leads to a brittle application. As more features get pushed out, more fires need to be fought. And eventually you're worried that any change might cause something unrelated to break.

In response, we swing the pendulum to the other side. We write extensive suites of integration tests that get run on every PR. Code coverage numbers need to be met before a PR can be approved. We aim to test every possible path, leaving no room for bugs to slip through. Unfortunately, our feedback loops begins to slow down. And meaningful features take longer to be released.

Both of these are extreme ends of the spectrum. Too little testing leads to brittle applications. Exhaustive integration tests slow down feedback loops and delay feature releases. So what is a happy middle ground? Here's what I think we should default to.

Write unit tests for all your changes. Testing error paths but not every combination of errors/inputs. Pick your edge cases that cover critical aspects of your code. Here's some examples:
- Test paths that have a complex outcome and/or an error you display to the user.

  if condition
    complex_outcome
  else
    error_to_user
  end

In general, you don't need to test well-established, trusted libraries. But you should manually validate library calls (see more on manual testing below).

  # Don't test that Rails will save a model to the database.
  model = Model.create
  # Don't test S3 library will delete the bucket
  s3Client.delete_bucket({  bucket: "bucketName", })

Test for public not private methods. Unexpected changes to private methods should be caught in the public method tests.

  # Test this method
  def public_method
    something = do_something
    do_something_with(something)
  end

  private

  # Don't test these methods
  def do_something
    ...
  end

  def do_something_with
    ...
  end

Test suites should run fast enough that by the time you make a coffee your build is complete (faster is better). That means unit tests over integration tests.
If possible, always manually test your change. Keep the process lightweight and repeatable by writing down your steps. Don't test the entire app behaviour on every change. But allow someone on your team to vet your change by running through the testing steps you wrote down. If you find yourself testing the same thing over and over again, look to automate it (i.e. write a unit test).
Manually testing is especially important if testing a third party service/api. Mocking in unit tests is needed for fast tests but you need to validate your mocks in the real world. Validating third party services/api is the purpose of integration tests. But arguably, a quick manual test may be all that's needed along with good observability.
Bonus points if you have a cloud developer environment or a developer preview on your PR that allows someone to validate your changes without having to run your code on their machine. This approach also promotes collaboration between your team!
Finally, have a robust CI/CD pipeline that lets you rollback bad builds, or feature flags to quickly turn off a breaking change. Having the right monitors in place will tell you if something is broken before it causes too much damage. Expect that bugs will happen even with a proper amount of testing.

While there's no one size fits all solution for the right amount of testing. I've found that these defaults have worked well in the past. They strike a balance between maintaining healthy developer feedback loops while minimizing game breaking bugs that make it to production.

Developer productivity with Github Codespaces

Jonathan Yeong — Fri, 09 Jun 2023 01:27:01 +0000

During my time at Shopify, I had the opportunity to work with a cloud development environment - and I was sold. Since then, I set out trying to recreate a similar experience for my personal development. Enter, Github Codespaces. I'd like to share my thoughts on using Codespaces and why I believe it will greatly benefit developer productivity.

Exploring Codespaces: My Motivations

There are a couple of reasons why I decided to explore GitHub Codespaces:

Escape from Bare Metal Setup: I'm tired of installing and managing specific versions of Ruby, Node, Postgresql across my various projects.
Onboarding Developers: one of my aspirations is to run my own company, I wanted to explore how cloud development environments like Codespaces could facilitate the smooth onboarding of new engineers.

The Github Codespaces Workflow

Firstly, install the Github CLI tool. Being able to use the terminal over the browser has significantly streamlined my experience. Once authenticated with the Github CLI, my workflow is as follows:

Creating a Codespace: Initially, I needed to create a Codespace. Note that if you cloned a project and you're in that folder, the Github CLI tool will use that project as the default

gh codespace create

Getting to Work: To start working on my application, I ran the following command, selecting the Codespace I had just created. There's a short delay as the environment is being set up:

gh codespace code

Astro Aside: Since I'm running the Codespace for my Astro site, I made an update to the dev command in the package.json file:

astro dev --host

This configuration binds the Astro local server to any IP, allowing proper connection when Codespace forwards the localhost port.

Codespace Logging: I also experimented with Dotfiles for my Codespace, which customize the Codespace developer environment. The Codespace logs proved invaluable for debugging my changes as the Codespace started:

gh codespace logs

Initial Considerations

Based on my early experience, here are a few considerations regarding Codespaces:

Overkill for Simple Projects: Codespaces is overkill for simple projects like my Astro blog. Given the minimal version dependencies and longer startup time compared to setting up my Astro project directly, it doesn't give that much benefit.
Treat Codespaces as Ephemeral: It's important to treat Codespaces as ephemeral environments. That means commit often! Any uncommitted changes will be lost when the Codespace is destroyed (which occurs by default every 30 days).

The Untapped Potential

I'm totally bought into the idea of cloud development environments. Github Codespace is one option there's also others such as Gitpod.

Imagine this situation, you're starting a new job, excitedly setting up a fresh codebase, only to encounter a mysterious, undocumented error. You turn to your onboarding buddy for help. They speak the dreaded words, "weird - works on my machine". A response that leaves you navigating a labyrinth of debugging all by your lonesome. Okay I'm being a little dramatic, but we've all experienced these "works on my machine" moments that can send you spiraling.

With a cloud developer environment like Codespaces, someone can get their project up and running in minutes rather then days. Think about the time and headache saved! These environments can be used for more then onboarding too. You could use them as developer previews, sharing a URL of your app running in the Codespace so anyone can test your changes.

While I haven't found a project to fully utilize Codespaces yet, I'm psyched to keep exploring it. Stay tuned, I'll keep this post updated.

The Tale of the Whitespace

Jonathan Yeong — Sun, 26 Sep 2021 00:29:40 +0000

Can you tell the difference between " " and " "? I certainly can't. But the computer can.

" ".ord
# => 32
" ".ord
# => 160

This whitespace led to one of the more confusing bugs that I've seen as a developer. I wanted to convert a string from Unicode to ASCII and this whitespace was causing my tests to fail. For the longest time, I trusted my eyes. It was only until I tested my assumptions that I realized that I wasn't dealing with any normal whitespace. I was dealing with a Unicode whitespace. Before I share how I solved the bug, I first want to do a deep dive into character encodings. Encodings like Unicode, ASCII, ANSI, ISO, are things we see all the time. But we hardly question, because it just works. Of course, until it doesn't.

Every character that we see on a computer screen, in your browser, or as a string object in your code, is represented as a number on the computer. To be specific, it's represented as a series of bits. This process of transforming a visual character into a number is done through character encoding. Character encoding is a map for the computer to transform this number into the character we see. It's a cipher, a key, to crack the code.

This number that we assign to a character is known as a code point. That is what " ".ord is outputting in the code above. If we take a look at a table of ASCII codes we can see that the codepoint 32 corresponds to a space in ASCII (see an ASCII table). The codepoint here is a decimal representation of the bits, if we were to break this down into binary for an ASCII encoding the bits 00100000 would map to the whitespace we see.

If we have something like ASCII doing this for us already, why do we need something like Unicode? The first manifestation of Unicode (Unicode 88) was an idea published in 1988 by Joseph D. Becker. Unicode was created to extend ASCII. Becker realized that ASCII wasn't enough to represent all languages around the world.

What is needed is a new international/multilingual text encoding standard that is as workable and reliable as ASCII, but that covers all the scripts in the world.

ASCII being a 7 bit encoding structure within an 8-bit byte could only represent 256 characters. The Unicode standard introduced three new encoding forms, UTF-8, UTF-16, UTF-32. Each of these encoding forms are interchangeable and they all extend the original ASCII encoding. UTF-8 is the most widely seen encoding format. It represents a string in a variable amount of 8 bit bytes, from one to four. If we were representing an ASCII character say A in Unicode we would have:

Encoding	Codepoint	Binary Representation	Character
ASCII	65	01000001	A
UTF-8	U+0041	01000001	A
UTF-16	U+0041	00000000 01000001	A
UTF-32	U+0041	00000000 00000000 00000000 01000001	A

See Unicode Table for A

Note: Unicode codepoints are represented via a U+ and the hexidecimal value. Using a function like .ord will convert that codepoint to a decimal.

As you can see there is a difference between the three Unicode formats. As mentioned above, UTF-8 is a variable width encoding. Where the width is between one to four 8 bit bytes. UTF-16 is also a variable length encoding but the width is one or two 16 bit bytes. And UTF 32 is a fixed width encoding that uses exactly 32 bits (four bytes). While UTF-8 might be the most popular, you might want to use UTF-16 if you wanted a balance between efficient access to characters with economical use of storage. Or UTF-32 if space wasn't a concern and you wanted faster access.

At this point, we know what character encodings are. We know that ASCII and Unicode are both types of character encoding. And that Unicode was introduced to extend ASCII so that we can represent all languages. We also know that an ASCII character can also be represented as a Unicode character, interchangeably. The ASCII bit representation is the basically the same as the Unicode bit representation. But what about the flip side to this statement? What if we want to represent Unicode characters as ASCII characters?

Transforming Unicode to ASCII

When building with Rails, there's a handy method that's found on the localization (I18n) library - .transliterate. Transliteration means to swap characters with something that looks similar. Not taking into account how the character sounds. For example, transliterating é will give e. Why would you do this? You might have a blog post with special unicode characters, but a URL isn't able to handle it. So we need to transform those characters to the ASCII approximation. When running transliterate using the whitespace example given in the beginning we have a problem.

# ASCII Space
I18n.transliterate("hëllo wōrld")
# => "hello world"

# Hard to tell but this line is a non breaking space!
I18n.transliterate("hëllo wōrld")
# => "hello?world"

The transliterate method doesn't know what to do with the unicode whitespace, which is a non breaking space (nbsp). Because it can't handle it, it will replace the nbsp with a question mark. Not something that we want. To solve this one use case, we can use regex. Unfortunately, in Ruby, the whitespace meta-character doesn't recognize non-ASCII characters (source) We can't use the regex meta-characters because they don't encompass non-ASCII characters.

# Unicode non breaking space
/\s/.match("hello world")
# => nil

# ASCII Space
/\s/.match("hello world")
# => #<MatchData " ">

Thankfully, we can use the POSIX bracket expressions to solve this issue:

# Unicode non breaking space
/[[:space:]]/.match("hello world")
# => #<MatchData " ">

# ASCII Space
/[[:space:]]/.match("hello world")
# => #<MatchData " ">

We can then use the bracket expression to substitute the non breaking space with an ASCII space.

I18n.transliterate("hëllo wōrld")
# => "hello?world"

I18n.transliterate("hëllo wōrld".gsub(/[[:space:]]/, " "))
# => "hello world"

While this solution works for unifying whitespaces. We start running into problems when we want to expand our solution. For example, the transliterator won't be able to transliterate unicode dashes. To expand this solution, I18n gives you the ability to add transliteration rules to a locale file:

# <locale>.yml
i18n:
  transliterate:
    rule:
      ü: "ue"
      ö: "oe"
      —: "-"

You can also set these rules using a Hash or a Proc. This solution is similar to how I18n.transliterate is set up. Under the hood, the transliterator has a hash that contains a certain set of Unicode to ASCII permutations (see source). If we wanted to create a transliterator based on some hash rules we can do something like this:

my_transliterator = I18n::Backend::Transliterator::HashTransliterator.new({"—" => "ASCII-DASH"})
my_transliterator.transliterate("hello—world")
#=> "helloASCII-DASHworld"

Deploying an Elixir app

Jonathan Yeong — Sat, 01 May 2021 02:16:10 +0000

I'm amazed by all the ways we can deploy an Elixir app. It's a little overwhelming (see: analysis paralysis). But it's a good problem to have. Especially now that Elixir has native support for deployment. Here are some notes that I wrote up while learning about deploying an Elixir app. Note, I didn't dive into where you should put your app. Just the deployment methods:

With Mix. You host the source code on the production server and you run a mix command to start the app, e.g. mix run --no-halt (with the corresponding flags).
Using Elixir Releases. This is a built in feature since Elixir 1.9. There's a command, mix release that will compile your elixir code and packages it into an artifact. This artifact will then be uploaded to a production server.
Using Distillery. Distillery is a library that is used to produce an Erlang/OTP release. It works with Elixir 1.6+, presumably solving release problems before 1.9 was released.

There are various features of each of these ways to release as well. When we deploy with Mix, we can run mix commands, e.g. mix phx.server. But there are some trade-offs. The source code is visible to anyone with access to the production server. And we don't have access to the Remote Observer.

Alternatively, to start our app with Elixir releases we can use a command similar to this: _build/prod/rel/my_app/bin/my_app start. With Elixir releases (and Distillery) we have access to the Remote Observer and our source code is hidden. Because we're compiling our app it also results in smaller slug sizes which means faster startup times ¹. However, we don't have access to Mix. Since Elixir 1.11, we have access to runtime.exs which is executed after the code is compiled. In a Phoenix app, this means renaming config/prod.secret.exs to config/runtime.exs and replacing use Mix.config with import Config.

Finally, a feature that Distillery has that Elixir releases doesn't is hot upgrades. With hot upgrades we can release a new version of our app without bringing the old app down. There is a lot of overlap between Distillery and Elixir releases. Distillery brings its own configuration so you shouldn't mix deployment methods. Choose one or the other. One reason you'd use Distillery over Elixir releases is if you need to deploy an Elixir app pre 1.9.

Further reading

https://gigalixir.readthedocs.io/en/latest/modify-app/index.html ↩