DEV Community: Archit Chandra

A Side Effect of Storing a Git Repository in iCloud Drive

Archit Chandra — Thu, 10 Aug 2023 13:25:52 +0000

Note: This article is part of my digital garden on architchandra.com and will keep evolving with time.

I've encountered some weird issues when trying to commit changes for my notes repository which is stored in a folder synced via iCloud. This article references iCloud but the same issues can also arise with other file sync services like Google Drive, Dropbox, OneDrive, etc.

I'm documenting my experience here in case someone ends up in my position in the future. I wouldn't put it past myself to forget my lesson and run into the same issue again (say thanks, future Archit).

Some Context First

I use Obsidian to create notes as Markdown files on my computer and use the Obsidian Git plugin to version control the changes via Git. The Obsidian vault in which I store my notes is stored in a folder synced using iCloud Drive.

I have this setup because I need to access my notes on my iPhone and two computers with the same Git credentials—my Mac Mini (daily driver) and my older yet portable MacBook Air (secondary computer). The logic is that iCloud will allow my notes to be accessible on all these devices seamlessly (courtesy of the Apple ecosystem) and Git will allow me to version-control my notes in a familiar and friendly way.

This works fine on most occasions but dives into a hellhole every once in a while.

The Problem (Fatal: Bad Object)

Sometimes, when I try to pull from or push to a remote repository, I get an error along the lines of fatal: bad object refs/heads/main 2. This is generally followed by a message that reads error: <remote-repo-name>.git did not send all necessary objects. I've also seen an error message reading fatal: Could not parse object <commit-hash> in such circumstances a couple of times.

I'm not a Git pro, so I got stumped by it the first time till I read the error message properly. And then suddenly, I went "What is this 2 doing in there?". It turns out that sometimes iCloud Drive can't properly reconcile the changes made on the same file on two different devices—iCloud's equivalent of a merge conflict. In such cases, it looks like iCloud Drive just adds a numbered suffix (like a 2) to the file modified more recently. Sometimes, there's just one suffixed file (the one without the suffix is missing or was renamed). Don't know why that happens but it creates the same kind of issue 🤷🏽‍♂️.

This might not be a huge problem in most circumstances but in my case, Git itself stops working. And the biggest purpose of using file-based notes gets thrown down the drain.

According to the error message, the files in question are in the .git/refs/heads folder, which is how Git keeps track of its HEAD reference. From what I understand, this situation violates one or both of the following rules around HEAD references:

Only one HEAD reference per branch is allowed in a repository (looking at normal repositories that we mortals use, not the fancy ones with multiple working trees). Git fails because it now encounters two HEADs.
Branch names and commit SHAs (possible HEAD references) do not allow spaces. Git fails because it encounters a space character in a HEAD reference.

The Fix That Works for Me

I would suggest taking care of a few things before tinkering with anything:

Make a copy of the .git folder in its current erroneous state so that you can get back to it if you manage to screw things up (fairly easy).
Check the state of the repository on all synced devices. If there are any uncommitted changes, make a note of them. Also, make a copy of any change that you would not like to lose. In my case, the most common culprit is the .obsidian/workspace.json file. The most common kind of change in it is information on open and recent tabs, which I don't care about much and can hence afford to lose—so I don't take a backup of this file.
Choose one computer on which you'll make changes and prevent any other synced computers from making changes while you're fixing things. You don't want your fixes to get corrupted by any conflicts getting generated from another device. For me, that means that I'll quit Obsidian on my secondary computer and only then make fixes on my primary computer.

For the fix itself, you want to have exactly one ref file per branch. If there's just one file (the one that has the suffix), removing the suffix from the filename should do it. If there are two files, delete the one with the suffix.

Sometimes, your latest change might have to be committed again. If any changes from another device are not showing up, use your backup of those changes to commit them again. After that, wait for about a minute for iCloud to sync all the changes and then open the other device to cross-check that everything is working there as expected.

Update: I recently decided to stop tracking the .obsidian/workspace.json file in Git. That has significantly reduced the frequency of such errors in iCloud.

Closing Thoughts

While many people say and Apple itself hints (suggests avoiding app folders) that it's a bad idea to keep version-controlled files in iCloud Drive, I have been using this setup more or less without hassle for the last six-seven months. It provides an essential service for my notes workflow—Git gives version control and iCloud gives syncing across devices. It's a combo that I wouldn't get easily or for free otherwise. There are, of course, services like Obsidian Sync to do the same thing but they're too expensive for someone sitting in a developing country and too opaque to trust blindly. Hopefully, it will keep working fine in the future too. 🤞🏽

Originally published on architchandra.com on July 19, 2023.

How to Specify a Node Version in Netlify Builds

Archit Chandra — Sun, 09 Apr 2023 14:29:17 +0000

Note: This article is part of my digital garden on architchandra.com and will keep evolving with time.

I keep coming across situations where I need to tell Netlify about the Node version to use while building my sites (eg. when building countmywords.in). Multiple times now, I’ve not been able to find the correct documentation related to this quickly. Guessing that I’m not the only one facing this issue, I decided to write a quick article to offer a solution.

Default Node Version

Netlify creates a Docker container to set up your site behind the scenes. The build image used to create this container defines the software versions installed (including Node).

If you don’t specify a Node version explicitly, Netlify uses the default Node version that comes pre-installed in its Docker container. Once you build a site using the default Node version, Netlify locks or pins your site to that Node version so that your site’s builds continue using the same Node version even when the default Node version in Netlify’s Docker build image changes with time. Your deployment workflow will thus not break when Netlify updates its build image to use a newer version of Node as the default.

For example, imagine that it’s late 2021 and you are using Node 16 to build your Eleventy site on Netlify. Node 16 is the latest version and it makes sense for Netlify to include it as the default Node version in its containers. Now, fast forward to March 2023. Netlify has updated the software versions included in its containers and Node 18 is now its default Node version. For some reason you were not able to update your code during this time and your site still requires Node 16. However, when you trigger a build on Netlify, your build still goes through without any errors. Because Netlify had pinned your site to use Node 16 (to match the Node version used in the previous build), it used Node 16 in the current build rather than Node 18 (which would only be used in new projects).

Explicitly Specify a Node Version

There are two main methods to specify a Node version in Netlify builds:

Set an environment variable called NODE_VERSION.
Create a .node-version or .nvmrc file that is stored in the base directory of your code repository.

You can specify the version information as the version number (full or partial) for any Node release or a string that nvm accepts. Some examples are 16, 16.2, 18.5.0, node and lts/fermium. You can read more about the supported inputs on nvm’s documentation.

Method 1: Set a NODE_VERSION Environment Variable

Netlify allows you to set environment variables to configure your site’s build and functionality. Among other things, you can use them to specify the versions of some software available during the build.

Netlify supports two kinds of environment variables- site and shared. Site environment variables are set at the site-level whereas shared environment variables are set at the team-level and all the sites owned by a team has access to them. In this article, you will use a site environment variable to set the Node version for a particular site’s build.

Further, Netlify offers multiple methods to set environment variables. To reduce decision fatigue, you will use their UI to set site environment variables. To do so, select a site from the Netlify dashboard and go to Site settings -> Environment variables.

Method 2: Create a Configuration File Supported by a Node Version Manager

Node version managers allow you to install multiple Node versions on a machine and switch to a particular one based on your project’s needs. Netlify’s build containers come installed with a Node version manager that can be configured using a file.

You can specify a Node version by creating a .node-version or .nvmrc file with the version information and adding it to your base directory.

An aside on these files. The .nvmrc file is fairly well-known these days as nvm has become the most popular version manager for Node. The .node-version file is a similar configuration file that is supported by a few other version managers.

Originally published at architchandra.com on March 28, 2023.