DEV Community: Steven Hicks

How To De-index Your Docs From Google (And Then Fix It)

Steven Hicks — Wed, 15 Nov 2023 16:35:03 +0000

1. The introduction

Software documentation is a critical element of developer experience. I don't believe this is a bold or disagreeable statement.

Think about how you engage with documentation. Occasionally, you know exactly what you're looking for — you could browse the documentation navigation to find the page you need. More often, you either don't know exactly what you're looking for, or you don't know where it lives in the documentation.

The majority of time you interact with documentation, you probably search for the help you need. That search might happen within the documentation site itself. It might happen in your favorite search engine. Wherever it happens, you expect to find helpful results to your search terms. If you don't find helpful results, you're likely to get stuck. Get stuck too many times, and you give up entirely. This is the nightmare of every tool built for developers.

Imagine the docs you most interact with. You know them reasonably well, but you've forgotten a detail that you know the docs can explain. Envision yourself searching for help on this topic. Imagine browsing the results for your query — and seeing....

0 results found.

Zero results??!! What??? You know it's in there somewhere. (╯°□°)╯︵ ┻━┻

The real-life story of de-indexing our docs

For months I worked on reviving the search results for Camunda's version 7 documentation. Our docs weren't giving 0 results for most queries, but they weren't far off. Known pertinent results were excluded consistently. Most of the results that came up were slightly related but not helpful.

Throughout this I learned unexpected details about SEO (Search Engine Optimization) — especially Google's flavor of SEO. In the end we were able to revive the massively deindexed content. I had my doubts that we'd get there.

What to expect from this article

This article describes my journey through this problem of vanishing search results in the Camunda 7 (C7) docs. There are 5 parts:

The introduction. You're almost finished reading that.
The disappearance.
The investigation.
The resolution.
The recommendation.

That final fifth part is the payoff. If you're managing a documentation site, and unsure how to handle versioned documentation from an SEO perspective, and just looking for guidance, jump there. Most guidance for handling duplicate content in regards to SEO ignores one prime use case — versioned documentation. I hope this article fills that gap.

If you prefer a video format, I've posted a summary of this experience on YouTube. It's told in a different way, but the message is the same.

2. The disappearance

Setting context

Camunda version 8 (C8) marked a significant change from version 7 (C7). While version 7 was intended for self-managed environments, version 8 is cloud-first. Both versions are still supported, at least until 2027.

The large differences in approaches and product features led us to rebuild the documentation for Camunda 8. C7 docs are hosted at docs.camunda.org; the version 8 documentation lives at docs.camunda.io. The two different sites are built with different tooling (Hugo vs Docusaurus).

My team, Developer Experience, manages the C8 docs; we interact with the C7 docs, but we mostly try to leave them alone. Basically, we try to keep the lights on, and not much more. There are issues I'd love to fix in the C7 docs, like the fact that they're tied to a very-specific very-old version of Hugo. But it's not worth the ROI for us...and there are so many other issues in the C8 docs that take priority.

This story is about the C7 docs. The ones that we try not to touch very much....unless something goes very very wrong with them.

Camunda 7 documentation details

The Camunda 7 product is currently on version 7.20. Each released version has its own documentation: 7.20, 7.19, 7.18, all the way down to 7.0. There are also two non-numeric versions of the documentation: latest (a mirror of whichever numeric version is latest), and develop (in-progress documentation for the next release). That's n+2 versions of the documentation, where n is the number of released numeric versions. The early versions were released over 8 years ago! There are a lot of docs.

Much of the documentation is duplicated across versions. For example, look how similar the "Introduction" page is between version 7.4 and version 7.19. There are a couple small differences, but probably 95% of the content has not changed, over 15 versions. As my 12yo would say, "foreshadowing...."

One other important detail — there is an in-site search for the C7 docs. Notably, it is built on a programmable Google search engine. Basically, that means that the search functionality on the site is powered by Google. A search query entered into the C7 docs search box gives basically the same results as entering the query on google.com and filtering by site:docs.camunda.org. Again, "foreshadowing...."

First report

At some point in 2021, the search experience on docs.camunda.org began to degrade. It was not sudden, or obvious. But search results became less helpful, and obvious hits on specific search queries began to disappear. A search for BPMN — a critical and core technology in the Camunda ecosystem, and definitely documented — returned zero helpful results. Since the in-site search is built on Google, the results were degraded both within the site, and directly from google.com.

The community noticed. I want to say we heard them....but we didn't realize how bad it was, or how much it was affecting people. We thought these were one-off complaints. We didn't prioritize the work to fix the search experience.

In mid-2022, we heard from Camunda's support team. They emphasized the problem. Every day, they help Camunda users find the answers to their problems. The degrading search experience was making that much harder. As a result, every member of the support team had effectively built a memory palace of the C7 documentation structure. The only way for them to find content in our docs was to already know where it existed.

Upon this discovery, I wrapped up other on-going projects, and we shifted our focus to fixing the C7 search experience.

3. The investigation

Finding answers in Google Search Console

Early in our investigation, we noticed something strange in the Google Search Console. Many of the docs for the current version were not indexed. They were filed under the category of "Duplicate without user-selected canonical" — meaning Google chose a different version of the page as canonical, and we didn't specify one with a <link rel="canonical"> hint.

This sounded familiar to me. When I learned to program in Ruby, I spent a lot of time searching ruby-doc.org for help. One thing I always found interesting was that I'd never get the latest version of the Ruby docs from my search. Searching Google for "ruby array map" does return a result from ruby-doc.org relatively near the top. But it's for Ruby version 2.7.0, while latest Ruby is currently somewhere in the 3s.

Our situation was much worse, though. We weren't getting any version in the results, let alone an older version.

Inspecting the older version pages in Google Search Console clearly revealed the reason. Our older version pages were excluded from the Google search index because they were "Excluded by ‘noindex’ tag".

Sure enough — our docs had a check in them for the version being rendered, and if it wasn't the current version, a <meta name="robots" content="noindex"> directive was applied to the page. We did this with the intention of convincing Google to index only the current version. Unfortunately, it did not have that effect.

Why we had zero search results

It was the combination of these two factors that caused Google to index very little of our content. We were telling Google explicitly not to index our older version pages with noindex directives; Google was choosing those older version pages as the canonical source, and therefore not indexing the latest version pages. 😅😬

This set me on a learning adventure. To me, it's obvious that in a documentation site, the latest version page is probably the one I want to find in a search. Why doesn't Google think that? And really, how does Google (and any other search engine) handle duplicate content?

How search engines handle duplicate content

Duplicate content happens all the time on the internet. Sometimes it's malicious (cue a generative AI ethics discussion), usually it's not. The usual example is a product page that can live in multiple categories, e.g. /tops/cowboy-shirt and /western-wear/cowboy-shirt.

We don't usually see the same page multiple times in search results, because a search engine chooses one as the canonical source. The canonical is chosen based on many factors -- which page is linked most by the rest of the internet, which page is referenced in a sitemap, etc. Website owners can even suggest a recommendation for a page's canonical URL, via a link rel=canonical hint.

I say "hint" because that's all link rel=canonical is. Google might, and in my experience often does, choose a different canonical than you specify. It's interpreting a collective story about your page from many different sources. Sometimes, it just doesn't agree with your suggestion — the rest of the internet convinces it to choose a different canonical.

Our docs weren't doing much to help Google interpret the collective story. We thought we were giving directives about canonicals by applying noindex to the old versions, but the noindex was applied separately from the choice of canonicals. We weren't submitting sitemaps with the true canonicals — in fact, we didn't even have sitemaps that were properly formed. It was basically 100% up to Google to figure out which version was canonical, without any of our input.

4. The resolution

We tried a lot of different things to convince Google to index the most recent version of our documentation. Not many of our attempts seemed to work!

And the feedback loop for each of them was horribly long! I'd experiment with something, and then wait weeks to see what happened. Even then, it was hard to tell if something worked. It was more obvious if it definitely didn't work.

Submitting a correct sitemap...unsuccessfully

When we realized Google wasn't working off of a sitemap, we went to submit ours, figuring it would be useful as a signal to Google about canonicals. It was rejected 😜 for being in an improper format. I wondered how long it had been in that state...

We corrected the sitemap's format. Our docs have a redirect rule set up so that if you visit a page without a version in the URL, it will redirect to the latest version of that page. We tried submitting a sitemap with these versionless redirecting URLs — no significant change was affected.

We tried a sitemap with the latest version hardcoded in the URLs. This had a more positive effect than the redirecting versionless URLs, but still not very significant.

Declaring canonicals...unsuccessfully

We shifted our focus to declaring canonicals via link rel=canonical. Since we have so many versions of documentation, we ran these experiments on only a few pages at a time, or sometimes an entire version at a time.

We started with pointing canonicals of the current version at the redirecting versionless URLs. We wanted to use the redirecting URLs so that we would never have to go back and change them. Google was unconvinced, and maintained its own opinion, usually canonicalizing an older version of the page. We also got new errors from the redirecting versionless URLs, about containing a redirect.

We tried using self-referential canonicals in the current version, and submitting the current sitemap. This had no effect.

Nothing happened when we applied the current version canonicals to older version pages, either. Because our older version pages still said noindex, Google wouldn't even bother to look at the canonical link. These pages remained canonical, but also de-indexed.

We also experimented with the sequencing of our submissions to Google's crawler. Would it make a difference if we submitted an older page first, then re-submitted a current version page? If we submitted an entire sitemap vs an individual page?

Sadly, no. Occasionally something good would happen, but results were not repeatable or predictable. It seemed like there was nothing we could do to convince Google to consistently choose our latest version docs as canonical. Each experiment took weeks to play out, and it was frustrating.

Building a comprehensive story...and waiting

We noticed that Google was doing a lot more crawling of our docs when we released a new version, and a flood of new pages came online. Given what we'd learned about how search engines choose a canonical source — by piecing together clues from a website and the rest of the internet — we decided to take one final half-court shot.

In a few months, we'd release version 7.19, and Google would crawl it thoroughly. If we could put together a comprehensive story, with correct canonicals on most versions, and an accurate sitemap, maybe this flurry of crawling activity would convince it to canonicalize and index the correct versions.

We weren't sure what to do about the noindex situation. My opinion was that the story would be more comprehensive with the older versions not noindexed. We ran an experiment to see if there was a positive effect on canonicals when I removed noindex. There was no positive effect. We decided to leave all older versions marked as noindex.

Months later, when we released our next version (7.19), we submitted a sitemap containing only the 7.19 pages, and crossed our fingers. And waited again.

Success!

Over the next few months, we received a couple notes from our support team that suggested things had improved.

Eventually I logged into Google Search Console to see if things had improved. It not only had improved, but it had improved remarkably.

90% of our 7.19 pages were indexed! I checked back later, and all but 4 of 500 pages were indexed. Google finally agrees with us! Current version pages are canonical. And more importantly, you get accurate results again when searching our docs.

5. The recommendation

Whew, the payoff! So you've got a documentation site, with duplicate content across versions. You don't want to re-live our awful experience of de-indexing your docs. What should you do?

Build a cohesive picture about the canonical source.

There's no one thing that resolved our situation. Search engines take a wholistic look at your website, and you need to make sure every version is telling a consistent story. Hints and signals compound. The more comprehensive story you tell, the more likely search engines are to agree with your canonical suggestions.

More specifically, in our experience that means:

Use link rel=canonical tags on all older version documents, pointed at the current version.
Submit a sitemap that contains absolute URLs of only the current version of documents.

There's not much there, but any slight deviation can cause havoc.

Some other important notes

link rel=canonical is only a suggestion!

It is definitely important to specify canonicals, but they aren't a guarantee. Search engines might very well choose a different page if the comprehensive story suggests your preferred canonical is incorrect.

Self-referential canonicals are probably not helpful.

The only time they're helpful is when the visited URL doesn't match the canonical URL. If links to your docs include query-string parameters for tracking, they might be helpful. That's not how we use our docs, so declaring a self-referential canonical would give a search engine as much information as declaring no canonical.

Probably don't use noindex.

This is straying a bit from science, as my experimentation suggested that removing the noindex tag from our older versions was actually detrimental to our situation. But hear me out.

All of my results at the time I ran that experiment were confusing, inconsistent, and reliable. I think removing noindex before The Big Recrawl (when we released a new version) would have also resulted in strong canonicalization success.
Google recommends that you don't use noindex to prevent canonicalization:

We don't recommend using noindex to prevent selection of a canonical page within a single site, because it will completely block the page from Search. rel="canonical" link annotations are the preferred solution.
Returning to my Ruby docs example from earlier — they have canonicalization problems, but their problems are way less severe than ours were. If we were indexing our old versions, our search wouldn't have broken. It would have served old versions for search results, but that's a much better situation for a stuck user than zero results.

Epic conclusion

This was the classic ambiguous and confusing software problem. The system was broken, and there was very little help pinpointing why. There are no examples on the internet about how to handle content duplication in versioned documentation. The horribly loose feedback loop when making changes, the unpredictability and inconsistency of canonical selection, it just all felt like I was powerless and guessing. As I got deeper into it, it became something I couldn't not solve. It haunted me a little. I had basically exhausted our appetite to figure it out when we took our final half-court shot. I'm glad it worked out, because that was probably my last attempt before saying "🤷🏼 sorry it just doesn't work."

Matches, Spoons, And The Relationship Between DX and UX

Steven Hicks — Thu, 16 Mar 2023 14:01:00 +0000

Developer Experience (DX) is kind of having a moment right now — and not necessarily a good one. In recent months there's been a lot of discourse pushing back on the importance of DX — or at least its importance relative to the importance of User Experience (UX). My favorite article to capture the moment is Redefining Developer Experience, by Cole Peters. Start there, if you aren't sure what I'm referring to.

I recently listened to a podcast episode that talked about the relationship between Developer Experience and User Experience. I don't recall what the episode said about this relationship....I do recall that I wasn't satisfied with it.

It's very possible that a part of me wants to write this article to make myself feel better about my life choices. I'm less than a year into my first job with the words "developer experience" in the title. Still, even if it's confirmation bias, I truly think I believe these things.

Since its conception, the field of Developer Experience has not defined itself clearly. As Cole suggests, this lack of a consistent interpretation or definition is certainly a significant factor in the disagreement on the importance of DX.

Not long ago I was at a conference, where I met two other people with the exact title as me — "Developer Experience Engineer." All three of us did completely different things. When I meet people and tell them my title, they often ask "what does that mean"; sometimes I introduce myself as "a Developer Experience Engineer...but I'm not sure what that means!" Beyond this specific title, there seems to be an identity crisis for the entire term "developer experience".

I'm not sure how you think about DX. I've got two metaphors, and one questionable psychological theory, to explain how I think about DX, why I think it is important, and how I see its relation to UX.

Burning matches

If you watch a cycling race, you're likely to hear the broadcasters talk about burning matches. The metaphor is this: at the beginning of a race, each rider starts with a book of matches. Each hard effort a rider gives burns one match from the book. Riders can choose when to burn each match...but once the matches are all burned, they have no hard efforts remaining.

You may have noticed this in any sport. When you're fresh, you can put in a hard effort. It feels great! But put in a bunch of hard efforts over time, and your body just can't give you anything more.

In a cycling race, it's a balancing act of burning the matches at the best time. If you can save a match for the sprint at the end of a race, you've got a chance of winning it. Burn them all too early, and you don't.

Spoon theory

Spoon theory is a metaphor originally used in the context of chronic illness, and which has since been used to describe many other scenarios.

Imagine a person starts their day with a limited number of spoons. Each activity throughout the day exhausts energy, and uses one spoon — even ordinary tasks. By the end of the day, their spoons are all gone, and the person has no remaining energy for even a small task.

Resting replenishes their spoons, so they can start over again the next day. But spoons need to be managed carefully every day, or the person runs the risk of exhausting their supply long before bedtime.

Ego depletion

Both these metaphors "rhyme" with the concept of ego depletion.

Ego depletion refers to the idea that our willpower is based on our mental energy, which is of limited supply. The more mental energy we have, the more likely we are to show self-control, and make choices that are better for us in the long term. As we use our mental energy throughout the day, our willpower is weakened, and we're more likely to make short-sighted choices for immediate satisfaction. Think food choices — you might find it manageable to avoid unhealthy foods early in the day...but after a long frustrating day of work, cookies are hard to resist. Maybe that's just me.

The concept of ego depletion is controversial. The validity of studies that demonstrate ego depletion are questioned. This is one of the many ideas that frustratingly falls into the bucket of "seems logical....no proof in either direction....and it might just be in your head."

But hey, I believe in it! Therefore it is real. For me, at least.

What does this have to do with building software?

Ego depletion, or something like it, appears in software development too. When a developer's tooling is working for them, they're able to put more effort toward the details that make a difference for the user. When the tooling fights them, their energy is exhausted, and they're less likely to put effort toward user-facing details.

If I burn two matches trying to test an asynchronous event handler on a React component, those matches are gone! If I spend three spoons fighting a weird webpack error, I'm becoming exhausted. Neither of these issues directly impact the user, but I've burned my energy on them. When something comes up late in the day that actually impacts the user — maybe something missing in regards to accessibility, maybe an important edge case in some component that only affects a small amount of users — my energy to deal with it appropriately depends on how much I've fought my tooling all day.

This is why I think Developer Experience matters. DX that works with you leads to better UX. Energy I must put into how I'm building a product pulls directly from energy I would have put into what I'm building.

I don't care as much about how shiny and hot the tools are that I'm using. I care that the tools I'm using are guiding me to a pit of success, where I don't need to invest a ton of effort to build a proper experience for the user.

There's certainly a risk of over-optimizing for DX at the expense of UX, in anything we build. Developers can easily overlook that imbalance because they aren't the actual product user. But even when prioritizing the user's experience, the developer's experience matters. If creating a good UX requires a lot of effort from developers, they just aren't going to do it.

The answer to the question "DX or UX?", as always, lies somewhere in the middle.

Breaking Problems Down: A Case Study

Steven Hicks — Thu, 01 Dec 2022 20:30:32 +0000

I've written in the past about strategies for breaking Pull Requests (PRs) into smaller pieces. In that article I give a number of recommendations for reducing the size of a PR....but it's still very abstract.

Recently I spent a couple months working on a large-ish project for the Camunda Platform 8 documentation. The size of the project forced me to practice what I preach. It involved moving a lot of content, which could easily have turned into massive PRs, and left overwhelmed reviewers with no choice but to rubber stamp "LGTM" (looks good to me) on them. I also worked in relative isolation on this project, and since I was much deeper into the work than my reviewers, easily-digested PRs were an absolute requirement.

So consider this article almost an addendum to my previous work. Instead of describing in the abstract, now I can point you at concrete examples!

Background: the project

Camunda Platform 8 includes a handful of components that work together to facilitate process orchestration. Most of them are always on the same version — but some of them aren't! Our Optimize component, which you might guess empowers you to optimize a modeled process, is on a completely different version number than the rest of the components. Where most components are currently on version 8.1, the latest Optimize release is 3.9.0.

Unfortunately, our docs weren't reflecting this. We treated the latest version of all components as version 8, even if that wasn't correct. And that was this project! Get the Optimize docs showing the correct version number.

As with anything, when you boil it down to a paragraph it sounds like way less work than the actual implementation. 😅

The work before the work

Small PRs start long before the PRs are opened. In this case, some up-front investigation helped identify ways to break the work down.

Early exploration to identify and understand the work

I had two goals with the early exploration:

To understand what work was needed, so that I could break it down.
To resolve some uncertainty about how the tooling supported solving our problem.

So I built some proofs of concept:

Along the way, I built a couple more proofs of concept when I ran into problems I wasn't sure how to solve:

This investigation and prototyping resulted in a much better understanding of the work. It even helped us identify work that, if done up front, would improve our ability to schedule and complete the remaining work in three important ways:

The critical output of breaking down work is not only the smaller pieces. It's also the knowledge of which pieces are the scariest, riskiest, and most uncertain, so you can solve those first.

Make the change easy before making the easy change

Of the 3 improvements listed above, I want to call out one in particular. Before writing a single line of code for the project, I wanted to make sure we could integrate incomplete changes a little bit at a time, especially at the beginning. As the project went on, and PRs started to resemble other previous PRs, it became less important to be able to integrate incrementally. But at the beginning, this was novel work -- we weren't sure what it would/should look like, and I wanted to feel safe introducing it in incomplete parts.

The first PR I opened for this project was to introduce a "next" version of the docs. With an "under construction" version, I felt free to make as many changes there as I wanted. I could deploy incomplete changes and show them to people for feedback.

This is not the first time I've referenced the following Kent Beck tweet, nor will it be the last:

// Detect dark theme var iframe = document.getElementById('tweet-250733358307500032-300'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=250733358307500032&theme=dark" }

Building a seam before introducing changes is always easier than doing those two things simultaneously.

Tracking the work

After the initial exploration and proposal, this project sat untouched for a month or two — for no important reason, there are just other things that I worked on. But the exploration gave us enough information to start tracking the work with some level of confidence.

There are a handful of different tools and artifacts we use to track projects at Camunda — Trello, Jira, GitHub, Google Docs... For the Developer Experience team in particular, we've moved to using GitHub to track almost everything. So for this project, I created a single issue to list all the things we'd have to do to complete this project. I initially filled it out from a high level, not too much detail, figuring I'd fill in more details as I learned them.

I have mixed feelings about this approach. I like that there is one place that tracks all of the work. I believe strongly in the importance of tracking the work publicly (or at least, visible to my team). This accomplishes that.

But by the end of the project the issue became pretty massive. If you're looking for something specific in that list of completed work, it's hard to find. Part of me thinks this might have been better served as a GitHub project, instead of an issue. I chose the artifact based on what I knew at the time, so I'm not holding it against myself for tracking it this way. I do think I'll be more conscious next time of how big an epic might grow when I decide how to track it.

Explaining the work

Knowing that I was working on this project mostly in isolation, it was critical to explain my work to reviewers who had less context. I learned some good habits about adding context to PRs while at Artsy. On this project I got to put them to good use — especially adding videos to demonstrate changes (direct link to video), and inline comments identifying the interesting changes.

A couple other good reasons to explain work at this level:

When anyone comes back to this in the future, there will be plenty of context. They shouldn't have to spend much time spelunking Slack or asking "hey I know this is a long shot, but do you remember why we wrote this line of code 6 months ago?"
It makes for a nice reference point whenever we want to do something similar in the future, or when someone on the internet asks a question about how to do this thing.

Separating infrastructure from routine work

Pull requests that combine significant infrastructural changes with routine changes are a recipe for losing the signal amidst the noise.

One example of this was mentioned above — introducing a "next" version of the docs, which was shipped separately from any content changes. After it was merged, I was free to twiddle with content all I wanted, but I wouldn't have wanted someone to have to review both types of changes in one place.

Another example — while I was moving Optimize documentation into its own section for the first time, I noticed that the multiple sets of versions were going to create a cross-linking mess. We'd end up with hard-coded versions in URLs when linking across the documentation, and have to update them whenever new versions were released. Before completing the first Optimize docs PR, I introduced an infrastructural enhancement for cross-linking.

Sequence in this case probably didn't matter too much; the importance to me was that I had two related but distinct changes, and I wanted to keep the history of them separate. Aside from making it easier to review, this approach prevents one ready feature from being held up by one disputed feature.

Splitting stories

Even though there was a ton of content to move in this project, there presented two natural ways to split the content into smaller pieces: by version, and by content section.

Splitting by version was something we noticed up front. We could iterate through the different versions, starting with the most recent version and ending with the oldest, and migrate content one version at a time. This also presented itself midway through the project as an opportunity to defer work. As we worked through newer versions, we realized that the oldest versions were less important to us, and we de-prioritized them.

Splitting by content section was not noticed up front. In fact I only discovered this natural seam on accident — by forgetting to do two of three content sections! 😅😬 When a stakeholder pointed out that I'd only shipped one section of the first version, I decided this was fine, and actually a good way to break things down for the other versions.

De-scoping mercilessly

One of my constant struggles in software development is deciding when a bug or edge case should be resolved with the work I'm doing, or if it can be done later. I have a high standard for "done" — sometimes too high. Perfect is the enemy of good, as....someone says. And I fall for perfection just about every time.

I've gotten better at de-scoping and deferring these kinds of things, and this project was an opportunity for me to demonstrate my progress. Many issues came up as I was working, and you can see in the epic for this project that we treated many of them as follow-up work instead of blocking issues.

Do you have any real life examples of breaking problems down into smaller pieces? Let me know!

Automating Pull Requests With Sed And The GitHub CLI

Steven Hicks — Mon, 21 Nov 2022 18:41:46 +0000

Obligatory xkcd comic on automation

TL;DR

Here's a Bash script to:

Take a branch name
Create a new branch based on the original
Call some scripts to update files
Use sed to update a config setting in another file
Commit the changes
Push up a GitHub PR with a lot of information filled in

On the other hand, if you've got some time...

Pull up a chair and listen to me talk about my personality again 🙄

My learning style looks something like this:

Run into a problem
Discover or remember a tool that could solve that problem
Learn that tool just enough to solve the problem
Never go back to that tool until the next problem wants it

It's kind of frustrating, actually — I would love to be the kind of person who dives deep into something new. I'd love to stay interested in a topic for more than one problem. Instead, I learn a little about a tool, and lose interest in it until I need it again. This is why this blog seems so scatterbrained, I think — no content strategy here, friends! Just a bunch of mismatched tangentially related solutions to very specific problems.

Nowhere is this more obvious than in my use of shell scripting. I know enough Bash to be ...very very casually dangerous. Like tortoise-shell-sunglasses-and-fake-leather-bomber-jacket dangerous.

Recently I got a chance to grow my shell scripting skills! When I recognized the need for sed, I was excited that I finally knew enough about it to recognize a case for it.

And as soon as I'd automated that, I recognized another opportunity — to get a little more knowledge of the GitHub CLI.

The use case, in detail

We have some older docs at Camunda that are built with Hugo. There are about 20 versions that we maintain, and each version is built from its own branch. This is a pain when you have to update more than one version. Thankfully, that doesn't happen very often.

But ooohhhhh, when it does! When it does!

We recently learned that we had sabotaged our docs by de-indexing most of them from Google search results. Yikes! Very long story short — when providing multiple versions of documentation, make sure you explicitly declare one version as the canonical. If you don't, Google will pick one for you! And it might pick one that you told it not to index!!

To resolve this, I wanted to apply canonical URLs to all versions of our docs. Which meant I had to make nearly identical changes to all ~20 versions/branches. The changes themselves were relatively simple:

Update the theme to include canonical URLs
Configure a site-level setting to use the correct base URL

Feel the pain before addressing it

More personality stuff, sorry. Here's one rule of developer experience that I feel pretty strongly about:

Before you address a problem, you should feel the pain of it as directly as possible.

Otherwise you're just guessing at the solution.

As a bonus, having a way to experience the problem gives you a nice feedback loop. You'll know you've fixed the problem when you no longer experience it.

All this to say, I had a hunch I might want to automate these PRs, but I didn't feel comfortable automating them until I knew what the repetition looked like. So I worked through the first few manually.

Once I'd been through the process a few times, I recognized the repetition. For each version of the docs, I needed to:

Manage my branch
Run a command to update a few files
Update a configuration file by changing one setting
Commit it all
Create a PR based on a template

1, 2, and 4 were already in my toolbox. 3 and 5 were not.

3. Updating a configuration file with sed

sed is a "stream editor"; for my needs, that effectively means it's a command line tool for editing the contents of a file. It's not just the editing that makes it the right tool to change a config setting — it also makes it easy to target text, by filtering the stream/file.

My specific use case was to modify the value of the baseURL configuration setting, to include a domain. So where the config file previously read:

baseURL: "/some/value"

I needed it to read:

baseURL: "https://docs.camunda.org/some/value".

I'd previously heard of sed, and previously used sed, but mostly in the context of copy-pasting things from the internet. This was the first time I'd looked at a problem and thought "oh, I could use sed to do that!"

And I'll be honest -- my understanding is still not far beyond cargo-culting. What I can say is that these are the arguments I needed:

-i '' to edit the file inline, rather than create a copy
-E to use "extended" regular expressions, because my regular expression used a capture group
's/...matching pattern.../...replacement pattern...'/g to substitute text for a regular expression match
the file name to edit

In the end, this is what my command looked like:

sed -i '' -E 's/baseURL: \"(.*)\"/baseURL: \"https:\/\/docs.camunda.org\1\"/g' config.yaml

This searches the config.yaml file for the baseURL setting, and injects the domain before its current value.

The most frustrating part about writing this command was figuring out that I needed the -E flag in order to use a capture group in my regular expression.

Yay, computers!

5. Creating a PR with the GitHub CLI

I'd previously opened many GitHub PRs, and I even use the GitHub CLI to do it every single time (although I abstract it behind an alias I call open-pr).

And initially, that's what I scripted. But I found myself going into the GitHub UI to update things repetitively. I figured there had to be some arguments I could use to change those values for me.

There are!

In particular, I was interested in these arguments:

--web: to open the web interface, so that I could make some final changes before creating the PR
-t: the title of the PR
-a: PR assignee, which I could set to myself with the identifier @me
-B: the base of the PR, meaning which branch it should be merged into
-b: the body of the PR.

I also assigned a bunch of variables ahead of time, to make my command easier to read:

gh pr create --web -t "$TITLE" -a @me -B $VERSION -b "$BODY"

Stitching it all together

You can see my final script in this gist. I've done my best to comment it well, but feel free to leave comments in there if anything doesn't make sense or could be improved.

Extra credit: automating the screenshots

When using the GitHub CLI to open a PR, I chose to use the web interface because there was still one repetitive task I didn't want to automate: adding screenshots to the body of the PR.

I'm certain there are tools to do this, but it seemed like a can of worms I didn't want to open. Getting everything but the screenshots was 95% of the win. I didn't mind having to take two screenshots and then paste them into the GitHub UI.

If you've automated this step as part of opening a PR, I'd love to hear about it!

Rule 4: Warm Up To Get Into Flow State Sooner

Steven Hicks — Thu, 25 Feb 2021 17:21:09 +0000

Deep coding work is like building a LEGO spaceship in my head. It takes a while to construct the model, and it takes focus to keep the model from collapsing. Once it collapses I have to rebuild the model from scratch. I've explained to my lovely interrupting children that this is why I get mad when I'm interrupted — what seems like a 30 second request to them is a much longer distraction for me.

I've estimated in my head that I need about 20-30 minutes to get into a state where I'm in the flow, the spaceship is fully constructed, and I can manipulate it like a master builder. I call this my "ease-in time." I think of it like critical mass, or a moat that I have to cross to get to an adventure.

I find writing to be similar — I need that ease-in time to get into a writing mindset before I feel effective. When I've got 45 minutes until bedtime, which is frequently when I write, it's easy to convince myself that it's not even worth starting. I'm only going to get 15 minutes of quality writing time after I ease in. Why bother?

During my ease-in time, I have very little self-control or direction. I'm mostly doing my best to keep myself from all the distractions I want to chase. It's sloppy. Sometimes I end up on Twitter. Sometimes I don't. The sessions when I end up on Twitter take longer to feel productive than the sessions when I don't.

Until recently I treated this as something I couldn't influence. I thought I just had to ride out my ease-in time, it takes what it takes, and I'd get to deep flow eventually.

But then I thought about athletes.

Athletes don't warm up on Twitter

Athletes don't just do whatever they want before a game — they warm up. Giannis Antetokounmpo doesn't go into a game cold — he warms up by dribbling and shooting jump shots. When I get on a bike, I'm not ready to crush my ride right away — it takes ten minutes of gradually increasing my cadence and power before my body feels ready. I don't just put on my shoes and run five miles — I do stretches and strengthening exercises. (Maybe you just put your shoes on and run but I am old and I want to avoid injury.)

Athletes don't warm up by scrolling through Twitter or watching incredible marble machine videos on Youtube. They do drills that warm their body up for the specific movements they'll be making.

With easing-in I've been allowing myself the time to warm up, but I wasn't guiding my time properly. I was taking jump shots to warm up for a bike ride. My warm-up time wasn't effective. It didn't actually warm me up for deep coding or writing.

What makes a good warm-up activity?

It's easy to start the activity. You're using it to warm up for the deep work — it shouldn't itself require a warm-up.
It's easy to stop the activity. It's important to set it aside when it's time to focus on your intention. You don't want to get sucked into your warm-up.
Specificity. This is the piece I was missing. Warm-ups should exercise the muscles you're going to use during your time block.

Writing warm-ups

Intentional writing warm-ups have reframed the amount of time I need to work on my writing practice. Whereas I used to see that 45 minutes before bed as not worth starting, I now see it as an opportunity to make at least a little bit of meaningful progress. I'm more likely to start a short writing session instead of conceding to Twitter or GCN for the night, because I can usually get myself into the writing mindset in ten minutes or less.

Some things I've done for writing warm-ups:

Time-boxed free-writing
Re-reading/editing an in-progress article
Picked a page out of the book "642 Things To Write About: Young Writers Edition". ("Young Writers Edition" because my partner teaches middle-school English online, and that's the book that's on our shelf 😅)

I'm sure this is probably a technique that real writers talk about, and there are probably lots of other good resources for this.

Coding warm-ups

Warming up to write code feels more natural to me because I've done this unintentionally for years. It feels different now that I'm applying intention, though. It feels like I have self-awareness and I've got agency in how long my warm-up will take.

Things I've tried for coding warm-ups:

Time-boxed work on a personal project. Emphasis on time-boxed, because I don't want to get sucked in. If you try this option, make sure this work doesn't require its own warm-up: right now I have a personal project that's always kind of floating around in my brain when I'm not working, so I can usually jump right into it.
Time-boxed PR Review. Yeah, I know I told you in the previous article that you should defer that to your smallest time slots, but I have found it a good way to ease into deep coding too.
Small learning challenges. My friend Barry recently shared an interactive TypeScript course with very small exercises. The exercises are tiny, so I can get a taste of coding without getting sucked in.

Things I want to try:

Writing a couple unit tests. They wouldn't even have to be tests for my focus project — the important thing is that I'd be warming up the "muscles" I'll be using. I haven't done this yet only because I keep forgetting to try it.
A code kata. The challenge here is to find a kata that's short enough to qualify as a warm-up instead of a full-fledged problem. It'd be easy to get sucked into this. I haven't put the effort into finding a short enough kata yet.

I'm not a robot

I know it seems like it but I'm not always on. Sometimes I self-indulge. Sometimes I still let my hair down and "warm up" by scrolling through Twitter.

I have more awareness of when I'm doing it though. And intention — I'm allowing myself to warm up in a way that I know is less effective as a treat. Like allowing myself to drink a Coke twice a week even though it's bad for me. (This is what I tell myself...and you...to convince us that I have agency and self-control. Is it working? 😬)

I also don't always need a warm-up. Sometimes my mind is in the right space from the start, and I'm able to jump right into my deep work. I try to do a check within the first few minutes of my time-block to see if I need an intentional warm-up. I think I need it about 50% of the time for writing, and maybe a little less for coding.

Warming up for deep work in small blocks of time is pretty new to me but so far it's been really effective. I spend a lot less time getting into the mindset I need to be effective. If you've got suggestions on how to warm up for deep work, I'd love to hear about them!

Rule 3: Categorize Tasks By Depth, And Work Them At The Right Time

Steven Hicks — Tue, 23 Feb 2021 19:18:57 +0000

I find an hour to be enough time to get into deep work. I haven't always felt this way, but I do now.

But what about even shorter blocks of time? What about those half-hour gaps between meetings? Is it worth starting anything meaningful during those, or should I just burn that time on Twitter?

As with everything, the answer lies somewhere in between. I don't think the ROI is worth getting into deep thought for a half-hour, but I do find those pesky half-hour blocks to be extremely useful for shallow work.

What qualifies as shallow work?

Making a personal phone call
Raising a question in Slack
Scheduling meetings
Creating tickets for new issues
PR review (if they're on the smaller side)
Small updates to docs
Investigating a tool I learned about recently
Answering a question I'm curious about

Many of these things require very little time, but "shallow work" isn't really about the amount of time involved for the task itself. It's about how much time it would take me to get into the level of focus required for the task. Everything in that list requires very little priming. I could drop any one of them immediately and spend very little time getting back into it an hour later.

Avoid shallow work during deep focus blocks

This is probably the most controversial thing I'll say in this series: procrastinate shallow work when you come across it during times of deep focus. Instead, track the task however you track your work, tag it as "shallow," and save it for a later tiny block of time.

When that next tiny block of time comes up, knock out as many shallow tasks as you can.

Ten seconds of distraction is better than five minutes of distraction

I've heard people say something to the effect of "if it takes less than five minutes, do it now." But I don't need any help getting distracted, especially when I'm trying to do deep work. Every unrelated task that you do during deep work is an opportunity to pull yourself out of your flow.

Stomp these distractions out. Recognize them as quickly as you can. Say "Not today, Satan," and save them for a time when you don't need your attention, focus, or deep thought.

My system for categorizing tasks by depth

What works for you will probably look different than what works for me, but I use Todoist to manage my day. I've shared more detail about how I use it, but I categorize my tasks using the labels feature.

When I'm in the middle of deep work and I come across a distracting task, I'll use Todoist's global Quick-Add shortcut (ctrl-cmd-A on a Mac) and add a task with the appropriate label: @shallow, @medium, or @deep.

And that's it! I'm distracted for less than a minute and I stay focused.

The rest of the day I'll handle my @shallow tasks during my extra-short time blocks, my @deep tasks during my longer time blocks, and...I don't really have a good description of when I handle the @medium tasks. I use it for tasks between @shallow and @deep but I don't have a simple heuristic to describe when I use it.

Next time we'll answer the question that actually prompted me writing this series: "How can I get into flow state more quickly so that I can accomplish something meaningful in smaller blocks of time?"

Rule 2: Slice Work Smaller

Steven Hicks — Sat, 20 Feb 2021 04:12:31 +0000

I'm a +7 perfectionist. I don't like to walk away from work until t's are crossed, i's are dotted, tests are written, and the work is done-done.

Perfectionism is a double-edged sword. While it produces beautiful work, it also prevents me from shipping things when they're "good enough," or even from collecting helpful feedback before I consider the work "complete." Sometimes it takes a non-trivial effort to convince myself to open a PR when the edges aren't yet polished.

My perfectionism has another negative effect — it prevents me from feeling accomplished when I have only a short amount of time to put toward the work. "One hour to put toward this feature? Pfffttt that'll only get me 10% done," my brain says. I dread even starting the work because I know I'm not going to "finish" within an hour.

The trick for me is to break work into smaller slices.

Smaller slices shift the meaning of "meaningful progress"

Instead of setting a goal of finishing an entire feature, I treat it like a good agile team treats their work — I break the feature into sub-features, or steps, or layers, or milestones. Anything that breaks the problem down into smaller problems.

These smaller tasks shift my perspective — a small block of time was not enough to finish an entire feature, but it's usually enough to make progress on at least one or two of the smaller tasks. The reframing lowers the bar for what "meaningful progress" means. An hour of work suddenly feels much more impactful. I don't have to fight the voice in my head that tells me it's not worth starting. My milestones for progress seem more attainable.

Smaller slices make me feel good

I've also now got dopamine on my side.

Large chunks of work feel nebulous. They make me feel bad. Like I'm never going to finish. Like I'm slow. Leaving a non-tasked story in "In Progress" for a week is frustrating. It's embarrassing to give the same update in standup several days in a row — "ummm I'm still working on the edit screen."

Smaller slices allow me to check things off a list more frequently, or to drag a sub-task from "In Progress" to "Done." This simple act feels like an accomplishment, complete with dopamine rush. No matter how many tasks remain or how small the task I checked off, it literally feels good when I complete one.

I actually left you a good heuristic above for knowing when you need to break your work down. If you're repeating your daily update...it might be time to break that work smaller. It will help you feel like you're making progress, and it'll give better visibility to the rest of your team.

A couple other good reasons to slice work smaller

It's easier for people to review a smaller scope of work. Large PR reviews often end in "LGTM" (looks good to me) because they're just too difficult to digest.
It tightens the feedback loop of delivery & iteration. Smaller chunks can be shipped faster. We can learn from them more quickly and iterate.

Slicing my work smaller helped me answer a very specific question I was trying to answer: "How can I accomplish something meaningful in a shorter amount of time?"

In the next article I'll answer another question: "How can I take advantage of those very short blocks of time?"

Rule 1: Dedicate Time. And Schedule It.

Steven Hicks — Mon, 15 Feb 2021 17:52:18 +0000

My first rule for maximizing productivity in small blocks of time is to dedicate time to a specific goal, and schedule it on my calendar.

An hour of time that has no declared focus often turns out to be exactly that — unfocused, scatterbrained, not dedicated to any one thing. In an unfocused hour I might feel like I make a bit of progress on several things, but rarely will I feel like I made significant progress on any one thing. Spending 10 minutes each on five things feels much less productive than spending 50 minutes on one thing. If I dedicate an hour to a specific task or project, I feel more focused from the start. There's nothing forcing me to stay dedicated...but having an intention gives me something to return to when distractions beckon.

The benefits compound when I schedule this dedicated time on my calendar. I feel more in control of my day. Control is a big thing for me — my worst days are the ones where I feel like I have no agency in what I'm doing. Dedicating time on my calendar for work that I want to accomplish satisfies my need for autonomy.

Recurring dedicated time on your calendar is a great way to build a practice, too. I've been using this strategy to strengthen my writing practice.

The hardest part about scheduling dedicated time is the discipline required to use it for its intended focus. For my first six months at Artsy, I had a weekly block of time on my calendar for "sharpening my tools" (h/t Jon Allured). I never once used it for its intended purpose.

What's the difference between the hour of sharpening I never did and the hour of writing I successfully dedicate every Monday? I think it's about specificity. I never really had an idea of what I'd do when it was time to sharpen. I wanted to keep my dev tools up to date, but that was such a nebulous goal that I never knew where to start.

With my writing hour, my goal is smaller — work on a blog article. Even if I don't know what the article is about yet, I know where to start...

And I'll tell you all about that later, in Rule 4.

Maximizing Productivity During Small Blocks Of Time

Steven Hicks — Fri, 12 Feb 2021 17:44:42 +0000

One of my primary self-improvement projects is to use my time more effectively, especially the small blocks in between meetings. I'm not a manager, but I have more of a manager's schedule than a maker's schedule. I willingly accept more meetings than most developers. I like the interaction, I like to get others unstuck, and I like to participate in cross-team conversations. The result: most of the empty time on my calendar is in blocks of an hour or less.

It's hard to do deep work in an hour or less. If I make full use of the hour, sure, I'll probably feel good about my progress...but that assumes I can get into the deep work promptly and that I don't get distracted. In reality, what looked on my calendar like a good hour often ends up feeling like 15 minutes of meaningful progress. If I'm lucky.

Thankfully, I have a lot of good role models to emulate at Artsy. Some of my most admirable coworkers accomplish meaningful work in the same size blocks of time that I would have previously dismissed as too small for meaningful work. They might not know it, but I've been watching and taking notes.

Over the next couple weeks, I'll share the rules I've developed for myself to work more effectively with my small blocks of time. I don't always follow them, but the days I do are the days I feel most successful, productive, and effective.

Tips For Working With Legacy Code, Courtesy Of My New Kitchen Faucet

Steven Hicks — Fri, 08 Jan 2021 16:36:19 +0000

This holiday break I replaced our kitchen faucet. The neck was loose and it wiggled, and there was a little bit of water starting to pool at the base. It wasn't leaking yet but it seemed like it might start leaking soon. There's probably a gasket or something I could have replaced but the faucet wasn't great and I had the time off of work. I figured now would be a good time to replace it.

We own an old⁽¹⁾ home. The main drawback of owning an old home is that every project takes five times longer than you expect it to. Anything that's hidden behind walls is made of old materials and it was introduced when building codes were far less safe. As you start opening things up during a project you find solutions from long ago, many of which are terrifying to work with. Like this electrical box containing 10 old wires with crumbling insulation that killed another of my holiday break projects:

My kitchen faucet project had all the makings of a classic old-home project. The instructions for installing the new faucet were very well-written and easy to follow! But my kitchen is old. Who would win — the new faucet or the old home?

The old home won. As things went wrong, I noticed parallels between my faucet installation project and working with legacy code. I've distilled the experience into a handful of tips.

Tip 1: Reframe "legacy code"

There doesn't seem to be a canonical definition of "legacy code", but it seems software engineers use the term to mean "code that I inherited that I must support." This definition isn't itself negative, but it has very negative connotations for most engineers. Since the code is inherited, it probably doesn't follow the patterns or style the new owner would like to see. It probably doesn't have enough test coverage to thoroughly describe its intentions or prevent regressions. There's likely technical debt that has been accrued over its lifetime. All of these factors make it difficult to change. But we need to change it, because it's still in production, and we still need to support it.

Much of what I've described above is purely factual, but our cognitive biases and past experiences with legacy code apply a layer of emotional distortion. The fundamental attribution error convinces us that the existing code wasn't well-factored or well-conceived by the original author. The hindsight bias convinces us that it could have been written much more simply and clearly. We remember the last time we had to support legacy code and how we broke production while trying to make a single small change. We start thinking it would be much easier to rewrite this code from scratch than to support it.

As I began my faucet project, I noticed similar feelings. The last time I'd tried to do a simple project in this house I found that mess of wiring, closed the box up, and called an electrician. How could the previous owners have been so careless with electricity? They must have had a death wish, I thought.

I started thinking about how if this were a software project, I'd want to rewrite the entire kitchen. Correction — I'd rewrite the entire house. The wiring is confusing and nasty all over, not just the kitchen. New construction would be so much easier to work with.

But I know "the big rewrite" is a trap in software. The hindsight bias and overuse of the phrase "technical debt" discredit the learning and the journey that went into creating legacy code. They overlook all of the features that are working well. If "the big software rewrite" were like building a new house from scratch, you'd move into a house with a beautiful kitchen and overall layout...but cardboard boxes for furniture, bathrooms with buckets for toilets, and nowhere to park your car.

This thought gave me a lot of appreciation for my old home. I don't want to do a big rewrite on it. The neighborhood is amazing. The house is the perfect size for us. We just finished our basement. Sure, the electrical is terrifying in most of the house and the main bathroom is peach, but just like existing software, it's a legacy. There's too much good stuff here to throw it all away and start over. Incremental improvement is the way to go — just like with a legacy software project.

Tip 1A: Dealing with integration problems is the job

In a talk that I love to give about getting unstuck, I encourage software developers to reframe our understanding of getting stuck on difficult problems. Getting stuck isn't something that prevents you from doing your job as a software engineer; it is the job of a software engineer. We get stuck and have the persistence to get unstuck. That's problem solving. We're good at it, and that's why we get paid to do this job.

This perspective ran through my mind as I thought about the things that typically go wrong with a home improvement project. I've always considered problems with old plumbing or electrical to be a nuisance. Steps that I shouldn't need to take. A distraction from the actual home improvement project. But I was wrong — integration with the existing plumbing/electrical/construction is the home improvement project.

It's similar to working with legacy code. When you introduce new code, you'll always face the challenge of integrating it with the existing system. Somehow you need to make a seam for your changes. Once your changes are introduced you need to verify they don't negatively affect the existing system. These aren't merely tasks that impede you from supporting legacy code...these tasks are the legacy code support.

Tip 2: Identify incremental improvement opportunities by de-risking

So now we've all gained appreciation for the existing system, and we're in agreement that the best way to make changes is through incremental improvement. Where do we start? How do we know which parts of the system to improve first?

Often the system will present the answer to you. Is there a subsystem that's flaky and needs to be reset often? A section of code that breaks every time you try to make a small change? Is there a service that was written by a departed teammate in a language that no current engineers feel confident supporting? Start there.

This is de-risking. You can't predict exactly when a line of code will fail, but you can predict which services or subsystems are likely to fail sooner rather than later. Identify them and improve them before they get a chance to fail.

With my kitchen sink, I'd been noticing water pooling at the base of the neck for a couple weeks. It was nothing catastrophic, but it was an indication of a weakness. It may not be failing now, but it would likely fail soon.

Tip 3: You own the code

The first integration problem I ran into with my kitchen faucet was that the window sill behind the faucet extends as a shelf, and it was in the way:

When we picked out a new faucet, I never considered that the faucet currently installed was probably deliberately chosen because of its short height. This allowed them to install it below the shelf. We chose a taller replacement faucet though, and the shelf prevented us from installing it.

My initial instinct was to return the faucet and find a shorter one. My partner suggested I cut a notch into the shelf. I resisted and thought "but this shelf is part of the house." She convinced me pretty quickly. It took us a while to pick out this faucet and we both really liked it, and this is our house, so we can do whatever we want with the shelf. We're planning on renovating the kitchen in the next few years, and it's very unlikely the shelf will survive that remodel. We could survive a few years with a notch in a shelf, especially if it means we enjoy the feature we use the most in our house.

So I cut a notch in the shelf with a reciprocating saw:

As I painted the new edges, I thought about how this was classic Steve — amplifying the importance of an existing feature at the expense of a new feature. I do this when I'm working with code. If code is already there and it's working well, I have an intense aversion to modifying it. I avoid refactoring anything that I don't need to change. This is extremely noticeable when I'm working with teammates who refactor freely. I get feelings of anxiety and discomfort as they make more changes than I would, and I get visions of a late night fixing an outage caused by a change that didn't need to be made.

My aversion to refactoring sometimes leads to a feeling of borrowing code — like the codebase isn't mine, it's the previous owner's, and I'm temporarily passing through. But like the flimsy shelf above my faucet, it's helpful to remind myself that I own this code. If there is code that doesn't look right, I can change it, because it's mine, and I'm the one supporting it.

Tip 4: Perfect is the enemy of done

My aversion to unnecessary refactoring has one major benefit: it prevents me from blowing a project into something much larger. When modifying a legacy codebase you regularly face decisions about how far to pull a loose thread.

There were several points in my faucet project where I caught myself in this kind of decision point. From the start, I wasn't sure if I'd be content with replacing only the faucet. It'd be nice to have a new sink too...but it wasn't as critical as the faucet. The entire kitchen needs a remodel...but that would be way too much work for a holiday project. The connectors on the faucet supply lines didn't fit the valves coming out of the wall, and I could have replaced the valves with something more modern...but that would have meant doing real actual plumbing. Even the window sill/shelf could have become a bigger project if I'd decided to remove it entirely.

Somewhere in between refactoring nothing and refactoring everything is a sweet spot for navigating these types of decision points. Practice helps you find that sweet spot. Time-boxing or the Pomodoro technique can be helpful for preventing yourself from pulling a refactoring thread too far. Being transparent and detailed in standup about the changes you're making makes it hard for you to go days with a vague update of "I'm still refactoring." Working with your team lead or product owner to prioritize the exact bits that need to change will help you avoid nebulous tasks with no end in sight.

And sometimes it comes down to showing self-restraint and acceptance that something is "good enough". It's okay to ship code that you wouldn't show in an interview. It's okay to not refactor the next code down the call stack. If you need it, I grant you permission to not solve all the problems today.

Oh, and here's my new kitchen faucet:

¹ as a resident of the Milwaukee area working for a company based in New York, I've discovered that "old home" is a very relative term, depending on where you live. My home was built in the 1930s — around here, that's pretty old. It's certainly not pre-revolutionary...but it's old enough to make home improvement difficult.

Generating Social Sharing Images In Eleventy

Steven Hicks — Thu, 17 Dec 2020 04:30:31 +0000

Inspired by Jason Lengstorf, I added social sharing images to all my blog posts on stevenhicks.me. This means that when you share an article from my site to a place like Twitter, you'll get a nice big card like this:

Sweet! I can't get enough of those 70s shag carpet vibes.

Prior art

Before I show you how I hooked this up to my eleventy site, consider this article by Stephanie Eckles about using puppeteer to generate social share images. If you want to use HTML & CSS to create your social sharing images, that is probably what you're looking for!

The method I chose⁽¹⁾ uses Cloudinary to overlay the article title onto a common social sharing image.

Most of what I needed was covered by Jason in his articles on adding text overlays in Cloudinary, designing a social sharing card, and auto-generating social share images with get-share-image. Thanks, Jason!

Heads up that the most time-consuming part was tweaking the Cloudinary text overlays. Lots of fiddling with cryptic arguments. It's literally pushing pixels to get text in the right place.

Emitting the image URLs in eleventy

Here's my addition to this problem space: a PR that shows everything I needed to do to hook up the images in eleventy!

There's not a lot there, but let's walk through it.

1. Add the `get-share-image` dependency

You'll do this with npm install @jlengstorf/get-share-image or yarn add @jlengstorf/get-share-image. I added it to my devDependencies because I care about separating dev dependencies from runtime dependencies. Maybe you don't care — I'm not going to arm-wrestle you over it.

2. Add an eleventy computed data file

Eleventy's computed data files inject computed properties into a page template for each page they apply to. Like maybe you want to compute a social sharing image URL that's based on the article title!

I added a file named blog.11tydata.js to the blog/ folder. I chose to put it in the blog/ folder because I only wanted to generate social images for my blog articles; it seemed silly to me to generate a social image for my about page that said "About". I'm not sure if the file name needs to start with blog, but that's what the docs did in their example (posts/posts.11tydata.js), so I just went with it.

The contents of blog/blog.11tydata.js:

const getShareImage = require('@jlengstorf/get-share-image').default;

module.exports = {
  eleventyComputed: {
    shareImage: (data) => {
      return getShareImage({
        // settings for cloudinary overlays
      });
    },
  },
};

We pull in the get-share-image dependency.
We export an object with a property named eleventyComputed.
Each property of eleventyComputed is a computed property that becomes available in your page templates. In our case, we compute a property named shareImage. The value of it is the result of a call to getShareImage with a bunch of configuration for the Cloudinary overlay.

This shareImage property gets computed for each page within blog/, based on its metadata (that's what the data argument passed into the function represents).

The only dynamic data here for my site was the title property that gets passed to getShareImage:

module.exports = {
  eleventyComputed: {
    shareImage: (data) => {
      return getShareImage({
        // ...
        title: data.title,
        // ...
      });
    },
  },
};

3. Emit the `shareImage` property in your template

I have one base page template for my site. It's based on the pug language.

I updated it to emit a shareImage in the appropriate meta tags if it exists:


meta(property='og:image' content=`${shareImage || 'https://www.stevenhicks.me/static/img/avatar.jpg'}`)
meta(property='twitter:image' content=`${shareImage || 'https://www.stevenhicks.me/static/img/avatar.jpg'}`)

All blog articles will have that shareImage computed property, so they'll emit their generated images. Pages like Home and About won't have a shareImage computed because I put the blog.11tydata.js file in the blog/ folder — so they'll get stuck with an image of my face. MY FACE!

Wrap it up, Steve

The PR ends up being 39 lines added — and over half of that is configuration settings for the text overlay. JavaScript is neat!

You likely found this article because you've already got an eleventy site, but if you don't, you should absolutely give it a look. It's a great option for building a blog or any other site where the data doesn't change frequently. I find it more intuitive than other popular options. This example especially demonstrates how well it's designed for generating dynamic content. Every time I come across a new problem I'm delighted to find there's a simple mechanism built into eleventy to solve it.

¹ Hahahahaha I act like I had agency in this decision but really I didn't see Stephanie's article until I had invested a lot of time in generating an image template based on Jason's articles. I'm as much a sucker for the sunk-cost fallacy as anyone, and here we are.

Sandwiches, Froyo, and Delivering A Cohesive Message

Steven Hicks — Mon, 16 Nov 2020 16:24:21 +0000

I love sandwiches. I love sandwiches. Yes — I would marry them, if I could.

This week my friend Jon asked me what I ate for lunch one day. It was the right day to ask. I spent probably too much time describing to him my current favorite sandwich — the amazing Phoenix sub from local chain The Chocolate Factory. It's not a complicated sandwich but it's got a specific vibe and it does it well. Jon was really impressed 🙄 and asked me if I was a professional sandwich-eater. I did not dispute.

It got me thinking about other sandwiches because, well, yummmm. The "Velvet Elvis" - an absolute chef's kiss that a Milwaukee bar stopped selling years ago, but still makes an appearance in my kitchen — peanut butter, banana, and bacon, "grilled" in a buttery frying pan. If you're a fan of salty & sweet it will make you cry. And a sandwich that I've only heard about — The Dennis — mentioned many times in our work Slack by my New York-based coworkers. It looks like a real monster.

Amazing sandwiches are amazing because they are cohesive. They push you to the edge of your comfort level but they show the restraint and respect to not push you over. They are adventurous...but responsibly adventurous.

Unlike my kids at the frozen yogurt place.

Andes Candies, Boba Balls, Raspberries, and sour Gummi Worms

My kids are currently 9 and 12. It's been a while since we've been to the local frozen yogurt joint (ugh, COVID), but when we've gone their combinations give me shivers. Oooo, they've got strawberry yogurt! And Reese's Pieces! And Andes Candies! Oooo, sour Gummi worms, brownie bites, raspberries, and marshmallows!

Don't get me wrong - each of things, on its own, is delicious:

Reese's Pieces
Andes Candies
sour Gummi Worms
brownie bites
raspberries
marshmallows

But together? An absolute nightmare. Barf city. I won't even finish the leftovers when my kids put together these nightmare concoctions.

I'm so hungry right now but what does this have to do with delivering a clear message?

When you're communicating a complex idea to someone, it's tempting to tell them everything you know. Every single detail you've learned is valuable to you. There are so many cool things you've learned about your topic, and you've put a lot of time into it. You just want to share all of it with your audience.

But is all of it necessary? Is some of it noise? How much of it will distract someone from the main idea you're teaching them?

Think of it like that trip to the frozen yogurt place. You're not my kids — you know better than to mix all the toppings you like in one dish. You'd rather craft something cohesive. Even though you love toasted coconut and Andes Candies and raspberries, you wouldn't dare put them all in the same dish.

Treat your communication and content the same way. Not every concept needs to be shared. Find the ones that fit together and create a cohesive narrative.

If you're drafting an email to your leader, they don't need to know everything that happened in the latest incident. Tell them the most important facts.

If you're writing a blog post, it doesn't have to be 5000 words long. Less is better — if you feel like you've got several stories to tell, tell them in separate blog posts.

If you're building a presentation, don't add slides just to fill time. Your listeners will learn a lot more from one cohesive story than three stories and 13 unrelated points.

If you're building a workshop or course, you don't have to teach every feature. Focus on the features that most align with your learning objectives. Give them resources to learn more when there could be side quests, but your workshop doesn't need to include the entire universe.

Cut out the Gummi Bears if you're going for a mint and white chocolate vibe. Skip the performance analysis if you're going for an intro to building an app. Be the Velvet Elvis. Be the Dennis. Be the Phoenix. Be cohesive. Cut out the noise that doesn't fit your narrative.