DEV Community: Adam Ormsby

Refactored: Hugo Shortcodes Module

Adam Ormsby — Wed, 08 Feb 2023 06:04:18 +0000

This post needed to be Refactored to work on dev.to. So here's a link, a summary, and a chance to check out my personal site where I have more freedom with my format. Cheers!

I previously started an open-source project for collecting user-created Hugo shortcodes, and you can add all of them to your Hugo site as a Hugo module. I filled the starter repo with some of the shortcodes I’ve made for my own site, and I’d love for it to become a place where everybody can share theirs with everyone. Theme-agnostic shortcodes are best, of course, but I really have no guidelines for additions right now. Let’s see where it goes!

Check out my original post for live samples!

aormsby / hugo-shortcodes

A collection of simple Hugo shortcodes, usable as a Hugo module

Per-App Language Preferences in Android 13

Adam Ormsby — Mon, 06 Feb 2023 12:00:00 +0000

Finally, some localization updates for Android! Android 13 was released in the summer of 2022, and it has a ton of cool new stuff to work with. Today, I’ll share a small piece I’m passionate about - per-app language settings.

Up until now, the active locale of a device was based solely on the device’s system-wide language settings. Supporting any kind of in-app option required building custom language pickers and managing user settings in our app’s preference cache. This has been somewhat improved with per-app language settings, and I’m hoping we continue to see more support in this area. Localization can play a huge part in product acceptance and market expansion, and the less custom stuff we have to build, the less we have to maintain down the line.

I had planned to write this post as more of an intro to configuring per-app language settings because I find the documentation to be a bit chaotic and hard to follow. But as I was gathering materials for that work, I found that the Android devs beat me to it! Not only have they blogged about it, but they’ve also added more info to the developer docs since the initial release - though I have to say, it’s still hard to follow with all the warnings and in-page links and… maybe just start with the blog post that I linked below.

So instead of that post, what I’ll do for you instead is give a short overview of the API updates - what’s provided and what isn’t - share some sample code, and talk a bit about what I think is missing from all this.

Recommended Android Resources

If you want to get more familiar with the new feature before reading on, here are the Android developer resources I recommend you look at:

Android Developers Blog - Per-App Language Preferences - Part 1 - basically, the post I sat down to write!
Android Docs - Per-App Language Preferences - messy, but good info if you’re patient

And here are two great videos sadly buried in the docs - one an overview of the per-app setting, and one a great intro into internationalization on Android.

API Feature Overview

What’s improved?

Application language settings can now be system-managed

This is big! After configuring per-app language settings, we can delegate managing a user’s language settings to Android’s system processes and stop worrying about managing language preferences ourselves. Unless you have special needs, you can throw out your custom-built system - you don’t need to maintain it anymore! Let the App Info page handle it for you.

Android now provides purpose-built classes to manage active locales

Static functions provided by the AppCompatDelegate and LocaleManagerCompat classes are now available to get, set, and store application locales, and between the two of them, you get access to all application- and system-level locale information you need. No more messing about with Java’s Locale class and forgetting to also update your active resource configuration - keep things simple with Android’s API.

All of this is backward compatible

If you use all the new classes and rely on the system to do the work, you can configure applications built for older versions of Android to also have most of the functionality of these features. Users won’t get the language selection option on your App Info page (oh well), but because you’ve configured the storage in your Manifest, you can use AppCompatDelegate to set and get locale info your users may have selected through a language picker or some other method.

This is particularly nice because it shows that Android is working to provide the per-app language experience to as many people as it can. I’m not sure how far back the compatibility goes, but I expect this could impact a lot of projects.

What’s missing?

There’s still no in-app language picker

I sort of understand this one. Every app is different, and there’s no good way to provide a language picker that everyone will want to use. We’d probably still see a bunch of custom implementations anyway. Honestly, I’m not convinced in-app language changes are something Android ever really intended to support - at least not without jumping through hoops. Even official Android documentation pushed developers to use the system-level locale and do full restarts to reload with a new language. And the per-app settings are great, but they’re still managed at a system level. I don’t know if a language picker is on their radar. You’ll still be building your own, thank you very much.

All this being said, you can absolutely navigate from your app directly to its system-managed language settings! It may not provide an ideal user experience, but the option is there. LinkedIn did it, which lines up with their track record on UX… but sometimes laziness can be good. I kind of like it.

Configuration is pretty clean… but also there’s potentially a lot of it

Initial configuration is not too bad - add the correct Compat libraries, create a locale XML file, update your manifest for backward compatibility, and add new API code to your app where needed. But how do you populate a language picker? None of those files generate in-app files for you to use, and I haven’t yet seen a function to retrieve all available app locales at runtime. You’ll need at least one other file for your language list, and maybe you’ll need duplicate lists to display language names for different locales. You get it all in and publish.

Now, you want to add a language to your application. Good luck remember all the files you need to add it to. It may not sound like much, but it’s like updating a single string in ten places. Will you remember to update them all? And will you notice if you don’t? Again, I love this feature. I’m just not a fan of how clunky it may end up being to use.

(I’ve built a little plugin that may help with some aspects of this, though! More on that in my next post.)

This feature feels incomplete

The documentation for the feature was crammed onto a single page, it’s filled with caveats about confusing known issues, and there don’t seem to be any finalized, stable libraries for us to import into our projects with confidence. I mean, who wants to use the questionable Appcompat 1.6.0-beta01 or higher? Is it safe? Are we going to have to update when something stable arrives? No idea.

Listen - I’m still excited about this feature, and you should be, too! But I’m also keeping an eye out for updates.

Sample Project

As promised, I have a sample project for you! I updated my l10n_samples project with support for per-app language preferences, and I added a page with a super simple language picker. You can change the language to watch the app update in real time. Bonus - since the new locale changer updates the whole app, you no longer need to update the system language to test out the other samples. (Knew I’d get that in there eventually!)

aormsby / android-l10n-samples

Some localization techniques for Android, read the blog posts on my site :)

Thanks for reading, and I hope you got something useful out of my love for internationalization. I would love to answer questions and chat about this new feature, so please comment with anything you have, and we can nerd out about all this together. Happy coding!

WilyKit: The MacBook That Became Wall Art

Adam Ormsby — Mon, 29 Aug 2022 12:00:00 +0000

Computers get old and die just like us. Parts fail. They become obsolete. You drop them a few too many times. It happens to the best of us. And while there are many ways to dispose of your computer - like donation and recycling - it can sometimes feel like letting go of a friend. I felt that way about WilyKit, so I turned it into wall art. 🙂

After 8 years of service, my 2013 MacBook Air finally died last year. Simple hard drive failure, totally fixable for a hundred bucks. But I needed a more powerful computer to help with my work, so I just bought a new one instead.

I had always planned on keeping the lid somehow, no matter how many times its dinged-up edges caught onto my bags, my clothes, and even my skin when they were feeling extra bloodthirsty. It was just so full of memories! I took that laptop around the world, wrote stories with it, learned to code on it, dropped it on its corners until it wouldn’t shut right, and stickered it full with the signposts of my youth. This was my companion, and I couldn’t just give it away.

WilyKit, the worldly MacBook Air

Finally, after a year of busyness, I performed the surgery. Removing the upper display was quite easy, but after that I was on my own. Though most of the remaining work boiled down to peeling away the layers, the screen material can get pretty nasty when it cracks! I probably should have worn long sleeves, goggles, and a mask to be extra careful with the ’exploding’ glass particles, but it was manageable. Gloves helped a lot. What would’ve helped more is if I hadn’t cracked the glass… but it was a fun project overall.

Bit by bit, I removed the bezel, many layers of screen, the hinge, and copious amounts of adhesive tape used to keep it all together… and voilà! An aluminum memory, all my own.

This computer had a life. And a death. Her name was WilyKit, and she can live on my wall forever.

Localization - Formatting Dates on Android

Adam Ormsby — Mon, 28 Mar 2022 12:00:00 +0000

Hey everybody, let’s talk localization! It’s a passion of mine that I’d like to spend more time on, and I want to share some tips and tricks I’ve picked up in my work. Hopefully, this is only the first of many localization posts to come. Keep an eye out for more in the future!

aormsby / android-l10n-samples

Some localization techniques for Android, read the blog posts on my site :)

Today, I want to look at date formatting for Android applications. When you want to display a date in your app, you have to keep in mind that your customers in different locales will expect dates to look the way they’re used to seeing them. But while there are a lot of date formatting options on Android, they aren’t all robust enough to ‘just work’ the way you need them to.

So first, let’s take a look at two common solutions that aren’t as great as they seem. Then I’ll show you a how to get you exactly the format you want every time.

LocalDate, LocalTime, LocalDateTime - from java.time

These newer Java 8 time classes were built to take the place of the old, clunky Date, Calendar, and DateFormat classes - which you should never rely on completely for localized dates because they’re junk. But this new java.time library is supposed to take care of all the problems developers had with the old ones. So… problem solved?

Maybe for Java devs, but this newer library only works with API 26 or higher on Android. Many projects still support older APIs, so these libraries simply aren’t an option them. However, if you don’t have to worry about that, give these a shot! I’ll see if I can spend some time with these in the future and share my thoughts.

SimpleDateFormat - from java.text

A ton of Android projects take advantage of SimpleDateFormat, and why wouldn’t you? Even the Android docs say this class is for formatting and parsing dates in a locale-sensitive manner. Sounds great, right? Well, I think it’s a bit of a trap.

Java’s SimpleDateFormat is a really attractive solution for anyone who previously tried the DateFormat class and found the formatting options to be really restrictive. DateFormat supports a limited number of pre-determined date-time arrangements, and plenty of desirable options are missing - like showing the month and date without the year. (The year is always there! Augh!! Hair pulling madness.) But with SimpleDateFormat, you can use a string value to explicitly build your own date format.

val myFormat = SimpleDateFormat("MMMM dd, yyyy")
// no locale specified, uses Locale.getDefault()

val date = Date()
println(myFormat.parse(date))

/* sample output
en_US, USA -- December 07, 2008
fr_FR, France -- décembre 07, 2008
*/

Super easy, right? Well, if you have keen eyes and you’re familiar with the date format in France, you’ve already noticed the sample output would be incorrect for that region. France puts the day number before the month!

wrong -> décembre 07, 2008
right -> 07 décembre 2008

And that’s the trap. The main reason to use SimpleDateFormat is to create your own format, but when you create your own format, it’s not really ‘locale-sensitive’ after all! And that’s a bit of a contradiction, I think.

In the end, your hard-coded string is the problem, and no matter what locale your app is running in, that format never changes to match. Now, there’s a way around this - you can extract your format string into a strings.xml resource file and add a ‘translated’ version of the format in the French resource.

<!-- res/values-en/strings.xml -->
<string name="date_format_1">MMMM dd, yyyy</string>

<!-- res/values-fr/strings.xml -->
<string name="date_format_1">dd MMMM yyyy</string>

This works fine, and it might seem reasonable at first. But if your project needs 10 different date formats in 10 different languages… that’s already 100 formatting strings you need to add to your resources and keep track of over time.

If you need that kind of specificity, give it a shot! But I don’t think you do, and I’m about to prove it. Let’s take a look a look at DateUtils - the tool that does all this junk for you.

DateUtils

DateUtils is the absolute bomb for date formatting on Android. It’s flexible, robust, and aboslutely is sensitive to your customer’s active locale. In fact, you’re not even allowed to specify the locale being used when you format with DateUtils, and for the large majority of common date formatting needs, this is absolutely what you want. You want the system to handle most things for you, and DateUtils does this well.

The flexibility in using DateUtils comes from its bitmask formatting flags. They allow you to specify exactly which date or time components you want to appear in your formatted date value, but instead of showing up in the order you choose (like with the string from before), they act as simple on/off switches independent of order or locale. Here’s an example.

println(
  DateUtils.formatDateTime(
    context, // needs a Context
    myDate, // the Date object to format
    DateUtils.FORMAT_SHOW_DATE // the format flag for showind 'date' (includes year)
  )
}

/* output:
en_US -> September 25, 2005
fr_FR -> 25 septembre 2005
zh_CN -> 2005年9月25日
*/

Let’s break this down a bit:

DateUtils.formatDateTime outputs a String value
First param is a Context
Second param is the Date you want to format
Third param is the set of format flags you want to apply to the Date

And that’s it! No explicit format strings, no translated format resources, no locale problems! It just works.

Let’s see how you can use the bit flags to remove the year now.

DateUtils.FORMAT_SHOW_DATE or DateUtils.FORMAT_NO_YEAR

/* output:
en_US -> September 25
fr_FR -> 25 septembre
zh_CN -> 9月25日
*/

The or bit operator takes both of the flags, combines them, and outputs a format with both flags enabled. This works with any number of formatting flags, including things like word abbreviations, numeric dates, showing weekdays, showing time, hiding the month, etc. Here are all the available flags for you.

DateUtils is truly the most flexible, easy-to-use, built-in date formatter I’ve come across for Android projects. If you aren’t using it yet, you should really think about switching to it today.

DateUtils and Kotlin Extension Functions

Want to supercharge your use of DateUtils? Use it in some Kotlin extension functions! Here are some fantastic samples from the Github project linked at the top of this post. These are ready for use in your project today!

Show only the time:

fun Date.asTime(context: Context): String =
    DateUtils.formatDateTime(context, time, DateUtils.FORMAT_SHOW_TIME)

Show date with year, numeric and abbreviation options:

fun Date.asDateWithYear(
    context: Context,
    numeric: Boolean = false,
    abbreviated: Boolean = false
): String {
    var formatFlags = DateUtils.FORMAT_SHOW_DATE
    if (numeric) formatFlags = formatFlags or DateUtils.FORMAT_NUMERIC_DATE
    if (abbreviated) formatFlags = formatFlags or DateUtils.FORMAT_ABBREV_MONTH

    return DateUtils.formatDateTime(context, time, formatFlags)
}

Dynamically show the time (if date is today) or the date (if date is not today):

fun Date.asTimeOrRecentDate(): String =
    DateUtils.formatSameDayTime(
        time,
        Date().time,
        DateFormat.LONG,
        DateFormat.SHORT
    ).toString()

And something super flexible!

fun Date.flexFormatted(
    context: Context,
    showWeekday: Boolean = false,
    showMonth: Boolean = true,
    showMonthDay: Boolean = true,
    showYear: Boolean = false,
    showTime: Boolean = false,
    numeric: Boolean = false,
    abbreviated: Boolean = false,
): String {
    var formatFlags = 0 // starts blank
    if (showWeekday) formatFlags = formatFlags or DateUtils.FORMAT_SHOW_WEEKDAY
    if (showMonth) formatFlags = formatFlags or DateUtils.FORMAT_SHOW_DATE
    if (!showMonthDay) formatFlags = formatFlags or DateUtils.FORMAT_NO_MONTH_DAY
    if (!showYear) formatFlags = formatFlags or DateUtils.FORMAT_NO_YEAR
    if (showTime) formatFlags = formatFlags or DateUtils.FORMAT_SHOW_TIME
    if (numeric) formatFlags = formatFlags or DateUtils.FORMAT_NUMERIC_DATE
    if (abbreviated) formatFlags = formatFlags or DateUtils.FORMAT_ABBREV_MONTH or DateUtils.FORMAT_ABBREV_WEEKDAY

    return DateUtils.formatDateTime(context, time, formatFlags)
}

Thanks for reading! I hope this little trip into the world of localization was as fun for you as it was for me. Please - ask questions, give opinions, and share your ideas in the comments! I’d love to swap date formatting stories with you. Cheers!

Using Hugo Modules Instead of Git Submodules

Adam Ormsby — Mon, 13 Dec 2021 06:00:00 +0000

Hey there, folks! I know I’m a little late to this party, but it’s time I talked about working with Hugo Modules. A while ago I wrote a post on building a Hugo site with Git submodules (see the series links above!), but let’s be honest - that’s a lot of work.

Setting up git submodules and managing them over time can be a headache, especially for larger projects, and that kind of configuration just brings in a lot of overhead you may not want to deal with long term. For a more manageable project, I now recommend Hugo modules.

Benefits of Hugo Modules

The Hugo module system is powered by Go modules, which I don’t really know anything about. Feel free to explore that on your own. What I do know is this –

Hugo modules are:

quick to configure
less work to implement in your project repo
great for customizing your site’s look without needing to fork a full theme
an awesome way to share open-source code for Hugo

No more managing git submodules. No more using complex commit/push/pull schemes to work with your site repository. No more confusion about how to modify and build somebody’s theme to fit your specific project needs. Everything you were using git submodules for can (and probably should) be converted to using Hugo modules.

How to Configure a Hugo Module

If all you want is to include a module in your project, setup is super simple. In your project configuration file (config.toml or yaml or whatever), add the path to the module you want to import. For online resources, add a full URL. For local resources, add a filepath.

[module]
  [[module.imports]]
    path = "github.com/aormsby/hugo-shortcodes"
  [[module.imports]]
    path = "github.com/aormsby/hugo-custom-layouts"
  [[module.imports]]
    path = "github.com/aormsby/papermod-custom"
  [[module.imports]]
    path = "github.com/adityatelange/hugo-PaperMod"

A few things to take note of:

This example is toml. Yaml has different syntax.
This is just configuration. You still need to initialize Hugo modules. (More on that shortly.)
The order of your modules is important!

Modules on the bottom will get loaded first, so keep anything you don’t want overridden higher up in the module chain. All conflicting files during module import will use the last file loaded. If you want to override anything from a module you can use this bottom-to-top loading to your advantage. I show an example of this later on.

Next, you have to tell your project that you want to import these modules on build and how to import them. In your terminal, run the following commands in order –

> hugo mod init github.com/<your_username>/<your_project>
# initializes the hugo module system in your project (I used my repo URL here, any string seems fine)

> hugo mod get
# gets a reference to the latest version of each module, stored in 'go.mod'

> hugo mod vendor
# optional, downloads a local copy of the modules

Now when you build your site, the Hugo module system pulls data from all of the paths in your configuration file and makes a build with all of the data from the included modules. This could be extra layouts, assets, i18n data, shortcodes, etc., and it’s all included in your site without a whole bunch of extra setup.

Keep in mind that the stored module references in go.mod are references to a specific commit on a github repo, not a branch. You should be able to configure these by modifying the references (or perhaps there’s a command for it I don’t know), but maybe think before you do. If you point to a branch head, for example, every build could pull in new changes you haven’t vetted for your site. Sticking with a specific commit hash is probably better here.

You can run hugo mod get -u to update references to all of your modules, but keep in mind this does not update the local ‘vendor’ copies. You have to run hugo mod vendor again for that.

A note on hugo mod vendor - So far I’ve seen two benefits of downloading local copies of your modules.

You can see all of the assets you’re building from local module folders and modify them as you wish. I’ve found it nice to play around with theme settings and then save the changes in my own modules to override them. Vendorizing creates a code playground for you in this way.
If you ever want to save a specific module configuration as part of a git commit, vendorizing allows you to include the modules at the state they’re in when you last confirmed a good build. It’s handy.

There’s a lot more you can configure with Hugo modules, but I never needed more than this for a simple site like mine. If you want to explore your options, I encourage you to check out Hugo’s module docs for more information.

How Modules Can Help You

As you saw before, I have four different modules I import for this site. Each has a different use case that has helped me organize my site and my assets better, and I want to share them here to give you some ideas on how you can take advantage of modules, too.

Basic Theme Customizations

I’m currently using the Paper Mod theme as the core of my layouts, but I wanted to customize a few pieces. If I was using git submodules like I used to, making customizations would have required forking the theme setting up a branch for my changes and referencing my entire theme fork as a submodule. But that’s overkill for a few small changes.

Instead of that, I created my own module called papermod-custom where I have a few custom layout files… and nothing else. Seriously, my module only has 4 files. This works because of the bottom-to-top module imports we talked about earlier. First, I import the core theme module that contains the base theme. Then my customizations are imported, and since I named my layouts the same as in the core theme module, any conflicting files are overridden with my customizations.

Boom. Customized theme with little to no effort.

My papermod-custom file hierarchy –

papermod-custom #root
 |-assets
   |-css
     |-extended
       |-userCustom.css # new file, additional styling
 |-layouts
   |-_default
     |-project-list.html # new file, additional layout
   |-partials
     |-home_info.html # name conflict, overrides core theme file
     |-social_icons.html # name conflict, overrides core theme file

Common Site Code

Let’s say you have 10 Hugo sites that you’re managing, and you have the same layout file in each of them. It’s an awesome layout file - why wouldn’t you reuse it? Well, instead of updating it 10 times whenever you feel it needs a change, you can save it in your own Hugo module that you import into all of those projects.

Perfect example - I like my 404 page, and I always want to enable disqus comments. I can put those two layout files into my hugo-custom-layouts module and import them wherever I want. Modules make it easy to share things.

Additional Elements

I like shortcodes, and I want to add someone’s public shortcodes to my site. If I start a collection of shortcodes and make it a Hugo module - like so - I can add them to any project with a quick module import. It’s not just true of shortcodes, of course. Any kind of Hugo-supported asset can be thrown into a module for a quick addition to your project.

I hope this shows you that Hugo modules are pretty fantastic to work with and how much nicer they are than git submodules. If I’d known that a year ago, maybe I never would have written about git submodules! No matter. We’re here now, and that’s what counts! Happy coding!

Github Action: Fork Sync With Upstream

Adam Ormsby — Sat, 10 Oct 2020 10:00:00 +0000

On github:

View action on Github
Follow @aormsby on Github

These days, automation is key to saving time and minimizing errors. But if you’ve never worked with automation, it’s good to start small. This was exactly my thought when I made my first Github Action. It’s available now on the Github Marketplace if you want to try it out - Fork Sync With Upstream. Let’s see what it does!

How It Works

Remember my post on making modifications to a Hugo theme? We forked a theme repository and saw how to make custom modifications to our local version of it. Whenever we wanted to pull updates from the source repo, we ran a bunch of git commands to pull new commits from the upstream repo and then push them to our fork. Syncing with commands like that is all well and good, but let’s save ourselves a little time by automating that process.

The Fork Sync With Upstream action takes care of all those updates for us without any work on our part. At its core, the action is a shell script that runs on a Github server and automatically does the following:

Checkout your fork (needed for the sync process)
Set the upstream repo to pull updates from
Check if there are any new commits to sync (and print the commit information)
Pull from the upstream repo
Push to the target branch of the target repo

Just add my action to a Github workflow file in your repo, and voila! All that update work you were doing manually is now automated!

Running the Fork Sync

If you’ve never made a Github workflow before, it’s a little bit confusing to figure out at first. (At least, it all was for me.) I recommend reading some of the Github Actions documentation to get a better feel for how all the pieces work together.

For those looking for a quick sample workflow, here’s the one I’m using in my fork of Hello Friend.

on:
  schedule:
    - cron: '0 7 * * 1,4'
    # scheduled at 07:00 every Monday and Thursday

jobs:
  sync_with_upstream:
    runs-on: ubuntu-latest
    name: Sync master with upstream latest

    steps:
    - name: Checkout master
      uses: actions/checkout@v2
      with:
        ref: master

    - name: Pull upstream changes
      id: sync
      uses: aormsby/Fork-Sync-With-Upstream-action@v1
      with:
        upstream_repository: panr/hugo-theme-hello-friend
        upstream_branch: master

    - name: Timestamp
      run: date

(Actual file at .github/workflows/wf-fork-sync.yaml.)

This may not be the only Action built for syncing with a remote, but it certainly has a better README than many of the other ones I’ve seen! For more details on using input variables and options, visit the official Marketplace page for Fork Sync With Upstream.

Behind The Scenes

Just some extra dev notes if you’re interested. 🙂

Github Actions currently have to be created as Docker container actions or JavaScript actions. I went with JavaScript because it’s more familiar to me, and I don’t need to run this action in any specific Docker-simulated environment.

The action.yaml file declares all the input variables needed for the action and then runs main.js. At the moment, main.js exists only to run the shell script upstream-sync.sh, which is where the sync really happens. This could change in the future.

I used both @actions/core and @actions/exec from the Github Actions Toolkit to simplify the functionality in main.js. The shell script is called by exec, which I think of as an ‘easy’ version of the exec function in node.js. Some of the actions and tools that Github has made are really quite useful.

Anyway, try it out! Let me know how you like it. I’m also planning on turning my shell deploy script into an action, so be on the lookout for that. And if you like my work, please consider buying me a coffee. Cheers!

View action on Github
Follow @aormsby on Github

How to Set Up a Hugo Site on Github Pages - with Git Submodules!

Adam Ormsby — Tue, 29 Sep 2020 22:00:00 +0000

Overview

I made this blog site using the Hugo static site generator because I wanted to build a simple site where I could have full control over every detail - not like using Wordpress, Squarespace, or Wix. With Hugo, I can add and remove templates and functionality at will, and it feels great to have that kind of power. You probably ended up here because you want that, too.

In this post, we’ll take a look at how to:

Create a new Hugo site
Host it on Github Pages
Use git submodules to separate our site’s key directories into manageable chunks

We’ll also go over the benefits (and drawbacks) of using submodules in our setup, and by the end of this post, we’ll have a new Hugo project hosted on Github Pages and ready for action.

Please note: I’ll talk a little about branching inside our submodules in the next post. To keep things simple, you can assume that I’ve stayed on the main ‘master’ branch of each git module. Also, I work on a Mac so all of my commands are Mac-specific.

Why Git Submodules?

Every git project is stored as a versioned code repository (a repo). Git submodules allow us to reference other repos within a project, which effectively puts a project inside a project (or a repo inside a repo). The submodule’s code can then be used by the main project, but the submodule maintains its own commit and branch history, which separates the projects and can be a pretty powerful thing.

In a Hugo project, most of what we see is source data - the raw files we manipulate before building our site. After Hugo builds our site, our ‘ready-to-publish’ data gets output to the public directory of our project. By turning the public folder into a submodule of the main project, we can treat it as a separate entity with a separate history. Let’s see what that process looks like.

Install Hugo and Create a Project

First things first, we need to install Hugo! We’ll run through the key Mac commands quickly here, but if you need any more help you can always check Hugo’s ‘Quick Start’ guide for more details or information about installing on Windows and Linux.

# 1. Open a terminal. Alternatively, you can VSCode's terminal - it's great!

# 2. If you don't have brew installed --
> /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

# 3. Install Hugo
> brew install hugo

# 4. Move to the directory where you want to create the project.
> cd [MY/DIRECTORY/NAME]

# 5. Create a new Hugo project. I'll use "hugo-gh" as my sample name.
> hugo new site [SITE_NAME]

So far, we’ve installed Hugo on our machine and created a new template project named hugo-gh. If you’re following Hugo’s Quick Start guide, you may be tempted to move straight on to adding a theme, running the local server, and seeing your site. We’ll absolutely get to all that, but since our goal here is to use git submodules to manage the different parts of our project, it would benefit us to go through some setup steps on Github first.

Push Project Files to Github

Let’s get our project versioned with Github right away! We’ll need to create two repositories on Github - one to host our source code and one to host our live site data.

Source Code Repository

Hot tip: If we create a repo without a README file it’s easier to avoid accidental history conflicts when pushing a local project to a fresh repo. We can always add one later.

When our new empty repo is ready, go back to the terminal window and input these commands to initialize our local repo and push to Github.

(If you already have a public folder in your project, delete it before doing any of this to save some headaches.)

# 1. Enter the project folder.
> cd hugo-gh

# 2. Initialize git locally.
> git init

# 3. Set our new Github repo as the remote for our local project
# (my sample URL - https://github.com/aormsby/hugo-gh-pages-source.git)
> git remote add origin https://github.com/[GITHUB_USERNAME]/[SOURCE_REPO_NAME].git

# 4. Stage all files for commit.
> git add .

# 5. Commit files.
> git commit -m "committing our hugo template"

# 6. Push to the remote master
> git push -u origin master

And there you have it! Our project source is now pushed to our Github repo.

You could absolutely set up your source repo to be the host of your site using Github Pages, and you may have good reasons for doing so. That would require a different setup that you can find steps for on Hugo’s Host on Github docs page. However, if we do this we ignore our chance to separate public into a separately versioned repo. To accomplish this, we have to create a second repository.

Live Site Data Repository

Let’s add our first submodule to the project! Remember, our goal is to separate the commit histories of our project source and our site build output to the public directory. First, we create another repo on Github for our public site data. I’ll call mine hugo-gh-pages-public. In our local terminal, run the following command from the project root.

# my sample URL - https://github.com/aormsby/hugo-gh-pages-public.git
> git submodule add https://github.com/[GITHUB_USERNAME]/[PUBLIC_REPO_NAME].git public

Sweet! We now have a submodule initialized and loaded into our project under the public directory, and it’s already connected to its remote origin. Let’s do a quick site build just to put something into the remote Github repo. The commands needed are a little tedious, but it’s good to know the process.

# 1. Perform a site build and output to 'public/' directory.
> hugo

# 2-4.
> cd public
> git add .
> git commit -m "first build"

# 5. Return to the project root.
> cd ../

# 6-7.
> git add .
> git commit -m "first build - update submodule reference"

# 8. Push the source project *and* the public submodule to Github together.
> git push -u origin master --recurse-submodules=on-demand

It’s super important when we build to make sure that we commit and push both the public submodule and the main project. For each submodule commit, the project updates its reference to the submodule to maintain its connection to the correct commit. The --recurse-submodules=on-demand command pushes all of the project modules at the same time.

submodule reference updated vied in the terminal after git status command

submodule reference on github after pushing

Enable Github Pages

On Github, our hugo-gh-pages-public repo now has some data. Go to the repo settings and find the Github Pages section. Set the Source of our site to master branch.

With these settings, Github Pages will be able to serve our site from the repo starting with the index.html file in its root directory. The live URL for the site is built from your user name and the project name. Here’s my sample URL - https://aormsby.github.io/hugo-gh-pages-public. At this point, our site won’t display anything except an error message since Hugo requires some page templates or a full theme to generate the pages. That’s fine. We just wanted to add some stuff to Github so we could access the repo’s Github Pages settings as we just did.

Benefits of Making a Submodule in ‘Public’

To recap, here’s our project setup so far.

Let’s take a look at some of the benefits of this system.

No accidental deploys - With our source code separated from our site data, it’s a lot harder to accidentally push unfinished work to our live site. Even if we make builds locally and push our source changes, we won’t push to our hugo-gh-pages-public repo without explicitly running commands to make that happen. *sigh of relief*
Separate rollbacks - If something goes wrong, the main project and public project can be separately rolled back to earlier commits. (Just remember to commit the modified reference updates to the main module!)
Branch work is easier - Changing branches in our source project will not change branches in our subproject (and vice versa). This is another nice safety buffer, and it gives us more options for branching in either module.
Ready for site staging - We can easily create development branches in the public submodule that could be served as development subdomains of our site and used for testing new features.

Some Drawbacks

Of course, it’s always wise to assess the risks of any setup, and there are many opinions on the ‘perfect system’ in the dev world - tell me about yours in the comments! To shine some light on the potential downsides of our submodule use, here are some of the concerns I’ve thought about.

Juggling multiple repos - With more submodules comes more administration. It doesn’t seem so rough for me, but right now I’m working on my own. Running interlinked, asynchronous repositories comes with its own challenges in a team setting, particularly if someone forgets to push updated submodule references to the source.
No site staging with Github Pages - While the public submodule can branch and easily store data used in a site staging system, Github Pages does not seem to support subdomains from different project branches. That’s more of a Github Pages drawback, but it means we can’t take full advantage of our setup just yet. Another hosting platform might offer a solution - I’ll be looking into some options in a future post.

Add a Theme to Our Site

Wait, we’re not finished just yet! Remember how our site doesn’t display anything except an error message right now? We need some templates to get things up and running. The easiest way to get some is to add one of the many Hugo themes to our project. Since I’m already using it on this site, I’ll use the Hello Friend theme in my example.

Many guides for adding a theme - including Hugo’s own docs - just tell you to clone the theme repo or download and copy it into your project’s themes directory. But hey, our theme today is submodules, so I propose that we add a theme to our project as a submodule since doing so would give us some nice dev powers over the theme.

Look at it this way - directly cloning a theme into our project has two major problems.

We can’t push our commits to the theme’s source repo, because we don’t have permission. This means we can’t make changes to the theme and then keep them versioned.
A cloned project in a subdirectory isn’t linked with the main project. When a submodule is added to a project, the main project adds a reference to it in a file named .gitmodules. Without that reference, the main project and the theme project aren’t connected, which could cause confusion or problems down the line.
Also, Git doesn’t allow repos inside repos that aren’t submodules. See this ‘simple dev’ post for details.

Let’s solve each of those problems in one fell swoop.

Make a fork of the Hello Friend theme so we have our own copy of it.
Add our forked repo as a submodule in our main project’s themes folder.

# Add the theme submodule from the root project folder
# my sample URL - https://github.com/aormsby/F-hugo-theme-hello-friend.git
> git submodule add https://github.com/[GITHUB_USERNAME]/[FORKED_THEME_REPO_NAME].git themes/hello-friend

Boom! Now we have our own fork of a cool theme referenced as a submodule of our Hugo project. We can make any changes we want and push them to our own repo. I’ll get into more advanced uses of theme submodules in another post, but for now we have the basics in place.

Things won’t be quite right without tweaking a few final settings, so let’s open the config.toml file in our project root so we can modify a few lines.

Add a line so our site builds using our new theme.

theme = "hello-friend"

Because we’re making a Github ‘project site’ we want to change thebaseURL to match our project URL.

baseURL = [GITHUB_USER_NAME].github.io/[PROJECT_NAME]

For example, mine will say aormsby.github.io/hugo-gh-pages-public. To understand the difference between setting up a project site and a user/organization site, see these Github Pages and Hugo docs on the matter.

There are a lot more theme settings that can be adjusted in your config file, but you can explore those on your own. Let’s deploy!

Test Site and Deploy

Running the terminal command hugo server in our project root spins up a local live preview of our site that can be viewed in a browser at localhost:1313. Run this now to check if our site is displaying anything (it should!), and then stop the process with Ctrl+C. Just like we did before, run the commands to build, commit, and push. (Note that we didn’t change anything in the theme, so we don’t have to push anything in that repo.)

> hugo
> cd public
> git add .
> git commit -m "build with theme"
> cd ../
> git add .
> git commit -m "build with theme - update submodule reference"
> git push -u origin master --recurse-submodules=on-demand

After a few minutes, refresh the live site at https://[USER_NAME].github.io/hugo-gh-pages-public and check out your new site! Congratulations, you’ve done it!

Next Steps

Theme Branches and Getting Updates - I’ve worked out a nice branch-based system for continuing to get updates from the original theme repo while adding and changing things our forked theme repo. It’s super useful. Next post!
Better Deploy - I’m sure you noticed that we had to use a plethora of terminal commands to add, commit, and push files to the main project and its submodules to maintain proper commit references. I wrote them out in this post for clarity, but this process is definitely something we can automate. Hugo’s docs provide a basic deploy script to do this, and I’m working on a greatly improved version of it that I’ll write about when it’s ready for action.
Custom domains / CNAME - You may want to set up a custom domain on Github Pages. Here’s some good documentation on how to do that.

Thanks for reading! ~

Hugo Themes - Making Modifications while Keeping the Source Up-To-Date

Adam Ormsby — Tue, 29 Sep 2020 22:00:00 +0000

Overview

In today’s post, I want to take a look at how we can keep our Hugo themes up-to-date. ‘But my theme works fine!’ you say. ‘Why would I ever need to update it?’ Well, when you download or clone a Hugo theme from its hosted git repository, you’re getting its latest version at that point in time. So even if the theme’s creator pushes updates to the code - with fixes, new features, and outside pull requests - you’ll still be stuck using the version you downloaded before. While this is fine in some cases, many of us want the newest updates so we can take advantage of a stronger, improved theme.

The key to this process is using git submodules to track and pull updates to a theme with just a few quick commands. If you followed my previous tutorial on adding a theme to your Hugo site as a git submodule, you already have a functional Hugo site with a (forked) theme included. Starting there, let’s see how to connect our theme to its remote source repo, which git commands we need to update it, and even how to safely make our own custom changes to the theme while keeping it in line with the source.

Just the Updates, Please

If we don’t intend to customize a theme at all, we can keep our workflow super simple. All we need to do is add the theme repo into our project as a submodule and periodically run a few git commands to check for updates. For the moment, let’s pretend we haven’t forked the theme repo and see what it would be like to clone the source directly. We’ll see an example with our fork in the next section.

Here’s a quick refresher command for adding a submodule to our project.

> git submodule add https://github.com/panr/hugo-theme-hello-friend.git themes/hello-friend

You might be thinking you can simply run git clone instead of adding the theme as a submodule, but our site is already set up as a git repo so that doesn’t work here. Git doesn’t like having .git files in a subdirectory without a .gitmodules component in the root. For more on that, see this ‘simple dev’ post.

In the example above, we added our submodule by pulling the theme’s source repo directly into our project. With the git remote command, we can easily verify that it’s connected to the correct remote repo.

> cd themes/hello-friend
> git remote --verbose
origin https://github.com/panr/hugo-theme-hello-friend.git (fetch)
origin https://github.com/panr/hugo-theme-hello-friend.git (push)

And there it is! With this setup, all we need to do is pull updates straight from the source. From the theme’s directory, we run the following terminal commands.

# optional: fetch any updates to view changes before merging
> git fetch origin
> git status
[status output here]

# pull updates and merge them automatically
> git pull

Since we’re not making any theme changes in this workflow, we shouldn’t expect any merging issues during the git pull. We don’t really need to fetch or check the status of the repo, but it’s always nice to see incoming changes before we bring them in.

And that’s it! Getting updates every so often is no problem if we don’t intend to make our own customizations to a theme. But hey - customization is where the fun really begins.

Updates and Theme Customization - A Parallel Branch Workflow

Here’s the scenario - We want to add a Hugo theme, make custom changes, and still to get updates from the theme’s original source. The method used above doesn’t work here for a few reasons:

We can’t push our custom changes to the theme’s source repo. We simply don’t have those permissions on the repo. Our local changes will not be safely stored anywhere, and any pulled updates may not be easily mergeable.
Incoming updates may require merges. If we’ve modified some of the theme files that are being updated in the source, then a simple git pull will not be enough to deal with the resulting conflicts.

Thankfully, a more advanced git workflow would help deal with these issues. Remember how we set up a forked theme repo in the last post? Now we get to take advantage of it. Crucially, the forked repo is ours, so we have full write access to it and don’t need to worry about repo permissions. We can do whatever we want.

Add a Remote Repo

However, our fork doesn’t automatically receive source updates either! Here, an additional step is needed to link our fork back to the source repo. Earlier, in the direct update workflow, we used the git remote command to verify that we had the correct remote repo connections. Running it in our forked theme directory, we see only the fork as our remote.

# my forked repo URLs as an example
> git remote --verbose
origin https://github.com/aormsby/F-hugo-theme-hello-friend.git (fetch)
origin https://github.com/aormsby/F-hugo-theme-hello-friend.git (push)

To get updates from the source repo, we need to add a second remote repo - yes, git can do this! We simply run this command in our working directory and check our remotes again to verify the connection.

# add a new remote repo connection
> git remote add upstream https://github.com/panr/hugo-theme-hello-friend.git

> git remote --verbose
origin https://github.com/aormsby/F-hugo-theme-hello-friend.git (fetch)
origin https://github.com/aormsby/F-hugo-theme-hello-friend.git (push)
upstream https://github.com/panr/hugo-theme-hello-friend.git (fetch)
upstream https://github.com/panr/hugo-theme-hello-friend.git (push)

Ta-da! Our local theme repo is now linked to both our forked repo and the source repo, which we called upstream. (This is a conventional label, but any name you choose is fine.) At this point, our setup already supports getting updates from the source, but the process is a little different than before.

Normally, when we’re updating directly from a single remote repo, all we need to do is pull the data to our local directory so that the local matches the remote. We’ll still do that here, but we explicitly pull data from the upstream repo we added. We’ll also have to make an additional push to our forked repo - origin - to update it with the new code. Here’s how that works compared to the previous example.

Left: simple update / Right: updating with a fork

Assuming you’re still on your master branch, here are the modified commands needed to pull and push the updates.

# optional: fetch any updates to view changes before merging
> git fetch upstream master
> git status
[status output here]

# pull updates and merge them automatically
> git pull upstream master

# push source updates to forked repo
> git push origin master

With no local changes to our theme, the commands run smoothly with no errors - and we want to keep it that way! Simple is good, and an update process that doesn’t result in merge conflicts is always nice. But how do we manage to avoid conflicts if we want to customize the theme? Simple - don’t do your work on master.

Branching, Customizing, and Merging Updates

Although you technically could make theme customizations on the master branch, this increases the chances of having to resolve merge conflicts every time you try to pull an update. It’s fine if that’s how you want to work, but I highly recommend avoiding it. In my experience, it’s often best to keep master sparkly clean and do your dirty work on a different branch.

For example, I do my theme in a separate branch that is always kept in parallel with master. I’ve named it working, as it’s the branch that I work in. (Not exactly clever, I know.) Since it has all of my theme customizations, it’s also the branch that I use when I build my site. Every so often, I switch back over to master, go through our update process, and then explicitly merge any new theme updates from master into working so that I have all the latest features in my working branch. Visually, it’s kind of like this.

This is just one possible theme customization workflow, but we can benefit from a setup like this. For example –

Our workspace is separate from our update environment. Getting theme updates will always be a painless process, and we have more control over merging when the time comes.
Commit histories are clearer. Histories are much easier to read if source updates and custom work don’t get tangled up, and this can be super helpful when performing more advanced git actions on a branch.
All the data is on our forked repo. The customizations and the pulled updates all live on our fork, which means we have full control of the data.
We can test on master before merging. If we’re unsure of how a new feature will display on our site, we can run a build using the master code. It’s a nice clean test bed before performing a merge.

The drawbacks are much the same as I mentioned in the last post with the addition that now we’d have at least one more working branch to manage. However, I’ve found this parallel branch workflow quite nice after getting more familiar with it. Perhaps you will, too.

Here are the commands we can use to get our working branch up and running. (You can choose a better name if you’d like.)

# create a new branch and switch to it
> git branch working
> git checkout working

We make some changes, commit our work, and push to our forked repo (see these commands for help). Time passes… and we want to get some updates on master.

# switch to master branch
> git checkout master

# quick update
> git pull upstream master
> git push origin master

Finally, we switch back to working and merge in the updates from master.

# switch to working
> git checkout working

# merge master into current branch (working)
> git merge master

I won’t get into resolving a merge here, so let’s cross our fingers and hope there are no conflicts to deal with.

something to look forward to :)

So yes - customizing your theme with this kind of workflow does take a bit of setup, and there are a few extra steps and things to keep track of to manage our code well. However, using git submodules and a parallel branch workflow do give us more leverage over our codebase, and I think that makes the extra work worthwhile. I’ve been using this workflow with my site for some time, and I don’t think I’d change anything about it.

That said, there are a number of other methods that people use for managing their themes, and it’s worth exploring them to see what fits your needs best. I recently read a post about managing your themes as Hugo modules by Nick Gracilla that I found quite interesting, and I think it’s worth checking out if you’re looking for an alternate way of doing things.

Next Steps

Better Deploy - I’m still working on a better site deploy script! It’s looking good so far, but it’s not quite ready.
Automatic Builds - This would still be great to have, so if I get around to it I’ll be sure to share my work.

Until next time! ~

Build and Deploy: A Shell Script for Hugo Sites

Adam Ormsby — Mon, 22 Jun 2020 04:30:00 +0000

Shell Scripts are Awesome

I don’t know about you, but I love writing code that automates my work. I finally had the chance to do just that, and now I’m the proud owner of a shell script that builds and deploys my Hugo site without any extra help from me - and it even works with my git submodule setup!

So why do this? Well, when I began working with Hugo, I ran across a rudimentary deploy script in their documentation. Since I was still familiarizing myself with Hugo, it hadn’t occurred to me yet to automate the build process, and I immediately fell in love with the idea. However, I could see the script would need major improvements to do what I wanted, and I was completely new to shell.

At first, I struggled a bit with writing shell code (especially keeping things POSIX-compliant for portability!), but I stumbled upon the shellscript.sh tutorials and dove straight in. Thanks to those fantastic tutorials, I now have a sweet script to show off and perhaps even a newfound love for the shell. Apart from variable expansion struggles, it’s really quite elegant!

I find my hugo-deploy.sh script does help keep builds following a standard process, but I think I’ll really see it shine when I get around to scheduling automated builds on a remote server. Exciting!

full script at end

Deploy Script Functionality

Be sure to place this script in your site’s root directory if you intend to use it. It’s built with my git submodule setup in mind. But I don’t think it would be too hard to modify it for something different.

Order of Operations

From start to finish, here’s what the script does.

Update the automatic build number (included in the git commit)
Check for any user options (see below)
Check if your active repo branches match your intended deploy branches
- Production and development branches can both be set
Check if your local repo is up to date (no merge conflicts allowed!)
(Optional) Clear out the site output folder before the Hugo build
Run Hugo build command
Git add, commit, and push to your specified remotes

If any point in this process fails, the script exits with an error message detailing where the issue occurred. You’ll have to fix the issue before running the script again. Thankfully, most of the issues are quick fixes. If you have to do a merge, that’s on you!

‘Settings’ Variables

There are some ‘settings’ variables at the top of the script that you may need to modify to match your project.

# BUILD/DEPLOY SETTINGS - edit as needed for your use case

PUB_SUBMODULE="public" # name of output folder where git submodule is located
IGNORE_FILES=". .. .git CNAME" # space-delimited array of files to protect when 'fresh' option is used
DEV_BRANCHES="dev dev" # development branches to build on and push to, 1-root, 2-pubmodule
PROD_BRANCHES="master master" # production branches to build on and push to, 1-root, 2-pubmodule

PUB_SUBMODULE holds the name of the folder where your public site submodule is located. The default output folder is public, which is where I have my submodule for the live site. If you’ve changed your output folder in the site config, you’ll need to update this to match it. The variable value is relative to the root directory, so if you’ve buried your output folder then set the path accordingly (e.g. my/public/build).

IGNORE_FILES is used with the ‘fresh’ option (see below). It acts as an array of filenames that will not be deleted from the output folder if you select the ‘fresh’ option. It’s mostly to protect non-Hugo files.

PROD_BRANCHES are the two production branches on which you intend to build and push your site data. The first branch name matches the preferred deploy branch for your site source data (in your root directory), and the second branch name matches the preferred deploy branch for your live site (the supposed submodule). These are the values the script uses to check your active branches in step 3 above. It’s a nice safety check.

DEV_BRANCHES works the same way for a development build, which is enabled through the user options.

User Options

Sometimes you need a little customization! That’s where my included script user options come into play. Hopefully these options can help you out a bit.

# Script Options
./hugo-deploy.sh [-d|-f] [-m "COMMIT_MESSAGE"] [-o "HUGO_OPTIONS"]

-d ‘dev’ => deploys to development branches set in DEV_BRANCHES list (default is PROD_BRANCHES)
-f ‘fresh’ => deletes public directory data before rebuild (skips files in IGNORE_FILES list)
-m ‘message’ => appends a custom commit message to the auto-build string, works exactly like git -m
-o ‘options’ (for Hugo build) => includes Hugo build options during deploy process (default none), all hugo build options are valid
-h => ‘help’

# Usage Example
./hugo-deploy.sh -d -f -m "Deploying like a rockstar!" -o "--cleanDestinationDir"

Fun Stuff: If you rename the script file to hugo-deploy.command, you can run the script with a double click! I haven’t actually tested this, but I think it just runs without options. Try it out!

Disclaimers / Seeking Shell Wisdom

I ran into some issues with this script that I couldn’t totally avoid. Please be aware of them if you try out my script.

Variable expansion - I’m aware that the variables I use in my various git commands are not double-quoted. (And just in case I didn’t notice, ShellCheck yelled about them in bright colors.) However, adding quotes or modifying the expansion caused the commands to be incorrectly formatted and resulted in failure. They work as they are now, but I’m curious to know if there’s a way to improve this. I’d love to get advice from experienced shell coders.
Limited Testing - I work on Mac OS, so I know this script works on that system - at least on my machine! I have no idea about Windows or Linux. If you try it out, please let me know!

The Full Script

View on Github