DEV Community: Christian Specht

Tinkerforge Weather Station, part 3 - Continuing the project after a decade

Christian Specht — Tue, 03 Sep 2024 21:00:00 +0000

Once upon a time…I bought a Tinkerforge Weather Station, wrote some C# code to interact with it, and managed to get it to run via Mono on a Raspberry PI with Linux.

My end goal was to measure temperature etc. in my garden, and somehow show it on my website.

(There are already two old posts about this on my website but not here, because they are from before I started mirroring my posts to here)

Then I put the weather station and the Raspberry PI in a drawer…and completely forgot about them. That was about eleven years ago!

Now I found both items again and decided to continue the project.

But of course, a decade is quite a long time for software - a lot of my original 2013 plan is not relevant anymore today:

The new plan

I don’t need Mono anymore. Ten years ago, the “new” .NET (a.k.a. .NET Core) didn’t even exist, but today I can just create a project in the current .NET Version (on Windows, because this is what I use) and it will just run on any Raspberry PI (very likely not on Windows).
Back then, I didn’t want to put the PI inside the weather station because I wanted to keep it in the house and also use it for other things (I didn’t explicitly write it, but I remember that I actually had home automation in mind). So I bought a Tinkerforge WIFI Master Extension.

But:

a) now there's the Raspberry PI Zero W, which is ridiculously cheap AND has Wi-Fi

b) Home Assistant, the home automation solution I evaluated so far, seems to "prefer" running on a dedicated machine anyway.

I have zero Home Assistant experience so far, but just from reading the documentation, it looks like that: the easiest installation method comes with its own operating system, and manually installing HA on an existing OS seems to be the hardest installation method.

⇒ it doesn't make sense to have a Raspberry PI running in the house 24/7 that does nothing but power the weather station. So the obvious choice is removing the Wi-Fi module and putting a Raspberry PI Zero W directly into the weather station.

(I still have the old Raspberry PI from 2013, but it has no Wi-Fi)
I wanted (and still want) to periodically save the current temperature, humidity etc. somewhere and do something cool with it.

In 2013, the Tinkerforge docs recommended Xively for this, but today their website doesn’t even load anymore, and according to Wikipedia they were bought by Google years ago.

In 2024, there are waaay more cloud services, and there's probably one that makes perfect sense for a requirement like this (and has a free tier that's enough for my needs).

I don't have any experience building stuff for/in the cloud yet, so I will use this project to get my feet wet.

So this is a but vague, but I'm planning to:
- periodically send temperature etc. somewhere to The Cloud™ and store it there (not for all eternity, just the last few days?)
- eventually use this data to create an auto-refreshing "this is the current weather in my garden" site with some nice graphs, weather.christianspecht.de or something like that.

The application so far

The code is here on GitHub: https://github.com/christianspecht/weather-station

It’s .NET 8 (the current LTS version). At the time of writing this post, it consists of nothing but a simple “Hello World” example (similar to the one in the 2013 blog posts), which shows the current temperature on the display.

Next (before I write the next post), I will add more functionality to the application, similar to the two tutorials in the Tinkerforge docs: read the values of all sensors and make use of the buttons to show different things on the display. And after that, I will start with the cloud stuff.

Migrating my blog from Jekyll to Hugo

Christian Specht — Thu, 30 May 2024 22:00:00 +0000

A few weeks ago, I migrated my blog for the third time…after WordPress, Blogofile and Jekyll it’s now powered by Hugo.

There’s nothing wrong with Jekyll. I just became more proficient with Hugo in the meantime (I hope I’m behind the steepest part of Hugo’s learning curve now…) and besides its insane build speed, I like the simplicity of installing/updating Hugo (compared to Ruby/Jekyll) on Windows machines.

And not to forget all the things I learned about processing images with Hugo.

My blog has a few old posts which directly load multiple larger images (e.g. this one). The images are not that large…but still, it would make more sense to auto-generate thumbnails and load just the thumbnails directly with the post. Not sure if it’s worth the effort, I don’t have that many posts with large images yet, but who knows…

Even though this is not the first Hugo site I built, I still learned a few new things about Hugo…and noticed some differences between Hugo and Jekyll:

Some things are easier in Hugo

…like getting the number of posts per year, for example.

Compare the Jekyll and Hugo versions of the “Archives” box in the sidebar. In Jekyll, I had to create logic for this myself, whereas Hugo supports it out of the box.

Hugo template code works ONLY in templates

In Jekyll, I could just create a page (the archive, for example) and write code like this directly in the page:

{% for post in site.posts %}
  <a href="{{ post.url }}">{{ post.title }}</a>
{% endfor %}

Hugo just ignores all template code inside regular pages. To build something like the archive page, you need the actual (empty) page, and a special template which contains all the logic and is used only by that single page.

Render hooks

Hugo has so-called render hooks, which allow things like auto-prefixing ALL image URLs with the complete site URL.

In my old Jekyll site, I created all hyperlinks (in all templates and in all Markdown pages) without domain, e.g. /page.html.

When I switched to Hugo, I decided to use “proper” hyperlinks with full URLs (https://christianspecht.de/page.html). Pasting {{ "/page.html" | absURL }} all over the main template (for the menus etc.) was one thing, but all images in all posts are also linked without domain (example).

But - as noted in the last paragraph - in Hugo it’s not even possible to put template code into regular pages, so I couldn’t just do something like {{ "/image.png" | absURL }}.

Hugo’s solution for this is called Render Hooks (there are more types of them, but I needed the ones for images).

You just need to create one file with what looks similar to a shortcode, and this causes Hugo to render all images with full URLs.

For example, the example Markdown image code linked above looks like this when rendered with this hook:

<img src="https://christianspecht.de/img/git-ssh-2.png" alt="Git Gui screen" />

No HTML comments

Hugo also removes all HTML comments when generating the output, including the ASCII art I have at the top of my blog’s HTML code.

Apparently the only way to get a  comment block into a Hugo site is converting it to Unicode, storing it in Hugo’s config file and loading it in the template with safeHTML.

Creating PHP pages is more complicated

My blog contains some PHP pages (for the project pages), and apparently it’s not that easy to generate PHP pages in Hugo.

Jekyll treats all files (as long as they contain YAML front-matter) equally and doesn’t really care about the file extension.

Hugo treats files with different extensions completely different. To create PHP pages with Hugo, I would have to create a copy of all layout files, just with .php instead of .html.

In the end, I decided to “cheat”, generate the project pages as .html files (so they use the same layout files as all other pages), and rename them to .php in the build script.

Implementing post series in Jekyll

Christian Specht — Fri, 29 Dec 2023 22:00:00 +0000

There were several occasions in the past where I wrote a “post series”, i.e. multiple posts about the same topic that belong together.

Until now, there was only one way to visually indicate this: including the name of the series in the post title, like this:

Cool Series, part 1: Intro
Cool Series, part 2: Implementation
etc.

Sometimes, I actually remembered to name the posts like this…sometimes I didn’t.

Recently, I noticed a few examples how others show post series on their blogs:

Ayende (example post)
- at the top of the post, there are separate “previous/next post” and “previous/next post in this series” links
- at the bottom of a post is a “More posts in this series” box
Steven van Deursen (example post)
- at the top of the post, there is a box with links to all posts in the series

“Previous/next post in this series” links sound tempting, but my blog is built with Jekyll, and it’s not that easy to determine the previous/next post that belongs to the same series.

But a box with links to all posts in the series, that’s doable. Here’s how I built it:

1. Defining the series in the YAML front-matter:

I just added a new variable called series to all relevant posts.

Example:

series: "Hugo/Lightbox image gallery"

This string is used to determine which posts belong together, so it must be 100% identical for all posts in the same series.

2. Adding the “series box” to the layout file:

I’m using a separate layout file (which inherits from the default layout) for posts.

The additional code for the “series box” goes into that file (so the box can only appear in posts), and here’s the complete code for it:

{% if page.series %}
<div>
  <h4>This post is part of a series: {{ page.series }}</h4>
  <ol>
    {% assign posts = site.posts | where: "series", page.series | sort: "date" %}
    {% for post in posts %}
    <li>{% if post.url == page.url %}{{ post.title }} <i>(this post)</i>{% else %}<a href="{{ post.url }}">{{ post.title }}</a>{% endif %}</li>
    {% endfor %}
  </ol>
</div>
{% endif %}

Nothing fancy here:

if page.series ⇒ only show the whole block if the current page actually has a series variable
assign posts = site.posts | where: "series", page.series | sort: "date" ⇒ load all posts that have the same series as the current post (including the current post)
loop them
- if it’s the current post, show the title only as text with “(this post)” behind it
- for all other posts, show them as links

And finally, here’s an example how the finished “series box” looks like.

Hugo/Lightbox image gallery: Renaming processed images

Christian Specht — Sat, 18 Mar 2023 22:00:00 +0000

After the last post, I thought I was finished with my Hugo/Lightbox image gallery, but there was one more thing I didn’t like: the filenames generated by Hugo’s image processing.

As a reminder, my current image gallery shortcode (including image captions from EXIF data and overlaying images with a logo) looks like this:

{{ $logo := resources.Get "img/overlay-logo.png" }}
{{ $image := (.Page.Resources.ByType "image") }}
{{ with $image }}
    {{ range . }}
    {{ $resized := .Fill "150x115 q20" }}
    {{ $x := sub .Width (add $logo.Width 10) }}
    {{ $y := sub .Height (add $logo.Height 10) }}
    {{ $withlogo := .Filter (images.Overlay $logo $x $y)}}
    <a href="{{ $withlogo.Permalink }}" data-lightbox="x" data-title="{{ with .Exif }}{{ .Tags.ImageDescription }}{{ end }}"><img src="{{ $resized.Permalink }}" /></a>
    {{ end }}
{{ end }}

Both the thumbnail ($resized) and the main image ($withlogo) are created with Hugo’s image processing features, which means that their final filenames are assigned at build time by Hugo, and they are ugly.

For example one of the images from my demo project: the original filename is notebook.jpg, but the the shortcode shown above produces this HTML:

<a href="http://localhost:1313/galleries/gallery1/img/notebook_hu3d03a01dcc18bc5be0e67db3d8d209a6_1586565_filter_6003090614434618376.jpg" data-lightbox="x" data-title="">
    <img src="http://localhost:1313/galleries/gallery1/img/notebook_hu3d03a01dcc18bc5be0e67db3d8d209a6_1586565_150x115_fill_q20_box_smart1.jpg" />
</a>

1. The easy part: renaming the main image

Starting with version 0.100.0, Hugo supports renaming files after processing them, via resources.Copy.

Unfortunately, there aren’t many examples out there yet. The only one in the docs (resources.Copy "images/mynewname.jpg") doesn’t help me, because it uses a fixed filename, and I need dynamic filenames because I’m dealing with lists of images.

But this one case is easy: I want to give the processed image its original filename back, the one that it had before processing.

As I’m looping all images anyway and processing each one in the loop, the original filename with its complete path, which resources.Copy needs, is still accessible via .RelPermalink.

(. is the current image)

So it’s just a matter of changing this line:

{{ $withlogo := .Filter (images.Overlay $logo $x $y)}}

…to this:

{{ $withlogo := .Filter (images.Overlay $logo $x $y) | resources.Copy .RelPermalink }}

As always, here’s the commit with the changes in the demo project.

The resulting HTML now looks like this:

<a href="http://localhost:1313/galleries/gallery1/img/notebook.jpg" data-lightbox="x" data-title="">
    <img src="http://localhost:1313/galleries/gallery1/img/notebook_hu3d03a01dcc18bc5be0e67db3d8d209a6_1586565_150x115_fill_q20_box_smart1.jpg" />
</a>

Already better than before, but the thumbnail still has an ugly filename.

2. The not-so-easy part: renaming the thumbnail

In past image galleries, I had used a suffix like _t for the thumbnails’ filenames, e.g. notebook.jpg would have a thumbnail named notebook_t.jpg.

This means that I would have to build the new file path from scratch:

path + original filename + "_t" + original file extension

So how do you do this in Hugo’s templating language?

You build strings via printf.

Example: {{ printf "%s%s" "foo" "bar" }} ⇒ foobar
There are multiple path.xyz methods which expect the complete path to a file and return parts of it (what I need: directory, filename, extension).

Example: {{ path.BaseName .RelPermalink }} ⇒ for notebook.jpg as the current image in a loop, like in the example above, this returns notebook

Syntax: Similar to a lot of other programming languages, nested function calls in Hugo’s template language must be in parentheses.

I.e. if I want to use the printf example from above, but instead of "foo" I want to pass the path.BaseName example (also from above), the correct syntax is:

{{ printf "%s%s" (path.BaseName .RelPermalink) "bar" }}

So this is the line of the shortcode that creates the thumbnails, which must be replaced:

{{ $resized := .Fill "150x115 q20" }}

And the new version which creates path + original filename + "_t" + original file extension filenames as intended:

{{ $thumbname := printf "%s/%s%s%s" (path.Dir .RelPermalink) (path.BaseName .RelPermalink) "_t" (path.Ext .RelPermalink) }}
{{ $resized := .Fill "150x115 q20" | resources.Copy $thumbname }}

The commit with this change is here.

3. Conclusion

Here is the final shortcode one last time - with EXIF image captions, logo overlays and pretty filenames:

{{ $logo := resources.Get "img/overlay-logo.png" }}
{{ $image := (.Page.Resources.ByType "image") }}
{{ with $image }}
    {{ range . }}
    {{ $thumbname := printf "%s/%s%s%s" (path.Dir .RelPermalink) (path.BaseName .RelPermalink) "_t" (path.Ext .RelPermalink) }}
    {{ $resized := .Fill "150x115 q20" | resources.Copy $thumbname }}
    {{ $x := sub .Width (add $logo.Width 10) }}
    {{ $y := sub .Height (add $logo.Height 10) }}
    {{ $withlogo := .Filter (images.Overlay $logo $x $y) | resources.Copy .RelPermalink }}
    <a href="{{ $withlogo.Permalink }}" data-lightbox="x" data-title="{{ with .Exif }}{{ .Tags.ImageDescription }}{{ end }}"><img src="{{ $resized.Permalink }}" /></a>
    {{ end }}
{{ end }}

For the same demo image as above (notebook.jpg), the HTML produced by this shortcode looks like this:

<a href="http://localhost:1313/galleries/gallery1/img/notebook.jpg" data-lightbox="x" data-title="">
    <img src="http://localhost:1313/galleries/gallery1/img/notebook_t.jpg" />
</a>

Hugo/Lightbox image gallery: Overlay images with logo

Christian Specht — Thu, 15 Sep 2022 20:00:00 +0000

Another nice addition to the Hugo/Lightbox2 image gallery: one of Hugo’s available image filters is able to overlay the gallery images with a logo.

Before we start, here’s the previous version of the gallery shortcode (including the EXIF caption from the last post) for reference:

/layouts/shortcodes/gallery.html:

{{ $image := (.Page.Resources.ByType "image") }}
{{ with $image }}
    {{ range . }}
    {{ $resized := .Fill "150x115 q20" }}
    <a href="{{ .Permalink }}" data-lightbox="x" data-title="{{ with .Exif }}{{ .Tags.ImageDescription }}{{ end }}"><img src="{{ $resized.Permalink }}" /></a>
    {{ end }}
{{ end }}

Like in the previous post, I will add the changes shown here as new commits in the hugo-gallery-example project in my code-examples repository.

Step 1: show logo

In order to be processed by Hugo, images must be either page resources or global resources.

The gallery images are page resources anyway, but the logo that will overlay all gallery images doesn’t belong to one single page. So it has to be a global resource.

The simplest way to make it a global resource is putting it into the assets folder.

Once it’s there, it can be applied to all gallery images by:

loading the logo with resources.Get
applying the filter, which overlays the gallery image with the logo
showing the changed image in the gallery instead of the original one

Here’s the commit with the changes.

The complete changed shortcode now looks like this:

{{ $logo := resources.Get "img/overlay-logo.png" }}
{{ $image := (.Page.Resources.ByType "image") }}
{{ with $image }}
    {{ range . }}
    {{ $resized := .Fill "150x115 q20" }}
    {{ $withlogo := .Filter (images.Overlay $logo 10 10)}}
    <a href="{{ $withlogo.Permalink }}" data-lightbox="x" data-title="{{ with .Exif }}{{ .Tags.ImageDescription }}{{ end }}"><img src="{{ $resized.Permalink }}" /></a>
    {{ end }}
{{ end }}

Step 2: move to bottom right

The coordinates are from the top left, so images.Overlay $logo 10 10 means 10 pixels from the top and 10 pixels from the left.

But I wanted to have the logo at the bottom right, so I needed to calculate the correct pixel values depending on the sizes of the gallery image and the logo.

The calculation itself is not difficult. All images in Hugo have properties for width and height, so I can access $logo.Width and $logo.Height (for the logo) and .Width and .Height (for the current gallery image).

For example, the correct x value for the logo’s top left corner (so that its bottom right corner is 10 pixels away from the right and from the bottom) is .Width - $logo.Width - 10,

Here’s some fancy ASCII art to visualize:

┌───────────────────────────────────────────────┐
│                                               │
│                                               │
│                                               │
│                                               │
│                          ┌─────────────┐      │
│                          │     LOGO    │      │
│                          └─────────────┘      │
│<---------- x -----------><-$logo.Width-><-10->│
│                                               │
└───────────────────────────────────────────────┘

<-------------------- .Width ------------------->

Same for the height: .Height - $logo.Height - 10

But I spent an insane amount of time getting the syntax right. The problem was the (for me) unusual syntax of Hugo’s template functions.

Getting used to the {{ name parameter1 parameter2 }} format and finding the correct math functions was one thing, but nesting two of these calls with all the correct brackets made my brain hurt.

But actually I just had to change this:

{{ $withlogo := .Filter (images.Overlay $logo 10 10)}}

…into this:

{{ $x := sub .Width (add $logo.Width 10) }}
{{ $y := sub .Height (add $logo.Height 10) }}
{{ $withlogo := .Filter (images.Overlay $logo $x $y)}}

Here’s the commit, and finally here’s the complete changed shortcode again:

{{ $logo := resources.Get "img/overlay-logo.png" }}
{{ $image := (.Page.Resources.ByType "image") }}
{{ with $image }}
    {{ range . }}
    {{ $resized := .Fill "150x115 q20" }}
    {{ $x := sub .Width (add $logo.Width 10) }}
    {{ $y := sub .Height (add $logo.Height 10) }}
    {{ $withlogo := .Filter (images.Overlay $logo $x $y)}}
    <a href="{{ $withlogo.Permalink }}" data-lightbox="x" data-title="{{ with .Exif }}{{ .Tags.ImageDescription }}{{ end }}"><img src="{{ $resized.Permalink }}" /></a>
    {{ end }}
{{ end }}

Once again, I am amazed by Hugo’s image manipulation features.

With these few lines of template code, I can now auto-create a thumbnail, pull the image title from the image’s EXIF data and overlay the image with a logo.

Hugo/Lightbox image gallery: Loading image captions from EXIF data

Christian Specht — Wed, 29 Jun 2022 21:00:00 +0000

When I figured out how to create an image gallery with Hugo and Lightbox2, there’s one thing I left out: image captions.

In Lightbox2 itself, this is straightforward. Here’s the generated HTML from the current version of my Hugo/Lightbox2 gallery, for one single image (image URLs shortened for brevity):

<a href="path/to/image.jpg" data-lightbox="x"><img src="path/to/thumbnail.jpg" /></a>

To add a caption, you just add a data-title attribute:

<a href="path/to/image.jpg" data-lightbox="x" data-title="My caption"><img src="path/to/thumbnail.jpg" /></a>

In my previous image gallery attempts with Jekyll, the list of images always came from a YAML list (either from a data file or directly from the respective page’s frontmatter).

In both cases, the list just had an additional property for the caption, which I used to fill the caption in the HTML attribute.

But in my Hugo gallery, the images are page resources, i.e. there is no actual “list of images” defined anywhere. The image files are stored in a subdirectory of the post, and Hugo reads them directly from the file system.

So it’s not obvious how/where to store captions per image - that’s why I omitted them in the first version of my Hugo gallery.

As a reminder/comparison, here’s the current version of the line from the shortcode that generates the <a> tag:

<a href="{{ .Permalink }}" data-lightbox="x"><img src="{{ $resized.Permalink }}" /></a>

(for the full shortcode, see the previous post)

Solution: Hugo and EXIF data

I wasn’t even aware that Hugo is able to access the EXIF data of images. I found it out by accident when reading this discussion in the Hugo forum.

First of all, how to edit an image’s EXIF data? There are probably many tools that can do this, but on Windows 10 (where I’m writing this text right now) you don’t even need a tool, Windows Explorer is enough:

Right-click on an image in the Explorer ⇒ Properties
Go to the Details tab and write the caption into the Title field:

To show this caption in the Hugo image gallery, the line in the shortcode must look like this:

<a href="{{ .Permalink }}" data-lightbox="x" data-title="{{ with .Exif }}{{ .Tags.ImageDescription }}{{ end }}"><img src="{{ $resized.Permalink }}" /></a>

New commit with the changes (shortcode and one image), in the existing example project in the code-examples repo.

That’s all - one changed line of code.

The more I learn about Hugo, the more I like it. As I said in the previous post: IMO the learning curve is insanely steep in the beginning, but apparently I have the hardest part behind me now. Now I’m at the stage where I’m amazed what Hugo can do with very little code.

Alternative solution: load captions from page’s front matter

This is the first solution I had, before finding out about Hugo’s EXIF feature. It works, but I like the EXIF solution better.

But for the sake of completeness, I wanted to show this solution as well.

Similar to my second Jekyll gallery, the list of captions is coming from the page’s front matter:

resources:
- src: "img/Pope-Edouard-de-Beaumont-1844.jpg"
  title: "Example caption 1"
- src: "img/esmeralda.jpg"
  title: "Example caption 2"

Note that the list must also contain the image’s name/path, so Hugo knows which text belongs to which image.

Then, the line in the shortcode must be changed like this to show the caption:

<a href="{{ .Permalink }}" data-lightbox="x" {{ if ne .Title .Name }}data-title="{{ .Title }}"{{ end }}><img src="{{ $resized.Permalink }}" /></a>

This works, but I didn’t like it because a) the data for the image gallery is now spread across two places (file system and front matter) and b) I have to duplicate the file name of each image in the front-matter again.

Creating an image gallery with Hugo and Lightbox2

Christian Specht — Mon, 10 Aug 2020 21:00:00 +0000

In the last few years, I built multiple static sites with Jekyll. Altogether, I’m happy with Jekyll, but there are a few things that could be better, with the build duration being on top of the list.

I wanted to evaluate another static site generator as an alternative, and I picked Hugo mainly because of its speed (I read multiple sources who said it’s really fast).

Compared to Jekyll, I find Hugo’s learning curve insanely steep, though…and it took me quite some time to grasp enough concepts until I was able to create a basic site.

At that point, I decided to create an image gallery first. That was one of my first experiments with Jekyll - after getting the basics to run, I built an image gallery with Lightbox2 in two versions (1, 2).

Here’s how to create the same result with Hugo:

Directory structure

Example code is here in the code-examples repo. While explaining the steps, I will link to the specific commit where I actually did the respective step.

This is Hugo’s basic structure for the content directory:

|       
+---content
| \---galleries
| | _index.md
| |   
| \---gallery1
| | index.md
| |   
| \---img
| esmeralda.jpg
| notebook.jpg
| Pope-Edouard-de-Beaumont-1844.jpg
| Victor_Hugo-Hunchback.jpg

Note the different _index.md and index.md:

_index.md is an index page, which just list its sub-items.
index.md (without underscore) is a “single page”, which means an actual content page.

Here’s the commit where I create the project with the .md files.

Plus, I needed some sample images for the gallery, so I just copied some from the example site provided with the Ananke theme.

Note the destination directory: the images are in a subdirectory of /content/galleries/gallery1, so they are part of /content/galleries/gallery1/index.md’s page bundle.

In Hugo terminology, the images are page resources, which means that we can list the current page’s images like this:

{{ with .Page.Resources.ByType "image" }}
    <ul>
    {{ range . }}
    <li>{{ .Permalink }}</li>
    {{ end }}
    </ul>
{{ end }}

But apparently, it’s not possible in Hugo to paste that code directly into the page for testing…so we need to create a shortcode first:

Shortcodes

For those who know Jekyll, Jekyll’s includes are called shortcodes in Hugo.

Shortcodes allow us to create things we need multiple times (like the code to show an image gallery :-) in a central HTML file - in this case, it’s the “list all images” code from the last paragraph:

/layouts/shortcodes/gallery.html:

{{ with .Page.Resources.ByType "image" }}
    <ul>
    {{ range . }}
    <li>{{ .Permalink }}</li>
    {{ end }}
    </ul>
{{ end }}

…and we can “include” it into other pages like this:

{{< gallery >}}

Here’s the commit with the changes - this will eventually become my “show image gallery for the current page” include.

This will output the following HTML:

<ul>
<li>http://localhost:1313/galleries/gallery1/img/Pope-Edouard-de-Beaumont-1844.jpg</li>
<li>http://localhost:1313/galleries/gallery1/img/Victor_Hugo-Hunchback.jpg</li>
<li>http://localhost:1313/galleries/gallery1/img/esmeralda.jpg</li>
<li>http://localhost:1313/galleries/gallery1/img/notebook.jpg</li>
</ul>

Auto-generating thumbnails

This is, besides the insane build speed, one of Hugo’s killer features for me: it’s able to batch-manipulate images at build time, which means that unlike my Jekyll image galleries, I just need to give Hugo the actual images, and it will auto-generate the thumbnails when building the site.

Here’s a simple example where I create a thumbnail for each image in the page bundle (150x115 pixels, JPEG quality 20%), and show it together with the original (larger) image:

/layouts/shortcodes/gallery.html:

{{ $image := (.Page.Resources.ByType "image") }}
{{ with $image }}
    <ul>
    {{ range . }}
    {{ $resized := .Fill "150x115 q20" }}
    <li>{{ .Permalink }}
        {{ $resized.Permalink }}</li>
    {{ end }}
    </ul>
{{ end }}

This will generate the following HTML:

<ul>
<li>http://localhost:1313/galleries/gallery1/img/Pope-Edouard-de-Beaumont-1844.jpg
    http://localhost:1313/galleries/gallery1/img/Pope-Edouard-de-Beaumont-1844_hu28e98c35df2fb9cf55bbe1469dad4e9d_67722_150x115_fill_q20_box_smart1.jpg</li>
<li>http://localhost:1313/galleries/gallery1/img/Victor_Hugo-Hunchback.jpg
    http://localhost:1313/galleries/gallery1/img/Victor_Hugo-Hunchback_hu0047bfedff79a029a47406b9671745f3_111947_150x115_fill_q20_box_smart1.jpg</li>
<li>http://localhost:1313/galleries/gallery1/img/esmeralda.jpg
    http://localhost:1313/galleries/gallery1/img/esmeralda_hueb36a26f61b343d8dba9f5eda0997ef5_54891_150x115_fill_q20_box_smart1.jpg</li>
<li>http://localhost:1313/galleries/gallery1/img/notebook.jpg
    http://localhost:1313/galleries/gallery1/img/notebook_hu3d03a01dcc18bc5be0e67db3d8d209a6_1586565_150x115_fill_q20_box_smart1.jpg</li>
</ul>

Note that in my real test project, I ignored the whole resources/_gen directory in source control. The Hugo docs tell me to commit the manipulated images, but I chose to ignore them because IMO they’re some kind of build artifact (like an executable, which I wouldn’t commit either).

For the local developer experience it doesn’t matter: on my local machine, Hugo only generates the images on the first build (because it doesn’t delete them before subsequent builds).

And the deploy will happen via CI, so I don’t care if the CI server builds a few seconds longer.

However, in the demo project linked here, I did commit the generated images.

Putting it all together

With all the “ingredients” prepared, we can now show the actual image gallery with Lightbox2:

/layouts/shortcodes/gallery.html:

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/lightbox2/2.11.1/css/lightbox.min.css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/lightbox2/2.11.1/js/lightbox.min.js"></script>

{{ $image := (.Page.Resources.ByType "image") }}
{{ with $image }}
    {{ range . }}
    {{ $resized := .Fill "150x115 q20" }}
    <a href="{{ .Permalink }}" data-lightbox="x"><img src="{{ $resized.Permalink }}" /></a>
    {{ end }}
{{ end }}

HTML output:

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/lightbox2/2.11.1/css/lightbox.min.css" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/lightbox2/2.11.1/js/lightbox.min.js"></script>

<a href="http://localhost:1313/galleries/gallery1/img/Pope-Edouard-de-Beaumont-1844.jpg" data-lightbox="x"><img src="http://localhost:1313/galleries/gallery1/img/Pope-Edouard-de-Beaumont-1844_hu28e98c35df2fb9cf55bbe1469dad4e9d_67722_150x115_fill_q20_box_smart1.jpg" /></a>

<a href="http://localhost:1313/galleries/gallery1/img/Victor_Hugo-Hunchback.jpg" data-lightbox="x"><img src="http://localhost:1313/galleries/gallery1/img/Victor_Hugo-Hunchback_hu0047bfedff79a029a47406b9671745f3_111947_150x115_fill_q20_box_smart1.jpg" /></a>

<a href="http://localhost:1313/galleries/gallery1/img/esmeralda.jpg" data-lightbox="x"><img src="http://localhost:1313/galleries/gallery1/img/esmeralda_hueb36a26f61b343d8dba9f5eda0997ef5_54891_150x115_fill_q20_box_smart1.jpg" /></a>

<a href="http://localhost:1313/galleries/gallery1/img/notebook.jpg" data-lightbox="x"><img src="http://localhost:1313/galleries/gallery1/img/notebook_hu3d03a01dcc18bc5be0e67db3d8d209a6_1586565_150x115_fill_q20_box_smart1.jpg" /></a>

This is it - this HTML will display the thumbnails, and clicking on them will open the actual images with Lightbox2.

Note: there is one additional commit with a CSS file in the demo repo - this is because of the Ananke theme.

Without that tweak, Ananke’s default CSS would stretch all images to the page width and show each one in a new row…which doesn’t make any sense for thumbnails.

Conclusion

So far, I like what I’ve seen from Hugo.

As I said before, I find the learning curve insanely steep compared to Jekyll (and the docs don’t explain the basic concepts enough, so my main problem was not knowing the exact terms for the things I was looking for).

But once I got past that stage, working with Hugo is very nice because it comes with a lot of stuff built-in, which I needed to do manually in Jekyll, like:

getting a list of images belonging to a specific page (although I think Jekyll has improved in the meantime - I wrote the “Jekyll image gallery” posts in 2014)
automatic index pages
and, of course, auto-generating thumbnails!

Creating custom OpenStreetMap tiles for my own tile server

Christian Specht — Thu, 29 Aug 2019 21:00:00 +0000

Once a year, I need to create a printable map with markers.

The map is for Sindorf trödelt, a website which I built for an annual garage sale in my hometown.

Users can register with their addresses, and I use Google Maps to show them on this map on the website.

The map is not active the whole year, so here’s a picture:

The organizers of the garage sale also need to print flyers and posters with this map, but using the exact same Google Map for this doesn’t make sense, since there’s a limit how many copies you are allowed to print with the free version of Google Maps.

Everybody involved in the garage sale is doing this in their spare time (me included, and I’m sponsoring hosting as well) so there’s simply no budget to pay for a Google Maps license which would allow to print more copies.

First solution

So I needed an alternative way to create a map for printing, and I managed to get something to work with OpenLayers and OpenStreetMap:

I based my code on some example code for a OpenLayers map with markers which I found online (I don’t have the link anymore) - it works well, here’s a simplified JSFiddle (right-click on the map and save as image)

But after the first flyers were printed, one problem came up: the font size for the street names was too small, the names weren’t readable anymore.

That was two years ago. Last year I didn’t have time to find a better solution, so they just printed bigger flyers :-)

This year, I finally had time to investigate this.

Running my own tile server

Sounds impressive, doesn’t it?

When you search for things like “how to increase font size in [random web mapping solution]”, you quickly find that:

most solutions use pre-made tiles by 3rd party providers, which are provided as images
to change the appearance of those tiles, you either need to change the provider or run your own tile server

Running my own tile server sounded scary, so at first I didn’t investigate this, but focused on creating the whole image with different tools.

In the end, I found Maperitive, which has its own scripting language, so it’s possible to write a text file with commands and execute it in Maperitive to create a map image from scratch.

While playing around with this, I found example code which saves the map as tiles (=image files) and uploads them via FTP.

That was my “aha” moment: a tile server is not a complex piece of software, but just a regular web server which serves pre-generated image files over HTTP.

So I just needed some try-and-error, until the tiles looked the way I wanted them to.

Here are the steps to do it with Maperitive (I used v2.4.3.0):

1. Creating my own ruleset

A ruleset is a file with Maperitive rendering rules. The look of the map can be changed in Maperitive (colors, details, text appearance), just by providing a different ruleset.

I needed something "simple" (without many details which clutter the view), so I looked at all the rulesets and picked the Google Maps ruleset. This is just a file in the Rules subfolder in the Maperitive application folder.

I want bigger fonts for the streets...so I copied that file, searched for font-size and increased some values for the street types that sounded right (some experimentation necessary). Here's a gist with the changes.

2. Getting OpenStreetMap data as XML

Quote from Maperitive's Ten Minutes Intro ⇒ Loading Map Data:

Using the browser, go to OSM map, choose a smallish area (let's say the size of a few city blocks) and then click on the Export tab. Select the OpenStreetMap XML Data radio button value, click on the Export button and you'll receive a file called map.osm from the server.

3. Writing a Maperitive script

This code goes into a text file with .mscript extension:

// this is the ruleset from step 1
use-ruleset location=sindorftiles.mrules

// this is the OpenStreetMap XML file from step 2
load-source sindorf.osm

// move and zoom the map, so that the whole town is visible on the screen
move-pos x=6.674573 y=50.904049 zoom=16

// now we could export the map to a file:
// export-bitmap file=map.png height=2000 width=1600

// ...but we want to generate tiles instead:
generate-tiles minzoom=16 maxzoom=16 subpixel=3 bounds=6.6554,50.8898,6.6924,50.9191

// upload via FTP:
ftp-upload host=tiles.sindorf-troedelt.de user=USERNAME pwd=PASSWORD remote-dir=/

A few notes about the generate-tiles command:

minzoom and maxzoom should match the zoom level from the move-pos command
generating tiles for one single zoom level is enough in this case - we just need this view of the map for printing, no more zooming necessary. The more zoom levels, the more tiles are generated!

For me, the FTP upload isn't really necessary because generating the tiles is a one-time thing. Uploading the tiles manually would be acceptable as well...but still, it's nice that Maperitive can do it.

4. Opening Maperitive and executing the script:

This is just a two-liner, calling Maperitive and passing the script from step 3:

rd /s /q Tiles
C:\Maperitive\Maperitive.exe "%~dp0\sindorftiles.mscript" -exitafter

I put this into a batch file, and copied all the files mentioned before to the same directory where the batch file is.

That’s it - running this will open Maperitive, which will generate the tiles, save them into the \Tiles subfolder and upload them to the web server.

Here is an example - on the left side an original OpenStreetMap tile, on the right side the same tile generated with Maperitive and my custom ruleset:

Loading my own tiles with OpenLayers

Back to the JSFiddle from the beginning:

At the bottom of the <script> block, there’s this:

var map = new ol.Map({
  target: 'map',
  layers: [
    new ol.layer.Tile({
      source: new ol.source.OSM()
    })
    ,vectorLayer1
  ],
  view: new ol.View({
    center: ol.proj.fromLonLat([6.674573,50.904049]),
    zoom: 16
  })
});

To change this in order to use my custom tiles, I just needed to use the XYZ source instead of the OpenStreetMap source which I’m using in the JSFiddle.

So it’s simply changing this line:

source: new ol.source.OSM()

…into this:

source: new ol.source.XYZ({url: 'https://tiles.sindorf-troedelt.de/{z}/{x}/{y}.png'})

/{z}/{x}/{y}.png matches the folder structure and file names generated by Maperitive’s generate-tiles command.