DEV Community: Gabriel Simmer

Tiny eink Dashboard

Gabriel Simmer — Thu, 18 Feb 2021 20:30:02 +0000

Lately I'm been pinning for new and exciting things to build. When I came across PiHut's case for the Raspberry Pi Zero and Waveshare Epaper Display, I quickly snapped it up, knowing I could put it to use - at the very least I would have my own eink display, which is a technology I am very keen on.

sidenote: I will be using eink and epaper interchangeably in this post. "E Ink" is a trademarked term but is typically used in a more generic form of "eink".

Having never done much, if anything, with the Raspberry Pi's hardware and "hat" capabilities, the first step was figuring out how the two interacted. Thankfully, Waveshare has a repository containing examples in both C and Python, the latter being the languages I opted for due to my familiarity. Setup is relatively straightforward, requiring one or two libraries fetched via apt-get followed by a git clone of the repository.

Hold on though! The Pi Zero I have on hand does not have WiFi! How does one connect it to the internet? It's relatively straightforward, truth be told, when using Windows or macOS (Linux is another story, but we'll get to that). The Pi Zero (I am unsure if this extends to the full fat Pi) has a "gadget mode", wherein it can appear as a network adapter and allow your computer to connect over this. From there you can share your actual internet connection with it. Adafruit has a nifty little guide here.

With the internet connection sorted and the repository cloned and tested, it was time to create a minimal viable product. This was more or less taking the example code and extracting the valuable bits, giving it a smaller footprint and easier to navigate layout. This simpler version has been tagged for your viewing pleasure.

One interesting note about this particular display is that the Python library draws on the display upside down. Why, I am not particularly sure, but it's a relatively easy thing to compensate for until I can figure out the root cause. This is why in the code the image is flipped at the end!

This ran fine, but a few days after I made the leap to Linux full time (again) and left my Windows install behind. I was under the impression that sharing the connection would be just as straightforward, but it turned out it was a bit more complex than that (it's entirely possible I was trying to rush through it and missed a critical piece) and I quickly gave up after many failed attempts to configure the necessary rules. Without this internet connection, the Pi was unable to sync the date and time, which was a small issue for a device showing the time. Since my desktop and Pi did, however, have a solid network connection, I figured I may as well leverage it to the best of my ability, and set out to rework the script in a more traditional webserver/client configuration, with my desktop generating the image. This has the added advantage of lessening the work the Pi itself has to do, hopefully lengthening the lifespan of it or the eink display.

There's nothing particular to note about the new codebase, beyond the minimal dependencies and routing system that I borrowed from a previous project, which looks for properly named files and functions in the lib/ directory before returning the 404. It was especially important to keep dependencies on the Pi-side minimal since the lack of internet connection would require me to manually fetch them and sftp them up. This lack of connection also constrained me to Python 2, since the dependencies I had previously installed were done under that version. To combat the largest issue regarding the time and date of the Pi, I ran a small snippet over SSH that synced the time with my desktop's with a small offset to account for lag, and hope to integrate this sync directly into the script. While this single snippet could have solved the lack of internet problem without needing to rewrite the application, I think it was a worthwhile time investment for future expansion.

update: since this post, I have set up an ntp server local to my desktop to sync the Pi with, so a lot of the problems regarding time being out of sync are solved for good! This sort of invalidates part of the need for the client <-> server, but oh well.

Overall, I'm very happy with the end product, especially with the new client/server structure. It will allow me to do some more complex things without worrying about bogging down the smaller CPU. The clock can lag behind by a few seconds, but that's not a deal breaker in my case. Porting the client to another Waveshare eink display should be as easy as switching out the display library and adjusting the parameters in the script, and I hope to obtain a larger display in the future to have more real estate for information.

What would you do with such a device connected to your computer? What information would you find useful? I'd love to gain some more inspiration for this!

CI/CD for Minecraft Resource Packs

Gabriel Simmer — Tue, 22 Sep 2020 16:51:18 +0000

I have recently been getting back into the world of modifying Minecraft Java Edition, and as a part of that I have been spending some time learning the ins and outs of the new resource pack system.

Historically, within Minecraft, it was possible to create "texture packs", which would change out textures of blocks, items, mobs, and sounds. This lead to the creation of a fairly sizeable community, and provided a fantastic outlet for many creatives playing Minecraft. However, this approach was rather simplistic, with some clear limitations, which were later solved with the current iteration of "resource packs". Resource packs extend beyond switching out textures, allowing for the modification of the physical model of items and blocks, changes to in-game text, and replacing the music completely. Both of these systems draw their resources from either .zip files or directories with a particular structure.

While resource packs are associated much more closely with the creative side of Minecraft, I couldn't help but examine the technical possibilities regarding them. Since a large amount of metadata is stored as JSON files, specifically when it comes to custom models and defining alternative models or textures, it's easy to leverage version control systems such as Git to track changes. This is where the continuous integration and continuous eployment side of things comes into play. While developing a resource pack for a server, I realized quickly that deployment of changes was going to be difficult (for some context, Minecraft servers can define a custom resource pack to use, which clients can opt to download upon connecting), and started to investigate what could be done to make this process easier. The solution was relatively easy, but makes a huge impact on productivity and shipping new models or textures.

Step one was finding a place to host the finalized zip file, so that client connecting to the server can fetch and apply it. To this end, I opted for a free F1-micro instance on Google Cloud. The only potentially limiting factor in this case is the 1GB of egress traffic from the server, but given the small size that the resource pack is currently (~11kb, so roughly ~1,100,000 downloads if my math is correct), I can safely ignore this for the time being. To serve the files themselves, I installed nginx and grabbed a certificate using certbot and LetsEncrypt.

With the basic hosting out of the way, we can focus on the process of delivering the zip file. For this project, I'm using sourcehut, and opted to use the available builds system for convenience (and to take a look at the competition!). There are three things to do to ensure the resource pack can be delivered:

Ensure the JSON resources are correctly formatted. Otherwise, models and textures will be broken. Unfortunately, checking the correctness of the data itself is another problem.
Package the resource pack as a zip file.
Upload this zip file to the server.

None of the steps above are necessarily "novel" when it comes to testing and delivering code, and was relatively simple to implement with the build manifest on sourcehut.

image: alpine/edge
packages:
  - zip
  - rsync
  - jq
secrets:
  # Secrets go here.
sources:
  - git@git.sr.ht:~username/repository
environment:
  deploy: username@deployment-server
tasks:
  - test: |
      cd repository
      # Ensure all JSON files are readable JSON.
      find . -type f -name "*.json" -exec cat {} + | jq . > /dev/null
  - build: |
      cd repository
      zip -r resource-pack.zip assets pack.mcmeta
  - deploy: |
      cd repository
      rsync -e "ssh -o StrictHostKeyChecking=no -i $HOME/.ssh/private-ssh-key" -avz resource-pack.zip $deploy:/public-directory

Fantastic! End of story, right? It very well could be, but we can take this a step further. Minecraft servers can also specify a SHA-1 hash so clients can verify the integrity of the resource pack before applying it. While in most cases this is not an issue, it is helpful to specify.

Rather than creating a small webserver to run continuously on an already constrained machine, I opted instead to leverage nginx and a simple static page, which is modified during the deployment process with the new SHA-1. This way, we save on resources, and there are fewer moving parts over all. The <code id="resource-pack.zip"></code> portion will be replaced when we deploy the resource pack!

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Resource Pack Repository</title>
    <link rel="stylesheet" href="https://unpkg.com/mvp.css">
</head>
<body>
    <header>
        <h1>Resource Pack Repository <small>server-ready</small></h1>
    </header>
    <main>
        <h3>Resource Pack</h3>
        <p>A basic resource pack!</p>
        <p>Download: <a href="/packages/resource-pack.zip"><code>/packages/resource-pack.zip</code></a></p>
        <p>sha1sum: <code id="resource-pack.zip"></code></p>
    </main>
</body>
</html>

This index.html file is then deployed to our nginx server's web directory, to be served up when a visitor arrives at the website.

We can then add the following commands to the deploy step in our build manifest.

# Fetch the old index file.
curl http://deployment-server > index-old.html
# Update the code block with the new SHA-1
sed -r "s/(<code id=\"resource-pack\.zip\">).*(<\/code>)/\1$(resource-pack.zip | awk '{print $1}')\2/g" index-old.html > index.html
# Send the index file back up to the web host
rsync -e "ssh -o StrictHostKeyChecking=no -i $HOME/.ssh/private-ssh-key" -avz index.html $deploy:/public-web-directory

And voila! We now have a static site that will always have the most recent SHA-1 hash available.

There are a few improvements that could be made here, such as minifying any JSON data before compressing, or testing the actual content to ensure it matches a schema, but given the small scope of this project, I'm happy with where it landed, and I can now focus on completing the rest of the resource pack. Feel free to ask any question in the discussion below!