DEV Community: C. Plug

How do you debug a Python script that crashes Python itself?

C. Plug — Tue, 02 May 2023 17:08:36 +0000

TL;DR

Use faulthandler.enable(); Python will print the stack trace to stderr before fully crashing.

Backstory

Today I tried to daemonize a Python script that calls request.get on macOS, and it was insta-dying for no apparent reason.
I tested without daemonization, but the problem only popped up when I daemonize it.

I was doing something like this:

from daemon import DaemonContext
from daemon.pidfile import PIDLockFile
import requests
import time


pid = PIDLockFile(path.join(home_dir, 'pidfile_watcher'))
fp  = open("out.log", "a+")
fpe = open("err.log", "a+")

with DaemonContext(
  pidfile=pid,
  stdout=fp,
  stderr=fpe
):
  while True:
    response = requests.get("https://example.com");
    print(response.text)
    time.sleep(60*10)

fp.close()
fpe.close()

Many researches, print()s and one final check on Console.app (the macOS standard logger) later, I found that it was segfaulting at requests.get().

Why did it take so long?

Because daemonized script don't properly log its death even when you pass a file pointer for stderr.

Also, crash report found in Console.app does not provide much information unless you write most of your program (note the word 'program' - this includes not only your script, but also Python packages and/or even Python itself)! So the only realistic bet may be to spam print(), which is already tedious because you need to properly tell DaemonContext to redirect stdout to a file (I didn't give any at first, which caused even more headaches).

The solution

It was faulthandler.enable() that I needed to call.

faulthandler — Dump the Python traceback — Python 3.13.0 documentation

This module contains functions to dump Python tracebacks explicitly, on a fault, after a timeout, or on a user signal. Call faulthandler.enable() to install fault handlers for the SIGSEGV, SIGFPE, ...

docs.python.org

This causes Python to print stack trace to stderr when signal is received (in macOS, segmentation fault causes a signal SIGSEGV; probably all *nix systems also does.) instead of straight-up dying on the spot. This method enabled me to pinpoint exactly which call was causing my script to crash.

What was the problem, by the way?

Turns out, this is a known, (probably) unavoidable issue when using request package in macOS.

To avoid this issue, you must drop proxy support as segfault occurs at process related to polling OS's proxy configuration, which is "not fork safe".

import os
os.environ['no_proxy'] = "*"

Beware of 'padding' in TextMeshPro

C. Plug — Mon, 06 Feb 2023 14:31:38 +0000

TextMeshPro renders text wrong?

When you want to display text in Unity (starting circa version 2021,) the text components default to TextMeshPro (TMP).
TextMeshPro makes you create an asset called "Font Asset" before you can display anything. It contains a Font Atlas, which is a collection of pre-rendered letters packed into a single image, whose mapping TMP keeps track of to display the text afterwards.

When making the atlas, you can set the sampling point size (SPS), which is the font size used to pre-render the letters. This also translates to the biggest font size at which TMP can safely display the letters (for the most part). For example, smaller SPS sometimes characters to be positioned incorrectly. Bigger SPS requires larger atlas size, but it helps you avoid weird issues... or does it?

Let's try an excessive case: we crank up the atlas size to maximum (8192 x 8192) and create the font atlas. With the auto sizing option for SPS, we should get a nicest result, right?

... ⁉️

Of course, I did not add any font-related effects or shader effects or something related to that. This is a TMP text displayed with the default setting with the Font Asset we just created.

Environment

Unity 2021.3.6f1
TextMeshPro 3.0.6

Let's see what happened

For this experiment, we use M+ Font (2p) which can be retrieved from Google Fonts (or directly from M+ Font distribution site)

M PLUS 2 - Google Fonts

M+ FONTS is a little nifty font family for everyday usage. Mplus 2 is a Sans Serif font with nine weights from Thin to Black, supporting 5,700+ Kanjis for Japan

fonts.google.com

We just created a Font Asset with the maximum possible atlas size. TMP's Font Asset Creator reports the atlas information, so let's take a look:

"Point Size" is the SPS that TMP has chosen for this atlas generation. It reports it's 1394. Quite excessive, but let's move on for now.

For comparison, let's also create a Font Asset with a sensibly-sized atlas:

The SPS this time is just 23.

Now what happens if we display text using those fonts?

The label using the big font asset has catastrophically failed to render the text correctly!

Why?

This issue has been raised in Unity Forum; the SPS becomes larger as the atlas size gets bigger... and it becomes too big for the character to be way too tightly packed! Each character needs to be spaced out for TMP to work correctly.
To space them out, you need to use, you guessed it, the Spacing property.

https://forum.unity.com/threads/text-mesh-pro-box-around-text-unwanted-effect-bug.665647/

(This link appears to redirect too many times here and I cannot seem to embed it. WAT?)

In the thread, the Sampling Point Size and Padding should satisfy the next formula:

\frac{\text{SPS}}{\text{Padding}} = 10

If you look at the log closely, this ratio is reported SP/PD Ratio. The value here should say 10% or somewhere around that value.

SPS defaults to Auto Sizing when you first introduce TMP to your project. This means that TMP will adjust the SPS according to the Font Atlas size. However, there is no way (and no apparent instructions) to set the Padding automatically (it defaults to 5 and can be as big as 64), so it may fly under your radar.

If you only need ASCII, the size of 512x512 ~ 1024x1024 should do. If you choose to set your SPS yourself, you can control it further and you don't even need to force the atlas to be a square. Just make sure that SPS/PD ratio is 10%, and find the atlas size that can pack the letters as tightly as possible.

(Setting SPS yourself. Notice that SPS/PD ratio is 10%.)

How the Unity's new Input System liberates the input detection from frame-rate

C. Plug — Sun, 22 Jan 2023 03:53:47 +0000

Unity's conventional input system is easy to use, but there is one issue with it - it is frame locked.

Not sure what that means? Let me show you: if you are new to Unity, chances are that you have read or even written this piece of code yourselves.

using UnityEngine;
public class Sample: MonoBehaviour {
  private void Update() {
    if (Input.GetKeyDown(KeyCode.Space)) {
        Debug.Log("Space pressed now");
    }
  }
}

This code has one major restriction - it can detect key presses only when the Update() is called.

Update() is called every frame, so as you may usually enable VSync and/or with everything going on with your actual game, you would be only checking if the key is pressed around 60 times per a second.

This is actually fine for most of the use cases - after all, it may be sufficient for you to check the key input every frame and there will be no disadvantages to the player.

But what if there was one?

There actually are several use cases.

You need to detect the mouse movement as a part of the player's action set, but any frame rate drop makes the movement jaggy, which you don't want.
The precise timing of the input is required to the point that you want to know when within the interval between the frames the key was pressed. (e.g., music games should be doing this or must find a workaround such that the judgment is bound to the frames.)

It's possible with the new input system!

Introduce `InputActionTrace`

(Disclaimer: I'm skipping how we migrate to the conventional Input System to the new one - look it up)

With the new Input System, we have a powerful class called InputActionTrace in our arsenal. You will be using this class as follows:

Create InputActionTrace
.SubscribeTo() the InputAction you want to check the input outside the confine of the frame rate
Directly foreach on the InputActionTrace
.Clear() the trace
When done, .UnsubscribeFromAll() and .Dispose() of the trace.

using UnityEngine;
using UnityEngine.InputSystem;
using UnityEngine.InputSystem.Utilities;

public class Sample: MonoBehaviour {
  [SerializeField] private InputActionAsset asset;
  private InputActionTrace _trace;
  private void Start() {
    _trace = new InputActionTrace();
    _trace.SubscribeTo(
      asset.FindActionMap("Default").FindAction("Action", true)
    );

    // Important - required to open up the potential of your input devices,
    // But keep in mind that NOT ALL DEVICES ARE COMPATIBLE.
    InputSystem.pollingFrequency = 1000;
  }

  private void Update() {
    foreach (var action in _trace) {
        var val = action.ReadValue<float>();
        // This is where you do stuff
    }
  }

  private void OnDestroy() {
    _trace.UnsubscribeFromAll();
    _trace.Dispose();
  }
}

For each frame, your _trace will contain everything occurred from the last frame to the current one; and it contains useful information!

`.ReadValue<T>()`

The value returned from the iterator can be used just like the regular InputAction object, for the most part. So you can ReadValue<T>() off of it like nothing has changed:

foreach (var action in _trace) {
  var val = action.ReadValue<float>();
}

...except that now you have every event happened from the last frame!

If you are drawing a line, then this will help you draw a smooth line even if your game runs at 5fps or something!

`.time` property

This is the great property to make use of - this is the timestamp of the action occurred! Combined with Time.realtimeSinceStartupAsDouble (as it shares the epoch with this property,) you can tell how many seconds before the frame the action occurred!

foreach (var action in _trace) {
  var diff = Time.realTimeSinceStartUpAsDouble - action.time;
  Debug.Log($"This action has occurred {diff} seconds before this frame.");
}

Sample

I have been making an input manager for my games, and with this example, I show the concept of using the .time property to retrieve the timestamp for the action precisely - take a look!
(I implement the input detection from the code side as much as possible, so there's stuff that is a bit off from many tutorials out there.)

Clpsplug / InputManager

Wrapper to handle Unity's new Input System's triggers from code. NOTE: It is not production ready yet.

Input Manager

WARNING!: This plugin is not stable yet!!

This plugin for Unity3D is the wrapper for using Unity's New Input Manager programatically (i.e., without using related components.)
With this plugin, you can:

Handle key presses as your custom enums
Handle key presses in a 'frame-unlocked' manner
"Hold frame count" with small effect from framerate fluctuation
Rebinding the assigned key (i.e., key config.)
- Output the custom bindings as serializable dictionary format
- "Duplicate keys" detection; if rebind causes one key bound to two actions, the plugin will try to swap the binds between them instead

How to use?

Please see the README inside the package.

Acknowledgements

The sample project includes a TextMeshPro-rendered version of M+ Font, which is avaiable under SIL Open Font License 1.1.

View on GitHub

In the FrameUnlockedSampleScene I prepared several labels that displays the data for the action - press the Space key to check the output, and also toggle VSync in the Game window.

Without VSync

Note the FPS - it's over 1000 so we won't be seeing values bigger than like 10ms difference for the most part.

With VSync

Now we're confined to 60fps - we should be seeing values around 16ms at most, which we do!

Notes

It seems there is a small GC allocation at the foreach section - it is small, but you should know that GC will occur periodically. The trace can still pick up the input actions missed during the dropped frames, so it shouldn't be too much of a problem, though.

Another point to make is that although this line:

    InputSystem.pollingFrequency = 1000;

causes the polling frequency to be 1000Hz, not all devices are compatible with this settings. Devices that are "polled" will be affected by this line. Although I don't have Windows PC to test, I heard that mice on Windows are most likely compatible with this. Keyboards may also be. If you set this to high number and you still don't get as much events as you hoped, then the device in question may not be getting polled (and instead, inputs are received from OS's API or something.)

It's still a good idea, though, because unexpected frame loss (= Update() failing to fire) can happen. In the example below, I'm playing my own music game, but I set Application.targetFrameRate = 5;. Notice that though this is a game that requires strict timing, I can keep getting "Perfect"s.

// Detect dark theme var iframe = document.getElementById('tweet-1608748818211930112-367'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1608748818211930112&theme=dark" }

Typeset LaTeX files with TeX Live on macOS and IntelliJ IDEA

C. Plug — Wed, 16 Nov 2022 01:00:00 +0000

So you want to typeset LaTeX locally

Nowadays, we have the luxury of 'cloud LaTeX services' like Overleaf, but sometimes you will run into timeout issues because you used a bit too many images.

It will never be a problem if you can just typeset LaTeX locally,
so here's the setup I use for my purposes.

Note that I need a CJK environment, or more specifically,
I need to be able to typeset Japanese characters.

Environment

macOS Monterey 12.6 (21G115)
TeX Live 2022
IntelliJ IDEA 2022.2.3 (Ultimate Edition (Student License should work as well, haven't tested Community))

Get MacTeX

On macOS, you want to use MacTeX to get TeX Live.
There are many ways you can get MacTeX,
but I used homebrew: brew install --cask mactex

Add TeX support to IntelliJ IDEA

IntelliJ IDEA – the Leading Java and Kotlin IDE

IntelliJ IDEA is undoubtedly the top-choice IDE for software developers. It makes Java and Kotlin development a more productive and enjoyable experience.

jetbrains.com

Yes, IntelliJ IDEA is originally for Java. But if you use Ultimate Edition, then you can use PHP, Ruby and Go, etc.
There are more languages (both programming and markup) it can support by installing plugins!

For LaTeX, you will use this:

TeXiFy IDEA - IntelliJ IDEs Plugin | Marketplace

plugins.jetbrains.com

This plugin enables not only syntax highlighting, but also TeX command completion
and Run Configuration template for LaTeX!
We will use this template to typeset our LaTeX on IDEA.

Setting up Run Configuration

This is where I stumbled a lot.

When typesetting Japanese, latexmk, which is a tool that chain-calls typesetting commands, is the tool to go.

We need to create a config file called latexmkrc.
This can exist either in your home dir or in the project root
with the filename .latexmkrc.

Most sources I searched suggests the following content as our latexmkrc.

#!/usr/bin/env perl
$pdf_mode         = 3;
$latex            = 'platex -halt-on-error';
$latex_silent     = 'platex -halt-on-error -interaction=batchmode';
$bibtex           = 'pbibtex';
$dvipdf           = 'dvipdfmx %O -o %D %S';
$makeindex        = 'mendex %O -o %D %S';

Now that we have the stuff, we look at the Run Configuration.

If the plugin is installed correctly, you should be able to find the LaTeX template.

And this is where stuff gets complicated:
Because of macOS's security measure, commands installed at specific paths requires user's explicit path specification.
And latexmk is installed at such a path.

For this reason, simply setting Compiler to Latexmk is not enough.

(...which is the very reason I suffered. I spent like an hour to learn this the hard way)

Usually, latexmk exists here:

/Library/TeX/texbin/latexmk

...or you can use which command and pbcopy the result.

which latexmk | pbcopy

if you didn't know, pbcopy is a macOS original command which copies output of command to clipboard!

Now you need to enable Select custom compiler executable path (required on Mac OS X) and paste the path!

[BY THE WAY]

PDF Viewer: With the plugin PDF Viewer, you should be able to select Built-in PDF Viewer.

Note, however, that this PDF viewer may fail to render Japanese characters
when the PDF file itself contain Japanese character.

To be absolutely sure, you may want to use macOS's Preview.app.

Scroll down a bit and you will find the following:

Item	Value
Main file to compile	This must be a full-path! (For some reason.)
Directory for output files	`{projectDir}/output` causes the output to go to `output` directory under the project root.
Output format	DEFAULT

...are the values to set.
I recommend that Directory for output files be set to {projectDir}/output
because LaTeX outputs all sort of files besides PDF (like 10 of them!!)

You probably don't want your project dir to be cluttered just because you've typeset your LaTeX file.

Give an appropriate name to Name:, save by clicking [OK],
And Run (green arrow at the top of the window) to compile the file!

By the way, you can find these settings within the red rectangle at Languages & Frameworks > TeXiFy
in the IDEA preferences.
Checking both of these will initiate compilation every time you save your LaTeX files.
Maybe worth trying if your PC can handle it.

Creating Custom Tags in Jekyll, with an actual example

C. Plug — Tue, 19 Jul 2022 03:58:45 +0000

Shields.io is a "service for concise, consistent, and legible badges in SVG and raster format, which can easily be included in GitHub readmes or any other web page."

Shields.io | Shields.io

Concise, consistent, and legible badges

shields.io

Their service takes query parameters to their API endpoint to generate a badge / shield. For example:

https://img.shields.io/static/v1?label=Find%20me%20on&message=GitHub&color=181717&style=flat&logo=github

becomes:

I was planning to use this in my personal blog, when I realized a problem; it is hard to maintain.
While it may be rare to change the content of those shields, it would be nice to have a maintainable way to put a shield on my blog.

Since I use Jekyll, I can define a custom 'Liquid tag.' Just like we can embed stuff from compatible services with liquid tags here in DEV.to, e.g., {% github repo %} and {% embed website %}, we can make a custom tag, say, {% shields_io payload %}, to display a shield.

How does Liquid tag on Jekyll work?

The Liquid tag on Jekyll takes the following format:

{% tag_name [tag_argument] %}

It has a tag name, and one optional argument.

See the problem here? There is at most one value accepted as an argument. Shields.io takes way more than that, and putting query parameter here does not solve anything.

There is still hope, though; let's take a look at the code we'll be writing.

module Jekyll
  class CustomTag < Liquid::Tag
    def initialize(tag_name, arg, parse_context)
      super
      # @type [String]
      @arg = arg
    end
    def render(_context)
      "The argument is #{@arg}" # return the render result here
    end
  end
end

The arg argument is the argument passed from the tag. It's a string. So we can do something with the input.

And looking back at the original problem, we are trying to pass a set of key-value pairs to an API endpoint.

So, I decided to pass a JSON payload here; it can be prettified for our purposes, and Ruby supports JSON deserialization out of the box.

How to turn it into the URL

Since Ruby can turn the input JSON into a hash, we can iterate on this hash and construct the query parameter.

So the idea is to deserialize the JSON and store it in a variable:

    def initialize(tag_name, input, parse_context)
      super
      # @type [Hash]
      @config = JSON.load(input.strip)
    end

And construct the query parameter. I also decided to include href and alt for other purposes into the JSON payload, but they are not relevant for the request. So I extract their values and remove it from the input hash before turning the rest into the query parameter.

def render(_context)
  href = @config[:href]
  alt = @config[:alt]
  @config.delete(:href)
  @config.delete(:alt)
  shield_tag = <<HTML
  <img src="https://img.shields.io/static/v1?#{hash_to_query}"
HTML
  if alt != nil
    shield_tag += " alt=\"#{alt}\" />"
  else
    shield_tag += " />"
  end
  if href != nil
    <<HTML
<a href="#{href}">
#{shield_tag}
</a>
HTML
  else
    shield_tag
  end
end

private
def hash_to_query
  @config.to_a.map { |k, v|
    "#{k}=#{v}"
  }.join '&'
end

Result

Here is my creation - copy the last two files and run a local server!

How to include secondary Jekyll website using submodule

C. Plug — Tue, 05 Jul 2022 08:55:52 +0000

Caveat

The technique explained in this article expects that you own an HTTP server that you can run ruby/bundler on and build your Jekyll site on.
This technique is not available for GitHub Pages.

The problem

After several month of Jekylling, I found it tedious to build my site locally and to upload it with scp, rsync, or something. I wanted my site to be built on the server, so that's what I did.
Then I realized I had another website which was hosting a blog at the "root directory" of your website while serving another Jekyll site at the different directly inside that.

Specifically, I had my dev blog and it also had product pages in the subdirectory and that page took advantage of _data directory and so on. It would be less confusing if you could source-control the two blogs separately by isolating them into different repositories.

https://example.com/  # A Jekyll site (A)
|- /2022/06/20  # Root level blog for site (A)
|- /another_product  # Separate Jekyll site (B)

I have tinkered around the Jekyll configuration and managed to pull it off, which I'll share with you here.

TL;DR

Create a source directory in your root blog project and move source there. Remember the dir name, you will use that later
Include your secondary site as a submodule at the project root
Edit your root site _config.yml to exclude that submodule from root site build procedure

  source: ["src"]  # The source directory you have created.

Build the submodule site

  cd submodule_site
  JEKYLL_ENV=production bundle exec jekyll build
  cd ..

In the output directory of your root Jekyll site, create a symlink (e.g., submodule_site) pointing to the output directory of secondary Jekyll site

  cd $PROJECT_ROOT/_site
  ln -s ../submodule_site/_site submodule_site  # This will be a part of your actual link!

Edit your root site _config.yml to include this line

  keep_files: ["submodule_site"]

Build both sites
Profit

First thing first

When you do something like this, you should consider moving your website source into a directory of your choice, as including another Jekyll website as a submodule may confuse Jekyll.
This is because, by default, most of the stuff in the project root is considered the source for the blog. Having another Jekyll project in such place can mangle everything.
Consult my another post on how to do this:

Use "source" Jekyll configuration to avoid deployment mistake

C. Plug ・ May 22 '22

#jekyll #ruby

Jekyll wants to play clean

If you put a symlink to the output directory of your submodule site into your source directory (by default it is your project root,) everything blows up at build time for some reason.

Now, the only thing we could possibly do is to create a symlink to the output dir of the submodule site into the output dir of the root site.

cd root_site/_site
ln -s ../submodule_site/_site submodule_site

But this does not work as everything in _site directory is wiped clean every build.

There is, however, a configuration key to stop this behavior.

The `keep_files` key

Configuration Options | Jekyll • Simple, blog-aware, static sites

The tables below list the available settings for Jekyll, and the various options (specified in the configuration file) and flags (specified on the command-line) that control them.

jekyllrb.com

You can use keep_files key to tell Jekyll to leave specific files/directories behind before performing a clean on build. This key works for symlinks, too!

keep_files: ["submodule_site"]

Time to build

By building both the submodule site and the root site, you will have created a working site in the _site directory. By symlinking the output dir of the root site to your webroot, you can serve the website!

Use "source" Jekyll configuration to avoid deployment mistake

C. Plug — Sun, 22 May 2022 12:40:14 +0000

By default, Jekyll includes into the website whatever exists at the project root minus some files and directories that has special meanings.

my_jekyll_site
|- GemFile  // safe by default
|- GemFile.lock  // safe by default
|- _config.yml  // safe by default
|- .gitignore  // safe by default
|- index.md  // included
|- _posts  // included (parsed and then included)
 |- ...

There is one problem with this setup. If you add a script file here, you are going to expose that file!

my_jekyll_site
|- Gemfile
|- ...
|- script.sh  // uh oh, you're exposing this file!

There can be multiple reasons for creating a non-Jekyll related scripts, examples being:

Automated build script for the server to publish your site

Yarn/Webpack config for your javascript needs

You could specify exclude key in your _config.yml, but this is merely a manual denylist - you may forget to add stuff until it is too late.

exclude: ["webpack.js"]

There is even better way:

The `source` key

This tells Jekyll to stop looking at the project root for the content to build; instead, it will use the directory you have specified.

source: "_src"

In this example, you put everything into _src directory and everything will work just as fine. No more need to update your exclude key!

One small notice is if you're generating stuff outside of Jekyll, e.g., transpiling your typescript, you need to point the input and output into the source directory.

I became tired of making manual deployment mistakes and this happened

C. Plug — Sat, 14 May 2022 15:58:24 +0000

This is the story of how I decided to automate the download process of Unity Cloud Build artifacts.

Clpsplug / ucb-webhook-receiver

Standalone Unity Cloud Build webhook receiver

Unity Cloud Build Webhook Handler

What is this?

This scripts boots a server that can accept webhook payloads from Unity Cloud Build and handle the artifact so that:

the latest artifacts are always at the specific paths
the previous artifacts are archived with the timestamp at when the archive occurs.

Why

This program was made to ensure that we don't have to download the artifacts from UCB by hand and the correct artifacts are in the correct directories, so that the same script can be used each time you update your game.

This may be achievable with Jenkins, but I just simply wanted to write this for personal educational purposes. So... whatevs.

This script is made with SteamCMD in mind, but if you are downloading the artifacts from Unity Cloud Build for uploading them to distributor using the similar tools, this script is for you.

Downloading them…

View on GitHub

A rather big set of binaries

Let me set the scene for you.

I have a game published on Steam - Twin Horizons. It's a music game, and it's made with Unity.

The repository I save the code employs the git-flow model, and whenever there is a commit on the base branch, it kicks builds on the Unity Cloud Build (thank goodness they finally supported IL2CPP for Windows.)

There are four targets, or types of the game executable, to be built. They are:

macOS version, Production Build
macOS version, Developer Build w/ development tools
Windows version, Production Build
Windows version, Developer Build w/ development tools

After they are built, I have to:

download them
place them on the specific location on my filesystem and
add an Addressable bundle to all of them.

This is what I need to do so that the tools provided by Steam can see the new build.

The problem

Several month ago, I received a message from one of my tester that his installation broke on the latest update. Ah s#!+, here we go again.

I gathered the logs. I tested on Unity. I even tested the built app. No luck. Heck, it was not even supposed to occur with the build he received.

...or so I thought.

Turns out, when I placed the executables, I made a mistake and placed the production version in the development version directory.

Alex E. Proimos, CC BY 2.0, via Wikimedia Commons

Thank goodness it was not the other way around.

...Come to think of it, I had a similar mess-ups in the past. I did avoid the catastrophe at that time because I at least had a temporary branch to test the game I have uploaded.

THAT'S IT! NO MORE HANDICRAFT!

(╯°□°)╯︵ ┻━┻

So I launched an investigation. I knew Unity Cloud Build support webhooks. So somehow, I had to receive them and automatically download them.

Where do I host the receiver, though?

I have a rented server, but it is for serving webpages. If I had done what I'm trying to do, eventually they will shut my account down for using too much of their processing capability. Remember, I have to run the uploader provided by the publisher on the machine that has the game.
I do have an VPS account, but they tend to be on the expensive side. I'm a broke-ass university student; Unless I want to set up a VPN or something to get a static IP, I want to avoid that.

At this point I thought, "it's kind of risky, but maybe I can try hosting it myself from my old PC and somehow expose it to the Internet."

That's when I found Loophole, a free tunneling service. If I can host an HTTP server that can receive the webhook from UCB, I can kick off automated binary downloads.

loophole / cli

Loophole. Instant hosting, right from your local machine. CLI.

Loophole CLI

https://loophole.cloud

Loophole CLI is one of the available loophole clients.

Installation

Head over to the releases page and get binary which is suitable for you.

Quick start

First create an account by executing

$ ./loophole account login

and following the instructions there, then execute

# Forward application running on local port 3000 to the world
$ ./loophole http 3000

# Forward local directory to the world
$ ./loophole path ./my-directory

# Forward local directory to the world using WebDAV
$ ./loophole webdav ./my-directory

Congrats, you can now share the presented link to the world.

For more information head over to docs.

Development

Testing

$ go test -v ./...

Running

# go run cli.go

Building

# go build -o loophole cli.go

View on GitHub

Now that the hosting method is decided, I had to learn how to listen to HTTP requests and run scripts in background simultaneously.

How do I make the server?

I wanted to make this as lightweight as possible. So using apache2 sounded very exaggerated to me; that is a full-featured web server.
Jenkins was not my option for a similar reason; if I was building the game on my machine, it would've made sense to use it. But the only goal here is to download a game and place it in the correct place.
I just wanted something lightweight that the only thing it does is to listen to a port, and worked on the language I'm comfortable with.

Since I wanted to make something in Python besides my research projects, I decided to code it in Python.

Few web searches later I found CherryPy, a lightweight web server in Python. It turns out it can safely spawn background tasks while it serves the webhook. Perfect.

What to download?

This is an example of the "Buiid Successful" payload - most of the information is redacted, but you get the idea:

The artifacts array has several other elements for you to download, including debug symbols. I omitted all but the one whose primary key is true. That is our game.

You can also generate secret key to check the payload - see the x-unitycloudbuild-signature header.

The result

Several filesystem f*ck-ups later, here's my invention:

Clpsplug / ucb-webhook-receiver

Standalone Unity Cloud Build webhook receiver

Unity Cloud Build Webhook Handler

What is this?

This scripts boots a server that can accept webhook payloads from Unity Cloud Build and handle the artifact so that:

the latest artifacts are always at the specific paths
the previous artifacts are archived with the timestamp at when the archive occurs.

Why

This may be achievable with Jenkins, but I just simply wanted to write this for personal educational purposes. So... whatevs.

This script is made with SteamCMD in mind, but if you are downloading the artifacts from Unity Cloud Build for uploading them to distributor using the similar tools, this script is for you.

Downloading them…

View on GitHub

This project receives webhooks from UCB, downloads the artifact from them, deploys them to the output folder while adding static accompaniments, should there be any.

Now I can be confident I will always deploy the correct binary! Also it was a good exercise to learn how filesystems are treated in Python.

Side note: this project can be deployed on Docker! If you cannot risk your main filesystem (as you should not,) I reckon this is the safest way to run this project.

Side side note: this code is not tested outside of my environment, so it may crash on you - if you experience any, issue reports are always welcome!

RVM did not work in the shell script, so I had to fix it

C. Plug — Tue, 10 May 2022 07:40:23 +0000

What happened

So I was building my portfolio site the other day and I decided to use Jekyll.

The server I'm renting can run Ruby via RVM, so with the power of symbolic links, I can pull the source on the server, build the static site there (which can be done in a breeze,) and safely serve it to the Internet.

But I ran into a problem - the script simple as this did not work:

#/usr/bin/env bash

set -eu

rvm use 2.7.2

RVM is not a function, selecting rubies with 'rvm use ...' will not work.

You need to change your terminal emulator preferences to allow login shell.
Sometimes it is required to use `/bin/bash --login` as the command.
Please visit https://rvm.io/integration/gnome-terminal/ for an example.

Interesting, so RVM I use at the terminal prompt is not /usr/local/rvm/bin/rvm but a shell function that gets injected as a part of the profile.

# Run type command directly from the shell prompt
$ type rvm
rvm is a function
(function gets printed out)

# shell script that contains `type rvm`
$ ./test.sh
rvm is /usr/local/rvm/bin/rvm

(NGL, I was fairly confused when they said "RVM is not a function..." Like, is RVM complaining that I tried to use RVM as a function when it cannot possibly be a function? Nope! What it means is that there is a shell function version of RVM and rvm must be a function to select Ruby interpreters.)

You need to source the profile yourselves

It turns out you must source the file provided by RVM. There can be two places that RVM may exist:

source "$HOME/.rvm/scripts/rvm" # $HOME installation
source "/usr/local/rvm/scripts/rvm" # System installation

But that did not quite do the trick

Apparently, RVM relies on unbound variables, so running the script with set -eu crashes it:

#!/usr/bin/env bash

set -eu

source "/usr/local/rvm/scripts/rvm"

rvm use 2.7.2

/usr/local/rvm/scripts/functions/support: Line 182: _system_name: Unbound variable

So we need to temporarily allow such variables while we run all the RVM commands.

Example working script

#!/usr/bin/env bash

set +u # WTF: Avoid RVM crashing due to unbound variables

source "/usr/local/rvm/scripts/rvm"
rvm use 2.7.2

set -eu # Restore strict error checking

bundle install
JEKYLL_ENV=production bundle exec jekyll build

Random thought

I remember having similar issue trying to use pyenv in a Jenkins-triggered shell script... It could've been an issue of the similar nature.

Here is My Take on Error-resistant Way of Naming Boolean Variables

C. Plug — Thu, 31 Mar 2022 03:11:32 +0000

Confusing bool(ean) variables

Pretty much all of us has done something similar to this:

bool isAvailable = true; // Is something in stock?

if (isAvailable) {
    System.Console.WriteLine("Not Avaialble"); // Uh oh
} else {
    System.Console.WriteLine("Available");
}

// Prints... "Not Available"...???

...or, an accidental reversal of the if logics.

This one is very simple as the only mistake is the swapped labels. However, if the content inside of the if gets bigger or in the case of else being no-op, you may start to make this kind of mistakes. Heck, I still do it.

But I wanted to put a stop to it and looked back all the code for the project I was working on. There I found a potential way that I can prevent some of the errors - defining a more robust naming convention.

Of course, this is not the silver bullet - some of you may even think that what I'm about to write is more confusing than ever! So use this convention at your discretions.

Naming the boolean values

Variables with type boolean can only take two values: true and false. For this reason, most of the use-cases are to store some state that can be answered by YES/NO.

Such nature of boolean usually leads us to naming them like:

is....
has...
can...

This is my base convention. When I declare a boolean value (be it local var or property,) I always prepend it with a be or an auxiliary verb.

This part is not much of the problem, however. It is the next part of the name that I found a potential problem.

Words that explain states

The part in question is a word or a phrase that explain what state the variable is supposed to mean. Let me call that part "State Words." I reckon that there are three categories for state words. Two of them being: Positive Words or Negative Words. E.g.,

Positive words
- Available
- Succeeded
Negative Words
- Unavailable
- Failed

I have found that improper mixing of those category leads to a burst of human error in writing codes for me.

Mixing the Categories

When the values refer to different states and those values use state words from different categories, it won't be an immediate problem. It may be naturally done for the simplicity or the readability of the logic/code.

using System;

bool hasFetchFailed = false; // Negative
bool isAvailable = false; // Positive

if (hasFetchFailed) {
  Console.WriteLine("Communication failed?!?");
  return;
}

if (isAvailable) {
  Console.WriteLine("Available");
} else {
  Console.WriteLine("Not Available");
}

But they still may become problematic once you start mixing them together.

if (!hasFetchFailed && isAvailable) {
  // is this when the fetch has failed or not? It is hard to deduce it within a second 
  // and also, we'll have a bad time when we try to reverse this expression.
}

And all the hell breaks loose when those variables points to the same state, but the state words used is from different categories. This situation is much more annoying as it occurs across the code within the project, like in different classes or methods. When the state is very important in the project, referring to it using both positive state words and negative ones is just asking for problem.

class A {
  bool hasSucceded; // This bool uses positive state word
}

class B {
  bool hasFailed; // but this one's negative!
}

As I found those cases in my code base, I came to a conclusion that I probably should decide which category to use for common states, and for others, I should decide once and stick with my decision all the way. Below is an example, but you could also prohibit one category altogether (although this is a bit too extreme.)

Ex)

State	Category	State Word(s)	Reason
Communication Result	Negative	failed	enables early return without negating the value
Stock availability	Positive	available	Less prone to make mistakes when writing code for Views

What about the last category?

At the start, I mentioned that there are three patterns for state words. The last one is, of course, words that does not fall within either of them.

Since this category is pretty much the very reason I wrote this article, allow me to explain my specific situation.

I created a rhythm game called Twin Horizons, which is played with two sets of four lanes that exist on the top and the bottom. This game provides "Full Lanes" mode where you use all the lane and "Half Lanes" mode where you use only the bottom four. Those modes are called lane styles.

And this game contains a logic where it is important to know which lane style the player has played. It is when I was looking back such part of my code base that I found something like this:

// This value exists as the parameter from the other scene
bool isHalf = ???; // by which I mean "Is the player playing in Half Lanes mode?"

// ...which is used later down the logic.
int rewardCoin;
if (isHalf) {
  rewardCoin = 700;
} else {
  rewardCoin = 800;
}

This "state word" is not even positive or negative!

And then I remembered that this very state word was causing all kinds of trouble when I was working on this part.

Let me put it in this way: What does isHalf == false even mean???

Well, I just explained what the game was, so we may have some idea, but can anybody, like literally anybody that knows nothing about the game realize that isHalf == false means that the player is playing in Full Lanes mode?

if (!isHalf) {
  // *We* know that this part is run when the "Full Lanes"
  // is played, but can *everybody& know?
}

What makes this matter worse is that this value is passed as a part of a struct that contains the information from the other scenes (for context, I use Unity for this game.) As the definition cannot be changed easily, we are locked down to using either isHalf or isFull.
If in some unfortunate situation where the procedure that run only on Half and the one on Full resides in the same scene, we're gonna be in a big trouble.

Some of you may have noticed: it may not be advisable to use boolean in this case. The sheer fact to express "is the player playing with Half Lanes style?" was wrong. The fact that these would be pretty much the only two lane styles in the game for foreseeable future did not help at all.

So to avoid the confusion altogether, I decided to ditch bool and use enum.

enum LaneStyle {
  Half,
  Full,
}

LaneStyle style;

This forces me to write which style I'm talking about which reduces the amount of confusion.

if (style == LaneStyle.Half) {
  // 100% for Half Lanes
}

Conclusion

We can name variables virtually however we want, and that flexibility makes us make bad choices. So how about purposefully restricting how we name them to avoid that?

Can anybody immediately know what the reverse of the state words mean?
- If yes... (e.g., Available --Negated--> Unavailable)
- Variables that points to the same state should be using the same state words - don't mix words from both categories
- If no...
- Consider not using boolean and try enum

Rules simple as this may help you write readable code and avoid human errors.

Extra talks (rants?)

! for negating bool being somewhat easy to miss

if (!isPlayable) {
  // I sometimes get confused b/c I did not notice 
  // the "!" at the start because the font is bad
  // And when "!" is in between parentheses, well......
}

When negating a boolean property of a class instance, the ! gets physically farther from the actual property name

if (!instance.isActive) {
  // When you only read `isActive` part...
  // well, say goodbye to your sanity
}

How to force Unity to target x64 or ARM64 for macOS builds

C. Plug — Thu, 24 Dec 2020 15:52:17 +0000

Unity 2020.2 adds support for Apple Silicon binaries (both native & Universal 2.) This means we need to decide which CPU type to use when building our game, either by hand or by code. How do you do this by code?

TL;DR

The API for switching the CPU type is not directly exposed as of 2.1f1, but you can use this workaround:

using UnityEditor;

// All but the last argument must be as they appear
EditorUserBuildSettings.SetPlatformSettings(
    "Standalone",
    "OSXUniversal",
    "Architecture",
    "x64" // Possible values: "x64" "ARM64" "x64ARM64"
);

Why

My project is built via Jenkins, which means the builds are kicked from the command line arguments.

Apparently, when you update to Unity 2020.2, the CPU type defaults to Universal 2 (x64 + ARM64.) However, it's often the case that you want not to use ARM64 because of the libraries you use. And it was the case for me.

How?

So I dug into my project files to find out that ${PROJECT_ROOT}/Library/EditorUserBuildSettings.asset remembers my build settings. This was bad news because this file was ignored by Git and I had three separate repository clones - one for development, the rest for building (to build dev/prod in parallel.) I needed to somehow replicate this file from the code!

I opened this asset file thinking this was a YAML file to find it being a binary file, but a large portion of it could be read as ASCII. I found this in the last of the file:

Standalone....iPhone....OSXUniversal....Architecture....x64....CreateXcodeProject....false

(this portion contained a lot of non-printing chars, which was replaced with ..... The number of periods is not to scale.)

Unfortunately, Unity does not expose these variables as an API, but there is a method you can tap into these raw values. The linked documentation is for one of the overloads, but you also have this one:

EditorUserBuildSettings.SetPlatformSettings(
    buildTargetGroupStr, builTargetStr, settingName, settingValue
);

As for buildTargetGroupStr etc., you need to set it as it appears in the .asset file - making the method look like the first code block.

Put that code in one of your IPreprocessBuildWithReport implementation, and you should be good to go.

Do keep in mind though, if you have this code in your project, you will not be able to specify your CPU type from the Build Settings window (it'll be ignored!)

Corrections are welcome, I wrote this late at night :P

Seriously, periodically do brew cleanup.

C. Plug — Tue, 13 Nov 2018 02:28:54 +0000

EDIT:

Seems like that since v2.0.0, homebrew decided that it should perform auto cleanup every 30 days, so it is rare to run into issues like below nowadays :) I'm keeping this up for historical reason (and possibly for other softwares that don't auto-cleanup themselves.)

I just found out that my Homebrew was keeping all the old installer for my upgradable Casks 😱

I ran brew cleanup to find to my horror:

_( _・ω)_<(me XD)@localhost H30 11/12 12:13:17)
> brew cleanup                                                                [~]
Removing: /usr/local/Cellar/apr/1.6.3... (60 files, 1.3MB)
Removing: /usr/local/Cellar/arpack/3.6.2... (22 files, 1.4MB)
...
Removing: /Users/(me XD)/Library/Caches/Homebrew/Cask/unity--2018.2.1f1,1a9968d9f99c.pkg... (940.3MB)
Removing: /Users/(me XD)/Library/Caches/Homebrew/Cask/unity--2018.2.6f1,c591d9a97a0b.pkg... (935.7MB)
...
Removing: /Users/(me XD)/Library/Caches/Homebrew/Cask/unity--2018.2.13f1,83fbdcd35118.pkg... (944.6MB)
Removing: /Users/(me XD)/Library/Caches/Homebrew/Cask/unity--2017.3.1f1,fc1d3344e6ea.pkg... (721.0MB)
Removing: /Users/(me XD)/Library/Caches/Homebrew/Cask/unity--2018.2.12f1,0a46ddfcfad4.pkg... (935.6MB)
Removing: /Users/(me XD)/Library/Caches/Homebrew/Cask/unity--2018.1.0f2,d4d99f31acba.pkg... (943.8MB)
...
==> This operation has freed approximately 19.9GB of disk space.

19.9GB?!?

I only have 250GB of space in total; that's a big chunk of data. If you are using Homebrew Cask to install your favorite apps, be sure to do brew cleanup (or use flag --cleanup on upgrades!)

DEV Community: C. Plug

How do you debug a Python script that crashes Python itself?

TL;DR

Backstory

Why did it take so long?

The solution

faulthandler — Dump the Python traceback — Python 3.13.0 documentation

What was the problem, by the way?

Beware of 'padding' in TextMeshPro

TextMeshPro renders text wrong?

Environment

Let's see what happened

M PLUS 2 - Google Fonts

Why?

How the Unity's new Input System liberates the input detection from frame-rate

Introduce InputActionTrace

.ReadValue<T>()

.time property

Sample

Clpsplug / InputManager

Wrapper to handle Unity's new Input System's triggers from code. NOTE: It is not production ready yet.

Input Manager

How to use?

Acknowledgements

Without VSync

With VSync

Notes

Typeset LaTeX files with TeX Live on macOS and IntelliJ IDEA

So you want to typeset LaTeX locally

Environment

Get MacTeX

Add TeX support to IntelliJ IDEA

IntelliJ IDEA – the Leading Java and Kotlin IDE

TeXiFy IDEA - IntelliJ IDEs Plugin | Marketplace

Setting up Run Configuration

Creating Custom Tags in Jekyll, with an actual example

Shields.io | Shields.io

How does Liquid tag on Jekyll work?

How to turn it into the URL

Result

How to include secondary Jekyll website using submodule

Caveat

The problem

TL;DR

First thing first

Use "source" Jekyll configuration to avoid deployment mistake

C. Plug ・ May 22 '22

Jekyll wants to play clean

The keep_files key

Configuration Options | Jekyll • Simple, blog-aware, static sites

Time to build

Use "source" Jekyll configuration to avoid deployment mistake

The source key

I became tired of making manual deployment mistakes and this happened

Clpsplug / ucb-webhook-receiver

Standalone Unity Cloud Build webhook receiver

Unity Cloud Build Webhook Handler

What is this?

Why

A rather big set of binaries

The problem

THAT'S IT! NO MORE HANDICRAFT!

loophole / cli

Loophole. Instant hosting, right from your local machine. CLI.

Loophole CLI

Installation

Quick start

Development

Testing

Running

Building

How do I make the server?

What to download?

The result

Clpsplug / ucb-webhook-receiver

Standalone Unity Cloud Build webhook receiver

Unity Cloud Build Webhook Handler

What is this?

Why

RVM did not work in the shell script, so I had to fix it

Introduce `InputActionTrace`

`.ReadValue<T>()`

`.time` property

The `keep_files` key

The `source` key