DEV Community: Rob Hoelz

Avoiding spurious wakeups with my ThinkPad X1 Yoga

Rob Hoelz — Sat, 01 Jan 2022 18:42:04 +0000

Last year I bought myself a ThinkPad X1 Yoga to use as my personal laptop - I've been pretty happy with it so far, but there's one problem I've run into that I've noticed a lot more lately - the laptop will sometimes wake up while the lid is closed, go back to sleep, and then repeat that whole process every few minutes. This can't be good for the hardware, so I dug into this issue and wanted to share my findings with anyone who's having the same problem running Linux on this laptop!

tl;dr - disable Thunderbolt wake events by running echo RP13 > /proc/acpi/wakeup

A bit of searching around the internet pointed me at /proc/acpi/wakeup, which lists and controls various ACPI devices that can trigger a wake-up, which ACPI state they work with, and whether or not they're currently allowed to wake the device up. I took a quick look to see which devices were allowed to trigger a wake:

$ grep enabled /proc/acpi/wakeup
GLAN S4 *enabled pci:0000:00:1f.6
XHC S3 *enabled pci:0000:00:14.0
RP09 S4 *enabled pci:0000:00:1d.0
RP13 S4 *enabled pci:0000:00:1d.4
PXSX S4 *enabled pci:0000:05:00.0
SLPB S3 *enabled platform:PNP0C0E:00
LID S4 *enabled platform:PNP0C0D:00

I'm pretty sure I want LID to bring my laptop out of sleep, and I don't know what SLPB is. I suspected that the culprit causing the wake-ups was one of the various PCI devices here, so I disabled all of those by running echo $DEVICE > /proc/acpi/wakeup as root for each of them (except for PXSX - for some reason writing that to /proc/acpi/wakeup didn't flip its state). I closed my laptop, and it didn't wake until I opened it back up again, huzzah!

I wasn't 100% satisified with this, though - I wanted to know if this showed up again in the future, and I wanted to know which device was causing the wake-ups. I figured that looking for WiFi connection events in my router's logs would serve as a pretty decent proxy for a wake-up event, so I set up grok exporter to look for those in my logs and export those as a counter named pavo_wifi_connections_total ("pavo" being my laptop's hostname). Once I had that metric in Prometheus, I wrote a simple alert rule to ping me if the laptop connected more than six times in the last hour: increase(pavo_wifi_connections_total[1h]) > 6

At this point, I conducted a few experiments by disabling/reenabling devices selectively. After some trial and error, I discovered that I only need to disable the RP13 device - whatever that is - to prevent the false wake-ups! I was curious, so I combed through lshw's output and discovered that RP13 corresponds to the laptop's Thunderbolt bus. So then I just wrote a little systemd service to disable that device upon every boot and called it day!

I'm not sure why that was triggering the wake-ups - maybe there's some weird electrical interference going on, or maybe the touchscreen goes through that bus somehow and there was some gunk on the screen generating events or something. I'm not going to dig into that any more, but if you know the answer or have an idea I'd love to hear about it!

Random zsh tips

Rob Hoelz — Sat, 24 Oct 2020 15:47:22 +0000

Random zsh tips

My friend fREW and I both use zsh, and we've picked up a number of tricks over the years. We decided to collaborate on a post sharing some assorted tips that can really improve the zsh experience!

Noclobber

This is a small thing, but it's saved me from many a headache. I (Rob) will often be working on some data, and I'll squirrel away a long-running or expensive computation to a temporary file, like so:

$ long-running-or-expensive-command > /tmp/data

All's well and good, until I search through history too far, or until I mistype a search that yields the above command line, and I overzealously hit enter - whoops, there goes that data! This is especially frustrating if the data you just clobbered changes over time, and there was a case in the old data that isn't present in the new! Or maybe you saved the data, got on a plane, and wanted to work with it.

Well, if you set noclobber in your zshrc, zsh will prevent you from making this mistake - it will refuse to run commands that would destroy a file via shell redirection (something like curl -Lo my-data.txt http://example.com/data.txt will still clobber my-data.txt). You can override this via the >| operator - personally, I prefer to explicitly rm the destination file and then re-run the command.

On a side note - wouldn't it be cool if the shell were smart enough to only clobber the destination file in the case of success, or if the shell somehow knew that long-running-or-expensive-command could result in different results? Or maybe it would be neat to have a facility to keep a history of temporary files as you work with them!

Vi Mode

Like noclobber, another little tweak that changed how I use zsh: bindkey -v to enable Vi mode! I was skeptical about this until I tried it, but if you're a Vim user, I recommend you at least try it out and see how you like it - I feel like it's made me so much more efficient at editing command lines over the years!

Widgets & Keybindings

zsh provides a feature called "widgets", which are really just functions that have a special relationship with the line editor. They're used to implement all of the various keybindings - for example, the default keymap assigns (most) keys to the self-insert widget, which inserts the character corresponding to the key you just typed into the line editor.

You can view the current keybindings via bindkey -L (if you're using vi mode, you might want to check out bindkey -L -M vicmd as well to see what widgets are bound in command mode) - this can reveal some interesting key combinations you didn't know about before! There are plenty of widgets that aren't bound to anything by default - you can find a bunch of interesting widgets in man zshzle and man zshcontrib.

It's also pretty easy to write your own - for example, I will sometimes accidentally type !4 instead of !$, which expands to the fourth entry in my history. This is never what I want, so I wrote a little widget to detect this situation and fix it to expand !$ instead.

Tips by fREW

Here are some random, but cool, features that I (fREW) really enjoy:

`push-line`

I bound the q key in the vicmd mode to push-line like this:

bindkey -M vicmd "q" push-line

What this does is set aside the current commandline, give you a fresh commandline to modify, and then pop the current line back. I typically use it in situations like this:

$ vi some/dir/some/file<ESC>q
$ mkdir -p some/dir/some/
$ vi some/dir/some/file

`AUTO_PUSHD`

In both zsh and bash you can use pushd some/dir to change directory intosome/dir, and then if you use popd you'll change directory back to where you came from. If you set the AUTO_PUSHD option, cd will automaticallypushd for you. I'll then use popd (or my alias for it, the symettricalmk) to get back to where I started.

global aliases

In zsh you can have aliases that are allowed anywhere in a commandline. I use G and L constantly below, and always forget that I've defined V even though I do that action manually a lot:

alias -g G="| grep"
alias -g L="| $PAGER"
alias -g V="| vim -"

Example:

$ long-out-is-long G needle # finds lines matching needle
$ curl https://blog.afoolishmanifest.com/ L # runs with pager

You can also pass arguments! This works fine:

$ ls G -P '^f'

History Events

This tip has been shared a lot over the years, but honestly it's one of my favorites, and I feel like it saves me so much time and typing! Let's say you're on the command line, and you issue the following commands:

$ mkdir a-really-long-directory-name
$ cd a-really-long-directory-name

It's no fun having to type that long directory name twice, is it? Fortunately, you can use history expansion (see "History Expansion" in man zshexpn for more details) to save yourself some time here:

You can use !$ to refer to the last argument of the previous command - so that cd above could be written as cd !$. This is probably my most-used expansion, especially with cd - I have even tweaked my cd so that if my previous command was a git clone, cd !$ will do the right thing and change directory into the repository I just cloned!
You can use !^ to refer to the first argument of the previous command - I sometimes will do something like mv file.txt dir/ and then vim !$/!^ to edit the file I just moved!
!! expands to the entire previous command line - you'll often come across this in the idiom sudo !! to re-invoke a command with sudo permissions, and I will sometimes do time !! to time a command I just ran. Another way I use this is diffing output of commands - I'll do something like some-command > /tmp/good-results, run some-other-command and eyeball the results, and then do combine <(!!) not /tmp/good-results to compare the two!
!* expands to the entire previous command line's arguments - so the same as !!, just without the command itself.
...:h, ...:t allows you to do some path manipulation on the expansion - :h removes a trailing path component, :t extracts the trailing path component, etc. It comes in handy if you do something like vim long/path/to/a/file.txt, and then you want to search the directory file.txt is in - it's just an ack SEARCH_STRING !$:h away!
!!:2 allows you to refer to the second argument of the previous command - you can use any number n in !!:n, but honestly I only ever find myself really using !!:2, because at a point it's easier to copy and paste or something rather than count how many tokens into your command line the argument is.

We hope you found something useful that you're itching to try out here! If you're interested in checking out our zsh configs, here they are:

Rob's
fREW's

Many thanks to fREW for contributing to this post!

Discovering your Prometheus via PromQL

Rob Hoelz — Sat, 26 Sep 2020 22:04:17 +0000

Discovering your Prometheus via PromQL

Knowing about all the metrics available on a Prometheus instance can sometimes feel impossible, so I'd like to share a trick I devised to making exploring those metrics easier!

One of Prometheus' strengths is its ability to ingest a good deal of data from a wide variety of data sources, due to the simplicity of its metrics format. However, if you work at an organization with multiple teams managing Prometheus exporters, you can end up feeling a little lost or overwhelmed with all of the metrics available to you, especially if you're on an infrastructure team that makes use of lower-level metrics for its alert rules. These kinds of metrics often come from third-party exporters your team didn't write, so you might not know all of the metrics available to you.

Prometheus does offer up lists of metrics in an autocomplete dialog when you start typing a query, and you can accomplish something similar if you're using Grafana. However, that workflow is more suited to "known unknowns" rather than "unknown unknowns" - it's more like searching a library for a book via the catalog search opposed to just browsing the stacks.

List all metrics for a given exporter

Let's say you want a list of all metrics exported by Prometheus when it scrapes itself - the trick inquestion looks like this:

count({job="prometheus"}) by ( __name__ )

Depending on your experience with Prometheus, you may or may not have seen that __name__ label before. __name__ is a special label in Prometheus - it's the name of the metric itself. In fact, foo{bar="baz"} is really just sugar for { __name__ ="foo",bar="baz"}!

Listing all "namespaces" for a given exporter

Running this against my own Prometheus, I get 202 results - here's a small sampling:

Metric	Value
prometheus_tsdb_isolation_low_watermark{}	1
prometheus_tsdb_lowest_timestamp_seconds{}	1
prometheus_rule_group_duration_seconds_sum{}	1
prometheus_sd_file_read_errors_total{}	1
prometheus_tsdb_compactions_total{}	1
go_memstats_gc_sys_bytes{}	1
prometheus_tsdb_head_min_time{}	1
prometheus_tsdb_head_truncations_failed_total{}	1
prometheus_tsdb_compaction_chunk_size_bytes_count{}	1
net_conntrack_dialer_conn_established_total{}	9
prometheus_tsdb_head_chunks_removed_total{}	1
prometheus_tsdb_tombstone_cleanup_seconds_count{}	1

There are two important takeaways here - first, the metrics aren't ordered by name, and in fact, the result ordering can change between executions. Because of this, I will often run these kinds of exploratory queries using promtool and pipe them to tools like jq or sort.

You can also see that the metrics follow a sort of namespacing scheme. We can use label_replace to extract the first level of "namespaces":

count(label_replace({job="prometheus"}, "metric_namespace", "$1", " __name__", "^([^_]+).*")) by(metric_namespace)

...which gives us a list that looks like this:

{metric_namespace="prometheus"}
{metric_namespace="promhttp"}
{metric_namespace="scrape"}
{metric_namespace="up"}
{metric_namespace="go"}
{metric_namespace="net"}
{metric_namespace="process"}

This listing can give us some hints on additional filters to use to zero in on what we're looking for - for example, if I'm interested in low-level metrics about how Prometheus communicates with things over the network, I can dig into the net "namespace" by adding __name__ =~"net_.*" to my vector selector! Since these "namespaces" are not documented within the metrics themselves, you often need to explore each one and see what you find to figure out which each one is about.

List all metrics for all exporters

You could tweak that first query slightly to list metrics for every job in your Prometheus:

count({ __name__!=""}) by ( __name__ )

One noteworthy thing about this query when compared to the first one is the __name__!="" bit - it might seem a bit strange if you're not used to it! It's just one possible shorthand for "give me all time series". The reason for that part is that PromQL requires that a vector selector has at least one label match that doesn'tmatch the empty string, so you can't use something like {} or { __name__ =~".*"}.

List all time series for a given exporter

Another change you could make to that initial query would be to list all time series - rather than metrics - that an exporter offers:

{job="prometheus"}

This allows you to see all label permutations for each metric. Be warned though: this can result in a lot of output, so I recommend adding some additional filters as you explore. This is another situation where I'd reach for promtool, which I mentioned above. It's not uncommon for me to use promtool to dump all of the time series to a file and then use shell tools (or even just explore the file in vim!) to get a better idea of what is available to me.

I hope you found this helpful, and that you now feel empowered to explore what kinds of metrics are available on Prometheus instances you use every day! Do you have any neat PromQL tricks you've discovered? If so, let me know!

Many thanks to Frew Schmidt, John Anderson, David Golden, Jonathan Yeong, and Liz Lam for reviewing this post!

What do you do with five minute chunks of time?

Rob Hoelz — Sat, 12 Sep 2020 02:01:53 +0000

What do you do with five minute chunks of time?

During my morning planning today, I was thinking about the highest priority work task I had on my plate. Without getting too into the details, I introduced a new component into our system, and it's been failing in... interesting ways. I can issue a command to invoke that failure, but it takes about five minutes to run its course. I figured that I might be alternating several times between provoking this failure and analyzing the new findings - so, I thought to myself: what's the best way to spend that time? Here are some thoughts I had on that:

Check Slack

This is what I used to do, since helping others out at work is part of my job, but it would often send my attention into a downward spiral. This is probably a bad idea - fortunately, I still get notifications for messages to me or my team's group if people need help, and I have a system in place to help me cut down on the other distractions Slack offers.

Work on another task for five minutes

This seems like the right answer, but I think this one is also a trap! Five minutes is a fair amount of time, but in my opinion, it's too short to justify the heavy cost of a context switch. Out of all likelihood, you're just going to end up doing the other task (or worse, both tasks) poorly. After all, your brain isn't a computer - if you think context switching is bad for your OS, it's worse for your mind!

Automate it into a longer cycle

In this case, I mean write some code to do the following:

Trigger the failure
Wait for it to pass
Record data
Maybe wait a minute or two
Repeat, say, for an hour

After that's started running, spend that hour working on the second-most important thing for the day, and then spend a deliberate chunk of time analyzing the results once those are in.

This has the advantage that you can reduce those expensive context switches while still collecting a lot of information for later analysis. Unfortunately, it does take time to write that code, and what if the first iteration yielded enough information for you to figure things out? As always, use your best judgement here!

Make that five minute chunk into a fifteen minute one

If you don't need to babysit the situation in question, trigger the issue and switch contexts to something else, but know (and be content with!) the fact that the thing will be finished for a while before you get back to it. The point here - surprise surprise - is to avoid the context switch!

Reduce the wait time

This doesn't apply to my problem today, but let's say you have a test suite that takes five minutes to run. It might be worth the effort to cut its run time in half if you can manage it!

Just sit with it

This is hard, because it can feel like a waste of time. But it keeps the current task loaded in your mind, and gives your mind a chance to let some of the facts settle and consider the bigger picture. You might think of an aspect of what you're working on that you hadn't considered, or you might be able to plan the kinds of analyses you'll run on the data you collect to get to your next cycle (or perhaps even the end of the task!) more quickly.

So, what'd I end up doing?

I went with the last one: I just sat with it. Or rather, I walked with it - I paced around my office and thought about the problem from different angles, and I wrote down some of my thoughts and hypotheses on my whiteboard. I feel like that allowed me to reach the solution more quickly, consider what error messages I'd been seeing that were red herrings, and take into account other things in play.

Like most things in software development - or even in life in general - that doesn't mean this is always the right solution. On a day where other developers are having more problems, I might be checking Slack more to chime in and help any way I can, or on a different type of task, I might invest some time in that automation.

Others?

Do you have strategies you employ in this situation that I didn't think of? If so, please let me know!

More on this from other people

If the idea of sitting with a problem and letting your mind do its thing resonates with you, you might enjoy the talk "Hammock Driven Development" by Rich Hickey. I've never used Clojure in anger, but I always like hearing what Rich has to say, which really makes me want to give it a shot!

I took a course on productivity this summer, and the creator Peter wrote a blog post on "toothbrush productivity", which this situation reminded me of. Having read books and taken other courses on productivity in the past, I found Peter's ideas refreshing - much like brushing your teeth - so I recommended checking it out!

This idea also got me thinking: what's the smallest time slice you should consider dividing your time into? I chose a fairly arbitrary number of fifteen minutes here, but I'm wondering if that's too small. The Pomodoro Technique hints that 25 minutes is better, and my fellow Milwaukeean developer Joe Clermont wrote a blog post today about keeping a work journal and thinking in 30-60 minute chunks.

Save time by focusing on the result - using Prometheus' textfile collector to avoid writing an exporter

Rob Hoelz — Mon, 07 Sep 2020 19:43:59 +0000

Save time by focusing on the result - using Prometheus' textfile collector to avoid writing an exporter

My friend Frew, as well as others, have said that our job as developers is not to write code, but rather to solve problems. I had a win in this regard relatively recently where I saved myself a good chunk of time by leveraging a Prometheus component for a purpose it may not have been designed for, but is remarkably well suited to.

A few months ago, I had a problem with my server - dhcpcd would get into a weird state roughly every 2 weeks and lose its IPv4 address, which is not great for a server that is hosting your personal site! I turned on debug logging for dhcpcd (of which there is a lot - I ended up putting the logs in their own journald namespace to enforce a separate disk quota!), and I noticed that dhcpcd started spitting out some strange errors roughly 24 hours before the address got lost.

I have monitoring in place to detect when my server is down, but it's no fun being under the gun investigating what's going on with dhcpcd while your site is down. I also use Prometheus to monitor various things, so to give myself some breathing room and time to investigate, I figured I could write a Prometheus exporter (Prometheus' term for a program that collects metrics for a Prometheus instance) for journald to export metrics about entry counts for (namespace, unit, priority) tuples, and then preemptively alert when (dhcpcd, dhcpd@eth0, ERROR)'s rate goes bananas.

Now, writing a Prometheus exporter isn't super hard, but it is work - and while a journald exporter might not be a bad tool to have in the future, I wasn't 100% confident I would need something so comprehensive. Was there a way I could get these logging metrics into my Prometheus without having to do the work of writing an exporter? Yes - through a very useful feature in Prometheus' node exporter: the textfile collector!

Node exporter is an exporter you can run that reports various metrics about the system it's running on: CPU usage, memory usage, process count, disk usage, etc. It does this through a series of collectors - you tell which collectors you're interested in when you start up node exporter. It will invoke the ones you asked for when it's scraped and serve up their metrics.

One of these collectors, as I mentioned, is the textfile collector. All it does is look for *.prom files in a directory of your choosing and serves up the metrics it parses out of them. This is ostensibly to cover things like custom system metrics that aren't available in node-exporter's considerable cadre of collectors, or things that would be difficult to add.

Fortunately for us, it also happens to work very well for trying out new metrics very quickly! You could write a cronjob to dump metrics out to a file - the format is very simple - and have some metrics to play with to decide whether or not you should invest more time on them in the future.

Instead of spending my time writing a journald exporter, I spent 10 minutes crafting a Perl program to dump this data into a file for node exporter to pick up. And I haven't felt the need for a journald exporter since - this just goes to show that if you focus on solving the problem rather than trying to throw code at it, you can save yourself a lot of time!

Of course, as with all things in software, there are tradeoffs - if you know for sure you will need such an exporter in the future, obviously it's cheaper to just write the exporter. So use your best judgement!

Surprising Logic in the Firefox Codebase

Rob Hoelz — Tue, 25 Aug 2020 21:01:54 +0000

Surprising Logic in the Firefox Codebase

Recently, I was curious about how Firefox's reader view implements its estimated reading time, so I spent a little time looking through the Firefox codebase. It turns out that it uses the readability package to extract the content of a page. Once it has that, it gets the length of that content and divides it by an estimated reading speed - and it was this part that surprised me:

https://searchfox.org/mozilla-central/rev/fa7f47027917a186fb2052dee104cd06c21dd76f/toolkit/components/reader/ReaderMode.jsm#598-629

 /**
   * Returns the reading speed of a selection of languages with likely variance.
   *
   * Reading speed estimated from a study done on reading speeds in various languages.
   * study can be found here: http://iovs.arvojournals.org/article.aspx?articleid=2166061
   *
   * @return object with characters per minute and variance. Defaults to English
   * if no suitable language is found in the collection.
   */
  _getReadingSpeedForLanguage(lang) {
    const readingSpeed = new Map([
      ["en", { cpm: 987, variance: 118 }],
      ["ar", { cpm: 612, variance: 88 }],
      ["de", { cpm: 920, variance: 86 }],
      ["es", { cpm: 1025, variance: 127 }],
      ["fi", { cpm: 1078, variance: 121 }],
      ["fr", { cpm: 998, variance: 126 }],
      ["he", { cpm: 833, variance: 130 }],
      ["it", { cpm: 950, variance: 140 }],
      ["jw", { cpm: 357, variance: 56 }],
      ["nl", { cpm: 978, variance: 143 }],
      ["pl", { cpm: 916, variance: 126 }],
      ["pt", { cpm: 913, variance: 145 }],
      ["ru", { cpm: 986, variance: 175 }],
      ["sk", { cpm: 885, variance: 145 }],
      ["sv", { cpm: 917, variance: 156 }],
      ["tr", { cpm: 1054, variance: 156 }],
      ["zh", { cpm: 255, variance: 29 }],
    ]);

    return readingSpeed.get(lang) || readingSpeed.get("en");
},

The fact that the reading speed is calculated in characters per minute - rather than words per minute - is what piques my interest, at least for English. I've seen many anecdotes about how English readers read in terms of whole words, rather than individual characters. There are many examples, such as this one, circulating the Internet of paragraphs with scrambled letters, but the only real research I was able to track down was Dyslexia: Cultural Diversity and Biological Unity (which I have yet to read 😅), which talks about how English and Italian dyslexics process words differently.

I find this property of English intriguing! It makes me wonder if we could develop a sort of "perceptual hash" for English misspellings, akin to image perceptual hashes, so that things like "careful" and "craeful" would hash to the same value, or even "definitely" and "definately" - although that latter example would probably be handled by a soundex function or something. I also wonder about a sort of "language aware distance function" that would be aware of common spelling mistakes such as "definately". Additionally, my notes from ElasticSearch: The Definitive Guide mention that "Damerau observed that 80% of human misspellings have an edit distance of 1", so maybe that could factor in? One thing that might throw a wrench into this is how much context factors how we read words.

If you know of research or existing algorithms/projects in this space, or if you have related ideas, please feel free to chime in - I'd love to hear about them!

Making Slack go slack-jawed: reducing the mental load around Slack notifications

Rob Hoelz — Tue, 11 Aug 2020 17:47:51 +0000

Making Slack go slack-jawed: reducing the mental load around Slack notifications

As I mentioned in a previous post, I use Slack at work, and I find it to be a constant source of distraction. I have taken to shutting it off for much of the day, which has the obvious disadvantage that if people need to reach me, they can't! In that post, I admit my weakness of needing to acknowledge unread badges and proposed a sort of "narrowing" features for Slack. I wanted to follow up on that and discuss some of the solutions I thought of, as well as the one I'm currently using.

Last month, I wrote a program I called "slack-jawed" - it took a list of Slack teams and set you to "do not disturb" for each of those teams, plus it muted a select set of channels for a period of time. That way, I could say slack-jawed 2h and go heads-down for two hours, but I would still get notifications for direct messages or for messages on channels that I should be paying attention to. Unfortunately, Slack's removal of personal access tokens and migration to only using app-based access tokens has sunk any hopes I had of using that - I can't install my slack-jawed app on every Slack team I'm on!

Another thing I tried is making use of Slack's sidebar sections for channels - basically, all of the non-essential work channels go into a "Fun" section. This works pretty well, but the systray icon still has an unread badge on it, and like I said, needing to acknowledge that is a weakness of mine.. Fortunately, I managed to find a solution that is Good Enough™ and took me next to no time to develop!

The recurring theme of my problem with Slack and distraction is that damn systray icon - I see it in the corner of my screen and feel compelled to acknowledge it. While working on focus and discipline is a worthwhile goal, and one that I'm striving towards, I figure why burn willpower if I don't have to - I use a powerful window manager that lets me do pretty much whatever I want! So I dug into Awesome's systray widget a bit, seeing if there's a way I could filter out the Slack systray icon using a little bit of Lua code!

Unfortunately, the freedesktop systray protocol is anything but simple - it actually uses XEmbed to embed application windows in the systray window! - and the logic for Awesome is written in C. If it were in Lua, I could at worst just copy the widget, make some modifications, and just run that - but since it's in C, I would have to fork the Awesome codebase and maintain my own package, and I don't know if that's worth the time. However, skimming over the systray protocol got me thinking - if systray icons are just regular windows, can't I just hide them?

To continue down this road, we need a few more tools in our toolbelt:

xwininfo - dumps out attributes (such as size, position, and child windows) about a window or a set of windows
xprop - dumps out X properties attached to a window
xdotool - conducts various X operations (such as moving, minimizing, and hiding) on windows without needing to write a program using Xlib or XCB

I did a little experiment - first I did xwininfo -root -tree to dump the entire window hierarchy on my desktop and used that to find the Slack systray icon's window ID, which I'll call $slack_icon_id. Then I used xdotool windowunmap $slack_icon_id and voila - it hid the icon! With a little more sleuthing by way of xprop, I was able to find a good set of criteria for identifying the Slack icon (it uses "slack" as its window class and it has the _XEMBED_INFO property), and with that, I have a little script that I run in a while loop that keep that pesky icon hidden:

$ for id in $(xdotool search --class slack); do
xprop -id $id | grep -q _XEMBED_INFO && xdotool windowunmap $id
done

This works, but honestly I'd rather have a little more diversity in the Slack client application space - especially an open source alternative. I've tried Scudcloud and Rambox in the past, but they both have their shortcomings. If anyone has any recommendations for a third-party Slack client that would help me manage distractions better, please let me know!

A little mistake I made in Go

Rob Hoelz — Sun, 26 Jul 2020 14:52:28 +0000

A little mistake I made in Go

I was working on a little Go project ~~last night~~ ~~a few nights ago~~ last weekend (boy it takes me way longer to finish blog posts these days!), and I wrote something like the following code, which has a bug in it:

/*
  type payload struct {
    filename string // the name of the file to search
    resultsChan chan []byte // the channel along which to return results
  }
*/

incomingWorkChan := make(chan payload)

go func() {
  for workUnit := range incomingWorkChan {
    defer close(workUnit.resultsChan) // make sure we always close the channel we're sending results back on
    err := doWork(workUnit.filename, workUnit.resultsChan)
    // error handling elided for brevity
  }
}()

I'm sure most seasoned Gophers can spot the problem here immediately, but this tripped me up for a bit, so I thought I'd share this so that others can benefit from my mistake. The problem here is that defer is function-scoped, not block-scoped, so Go basically builds up a long list of channels to close via a long list of deferred functions to run when the goroutine eventually exits. Not only is this is a waste of memory, but since I'm trying to close a channel here, and there's another goroutine elsewhere in the program that depends on that channel getting closed to proceed, my program deadlocks, which is never a good thing!

There are three solutions I can think of off the top of my head:

Just avoid `defer` entirely

go func() {
  for workUnit := range incomingWorkChan {
      err := doWork(workUnit.filename, workUnit.resultsChan)
      close(workUnit.resultsChan)
      // error handling elided for brevity
  }
}()

I'm not a huge fan of this, since if you add code to that loop in the future you need to be extra careful that you close(workUnit.resultsChan) in every possible case. It is, however, what I ended up doing, since I don't forsee that loop's body growing much - hopefully it doesn't bite me! 🤞

Call an anonymous function in the for loop and have the `defer close(...)` happen there:

go func() {
  for workUnit := range incomingWorkChan {
    func() {
      defer close(workUnit.resultsChan)
      err := doWork(workUnit.filename, workUnit.resultsChan)
      // error handling elided for brevity
    }()
  }
}()

Although this solution doesn't have the same future-proofing problems as the previous, I'm not a huge fan - I don't really like the "inline anonymous function call" in Go, except for go func() { ... }() and defer func() { ... }(). I'm not really sure why I'm ok with those two forms and not the other - maybe it's because I prefer block-based scoping and the the func() { ... }() feels like a hack to introduce something akin to block-based scoping here? Or maybe because the others are more established Go idioms in my mind?

Minor segue - one thing I find interesting about Go is the number of idioms present in the use of the language. Establishment of idioms is not specific to Go by any means - I think that's just a result of the language being used - but one of the selling points of Go is its simplicity, and that makes me think of Esperanto in a way. In particular, this quote I read when I was getting into constructed languages oh so long ago:

As for Esperanto, I don't know if Esperantists speak the language at home for their children to hear so that they learn it as a (second) native tongue. If they do, the kids will probably be producing changes very slowly over the years (if they do the same with their own children, and so on). This perhaps would horrify doctor Zamenhof and his followers, but it would be a sure sign that the language is indeed used for communication and is alive, a natural(ized) language among peers.

Call a named function with the channel, and have the `defer close(...)` happen within that function:

func someFunc(filename string, resultsChan chan []byte) error {
  defer close(resultsChan)
  return doWork(filename, resultsChan)
}

go func() {
  for workUnit := range incomingWorkChan {
    err := someFunc(workUnit.filename, workUnit.filename)
    // error handling elided for brevity
  }
}()

This could easily happen in doWork itself, but I kind of like decoupling the closing of the channel from the generation of the results - if other functions are needed to generate results, I don't need to modify someFunc later on, or wonder why my program is trying to write to a closed channel. This is probably the solution I like the most, since you're protected from forgetting a close(workUnit.resultsChan) but you don't have to deal with the anonymous function weirdness.

(Your idea here!)

What about you? Have you encountered this problem in the past, and if so, how did you solve it? Let me know!

Using udev to disable my infrared camera on Linux

Rob Hoelz — Fri, 01 May 2020 01:30:49 +0000

Using udev to disable my infrared camera on Linux

My work laptop has two cameras - one of the cameras is a regular, run-of-the-mill camera, and since I'm remote (well, due to COVID-19, I guess almost everyone is these days!), I use it for video meetings a lot. The other camera is an infrared camera, and to be honest, I'm not sure what it's for. All I know is that because it's /dev/video0, most applications treat the infrared camera as the default.

Normally Firefox remembers which camera I used last (I typically use a third, external USB camera), but lately I've been moving throughout the house throughout the work day, trying to change my surroundings, so Firefox is constantly confused as to which cameras are attached to my laptop. I'm sick of needing to manually select away from the IR camera all the time, so I decided to disable the IR camera for once and for all.

Looking around on the Internet for "disable Linux camera" provides no shortage of solutions - the most popular of which is "cover it up", which I've done with a piece of duct tape, but doesn't solve the camera selection problem. The next most popular solution is to the blacklist the uvcvideo kernel module, which will disable my IR camera, but along with all of my other cameras, so that's a no-go. Fortunately, in my travels, I stumbled upon bConfigurationValue, which if set to 0 will disable a camera. So now all I needed to do was find the camera's USB bus:

$ sudo lshw
<snip>
    description: Generic USB 
    bus info: usb@1:5

...and then navigate to its location in sysfs, and write a 0 to bConfigurationValue to disable the camera:

$ echo 0 | sudo tee /sys/bus/usb/devices/1-5/bConfigurationValue

And it works! Or doesn't, in this case! 🎉

Now, this disables the camera, but it's not permanent - it'll be reenabled the next time I reboot. I could write a systemd unit that runs on every boot to just write 0 to that sysfs file, but I wanted to do things properly and use udev, since that's what it's there for.

I took way too much time figuring out how to do this with udev; the short version is that I used udevadm --monitor to watch for changes, and then I did rmmod uvcvideo && modprobe uvcvideo to force the kernel to run the udev rules. I also used a bogus RUN{program}+="..." statement in my udev rules to spit out debugging information to a file in /tmp/ for situations in which I found udevadm --monitor lacking (although in retrospect I probably could've used udevadm test). I learned some interesting things from this exploration - I learned that there's an important difference between ATTR and ATTRS (the latter is what I wanted), and I learned that you need to match an attribute in your dev in order to interpolate $attrs{attribute} into a RUN statement. However, no matter what I did, I couldn't seem to get bConfigurationValue to get updated! I decided to walk away and pick this up again the next morning.

Well, I'm glad I did that, because something dawned on me - looking at my RUN{program} output, I noticed that devpath was always something underneath /sys/bus/usb/devices/1-5 - so that directory must represent some sort of controller higher up in the device hierarchy, rather than the camera itself. Looking at the output for udevadm info --attribute-walk --path=/sys/bus/usb/devices/1-5, I noticed that it's governed by the usb module, rather than uvcvideo, so that makes sense why my udev rules weren't getting triggered, since I was only reloading the video module. I was loathe to remove and load the usb module to try this out, but running udevadm test seemed to apply the rule just fine!

This ended up being the magic rule I needed to add to make my change permanent:

ACTION!="add|change", GOTO="camera_end"

ATTRS{idVendor}=="5986",ATTRS{idProduct}=="1141",ATTR{bConfigurationValue}="0"

LABEL="camera_end"

So that removes a major annoyance of mine with my cameras - the next step is to try to figure out if I can change the enumeration order so that if the external web camera is plugged in, it comes up first and is preferred. A cursory look at the Firefox code seems to indicate that it always starts from /dev/video0, and as far as I understand udev, I can't really shuffle the order of those entries around. Some alternatives that have come to mind are 1) disabling the internal camera if the external camera is plugged in (more udev, yay!), or figuring out where Firefox stores its preferred camera information and messing with that. I rather like the first approach, especially since I have a script that I use to take my picture for Slack and it would benefit from this as well.

Strategies, Observations, and Open Questions for Working From Home

Rob Hoelz — Sun, 22 Mar 2020 14:08:07 +0000

With the COVID-19 crisis forcing more people to sequester themselves at home and work remotely, there has been no shortage of articles describing how to maintain your productivity and sanity when working from home. I've been working from home for 3½ years, and I've learned a lot in the meantime, so I thought I'd share my own insights on the matter! There are a lot of commonalities among the various articles - things like "get dressed", "use a walk around the block as a commute" - I won't rehash these. Rather, I'd like to introduce you to some tips and tricks I haven't really seen discussed elsewhere - on top of these, there are some open questions I've been mulling over in my head for some time. If you read this post and have some thoughts on potential solutions to these questions, please chime in!

Observation: Multitasking is more alluring

This is something I just noticed, since I recently visited the home office in Santa Monica - I find that it's way more tempting to try to multitask, which in my opinion is a killer of both productivity and quality of work. Let me provide a more concrete example - let's say your teammate has a question for you about how something works. If you're in an office, they might drop by your desk, or they might catch you on your way to the kitchen, or they might Slack you and ask if you could drop by their desk. Whether it's at your desk, their desk, or chatting in the kitchen, they typically have your undivided attention - even if you see notifications coming in, social conventions keep your focus on the person you're helping.

In contrast, if I see a request for help in Slack, I can be chatting with that person, but I can also flip to another window to make a change to some code so that can run in the background for a bit and I can answer another question in another channel and I can check out the latest post in the gaming channel and oh I'll just run upstairs and start another pot of coffee brewing and...

You see where I'm going with this? It's way too easy to multitask here!

What do we do about this? I'm a compulsive "I need to acknowledge unread badges" kind of person, so I wish Slack had a "narrowing" feature (akin to Emacs' narrowing feature) so that I could more easily focus on one conversation. Alternatively, I'd like to get into more of a habit of starting up a video chat when helping someone - I think the nature of video chat would facilitate faster communication and enforce some of those social conventions I mentioned above.

Strategy: Show up five minutes early for video meetings

If you're going to a meeting in an office, sometimes you'll wrap up what you're doing five minutes before the meeting to grab a cup of coffee or something, and maybe you'll walk with a coworker with the same agenda and you'll chat along the way. And naturally some casual conversation will happen among the first few arrivals at the room while they wait for everyone else to show up. I've been thinking about employing a strategy of explicitly showing up on the video call a few minutes early to encourage this kind of informal chat and just do a little catching up with my teammates.

Strategy: Change your Slack profile picture every day

A few months ago, Mark Dominus started to change his Slack picture every day and mentioned it at our company's fall Tech Week. I thought this was a cool idea, so I started doing it myself, and I eventually automated most of it (the script is a abominable pile of Python that's currently not broken, but I'd be happy to clean it up and share it if enough people are interested!). This has served as a conversation starter among people at work, and humanizes people who are sometimes just a screen name in Slack. My coworker Jeremy even started a dedicated channel at work - #ilooklikethistoday - for more of a "river of profile changes" feel, which has been a real hit!

Strategy: Treat yourself if the home office is getting treated

Now, this tip doesn't really apply during the COVID-19 crisis, but I think it's good to keep in mind. Occasionally, the home office will have an all hands meeting, and we remotes tune in via Zoom. The locals often get something like donuts provided, and sadly those don't hold up over Ethernet. So, rather than feel jealous, I've taken to picking up donuts for my family and enjoying them during the all hands meeting. I also post a picture on Slack - it's another conversation starter!

Strategy: Adjust your start/end time

Now, this one only applies if your home office is in a different time zone, the time difference is small enough that such an adjustment makes sense, and if your employer is comfortable with more flexible hours (although chances are they'd prefer this!), but I personally found it harder to work my typical 9-5 in Central Time with Californa being two hours behind. I adjusted my schedule to something more like 10-6, which helped a ton! Well, until recently - my kiddo has been more demanding and more adventurous lately, so my wife often needs my help earlier than 6, so I'm working on adjusting my schedule to help my family more.

Strategy: During video meetings, sometimes you need to be a little rude

I hate being rude - I'm not saying I never am, but I try not to be and when I realize I may have been rude in retrospect, I agonize over it. That being said, it's often hard to get a word in edgewise when you're in a video meeting with some participants in a meeting room at an office, and some participants on camera. For one, you don't have your body language to help you - but the amount of time we allocate for pauses in conversation is surprisingly short, as is mentioned in an episode of one of my favorite podcasts, Lingthusiasm:

There’s actually so little pausing in some interactional – in many interactional contexts – that it is too short for the brain to actually, really have prepared for it without doing some clever anticipation about how that person’s speech goes, and how interaction goes. So, the average time between me saying something and someone replying is, like, 200 milliseconds, which is, like, the speed of a blink.

200 milliseconds is so short, and with the extra latency introduced by talking over the Internet, you're at an even further disadvantage. I was talking with my boss about this, and he said "sometimes you have to be a little rude", which I've tried to take to heart. So now I try to butt in a little bit, even if the first word out of my mouth during the interjection is just "Sorry!".

Question: Keeping work at work (time)

I have a bad habit - sometimes I'll work my regular hours, and then at the end of my day I'll leave my work laptop on so I can "check a few things" or "finish one more thing" or "make a small tweak to my program" after dinner and chores. ZipRecruiter certainly doesn't expect this of me, and the company is very clear on the importance of work/life balance. This is something I struggle with personally, and I would appreciate any tips about how to combat this! It might be as simple as "close your laptop at the end of your day - no exceptions"!

Question: Maintaining social connections and absorbing insights while remote

This one is hard - I don't know if there's going to be an answer for this. I really like going out to Santa Monica to visit - every time I get a chance to catch up with coworkers, and chat about cool random things. And missing out on that sucks - when Brian Fitzpatrick left Google, it came up again a few days later on Slack, and I wondered aloud what ever happened with his Perkeep project. Someone replied "yeah, we were talking about that on Friday", and it bummed me out to not have been part of that conversation. Obviously I don't expect my coworkers to hold every in-person conversation on Slack, nor do I expect them to invite me to a Hangout so I can dial in to informal office chat, but I'd really like to see if there's some sort of middle ground I can access as a remote.

Well, I hope you found that helpful - and if you have any feedback on my thoughts on the matter, or if you have ideas on how to resolve some of the questions I'm pondering, feel free to reach out!

Using SQLite's fsdir to effect a quick search index

Rob Hoelz — Tue, 25 Feb 2020 14:08:02 +0000

Originally published at https://hoelz.ro/blog/using-sqlites-fsdir-to-effect-a-quick-search-index

At work, we have a wiki we use for knowledge sharing. It's implemented on top of gitit, which works pretty well for our purposes, but sometimes I find the search functionality lacking. What I'd like is something that has more advanced full text search-like features - things like stemming (so that searching for "databases" finds documents that mention "database") and ranking by query relevance (so that documents that mention "database" many times are ranked higher than those that just mention "database" once).

Well, SQLite happens to have two pieces of functionality that can help in this regard: the FTS extension, which implements those features I just mentioned, and the fsdir table-valued function, which we can use to treat the filesystem like a table. Since gitit stores its pages in a Git repository, all we need to do is clone the repository and run a few SQL snippets to build our search index:

$ git clone work:wiki
$ sqlite3 wiki-search.db "create virtual table page_search using fts5 (path, page, tokenize='porter unicode61')"
$ sqlite3 wiki-search.db "insert into page_search select name, data from fsdir('wiki')"

Now we can search the wiki and benefit from those features: select path, snippet(page_search, 1, '', '', '...', 30) from page_search where page_search match 'my query' order by rank

Granted, it would be nice to do things like extract the front matter for each page, along with things like edit dates, but this works well enough.

Unsung Heroes of the Command Line

Rob Hoelz — Thu, 03 Jan 2019 20:16:09 +0000

Originally published at https://hoelz.ro/blog/unsung-heroes-of-the-command-line

Since the command line is my primary way of interacting with my computer, I often will take steps to optimize my usage of it. Most command line users know the basics: ls, cd, grep, etc - these are instrumental building blocks in using the command line. However, there are often more specialized programs out there that can really save you some time, and I'd like to share a few notable examples of these kinds of specialized programs that have made my life easier over the last year or two. If you don't know them, hopefully you'll find them useful; if you do know them, hopefully it will just indicate that we both have good taste in our tools. =)

mojo

Over the past few years, I find myself doing more and more work with data online. Calling out to various web services or fetching content from a URL has become an essential part of being a developer. When it comes down to simply fetching the data, I reach for curl. However, if I have to do some slicing and dicing, I look to my good friend mojo.

mojo is a command line utility shipped with the Mojolicious project - it's a kind of web toolkit for Perl programmers. You don't need to be skilled in Perl to make use of mojo, however. mojo get allows you to fetch the contents at a URL, select particular elements via CSS3 selectors, and then transform the result using various methods. For example, let's say I want to get the src attribute of each img element on a page. This is how I could invoke mojo get to get the job done:

    $ mojo get http://example.com img attr src

The img part is of course the element selector; the arguments following it are a method name (any method in Mojo::DOM may be used) plus any arguments for the method. I've found mojo get to be invaluable in scraping a document and extracting some content from it for use further down the command line.

jq

Continuing along the online data trend, I find myself working with JSON pretty often too. Instead of writing a script to extract data from a JSON, you can use jq. It's a handy program that describes itself as "sed for JSON data", and I feel like it lives up to this idea. Just invoking the . operator (which selects the current object) will pretty-print a chunk of JSON:

    $ cat chunk-of.json | jq .

You can select a particular key in a document by adding it after the . operator:

    $ cat chunk-of.json | jq .result

jq's selector syntax is pretty easy to understand and to recall - I can reliably remember how to do the basics each time I use it. For other instances, the documentation is always available for consultation!

uniprops & unichars

I try to write software to be Unicode-aware, which often means something like checking for the Letter property rather than simply looking for a-z and A-Z. To help me in this effort, I make use of two scripts from the Unicode::Tussle CPAN distribution: uniprops and unichars.

To use uniprops, you can give a character or codepoint, and it will tell you about all of the Unicode properties that character exhibits. For example:

    $ uniprops а
    U+0430 ‹а› \N{CYRILLIC SMALL LETTER A}
        \w \pL \p{LC} \p{L_} \p{L&} \p{Ll}
        All Alnum X_POSIX_Alnum Alpha X_POSIX_Alpha Alphabetic Any Assigned
           InCyrillic Cyrillic Is_Cyrillic ID_Continue Is_IDC Cased Cased_Letter LC
           Changes_When_Casemapped CWCM Changes_When_Titlecased CWT
           Changes_When_Uppercased CWU Cyrl Ll L Gr_Base Grapheme_Base Graph
           X_POSIX_Graph GrBase IDC ID_Start IDS Letter L_ Lowercase_Letter Lower
           X_POSIX_Lower Lowercase Print X_POSIX_Print Unicode Word X_POSIX_Word
           XID_Continue XIDC XID_Start XIDS

unichars approaches the property problem from the opposite angle; instead of telling you which properties a character has, it gives you a list of which Unicode characters have a particular property:

    $ unichars '\p{Cyrillic}
     Ѐ  U+0400 CYRILLIC CAPITAL LETTER IE WITH GRAVE
     Ё  U+0401 CYRILLIC CAPITAL LETTER IO
     Ђ  U+0402 CYRILLIC CAPITAL LETTER DJE
     Ѓ  U+0403 CYRILLIC CAPITAL LETTER GJE
     Є  U+0404 CYRILLIC CAPITAL LETTER UKRAINIAN IE
     Ѕ  U+0405 CYRILLIC CAPITAL LETTER DZE
     І  U+0406 CYRILLIC CAPITAL LETTER BYELORUSSIAN-UKRAINIAN I
     Ї  U+0407 CYRILLIC CAPITAL LETTER YI
     Ј  U+0408 CYRILLIC CAPITAL LETTER JE
     Љ  U+0409 CYRILLIC CAPITAL LETTER LJE

combine

If you've done text processing on the command line, there's a good chance you're familiar with comm(1). It's a program that you give two text files along with an option switch that indicates different set operations to perform on the lines in those files. For example, comm -13 A B prints lines unique to B, and comm -12 A B prints lines in both A and B. From my examples, you get a good idea of how difficult comm can be to use; I would always have to consult the man page or sit and think the invocation I would need to make to get the results I needed. Frustrated, I cried out to the Internet:

[ɹɑb hɛlt͡s]

@hoelzro

comm(1) has the weirdest way of specifying what you want, but it's so useful =/

16:37 PM - 30 Dec 2015

1 0

Fortunately, a fellow programmer heard my plea:

Vladislav Naumov

@vnaum

@hoelzro COMBINE(1) is way more straightforward if you want to compare sets of lines.
apt-get install moreutils

17:02 PM - 30 Dec 2015

1 0

Ever since that fateful day, I have not used comm once. combine does exactly what I need, and does so using a syntax that's so straightforward. My examples from above change to combine B not A and combine A and B, respectively.

Discovering More Tools

combine is part of the moreutils package, available on many platforms; I recommend looking at other tools in the package to see if you find something else that solves a problem you have, or better yet, a problem you didn't know you had!

If jq apppeals to you, but you work with other kinds of structured data, you may find this repository on GitHub interesting:

https://github.com/dbohdan/structured-text-tools

Are you there any specialized tools you feel make your life a lot easier? If so, please let me know!

DEV Community: Rob Hoelz

Avoiding spurious wakeups with my ThinkPad X1 Yoga

Random zsh tips

Random zsh tips

Noclobber

Vi Mode

Widgets & Keybindings

Tips by fREW

push-line

AUTO_PUSHD

global aliases

History Events

Discovering your Prometheus via PromQL

Discovering your Prometheus via PromQL

List all metrics for a given exporter

Listing all "namespaces" for a given exporter

List all metrics for all exporters

List all time series for a given exporter

What do you do with five minute chunks of time?

What do you do with five minute chunks of time?

Check Slack

Work on another task for five minutes

Automate it into a longer cycle

Make that five minute chunk into a fifteen minute one

Reduce the wait time

Just sit with it

So, what'd I end up doing?

Others?

More on this from other people

Save time by focusing on the result - using Prometheus' textfile collector to avoid writing an exporter

Save time by focusing on the result - using Prometheus' textfile collector to avoid writing an exporter

Surprising Logic in the Firefox Codebase

Surprising Logic in the Firefox Codebase

Making Slack go slack-jawed: reducing the mental load around Slack notifications

Making Slack go slack-jawed: reducing the mental load around Slack notifications

A little mistake I made in Go

A little mistake I made in Go

Just avoid defer entirely

Call an anonymous function in the for loop and have the defer close(...) happen there:

Call a named function with the channel, and have the defer close(...) happen within that function:

(Your idea here!)

Using udev to disable my infrared camera on Linux

Using udev to disable my infrared camera on Linux

Strategies, Observations, and Open Questions for Working From Home

Observation: Multitasking is more alluring

Strategy: Show up five minutes early for video meetings

Strategy: Change your Slack profile picture every day

Strategy: Treat yourself if the home office is getting treated

Strategy: Adjust your start/end time

Strategy: During video meetings, sometimes you need to be a little rude

Question: Keeping work at work (time)

Question: Maintaining social connections and absorbing insights while remote

Using SQLite's fsdir to effect a quick search index

Unsung Heroes of the Command Line

mojo

jq

uniprops & unichars

combine

Discovering More Tools

`push-line`

`AUTO_PUSHD`

Just avoid `defer` entirely

Call an anonymous function in the for loop and have the `defer close(...)` happen there:

Call a named function with the channel, and have the `defer close(...)` happen within that function: