DEV Community: Adam Parkin

Using namedtuple's to convert dicts to objects

Adam Parkin — Fri, 25 Aug 2023 22:06:00 +0000

A friend shared this with me today and I thought it was pretty neat. If you have a dict, and you want to convert it to an object where the object properties are the keys from the dict, and the values are the values from the dict, you can use a namedtuple to do so. For example:

>>> some_dict = {"name": "My name", "func" : "my func"}
>>> import namedtuple
>>> SomeClass = namedtuple("SomeClass", some_dict.keys())
>>> as_an_object = SomeClass(**some_dict)
>>> as_an_object.name
'My name'
>>> as_an_object.func
'my func'

Won't handle nested dicts (the sub-dicts will still be dicts on the constructed object), but for a quick and dirty way to convert a dict to an object, this seems pretty handy.

Using the splat operator you can also save a line of code:

>>> as_an_object = namedtuple("SomeClass", some_dict.keys())(**some_dict)
>>> as_an_object
SomeClass(name='My name', func='my func')

F-Strings Are F'ing Cool Part 2

Adam Parkin — Sun, 11 Jul 2021 22:27:00 +0000

So in a previous post a couple years ago I wrote about some of the reasons the f-string functionality introduced in Python 3.6 was so useful. Today I learned a few new tricks with them from a recent episode of the Python Bytes Podcast so wanted to revisit the topic & share.

Debugging

So how many times have you been debugging, wanted to know what the value of a variable was at a particular point, and then wrote something like:

>>> myvariable = 42
>>> print(f"The value of myvariable is: {myvariable}")
The value of myvariable is: 42

Ok, putting aside the fact I'm using print statements for debugging (😱), this is extremely common. But what if you had a second variable? Ok, so then you add it as well:

>>> myvariable = 42
>>> myothervariable = 99
>>> print(f"The value of myvariable is: {myvariable} myothervariable: {myothervariable}")
The value of myvariable is: 42 myothervariable: 99

And so on. But I'm lazy, I don't want to repeat the variable name twice, so as of Python 3.8 there's a new specifier to aid with debugging. By appending an equal sign (=) to the expression the both the name & value get emitted:

>>> myvariable = 42
>>> myothervariable = 99
>>> print(f"The value of {myvariable=} {myothervariable=}")
The value of myvariable=42 myothervariable=99

Super handy. What's really crazy is it works with expressions too:

>>> myvariable = 42
>>> myothervariable = 99
>>> print(f"Calculation: {(myvariable * 3.14159 / myothervariable)=}")
Calculation: (myvariable * 3.14159 / myothervariable)=1.3327957575757574

This can be handy when printing out properties of an object:

>>> class Foo:
... bar = 42
...
>>> f = Foo()
>>> print(f"{f.bar=}")
f.bar=42

It's also particularly handy if you have a function which takes arbitrary parameters via args and kwargs and you want to see what the values of them are:

>>> def foo(*args, **kwargs):
... print(f"{args=} {kwargs=}")
...
>>> foo(242, 2342, "adlfk", arg1=9234, arg2="dfslkj")
args=(242, 2342, 'adlfk') kwargs={'arg1': 9234, 'arg2': 'dfslkj'}

Common Format Specifiers

So one of the things I talked about in my last post on F-strings was how you can use all the same format specifiers you used to use with .format() with F-strings. A common example of this is to format a float to a specific number of decimal places:

>>> import math
>>> print(f"Pi to 3 digits: {math.pi:.3f}")
Pi to 3 digits: 3.142

Or formatting datetimes:

>>> from datetime import datetime
>>> print(f"Today is {datetime.now():%B %d, %Y}")
Today is July 11, 2021

This is neat and all, but oftentimes these format strings are used in many places in a project. Say for example you want all dates in your project to be formatted like the example above. One approach would be to repeat that format string (%B %d, %Y) in each place you format a date. The problem though is if that format changes (say in the future you want 2 digit years, or want to include the day of the week, etc) you'd then have to search & replace every spot where you've used that format.

Ok, so this is software engineering 101, time for the "Don't Repeat Yourself" (or DRY) principle to apply, so maybe you extract out a function to format a datetime:

>>> def format_date(datetime_obj):
... return f"{datetime_obj:%B %d, %Y}"
...
>>> print(f"Today is {format_date(datetime.now())}")
Today is July 11, 2021

Which works and is perfectly reasonable, but now you have this little function floating around which makes the f-string expression a little more cumbersome. One could also make a case that defining a function that is a single line of code is a bit overengineered (I'm not particularly sympathetic to this view myself, but some are). And really the format string is a constant, so arguably it's a little weird to have a function to express it's application.

But, here's where my mind was blown -- you can nest f-string expressions:

>>> DATETIME_FORMAT='%B %d, %Y'
>>>
>>> print(f"Today is {datetime.now():{DATETIME_FORMAT}}")
Today is July 11, 2021

Now our format expression is a single constant variable, and we just defer to it wherever we want to format a datetime. If that format changes, we simply update the constant. Really useful trick.

VS Code Tip Of The Day - Selectively Adding Files To A Git Commit

Adam Parkin — Sat, 03 Apr 2021 17:26:00 +0000

Small tip: sometimes when working on something I find myself making changes to many files. I then only want to include a few of those files in my next Git commit. To date I've always just manually done a git add <file> in the terminal for each file I want to add, but this is tedious if there's many of them.

Turns out there's an easy way to do this in VS Code. If you go to the Source Control item on the left nav (the icon that looks like a branch), you’ll see a list of all untracked and modified files. For example:

In this screenshot you can see I have a number of files that have been modified (the ones with the "M" beside them) and two new (untracked) files (the ones with the "U" beside them).

If you want to see what's changed in any of the modified files, clicking the item will bring up a diff window. If you then want to include (or "stage") this file for the next commit, right-click it and pick "Stage Changes":

Repeat for each file you want to include, and then do a git commit to complete the commit. Alternatively if you're anti-terminal you can click the checkmark on this same view to complete the commit.

The Wild World of Apple Silicon

Adam Parkin — Sat, 06 Mar 2021 18:33:00 +0000

I recently took the plunge and obtained a shiny new M1-powered Macbook Air. For those unfamiliar, last year Apple announced that they were now building machines with a brand new ARM-based architecture, making the switch from the long-lived x86 Intel architecture. This brought promises of amazing battery life, amazing performance, and terrifying compatibility issues. Now that I've been living with this machine for a week or two, I thought I'd recap my experience both setting it up, any gotchas or surprises along the way, as well as my experiences around how well the new architecture works as experienced through the lens of a developer.

Lesson 1: Rosetta Works, But Sucks Battery Like You Wouldn't Believe

Everything I've run through Rosetta has been flawless from a functionality perspective. Having said that though: anything run through Rosetta does seem to suck battery life. And not just apps that are normally CPU intensive. For example: I found that having Dropbox (which doesn't support M1), Itsycal, and Spectacle constantly running in my menu bar all seemed to have a significant drain on battery life. I've since switched from Dropbox to Sync, from Spectacle to Rectangle, and have uninstalled Itsycal as I still haven't found an M1-powered replacement.

Lesson 2: Which Apps Are M1 Ready is Really Random

So of these, which would you expect are M1 ready right now?

Slack
Chrome
Firefox
Visual Studio Code
Sublime Text
Dropbox
Docker

If you answered the first three, then kudos to you, though until very recently (ie within the last week or so) VS Code only had M1 support via Insiders. It looks like Sublime Text 3 will never support M1 and ST4 is still a long ways off, which for a paid product used by a lot of Mac users is truly mind-blowing to me. The fact that Dropbox still doesn't have M1 support is just inexcusable at this point (particularly given it's an "always running" app). Docker has a preview version that's been out for some time, but full support still seems like a long ways off. Side note: I haven't tried the preview version, and I don't plan on it as there's been mixed reports on how stable it is (a positive take and a less positive take).

Lesson 3: Homebrew is Ready, but Your Obscure Package Might Not Be

With Homebrew 3.0, the popular package manager is now M1-ready. I can happily report that the vast majority of packages I use are M1-native. I installed python3, git, git-extras, bash-completion, pyenv, pipx, starship, the_silver_searcher, hugo, watch and a bunch of others without issue, and all seem to be M1 as reported in Activity Monitor.

So what happens when something isn't?

$ brew install hadolint

Updating Homebrew...
==> Auto-updated Homebrew!
Updated 1 tap (homebrew/core).
==> New Formulae
bas55 delve geph4 kotlin-language-server latino libpipeline openmodelica oras sqlancer
==> Updated Formulae
Updated 267 formulae.

Error: hadolint: no bottle available!
You can try to install from source with:
  brew install --build-from-source hadolint
Please note building from source is unsupported. You will encounter build
failures with some formulae. If you experience any issues please create pull
requests instead of asking for help on Homebrew's GitHub, Twitter or any other
official channels.

I asked about this on Github and since hadolint is built with ghc and ghc isn't M1 ready (and likely won't be for some time) you either live without the package, install a separate Rosetta-based brew installation, or obtain the package from some other means (in the case of hadolint this is what I did: there are self-contained binaries on their Github, so I threw the latest in my path)

One pro-tip: this site is awesome for seeing if a package you're interested in is M1-ready or not: https://doesitarm.com/kind/homebrew/

Lesson 4: Homebrew Is Different Now

One minor gotcha I ran into is that Brew installs to a different directory: /opt/homebrew. If you have scripts (think things like .bashrc & the like) that reference the old brew path they'll have to be tweaked.

Lesson 5: Instant On is Amazing

This is a minor thing, and honestly I didn't think I'd like it as much as I do, but M1 Macbooks feature an "instant on" wake up from sleep. And it truly is "instant". Ie before I've completely opened the lid of my MBA the screen is already on and awaiting input. This even happens when I have an external display connected. Contrast this with my Intel-based Macbook Pro for work which takes a good 30 seconds to resume from sleep (often longer if I have external displays connected).

Surprisingly this meant I didn't bother installing Amphetamine on this machine since there's no point -- I don't care if my M1 Mac falls asleep as it wakes up so damn fast.

Lesson 6: Big Sur Is Less Good

This is less dev-orientated, but I really don't like Big Sur. The new Notification Center is annoying and just wastes space on my menubar. Toast notifications look much bigger on screen (so are more jarring). Lots of little annoyances with it, none of which are dealbreaking, but if I had my way I'd have Catalina instead of Big Sur on this machine (alas, not an option).

In Summary

This machine is awesome. It's expensive (as all Macs are), but is crazy fast, and (once you get rid of all your Intel apps) sips battery very lightly.

Docker is really the only thing that I miss at this point from turning this into a real dev machine. Hopefully full M1 support will arrive for that though I can't help but wonder if Docker will ever be completely compatible (if you build a Docker image on M1, can you run that image on an Intel based machine?)

Using Starship For Terminal Prompt Goodness

Adam Parkin — Sat, 27 Feb 2021 18:39:00 +0000

Recently I had the good fortune of attending the 2021 iteration of the Pycascades conference, and there was at least one talk that mentioned Starship and at least one other where the presenter happened to be using it. For those not in the know, Starship is a cross-shell compatible terminal prompt generator written in Rust that is crazy fast and crazy customizable.

I was intrigued, and decided to take my (reasonably sophisticated) Bash prompt and Starship-ify it. In this post I'll outline some of the things I went through, lessons learned, and hopefully impart some advice on how to do things with it.

To begin with, here was my old bash prompt:

That prompt has a bunch of things in it, and shows:

exit code of the last command (0 in this case)
the current day & time, along with the TZ info
the time since I last rebooted (2 days in this case)
my current directory
the name of the current branch I’m on (starship in this case) and the asterisk to indicate there’s uncommitted changes
The lambda symbol as my input symbol (not a Half-Life reference, more a reference to my functional programming days)

Not shown in that screenshot is that when I have a Python virtual environment activated it also gets displayed in the prompt.

The "code" for this that resided in my .bashrc file:

function uptimeinfo {
    uptime | perl -ne 'if(/\d\s+up(.\*),\s+\d+\s+users/) { $s = $1; $s =~ s/^\s+|\s+$//g; print $s; }'
}

function proml {
  local BLACK="\[\033[0;30m\]"
  local GRAY="\[\033[1;30m\]"
  local RED="\[\033[0;31m\]"
  local LIGHT\_RED="\[\033[1;31m\]"
  local GREEN="\[\033[0;32m\]"
  local LIGHT\_GREEN="\[\033[1;32m\]"
  local BROWN="\[\033[0;33m\]"
  local YELLOW="\[\033[1;33m\]"
  local BLUE="\[\033[0;34m\]"
  local LIGHT\_BLUE="\[\033[1;34m\]"
  local PURPLE="\[\033[0;35m\]"
  local LIGHT\_PURPLE="\[\033[1;35m\]"
  local CYAN="\[\033[0;36m\]"
  local LIGHT\_CYAN="\[\033[1;36m\]"
  local LIGHT\_GRAY="\[\033[0;37m\]"
  local WHITE="\[\033[1;37m\]"
  case $TERM in
    xterm*)
    TITLEBAR='\[\033]0;\u@\h:\w \D{%a %b %d %Y %l:%M%p (%Z%z)}\007\]'
    ;;
    *)
    TITLEBAR=""
    ;;
  esac

PCOLOR="\[\033[\$(promptcol)\]"

# note that in the following prompt the error code item (\$?) must be the
# first item in the prompt. Otherwise it'll show the errorcode for the last
# command executed in producing the prompt.
PS1="${TITLEBAR}\
$BLUE [$GREEN[\$?] [\D{%a %b %d %Y %l:%M%p (%Z%z)}] [Up: \$(uptimeinfo)] $BROWN\u@\h:\w $LIGHT\_GRAY\$(\_\_git\_ps1)\
$BLUE]\
\n$PCOLOR λ $LIGHT\_GRAY"
PS2='> '
PS4='+ '
}
proml

That's a lot, and to be honest, I might be missing some of the code as my .bashrc file is full of random little snippets.

It's also shell-specific. If I ever wanted to move to Zsh, or Fish, or whatever, I'd have to re-invent that (and this has been an impediment for me to switching shells).

So I took this as a starting point and wanted to recreate it in Starship. Starting point was to install Starship with Brew:

brew install starship

Easy enough. Next is to initialize it in your shell of choice. In my case for Bash this was adding the following to my .bashrc:

eval "$(starship init bash)"

There are equivalent instructions on the Starship website for other prompts

Lastly, you need a ~/.config/starship.toml file where you'll configure your prompt.

touch ~/.config/starship.toml

Now open up a new terminal and you should see the default Starship prompt which is actually quite sophisticated out of the box:

Your output may vary, as many of the items in a Starship prompt are dynamic depending on your current context. In this you can see:

I'm in a directory called dotfiles
I'm currently in a Git repo, but not on any branch instead checked out an arbitrary commit with SHA 931e5c4
The coffee cup has to do with Java, but I don't have a valid JDK installed so it's not showing what version
My current Python environment is Python 3.9.2
My AWS environment is configured to communicate with the ca-central-1 region

That's a lot! Note that each of those things is what Starship calls a Module. This is one of the key things about Starship is that each part of your prompt is made up by a distinct module that you configure. So that prompt is currently displaying the Directory Module, the Git Branch Module, the Git Commit Module, the Python Module, and the AWS Module. Technically it's showing a few others, but this gives you an idea of how Starship composes your prompt by stringing together some modules.

The full list of all modules can be found at: https://starship.rs/config/

Ok, lets get started trying to configure this prompt to be like my old one. I opened up my starship.toml and added the contents from the example on the Starship docs:

# Don't print a new line at the start of the prompt
add\_newline = false

# Replace the "❯" symbol in the prompt with "➜"
[character] # The name of the module we are configuring is "character"
success\_symbol = "[➜](bold green)" # The "success\_symbol" segment is being set to "➜" with the color "bold green"

# Disable the package module, hiding it from the prompt completely
[package]
disabled = true

With this you probably won't see much change, aside from the symbol changing from a > to a ➜ due to the configuration of the character module. See, everything in a Starship prompt (including the symbol at the end) is a module. Well, mostly. 😜

Ok, so it currently shows the directory, but not like my old prompt where A) it was a yellow-ish colour, and B) showed the full path. Looking at the docs for the Directory module, I gave this a try to configure it:

[directory]
truncation\_length = 100
truncate\_to\_repo = false
style = "yellow"
format = "[:$path]($style)[$read\_only]($read\_only\_style) "

Let's explain this a little bit to give a feel:

truncation_length controls how many directories deep you have to be before Starship will abbreviate the directory name in your prompt. I rarely go very deep and to be honest when I do I still want to see the full path so I made the number rediculously high so that it never truncated
truncate_to_repo is a special setting that controls if the directory is truncated to the root of the Git repo you are currently in. Again, I don't like this (I want to see the full path), so disabled it
style is a common setting on (I believe) every module and controls the colour of the module when rendered. In this case saying "yellow" to match my old prompt. Styles are covered in depth in the docs
format is another common setting across every module and controls effectively the "layout" of the module.

It's worth digging into the format directive there, as understanding this goes a long way in understanding how you control Starship's output. The expression:"[:$path]($style)[$read_only]($read_only_style) " says start a text group(this specified by the [ and ] delimiters) and have it output a colon (:) followed by the value of the $path variable. Each module has its own set of variables that get populated with values that are relevant to that module (in this case $path ends up being the full path of the current working directory). The brackets that follow a text group specify a style string. You might wonder "but isn't that what the style setting is for?" And yes, but essentially the style setting defines the "default" style within a module, and style strings within the format can override that. In this case, $stylecorresponds to the style setting defined in the configuration for the module (in this case "yellow"). You can see that later in this definition I have the $read_only variable (which gets displayed when the current working directory is read only) and has a different style defined for that scenario (the default $read_only_style is "red", but you could change that in this configuration by adding a read_only_style="blue" setting to the directory config). In any case the relevant part of the docs on format strings is here.

Clear as mud? Admittedly, this does take a bit to get your head around (or did for me), but basically you override settings in the config as appropriate to tweak each module to your liking.

Ok, that's fine, but Adam how do we control the order of items in the prompt? And that's a good question that took me a little while to figure out. Turns out that the prompt as a whole has a format setting. The default is to show all modules, this is from the docs:

format = "$all"

# Which is equivalent to
format = """
$username\
$hostname\
$shlvl\
$kubernetes\
$directory\
$git\_branch\
$git\_commit\
$git\_state\
$git\_status\
$hg\_branch\
$docker\_context\
$package\
$cmake\
$dart\
$dotnet\
$elixir\
$elm\
$erlang\
$golang\
$helm\
$java\
$julia\
$kotlin\
$nim\
$nodejs\
$ocaml\
$perl\
$php\
$purescript\
$python\
$ruby\
$rust\
$swift\
$terraform\
$vagrant\
$zig\
$nix\_shell\
$conda\
$memory\_usage\
$aws\
$gcloud\
$openstack\
$env\_var\
$crystal\
$custom\
$cmd\_duration\
$line\_break\
$lua\
$jobs\
$battery\
$time\
$status\
$character"""

This is what controls the order of modules. Move an item up, and it'll appear earlier in the prompt, move down to move it later in the prompt. Personally I don't like this, as it means if you want to change the order of items you have to override the entire format string. It'd be nice if there was an "index" value or something on each module that could determine ordering, but oh well.

In any case it also provides a global "completely hide" ability for a module -- if you remove it from the format then it won't be displayed. Note that I don't think hiding from the format is the same thing as disabling a module. Each module has a disabled setting which (if true) disables that module (so won't get displayed, and I believe not evaluated).

Ok, with this I continued on and got most of my old prompt in place:

# Don't print a new line at the start of the prompt
add\_newline = false

[character]
success\_symbol = " [λ](grey)"
error\_symbol = " [λ](bold red)"

[directory]
truncation\_length = 100
truncate\_to\_repo = false
style = " yellow"
format = "[:$path]($style)[$read\_only]($read\_only\_style) "

[git\_branch]
symbol = ""
style = "bold white"
format = '[\($symbol$branch\)]($style) '

[git\_status]
# I don't care about untracked files or that there's a stash present.
untracked = ""
format = '([\[$conflicted$deleted$renamed$modified$staged$behind\]]($style) )'
modified = '\*'

[status]
disabled = false
format = '[\[$status - $common\_meaning\]](green)'

This is close, but is missing the current time and "uptime". And those proved to be wrinkles for me. My previous time showed the full current date & time along with the current timezone info (ex: PST-0800). There's a time module that can recreate all of this except the timezone name. It should, given the docs on format strings for the underlying Chrono library, but turns out there's a bug there that causes that to not work.

But, Starship supports custom commands to be in a module, so I added a custom command to just defer to the standard date command on *-nix type systems:

[custom.tztime]
command = 'date +"%a %b %d %Y %l:%M%p (%Z%z)"'
when = "true"
format = '[\[$symbol($output)\]](green)'

This gives me the time as it was before. The when bit there is true so that this command is always displayed. We'll get to how to control where custom commands show up in a minute, but there was one more custom command I needed for my system uptime. The way I did this in my old prompt was a Bash function:

function uptimeinfo {
    uptime | perl -ne 'if(/\d\s+up(.\*),\s+\d+\s+users/) { $s = $1; $s =~ s/^\s+|\s+$//g; print $s; }'
}

And then I used uptimeinfo in my prompt. But that's Bash-specific, so instead I created a little shell script called uptime.sh with the following contents:

#!/bin/sh

echo "[`uptime | perl -ne 'if(/\d\s+up(.\*),\s+\d+\s+users/) { $s = $1; $s =~ s/^\s+|\s+$//g; print $s; }'`]"

Which just does the same thing, echoes out the system uptime with it filtered through Perl to make it more concise. Now the starship config:

[custom.uptime]
command = "uptime.sh"
when = "true"
format = "[$symbol($output)](green)"

Now back to that layout question: we control the order of things by the global format setting, but how do we refer to custom commands? Like this:

format = """
$status \
${custom.tztime} \
${custom.uptime} \
$username\

.... rest of the file ...

Ie ${custom.<your custom module name>}. Ok with all that, I then had everything I needed, and continued to flesh out my prompt. My final config:

format = """
$status \
${custom.tztime} \
${custom.uptime} \
$username\
$hostname\
$shlvl\
$kubernetes\
$directory\
$git\_branch\
$git\_commit\
$git\_state\
$git\_status\
$docker\_context\
$package\
$cmake\
$nodejs\
$perl\
$python \
$ruby\
$rust\
$terraform\
$vagrant\
$nix\_shell\
$conda\
$aws \
$env\_var\
$cmd\_duration\
$line\_break\
$character"""

# Don't print a new line at the start of the prompt
add\_newline = false

[aws]
format = '\[AWS: [$profile/($region)]($style)\]'
symbol = ''
style = 'bold white'

[character]
success\_symbol = " [λ](grey)"
error\_symbol = " [λ](bold red)"

[cmd\_duration]
min\_time = 1000

[directory]
truncation\_length = 100
truncate\_to\_repo = false
style = " yellow"
format = "[:$path]($style)[$read\_only]($read\_only\_style) "

[git\_branch]
symbol = ""
style = "bold white"
format = '[\($symbol$branch\)]($style) '

[git\_status]
# I don't care about untracked files or that there's a stash present.
untracked = ""
format = '([\[$conflicted$deleted$renamed$modified$staged$behind\]]($style) )'
modified = '\*'

[python]
format = '[${symbol}${pyenv\_prefix}(${version} )(\($virtualenv\))]($style)'

[status]
disabled = false
format = '[\[$status - $common\_meaning\]](green)'

[custom.tztime]
command = 'date +"%a %b %d %Y %l:%M%p (%Z%z)"'
when = "true"
format = '[\[$symbol($output)\]](green)'

[custom.uptime]
command = "uptime.sh"
when = "true"
format = "[$symbol($output)](green)"

[env\_var]
variable = "0"

#### Disabled modules ####

# add these back to format if you want them:
# $time\
# $hg\_branch\
# $dart\
# $dotnet\
# $elixir\
# $elm\
# $erlang\
# $golang\
# $helm\
# $java\
# $julia\
# $kotlin\
# $nim\
# $ocaml\
# $php\
# $purescript\
# $swift\
# $zig\
# $memory\_usage\
# $gcloud\
# $openstack\
# $crystal\
# $lua\
# $jobs\
# $battery\
[hg\_branch]
disabled = true
[dart]
disabled = true
[dotnet]
disabled = true
[elixir]
disabled = true
[elm]
disabled = true
[erlang]
disabled = true
[golang]
disabled = true
[helm]
disabled = true
[java]
disabled = true
[julia]
disabled = true
[kotlin]
disabled = true
[nim]
disabled = true
[ocaml]
disabled = true
[php]
disabled = true
[purescript]
disabled = true
[swift]
disabled = true
[zig]
disabled = true
[memory\_usage]
disabled = true
[gcloud]
disabled = true
[openstack]
disabled = true
[crystal]
disabled = true
[lua]
disabled = true
[jobs]
disabled = true
[battery]
disabled = true

# Until these get resolved, doing my own datetime with date:
# https://github.com/starship/starship/discussions/2360#discussioncomment-391911
# https://github.com/chronotope/chrono/issues/288
[time]
disabled = true
# format = '[\[$time\]](green) '
# time\_format = "%a %b %d %Y %l:%M%p (%z)"

Note current version (in case I revise in the future) is at: https://github.com/pzelnip/dotfiles/blob/mainline/.config/starship.toml)

This gives a prompt like the following:

Pretty sweet, lots of dynamicism where needed, but still has all the things I liked from before. Definitely took some time to get this just the way I liked it, but am happy with the result, and as a bonus: now I have the same prompt if I'm in Bash, Zsh or whatever.

Building A VS Code Extension Without Installing Node By Docker Magic

Adam Parkin — Sat, 27 Feb 2021 00:14:00 +0000

I recently wanted to work through the tutorial for building a VS Code extension, but the first step is to install Node.js locally, which, well, I don't want to do. Sorry to the Node peeps out there, but I don't want to touch the (large and rather complex) Node ecosystem just to try out building a VS Code extension. So I then thought, "Hey, you can install Node on a Linux box, so why can't I just do it inside a Docker container?"

And of course, you can, and not only that, but with the magic that is the VS Code Remote Containers extension you can even have VS Code work as if it's all on your local host machine. Let's give this a try.

Install The Pre-Requisites

First step: install the Remote Development extension pack from Microsoft. This will allow you to treat a running container, ssh connection to a remote machine, or Linux environment via WSL (assuming you are a Windows person) as though it's a local folder. If you've never played with this before, it really is worth checking out as it's amazing.

I'm going to assume you also have Docker installed. If not, follow the instructions on their site, or any one of the 59 million or so tutorials online.

Create the Dockerfile

Ok, now let's create a Dockerfile that has what we need. The VS Code tutorial mentions you need the following:

Node.js
Git
Yeoman
The VS Code Extension Generator

Ok, all those things boil down to some basic terminal commands, but as a simple starting point, let's use the official Node.js Docker image to get Node.js. To do this, make node:10-alpine the base image, and then install those dependencies:

FROM node:10-alpine

RUN apk add --update git bash
RUN npm install -g yo generator-code

USER node

ENTRYPOINT /bin/bash

WORKDIR /home/node

Breaking this down a bit:

the FROM line says the image to use as a base image is the node:10-alpine image from Dockerhub. This gives you npm & whatnot already installed.
the apk add line installs Git and Bash (alpine doesn't have Bash installed by default)
the npm install -g line installs Yeoman and the VS Code Extension Generator
the USER line creates a user called node, which you need as otherwise yo fails when you run it in the container due to permission issues (doesn't seem to like running as root)
the ENTRYPOINT says when a container is started from this image, start off by running /bin/bash
the WORKDIR says when a container is started from this image, start in the /home/node directory

Save this file as Dockerfile. Note that I used node:10-alpine (so a 10.x version of Node), feel free to replace with a newer version if you want (I have no idea what version the VS Code Extension Generator wants).

Build The Docker Image

Now you want to build the Docker image, run this command in the same directory as the Dockerfile:

docker build -t vscodeenv:latest .

The vscodeenv name is totally arbitrary, feel free to name it whatever you want, but that's the name I'll use for this blog post. You'll see a bunch of output, and after it's done, you should be able to see the built image when you do a docker images:

$ docker images
REPOSITORY TAG    IMAGE ID     CREATED        SIZE
vscodeenv  latest 37d9e66fffbc 48 minutes ago 186MB

Run the Image

Now we have a built Docker image with all the tooling you need for the tutorial. Next step is to spin up a container from this image. I'm sure if you get into the VS Code devcontainer.json stuff you could do this from within VS Code, but I just do it from the terminal:

$ docker run -it --name vscodeenv -v /Users/aparkin/dockerpath/:/home/node --rm vscodeenv:latest
bash-5.0$

Replace /Users/aparkin/dockerpath/ with the full path to a directory where you want to put your extensions code (it's perfectly fine to be the same directory where you put theDockerfile).

Note that this just drops you into a bash shell inside the running container. Leave this window open, so long as this is open your container will be running. Once you type "exit" here, the container will be terminated (which we don't want to do until we're done working on our little extension).

To break the docker run command down a bit, the key bits:

-it means run in interactive mode (ie where you can type in commands)
--name vscodeenv gives the container the name vscodeenv (again, this is arbitrary)
-v ..... tells it to mount the path you specified on your local host filesystem into /home/node in the running container (so any files in the path you specify will show up in the container inside /home/node)
--rm tells Docker to delete the container once you exit

Create The Skeleton Extension

The VS Code tutorial indicates after installing everything, you should run the generator. Do this in the running docker container with the yo code command:

bash-5.0$ yo code

     _-----_ ╭──────────────────────────╮
    | | │ Welcome to the Visual │
    |--(o)--| │ Studio Code Extension │
   `---------´ │ generator! │
    ( _´U`_ ) ╰──────────────────────────╯
    / ___A___ \ /
     | ~ |
   __'.\_\_\_.'__
 ´ ` |° ´ Y `

? What type of extension do you want to create? (Use arrow keys)
❯ New Extension (TypeScript)
  New Extension (JavaScript)
  New Color Theme
  New Language Support
  New Code Snippets
  New Keymap
  New Extension Pack
  New Language Pack (Localization)

This generator walks you through creating your first extension. Following VS Code's tutorial I picked:

New Extension (TypeScript)
Name of my extension: "AdamsExtension"
Identifier of my extension: "adamsextension" (the default)
Description I entered random gibberish
Initialize a Git repo: yes
Bundle with Webpack: no
Package manager: npm

After that, it goes ahead and installs all the various npm dependencies -- all within the docker container.

Attach To the Container

Now in VS Code open the command palette and search forRemote-Containers: Attach to Running Container.... Pick this, and then your running container called vscodeenv should appear in the list:

Pick it, and VS Code will open a new Window "attached" to the running container. For more details, consult the official docs.

Now click on "Open Folder" and navigate to your adamsextension (or whatever you called your extension) folder and click OK. You then get a VS Code window "attached" to the running docker container, with your test extension open and ready to play with. Here's a screenshot to give an idea:

Now you can hit F5 and VS Code will open up a new Extension Development Host window with your test extension loaded. In that window you should be able to search for the "Hello World" command in the command palette and run the command.

Profit

And that's it, no Node.js or any other tooling installed to your local machine other than VS Code & Docker. Once you're done playing around, exit out of the running Docker container (enter "exit" in the bash prompt) and the container will be terminated, and all the files you created will remain in the path you mounted into the container.

If you want to later pick up where you left off, just run the same docker run command as before, re-attach to the container, and re-open the folder.

Git Image Diff with iTerm2

Adam Parkin — Sun, 14 Feb 2021 20:37:00 +0000

So iTerm2 has this neat imgcat command that allows you to "cat" or view an image right in the terminal. It's pretty sweet, but I had an idea: what if we used that with Git for diffing changed images in agit diff?

This seems like it should be possible, you can replace what git does when you do a diff of two different binary files. For example, the spaceman-diff package by Zach Holman does this to do a diff of two images as ASCII art.

We want to do the same, but instead of seeing an ASCII art version of an image, it'd be cool to see the two versions of the image itself right in the terminal.

Setting Git Attributes

The first thing is create a file in your home directory called ~/.config/git/attributes. In here you can define a mapping between file types and what command Git will run when doing a diff of those filetypes. In my case I entered the following:

\*.png diff=image
\*.jpg diff=image
\*.gif diff=image
\*.jpeg diff=image

This tells git that when doing a diff of a file that ends with png, jpg,gif, or jpeg to use the image config.

Set Up The Image Config

Now, in your ~/.gitconfig file, add the following:

[diff "image"]
    textconv = imgcat

This tells Git to use iTerm's imgcat script for converting the binary image file to text. This seems weird, but this is actually how iTerm's imgcat command works: it converts the binary image into a textual stream that iTerm knows how to understand and then render as an actual image.

Put imgcat In Your Path

Next is put iTerm's imcat script somewhere on your path. You can download it from here. Save that somewhere, make sure you chmod it to be executable (chmod +x imgcat), and then throw it into some directory on your path. You can confirm it's there by typing imgcat into an iTerm window:

$ imgcat
Usage: imgcat [-p] filename ...
   or: cat filename | imgcat

Note that you have to put this script in your path, you can't just rely on the version that's inlined into your terminal with iTerm's shell integration.

Profit

Now when you change an image in a Git repo and do a git diff while in iTerm you'll see a preview of the original image and the changed image. Example:

The above image is the original, the 2nd image is what I changed it to. Note that because imgcat is iTerm specific, it won't work in other terminals. If you do a git diff in a different terminal (ex: the integrated terminal in VS Code) you'll see just the ordinary blank output:

Python manage.py pytest or Making Django Use Pytest

Adam Parkin — Sun, 19 Apr 2020 01:51:00 +0000

This post was originally posted on Apr 18, 2020 to https://www.codependentcodr.com/python-managepy-pytest-or-making-django-use-pytest.html#python-managepy-pytest-or-making-django-use-pytest

Edit: For those who prefer visual content, I've recorded this as a video on Youtube which you can find at: https://youtu.be/toKW2YLnGFQ

Ok, so I've had to convert some Django projects to use Pytest as the test runner rather than the built in one that Django uses. This is actually pretty straightforward, and I even recorded a Youtube video showing the process.

That's all fine and good, but one of the complaints I've heard from Django-ista's (is that a term? Djangoites? Django Devotees?) is that it means now the good old normal python manage.py test no longer works.

So challenge accepted, as one can certainly create custom manage.py commands in Django.

Python Manage.py pytest

So first challenge is "how do we run pytest from Python?" as normally you run Pytest as a command line tool. As it turns out there's docs on how to do this on Pytest's site.

The trick is to import pytest and then call pytest.main() passing in the same command line arguments you'd give to Pytest in the terminal to thatfunction. As an example:

pytest.main(['--lf'])

Would be the same thing as doing pytest --lf on the command-line. Easy peasy.So started a basic custom management command in a file called pytest.py and put it into my Django project's management/commands directory.

import pytest
from django.core.management.base import BaseCommand

class Command(BaseCommand):
    help = "Runs tests with Pytest"

    def handle(self, \*args, \*\*options):
        pytest.main()

This works, in that now I can do python manage.py pytest and it'll run Pytest as if I just ran the pytest executable in the current directory.

Cool, but how do I start passing arguments? Normally in a custom Django management command you define a add_arguments function and use the argparsemodule to define the expected arguments for your custom command. In this case though, I essentially want the interface to Pytest, which would be non-trivial to recreate by hand (there's a lot of options on that Pytest executable).

But, with argparse, there is a way to essentially say "accept any arguments",and that's the argparse.REMAINDER value for the nargsparameter.

argparse.REMAINDER. All the remaining command-line arguments are gathered into a list. This is commonly useful for command line utilities that dispatch to other command line utilities

Perfect, that's exactly what I want. Adding to our management command is straightforward:

import argparse
import pytest
from django.core.management.base import BaseCommand

class Command(BaseCommand):
    help = "Runs tests with Pytest"

    def add\_arguments(self, parser):
        parser.add\_argument("args", nargs=argparse.REMAINDER)

    def handle(self, \*args, \*\*options):
        pytest.main(list(args))

Now whatever arguments we pass to our manage.py pytest command will be passed directly through to the pytest.main function. Exactly what I want, and super concise.

But That's Still Different

At this point it worked, but I could still hear those nagging voices saying,"yeah but manage.py pytest is not the same as manage.py test". Fine, as it turns out though you can override any of the built-in manage.py commands.

The trick is to just create a custom management command with the name of the command you want to override, and make sure your app is the last one in Django'sINSTALLED_APPS setting. So in our case we could just rename ourmanagement/pytest.py file to management/test.py and it'd work. But I kindaliked having both (ie both manage.py pytest and manage.py test being effectively an alias to it). So I created a management/test.py file and put in the following:

# Override the built in Django manage.py test with pytest
from djangotest.management.commands.pytest import Command

Yup, that's it, 1 line of code. Now doing a python manage.py test runs Pytest as the test runner.

Did this all on a test project, source is up on Github at:https://github.com/pzelnip/djangotest

Or if you just want to see the management commands, they're at:https://github.com/pzelnip/djangotest/tree/master/djangotest/management/commands

Visual Studio Code Tasks and Split Terminals

Adam Parkin — Tue, 19 Feb 2019 16:43:03 +0000

This post was originally posted on Feb 10, 2019 to https://www.codependentcodr.com/visual-studio-code-tasks-and-split-terminals.html#visual-studio-code-tasks-and-split-terminals

So as a big Visual Studio Code fan, I've long made use of the tasks feature. The most recent (January 2019, 1.31) update added a cool new feature related to this that I've been waiting for for some time. I thought I'd do a little write up about this and how I use tasks with VS Code, particularly as a Pythonista.

Task Basics

As a starting point, to give a basic idea of what tasks are, they're effectively little shortcuts to terminal commands that you can trigger from within VS Code. They're commonly used for things like triggering build tasks, or starting up a local dev server, etc. The thing that makes them nice is that they can be triggered from the command pallette much like normal VS Code commands. For example, when working on this blog, I'll use a task to fire up a local dev server to test out content before committing/pushing it. It looks something like this:

At this point I can then go to http://localhost:8000 and see the content I've been working on. Handy. To create a task, you open up the command pallette and pick "Tasks: Configure Task" and you'll be prompted with some default template tasks, or the option to "Create tasks.json file from template" which gives you total control and is the option I use.

A tasks.json file contains a number of JSON blobs which define your tasks. They look something like:



{
    // See https://go.microsoft.com/fwlink/?LinkId=733558
    // for the documentation about the tasks.json format
    "version": "2.0.0",
    "tasks": [
        {
            "label": "Run Server",
            "type": "shell",
            "command": "source $(dirname ${config:python.pythonPath})/activate && make devserver"
        },
    ]
}

This is the definition for my run local dev server that I showed in the video. You can have as many tasks as you want, the tasks property is just a list of these definitions. The full list of properties and options are
in Microsoft's excellent docs.

Common Python Tasks

So now that we have an idea of what tasks are, what are some of the neat things you can do with them, particularly from the perspective of a Python developer? These are some of the common ones I set up, most of which are Django related, since much of my day job is working in that framework:

Running a Dev Server



{
    "label": "Run Server",
    "type": "shell",
    "command": "${config:python.pythonPath} manage.py runserver --noreload",
},

This is basically the analogy to the task I showed previously, just using Django's runserver command. One thing to note about this: note that the label is the same as the one for my blog project. tasks.json files are stored per-project, but one neat thing is you can assign a hotkey to a given task. In my keybindings.json I
have:



{
    "key": "cmd+shift+r",
    "command": "workbench.action.tasks.runTask",
    "args": "Run Server"
},

This allows me to start a local dev server by simply hitting a hotkey, and so long as I name that "start up a local dev environment" task the same on each project, it's the same keystroke to start up a local dev environment.

Hitting a Health Check URL



{
    "label": "Healthcheck (requires running server)",
    "type": "shell",
    "command": "curl http://127.0.0.1:6100/health"
},

Once I have a local server running, it's handy to be able to quickly hit the
health check url (you added a health check to your API right?). Again, this is small, but handy as it saves me the trouble of tabbing over to a terminal window, typing out the curl command, realizing that this project runs on a different port, going to look that up, etc, etc, etc.

Running Unit Tests



{
    "label": "Run Unit Tests",
    "type": "shell",
    "group": {
        "kind": "test",
        "isDefault": true
    },
    "command": "${config:python.pythonPath} -m pytest -rxXs --ds=projectname.settings.local_test --random-order"
},

Whenever possible I use pytest for running my unit tests. Normally this is run from the command line as something like pytest <name of directory containing tests>. The problem though is that pytest gets installed to a virtual environment, so how do I give the full path to the virtual env without making the task machine specific? The answer is I run it as a module and just use the config:python.pythonPath variable to reference whatever the current Python environment is. The other options are some common ones I feed to pytest, ex the --ds switch is for specifying the DJANGO_SETTINGS_MODULE environment variable. --random-order uses the Pytest Random Order plugin to run the tests in a random order on each test run (which has discovered bugs in my code/tests).

I also set a hotkey for this task:



{
    "key": "shift+cmd+f11",
    "command": "workbench.action.tasks.test"
},

This makes use of the kind property of the task definition.

Update Dependencies



{
    "label": "Update Python Dependencies",
    "type": "shell",
    "command": "${config:python.pythonPath} -m pip install -r requirements.txt --upgrade && ${config:python.pythonPath} -m pip install -r requirements-dev.txt --upgrade"
},

I still use requirements.txt files (I really should spend the time to learn pipenv, but alas). With this task I can quickly update all my project's dependencies. I also separate out my project's dependencies and my project's dev dependencies (think things like pytest or pylint) into separate files. The reason for this is that I can then let my dev dependencies "float", and most projects I work on also build a Docker image at the end of the day, so separating the dependencies allows me to only install the dependencies needed for running the project into the Docker image, which cuts down on image size.

Many Many More

This is just scratching the surface, any time I find myself commonly running commands in a terminal window on a project, I'll spend the minute or so to turn that into a VS Code task.

Lastly, one of the key points here is that I do essentially these same tasks on any project I work on and I just tweak the specific commands for the particular project. This creates a common/familiar workflow for me regardless of if it's a Django project, Flask, or even an entirely different tech (I had a Java project with a REST API and I created many of the same tasks for that).

New Tricks

As mentioned, in the January 2019 update they added a new feature related to tasks that I'm a huge fan of: Task Output Split Terminals

This allows you to have a task spawned into a split terminal window to another (already running) task. This is really handy when you have say a task for running the dev server and another task for say tailing the log file of that server as you can have them appear side-by-side in the integrated terminal.

This was particularly useful for a project I work on at work where I have a Django-based server, which speaks to another local dev server via a socket connection. Previously I had tasks set up for both of these, and I'd have to fire up each one individually, and switch between multiple terminal windows to see the output of each. Now I can have them show up side by side in one view. The way this works is by sharing the same group property in the task's presentation property:



{
    "label": "Run Server",
    "type": "shell",
    "command": "${config:python.pythonPath} manage.py runserver --noreload",
    "presentation": {
        "group": "groupServerStuff"
    }
},

All tasks with the same group will open up as another split terminal pane in the same terminal window. Very nice.

This got me to thinking though: rather than start each task individually, is there a way to have tasks "call" or "spawn" other tasks? And as it turns out there is:



{
    "label": "Run Server",
    "dependsOn": [
        "Run TCP Server",
        "Run Django Server",
        "Tail Log File"
    ]
},
{
    "label": "Run Django Server",
    "type": "shell",
    "command": "${config:python.pythonPath} manage.py runserver --noreload",
    "presentation": {
        "group": "groupServerStuff"
    }
},
{
    "label": "Run TCP Server",
    "type": "shell",
    "command": "${config:python.pythonPath} scripts/tcp_server.py",
    "presentation": {
        "group": "groupServerStuff"
    }
},
{
    "label": "Tail Log File",
    "type": "shell",
    "command": "tail -f /tmp/logfile.txt",
    "presentation": {
        "group": "groupServerStuff"
    }
},

Check out that Run Server task -- it spawns three other tasks I have defined: "Run Django Server" (which was my previous "Run Server" task), "Run TCP Server" (the simulated socket server), and "Tail Log File" which just tails the logfile that Django is logging to.

And of course, because it's called Run Server the same hotkey I defined previously will spawn up a new terminal window split 3-ways with these tasks running. All with a single keystroke. That's pretty powerful stuff!

In any case, I hope this was a useful overview of Tasks in VS Code. Have you come up with any creative uses for them? Lemme know in the comments!

Disabling Pylint Messages

Adam Parkin — Sun, 12 Aug 2018 22:20:00 +0000

This post was originally posted on Aug 12, 2018 to https://www.codependentcodr.com/disabling-pylint-messages.html#disabling-pylint-messages

Small tip of the day since I keep having to look this up. If you use
Pylint for static analysis of your code, you might find that you'll want to disable a particular rule for a particular line or file. That is, you don't want to permanently disable this rule, but just for this one special spot where the rule doesn't make sense.

I find this particularly common in unit test code as my test method names tend to be long, violating rule C0103 "Name doesn't conform to naming rules". For example, a test name might look like:

def test_message_builder_generates_correct_bytestring_when_no_argument_supplied():

Which is very self-descriptive, and when that test fails, it makes it much
easier to figure out what might've gone wrong. The problem is that Pylint will flag that line since it's greater than 30 characters long, violating the style
guidelines. We could disable this rule across the entire codebase, but outside
of tests, the rule makes sense.

This is where local disables come in, which take the form of comments:

# pylint disable=C0103
def test_message_builder_generates_correct_bytestring_when_no_argument_supplied():

This will suppress code C0103 for the remainder of the scope (module or block), or until it's re-enabled:

# pylint disable=C0103
def test_message_builder_generates_correct_bytestring_when_no_argument_supplied():


# still disabled here...

# pylint enable=C0103
def but_not_disabled_here_so_this_name_will_get_flagged_by_pylint():

You can also (and it's generally better practice) use the "verbose name" for a
particular code rather than the shorthand code. For example, this is
equivalent:

# pylint disable=invalid-name
def test_message_builder_generates_correct_bytestring_when_no_argument_supplied():

A question then becomes "how do I know what the verbose name for a code is?" And the answer is to use the --list-msgs argument to Pylint on the command line, and it'll spit them all out:

$ pylint --list-msgs
:blacklisted-name (C0102): *Black listed name "%s"*
  Used when the name is listed in the black list (unauthorized names).
:invalid-name (C0103): *%s name "%s" doesn't conform to %s*
  Used when the name doesn't conform to naming rules associated to its type
  (constant, variable, class...).
:missing-docstring (C0111): *Missing %s docstring*
  Used when a module, function, class or method has no docstring.Some special
  methods like __init__ doesn't necessary require a docstring.
:empty-docstring (C0112): *Empty %s docstring*
  Used when a module, function, class or method has an empty docstring (it would
  be too easy ;).

... more lines ...

One extra tip about Pylint: if you use Visual Studio Code you can turn on Pylint as your linter, and warnings will get put into the problems view. To turn it on set the following in your VS Code settings (assuming you've installed the Python extension from Microsoft):

"python.linting.enabled": true,
"python.linting.pylintEnabled": true,

Note that these are both on by default (at least in the current version). Once you save a Python file, Pylint warnings will show up in the problems view:

Note that that screenshot also illustrates how you can filter the problems view down to show just Pylint warnings by entering "pylint" into the search box. Also note that clicking any of those will open up that file in VS Code and navigate to the line in question. Really handy.

Docker and Image Sizes

Adam Parkin — Sun, 12 Aug 2018 02:30:00 +0000

This article was originally posted at: https://www.codependentcodr.com/docker-and-image-sizes.html on Aug 12, 2018

So this surprised me. I had a Dockerfile that looked basically like this:

FROM alpine:latest

RUN apk add --no-cache --update \
    python3 nodejs-current-npm make git curl

RUN python3 -m ensurepip
RUN pip3 install --upgrade pip

RUN npm install -g markdownlint-cli

# needed for one of the packages in requirements.txt
RUN apk add --no-cache --update python3-dev gcc build-base

COPY requirements.txt /build/requirements.txt
RUN pip3 install -r /build/requirements.txt

Building this image resulted in an image that docker images reported as 379MB insize. That's a little large so I wanted to trim.

Since those packages installed just before copyingrequirements.txt to the image were only there to be able to install a package, there's no reason for them to remain in the image. Cool, so we can turf them to save on image size:

FROM alpine:latest

RUN apk add --no-cache --update \
    python3 nodejs-current-npm make git curl

RUN python3 -m ensurepip
RUN pip3 install --upgrade pip

RUN npm install -g markdownlint-cli

# needed for one of the packages in requirements.txt
RUN apk add --no-cache --update python3-dev gcc build-base

COPY requirements.txt /build/requirements.txt
RUN pip3 install -r /build/requirements.txt

# cleanup unneeded dependencies
RUN apk del python3-dev gcc build-base

Sweet, and this resulted in an image size of 381MB, a savings of NEGATIVE 2MB. Wait.... WAT?

So I removed some stuff and ended up with an image that's a few MB's larger? How does that work?

And this is where if we want to get technical, we start talking about how Docker uses a layered filesystem and as such (not entirely unlike Git repos) once something is added to an image, it can't really (or at least easily) be removed.

See this issue which mentions what I'm talking about: https://github.com/gliderlabs/docker-alpine/issues/45

So what do we do? Well, we combine the operations into a single Docker instruction:

FROM alpine:latest

RUN apk add --no-cache --update \
    python3 nodejs-current-npm make git curl

RUN python3 -m ensurepip
RUN pip3 install --upgrade pip

RUN npm install -g markdownlint-cli

COPY requirements.txt /build/requirements.txt

RUN apk add --no-cache --update python3-dev gcc build-base && \
    pip3 install -r /build/requirements.txt && \
    apk del python3-dev gcc build-base

Because the add & the removal of the apk packages are a single Docker instruction they don't inflate the size of the built image (you can think of layers as being "checkpoints" after each instruction in a Dockerfile).

With this change my image size dropped significantly. How much? Let's let the tool tell us:

$ docker images
REPOSITORY                   TAG                 IMAGE ID            CREATED              SIZE
someimage                    combineops          1744771da3fa        About a minute ago   216MB
someimage                    removepckgs         fc5877e2afad        4 minutes ago        381MB
someimage                    original            b6e5e43b22e0        5 minutes ago        379MB

That is, it dropped from 379MB to 216MB. Not a bad savings at all.

This is a classic "time vs space" tradeoff though. Because I had to move the requirements.txt line up, that means that builds of this image are often slower (because of the way the Docker cache works, if I change the requirements.txt file then it'll have to install those apk packages any time requirements.txt changes). However, I think the savings in space (40%+) is worth it.