DEV Community: Jürgen Hermann

OpenSCAD Model With Animation Video and MakerWorld Multi-Plate Support

Jürgen Hermann — Sat, 29 Aug 2026 09:41:58 +0000

Contents

Motivation and Purpose
Using an Animation to Visualize Key Concepts
Working With the Model
- 1. Get the Files
- 2. Choose the Dimensions
- 3. Inspect the Assembly
- 4. Export the MakerWorld Plates
- 5. Attach the Hoses
Create the Model Animation
Reference Links

Motivation and Purpose

I made a parametric bayonet connector for AC hoses, to attach such hoses to a mobile AC unit and a typical window kit taking the hot air outside. It is part of my Air Conditioning Collection on MakerWorld.

And of course, I prefer code over using some GUI CAD application, leading straight to OpenSCAD as the established standard for 3D-models-as-code. It is also one of the few ways MakerWorld models can be made customizable by the end user.

The connector allows attaching a hose with a simple push and a short twist, instead of needing a threaded joint or tools every time the hose is removed.

The design has three printable parts:

A female connector with bayonet slots on the inside.
A male connector with matching lugs.
A female adapter with a wider fitting section for joining to an existing tube, like the one on an air intake cover.

The male lugs fit into the slots in the female connector. Push the parts together, twist them, and the lugs travel along the horizontal parts of the L-shaped slots. This is the same basic idea used by bayonet light fittings, camera mounts, and other quick-release connectors. See the Bayonet mount overview for useful background.

The hose itself is held on the printed connector with a worm-drive hose clamp. A screw on the clamp pulls the perforated band tight around the hose. See the Hose clamp reference for an explanation of that mechanism.

Using an Animation to Visualize Key Concepts

The OpenSCAD file contains code that creates a 10-scene animation. It shows the female connector turning to expose the slots, the hose and male connector moving into position, the male part twisting to lock, and the parts separating again. It also emphasises the parametric nature of the model by showing a change of the two main parameters, hose diameter and fitting height.

The animation is not just decoration — it makes the hidden bayonet slots easier to understand before printing the parts.

Working With the Model

1. Get the Files

Clone the things repo to get all required model files.

Install the OpenSCAD Nightly Build to view SCAD files, or alternatively add a SCAD live preview extension like Carve to VS Code.

The hose-connector-multi-plate.scad script targets some special features offered by MakerWorld, using the bespoke mw_plate_N() module names. To see this, open the MakerWorld Parametric Model Maker, where you can also download a 3MF file for your slicer.

For creating the animation video, the project also includes a subtitle file named subtitles.srt.

2. Choose the Dimensions

Open the MakerWorld Parametric Model Maker link. Wait for the model to load, then use the parameter controls to enter your measurements in millimetres:

Set hose_diameter to the inside diameter of your hose.
Set hose_connector_height to the length of the straight fitting section.
Adjust wall_thickness if you prefer a different wall size.
Adjust tolerance_gap if your printer consistently makes holes too tight or too loose.

The default values are a 150 mm hose diameter, a 45 mm fitting height, a 2.5 mm wall thickness, and a 0.15 mm tolerance gap. The diameter and length are limited by what fits into your printer's build volume.

Measure your real hose before entering the diameter value; nominal hose sizes are not always the same as their measured inside diameter. Check the preview after changing the parameters and make sure the hose can slide over the fitting by using the measuring tool in your slicer. Section 4 explains how to download the customized 3MF with all three plates.

3. Inspect the Assembly

Open hose-connector-multi-plate.scad in OpenSCAD and press Preview (F5) or Render (F6). The script starts with the parts arranged as an assembly. The file uses lower detail for previews and finer curves for final output, so the preview should be faster than a final render.

The variables animate = 0 and local = 0, set at the top of the script, fit the model maker case. Comment the line with those settings and uncomment the line below it for local use and watching the animation within OpenSCAD

The MakerWorld model creator looks out for the three modules mw_plate_1(), mw_plate_2(), and mw_plate_3(), placing each printable part on its own plate.

4. Export the MakerWorld Plates

Upload the SCAD file through the MakerWorld Parametric Model Maker link above and set your dimensions in the customizer. The generated download is a 3MF containing three plates:

Plate 1: the female hose connector.
Plate 2: the male hose connector.
Plate 3: the female adapter with the tube extension.

Open the 3MF in your slicer, check the fit and orientation, and slice the parts. The models are intended to print without supports on a printer that can handle 45° overhangs.

Do not use PLA for these parts. Use a material with better heat and UV resistance instead, such as PETG or PCTG, and choose a material and print settings suitable for the temperatures and environment around your AC equipment.

5. Attach the Hoses

Push each hose over its connector. Do not forget to fit a worm-drive hose clamp over the hose end before that, and tighten it carefully. The clamp is not included in the animation, but it is needed to secure the hose to the printed connector.

To connect the printed parts, line up the male lugs with the female slot openings, push the male connector fully home, and twist in the direction marked on the outside of the female connector housing. To remove it, twist in the opposite direction.

Create the Model Animation

The SCAD file divides the animation video into 10 scenes and uses OpenSCAD's $t animation value to progress through them, going from 0.0 to 1.0 in steps depending on the amount of requested frames. OpenSCAD documents $t, animation steps, and camera variables in its animation documentation. The OpenSCAD documentation page also links to the language reference and command-line manual.

From a terminal in the model folder, run the commands from the README. On Windows, OpenSCAD is likely not on your PATH and a working command may look like this:

"C:\Program Files\OpenSCAD (Nightly)\openscad.exe" -o frame.png --colorscheme "Nocturnal Gem" --animate 180 --camera=400,400,-300,0,0,50 hose-connector-multi-plate.scad

convert -delay 7 -loop 0 frame*.png hose-connector-animation.gif

ffmpeg -y -framerate 6 -i frame%05d.png -vf "subtitles=subtitles.srt" -c:v libx264 -pix_fmt yuv420p hose-connector-animation.mp4

The first command asks OpenSCAD for 180 animation frames and defines the camera position by specifying an eye line. The second combines the generated frames into a looping GIF using ImageMagick. The last command creates a MP4 video at six frames per second and burns in the captions using FFmpeg. On Windows, these last two commands are probably better executed in a WSL virtual machine.

On Linux or macOS, replace the Windows executable path with openscad if it is on your PATH. Recent ImageMagick installations may use magick instead of convert, so the second command may be:

magick -delay 7 -loop 0 frame*.png hose-connector-animation.gif

The frame files are temporary render output. Keep the GIF or MP4 and remove the individual PNG files when you are finished and do not need them anymore.

Reference Links

SVG Thumbnails in Windows 11 with Microsoft PowerToys

Jürgen Hermann — Sun, 27 Aug 2023 21:37:49 +0000

Install the PowerToys from the Microsoft Store so you get the Explorer extensions as shown in the video below.

For avid Inkscape users, this is a great addition and gets you the same level of convenience any Linux system offers by default. 😏 🐧

https://youtu.be/ZAGDNwk7Bd8?si=BCxTVhc04yNb8HD8

🗣 Communication Hack – Use Labelled Links

Jürgen Hermann — Tue, 31 Mar 2020 09:09:08 +0000

Do your readers a favor and use the “Copy as Markdown for Chrome & Firefox” add-on to post web links.

When you post links to some information bit on the web or your intranet, using raw technical links typically reduces information content. For example, a link featuring just a post number (…/post/0815) is purely technical, and takes up a lot of space while having almost zero information content – it just tells your audience the target site via the domain, and that is all.

🌐 🚑 Copy as Markdown for Chrome & Firefox to the rescue.

The above link was created like this:

Using that context menu item copies the link including its label (or the current selection if there is one, like shown above) as markdown text into your clipboard. Markdown is a universal format that can be pasted into lots of channels: Rocket.Chat and other web chats, blog sites like this one, many wikis (like Atlassian Confluence), etc.

In Confluence, the trick to know is the “Insert Markup” menu item in the editor (Ctrl-Shift-D), enabling you to paste the markdown text almost directly into any page – just make sure the format drop-down above the text area says markdown and not Confluence wiki markup.

So do your audience a favor, install that add-on, and in future provide them with readable links that allow them to judge whether they need to click-through on that link.

Software Development is the Metamorphosis of Caffeine into Code

Jürgen Hermann — Sun, 29 Mar 2020 20:58:39 +0000

A list of my personal coffee gear – a long answer to What type of coffee you like?

Being able to brew a decent cuppa joe at home is more essential than ever, in times without access to my company's professional WMF coffee machines – about 7.5k€ if you buy them, a little out of my price range, and see the note on cleaning further below. 🏡 🍵 😋

Let's start with a simple list of my gear, including current prices (from amazon.de).

Product	Price
Aerobie AeroPress A80	29€
Accessory: Fellow Prismo	29€
Kitchen Timer	14€
Digital Scale	14€
Kettle: Uarter 2000W	42€
Coffee Grinder: Graef CM 800	111€

The mentioned prices should give you a hint whether you want to go down this specific rabbit hole. Keep in mind that people spend a whole lot of money on their hobbies, or coffee-2-go for that matter – so just regard drinking a decent cup as your hobby, then it's all normal and not snobbish at all. 😀

The Coffee

Finding the brand of coffee you like is all up to you – taste is subjective by nature, and that also includes milk and sweeteners.

However, there are a few guiding principles: get it as fresh(ly roasted) as possible, get whole beans, grind them right before brewing your cup, and get 100% Arabica only (cheap coffee is of the Robusta variant, or a blend). Spoiled coffee beans are uneven in shape and color, and are brittle with a lot of crumbling – avoid. One kg of non-specialty coffee beans will go for around 15€ in Germany, and a little more for fair-trade brands.

The Coffee Maker

The AeroPress is hailed by many as the way for brewing single cups of coffee. It is certainly cheaper and way more sanitary than the usual fully automatic coffee machines for end-consumers, which normally develop a nice population of mildew and other gunk in their insides, unless you religiously clean them every day. For the AeroPress, you need 15 seconds of rinsing under running water.

I evolved to the AeroPress via several simple electric coffee makers, an espresso machine, a fully automatic coffee maker, back to the simple things with various french presses, to finally settle on what I use now.

As for dosage, I typically go for 25 g coffee per cup of about 400 ml beverage. With the AeroPress, you get a pretty concentrated brew, but adding more hot water after filling the cup gets you to your preferred strength. And there's a lot of other variables involved: dark or light roast, grind size, water temperature, extraction time. You have to experiment in the beginning to get to your personal favorite recipe.

Because volumetric measurement definitely torpedoes repeatability of recipes, and extraction time is quite important, you also need these two things (possibly also as scales with an integrated timer):

You'll also find plenty of videos on the net showing you how to use an AeroPress, including all the variants and special recipes people came up with.

The Fellow Prismo is an accessory to replace the original filter holder, featuring a pressure-sensitive valve and including a stainless steel filter. Using it you can make espresso-style coffee with the AeroPress, but I got mine because it is a way to use the so-called ‘inverted’ style of AeroPress recipes, without the involved hazard of juggling hot brown water around.

The Grinder

Using a good grinder is just as essential as using good coffee. You can ruin great coffee with a bad grinder, e.g. one that produces particle sizes from dust to grainy, all at the same time. And no grinder on this world can make stale or spoiled coffee beans any good.

I have the CM 80 as shown above, but the available model is the CM 800 featured in the list at the top.

If you get no workout otherwise, a very good manual burr grinder is also an option. 💪

The Kettle

I replaced my plain old steel kettle by a glass one with temperature control (in 5°C steps) and a few other gimmicks. I really like the glass, especially when filling it – no narrow gauges you can only see from one specific angle, or similar things.

Closing Thoughts

I never was a friend of coffee shops and their inherent waste of resources and, well, money. I rather invest that money and some time to drink my first coffee at home, or take it with me in a steel vacuum bottle or cup. Working for a company offering great free coffee helps too.

I can also recommend watching James Hoffmann on YouTube. Yes, he is a “coffee professional” (i.e. a barista by trade), but his tips are still quite grounded and useful for mere mortals, like a recent test of water filters. He's also a funny guy, watch his tests of various coffee drinks for a good laugh.

Photo credits: Adafruit Industries [CC BY-NC-SA 2.0]

🌠 Listing My GitHub Stars

Jürgen Hermann — Sun, 29 Mar 2020 14:01:41 +0000

starred is a Python tool to create a repository that lists all your GitHub stars by topic.

Having a list of your stars organized by topic helps you as well as others to explore that database of curated know-how from another angle than what GitHub's web interface provides. On the “Stars” tab of your profile page, you can filter by language / technology or a keyword, and sort the list by history, popularity, or activity.

The starred CLI tool gets all your stars via the GitHub API and writes them to a README file that is then committed into a dedicated repository in your account.

The end result looks like this:

This also makes it easy to search for repositories and their descriptions, by using the in-page search of your bowser (Ctrl-F).

While we're on the topic of searching for keywords in your stars, the oh-my-stars tool does just that.

But that has to wait for another post…

Image credits: John Vermette

Packaging Python Applications for Debian

Jürgen Hermann — Sat, 28 Mar 2020 05:00:00 +0000

Easily deploy any Python application in form of an ‘omnibus’ Debian packaxge, i.e. one that contains all the application's dependencies, just like in a Java WAR. A basic understanding of Debian packaging, the Linux command prompt, and Python tooling is assumed.

Introduction

In this article, I'll show how to use dh-virtualenv to create self-contained Debian packages to deploy a Python application. The resulting package is very similar to a executable JAR that you can start via java -jar, in that it contains all the moving parts except Python itself, without influencing or being influenced by version requirements of other applications. This also frees you from being restricted to the dependencies and their versions found on your target platforms, and makes porting to several different target environments easier.

The advantage of using a Debian package for deployment as opposed to the native Python tool chain is that you are less dependent on typical development tools and services, i.e. to deploy to QA or production environments you need neither Internet access nor any compiler suites (for extension packages). To achieve the same with direct use of virtualenv and pip, you'd need to have an in-house PyPI repository accessible from production networks, and also release any extension packages as wheels pre-built for the target platform. Removing and updating an application is also much easier with Debian packages.

To use dh-virtualenv, you just have to extend your existing application project with a debian subdirectory – project meta-data like pip requirements and so on will be leveraged to build the final package, i.e. common tasks are delegated to the standard Python eco-system.

Note that just like with any other form of omnibus packaging, you take over the responsibility to release security updates of the contained dependencies in a timely manner.

How does it work?

dh-virtualenv is a debhelper plugin that extends the normal Debian tool chain for package building with the ability to create a Python virtualenv (an isolated Python environment), and then wrap that into the final Debian package.

Depending on the details of the application, you often also have to provide some kind of configuration of the software itself, and possibly some means to run it as a service. This can be done in several ways:

add a debian/«pkg».install descriptor to add configuration files to the Debian package.
provide a Puppet recipe or Ansible playbook that deploys the package and integrates it into the system.
embed (default) configuration into the application's Python package (via the include_package_data option of setuptools).

All those can be combined, e.g. provide defaults via Python package data, and then add external configuration that only provides values specific to the concrete host installation.

A real-world example is the devpi supervisor ERB template that serves both the purpose of passing configuration to the application process (via command line options), and also starting and controlling that process (i.e. handle demonization and automatic startup on boot).

Installing the build tools

Unsurprisingly, you need to install dh-virtualenv to use it. Since it is architecture independant, you can choose to use a recent release offered by Debian sid whatever your build platform is.

If this is your first time to build a Debian package, you also need to add the basic tools for that:

sudo apt-get install build-essential debhelper devscripts equivs

Finally, to take advantage of the available template for easily adding an inital debian directory, install the cookiecutter tool. Note that you can opt to build packages in a Docker container instead, with only Docker as a requirement on your build host.

Packaging an example project

To add the necessary debian directory with minimal effort, you can use the dh-virtualenv-mold cookiecutter. The following commands basically repeat what the integration test script of that project does, namely instantiate a Python project and then add debianization on top of it.

To provide common defaults to cookiecutter, it makes sense to have a ~/.cookiecutterrc file similar to the one I use.

Let's first create a sample project:

mkdir -p ~/tmp/dh-venv-blog
cd ~/tmp/dh-venv-blog
cookiecutter --no-input \
    "https://github.com/borntyping/cookiecutter-pypackage-minimal.git"
cd cookiecutter_pypackage_minimal/
python3 setup.py build

You can of course also use one of your own, then just check that out instead. Next, we add the debian directory:

cookiecutter --no-input \
    "https://github.com/Springerle/dh-virtualenv-mold.git"
dch -r "" # insert proper date & distro

Using --no-input causes the template's defaults to be accepted – it avoids answering all the template's prompts. After all, this is just a demo not requiring sensible inputs. Take the time to have a look at what's in the debian directory.

You're now able to build the package and if that succeeds, print the contained meta data:

dpkg-buildpackage -uc -us -b
dpkg-deb -I ../pyvenv-foobar_*.deb

The last command should show you something like this:

Package: pyvenv-foobar
Version: 0.1.0
Architecture: amd64
Maintainer: Jürgen Hermann <jh@web.de>
Installed-Size: 12877
Pre-Depends: dpkg (>= 1.16.1), python3, python3-venv
Section: contrib/python
Priority: extra
Homepage: https://github.com/jschmoe/foobar
Description: A Python package and its dependencies packaged up 
    as DEB in an isolated virtualenv.

Finally, install the new package via dpkg -i, or upload it to a repository and use it from there with apt.

Real-world examples

These are examples of dh-virtualenv packaging for non-trivial applications:

The last two show how to integrate Python web applications into systemd, instead of using supervisor like the devpi example. The jupyterhub one also demonstrates the integration of a Python project with server-side Javascript (in a NodeJS environment).

Release of rudiments v0.4.0

Jürgen Hermann — Fri, 27 Mar 2020 20:59:04 +0000

New release of my Python package with fundamental elements for any Python project, like configuration handling.

rudiments is a Python package that collects ‘miscellaneous’ functionality that is unspecific in nature and shared among many projects. The package provides runtime support for my Springerle cookiecutters, which enable you to quickly create new Python projects that work out of the box.

The new release 0.4.0 fixes some minor problems regarding newer Click versions, and path handling under Windows. It also drops Python 2.7 support, so Python 3.5 or higher is required now.

See the rudiments documentation for details on using the contained features.

Image credits: annola on Flickr

Embedding Graphs Into Your Sphinx Documents

Jürgen Hermann — Wed, 25 Mar 2020 05:00:00 +0000

Embed GraphViz ‘dot language’ graphs into your documentation, and hot-link the nodes to any HTTP resource.

The sphinx.ext.graphviz extension allows you to directly embed GraphViz ‘dot language’ graphs into your document files. They are then rendered to PNG or SVG images, which get added to your generated HTML documentation. Using SVG allows you to hot-link your nodes to any HTTP resource.

Before use, you have to activate the extension with just a few changes to your docs/conf.py

extensions = [
    # …
    'sphinx.ext.graphviz',
]

# …

# -- GraphViz configuration ----------------------------------
graphviz_output_format = 'svg'

This is an example for what you can then add to your documentation:

As long as the nodes have a href attribute, the SVG rendering contains them and thus node labels become clickable hyperlinks.

And here's the related markup that needs to be added to one of your .rst files:

.. graphviz::
    :name: sphinx.ext.graphviz
    :caption: Sphinx and GraphViz Data Flow
    :alt: How Sphinx and GraphViz Render the Final Document
    :align: center

     digraph "sphinx-ext-graphviz" {
         size="6,4";
         rankdir="LR";
         graph [fontname="Verdana", fontsize="12"];
         node [fontname="Verdana", fontsize="12"];
         edge [fontname="Sans", fontsize="9"];

         sphinx [label="Sphinx", shape="component",
                 href="https://www.sphinx-doc.org/",
                 target="_blank"];
         dot [label="GraphViz", shape="component",
              href="https://www.graphviz.org/",
              target="_blank"];
         docs [label="Docs (.rst)", shape="folder",
               fillcolor=green, style=filled];
         svg_file [label="SVG Image", shape="note", fontcolor=white,
                   fillcolor="#3333ff", style=filled];
         html_files [label="HTML Files", shape="folder",
                     fillcolor=yellow, style=filled];

         docs -> sphinx [label=" parse "];
         sphinx -> dot [label=" call ", style=dashed, arrowhead=none];
         dot -> svg_file [label=" draw "];
         sphinx -> html_files [label=" render "];
         svg_file -> html_files [style=dashed];
     }

For all this to work, you need the GraphViz suite of tools installed on the machine that renders the documentation.

A JupyterHub Showcase: DevOps Intelligence

Jürgen Hermann — Sun, 22 Mar 2020 06:00:00 +0000

Originally published in 2019 as DevOps Intelligence in the Jupyter Blog.

DevOps Intelligence turns data from software development and delivery processes into actionable insight, just like BI does for the business side. Jupyter is the ideal instrument for that, with its combination of powerful coding environments and a user interface facilitating experimentation with ultra-short feedback cycles.

A Jupyter-based setup supports risk analysis and decision making within development and operations processes – typical business intelligence / data science procedures can be applied to the ‘business of making and running software’. The idea is to create feedback loops, and facilitate human decision making by automatically providing reliable input in form of up-to-date facts. After all development is our business — so let's have KPIs for developing, releasing, and operating software.

Typical Use-Cases

Here are some obvious application areas where data analysis can be helpful on the technical side.

Migration processes of all kinds (current state, progress tracking, achievement of objectives)
Inventory reporting for increased transparency and support of operational decisions
Automate internal reporting processes to free up scarce assets and human expertise

Platform Architecture

A simple JupyterHub setup can enable you to do analysis on your already available but under-used and hardly understood data, without any great investment of effort or capital. By adding a single JupyterHub host, you can use the built-in Python3 kernel to access existing internal data sources.

The following diagram shows what role JupyterHub can play in an existing environment.

To make such a deployment easy, the 1and1/debianized-jupyterhub project provides a JupyterHub service including a fully equipped Python3 kernel as a single Debian package – only Python3, NodeJS, and Chromium packages must be installed in addition to the jupyterhub one. If you raised an eyebrow on Chromium being in that list, it's used by JavaScript-based visualization frameworks to render PNG images.

Including a NginX-powered SSL off-loader, the complete setup can be done in under an hour.

Use-Case: Migration Reporting

At the time of this writing (early 2019), a widespread challenge is migration from Oracle Java to other vendors, and also to start migration from Java 8 to newer versions (Java 11). If you do that at scale across many machines and teams, you definitely need some kind of governance, and constant feedback on the current status and the rate of progress.

What follows is an excerpt from a productive notebook, with anonymized data about AdoptOpenJDK deployments. That data was originally retrieved from a system called “Patch Management Reporting”, which collects information about installed packages for all hosts in the data center. We're in the yellow “Data Sources” box of the above figure here.

First off, we read the data and show the value sets of categorical columns, plus a sample.

import numpy as np
import pandas as pd

raw_data = pd.read_csv("../_data/cmdb-aoj.csv", sep=',')
print('♯ of Records: {}\n'.format(len(raw_data)))

for name in raw_data.columns[1:]:
    if not name.startswith('Last '):
        print(name, '=', list(sorted(set(raw_data[name].fillna('')))))

print(); print(raw_data.head(3).transpose())

♯ of Records: 104

Distribution = ['Debian 8.10', 'Debian 8.11', 'Debian 8.6', 'Debian 8.9',
                'Debian 9.6', 'Debian 9.7', 'Debian 9.8']
Architecture = ['amd64']
Environment = ['--', 'DEV', 'LIVE', 'QA']
Team = ['Team Blue', 'Team Green', 'Team Red', 'Team Yellow']
Installed version = ['11.0.2.9-83(amd64)', '11.0.2.9-85(amd64)', '8.202.b08-66(amd64)',
                     '8.202.b08-83(amd64)', '8.202.b08-85(amd64)']

                                    0                   1                   2
CMDB_Id                     108380195           298205230           220678839
Distribution              Debian 8.11          Debian 9.6         Debian 8.11
Architecture                    amd64               amd64               amd64
Environment                       DEV                  --                 DEV
Team                         Team Red            Team Red            Team Red
Last seen            2019-03-18 06:42    2019-03-18 06:42    2019-03-18 06:42
Last modified        2019-03-18 06:42    2019-03-18 06:42    2019-03-18 06:42
Installed version  11.0.2.9-83(amd64)  11.0.2.9-83(amd64)  11.0.2.9-83(amd64)

Next comes the usual data cleanup. The Distribution column is a bit diverse, and not everyone has Debian codenames and associated major versions memorized. The map_distro function fixes that.

def map_distro(name):
    """Helper to create canonical OS names."""
    return (name.split('.', 1)[0]
        .replace('Debian 7', 'wheezy')
        .replace('Debian 8', 'jessie')
        .replace('Debian 9', 'stretch')
        .replace('Debian 10', 'buster')
        .replace('squeeze', 'Squeeze [6]')
        .replace('wheezy', 'Wheezy [7]')
        .replace('jessie', 'Jessie [8]')
        .replace('stretch', 'Stretch [9]')
        .replace('buster', 'Buster [10]')
    )

Together with other cleanup steps, the mapper function is applied in a dfply pipeline. The result can be controlled by showing a sample of data points with unique version numbers.

from dfply import *

cleaned = (raw_data
    >> mutate(Version=X['Installed version'].str.split('[()]', 1, expand=True)[0])
    >> mutate(Environment=X.Environment
              .fillna('--').str.replace('--', 'N/A').str.upper())
    >> mutate(Distribution=X.Distribution.apply(map_distro))
    >> drop(X.CMDB_Id, X['Last seen'], X['Last modified'], X['Installed version'])
)

print((cleaned >> distinct(X.Version)).transpose())

                       0            13            14            62  \
Distribution   Jessie [8]  Stretch [9]    Jessie [8]   Stretch [9]
Architecture        amd64        amd64         amd64         amd64
Environment           DEV          N/A           DEV           DEV
Team             Team Red    Team Blue      Team Red     Team Blue
Version       11.0.2.9-83  11.0.2.9-85  8.202.b08-83  8.202.b08-85

                        68
Distribution    Jessie [8]
Architecture         amd64
Environment            DEV
Team             Team Blue
Version       8.202.b08-66

Time to present the cleaned up data, starting with a table of teams and their number of installed packages. In the production notebook, an API of the corporate identity management is used to enrich the table with contact data of the team leads. Having the organizational data available also makes it possible to filter or aggregate the data by business units.

counts = cleaned.groupby(['Team']).size()
print(counts.reset_index(name='Count'))

          Team  Count
0    Team Blue     42
1   Team Green     16
2     Team Red     45
3  Team Yellow      1

To create a heatmap of how diverse a team's version spectrum is, we calculate percentages of versions per team.

percentage = cleaned.groupby(['Team', 'Version']).size().reset_index(name='Count')
percentage = percentage.assign(Percent=
    percentage.apply(lambda x: 100.0 * x[-1] / counts[x[0]], axis=1))
print(percentage.head(3))

        Team       Version  Count    Percent
0  Team Blue   11.0.2.9-85      1   2.380952
1  Team Blue  8.202.b08-66      1   2.380952
2  Team Blue  8.202.b08-85     40  95.238095

HoloViews makes creating the heatmap including a label overlay a breeze.

import holoviews as hv
hv.extension('bokeh')
publish = 1  # publishing or interactive mode?

heatmap = hv.HeatMap(percentage[['Version', 'Team', 'Percent', 'Count']]).opts(
    hv.opts.HeatMap(
        title='Version Distribution by Team',
        width=480, xrotation=25,
        zlim=(0, 100), cmap='kbc_r', clipping_colors=dict(NaN='#ffffe0'),
        colorbar=True, tools=['hover'], toolbar=None if publish else 'right',
    )
).sort()

label_dimension = hv.Dimension('Percent', value_format=lambda x: '%.1f' % x)
labels = hv.Labels(heatmap, vdims=label_dimension).opts(
    hv.opts.Labels(
        text_color='Percent',
        text_font_size='10pt',
        text_font_style='bold',
    )
)

chart = heatmap * labels

if publish:
    import time
    import phantomjs_bin
    from IPython.display import HTML, clear_output

    %env BOKEH_PHANTOMJS_PATH={phantomjs_bin.executable_path}
    chart_img = 'img/devops/aoj-heatmap.png'
    hv.save(chart, chart_img)
    chart = HTML('<img src="{}?{}"></img>'.format(chart_img, time.time()))
    clear_output()

chart

From the heatmap, you can easily glance whether a team uses predominantly one version, and how recent the used versions are.

Conclusion

Using a platform powered by Project Jupyter and a big chunk of the scientific Python stack lets you easily mold your data into the shape you need, and then choose from a wide range of visualization options to bring your message across.

👍 Image credits: Devops-toolchain

Using Python3's ‘venv’ with tox

Jürgen Hermann — Sat, 21 Mar 2020 22:27:28 +0000

Using the built-in Python module in favor of virtualenv to create your testing or project automation environments.

tox is a generic virtualenv management and test command line tool, especially useful for multi-environment testing. It has a plugin architecture, with plenty of both built-in and 3rd party extensions.

This post assumes you are already familiar with tox and have a working configuration for it. If not, check out its documentation.

In order to make tox use the built-in virtual environment venv of Python 3.3+, there is a plugin named tox-venv that switches from using virtualenv to venv whenever it is available.

tox-dev / tox-venv

Use Python 3 venvs for Python 3 test environments

Typically, venv is more robust when faced with ever-changing runtime environments and versions of related tooling (pip, setuptools, …).

To enable that plugin, add this to your tox.ini:

[tox]
requires = tox-venv

That merely triggers tox to check (on startup) that the plugin is installed. You still have to add it to your dev-requirements.txt or a similar file, so it gets installed together with tox. You can also install tox globally using dephell jail install tox tox-venv – see the first post in this series for details.

The end result is this (call tox -v to see those messages):

py38 create: …/.tox/py38
  …/.tox$ /usr/bin/python3.8 -m venv py38 >…/log/py38-0.log

And there you have it, no more virtualenv package needed. 🎉 🎊

Add level setting to dashboard

Jürgen Hermann — Sat, 21 Mar 2020 11:42:43 +0000

On the dashboard, in the meta data summary box after the "Manage" button, please include the level setting. Otherwise it is tedious to impossible to maintain them after the fact, or get an idea whether your level settings are consistent across posts.

Sort Order of Posts in a Series?

Jürgen Hermann — Sat, 21 Mar 2020 10:37:28 +0000

I'm a bit surprised about the (in a timeline) 2nd post Blogging About My Repositories to Guide Interest being the first in the generated list of posts. Any hints to fix this?

Maybe related to the original date of posts at the source? If that, it might be a worthwhile feature to allow explicit ordering (series: #«PRIO» «TITLE», with prio defaulting to the UNIX timestamp used currently), or else sort by order of publishing on dev.to instead of any sources.