DEV Community: stereobooster

The test pyramid is a fiction until you define 'unit'

stereobooster — Wed, 04 Nov 2026 21:19:14 +0000

The test pyramid says: write mostly unit tests, fewer integration tests, a few end-to-end tests. That instruction means something only if "unit test" does. It has at least five definitions in circulation, and they disagree. Count one suite under two of them and you can get a pyramid and a diamond from the same tests.

So "do we follow the pyramid?" has no answer until a team picks a definition. And the problem the pyramid was meant to solve, a slow and flaky suite, doesn't need the pyramid at all. You can see it directly, and fix it directly.

Five definitions of "unit test"

1. A test of one small piece of the design. IEEE's glossary defines unit testing as "Testing of individual hardware or software units or groups of related units", and admits in a note that the relationship between "module", "component" and "unit" "is not yet standardized"¹. Software Engineering at Google says "Units can be as small as an individual function or as broad as a set of several related packages/modules"².

2. A test with every collaborator replaced by a double. Martin Fowler calls these solitary tests: "mockists insist upon solitary unit tests, while classicists prefer sociable tests"³.

3. A test that touches nothing outside its process. Michael Feathers: "A test is not a unit test if: It talks to the database / It communicates across the network / It touches the file system / It can't run at the same time as any of your other unit tests / You have to do special things to your environment (such as editing config files) to run it"⁴. Google's testing blog: "A Small test equates neatly to a unit test"⁵.

4. A test that no other test can affect. Kent Beck's draft of Test-Driven Development: By Example says "tests should be able to ignore each other completely"⁶. Fowler recalls that "when xunit testing began in the 90's we made no attempt to go solitary unless communicating with the collaborators was awkward"³.

5. A test of one behaviour through a public interface. Roy Osherove's unit is "a single logical functional use case in the system that can be invoked by some public interface (in most cases)"⁷. Ian Cooper: "the unit of isolation is the test not the thing under test"⁸.

Even ISTQB, the body behind the most common tester certification, hasn't settled it. Its syllabus says "Component testing (also known as unit testing)"⁹. Its glossary dropped "unit testing" as a synonym in August 2025, then brought it back in April 2026 as a separate term: "the smallest part of code testable in isolation"¹⁰, without saying what isolation means.

Fowler's verdict is that the term is "very ill-defined", and "really it's a situational thing - the team decides what makes sense to be a unit"³. That's reasonable advice. The trouble starts when the team never decides.

The same test, different layers

Take a few ordinary tests and file them under each definition:

Test	Small piece	All doubles	No I/O	Independent	One behaviour
A pure function, called directly	unit	unit	unit	unit	unit
A class with its real in-memory collaborators	depends	integration	unit	unit	unit
A class with every collaborator mocked	unit	unit	unit	unit	arguably not
An HTTP handler, only the database faked	integration	integration	unit	unit	unit
A function that writes a temp file	unit	?	integration	unit	unit

Only the pure function is a unit test under every definition. Everything else moves between layers depending on who files it, and nothing in the code changes.

One suite, two shapes

Fabian Trautsch and Jens Grabowski took 10 open-source Python projects and classified their tests against the IEEE and ISTQB definitions. Their conclusion: "developers believe that they are developing more unit tests than they actually do"¹¹. Depending on the project and the definition, the share of real unit tests ranged from
0% to 68.30%.

aws-cli shows how much the definition matters. Its developers labelled about 100 tests per revision as unit tests. Under IEEE's definition about 60% of those qualified; under ISTQB's, about 3%¹¹. The tests are the same in both counts; only the definition changes.

The tests that don't qualify don't disappear. The study only says they aren't unit tests, but tests written as unit tests rarely drive the whole system end to end, so they belong in the middle layer. Under IEEE's definition, aws-cli keeps a wide base. Under ISTQB's, it has a thin base and a wide middle. Add any end-to-end tests on top and the same suite is a pyramid under one standard and a diamond under the other.

Neither drawing is wrong. The pyramid simply has no shape until someone says what goes in the bottom layer, and most teams never do.

You don't need the shape to find slow tests

The pyramid's case was economic: unit tests are fast and reliable, so have many of them. But speed is not something you infer from a diagram. Every test runner reports how long each test takes. Sort by duration and the slow tests are at the top, whatever folder they live in and whichever definition they would satisfy.

Look at why they're slow and the answer is almost never "it covers too much code". It's what the test touches: a real database, the network, the disk, a sleep. And tests are flaky for a related reason: they share something, a database, a file, a global, the clock, so the result depends on what ran before or alongside them.

Fix it directly

Two of the five definitions are really engineering rules, and they're the two that matter for speed:

Make every test independent (definition 4). A test that sets up its own state and shares nothing can run in any order and in parallel. That alone removes a whole class of flakiness and lets you add cores instead of waiting.
Stub the slow parts (definition 3). Replace the database, the network and the clock with in-process fakes where the test isn't about them. Fake the expensive edge, not every collaborator.

Neither rule says how much code a test may cover. An HTTP handler test with the database faked runs in one process, in milliseconds, in parallel with everything else. Under two of the five definitions it's an integration test; for speed it doesn't matter. Cooper makes the same point about TDD: "it's the isolation of the test that is driving
that not the isolation of the code under test"⁸.

The other argument for small tests is that a failure points at the bug. Trautsch's thesis found that in the 27 projects it studied: "unit tests are better able to pinpoint the source of a defect"¹². How much that matters depends on the application, not on the test. If an error surfaces with a clear stack trace, nothing swallows it, there isn't much indirection between the call and the fault, and you can run the test locally under a debugger, a failing integration test is quick to trace. If errors get caught and re-raised as something generic, small tests do most of the pointing. Either way it's a question of debuggability, not of counting layers. The follow-up study found no difference in what the two kinds of test catch: "We could not find any evidence that one test type is more capable of detecting certain defect
types than the other one"¹³.

So keep the pyramid if your team has defined "unit" and finds the picture useful. If it hasn't, skip the argument about the shape. Measure which tests are slow and flaky, make them independent, and stub what makes them slow.

I keep a knowledge base on software quality where every claim has a source. Terminology maps unit, integration and end-to-end onto Google's scope and size. The axes covers what the pyramid doesn't ask: where the inputs come from and what decides pass
or fail.

IEEE Standard Glossary of Software Engineering Terminology, IEEE Std 610.12-1990, entries "unit testing" and "unit". ↩
Titus Winters, Tom Manshreck, Hyrum Wright (eds.), Software Engineering at Google, O'Reilly, 2020, chapter 12, "Unit Testing". ↩
Martin Fowler, "Unit Test", bliki, 2014 (revised 2017). ↩
Michael Feathers, "A Set of Unit Testing Rules", Artima, 2005. ↩
Simon Stewart, "Test Sizes", Google Testing Blog, 2010. ↩
Kent Beck, Test-Driven Development: By Example, "Isolated Test" (pre-publication draft, 2002; the printed edition may word it differently). ↩
Roy Osherove, "Unit test: a definition", The Art of Unit Testing companion site. ↩
Ian Cooper, TDD, Where Did It All Go Wrong?, talk, DevTernity 2017 (first given 2013); quoted from the recording. ↩
ISTQB, Certified Tester Foundation Level Syllabus v4.0.1, 2024, §2.2.1. ↩
ISTQB Glossary, "unit testing", read November 2026. ↩
Fabian Trautsch, Jens Grabowski, "Are there any Unit Tests? An Empirical Study on Unit Testing in Open Source Python Projects", ICST 2017, doi:10.1109/ICST.2017.26. ↩
Fabian Trautsch, "An Analysis of the Differences between Unit and Integration Tests", PhD thesis, University of Göttingen, 2019, doi:10.53846/goediss-7393. ↩
Fabian Trautsch, Steffen Herbold, Jens Grabowski, "Are Unit and Integration Test Definitions Still Valid for Modern Java Projects? An Empirical Study on Open-Source Projects", Journal of Systems and Software 159, 2020, doi:10.1016/j.jss.2019.110421. ↩

Run Cystoscape.js with Node.js

stereobooster — Thu, 12 Oct 2023 16:33:48 +0000

Why? If you need to generate an image on the server side.

At first, I thought it was impossible. There is cytosnap that uses Puppeteer (headless browser). Then, for fun, I rewrote it with Tauri.

But later, I though, what if it would be possible to run it with Node.js. And, indeed, it’s possible.

Solution

Add polyfils for the browser (it also includes canvas):

class XMLSerializer {
 serializeToString(node) {
 return serialize(node);
 }
}
global.XMLSerializer = XMLSerializer;

const dom = new JSDOM(`<!DOCTYPE html><div id="cy"></div>`);
global.window = dom.window;
global.document = dom.window.document;

Explicitly set bounding box

source.layout.boundingBox = {
 x1: 0,
 y1: 0,
 x2: source.width,
 y2: source.height,
};

Generate a graph the same way you would in the browser:

const container = dom.window.document.querySelector("#cy");
const cy = cytoscape({ container, ...defaults, ...source });
cy.layout(source.layout).run();

For now I was able to render it only as SVG:

await loadExtension("svg");
res = cy.svg({
 bg: source.background,
 full: true,
});

In order to not polute global scope with polyfills we can run this script as sub-process:

const executablePath = `bin/cyto-nodejs.js`;
const bin = spawn(executablePath, args, {
 windowsHide: true,
});

That’s it.

Source code

Source code is here: https://github.com/stereobooster/cyto-nodejs

Hugo ideal image

stereobooster — Tue, 10 Oct 2023 23:14:33 +0000

I created image components a couple of times before:

But browsers keep improving - now almost all modern browsers support:

I checked existing solutions - they either don’t do what I want or are complicated (for my taste):

Features

What I expect from an “ideal” image:

Responsiveness
Lazy-loading
Modern LQIP
Portable markdown links
Dark mode

Implementation

There are different ways to implement image components in Hugo:

I think the most pragmatic way would be to implement it as “partial” which accepts an image as a resource. This way it can be reused in render hooks and shortcodes, and the resolution of the image would be the responsibility of the caller.

It can look something like this:

{{ partial "picture.html" (dict "img" $img "alt" $.Text) }}

Partial picture.html would be responsible for: detecting width and height, resizing the image, and rendering HTML. There are no special requirements for HTML or CSS so it would be a pretty portable solution (copy one file).

Responsiveness

Component would be responsible for resizing image and rendering srcset attribute. Basic code:

srcset="
{{- (.Resize "330x").RelPermalink }} 330w,
{{- (.Resize "660x").RelPermalink }} 660w,
{{- (.Resize "1024x").RelPermalink }} 1024w,
{{- (.Resize "1320x").RelPermalink }} 2x"
src="{{ (.Resize "660x").RelPermalink }}"

But :

it suppose to resize only raster images (not SVG)
it should not upscale image

Additionally we can provide webp version of the image. Basic code:

<picture>
<source
srcset="
{{- (.Resize "330x webp").RelPermalink }} 330w,
{{- (.Resize "660x webp").RelPermalink }} 660w,
{{- (.Resize "1024x webp").RelPermalink }} 1024w,
{{- (.Resize "1320x webp").RelPermalink }} 2x"
src="{{ (.Resize "660x webp").RelPermalink }}"
/>
<img ... />
</picture>

Component only responsible for HTML part. You probably would need to add some CSS, for example:

img {
 max-width: 100%;
 height: auto;
}

Lazy-loading

In order to use lazy-load it is enough to add:

loading="lazy"

and optionaly:

decoding="async"

But :

it is important to set width and height of the image, to prevent reflow - browser would know ratio of the image upfront and will reserve space for it without downloading it

Which is pretty easy to do with Hugo

width="{{ .Width }}" height="{{ .Height }}"

But :

it works only for raster images

For SVG we need to do something like this:

{{ if (eq .MediaType.SubType "svg") }}
{{ $width := ""}}
{{ $height := ""}}
{{ $svgContent := .Content }}
{{ range (findRESubmatch `<svg[^>]*width=["']([.0-9]*)["'a-zA-Z]` $svgContent 1) }}
{{ $width = index . 1 }}
{{ end }}
{{ range (findRESubmatch `<svg[^>]*height=["']([.0-9]*)["'a-zA-Z]` $svgContent 1) }}
{{ $height = index . 1 }}
{{ end }}
{{ if (eq "" $width $height) }}
{{ range (findRESubmatch `<svg[^>]*viewBox=["']?([.0-9]*) ([.0-9]*) ([.0-9]*) ([.0-9]*)` $svgContent 1) }}
{{ $width = index . 3 }}
{{ $height = index . 4 }}
{{ end }}
{{ end }}
{{ if (eq "" $width $height) }}
{{ warnf "Can't detect width and height for SVG %s" .RelPermalink }}
{{/* do not use lazy without dimensions */}}
{{ $lazy = false }}
{{ end }}
{{ end }}

This code reads SVG as text and uses regular expression in order to match width, height, and viewBox. Important : it should not match values with %, because they are useless in this case. On the other hand - any absolute units can be matched, like px.

LQIP

There are many LQIP approaches. I like, so called, modern LQIP, which uses 20px, 20% quality webp image as base64.

<picture
{{ if $lqip }}
{{ $bg := (.Resize "20x webp q20").Content | base64Encode }}
style="background-image:url(data:image/webp;base64,{{ $bg }});background-size:cover"
{{ end }}>

I already introduced picture in the markup, so I will reuse it as “holder” for LQIP.

But :

for transparent images (png, webp) LQIP may be problematic, so there suppose to be a way to disable it
I don’t want to generate LQIP for SVG, because I would need to rasterize SVG first, which is too much trouble

Additionaly you may want to add CSS in order to make picture of the same size as image, for example:

picture {
 display: block;
}

To improve appearance it is recomended to add blur:

picture > img {
 -webkit-backdrop-filter: blur(10px);
 backdrop-filter: blur(10px);
}

Image is on “top” of picture. While it is loading it will be transparent. So we can use it add blur to the background. Important : this trick doesn’t work in Firefox, see Bug 1556156.

Portable markdown links

This one is easy. Partial take resource rather than path. So it is responsibility of caller to resolve path.

Dark mode

I’m not sure yet how to implement “dark mode” for images. From an HTML point of view it is trivial to implement:

<picture>
 <source media="(prefers-color-scheme: dark)" srcset="..." />
 <source media="(prefers-color-scheme: light)" srcset="..." />
 <img src="..." />
</picture>

But how to do it from Markdown’s point of view? Some options are:

convention. If there is an image img.png and img_dark.png, the system can automatically find the second image and use it for dark mode
shortcode. Will work, but this is not a markdown solution
Use some kind of marker in the url to distinguis dark-/light-mode images. For example, #gh-dark-mode-only or #gh-light-mode-only. See Specifying the theme an image is shown to
SVGs don’t need a special solution, they can handle it internally. See Creating SVG that appears black in light mode and light in dark mode
for black and white images filter: invert(1) would do the trick

Other

I want to keep number of params minimal. Currently it supports only:

img - image as resource
alt - string, alternative text
class - string, for styling
lazy - boolean, whether it should use lazy loading or not
webp - boolean, whether it should generate webp version or not
lqip - boolean, whether it should generate LQIP or not

Source code

My current implementation is here:

Tauri instead of Puppeteer or Playwright?

stereobooster — Tue, 03 Oct 2023 11:14:09 +0000

Tauri and Puppeteer/Playwright have different use cases. So it seems nothing to compare. But there is (at least) one use case where you may try to use Tuari instead of a headless browser - “snapshots”. For example:

react-snap - HTML snapshot, uses Puppeteer
mermaid-cli - SVG/image snapshot, uses Puppeteer
cytosnap - image snapshot, uses Puppeteer

Why? Because Tauri uses provided by OS browser and produces small binaries (let’s say ~10 MB). While Puppeteer and Playwright need a full browser (let’s say ~200 MB). And Tauri probably can be faster.

Experiment

Let’s see if this will work in practice. For an experiment, I decided to recreate cytosnap with Tauri. Functionality is trivial:

read input from file or STDIN (Elements JSON)
generate a graph with the help of Cytoscape.js
write result (image from canvas) to file or STDOUT
and this is supposed to be CLI, not GUI-app

CLI

CLI applications are not a typical use case for Tauri, but it is possible.

Hide window (tauri.conf.json):

"windows": [{ "visible": false, ...

Hide icon from Dock bar (Mac OS only):

#[cfg(target_os = "macos")]
use tauri::ActivationPolicy;

tauri::Builder::default()
    .setup(|app| {
        #[cfg(target_os = "macos")]
        app.set_activation_policy(ActivationPolicy::Accessory);
        Ok(())
    })

Add CLI arguments (tauri.conf.json):

 "cli": {
      "description": "Render graphs on the server side with Cytoscape.js, getting image file as output",
      "args": [
        {
          "name": "source",
          "short": "s",
          "takesValue": true,
          "multiple": false,
          "multipleOccurrences": false
        },
        {
          "name": "destination",
          "short": "d",
          "takesValue": true,
          "multiple": false,
          "multipleOccurrences": false
        }
      ],
    },

Overcome security fences

Because Tauri’s main use case is GUI, they care about security and limit what files/folders the application can access. Which doesn’t work for the way people use CLI. So I had to write my own Tauri commands to write and read files, for example:

#[tauri::command]
fn write_destination(dst: String, res: String) -> Result<String, String> {
    if dst == "" {
        print!("{}", res);
        Ok(String::from("ok"))
    } else {
        let bytes = base64::engine::general_purpose::STANDARD.decode(res);
        let path = absolute_path(PathBuf::from(dst)).unwrap();
        match bytes {
            std::result::Result::Ok(v) => {
                std::fs::write(path, v).unwrap();
                Ok(String::from("ok"))
            }
            std::result::Result::Err(e) => Err(e.to_string()),
        }
    }
}

Note : I need to pass binary data from the front end to the rust, so I use Base64.

Distribution

Initially, I wanted to distribute this CLI as binary inside the npm package. But then I realized that Tauri can’t really produce portable binaries. Tauri relies on the OS’s browser - which is a neat trick to shrink down the size of the binary, but this is what makes it less portable. Trade-offs as always.

Not to stop an experiment I decided to distribute at least binary for Mac OS in npm. And it works (Mac OS only):

npx @stereobooster/cyto-snap -s g2.json -d g2.png

Size of the npm package

npm notice package size: 2.9 MB
npm notice unpacked size: 7.4 MB

I didn’t do any Tauri optimizations, so it probably can be smaller. Also, this is Mac OS-only binary, it will be bigger if we pack binaries for all platforms in one npm package.

And it has 0 dependencies.

Source code

Warning : because this is an experiment, I didn’t try to make it perfect. Just made it work.

Source code: https://github.com/stereobooster/cyto-snap

Next steps

Homebrew

The idea of distributing binary in npm failed, so I think to try to distribute it via Homebrew. Homebrew works on Mac OS, Windows (WSL 2), and Linux (though they have their own package managers).

Produce standard desktop installers for Tauri (tauri-apps/tauri-action)
Upload installers to GitHub releases
Run installers in silent mode with the Homebrew formula

Full automation with GitHub Actions

It has a lot of steps to produce the final package. I started automation but didn’t finish it. Ideally, it should:

Build application
Run tests (I do integration tests with odiff)
Create a tag and push it
Create release and upload all binaries
Update the Homebrew formula and publish it
Publish the npm package (but it is not very useful without portable binaries)

Conclusion

This approach may work with Homebrew distribution
- It is sad that you can’t publish binaries to npm, which would make JS-developer use other means to install it (Homebrew, apt-get, cURL, etc)
- On the other hand, it is fully independent of npm (and Node.js ecosystem in general), so can be used by none-JS developers
This was my first time using Tauri, and it is awesome

Distributing executable binaries in npm package

stereobooster — Sun, 01 Oct 2023 09:28:23 +0000

Let’s say you need to distribute binary in npm. I’m not talking about node extensions (node-gyp, napi, etc.). I’m talking about standalone executables, like CLI applications.

We have options. How to distribute binary :

host executables on some server, for example, GitHub releases, and download appropriate (e.g. for given OS and architecture) binary on installation
put all executables (for all OSs and architectures) in one npm package
create sub-packages with binary for each OS and architecture and one “root” npm package which would list all sub-packages as optional dependencies

How to run the binary :

Use JS wrapper which would call actual executable with exec, spawn, etc.
On installation figure out where Node puts binaries and overwrite it with your binary from the package
Use JS script to call node binding (*.node) or WASM

Tools and blog posts :

name	distribution	execution
binary-install	1	1 (`spawnSync`)
binwrap	1	1 (`spawn`)
bin-wrapper	1	1 (`spawn`, …)
prebuild-install	1	3
blog.xendit.enginee	2	2
hspak.dev	2	2
prebuildify	2	3
blog.orhun.dev	3	1 (`spawnSync`)
napi-rs	3	3

Example projects :

name	distribution	execution
odiff	2	2
@astrojs/compiler	2 (WASM)	3
esbuild	3	1 (`execFileSync`)
biome	3	1 (`spawnSync`)
Turborepo	3	1 (`execFileSync`)
lightningcss	3	3

Useful modern tools for static websites

stereobooster — Tue, 26 Sep 2023 15:50:02 +0000

This list is about tools beyond static hosting and static website generators. There are already enough list about that.

Analytics

In old times there was basically one default choice: Google analytics. What is bad about GA:

JS size (about 49kb). There alternative clients though, like this one
Blocked by default by some browsers and adblockers
no GDPR compliance (e.g. you always need to provide cookies banner)

But there are a lot of better alternatives (just to name few):

https://tinyanalytics.io/ - 1 site for free
https://umami.is/ - 3 sites for free
https://swetrix.com/ - open source, starts from $5/month

See more alternatives here:

Also some “static hostings” provide analytics, for example:

Cloudflare Pages there is a free tier
Netlify $9/month

Search

In old times people would use “Google custom search engine” (I can’t find it anymore, I guess, they discounted it). There is as well Algolia. But who needs third party paid solution, when you can do it for free.

Recent trend is to generate search index during site generation and then use client side library to do the search. It will probably won’t scale for a huge website, but for personal blog more than enough.

Some options (not a full list):

Honorable mentions:

typesense - open-source, but needs server
docsearch - free Algolia search for developer docs

Or you can do something crazy, like hosting SQLite database over HTTP.

Comments

In old times people would use Disqus. But again, it is not GDPR compliant out of the box and there is huge JS payload (at least it was, when I used it last time).

One of potential solution for developers is to reuse github comments or discussions as comments for your website. Most developers have github account anyway (I know there are people, who are strongly opposed to Github and prefer Gitlab or self-hosted solutions, but this is discussion for another time). For example:

Other option is not to host comments directly on your website, but put a link to a place where people can discuss (especially if you crosspost), for example: hacker news, dev.to, reddit, twitter etc.

Runtime type validators

stereobooster — Sun, 24 Sep 2023 12:00:00 +0000

Where can be used

validate http request from client (at server side)
validate http response from server (at client side)
validate data from untyped storage, for example localStorage, JSON columns in DB, etc.
validate env variables
validate CLI arguments
validate configuration files
validate data from URL (query params)

Features

Basic validation - can validate static types at runtime (can be restricted to JSON type)
Serialisation / deserialisation - for example, TS/JS supports Date, but JSON doesn’t. So you need to do conversion in order to support it
Compatibility with other schemas - there are tools which can convert from- or to- another schemas, for example JSONSchema, OpenAPI etc
Strict or permissive mode - would object be accepted if it contains extra fields
Helpful error messages - if in case of failed validation it points what exactly went wrong or simply dismisses the input
AOT (ahead of time) - can it use type information in order to produce validation libraries or faster serialiser, deserialiser
Custom types - can user create custom types, for example type for email, instead of string

Libraries

Not a full list, but rather a high level overview

Fast

typia
- Doesn’t need extra schema definition. Uses AOT to generate validators, serialisers and deserialisers directly from TypeScript types
- Supports protocol buffers
typebox
- Supports all JSON, JS, JSONSchema, but for TS discriminated unions you need to use typebox-validators
- Can use JIT (and AOT?)
ts-runtime-checks
- Uses AOT

See: Runtype Benchmarks

Other

type-o-rama

Text to diagram

stereobooster — Sat, 23 Sep 2023 14:00:00 +0000

aka diagrams as code

I dreamt about it for a long time. Idea is to allow to express diagrams with a text, this way you can draw diagram while you are writing you markdown file. Without need to switch to another tool. And this diagram becomes part of the document and can be stored in git.

Now when Hugo supports render hooks for code blocks and in MDX you can do this too (to use with Astro, etc.) it’s time to revisit the subject.

Tools

Names	Type	Integration	Formats	Supported diagrams
mermaid	Declarative	browser*	SVG	Flowchart, UML, ER, Gant…
graphviz	Declarative	CLI, library	SVG, raster	Graphs (flowchart, network, etc)
d2lang	Declarative	CLI (go)	SVG, raster…	Flowchart, UML, ER…
pikchr	?	CLI, library	SVG	Kind of anything
Diagon	Declarative	CLI, library	ASCII-art	Flowchart, Sequence…
PlantUML	Declarative	CLI	SVG, raster, ASCII-art	UML…
gnuplot	?	CLI	SVG, raster, ASCII-art	plots
Tikz	?	LaTeX*	PDF*	anything

This is birdeye overview. There are nuances. I didn’t include some libraries that:

has very narrow use-case (only one type of digram)
are less popular or not supported (abandonware)

Mermaid

Most popular, because it was adopted by github
Works only in the browser. There is a cli, but it just a hack with headless browser (puppeteer)
Theme-able
syntax highlightning

Graphviz

Mature and wide-spread. I think, it was number one for graphs (flowchart, network, DAG) before github integrated mermaid
Originally CLI tool, but there are as well libraries, for example WASM-based, which allows to use Graphwiz on the server and in the browser
syntax highlightning

d2lang

Wants to take over the world
Originally CLI tool, but there is an attempt to convert to WASM library
It worth to mention that it packs headless browser (playwright)
Theme-able
syntax highlightning

Pikchr

“PIC-like markup language for diagrams in technical documentation”
To me it feels more on declarative side, but you can do some imperative things as well
There is WASM library and go port
syntax highlightning

Diagon

“Diagon is an interactive interpreter. It transforms markdown-style expression into an ascii-art representation.”
It is the only one option on the list which outputs (only) ASCII-art. But you can feed it to another tool which will convert it to SVG
There is WASM library, but it’s not distributed as npm package

Tikz

The most famous option between people who use LaTeX
I can’t tell if this more declarative or imperative approach (kind of both)
There is a project which compiles Tikz to WASM (demo), but it’s not distributed as npm package

gnuplot

This is the only one solution on the list which focuses solely on plots. There are a lot of plot libraries and I don’t want to list them all here. The difference from other solutions:

It is standalone, while other solutions are DSLs/libraries inside bigger language, for example matplotlib “requires” python
it is CLI, where is many other solutions are GUI

One way or another I felt like I need to include at least one of those on the list. Originally written in C, so there is a chance of porting it to WASM (see also MobileGnuplotViewerQuickWASM)

I can’t tell if this more declarative or imperative approach (kind of both)

Syntax highlightning:

ASCII-art

Editors:

Special purpose

Other

Other tools (I’m pretty sure there are much more projects which I’m not aware of):

penrose. This one is interesting - I need to read more about it
vega-lite
mscgen
- mscgen_js
flowchart.js
js-sequence-diagrams
kroki
dbml-renderer
dpic
erd
actdiag
nomnoml
UMLet

Instead of conclusion

I’m looking for solution:

to embed diagrams in markdown file (inside fenced code block)
ideally it should be possible to generate on the server (during conversion of markdown to HTML)
ideally it should generate SVG, If we inline SVG (instead of using <img>) it can support
- dark mode via classes or css variables
- it can be animated
- it can contain html links (<a>)
- you can use Cmd + F to search text in it
- we can add zoom/pan to it as in github
- sometime you need to have standalone file as well, which you can include as image for crossposting
it should be lightweight (so no headless browsers)
In some cases it make sense to generate visualisation on the client side (in the browser), for example:
- fractals with infinite zoom
- interactive visualisations, like in g9 or Road to Reality Essays. In this case they should use high performance solutions (canvas or WebGL)

Diagrams that I’m interested in:

graphs (aka network, flowchart, DAG, etc.)
- Hasse diagrams
- subway-scheme like diagrams. For example see: gitgraph, metrosets
Euler diagrams (which are often confused with Venn)
xkcd-style plots
sankey diagrams

One more twist to this story. Because those diagrams can be represented as text you can use GhatGPT to generate diagrams.

Portable markdown links

stereobooster — Fri, 22 Sep 2023 12:00:00 +0000

aka relative links, file path links. Software supports portable markdown links if for content like this:

[test](folder/test.md)

For editor : user can Cmd + Click the link and editor will jump to that file
For static website generator : file pahts are replaced by URLs according generation scheme, for example: [test](folder/test.md) → <a href="/folder/test/">test</a>

Example

portable-hugo-links shows how same links work in Github and website generated by Hugo.

Features

Links: [readme](/readme.md), which also includes resolution of relative and absolute paths:
- [readme](../readme.md)
- [readme](./readme.md)
- [readme](readme.md)
Anchors: readme
- for editor: Cmd + Click should navigate to specified section
- for static website generator: anchors should be preserved in html links
Images: ![alt]_(img.jpg)
- for editor’s preview: images should be displayed
- for static website generator: images should be displayed

Who supports it?

Software	Does it support PML
Github	Yes
VSCode	Yes
Hugo	Yes, with configuration
Obsidian	Yes
Docusaurus	Yes, see documentation
Markdown Language Server	Yes
markdown-links	Yes
Jekyll	Yes, with plugin
Astro	No, see discussion
Next.js	No, AFAIK
Gatsby	No, AFAIK
…

Naming

Why do I call it portable? Because this simple convention allows to use same markdown files with different software wihtout modifications. In the same vein, we can think of portable markdown frontmatter, etc.

A bit of convention over configuration goes long way.

Speed up navigation between pages

stereobooster — Tue, 19 Sep 2023 08:00:00 +0000

Basic idea - navigating between pages is slow, because:

browser waits for network response
it is expensive to redraw the whole page from scratch. Much faster would be to replace innerHTML of the document. At least this is common belief, but there are nuances to that.

History

First solution appeared in 2010. See defunkt/jquery-pjax

Around 2015 "olympic torch" was taken over by turbolinks. Which was part of Rails stack and later replaced by Turbo Drive.

Around the same time there were also:

In 2018 Google jump in on that boat with quicklink

Note: later doesn't mean better, because development didn't stop at the release date. For example, swup recently released v4.

Is it still alive?

Yes.

One would think that with SPA trend (started around 2015) nobody would care about this kind of solutions, but static website still popular. For example Hugo, Astro and others

The devil is in the details

I put all those libraries in one category but they are actually different and use different assumptions. For example,

PJAX uses History API and AJAX (or fetch). It takes over browser navigation mechanism
quicklink on the other side bothers itself only with prefetching/prerendering

Browser history

So we have two main approaches:

don't take over browser navigation
- pros:
- easy to integrate
- cons:
- animations are not possible
- state of the page reseted, like selected tabs, collapsed menu, scroll position in sidebar
- examples: quicklink, @astrojs/prefetch, astro-hover-prefetch, swup preload
take over browser navigation
- pros:
- hard to integrate
- cons:
- animations are possible
- state of the page can be preserved but needs additional work
- examples: pjax, barba, swup, astro-spa

Adaptability

Besides browser navigation we need to take into account:

we don't want to prefetch all the links
- quicklink uses Intersection Observer to detect links within the viewport
- Other approach is to do this on hover, focus, tap
- Some libraries allows to specify which links to prefetch
- Or we can statistically guess
we don't want it to do while browser is busy doing something else. You don't want to degrade user experience
- quicklink uses requestIdleCallback
Making additional requests on mobile devices may cost user additional money
- quicklink uses navigator.connection.effectiveType to detect slow networks, and navigator.connection.saveData to detect data-saver mode

Related functionality

Main subject is to speed up navigation between pages, but there are related functionalities:

animate page navigations, like in swup and barba
replace only part of the page, for example to filter list or table, like in Turbo Frames, swup fragment, astro view transitions
show page preview on hover, like in Obsidian. See also previewbox and astro-link-preview
highlight section if navigated with anchor, like in StakOverflow

I mainly concern myself with static web sites, but in the context of dynamic websites there is so called "HTML over wire" approach:

Complexity with browser history

Basic implementation can look like this

Add global click event listener
on click you fetch remote content
extract element that needs to be replaced (for example main)
replace content with target.innerHTML = response
Push new item to history with history.pushState(state, "", url);

But you need to take care of:

removing event handlers from old HTML
attaching event handlers to new HTML
replacing title, maybe announce it to screen reader
replacing styles
replacing scripts
maybe provide a hook so that it would be possible to integrate analytics
scrolling to top of the page
listen to history for back navigations
maybe resolve relative links
it doesn't preserve state of scroll or tabs

It is an error-prone approach.

To be continued...

While I was doing the research on the subject I stumbled upon:

I will need more time to finish the research

Markdown parsers

stereobooster — Sun, 17 Sep 2023 10:00:09 +0000

JS

micromark used by
- mdast-util-from-markdown used by
- remark-parse used by
  - prettier
  - mdx, which is used by
  - Next.js
  - Gatsby
  - Docusaurus
  - etc.
  - mdc, which is used by
  - Nuxt
- Astro
markdown-it used by
- 11ty
- vuepress
- vitepress
showdown used by (based on what they say)
- GoogleCloudPlatform
- Meteor
- Stackexchange - forked as PageDown
remarkable
- used by Docusaurus
marked used by
- Hexo
- docsify
markup-it used by
- gitbook
commonmark.js
- is the JS reference implementation of CommonMark

WASM

markdown-wasm
- uses md4c
Wasm Markdown
- uses pulldown-cmark
parsedown
- uses pulldown-cmark

Rust

pulldown-cmark used by
- mdBook
- cobalt
zola-markdown
- uses pest.rs
- used by zola
markdown-rs
- Sibling to micromark
comrak
- Rust port of cmark-gfm
markdown-it-rust
- Rust port of markdown-it library.
Snekdown

Golang

C

cmark-gfm
- used by Github
- is an extended version of the cmark
cmark
- is the C reference implementation of CommonMark
md4c

Zig

koino
- Zig port of comrak, which is Rust port of cmark-gfm
markzig archived

Tree-sitter

Tree-sitter is a parser generator tool and an incremental parsing library. There are bindings for many languages (including Rust and JS)

Ruby

jekyll-commonmark
- used by jekyll
- uses cmark-gfm
redcarpet used by
- slate
- jekyll

Python

python-markdown used by
- MkDocs
- Pelican
marko

Clojure

clarktown

Elixir

Other

Crticism

Component libraries trends

stereobooster — Sat, 16 Sep 2023 13:00:00 +0000

Headless and accessible

aka unstyled, renderless

Main idea is to provide all the logic for component (often accessibility) without forcing any visual appearance.

Reach UI
Radix primitives
headlessui
ariakit
not a component library, but worth to mention React Spectrum

Based on a design system

Main idea it is based on thoroughly documented design-system, so it gives more than just components. There can be a lot of implementations based on the same design system. For example, for material design:

Other design systems:

https://designsystemsrepo.com/design-systems/

For mobile development

Main idea is to support mobile devices, for example, bottom navigation bar, gestures, safe-areas, action-sheet etc. Can be used for developing native apps with WebView (PhoneGap/Cordova) or for PWAs.

All react-native-web component libraries are also in this category:

Based on Tailwind

Main idea: it is based on Tailwind. While CSS-in-JS in very unstable situation and they keep inventing zero-runtime solutions, Tailwind just works. So there is whole new wave of component libraries:

Not a component library

This is new interesting approach. We already have enough good component libraries, so instead of creating another one, this tool allows to re-use existing. It is more like collection of snippets.

See shadcn/ui.

DEV Community: stereobooster

The test pyramid is a fiction until you define 'unit'

Five definitions of "unit test"

The same test, different layers

One suite, two shapes

You don't need the shape to find slow tests

Fix it directly

Run Cystoscape.js with Node.js

Solution

Source code

Hugo ideal image

Features

Implementation

Responsiveness

Lazy-loading

LQIP

Portable markdown links

Dark mode

Other

Source code

Tauri instead of Puppeteer or Playwright?

Experiment

CLI

Overcome security fences

Distribution

Size of the npm package

Source code

Next steps

Homebrew

Full automation with GitHub Actions

Conclusion

Distributing executable binaries in npm package

Useful modern tools for static websites

Analytics

Search

Comments

Runtime type validators

Where can be used

Features

Libraries

Fast

Popular

Other

Related

Text to diagram

Categories

Types

Features

Tools

Mermaid

Graphviz

d2lang

Pikchr

Diagon

Tikz

gnuplot

ASCII-art

Special purpose

Other

Instead of conclusion

Portable markdown links

Example

Features

Who supports it?

Naming

Speed up navigation between pages

History

Is it still alive?

The devil is in the details

Browser history

Adaptability

Related functionality

Complexity with browser history

To be continued...

Markdown parsers

JS

WASM

See also

Rust

Golang