DEV Community: stereobooster

Run Cystoscape.js with Node.js

stereobooster — Thu, 12 Oct 2023 16:33:48 +0000

Why? If you need to generate an image on the server side.

At first, I thought it was impossible. There is cytosnap that uses Puppeteer (headless browser). Then, for fun, I rewrote it with Tauri.

But later, I though, what if it would be possible to run it with Node.js. And, indeed, it’s possible.

Solution

Add polyfils for the browser (it also includes canvas):

class XMLSerializer {
 serializeToString(node) {
 return serialize(node);
 }
}
global.XMLSerializer = XMLSerializer;

const dom = new JSDOM(`<!DOCTYPE html><div id="cy"></div>`);
global.window = dom.window;
global.document = dom.window.document;

Explicitly set bounding box

source.layout.boundingBox = {
 x1: 0,
 y1: 0,
 x2: source.width,
 y2: source.height,
};

Generate a graph the same way you would in the browser:

const container = dom.window.document.querySelector("#cy");
const cy = cytoscape({ container, ...defaults, ...source });
cy.layout(source.layout).run();

For now I was able to render it only as SVG:

await loadExtension("svg");
res = cy.svg({
 bg: source.background,
 full: true,
});

In order to not polute global scope with polyfills we can run this script as sub-process:

const executablePath = `bin/cyto-nodejs.js`;
const bin = spawn(executablePath, args, {
 windowsHide: true,
});

That’s it.

Source code

Source code is here: https://github.com/stereobooster/cyto-nodejs

Hugo ideal image

stereobooster — Tue, 10 Oct 2023 23:14:33 +0000

I created image components a couple of times before:

But browsers keep improving - now almost all modern browsers support:

I checked existing solutions - they either don’t do what I want or are complicated (for my taste):

Features

What I expect from an “ideal” image:

Responsiveness
Lazy-loading
Modern LQIP
Portable markdown links
Dark mode

Implementation

There are different ways to implement image components in Hugo:

I think the most pragmatic way would be to implement it as “partial” which accepts an image as a resource. This way it can be reused in render hooks and shortcodes, and the resolution of the image would be the responsibility of the caller.

It can look something like this:

{{ partial "picture.html" (dict "img" $img "alt" $.Text) }}

Partial picture.html would be responsible for: detecting width and height, resizing the image, and rendering HTML. There are no special requirements for HTML or CSS so it would be a pretty portable solution (copy one file).

Responsiveness

Component would be responsible for resizing image and rendering srcset attribute. Basic code:

srcset="
{{- (.Resize "330x").RelPermalink }} 330w,
{{- (.Resize "660x").RelPermalink }} 660w,
{{- (.Resize "1024x").RelPermalink }} 1024w,
{{- (.Resize "1320x").RelPermalink }} 2x"
src="{{ (.Resize "660x").RelPermalink }}"

But :

it suppose to resize only raster images (not SVG)
it should not upscale image

Additionally we can provide webp version of the image. Basic code:

<picture>
<source
srcset="
{{- (.Resize "330x webp").RelPermalink }} 330w,
{{- (.Resize "660x webp").RelPermalink }} 660w,
{{- (.Resize "1024x webp").RelPermalink }} 1024w,
{{- (.Resize "1320x webp").RelPermalink }} 2x"
src="{{ (.Resize "660x webp").RelPermalink }}"
/>
<img ... />
</picture>

Component only responsible for HTML part. You probably would need to add some CSS, for example:

img {
 max-width: 100%;
 height: auto;
}

Lazy-loading

In order to use lazy-load it is enough to add:

loading="lazy"

and optionaly:

decoding="async"

But :

it is important to set width and height of the image, to prevent reflow - browser would know ratio of the image upfront and will reserve space for it without downloading it

Which is pretty easy to do with Hugo

width="{{ .Width }}" height="{{ .Height }}"

But :

it works only for raster images

For SVG we need to do something like this:

{{ if (eq .MediaType.SubType "svg") }}
{{ $width := ""}}
{{ $height := ""}}
{{ $svgContent := .Content }}
{{ range (findRESubmatch `<svg[^>]*width=["']([.0-9]*)["'a-zA-Z]` $svgContent 1) }}
{{ $width = index . 1 }}
{{ end }}
{{ range (findRESubmatch `<svg[^>]*height=["']([.0-9]*)["'a-zA-Z]` $svgContent 1) }}
{{ $height = index . 1 }}
{{ end }}
{{ if (eq "" $width $height) }}
{{ range (findRESubmatch `<svg[^>]*viewBox=["']?([.0-9]*) ([.0-9]*) ([.0-9]*) ([.0-9]*)` $svgContent 1) }}
{{ $width = index . 3 }}
{{ $height = index . 4 }}
{{ end }}
{{ end }}
{{ if (eq "" $width $height) }}
{{ warnf "Can't detect width and height for SVG %s" .RelPermalink }}
{{/* do not use lazy without dimensions */}}
{{ $lazy = false }}
{{ end }}
{{ end }}

This code reads SVG as text and uses regular expression in order to match width, height, and viewBox. Important : it should not match values with %, because they are useless in this case. On the other hand - any absolute units can be matched, like px.

LQIP

There are many LQIP approaches. I like, so called, modern LQIP, which uses 20px, 20% quality webp image as base64.

<picture
{{ if $lqip }}
{{ $bg := (.Resize "20x webp q20").Content | base64Encode }}
style="background-image:url(data:image/webp;base64,{{ $bg }});background-size:cover"
{{ end }}>

I already introduced picture in the markup, so I will reuse it as “holder” for LQIP.

But :

for transparent images (png, webp) LQIP may be problematic, so there suppose to be a way to disable it
I don’t want to generate LQIP for SVG, because I would need to rasterize SVG first, which is too much trouble

Additionaly you may want to add CSS in order to make picture of the same size as image, for example:

picture {
 display: block;
}

To improve appearance it is recomended to add blur:

picture > img {
 -webkit-backdrop-filter: blur(10px);
 backdrop-filter: blur(10px);
}

Image is on “top” of picture. While it is loading it will be transparent. So we can use it add blur to the background. Important : this trick doesn’t work in Firefox, see Bug 1556156.

Portable markdown links

This one is easy. Partial take resource rather than path. So it is responsibility of caller to resolve path.

Dark mode

I’m not sure yet how to implement “dark mode” for images. From an HTML point of view it is trivial to implement:

<picture>
 <source media="(prefers-color-scheme: dark)" srcset="..." />
 <source media="(prefers-color-scheme: light)" srcset="..." />
 <img src="..." />
</picture>

But how to do it from Markdown’s point of view? Some options are:

convention. If there is an image img.png and img_dark.png, the system can automatically find the second image and use it for dark mode
shortcode. Will work, but this is not a markdown solution
Use some kind of marker in the url to distinguis dark-/light-mode images. For example, #gh-dark-mode-only or #gh-light-mode-only. See Specifying the theme an image is shown to
SVGs don’t need a special solution, they can handle it internally. See Creating SVG that appears black in light mode and light in dark mode
for black and white images filter: invert(1) would do the trick

Other

I want to keep number of params minimal. Currently it supports only:

img - image as resource
alt - string, alternative text
class - string, for styling
lazy - boolean, whether it should use lazy loading or not
webp - boolean, whether it should generate webp version or not
lqip - boolean, whether it should generate LQIP or not

Source code

My current implementation is here:

Tauri instead of Puppeteer or Playwright?

stereobooster — Tue, 03 Oct 2023 11:14:09 +0000

Tauri and Puppeteer/Playwright have different use cases. So it seems nothing to compare. But there is (at least) one use case where you may try to use Tuari instead of a headless browser - “snapshots”. For example:

react-snap - HTML snapshot, uses Puppeteer
mermaid-cli - SVG/image snapshot, uses Puppeteer
cytosnap - image snapshot, uses Puppeteer

Why? Because Tauri uses provided by OS browser and produces small binaries (let’s say ~10 MB). While Puppeteer and Playwright need a full browser (let’s say ~200 MB). And Tauri probably can be faster.

Experiment

Let’s see if this will work in practice. For an experiment, I decided to recreate cytosnap with Tauri. Functionality is trivial:

read input from file or STDIN (Elements JSON)
generate a graph with the help of Cytoscape.js
write result (image from canvas) to file or STDOUT
and this is supposed to be CLI, not GUI-app

CLI

CLI applications are not a typical use case for Tauri, but it is possible.

Hide window (tauri.conf.json):

"windows": [{ "visible": false, ...

Hide icon from Dock bar (Mac OS only):

#[cfg(target_os = "macos")]
use tauri::ActivationPolicy;

tauri::Builder::default()
    .setup(|app| {
        #[cfg(target_os = "macos")]
        app.set_activation_policy(ActivationPolicy::Accessory);
        Ok(())
    })

Add CLI arguments (tauri.conf.json):

 "cli": {
      "description": "Render graphs on the server side with Cytoscape.js, getting image file as output",
      "args": [
        {
          "name": "source",
          "short": "s",
          "takesValue": true,
          "multiple": false,
          "multipleOccurrences": false
        },
        {
          "name": "destination",
          "short": "d",
          "takesValue": true,
          "multiple": false,
          "multipleOccurrences": false
        }
      ],
    },

Overcome security fences

Because Tauri’s main use case is GUI, they care about security and limit what files/folders the application can access. Which doesn’t work for the way people use CLI. So I had to write my own Tauri commands to write and read files, for example:

#[tauri::command]
fn write_destination(dst: String, res: String) -> Result<String, String> {
    if dst == "" {
        print!("{}", res);
        Ok(String::from("ok"))
    } else {
        let bytes = base64::engine::general_purpose::STANDARD.decode(res);
        let path = absolute_path(PathBuf::from(dst)).unwrap();
        match bytes {
            std::result::Result::Ok(v) => {
                std::fs::write(path, v).unwrap();
                Ok(String::from("ok"))
            }
            std::result::Result::Err(e) => Err(e.to_string()),
        }
    }
}

Note : I need to pass binary data from the front end to the rust, so I use Base64.

Distribution

Initially, I wanted to distribute this CLI as binary inside the npm package. But then I realized that Tauri can’t really produce portable binaries. Tauri relies on the OS’s browser - which is a neat trick to shrink down the size of the binary, but this is what makes it less portable. Trade-offs as always.

Not to stop an experiment I decided to distribute at least binary for Mac OS in npm. And it works (Mac OS only):

npx @stereobooster/cyto-snap -s g2.json -d g2.png

Size of the npm package

npm notice package size: 2.9 MB
npm notice unpacked size: 7.4 MB

I didn’t do any Tauri optimizations, so it probably can be smaller. Also, this is Mac OS-only binary, it will be bigger if we pack binaries for all platforms in one npm package.

And it has 0 dependencies.

Source code

Warning : because this is an experiment, I didn’t try to make it perfect. Just made it work.

Source code: https://github.com/stereobooster/cyto-snap

Next steps

Homebrew

The idea of distributing binary in npm failed, so I think to try to distribute it via Homebrew. Homebrew works on Mac OS, Windows (WSL 2), and Linux (though they have their own package managers).

Produce standard desktop installers for Tauri (tauri-apps/tauri-action)
Upload installers to GitHub releases
Run installers in silent mode with the Homebrew formula

Full automation with GitHub Actions

It has a lot of steps to produce the final package. I started automation but didn’t finish it. Ideally, it should:

Build application
Run tests (I do integration tests with odiff)
Create a tag and push it
Create release and upload all binaries
Update the Homebrew formula and publish it
Publish the npm package (but it is not very useful without portable binaries)

Conclusion

This approach may work with Homebrew distribution
- It is sad that you can’t publish binaries to npm, which would make JS-developer use other means to install it (Homebrew, apt-get, cURL, etc)
- On the other hand, it is fully independent of npm (and Node.js ecosystem in general), so can be used by none-JS developers
This was my first time using Tauri, and it is awesome

Distributing executable binaries in npm package

stereobooster — Sun, 01 Oct 2023 09:28:23 +0000

Let’s say you need to distribute binary in npm. I’m not talking about node extensions (node-gyp, napi, etc.). I’m talking about standalone executables, like CLI applications.

We have options. How to distribute binary :

host executables on some server, for example, GitHub releases, and download appropriate (e.g. for given OS and architecture) binary on installation
put all executables (for all OSs and architectures) in one npm package
create sub-packages with binary for each OS and architecture and one “root” npm package which would list all sub-packages as optional dependencies

How to run the binary :

Use JS wrapper which would call actual executable with exec, spawn, etc.
On installation figure out where Node puts binaries and overwrite it with your binary from the package
Use JS script to call node binding (*.node) or WASM

Tools and blog posts :

name	distribution	execution
binary-install	1	1 (`spawnSync`)
binwrap	1	1 (`spawn`)
bin-wrapper	1	1 (`spawn`, …)
prebuild-install	1	3
blog.xendit.enginee	2	2
hspak.dev	2	2
prebuildify	2	3
blog.orhun.dev	3	1 (`spawnSync`)
napi-rs	3	3

Example projects :

name	distribution	execution
odiff	2	2
@astrojs/compiler	2 (WASM)	3
esbuild	3	1 (`execFileSync`)
biome	3	1 (`spawnSync`)
Turborepo	3	1 (`execFileSync`)
lightningcss	3	3

Useful modern tools for static websites

stereobooster — Tue, 26 Sep 2023 15:50:02 +0000

This list is about tools beyond static hosting and static website generators. There are already enough list about that.

Analytics

In old times there was basically one default choice: Google analytics. What is bad about GA:

JS size (about 49kb). There alternative clients though, like this one
Blocked by default by some browsers and adblockers
no GDPR compliance (e.g. you always need to provide cookies banner)

But there are a lot of better alternatives (just to name few):

https://tinyanalytics.io/ - 1 site for free
https://umami.is/ - 3 sites for free
https://swetrix.com/ - open source, starts from $5/month

See more alternatives here:

Also some “static hostings” provide analytics, for example:

Cloudflare Pages there is a free tier
Netlify $9/month

Search

In old times people would use “Google custom search engine” (I can’t find it anymore, I guess, they discounted it). There is as well Algolia. But who needs third party paid solution, when you can do it for free.

Recent trend is to generate search index during site generation and then use client side library to do the search. It will probably won’t scale for a huge website, but for personal blog more than enough.

Some options (not a full list):

Honorable mentions:

typesense - open-source, but needs server
docsearch - free Algolia search for developer docs

Or you can do something crazy, like hosting SQLite database over HTTP.

Comments

In old times people would use Disqus. But again, it is not GDPR compliant out of the box and there is huge JS payload (at least it was, when I used it last time).

One of potential solution for developers is to reuse github comments or discussions as comments for your website. Most developers have github account anyway (I know there are people, who are strongly opposed to Github and prefer Gitlab or self-hosted solutions, but this is discussion for another time). For example:

Other option is not to host comments directly on your website, but put a link to a place where people can discuss (especially if you crosspost), for example: hacker news, dev.to, reddit, twitter etc.

Runtime type validators

stereobooster — Sun, 24 Sep 2023 12:00:00 +0000

Where can be used

validate http request from client (at server side)
validate http response from server (at client side)
validate data from untyped storage, for example localStorage, JSON columns in DB, etc.
validate env variables
validate CLI arguments
validate configuration files
validate data from URL (query params)

Features

Basic validation - can validate static types at runtime (can be restricted to JSON type)
Serialisation / deserialisation - for example, TS/JS supports Date, but JSON doesn’t. So you need to do conversion in order to support it
Compatibility with other schemas - there are tools which can convert from- or to- another schemas, for example JSONSchema, OpenAPI etc
Strict or permissive mode - would object be accepted if it contains extra fields
Helpful error messages - if in case of failed validation it points what exactly went wrong or simply dismisses the input
AOT (ahead of time) - can it use type information in order to produce validation libraries or faster serialiser, deserialiser
Custom types - can user create custom types, for example type for email, instead of string

Libraries

Not a full list, but rather a high level overview

Fast

typia
- Doesn’t need extra schema definition. Uses AOT to generate validators, serialisers and deserialisers directly from TypeScript types
- Supports protocol buffers
typebox
- Supports all JSON, JS, JSONSchema, but for TS discriminated unions you need to use typebox-validators
- Can use JIT (and AOT?)
ts-runtime-checks
- Uses AOT

See: Runtype Benchmarks

Other

type-o-rama

Text to diagram

stereobooster — Sat, 23 Sep 2023 14:00:00 +0000

aka diagrams as code

I dreamt about it for a long time. Idea is to allow to express diagrams with a text, this way you can draw diagram while you are writing you markdown file. Without need to switch to another tool. And this diagram becomes part of the document and can be stored in git.

Now when Hugo supports render hooks for code blocks and in MDX you can do this too (to use with Astro, etc.) it’s time to revisit the subject.

Tools

Names	Type	Integration	Formats	Supported diagrams
mermaid	Declarative	browser*	SVG	Flowchart, UML, ER, Gant…
graphviz	Declarative	CLI, library	SVG, raster	Graphs (flowchart, network, etc)
d2lang	Declarative	CLI (go)	SVG, raster…	Flowchart, UML, ER…
pikchr	?	CLI, library	SVG	Kind of anything
Diagon	Declarative	CLI, library	ASCII-art	Flowchart, Sequence…
PlantUML	Declarative	CLI	SVG, raster, ASCII-art	UML…
gnuplot	?	CLI	SVG, raster, ASCII-art	plots
Tikz	?	LaTeX*	PDF*	anything

This is birdeye overview. There are nuances. I didn’t include some libraries that:

has very narrow use-case (only one type of digram)
are less popular or not supported (abandonware)

Mermaid

Most popular, because it was adopted by github
Works only in the browser. There is a cli, but it just a hack with headless browser (puppeteer)
Theme-able
syntax highlightning

Graphviz

Mature and wide-spread. I think, it was number one for graphs (flowchart, network, DAG) before github integrated mermaid
Originally CLI tool, but there are as well libraries, for example WASM-based, which allows to use Graphwiz on the server and in the browser
syntax highlightning

d2lang

Wants to take over the world
Originally CLI tool, but there is an attempt to convert to WASM library
It worth to mention that it packs headless browser (playwright)
Theme-able
syntax highlightning

Pikchr

“PIC-like markup language for diagrams in technical documentation”
To me it feels more on declarative side, but you can do some imperative things as well
There is WASM library and go port
syntax highlightning

Diagon

“Diagon is an interactive interpreter. It transforms markdown-style expression into an ascii-art representation.”
It is the only one option on the list which outputs (only) ASCII-art. But you can feed it to another tool which will convert it to SVG
There is WASM library, but it’s not distributed as npm package

Tikz

The most famous option between people who use LaTeX
I can’t tell if this more declarative or imperative approach (kind of both)
There is a project which compiles Tikz to WASM (demo), but it’s not distributed as npm package

gnuplot

This is the only one solution on the list which focuses solely on plots. There are a lot of plot libraries and I don’t want to list them all here. The difference from other solutions:

It is standalone, while other solutions are DSLs/libraries inside bigger language, for example matplotlib “requires” python
it is CLI, where is many other solutions are GUI

One way or another I felt like I need to include at least one of those on the list. Originally written in C, so there is a chance of porting it to WASM (see also MobileGnuplotViewerQuickWASM)

I can’t tell if this more declarative or imperative approach (kind of both)

Syntax highlightning:

ASCII-art

Editors:

Special purpose

Other

Other tools (I’m pretty sure there are much more projects which I’m not aware of):

penrose. This one is interesting - I need to read more about it
vega-lite
mscgen
- mscgen_js
flowchart.js
js-sequence-diagrams
kroki
dbml-renderer
dpic
erd
actdiag
nomnoml
UMLet

Instead of conclusion

I’m looking for solution:

to embed diagrams in markdown file (inside fenced code block)
ideally it should be possible to generate on the server (during conversion of markdown to HTML)
ideally it should generate SVG, If we inline SVG (instead of using <img>) it can support
- dark mode via classes or css variables
- it can be animated
- it can contain html links (<a>)
- you can use Cmd + F to search text in it
- we can add zoom/pan to it as in github
- sometime you need to have standalone file as well, which you can include as image for crossposting
it should be lightweight (so no headless browsers)
In some cases it make sense to generate visualisation on the client side (in the browser), for example:
- fractals with infinite zoom
- interactive visualisations, like in g9 or Road to Reality Essays. In this case they should use high performance solutions (canvas or WebGL)

Diagrams that I’m interested in:

graphs (aka network, flowchart, DAG, etc.)
- Hasse diagrams
- subway-scheme like diagrams. For example see: gitgraph, metrosets
Euler diagrams (which are often confused with Venn)
xkcd-style plots
sankey diagrams

One more twist to this story. Because those diagrams can be represented as text you can use GhatGPT to generate diagrams.

Portable markdown links

stereobooster — Fri, 22 Sep 2023 12:00:00 +0000

aka relative links, file path links. Software supports portable markdown links if for content like this:

[test](folder/test.md)

For editor : user can Cmd + Click the link and editor will jump to that file
For static website generator : file pahts are replaced by URLs according generation scheme, for example: [test](folder/test.md) → <a href="/folder/test/">test</a>

Example

portable-hugo-links shows how same links work in Github and website generated by Hugo.

Features

Links: [readme](/readme.md), which also includes resolution of relative and absolute paths:
- [readme](../readme.md)
- [readme](./readme.md)
- [readme](readme.md)
Anchors: readme
- for editor: Cmd + Click should navigate to specified section
- for static website generator: anchors should be preserved in html links
Images: ![alt]_(img.jpg)
- for editor’s preview: images should be displayed
- for static website generator: images should be displayed

Who supports it?

Software	Does it support PML
Github	Yes
VSCode	Yes
Hugo	Yes, with configuration
Obsidian	Yes
Docusaurus	Yes, see documentation
Markdown Language Server	Yes
markdown-links	Yes
Jekyll	Yes, with plugin
Astro	No, see discussion
Next.js	No, AFAIK
Gatsby	No, AFAIK
…

Naming

Why do I call it portable? Because this simple convention allows to use same markdown files with different software wihtout modifications. In the same vein, we can think of portable markdown frontmatter, etc.

A bit of convention over configuration goes long way.

Speed up navigation between pages

stereobooster — Tue, 19 Sep 2023 08:00:00 +0000

Basic idea - navigating between pages is slow, because:

browser waits for network response
it is expensive to redraw the whole page from scratch. Much faster would be to replace innerHTML of the document. At least this is common belief, but there are nuances to that.

History

First solution appeared in 2010. See defunkt/jquery-pjax

Around 2015 "olympic torch" was taken over by turbolinks. Which was part of Rails stack and later replaced by Turbo Drive.

Around the same time there were also:

In 2018 Google jump in on that boat with quicklink

Note: later doesn't mean better, because development didn't stop at the release date. For example, swup recently released v4.

Is it still alive?

Yes.

One would think that with SPA trend (started around 2015) nobody would care about this kind of solutions, but static website still popular. For example Hugo, Astro and others

The devil is in the details

I put all those libraries in one category but they are actually different and use different assumptions. For example,

PJAX uses History API and AJAX (or fetch). It takes over browser navigation mechanism
quicklink on the other side bothers itself only with prefetching/prerendering

Browser history

So we have two main approaches:

don't take over browser navigation
- pros:
- easy to integrate
- cons:
- animations are not possible
- state of the page reseted, like selected tabs, collapsed menu, scroll position in sidebar
- examples: quicklink, @astrojs/prefetch, astro-hover-prefetch, swup preload
take over browser navigation
- pros:
- hard to integrate
- cons:
- animations are possible
- state of the page can be preserved but needs additional work
- examples: pjax, barba, swup, astro-spa

Adaptability

Besides browser navigation we need to take into account:

we don't want to prefetch all the links
- quicklink uses Intersection Observer to detect links within the viewport
- Other approach is to do this on hover, focus, tap
- Some libraries allows to specify which links to prefetch
- Or we can statistically guess
we don't want it to do while browser is busy doing something else. You don't want to degrade user experience
- quicklink uses requestIdleCallback
Making additional requests on mobile devices may cost user additional money
- quicklink uses navigator.connection.effectiveType to detect slow networks, and navigator.connection.saveData to detect data-saver mode

Related functionality

Main subject is to speed up navigation between pages, but there are related functionalities:

animate page navigations, like in swup and barba
replace only part of the page, for example to filter list or table, like in Turbo Frames, swup fragment, astro view transitions
show page preview on hover, like in Obsidian. See also previewbox and astro-link-preview
highlight section if navigated with anchor, like in StakOverflow

I mainly concern myself with static web sites, but in the context of dynamic websites there is so called "HTML over wire" approach:

Complexity with browser history

Basic implementation can look like this

Add global click event listener
on click you fetch remote content
extract element that needs to be replaced (for example main)
replace content with target.innerHTML = response
Push new item to history with history.pushState(state, "", url);

But you need to take care of:

removing event handlers from old HTML
attaching event handlers to new HTML
replacing title, maybe announce it to screen reader
replacing styles
replacing scripts
maybe provide a hook so that it would be possible to integrate analytics
scrolling to top of the page
listen to history for back navigations
maybe resolve relative links
it doesn't preserve state of scroll or tabs

It is an error-prone approach.

To be continued...

While I was doing the research on the subject I stumbled upon:

I will need more time to finish the research

Markdown parsers

stereobooster — Sun, 17 Sep 2023 10:00:09 +0000

JS

micromark used by
- mdast-util-from-markdown used by
- remark-parse used by
  - prettier
  - mdx, which is used by
  - Next.js
  - Gatsby
  - Docusaurus
  - etc.
  - mdc, which is used by
  - Nuxt
- Astro
markdown-it used by
- 11ty
- vuepress
- vitepress
showdown used by (based on what they say)
- GoogleCloudPlatform
- Meteor
- Stackexchange - forked as PageDown
remarkable
- used by Docusaurus
marked used by
- Hexo
- docsify
markup-it used by
- gitbook
commonmark.js
- is the JS reference implementation of CommonMark

WASM

markdown-wasm
- uses md4c
Wasm Markdown
- uses pulldown-cmark
parsedown
- uses pulldown-cmark

Rust

pulldown-cmark used by
- mdBook
- cobalt
zola-markdown
- uses pest.rs
- used by zola
markdown-rs
- Sibling to micromark
comrak
- Rust port of cmark-gfm
markdown-it-rust
- Rust port of markdown-it library.
Snekdown

Golang

C

cmark-gfm
- used by Github
- is an extended version of the cmark
cmark
- is the C reference implementation of CommonMark
md4c

Zig

koino
- Zig port of comrak, which is Rust port of cmark-gfm
markzig archived

Tree-sitter

Tree-sitter is a parser generator tool and an incremental parsing library. There are bindings for many languages (including Rust and JS)

Ruby

jekyll-commonmark
- used by jekyll
- uses cmark-gfm
redcarpet used by
- slate
- jekyll

Python

python-markdown used by
- MkDocs
- Pelican
marko

Clojure

clarktown

Elixir

Other

Crticism

Component libraries trends

stereobooster — Sat, 16 Sep 2023 13:00:00 +0000

Headless and accessible

aka unstyled, renderless

Main idea is to provide all the logic for component (often accessibility) without forcing any visual appearance.

Reach UI
Radix primitives
headlessui
ariakit
not a component library, but worth to mention React Spectrum

Based on a design system

Main idea it is based on thoroughly documented design-system, so it gives more than just components. There can be a lot of implementations based on the same design system. For example, for material design:

Other design systems:

https://designsystemsrepo.com/design-systems/

For mobile development

Main idea is to support mobile devices, for example, bottom navigation bar, gestures, safe-areas, action-sheet etc. Can be used for developing native apps with WebView (PhoneGap/Cordova) or for PWAs.

All react-native-web component libraries are also in this category:

Based on Tailwind

Main idea: it is based on Tailwind. While CSS-in-JS in very unstable situation and they keep inventing zero-runtime solutions, Tailwind just works. So there is whole new wave of component libraries:

Not a component library

This is new interesting approach. We already have enough good component libraries, so instead of creating another one, this tool allows to re-use existing. It is more like collection of snippets.

See shadcn/ui.

Distributing CSS in npm package

stereobooster — Sat, 09 Sep 2023 15:00:00 +0000

Introduction

Typical problem: you want to distribute React (Solid, Vue, etc.) component in npm package. Most likely it will need some kind of styles and maybe assets (images, svg, fonts). How you're gonna do it. Two options:

distribute CSS files
CSS-in-JS

Distributing CSS has following issues:

You need explicitly include those files (dependency)
You may need to process them (with bundler/compiler) in order to adjust paths
They may result in dead code (e.g. code that is never used)
Styles may clash (global namespace, isolation)
There can be issues with non-deterministic resolution
No customisation (no theme support)

Distributing CSS-in-JS:

resolves all CSS issues, but also
adds runtime penalty
increases bundle size

This is classical point of view, but there is a twist. Picture may change a bit with:

atomic CSS
CSS variables (custom properties)
zero-runtime CSS-in-JS (compilation)

CSS-in-JS alternatives

Before we continue let's mention alternatives:

atomic CSS UI kit - Tailwind it the most popular option here
CSSModules

If you want to distribute components (in npm package) you would have to compile those to CSS and distribute this file, which brings us back to CSS issues.

Note: there is a way to compile Tailwind inside the npm package

Example: @dlarroder/playground (blog post)

It contains global styles:

*,
:after,
:before {
  border: 0 solid #e5e7eb;
  box-sizing: border-box;
}

You may be able to customize it via CSS variables

*,
:after,
:before {
  --tw-ring-color: rgba(59, 130, 246, 0.5);
}

But there is no type-safety. I can do a typo --tw-ring-col: rgba(59, 130, 246, 0.5); and nothing will notify me about the error

What if you use two component libraries that use different versions of Tailwind. There will be clash
CSS file may contain all styles for all components, but if I use only one component all the rest of styles will be a dead code

CSS-in-JS issues

The biggest problem is runtime penalty. Your components would need to parse code for CSS (template literals or JS object), do a vendor specific prefixing, and inject styles and all this happens in the main thread.

Second problem is increased bundle size, which includes runtime itself and CSS expressed in JS.

Don't forget about server side rendering, which also would need special handling.

As the solution you may use one of zero-runtime approaches (in alphabetic order):

compiledcssinjs
linaria
macaron uses vanilla-extract
panda
style9
styled-vanilla-extract uses vanilla-extract
tamagui inspired by vanilla-extract
unocss
vanilla-extract

But then again if you "compile" CSS-in-JS before distributing via npm you will end up with "style" file. So you need to distribute it as is and compilation should be done by the consumer, which may be problematic. Because there are a lot of bundlers/compilers, for example:

webpack/babel
vite/esbuild
turbopack/swc
etc

So you either will vendor-lock your consumers to one solution or CSS-in-JS need to provide all options.

solution	vite	esbuild	webpack	next	parcel	rollup	babel	cli
vanilla-extract	+	+	+	+	+	+
linaria	+	+	+			+	+
panda								+
compiledcssinjs			+		+		+

Note: this is not a fair comparison - devil is in details.

Real-world experience

What do big component libraries (UI kits) choose to use.

Use compile-time CSS-in-JS

mui: [RFC][system] Zero-runtime CSS-in-JS implementation. But currently uses emotion
Chakra UI: The future of Chakra UI. Currently uses panda. Before was using emotion.

Use Tailwind

Use runtime CSS-in-JS

baseweb uses styletron
mantime uses emotion based css-in-js library
fluentui: Motivations for moving away from SCSS. Currently uses @fluentui/merge-styles
evergreen uses ui-box
grommet uses styled-components
All react-native-web based systems:
- paper
- RNUI
- magnus
- React Native UI Kitten
- gluestack-ui, successor for NativeBase

Use "nothing"

There is a trend for un-styled components (aka renderless, headless). They use "nothing", but this becomes responsibility of consumer to provide styles (including essential ones):

radix-ui primitives uses style prop and CSS variables
ariakit uses style prop and CSS variables
headlessui can be used with Tailwind
reach-ui uses CSS for essential styles
@mui/base uses useUtilityClasses

DEV Community: stereobooster

Run Cystoscape.js with Node.js

Solution

Source code

Hugo ideal image

Features

Implementation

Responsiveness

Lazy-loading

LQIP

Portable markdown links

Dark mode

Other

Source code

Tauri instead of Puppeteer or Playwright?

Experiment

CLI

Overcome security fences

Distribution

Size of the npm package

Source code

Next steps

Homebrew

Full automation with GitHub Actions

Conclusion

Distributing executable binaries in npm package

Useful modern tools for static websites

Analytics

Search

Comments

Runtime type validators

Where can be used

Features

Libraries

Fast

Popular

Other

Related

Text to diagram

Categories

Types

Features

Tools

Mermaid

Graphviz

d2lang

Pikchr

Diagon

Tikz

gnuplot

ASCII-art

Special purpose

Other

Instead of conclusion

Portable markdown links

Example

Features

Who supports it?

Naming

Speed up navigation between pages

History

Is it still alive?

The devil is in the details

Browser history

Adaptability

Related functionality

Complexity with browser history

To be continued...

Markdown parsers

JS

WASM

See also

Rust

Golang

C

Zig

Tree-sitter

Ruby

Python

Clojure