DEV Community: Christian Bewernitz

Release 0.9.0 of `@xmldom/xmldom`

Christian Bewernitz — Thu, 29 Aug 2024 20:06:11 +0000

Context

xmldom is a javascript ponyfill to provide the following APIs that are present in modern browsers to other runtimes:

convert an XML string into a DOM tree ```

new DOMParser().parseFromString(xml, mimeType) => Document
- create, access and modify a DOM tree
new DOMImplementation().createDocument(...) => Document
- serialize a DOM tree back into an XML string
new XMLSerializer().serializeToString(node) => string

Source: xmldom readme

History

Since I started contributing to the forked xmldom library in June 2020, there have been 40 releases.

It is a very interesting and challenging project and will most likely stay that way for quite a while.

According to GitHub over 50 people have contributed to it since it was forked.

Thank you again to all contributors.

And this doesn't count all the people that managed to make the move from the original unscoped xmldom package, to the scoped @xmldom/xmldom package version 0.7.0 to get all security fixes.
The most recent version released as the lts tag is 0.7.13.

The last version with breaking changes was 0.8.0 which was released on Dec 22, 2021, almost 3 years ago.
The most recent version released as latest is 0.8.10.

0.9.0 (2024-08-29)

But what I want to talk about today is all the stuff that has been released under the next tag since October 2022.

I'm really excited about those changes since they are providing a clear foundation for potential future changes.

TLDR: More alignment with the specs, and differences are made as explicit as possible.

1. Enforcing `mimeType` to give back control

One aspect that makes the implementation complex, is that there are different rules for parsing XML vs HTML.
xmldom (to some degree) "supported" both flavors from the beginning. It was even not required to pass a mimeType at all: What rules to apply was decided based on the current default namespace of the XML string/node that was currently being parsed.

This ends with 0.9.0: From now on the mimeType in DOMParser.parseFromString(xml, mimeType) is mandatory and is the only thing that is ever checked to decide whether to apply XML or HTML rules. Basta.

And that information is preserved in the resulting Document (new type property), so when serializing it, the proper rules are applied again.

This was a massive (and potentially breaking) change, but I'm really excited it is ready, since it made tons of related bug fixes possible/way simpler to implement and also reduces the complexity of the API and the implementation.

Additionally it now only accepts the mime types specified, and throws a TypeError in any other case.

Strictness and Error handling

An aspect that personally confuses me about the error handling of the native browser API is that it always returns a Document and if something went wrong, a parsererror node will be the first child of the body:

Since error handling never worked this way in xmldom but the existing error handling was very complex and confusing and badly documented, 0.9.0 simplifies it and now has a (way more) consistent behavior towards any potential error that happens during parsing:
It throws a ParseError 🎉, e.g. in one of the following cases:

In previous versions it was possible for some non well-formed XML strings, that the returned Document would not have a documentElement, which will most likely lead to TypeErrors later in the code.
several non well-formed XML strings will now properly be reported as fatalError which now always prevents any further processing.
several things that have previously not been reported as an error or only have been reported as a warning are now also reported as a fatalError

There are still cases left which are reported as a warning (especially when parsing HTML) or as an error which do not stop the data from being processed, but the new error handling makes it very easy to decide how strict the code that uses xmldom needs to be.

The (non spec compliant) option that can be passed to the DOMParser constructor is called onError.
it takes a function with the following signature:



function onError(level:ErrorLevel, message:string, context: DOMHandler):void;

ErrorLevel is either warning, error or fatalError
xmldom already provides an implementaiton for the two most common use cases:
- onErrorStopParsing to throw a ParseError also for all error level issues
- onWarningStopParsing to throw a ParseError also for all error level issues

It is a recommendation to apply one of them to stop processing XML on the first signal of anything unexpected:



// prevent parsing of XML that has errors

new DOMParser({onError: onErrorStopParsing}).parseFromString(...)

// prevent parsing of XML that has warnings

new DOMParser({onError: onWarningStopParsing}).parseFromString(...)

`compareDocumentPosition`, extended HTML entities , `null` instead of `undefined`, ...

Another fork of the original xmldom repository made it's way back into our repo by extending the HTML entities to the complete set (also available in 0.8.x) and porting over the implementation of the compareDocumentPosition API. Thank you, and welcome @zorkow

Along the way several places where xmldom so far returned undefined instead of null, have been fixed to adhere to the spec.

And I discovered that the former author seems to have preferred iterating from the end of a list in so many places, that attributes were processed in the reverse order in multiple places, which is now fixed.

The implementation of the removeChild API changed quite a bit, to comply to the spec and throws a DOMException when it should.

And 3 related bugs were fixed in a way that clearly states what the future direction of xmldom is:
Support for lax HTML parsing rules will only be provided if proper strict XML parsing doesn't suffer from it.
The former (broken) "support" for automatic self closing tags in HTML is gone.

coctype internalSubset

More recently @shunkica invested a huge amount of time end effort to fix tons of issues in the former handling of the internalSubset part of the !DOCTYPE.

It is now preserved as part of the internalSubset property of the doctype of a Document and many wrong doctype declarations are now correctly detected as such and reported as a fatalError.

Also thanks to @kboshold for the latest bug fix in this area.

Along the way we created a new module containing regular expressions for the relevant grammar, and correctness checks are based on those and they are properly covered by tests.

It is not the goal of xmldom to become a validating parser, but this a great step to support those documents that come with more complex DTDs.

And there is even more

Up to now development was done using Node v10, since this is also the lowest version xmldom currently supports. As part of the work on the upcoming version, I decided to switch to v18 for development, since more and more devDependencies also made this a minimum requirement. This will be the new minimum runtime version for the time being starting with this release.

I initiated a public poll / dicussion to ask people which version of Node or other runtimes they need support for.
The next breaking release will most likely drop support for some older Node versions, if there is no feedback indicating something different.

Along the way plenty of APIs have received jsdoc comments with proper types.

Thank you

for taking the time to read through all of this.

Those are quite some changes, and I'm very excited to be able to ship those.

I hope you are as excited as I am :)

If you need more details you can go through the very detailed changelog, or head over to the repository and join or start a discussion or file an issue.

Tired of approving dependency PRs?

Christian Bewernitz — Sat, 19 Aug 2023 17:01:56 +0000

Dependency PRs?

I'm a huge fan of https://www.mend.io/renovate/ for updating dependencies PR by PR for multiple reasons.

So much that I even created a shared config at work with some nice docs.

Well, they have a automerge option, right?

Yes, and recently it is implemented by enabling "automerge" in GitHub!

But for most repositories it is reasonable to require somebody to review code changes before they land, which is why we have branch protection with some status checks and at least one reviewer approval.

And since we don't want the dependency PRs to get into the way, the default option we use is to only automatically create and update them outside of office hours.

And to reduce cost for GitHub actions that are run for each dependency update PR, we only ever create one at a time.

Repeated manual work...

the creation of dependency update PRs is automated
and the testing is automated
for landing them it still requires a human to approve. Outside of office hours. (So it's either unlikely to get more than a handful landed per day, or it encourages people to work outside of office hours.)

And this means they usually pile up, as it visible on the nice Dependency Dashboard renovate offers.

... should be automated

I had some attempts on researching different options to solve this problem, like adding some github app and configuring them or adding github actions that would take care of it, but it all sounded quite complicated, so I never really did something about it.

Recently I listened to https://changelog.com/jsparty/287 and learned about https://val.town
I registered during listening to the episode (using my GitHub account) and went through the short introduction tour.

I didn't immediately have an idea what to do with it, but I was sure it wouldn't take long.

Last week it hit me:
How difficult would it be to create a val that acts as a webhook for github that would automatically approve PRs that are qualified?

githubWebhookApproveDependencyPRs | @karfau | Val Town

A webhook to approve dependency PRs created by bots The webhook can be configured on the repo or on the org level it needs to receive the workflow_runs events it needs to have the webhookSecret configured (line 7) it needs to send json payload It will only approve if all of the following conditions are met: the event action is completed and the workflow_run.conclusion has to be success the event is related to exactly one pull request authored by one of the users listed in allowedAuthors (line 5) the githubApiToken configured in line 9 needs to have repo access to the related repository a branch protection has to be configured that requires a review and that requires at least one status check the current value for dryRun is false (line 3) If it approves the PR, it leaves a comment pointing to the website of this val.

val.town

Turns out, it wasn't that difficult, even though I was new to the platform.

And I received some quick and helpful support on Discord when I was stuck. Thx for that.

And the best thing about it is, that you can fork it an use it on your own.

I published my first deno package

Christian Bewernitz — Thu, 06 Jan 2022 08:03:21 +0000

Cover Image by Cmglee - Own work, CC BY-SA 3.0

What was it like to do that?

It was so much fun compared to all previous experiences of publishing a (typescript or javascript) package.

What made it really easy for me to get up to speed with deno:

Of course the experience to "just execute typescript" without having to first research the "currently best way to transpile the code in this use case", is just mind blowing again and again.

Instead of picking from tons of tools, I was regularly visiting the deno manual.
Even when I didn't expect something to be provided out of the box, regular searching pointed me to it.

Deno comes with a test runner and provides basic assertions that can also take care of collecting coverage data.
It also comes with a linter and a (code and docs and code in docs) formatter and there is more.

But the most fun part was publish the package:

I claimed the library name. (I didn't need to create an account for that!)
I followed the instructions to add webhook to my github repository
I created a release on GitHub. This is the step that needs to be repeated to publish a new version.

(Did you know that the versions on https://deno.land/x are immutable and can not be dropped?)

So what package and why?

Thx for asking ;)

I recently analysed npm packages that are still using an insecure version of another library I'm maintaining, to judge if it makes sense to provide an upgrade PR. Since there are over 2000 dependents on all older versions, and at that point in time over 60 on the versions I care about, I needed to prioritize the packages. So I can first provide update the PRs that have a reasonable impact on reducing the usage and have a good chance of being merged.

I still have that idea "Maybe I just didn't use the correct search terms?", because I couldn't find any existing/similar library. If you know about something comparable please let me know.

karfau / scor

Calculate scores for numeric values or items, and get the "total score" (aka "arithmetic mean" or "weighted arithmetic mean") from multiple scores.

scor

Calculate scores for numeric values or items, and get the "total score" (aka arithmetic mean or weighted arithmetic mean) from multiple scores.

Usage

Imagine you

have a long list of items to work on, and you want to prioritize them
want to show the most relevant items to a user before showing more

For example, let's look at npm packages. Possible criteria are:

number of maintainers
number of dependencies (direct/transient)
time since last published version
version (major < 1?) / dist-tags
weekly downloads
source code repo attributes (e.g. GitHub stars/forks)
quality?
...?

The different relevant values come in very different "shapes". Once all the data is gathered per package, depending on the use case the different values are more or less relevant.

import {
  createToMean,
  distributeWeights,
  scorForItems,
} from "https://deno.land/x/scor/scor.ts";
import { getPackagesData } from "./npm.ts";
const packages = await getPackagesData()

…

View on GitHub

It was developed using TDD, so the coverage is 100%.

To use it in your deno code you can import it like this:

import {scor} from "https://deno.land/x/scor@v1.0.0/scor.ts";

Make sure to use the latest version:

I'm quite happy about the existing documentation (the readme and doc comments), so I won't repeat it here.

I'm really curious to see if it's understandable, so please let me know.

What' the name about?

Well, I checked for suitable names that are still available as an (unscoped) npm package.

So, are you going to publish it to npm?

Maybe, it's currently the last thing on my short list of todos.

First I want to collect some feedback, to see if it's worth the additional effort. If you think it is, feel free to create PR.

Automating some annoying clicks

Christian Bewernitz — Wed, 30 Sep 2020 06:44:14 +0000

I just automated away something that annoyed me:

Whenever I'm in some foreign WIFI (e.g. in the train) I need to find the right URL to open in the browser to confirm some dialog or login before having access to the internet.

I recently found out that I can just enter the IP-address of the primary DNS server from my network information into the browser as the most reliable way.

But that's still to many clicks for doing it multiple times a day (e.g. when switching between trains), so I composed a few bash commands to do that for me:

xdg-open "http://$(grep "nameserver" /etc/resolv.conf | tr  -d "nammeserver ")" > /dev/null &

Let's see what's happening here from inside out:

$ grep "nameserver" /etc/resolv.conf
nameserver 192.168.8.1

delivers the current primary DNS,

... | tr -d "nameserver "

drops the first word and space from the previos command so that only the IP address is left,

"http://$(...)"

prefixes the output from above commands (executed in a sub-shell) to make it a URL

xdg-open "..." > /dev/null &

opens the default browser (because the string is a URL),

> /dev/null: without writing some message like "Opening in existing browser session." to the terminal
&: in the background (without blocking the shell)

I added the whole thing to a new wifi alias in my ~/.extend.bashrc:

alias wifi='xdg-open "http://$(grep "nameserver" /etc/resolv.conf | tr  -d "nammeserver ")" > /dev/null &'

so it's always on my fingertips :)

PS: It's not working everywhere (e.g. not in the German ICE trains, thanks Deutsche Bahn)
and the next hit of Ctrl+C adds the output

[n]   Done     ...the command...

So I'm very open for suggestion of how to improve it.

Gute Gewohnheiten für sichere Internet-Nutzung

Christian Bewernitz — Mon, 30 Dec 2019 02:06:39 +0000

Dies ist eine Übersetzung von Healthy habits for cyber security (EN), die Links die im Englischen Artikel existieren, im Artikel verweisen auf die gleichen (Englisch sprachigen) Webseiten wie im übersetzten Artikel und enden mit (EN). Links die ich zur Erläuterung hinzugefügt habe enden mit (DE).

So wie in dieser Jahreszeit sich (wieder) eine Erkältung holen, ist das Risiko einem technischem Angriff zu "erliegen" ständig präsent.
Sowohl eine ausgeklügelter Angriff auf der Grundlage sozialer Netzwerke als auch eine Phishing-Mail ohne Grammatik können echte Schäden anrichten.
Niemand der über das Internet kommuniziert ist immun.

So wie gründliches Händewaschen oder Impfungen können Gewohnheiten das Risiko verringern unabsichtlich einen Angriff weiter zu verteilen.
Da das neue Jahr oft zu neuen Gewohnheiten inspiriert, schlage ich hier ein paar vor, die dabei helfen können, Dich und die Menschen um Dich herum zu schützen.

1. Frag nach

Es wird immer schwieriger einen Angriff an der Art der Nachricht zu erkennen.
Nachrichten mit gefährlichen Links kommen nicht immer von Fremden. Manchmal sehen sie aus wie alltägliche Kommunikation oder sehen danach aus als ob sie von jemandem sind den man kennt oder mit dem man arbeitet.
Angreifer nutzen subtile aber tief verwurzelte kognitive Verzerrungen (DE) die es oft ermöglichen den gesunden Menschenverstand zu „überspringen“. So das der natürliche Impuls ist zu klicken.

Zum Glück gibt es eine einfache Gewohnheit um diese Art von Attaken zu verhindern: Vor dem Klick nachfragen.

Vielleicht bekommst Du eine Mail von einem Freund der dringend Hilfe braucht oder von Deinem Chef der gerade in ein Flugzeug steigt.
Es könnte so verlockend oder mysteriös sein wie eine direkte Nachricht von einem Bekannten der einen Link schickt und fragt „😂 Bist Du das?“

Man muss geistesgegenwärtig sein um die Reaktion zu verhindern auf die diese Angriffe lauern, aber die Abwehr ist schnell und unkompliziert. Schicke eine Textnachricht, rufe kurz and oder geh den Gang runter und frage „Hast Du mir das geschickt?“

Wenn die Nachricht echt ist, tun ein paar mehr Minuten für die Frage nicht weh. Wenn sie nicht echt ist, hast Du die Person von der es ausging sofort darüber informiert, das sie kompromittiert wurde. Und das kann sehr dazu betragen solch einen Angriff aufzuhalten.

2. Nutze Ende-zu-Ende verschlüsselte Nachrichten und ermutige andere dazu

Auf die selbe Art wie Impfungen das Risiko verringert, selbst zur Ansteckungsgefahr zu werden, sind sowohl Du als auch Menschen die Du kennst sicherer wenn Du verschlüsselte Kommunikation verwendest.
Ermutige Deine Freunde, Kollegen und Tante Matilda zu einer App wie Signal zu wechseln. Dadurch trägst Du dazu bei, dass weniger Menschen Nachriten Dienste nutzen die leicht angreifbar sind.

Das bedeutet nicht, das Du aufhören musst angreifbare Dienste zu nutzen. Stattdessen betrachte es als eine Hierarchie. Nutze Signal für wichtige Nachrichten, denen man vertrauen muss, wie Bitten um Geld oder Absprachen zu einer Reise.
Benutze alle andere Kanäle, wie SMS oder soziale Netzwerke, nur für „unwichtige“ Kommunikation.
Wenn jetzt Bitten oder Links die wichtig zu sein scheinen, über „unwichtige“ Kanäle ankommen, wirst Du wahrscheinlich eher vorsichtig sein.

3. Stecke keine „dreckigen“ USB-Stecker in deinen ***

Du würdest Deine Zähne ja auch nicht mit einer Zahnbürste putzen die Du in einer Gasse gefunden hast. Warum also würdest Du also eine USB-Gerät anstecken, von dem Du nicht weisst wo es war?!
Wir können zwar noch sagen das die menschliche Neugierde clever ausgenutzt wurde, wenn jemand einen gefundenen USB Stick in seinen Computer steckt (EN).
Aber wir würden wohl kaum eine öffentliche Telefon-Ladestation (EN) oder ein selbst gekauftes USB-Kabel (EN) verdächtigen.
Sogar scheinbar harmlose USB Eingabegeräte (EN) oder wiederaufladbare (EN) Geräte können ein Risiko darstellen.

Anders als bei Mails und einigen Diensten zum Versenden von großen Dateien, bei denen die Dateien gescannt und gefiltert werden, bevor sie Deinen Computer erreichen, bedeutet das Einstecken per USB eine direkte und ungeschützte (EN) Verbindung.
Sobald diese Verbindung besteht, braucht der Nutzer nichts mehr zu tun, damit eine ganze Menge schlechter Dinge passieren kann. Es ist für Angreifer sehr einfach auf diesem Weg Schadsoftware (DE) oder Ransomware (DE) auf einem Rechner oder Mobil-Telefon zu installieren.

Natürlich muss auch nicht auf den Komfort von USB verzichtet werden. Statt mit fragwürdigem Verhalten gegenüber USB Geräten zu beginnen, spare einfach nicht beiKauf von USB Geräten und Kabeln.
Wenn es in Deinen Computer gesteckt werden soll, stelle sicher das Du extra aufmerksam bist.
Kauf es direkt von einem Hersteller (wie den Apple Store) oder Verkäufer die Ihre Lieferkette überwachen.
Stecke wiederaufladbare Geräte nicht an Deinen Rechner. Nutze stattdessen einen Stecker mit USB Anschlüssen (EN).

Praktiziere gute Sicherheits-Gewohnheiten

Deine Geräte gesund und munter zu halten ist eine Frage von guten Gewohnheiten. So wie sich gegen eine Grippe zu schützen, können gute Gewohnheiten Dich und die um Dich herum schützen.
Mach solche gewissenhaften Gewohnheiten zum Teil Deiner Vorhaben für das neue Jahr - oder fange einfach gleich damit an.

Ich wünsche euch angenehme und sichere Feiertage.

My first published npm package is called runex

Christian Bewernitz — Tue, 24 Dec 2019 14:39:07 +0000

Some months ago I got tired of all those tiny differences that you need to consider when writing CLI scripts for node.

So in the last days I took the time to rewrite a module that I have already copied into more than three repositories:

karfau / runex

Run (javascript) module export as a script

runex

Run module export as a node or npx script.

(See Why not ... for alternative approaches.)

When to use

So you have some code that you want to be able to run from the command line You can of course just write it down into a file and run it with node ./script.js Maybe you go one more step and add a hashbang and make it executable so on a linux shell you run it with just ./script.js. But this way you can not import the file without executing all the code. Wrapping all the code into a function and executing it if (require.main === module) helps with that. You also manage to parse those arguments you need, maybe using one of the available libraries.

Are you able to also call your function from code with those arguments?
Do you need to make any async call (like…

View on GitHub

The first version has already been published to npm:
https://npmjs.org/package/runex

So if your module exports a method named run it can now be used as a CLI tool:

npx runex path/to/file.js

I still have some ideas for features to implement, but I'm very happy it's out there now.

Merry X-Mas and Let me know what you think

Advent github repo cleanup

Christian Bewernitz — Sat, 07 Dec 2019 21:53:23 +0000

Today I went through the list of my repositories on github to do some cleanup:

I deleted forks that didn't have any (forks or) branches from me that I wanted to keep.

I archived all repositories related to Actionscript/Flash.

It was a bit tedious but it also feels good.

The biggest surprise was when I opened the no longer existing repository karfau/bachelor-thesis, since I expected to see some latex source code but instead it showed the well known instructions for the first commit.

Since it was created at least nine years ago, that might be a new record for a repo not being initialized? (Since there is no trace of it in my activities I can not prove that one.)

I'm quite sure I used git when writing the thesis, but obviously I did never push that code. I wonder if I can still find a copy of it on some old hard drive or CD...

Feel free to share what kind of repos you archived/deleted and what interesting things you (re)discovered while doing that.