DEV Community: puffball1567

I Built a Kawaii Native GUI with Nim and SDL3

puffball1567 — Sat, 22 Aug 2026 11:34:31 +0000

Native GUI demos often look like settings panels, administrative dashboards, or collections of standard controls. Those examples are useful, but they leave another question unanswered: can the same native UI foundation express a playful, consumer-facing visual language?

Clay Board Style System v0.5.0 includes a new Kawaii Companion demo built to explore that question. It renders a complete daily-companion concept in an SDL3 window with a soft color palette, mixed Japanese and English text, cards, progress indicators, and a two-head-tall character drawn directly through retained Canvas commands.

There is no browser or WebView behind this screen, and the character is not a PNG, SVG, or 3D model. The demo is assembled from the same public Box, Text, Canvas, Style, layout, and retained-rendering primitives intended for ordinary CBSS applications.

Clay Board Style System is abbreviated as CBSS throughout the rest of this article.

Why visual range matters in native GUI development

A UI toolkit can support rows, columns, text, borders, and input controls while still making every application look structurally similar. That is enough for many tools, but creative applications and consumer-facing desktop software also need control over composition, typography, color, illustration, spacing, and visual hierarchy.

The Kawaii Companion demo is therefore not mainly a test of whether CBSS can draw rounded rectangles. It tests whether its existing primitives can be composed into a coherent visual system that feels different from a conventional native GUI demo.

The screen combines:

a two-column application layout built from retained Box nodes;
Japanese and English text shaped through the configured text engine and font fallbacks;
cards, borders, shadows, gradients, spacing, and rounded geometry expressed through Style values;
a character illustration produced from Canvas drawing commands rather than an imported character asset;
several visually distinct sections that still share one palette and typography system.

This is important for CBSS because the project aims to provide reusable presentation primitives, not a fixed collection of applications that all inherit one visual identity.

Building a kawaii desktop UI without HTML or a WebView

The authoring model is CSS-inspired, but CBSS is not a browser CSS implementation. The demo creates retained nodes and typed style values in Nim. CBSS then resolves style, computes layout, prepares text and paint data, and sends the result through its SDL3 backend.

Nim application
  -> retained Box, Text, and Canvas nodes
  -> typed CBSS Style declarations
  -> style resolution, layout, text, and paint
  -> SDL3 backend
  -> native desktop window

For example, the left companion panel is a normal Box with layout and visual declarations:

let companion = ui.box(uiStyle([
  width(426), height(632), padding(23), gap(11),
  flexDirection(fdColumn),
  decl("background-color", colorValue(white)),
  borderRadius(8),
  borderWidth(1),
  decl("border-color", colorValue(rgba(0.31, 0.20, 0.38, 0.08))),
  decl("box-shadow", shadowValue(
    px(0), px(16), some(px(34)), some(px(-18)),
    some(rgba(0.30, 0.18, 0.40, 0.24))
  ))
]), parent = some(main))

Names such as padding, gap, flexDirection, background-color, and box-shadow make the intent familiar to web developers. Internally, however, these declarations feed CBSS's retained native UI pipeline rather than a DOM and browser rendering engine.

Drawing the character with retained Canvas commands

The character is the most immediately visible part of the screenshot, but it does not require a separate illustration file. The demo constructs a Canvas2D, records drawing commands, and mounts that canvas into the retained UI tree:

let mascot = newCanvas2D()
mascot.drawMascot()

discard ui.canvas(
  mascot,
  uiStyle([width(380), height(378)]),
  parent = some(companion),
  code = "daily-companion-character"
)

drawMascot layers rounded shapes, lines, a quadratic path, solid fills, and Oklab-interpolated linear gradients. The face, hair, clothing, highlights, shadow, and background decorations are all described in code. The result remains a 2D retained drawing; the sense of depth comes from layering, color, and highlights rather than a 3D asset pipeline.

This does not mean every application should generate all artwork procedurally. It shows that custom visual elements can live beside styled layout and text without requiring a separate rendering model for each component.

Mixed Japanese and English typography in a native UI

The screenshot also includes きょうも、いい日。 alongside English headings and supporting text. The demo configures the CBSS text path with font fallbacks before building the interface:

var fonts = initFontRegistry()
fonts.addFallbackFamily("Noto Sans")
fonts.addFallbackFamily("Noto Sans CJK JP")

var cosmic = initCosmicTextEngine(fonts)
ui.configureTextLayout(cosmic.textEngine(), fonts)

This is a small detail visually, but an important application-level test. A styled layout is not convincing if its text path works only for a narrow set of Latin demo strings. The Kawaii Companion screen exercises mixed-script shaping and fallback inside the same retained layout.

Run the Kawaii Companion demo with Nim and SDL3

The complete source is included in the v0.5.0 tag:

git clone https://github.com/puffball1567/clay-board-style-system.git
cd clay-board-style-system
git checkout v0.5.0
nimble setupBundled
nimble kawaiiCompanionDemo

The current package requires Nim 2.2 or newer. Rust and Cargo are also needed to build the native text and image bridges. Linux x86_64 with SDL3 is the Tier 1 runtime target; the portable test suite also covers Windows x86_64 and macOS arm64, while their complete native runtime paths still need broader contributor validation.

The full example is available in examples/kawaii_companion_demo.nim.

A native UI should not decide the personality of the application

The main result of this demo is not that CBSS has a built-in “kawaii theme.” It does not. The result is that application code can construct this visual language from general-purpose native layout, text, Canvas, and style primitives.

Native GUI development should not force every application toward the same visual personality. Kawaii Companion is one more test of whether CBSS can provide a familiar, expressive authoring model while remaining a native UI system rather than embedding a browser.

Perry vs scriptc: Comparing Two TypeScript-to-Native Compilers

puffball1567 — Mon, 17 Aug 2026 16:19:37 +0000

Two open-source projects now make a similar promise: write TypeScript or JavaScript, compile it ahead of time, and ship a native executable without requiring Node.js on the target machine.

Those projects are Perry and scriptc.

Their headlines sound similar, but they are not trying to build exactly the same product. Perry is growing into a cross-platform application platform with native UI, mobile targets, Node-compatible APIs, threading, and native integrations. scriptc is more narrowly centered on compiling ordinary TypeScript while making the boundary between static code, dynamic JavaScript, and unsupported code visible.

That distinction matters more than a benchmark table.

This comparison is based on each project's official repository, documentation, and published test methodology as reviewed on August 18, 2026. Perry was reviewed at commit e53de81, and scriptc at commit a737fde. Both projects are moving quickly, so check their current documentation before making a production decision.

Perry vs scriptc: the short answer

Question	Perry	scriptc
Primary direction	A broad native application platform built around TypeScript	A TypeScript-to-native compiler with an explicit static/dynamic/rejected contract
Frontend	SWC parser, then Perry's HIR and transforms	The TypeScript compiler for parsing and type checking, then a typed IR
Native backend	LLVM code generation and a system linker	LLVM IR compiled by clang; a readable C backend is also available for inspection
Default native path	No JavaScript engine; an optional JS runtime requires explicit opt-in	No JavaScript engine in a static build; `--dynamic` explicitly embeds quickjs-ng
TypeScript model	A practical subset; most type annotations are erased	Type information drives lowering, generic specialization, and checked boundaries
Node.js compatibility	A large native implementation of Node and web APIs, plus package-specific bindings	Supported Node APIs are implemented in the native runtime and tested against Node behavior
npm strategy	Native bindings, selected packages compiled through `compilePackages`, or an explicitly enabled JS runtime	npm package code normally enters an explicit dynamic island; static npm compilation is experimental
Native UI	Built-in declarative UI mapped to AppKit, UIKit, GTK4, Win32, and the DOM	No comparable application UI framework is documented
Cross-platform scope	Desktop, Apple platforms, Android/Wear OS, and Web/WASM	macOS, Linux, Windows, WASI, plus mobile library-mode archives
License	MIT	Apache-2.0

In practical terms, Perry is the more application-oriented choice. scriptc is the more compiler-contract-oriented choice.

What does "compile TypeScript to native" actually mean?

Neither project simply runs tsc and packages the resulting JavaScript. Both lower supported TypeScript or JavaScript constructs into native code and link support code for features such as strings, arrays, asynchronous work, and operating-system APIs.

For Perry, the documented pipeline is:

TypeScript
  -> SWC parser
  -> Perry HIR
  -> optimization and lowering passes
  -> LLVM
  -> object file
  -> system linker
  -> native executable

Perry's architecture documentation also describes its runtime library, including JavaScript value representations, garbage collection, arrays, strings, and UI handles.

For scriptc, the documented pipeline is:

TypeScript
  -> TypeScript compiler: parse and type-check
  -> lowering
  -> typed IR
  -> LLVM IR
  -> clang
  -> native executable

scriptc also retains a readable C backend for debugging and inspection. Its How It Works documentation says that the LLVM backend is the production default and that runtime features are split into link-gated units.

This is an important clarification: "no runtime" in compiler marketing should usually be read as "no Node.js or general JavaScript engine in the normal static output," not "the executable contains no support code at all." Both projects link native runtime components needed to preserve useful JavaScript behavior.

The biggest architectural difference: how TypeScript types are used

Perry uses SWC to parse TypeScript and lowers the result into its own high-level intermediate representation. Its documentation describes support for a practical subset of TypeScript and states that interfaces, type aliases, and most annotations are erased. Perry also documents that annotations do not automatically become runtime validators. These rules are explicit in Perry's supported features and limitations.

scriptc uses the TypeScript compiler itself for parsing and type checking. It then uses the checker's type and narrowing results when building its typed IR. Generics can be monomorphized, unions can become tagged representations, and crossings from dynamic code into statically typed code can be validated at runtime. The project documents this model in How It Works.

That does not mean scriptc accepts every program that tsc accepts. Its limitations page says it uses its own type world based on es2025 plus its ambient declarations, and it deliberately tightens some signatures to match what the compiler can implement. A program may therefore need changes even if it already passes ordinary TypeScript checking.

The trade-off is clear:

Perry owns more of the language-to-runtime mapping and can shape it around a broad native platform.
scriptc treats TypeScript's type information as part of the compilation contract and exposes a stricter boundary around what it can prove and lower.

Static compilation, dynamic JavaScript, and npm packages

Both projects prefer engine-free native code, but neither can statically compile arbitrary JavaScript behavior. Dynamic property patterns, runtime-generated code, and real-world npm packages eventually force a choice: reject the code, port it, replace it with a native implementation, or embed an interpreter.

Perry's npm and JavaScript runtime strategy

Perry offers several paths for dependencies:

built-in or third-party native bindings can map familiar package APIs to Rust functions linked into the executable;
selected pure TypeScript or JavaScript packages can be pulled into Perry's compiler through perry.compilePackages;
JavaScript files that need interpretation can use Perry's optional JS runtime.

The important safety and deployment detail is that the interpreter is not supposed to appear silently. Perry's current JS runtime opt-in documentation describes perry-jsruntime as QuickJS-based and says the build refuses to link it unless the application opts in through configuration, a CLI flag, or an environment variable.

Perry also labels its current compilePackages porting workflow as experimental. Packages based heavily on features such as Proxy, or packages that depend on Node's native addon ABI, may need replacement or a Perry-specific native binding.

scriptc's dynamic island

scriptc divides a program into three visible tiers:

code compiled statically;
code that can run only when --dynamic is enabled;
code that is rejected.

Its coverage command reports how many statements compile statically and identifies dynamic or unsupported sites with diagnostic codes. A static build never silently adds an engine.

When --dynamic is enabled, scriptc embeds quickjs-ng for npm package implementations and any-typed code. Values crossing back into static code are copied and validated. The npm dependencies documentation explicitly warns that CPU-bound dependency code in this island is slower than V8 and that its Node built-ins are shims rather than Node itself.

scriptc also has an experimental --npm-static mode, but its own documentation says package coverage is partial and unsupported sites can remain deferred or prevent static compilation.

The key difference is not that one project has an escape hatch and the other does not. Both do. The difference is how prominently the boundary is exposed. scriptc makes the static/dynamic split a first-class report for each program, while Perry combines native compilation with package bindings, package porting, and explicit interpreter opt-in as part of a wider application platform.

Node.js compatibility: published claims are not the same as guarantees

Perry documents native implementations for a broad set of Node modules and web APIs. Its current supported-features page reports roughly 97% passing results across 53 node:* modules from Node's own test suite, with roughly 95% overall Node/TypeScript compatibility. Those are Perry's published project results, not an independent certification that every Node application will compile unchanged.

The limitations remain significant for some projects. Perry documents restrictions around runtime-generated code, general user-space CommonJS require(), reflection, and dynamic language features. Its package-porting guide also explains why Node-API/N-API, NAN, V8, or libuv-based native addons cannot simply pass through compilePackages.

scriptc takes a different evidence approach. Its repository says the test corpus runs each program under Node and as a compiled binary, then compares stdout, stderr, and exit codes byte for byte. It also reruns the corpus with AddressSanitizer and a reference-count audit. That methodology is documented in both the repository README and How It Works.

However, scriptc does not claim universal Node compatibility. Its limitations documentation lists deliberate behavioral differences, unsupported standard-library surfaces, constraints on any, dense-array behavior, FFI restrictions, and differences inside the dynamic island.

So the fair reading is:

Perry publishes a wide compatibility surface and a high project-reported pass rate.
scriptc emphasizes differential testing and enumerated divergences for the narrower surface it accepts.
neither result means that an arbitrary Node.js application or npm dependency graph is guaranteed to work.

Cross-platform native UI, mobile apps, and WebAssembly

This is where Perry's broader product scope becomes most obvious.

Perry includes a declarative perry/ui framework. Its widgets map to AppKit on macOS, UIKit on iOS, GTK4 on Linux, Win32 on Windows, and DOM elements on the web. Perry's platform overview also covers Android, Wear OS, watchOS, tvOS, visionOS, and WebAssembly-related targets.

scriptc targets macOS, Linux, Windows, and WebAssembly through WASI Preview 1. Its current platform documentation also describes iOS and Android static archives in library mode, intended to be linked into a host mobile application. It does not document a Perry-like cross-platform widget framework.

That creates a straightforward decision boundary. If the goal is to build a native desktop or mobile interface directly from the same TypeScript-oriented platform, Perry is attempting to provide much more of the stack. If the goal is to compile typed application logic, a CLI, a server, a WASI module, or a library core while keeping the host UI separate, scriptc's narrower model may be easier to reason about.

What about TypeScript-to-native performance?

Perry publishes an open benchmark harness and reports both workloads it wins and workloads where Node.js or Bun is faster. That is better evidence than a table containing only favorable cases, and the exact harness is linked from Perry's README.

But those numbers do not answer "Is Perry faster than scriptc?" scriptc uses different runtime representations, a different compatibility contract, and different dynamic fallback behavior. No same-program, same-commit, same-machine comparison between the two projects was performed for this article.

A fair benchmark would need to separate at least these cases:

fully static code in both compilers;
code that activates either project's optional JavaScript engine;
startup time, steady-state throughput, peak memory, and binary size;
correct output and error behavior, not only elapsed time;
the exact compiler commits, optimization flags, toolchain, and operating system.

Until that experiment exists, a performance winner would be speculation.

Which TypeScript native compiler should you try?

Try Perry first when:

native desktop or mobile UI is central to the project;
you want a broad batteries-included platform around TypeScript;
Perry's native modules or package-specific bindings cover your stack;
its real-thread APIs and cross-platform application targets are important.

Try scriptc first when:

you want the TypeScript compiler's type information to drive native lowering;
you need a report showing exactly which parts of your application stay static;
explicit rejection is preferable to silently accepting unsupported behavior;
you are building a CLI, server, WASI module, or embeddable native library without needing a built-in UI toolkit.

For either project, start with a representative slice of the real application rather than a Fibonacci demo. Include the npm dependencies, Node APIs, error paths, asynchronous behavior, target platforms, and deployment constraints that will determine whether the experiment succeeds.

Conclusion

Perry and scriptc share an attractive headline: compile TypeScript to native code and ship without requiring Node.js. Under that　headline, they optimize for different outcomes.

Perry is building outward toward a complete cross-platform native application environment. scriptc is building inward around an inspectable compilation contract: static, explicitly dynamic, or rejected.

That is why the useful question is not simply "Which compiler is faster?" It is "Which project's compatibility boundary and product scope match the application I want to build?"

Primary sources

I Built CSS-Like Native UI Animations with Nim and SDL3

puffball1567 — Thu, 13 Aug 2026 02:03:48 +0000

Can CSS-like keyframe animations and hover transitions work in a native GUI without a browser or WebView? Clay Board Style System v0.4.1, released on August 12, 2026, includes a new SDL3 demo that shows them running in a native UI built with Nim.

Select the preview to open the full MP4 recording.

Clay Board Style System is abbreviated as CBSS throughout the rest of this article.

Why CSS-like animation is useful in native GUI development

The first version of a native UI animation often starts with a timer and a mutable position:

every frame
  -> calculate elapsed time
  -> interpolate a value
  -> update the widget
  -> rebuild what changed
  -> request another frame

That is manageable for one object. It becomes harder when one component combines a hover transition, several named animations, delays, easing functions, alternating directions, cancellation, and reduced-motion behavior.

The host application also needs to distinguish work that changes layout from work that changes only presentation. Moving a box with a paint transform should not require the complete style and layout pipeline to run again on every frame.

CBSS addresses this by keeping motion declarations in the style model and sampling only active tracks when their next frame deadline arrives.

What the Clay Board Style System v0.4.1 animation demo shows

The declarative motion runtime itself landed in v0.4.0 on August 11. Version 0.4.1 is a smaller, intentionally focused release: it adds a polished example, GIF and MP4 recordings, and roadmap documentation for exposing declarative motion through the C ABI in Version 0.5 and later.

The new demo makes four behaviors visible in one native SDL3 window:

automatic horizontal travel with eased keyframes;
an in-place 2D flip expressed with a typed transform;
background color, text color, and opacity animations sharing one retained node;
a reversible hover transition for color, opacity, translation, scale, and rotation.

This matters because an API can look complete in unit tests while still being awkward to compose. The demo exercises multiple tracks, negative delays, alternate directions, inherited text color, hover state changes, paint, hit testing, and the SDL3 host loop together.

Several objects moving smoothly at once is not unusual by itself; mature UI frameworks already do that. The purpose of this demo is to show that CSS-inspired keyframes and transitions can compose inside one retained native UI model, using typed Nim declarations and an SDL3 event loop.

Writing native keyframe animation without a manual interpolation loop

The horizontal movement in the demo is registered as named style keyframes. The application describes the values at four offsets; the motion runtime owns timing and interpolation between them.

ui.registerStyleKeyframes(styleKeyframes("ease-shuttle", [
  styleKeyframe(0, [
    decl("transform", transformValue(translate(px(0), px(0))))
  ]),
  styleKeyframe(0.12, [
    decl("transform", transformValue(translate(px(0), px(0))))
  ]),
  styleKeyframe(0.88, [
    decl("transform", transformValue(translate(px(350), px(0))))
  ]),
  styleKeyframe(1, [
    decl("transform", transformValue(translate(px(350), px(0))))
  ])
]))

The component then associates the animation name with its duration, timing function, iteration count, direction, and fill mode:

uiStyle([
  animationNames("ease-shuttle"),
  animationDurations(2.6'f32),
  animationTimingFunctions("cubic-bezier(0.42, 0, 0.58, 1)"),
  decl("animation-iteration-count", keyword("infinite")),
  animationDirection(adAlternate),
  animationFillMode(afBoth)
])

The syntax is CSS-inspired, but the values are authored through typed Nim helpers. There is no DOM, WebView, JavaScript animation loop, or string selector involved.

Running multiple CSS-like animations on one native UI node

The color panel demonstrates a case that is easy to underestimate: two animations run on the same node with different durations and delays.

uiStyle([
  animationNames("palette-cycle", "opacity-breathe"),
  animationDurations(6.2'f32, 3.1'f32),
  animationDelays(0.0'f32, -0.9'f32),
  animationTimingFunctions("ease-in-out", "ease-in-out"),
  decl("animation-iteration-count", keyword("infinite, infinite")),
  animationDirections(adAlternate, adNormal),
  animationFillModes(afBoth, afBoth)
])

CBSS keeps the tracks independent and applies declaration order when animated values overlap. CSS-like list cycling is supported for animation names, durations, delays, timing functions, iteration counts, directions, fill modes, play states, and composition.

Creating a reversible hover transition in a native GUI

The interactive panel starts with normal style and adds a hover style. The transition declarations describe how the runtime moves between the two states.

transitionProperties("background-color", "opacity", "transform")
transitionDurations(0.42'f32, 0.24'f32, 0.52'f32)
transitionTimingFunctions(
  "ease-out",
  "ease-out",
  "cubic-bezier(0.16, 1, 0.3, 1)"
)

When the pointer enters or leaves the button, CBSS resolves the new state and reconciles the transition once. Active samples then update paint data and, while geometry changes, hit-test data. Reversing the hover state starts from the current presented value rather than visibly jumping to an endpoint.

How the SDL3 animation loop stays event-driven

Declarative motion does not mean rendering continuously when nothing is active. The scheduler computes the next deadline and lets the SDL3 event loop wait until either an input/window event arrives or that deadline is due.

state or input event
  -> resolve style and reconcile motion once
  -> schedule the next active-track deadline

deadline arrives
  -> sample active transitions and keyframes
  -> refresh paint and, if required, hit data
  -> render
  -> wait for the next event or deadline

The important boundary is between layout and presentation. The demo's transforms, colors, and opacity can be sampled through paint and hit updates without running style resolution or layout on every animation frame. Layout still runs when a real layout-affecting change, such as a resize, makes it necessary.

Run the Nim and SDL3 native UI animation demo

The reproducible source for the recording is included in the v0.4.1 tag:

git clone https://github.com/puffball1567/clay-board-style-system.git
cd clay-board-style-system
git checkout v0.4.1
nimble setupBundled
nimble declarativeMotionDemo

The current requirements are Nim 2.2 or newer plus Rust and Cargo for the native text and image bridges. Linux x86_64 with SDL3 remains the Tier 1 runtime target. Windows x86_64 and macOS arm64 are covered by the portable CI suite, but their complete native runtime paths still need contributor validation.

The complete example is available in examples/declarative_motion_demo.nim.

Current limits of CSS-like native UI animation in CBSS

Version 0.4.1 remains a developer preview, so public APIs may change before 1.0.

The current declarative motion path focuses on opacity, foreground and background colors, and typed 2D transforms. Additional values, filters, 3D transforms, and CPU effects remain roadmap work. Reduced-motion handling exists in the Nim runtime, while foreign-language keyframe registration and transition control through the C ABI are planned for Version 0.5 and later; v0.4.1 documents that boundary but does not claim it is already implemented.

Release and demo timestamps

GitHub lists v0.4.1 as a non-prerelease published at 2026-08-12 11:26:51 JST (02:26:51 UTC). The GIF and MP4 demo assets were uploaded to the same release during the 11:26 JST minute. Both URLs in this article are pinned to v0.4.1 rather than the moving main branch.

Implementing CSS ex and ch Units in a Native UI Style Engine

puffball1567 — Thu, 06 Aug 2026 12:57:16 +0000

The devel branch of Clay Board Style System now supports four more font-relative length units: ex, ch, rex, and rch.

For readability, I will refer to Clay Board Style System as CBSS below.
CBSS is a CSS-inspired primitive engine for native GUI toolkits written mainly in Nim. It does not use a DOM or WebView, and it is not intended to reproduce the entire browser platform.

This update began with a small-looking requirement:

decl("width", ex(20))
decl("min-width", ch(12))

Adding four unit tags was easy. Resolving them correctly was not.

Unlike px, these units depend on the selected font. Unlike em, they cannot be calculated from font-size alone. The style resolver needs information from the text engine, but the core layout and style layers should not depend on one concrete font library.

This devlog explains how commit
8629108
addresses that boundary.

The feature is currently on devel and is intended for the v0.4 development line. It is not part of the current v0.3.2 release.

What ex, ch, rex, and rch mean

The distinction between the units matters to the implementation:

Unit	Reference used by CBSS
`ex`	x-height of the current element's selected font
`ch`	advance measure of the `0` glyph in the current selected font
`rex`	root element's `ex` value
`rch`	root element's `ch` value

These definitions follow the font-relative units in the CSS Values and Units specification.

The specification also defines a 0.5em fallback when the x-height or the horizontal 0 advance cannot be determined.

CBSS is CSS-inspired rather than a conforming browser engine, and the current adapter has one implementation difference worth making explicit. The CSS specification defines font-relative lengths without shaping, while the current cosmic-text adapter obtains ch by shaping "0" with the resolved text
configuration. This article describes the CBSS contract as it exists in this commit, not a claim of complete CSS conformance.

That fallback is important for a native style engine. CBSS must still behave deterministically in headless tests, through its C ABI, or when an application has not installed a full text engine.

Why font-relative units cross subsystem boundaries

A normal absolute length can be resolved inside the style system:

20px -> 20 layout pixels

An ex value needs a longer path:

font declarations
  -> selected font face and variation settings
  -> x-height in font units
  -> scale by the computed font size
  -> resolve ex into layout pixels

ch also depends on the font selection, but its reference is the shaped advance of the 0 glyph.

CBSS already has a replaceable TextEngine abstraction. Text shaping is provided by a cosmic-text adapter today, while layout and style resolution remain independent of cosmic-text. Importing the cosmic-text bridge directly into the property resolver would have broken that boundary.

The new implementation therefore introduces a small callback contract instead:

type
  FontUnitMetrics = object
    version: uint32
    xHeight: float32
    zeroAdvance: float32

  FontUnitMetricsResolver = proc(
    style: ComputedTextStyle
  ): FontUnitMetrics {.closure.}

The style resolver describes the resolved font selection. The installed text adapter answers with only the two measurements required for these units. Neither side needs to own the other subsystem.

Why the metrics contract is versioned

FontUnitMetrics carries a contract version even though it currently contains
only two values.

That gives CBSS an explicit compatibility boundary. A future text adapter may be built separately from the style engine, especially when accessed through a native language boundary. Returning structurally similar data is not enough if the two sides disagree about its meaning.

The resolver rejects an incompatible version and validates the fields independently. A non-finite or non-positive x-height does not invalidate a usable zero advance, and vice versa. Each invalid field falls back to 0.5em.

The same fallback is used when no provider is installed:

let halfEm = fontSize * 0.5

FontUnitMetrics(
  version: fontUnitMetricsContractVersion,
  xHeight: halfEm,
  zeroAdvance: halfEm
)

This keeps the default debug engine, headless style tests, and the C ABI deterministic without pretending that the fallback is a measured font metric.

Resolution order is the difficult part

The style resolver cannot ask for current font metrics before it knows the current font. At the same time, a declaration such as this refers to font metrics:

decl("font-size", ex(2))

The implementation resolves a node in stages:

Partition font-size, line-height, other font-* descriptors, and the remaining declarations.
Resolve font-size with the parent font metrics available.
Resolve the descriptors needed to choose the current font.
Ask the metrics provider for the current x-height and zero advance.
Resolve line-height and the remaining properties with the appropriate current and root metrics.
Pass stable root metrics and current metrics to child resolution.

This avoids a cycle and follows the important CSS distinction: font-relative units used in font-* properties resolve from the parent metrics, while ex and ch used in line-height continue to use the element's own metrics.

Root variants need another invariant. rex and rch must continue to refer to the root text style while descendants and independently invalidated subtrees are resolved. CBSS stores the root x-height and root zero advance in the resolution environment rather than recalculating them from each descendant.

Measuring real metrics with cosmic-text

The default full text adapter crosses from Nim into a Rust bridge built around cosmic-text.

For ex, the bridge selects the configured font face, applies variable-font settings, reads the face's x-height through ttf-parser, and scales the result from font units to the computed font size. If the font does not provide a valid x-height, the bridge returns the half-em fallback.

For ch, it shapes the string "0" with the resolved text configuration and uses the resulting layout-run width as the zero advance. If shaping cannot produce a positive finite value, it also falls back to half an em.

The metrics are cached by the resolved font configuration. This matters because style resolution may ask the same question for many nodes. Reopening font data or shaping 0 for every declaration would turn a small relative-unit feature into repeated text-engine work.

The cache is deliberately bounded. In this commit, the text adapter retains at most 128 font-metric entries.

The public units remain typed

The Nim API adds explicit constructors:

ex(2)
ch(3)
rex(1)
rch(8)

Internally, the corresponding unit kinds were appended after the existing public ordinals. This preserves the numeric values of earlier unit tags.

The versioned C ABI follows the same append-only rule. Its ABI version moves from 0x00010007 to 0x00010008, and the four new unit constants are added without renumbering the existing ones.

The C ABI uses the deterministic fallback metrics because a concrete text engine remains an application-side adapter concern at that boundary. A C consumer can therefore use ex and ch, but it should not mistake the fallback for a measurement from a particular installed font.

What the tests protect

The commit adds a dedicated 213-line unit test file and expands the cosmic-text and C-consumer tests.

The test matrix covers:

stable public unit ordinals;
0.5em behavior without a metrics provider;
metrics based on the current font family and size;
stable rex and rch values across descendants;
ex and ch inside line-height;
independent fallback for invalid provider fields;
diagnostics when standalone resolution has no font context;
reuse of root and parent metrics during subtree resolution;
cosmic-text x-height and zero-advance reporting;
agreement between the reported zero advance and measured 0 width;
C ABI construction and layout using the fallback path.

This is more test surface than the four new enum values suggest, but most bugs in relative-unit handling come from resolution context rather than arithmetic.

The larger design lesson

CSS-inspired native UI is not mainly a matter of copying property names.
Familiar syntax carries assumptions about font selection, inheritance, resolution order, fallback, and root-relative state.

For CBSS, the useful design was not to make the style resolver understand cosmic-text. It was to define the smallest versioned question the resolver could ask a text engine:

Given this resolved text style,
what are its x-height and zero-glyph advance?

That keeps the style system replaceable, the text engine replaceable, and the headless fallback deterministic.

The result is a small author-facing feature built on an explicit subsystem boundary:

ui.box(uiStyle([
  decl("font-size", px(18)),
  decl("width", ch(24)),
  decl("min-height", ex(3))
])):
  ui.text("Font-relative native layout")

Repository:
puffball1567/clay-board-style-system

About cosmic-text

This implementation builds on cosmic-text, a pure Rust library for multiline text shaping, layout, rendering, font fallback, and editing.

cosmic-text was created by Jeremy Soller and is developed in the pop-os/cosmic-text repository with contributions from its open-source community. CBSS uses it through a Rust bridge for native text shaping and measurement. The font-relative unit work in this devlog would be much larger without that foundation.

Why Filtered Vector Search Returns Fewer Than K Results

puffball1567 — Thu, 06 Aug 2026 12:36:51 +0000

Vector similarity is rarely the only rule that decides whether a result is useful.

A support assistant may need documents for one tenant and one product version.
A code assistant may need the active repository and branch. An internal RAG system may need to enforce permissions, publication status, language, and a date range before it returns any text to a model.

These constraints are commonly called metadata filters. But adding a filter to an API request does not tell us when the filter is applied or how much vector search work it avoids.

That distinction matters:

A filter can reduce the final result set without reducing the vector search.

This article focuses on that execution boundary. A broader guide to reducing the candidate set is available in
RAG Retrieval Optimization: Reduce Vector Search Before Ranking.

Pre-filtering vs post-filtering in vector search

Consider a search over one million document vectors:

tenant_id = "acme"
product = "billing"
language = "en"
version = "2026.2"
published = true

Suppose only 2,000 vectors satisfy all five conditions. The application asks for the ten most similar eligible documents.

There are several ways a system can execute that request:

Strategy	Simplified execution
Post-filtering	Search broadly, then remove results that fail the filter
Overfetch and post-filter	Search for more than ten candidates, filter them, then keep ten
Pre-filtering	Build an eligible set first, then search within it
Filter-aware ANN search	Use metadata indexes while traversing the vector index
Namespace or partition search	Select a smaller stored subset, then run vector search
Application routing	Route directly to a known tenant, repository, or data neighborhood

All six can expose an API that looks like "vector search with metadata filtering." Their cost and failure behavior are not the same.

Why post-filtered vector search returns fewer than k results

Post-filtering first asks the vector index for its nearest results and then removes candidates that do not satisfy the metadata conditions.

If the unfiltered search returns 50 candidates but only 0.2 percent of the corpus is eligible, many requests may produce fewer than ten valid results.

Some may produce none, even when matching documents exist elsewhere in the index.

Overfetching reduces that risk:

wanted results: 10
initial vector candidates: 500
apply metadata filter
return the first 10 eligible results

But the correct overfetch factor depends on filter selectivity and on how eligible vectors are distributed in the vector space. A fixed multiplier may work for one tenant and fail for another. A retry loop can improve completeness, but it adds work and makes tail latency harder to predict.

Post-filtering is still reasonable when filters are broad, the candidate pool is already small, or occasional short result sets are acceptable. It is a poor default for strict authorization boundaries or highly selective filters.

Weaviate's documentation describes the same two post-filtering risks: an unpredictable result count and the possibility that a restrictive filter leaves no match in the initial vector results. Its implementation uses pre-filtering with an allow list instead.

Weaviate filtering concepts

Pre-filtering changes the eligible search space

Pre-filtering determines which records are eligible before similarity ranking.
A simplified plan looks like this:

authorization filter
  -> tenant and product scope
  -> eligible vector IDs
  -> vector similarity search
  -> optional reranking
  -> context construction

This makes the result contract easier to understand: the nearest neighbors are selected from the eligible set rather than selected globally and checked later.

However, "pre-filtering" does not necessarily mean that the engine performs an exact scan over every eligible vector. An implementation may combine an inverted metadata index with an ANN graph, pass an allow list into graph traversal, choose between exact and approximate search based on cardinality, or use another filter-aware strategy.

The important questions are therefore practical:

Is eligibility established before results enter the top-k set?
Can the vector index traverse efficiently under a selective filter?
Does the engine fall back to an exact scan for some filter shapes?
Which metadata fields require their own indexes?
What happens when the eligible set is empty or extremely small?

For example, Qdrant recommends creating payload indexes for fields used in filters, preferably before ingestion. Its documentation treats fields such as availability, location, and price as business constraints that embeddings do not express reliably.

Qdrant filtering documentation

How filter selectivity changes vector search performance

Filter selectivity is the fraction of the corpus that remains eligible. A filter matching 800,000 of one million records has 80 percent selectivity. A filter matching 2,000 records has 0.2 percent selectivity.

That number can change the best execution plan:

a broad filter may add metadata work without removing many vector candidates;
a selective post-filter may discard almost every ANN result;
a selective pre-filter may make an exact scan of the eligible set practical;
an intermediate filter may need filter-aware graph traversal;
a stable and highly selective scope may be better represented by routing or partitioning.

Do not infer performance from the final result count. Two queries can both return ten rows while one considers thousands more vector candidates. Record eligible cardinality and vector work for every important filter shape.

Metadata correctness comes before metadata performance

Filtering is not only an optimization. Some filters define whether a document may be considered at all.

A useful order for a RAG retrieval pipeline is:

enforce authorization and tenant isolation;
select a stable application scope such as repository, product, or version;
apply dynamic metadata such as status, language, and date;
perform vector or hybrid candidate ranking;
rerank the bounded candidate set;
project fields and enforce the context budget.

Authorization should not depend on whether an ineligible vector happens to miss the top-k cutoff. It should be enforced as a hard boundary independently of relevance ranking.

The metadata model also needs explicit semantics. For example:

Field	Useful question
`tenant_id`	Is this a security boundary, a routing boundary, or both?
`repository`	Can one query intentionally search multiple repositories?
`version`	Should older versions be excluded or merely ranked lower?
`language`	Is fallback to another language allowed?
`published`	Can drafts ever enter model context?
`valid_from` and `valid_to`	Which time is authoritative for the request?

A vector database cannot infer these policies from an embedding. The application still owns the meaning of the fields and the rules for combining them.

Tenant-scoped vector search: filter or namespace?

Namespaces and partitions can reduce the search space before vector ranking, but they are not a replacement for metadata indexes.

A good routing or partition boundary is usually:

known before retrieval;
stable enough that records do not move constantly;
selective enough to exclude substantial unrelated data;
meaningful to authorization or application behavior;
limited to a manageable number of scopes.

Tenant, repository, product, and corpus source often fit. Free-form tags, temporary UI filters, arbitrary price ranges, and frequently changing status values usually fit metadata filtering better.

For multi-tenant vector search, the choice depends on the workload. A namespace or routed partition makes tenant scope explicit and can avoid cross-tenant search work. A tenant_id metadata filter is more flexible when queries may legitimately span tenants or when creating many physical scopes would be operationally expensive.

Neither choice automatically implements authorization. Tenant identity must come from an authenticated context, and every retrieval path must enforce the same policy.

This produces a layered design:

stable scope
  -> dynamic metadata filter
  -> vector or hybrid ranking
  -> reranking
  -> LLM context

The stable scope makes the first problem smaller. Metadata filters express conditions inside that scope. Vector similarity orders the remaining semantic candidates.

How to measure vectors scanned in filtered vector search

Returning ten results does not mean that only ten vectors were considered.
When evaluating filtered vector search, record at least:

total vectors in the corpus;
vectors in the selected namespace or partition;
records eligible after metadata filtering;
vector candidates visited or scored;
results remaining after filtering;
recall at k for the eligible ground truth;
empty-result and short-result rates;
p50 and p95 retrieval latency;
candidates passed to the reranker;
tokens passed to the model.

Test several filter selectivities. A query matching 80 percent of a collection has a different execution shape from one matching 0.1 percent. Also test correlation: eligible vectors may be clustered together in the vector space or scattered across it.

Finally, include a deliberately wrong scope in the evaluation. A narrow query can be fast because it excluded the correct documents. Latency improvement is not useful unless recall and authorization correctness remain intact.

A practical decision checklist

Before shipping vector search metadata filtering, ask:

Which filters are authorization rules rather than relevance hints?
Are filters applied before or after the vector top-k is formed?
Which filter fields have indexes?
How does performance change as filter selectivity changes?
Can a stable namespace or application route reduce the search space first?
Does overfetching hide an incomplete post-filtering design?
Are recall, vectors examined, latency, and context tokens measured together?
What is the fallback when the chosen scope is wrong or empty?

Filtered vector search FAQ

Does metadata filtering reduce vector search work?

Only when the engine uses the filter before or during vector candidate selection, or when the application routes the query to a smaller scope. A post-filter can reduce the returned rows while leaving the original vector search unchanged.

Why does vector search return fewer results than k?

The corpus may contain fewer than k eligible records. If enough eligible records do exist, a post-filter may have removed too many of the initial ANN candidates, or an approximate filtered search may have exhausted its search budget before finding k matches.

Does pre-filtering reduce vector search recall?

Recall must be measured against the eligible ground truth. A correct filter intentionally excludes out-of-scope records, but a wrong filter can exclude the answer. Filtered ANN traversal can also have different recall behavior from an unfiltered graph, so it should be compared with exact search over the same eligible set.

Should tenant ID use a metadata filter or a namespace?

Use a namespace or routing boundary when tenant scope is stable, selective, and almost every query belongs to one tenant. Use an indexed metadata filter when the scope is dynamic or legitimate cross-tenant queries are common. Some systems combine both: route to a tenant-level scope, then apply metadata filters inside it.

A different approach: placement-aware retrieval in KoutenDB

Metadata filtering is not the only way to reduce vector work before ranking.
When a useful search scope is predictable while data is being stored, placement itself can provide the first candidate boundary.

KoutenDB is an open-source document and vector database written in Nim.

KoutenDB does not implement a ring as a conventional metadata filter. A ring is an application-defined locality coordinate chosen when data is placed. The same coordinate can later become the starting scope for retrieval.

For example, documentation for different Laravel releases can remain in separate rings while a stellar lens records that they belong to the same framework context. The lens changes visibility metadata; it does not copy the document payloads between rings:

import koutendb

var db = koutendb.open(dataDir = "data")
db.attachStellar("framework/laravel", "docs/laravel/10")
db.attachStellar("framework/laravel", "docs/laravel/11")

let queryVec = @[1.0'f32, 0.0'f32]
let relatedVersions = db.readStellar("framework/laravel")
let hits = db.retrieve(
  queryVec,
  ring = "docs/laravel/11",
  budget = 8
)

The stellar read keeps the Laravel 10 and Laravel 11 results grouped by their original ring, so the application can see them as related without collapsing their version boundaries. subrings can narrow a stellar read when only one member coordinate is needed.

The current vector retrieve API does not use a stellar lens as an implicit vector filter. After the appropriate member ring is selected, KoutenDB scans the vectors stored in that ring, computes their exact cosine similarity, and returns up to eight results. It does not first run a global vector search and discard results from other versions afterward.

This is placement-aware retrieval rather than general-purpose metadata filtering. Rings preserve stable distinctions such as framework version, tenant, product, language, or document family. Stellar lenses can expose related rings through one read context without erasing those distinctions.
Frequently changing conditions such as publication status or price may still need ring-read filters, application logic, or an indexed metadata filtering system. A ring or lens also should not be treated as authorization by itself;

authentication and access policy remain separate concerns.

The trade-off is explicit. KoutenDB works best when the application, import rule, or operator can express a useful locality while storing the data. If no meaningful locality is known, retrieval without a ring becomes a broad exact scan. KoutenDB also exposes retrieval statistics such as totalVectors,
scanned, skippedVectors, and candidateReduction, making the amount of vector work observable instead of inferring it from the number of returned results.

The goal is not to avoid vector search. It is to make vector search operate on the smallest valid candidate set. Metadata filters determine which records are\ eligible. When a useful scope is known in advance, placement-aware retrieval can make that eligible set smaller before ranking starts.

KoutenDB v0.10.0: 72 Hours, Three Persistent Nodes, and Zero Client Errors

puffball1567 — Mon, 03 Aug 2026 12:23:51 +0000

I released KoutenDB v0.10.0.

Release:

https://github.com/puffball1567/koutendb/releases/tag/v0.10.0

KoutenDB is a ring-oriented document and vector database written in Nim. Its core idea is to make an application's locality boundary part of the read path:
place related data together, select that boundary before retrieval, and avoid making unrelated records candidates in the first place.

That idea needs more than a clean benchmark. A database also has to keep running while it writes persistent state, serves reads, tracks topology state, and eventually shuts down for verification.

For v0.10.0, I ran a local persistent three-node cluster continuously for 72 hours. The result was simple and useful: the run completed with zero client errors, and all three data directories passed offline verification after the servers stopped.

What the endurance run did

The runner started three local TCP nodes with persistence enabled and exercised
a mixed workload every 250 ms for 259,200 seconds.

The workload included:

TCP writes;
point reads using IDs returned by earlier writes;
JSON projection queries;
bounded ring reads with sort and limit;
ring-scoped exact vector retrieval;
metrics collection;
a final snapshot and offline verification after shutdown.

The runner builds its own copy of the server, CLI, and workload binaries under the run directory before it starts. This matters because rebuilding the source tree during a long run cannot silently change the executable being tested.

The exact run used commit 87c755c9130ec0bbf70a3903c75fd2bdae8b084b.

Final result

Operation	Completed
PUT	969,281
returned-ID GET	969,281
projection query	969,281
bounded ring read	969,281
ring-scoped retrieve	96,928
metrics read	48,464
client errors	0

That is 4,022,516 completed logical client operations across the 72-hour period.

After the workload finished, the runner took its final snapshot, stopped the three nodes, and ran offline kouten verify against every persistent data directory. All three verifications succeeded.

The final operational counters also showed:

zero pending handoffs;
zero handoff queue depth;
zero handoff failures, stale acknowledgements, or queue-full events;
zero remaining migrations;
zero universe-sync errors;
zero global retrieval requests.

The last point is important for KoutenDB's design. The workload used ring-scoped retrieval, and the metrics confirmed that it did not silently fall back to a cluster-wide retrieval path.

Why this is more useful than a startup smoke test

A startup test can show that a server accepts a request. It does not show what happens after hundreds of thousands of persistent writes, repeated read paths, periodic metrics calls, topology bookkeeping, and a full stop-and-reopen boundary.

This run specifically exercised the persistent cluster path over time:

records were written to disk-backed nodes;
later requests read data that had been accumulated by the same run;
cluster handoff and migration counters were observed throughout the run;
the servers were stopped cleanly;
the on-disk state was opened and verified offline.

It does establish a concrete baseline: KoutenDB can sustain a mixed, persistent local cluster workload for three days without a client-visible error or an offline integrity failure.

What changed in v0.10.0

The endurance runner is one part of a larger operational hardening release.

Operational configuration can now be loaded and verified before a server is used.
Write guardrails, audit events, capacity thresholds, and backup verification provide explicit operational checks.
Explicit scale-in migration and rolling topology activation add controlled drain, handoff, and progress behavior.
Cluster retrieval now remains in the requested ring rather than falling back to a cluster-wide scan.
Physical placement is decoupled from the logical orbit schedule, so a logical orbit boundary does not stall local request processing.
Handoff I/O is kept off the request loop, and stale handoff replay cannot resurrect an older mutation.
The optional FAISS backend was removed. Vector retrieval now follows the core KoutenDB model directly: select a ring first, then perform exact cosine ranking over that bounded candidate set.

These are not unrelated features. They move the project from a collection of locality experiments toward an operable persistent service: know what the server is configured to do, constrain unsafe operation, observe background state, and verify the data after a long workload.

Reproducing the run

The runner is included in the repository:

KOUTEN_SOAK_SECONDS=259200 \
KOUTEN_SOAK_WORKDIR=/tmp/koutendb-soak-72h \
examples/soak_72h.sh

It writes JSON Lines progress, node logs, final metrics, a final snapshot, offline verification output, the workload configuration, and a completed.ok marker only after all final verification steps succeed.

The full configuration and scope are documented here:

https://github.com/puffball1567/koutendb/blob/main/docs/soak-testing.md

The run is intentionally not a CI job. A three-day endurance test is useful because it is long enough to observe persistent operational behavior; it would make normal pull-request feedback unusably slow.

The next reliability question

This result validates the current local, disk-backed, buffered-durability cluster mode. The next endurance variants are clear: strong-durability writes, TLS and authenticated transport, Docker/container deployment, and eventually multiple machines.

Those tests extend this baseline rather than replace it. The v0.10.0 result is now a documented, reproducible reference point for future changes to storage, handoff, topology, and retrieval behavior.

Source:

https://github.com/puffball1567/koutendb

Clay Board Style System update: Link Navigation

puffball1567 — Mon, 03 Aug 2026 11:52:50 +0000

Clay Board Style System now provides typed native Link Navigation.

For readability, I will refer to Clay Board Style System as CBSS below.
CBSS is only an abbreviation used in this article; the project's official name
is Clay Board Style System.

The feature lets a native application move between screens with a flow similar
to a Single-Page Application: the window remains open, the navigation menu
remains in place, and only the active screen changes.

In this tutorial, we will build a small navigation menu with three destinations:

Home
Projects
Settings

Clicking a menu item switches the visible screen inside the same native SDL3
window. No DOM, browser history, WebView, or URL-string router is required.

Run the finished demo

The complete interactive demo is already included in the repository:

git clone https://github.com/puffball1567/clay-board-style-system.git
cd clay-board-style-system
nimble setupBundled
nimble navigationDemo

The full source is available in
examples/navigation_demo.nim.

Step 1: Define the screens

Start by describing the destinations as a normal Nim enum:

import clay_board_style_system

type Screen = enum
  homeScreen,
  projectsScreen,
  settingsScreen

let navigator = initStackNavigator(homeScreen)

navigator now owns the screen history. The initial destination is
homeScreen.

Because destinations are Nim values, invalid destinations are caught by the
compiler instead of becoming malformed route strings at runtime.

Step 2: Define a small set of styles

The navigation behavior does not impose a visual design. These styles create a
simple sidebar and content area:

proc appStyle(): UiStyle =
  uiStyle([
    decl("width", px(900)),
    decl("height", px(520)),
    decl("flex-direction", keyword("row")),
    decl("background-color", colorValue(rgb(0.04, 0.05, 0.08)))
  ])

proc menuStyle(): UiStyle =
  uiStyle([
    decl("width", px(220)),
    decl("height", px(520)),
    decl("padding", px(20)),
    decl("gap", px(10)),
    decl("flex-direction", keyword("column")),
    decl("background-color", colorValue(rgb(0.07, 0.09, 0.13)))
  ])

proc contentStyle(): UiStyle =
  uiStyle([
    decl("width", px(680)),
    decl("height", px(520)),
    decl("position", keyword("relative")),
    decl("overflow", keyword("hidden"))
  ])

proc screenStyle(): UiStyle =
  uiStyle([
    decl("position", keyword("absolute")),
    decl("left", px(0)),
    decl("top", px(0)),
    decl("width", px(680)),
    decl("height", px(520)),
    decl("padding", px(28)),
    decl("gap", px(12)),
    decl("flex-direction", keyword("column")),
    decl("background-color", colorValue(rgb(0.04, 0.05, 0.08)))
  ])

proc linkStyle(): UiStyle =
  uiStyle([
    decl("width", px(180)),
    decl("height", px(40)),
    decl("padding", px(10)),
    decl("align-items", keyword("center")),
    decl("background-color", colorValue(rgb(0.11, 0.15, 0.21))),
    decl("border-radius", px(6)),
    decl("cursor", keyword("pointer"))
  ])

proc linkTextStyle(): UiStyle =
  uiStyle([
    decl("font-size", px(14)),
    decl("line-height", px(20)),
    decl("color", colorValue(rgb(0.91, 0.94, 0.98)))
  ])

These are ordinary CBSS styles. A component library can replace all of them
without changing the navigation code.

Step 3: Build the menu and screens

Keep the three screen roots so they can be registered with the navigator. We
also keep the Back and Forward buttons for the next step.

type AppView = object
  home: NodeHandle
  projects: NodeHandle
  settings: NodeHandle
  backButton: ButtonHandle
  forwardButton: ButtonHandle
  refreshProjects: ButtonHandle
  notifications: CheckboxHandle

proc buildView(ui: UiRoot; navigator: Navigator[Screen]): AppView =
  ui.box(appStyle()):
    ui.box(menuStyle()):
      ui.text("My application")

      ui.link(
        navigator,
        homeScreen,
        "Home",
        style = linkStyle(),
        textStyle = linkTextStyle()
      )

      ui.link(
        navigator,
        projectsScreen,
        "Projects",
        style = linkStyle(),
        textStyle = linkTextStyle()
      )

      ui.link(
        navigator,
        settingsScreen,
        "Settings",
        style = linkStyle(),
        textStyle = linkTextStyle()
      )

      result.backButton = ui.button("Back")
      result.forwardButton = ui.button("Forward")

    ui.box(contentStyle()):
      ui.box(result.home, screenStyle()):
        ui.text("Home")
        ui.text("Welcome to the native application.")

      ui.box(result.projects, screenStyle()):
        ui.text("Projects")
        ui.text("Project data will be displayed here.")
        result.refreshProjects = ui.button("Refresh projects")

      ui.box(result.settings, screenStyle()):
        ui.text("Settings")
        result.notifications =
          ui.checkbox("Enable notifications", checked = true)

The hierarchy is visible directly in the code:

application
├── navigation menu
│   ├── Home Link
│   ├── Projects Link
│   ├── Settings Link
│   ├── Back button
│   └── Forward button
└── content area
    ├── Home screen
    ├── Projects screen
    └── Settings screen

ui.link() is a style-neutral native Link. It handles pointer activation,
Enter-key activation, keyboard focus, disabled state, and accessibility
semantics.

Step 4: Register the screens

Build the view once, then associate each typed destination with its screen root:

let ui = initUiRoot()
let navigator = initStackNavigator(homeScreen)
let view = buildView(ui, navigator)

let host = initNavigationScreenHost(ui, navigator)
host.registerScreen(homeScreen, view.home)
host.registerScreen(projectsScreen, view.projects)
host.registerScreen(settingsScreen, view.settings)

var interaction = initInteractionState()

if not host.sync(interaction):
  raise newException(ValueError, "initial screen could not be activated")

At this point, only Home is active.

When a Link receives a click or Enter-key event, it updates the navigator. After
the current platform-event batch, synchronize the screen host:

let screenChanged = host.sync(interaction)

In a real application this line belongs in the central event loop, after CBSS
has dispatched the current batch of SDL3 events. The complete demo shows that
integration without hiding it behind tutorial pseudocode.

Only the active screen participates in layout, painting, hit testing, keyboard
focus, and the visible accessibility tree. The other screens remain retained
but inert.

That is the SPA-like part: switching screens does not reconstruct the complete
application tree or open another native window.

Add Back and Forward

The same navigator provides history operations:

view.backButton.onClick = proc(event: DispatchResult): bool =
  if not navigator.back():
    echo "Already at the first screen"
  true

view.forwardButton.onClick = proc(event: DispatchResult): bool =
  if not navigator.forward():
    echo "Already at the latest screen"
  true

push, replace, back, and forward operate on typed history entries. Two
visits to Projects remain two separate entries, which allows CBSS to restore
focus for the correct visit when navigating through history.

Request data with Joubako

Navigation belongs to CBSS, while HTTP communication belongs to the
application's data layer.

I am also developing
Joubako, a separate async transport
client for Nim. It supports HTTP(S), typed JSON, standard Nim await,
result-aware callback composition, WebSockets, local IPC, and optional NIF/BIF
data exchange.

It can be started from a normal CBSS event handler without coupling networking
to the navigation system:

import std/asyncdispatch
import joubako

type Project = object
  id: int
  name: string

let api = newClient(newHttpTransport(), "https://api.example.com/")

proc loadProjects() {.async.} =
  let outcome = await api.getJson("projects", seq[Project])
  if outcome.isErr:
    echo "Request failed: ", outcome.error.msg
    return

  echo "Loaded projects: ", outcome.value.len
  # Update application state and mark the affected view dirty here.

view.refreshProjects.onClick = proc(event: DispatchResult): bool =
  asyncCheck loadProjects()
  true

Joubako is not bundled with CBSS and is not required for Link Navigation. An
application can use Joubako, another HTTP client, IPC, an FFI service, or a
fully local data source behind the same event-handler boundary.

Optional features

The basic menu above is enough for normal in-process screen navigation. CBSS
also provides optional APIs for:

focus restoration for each history entry;
animated screen transitions;
validated application deep links;
replacing one retained screen without rebuilding the application; and
injecting a custom navigation driver.

These features are documented in the
Native Navigation guide.

Why use typed destinations?

A browser URL is useful when the URL is the application's public address. A
native in-process screen does not always need that string boundary.

With CBSS:

navigator.push(projectsScreen)

is checked against Screen by Nim. A larger application can replace the enum
with a variant object that carries typed parameters, such as a project ID.

External deep links can still be decoded into the same destination type at the
edge of the application. Internal navigation remains typed.

Current status

Clay Board Style System v0.3 is a developer preview. Linux x86_64 with SDL3 is
currently the Tier 1 runtime. Windows and macOS portable builds run in CI, while
complete runtime validation on those platforms remains contributor-driven.

Link Navigation is covered by unit tests, retained-screen tests, transition and
focus tests, performance checks, and a real SDL3/Wayland E2E scenario.

Links

Feedback on the Link API and the navigation-menu authoring experience is very
welcome.

RAG Retrieval Optimization: Reduce Vector Search Before Ranking

puffball1567 — Mon, 03 Aug 2026 04:21:03 +0000

Most RAG performance advice begins at the ranking stage: choose a faster embedding model, tune an ANN index, reduce the result count, add a reranker, or cache common queries.

Those are useful techniques. But RAG retrieval optimization should start one step earlier:

Why is this query considering these vectors at all?

A request often already contains a reliable boundary: tenant, repository, product, language, document type, version, date range, permission scope, or the object currently open in the application. Applying that knowledge before vector ranking can reduce RAG latency, vector-search memory use, and unrelated context.

This is the difference between searching a whole corpus for similar documents and searching the authorized, relevant part of that corpus. It is also the specific problem that a locality-aware RAG database such as
KoutenDB explores.

Vector search metadata filtering is not one operation

Vector search metadata filtering commonly restricts a query by fields such as:

tenant_id = "acme"
AND product = "billing"
AND language = "en"
AND published = true
AND version = "2026.2"

Those constraints are necessary for both relevance and security. Their placement in the read path, however, changes the cost:

Strategy	What happens
Filter after broad retrieval	Rank widely, then discard ineligible candidates.
Filter-aware vector index	Use indexed metadata while selecting vector candidates.
Namespace or partition selection	Select an eligible subset, then search it.
Application-local routing	Route directly to a known tenant, source, or related-data neighborhood.

The first approach can still enforce correct access rules, but it may score vectors that should never have been candidates. The latter two are valuable when the application already knows a stable boundary.

For a multi-tenant support assistant, the tenant boundary is not merely a relevance hint. It is an authorization rule and a natural first search scope.
For a code assistant, the active repository and branch often serve the same purpose. For a product assistant, version and language can rule out most of the corpus before similarity becomes useful.

RAG optimization starts with the working set

The working set is everything a request touches: candidate vectors, metadata,payload bytes, reranker inputs, and finally model context. Reducing it changes several costs at once:

fewer vectors to score;
less vector memory to read;
fewer payloads to filter and project;
fewer candidates for a reranker;
less irrelevant context competing for the prompt budget.

This is not an argument for creating a partition for every tag. A pre-ranking boundary should be stable, known at request time, meaningful to the application,and strong enough to exclude large amounts of unrelated data. Tenant, source,product, and document-version boundaries often qualify; free-form tags usually do not.

There is also an essential quality check: a smaller scope is only an optimization if it retains the documents the user needs. A wrong partition can be fast because it is wrong.

KoutenDB: locality first, exact vector ranking second

KoutenDB is an open-source, embedded-capable document and vector database written in Nim. It is not a general replacement for PostgreSQL, a mature ANN vector database, or a global secondary-index engine.

Its vector path makes a focused trade-off:

the application chooses a ring, a semantic locality boundary;
KoutenDB retrieves candidates from that ring;
it performs dependency-free exact cosine ranking over that bounded set.

For example, a RAG application that already knows the tenant and product can place and query documentation in a corresponding ring:

import koutendb

var db = koutendb.open(dataDir = "data")
let hits = db.retrieve(
  @[1.0'f32, 0.0'f32],
  ring = "tenant/acme/product/billing",
  budget = 8
)

The ring is chosen by application logic, policy, or an import rule; KoutenDB does not claim to infer the right security or business boundary from embedding similarity. JSONL ingestion can derive rings from a field, for example with a tenant field and a tenant prefix.

Filters and projections still narrow results, but only after the local ring has been selected. This boundary is important: KoutenDB is a good fit when an application can name a meaningful pre-ranking scope. It is not the right primary tool when every query must perform global, cross-corpus discovery.

Measure vectors scanned, not only results returned

Returning fewer hits does not prove that a system did less search work. A useful RAG database should expose whether unrelated candidates were skipped before ranking.

KoutenDB reports total vectors, scanned candidates, skipped vectors, rings touched, candidate reduction, payload bytes, and estimated tokens. Its included working-set benchmark uses 10,000 vectors across 100 rings. In one local run, global retrieval scanned 10,000 vectors per query, while routed retrieval scanned 100: a 99% reduction. Measured latency in that run was 1,954.9 microseconds for the global path and 31.4 microseconds for the routed path.

An included RAG-style test kept recall at 1.000 for the correctly routed ring while reducing scanned candidates from 400 to 40 and estimated tokens from 615.2 to 231.6. It also demonstrates the failure case: selecting the wrong ring keeps the small scan but produces zero recall.

These are local synthetic measurements, not a universal latency claim or a claim that KoutenDB beats every vector database. They demonstrate a narrower invariant: candidates in unrelated rings are skipped before exact vector scoring, rather than discarded after a broad search.

When locality-first retrieval is useful

This approach is particularly relevant for:

multi-tenant RAG and tenant-based data isolation;
support systems scoped to a customer, product, or deployment;
code assistants scoped to a repository or service;
documentation assistants with known language and version;
local AI or desktop applications with a bounded local corpus;
systems that need nearby application history as well as similar chunks.

It is less suitable when global discovery is the core job, or when the application cannot identify a trustworthy boundary before retrieval. In that case, a broad ANN-oriented vector database with filter-aware indexing is often the better primary retrieval layer.

A checklist for RAG retrieval optimization

Before tuning a model or index, ask:

Which tenant, permission, source, version, or task boundary is known before retrieval?
Is that boundary enforced before candidates are ranked?
How many vectors are scanned, rather than merely returned?
Does scoped retrieval preserve recall on representative queries?
Can the system explain why each document was eligible?
Are payload projection and context budgets applied before the LLM call?
Can the same routing rules be validated offline?

The answer may lead to namespaces, metadata indexes, database partitions, or a locality-aware database. The common requirement is to make the pre-ranking boundary explicit and measurable.

Conclusion

The best way to reduce RAG cost or RAG latency is not always a more complicated ranking algorithm. Often it is avoiding a ranking problem that the application already knows is irrelevant.

Use metadata filtering for correctness. Use tenant, source, version, and task locality to reduce the eligible search space when those boundaries are trustworthy. Then measure scanned candidates, recall, latency, and context size together.

That is the design space KoutenDB explores: a database for RAG where application-level locality becomes part of the retrieval path before exact vector ranking begins.

KoutenDB v0.9.0: From a Locality Experiment to a Testable Database

puffball1567 — Sun, 02 Aug 2026 11:09:04 +0000

I released KoutenDB v0.9.0.

Release:

https://github.com/puffball1567/koutendb/releases/tag/v0.9.0

KoutenDB is a ring-oriented NoSQL document and vector database written in Nim.
Its central idea is deliberately narrow: when an application already knows a useful locality boundary, that knowledge should reduce the data a request has to inspect.

Previous releases established the data model, public API, native C ABI, and language-driver foundation. v0.9.0 concentrates on a harder question: can that locality model be tested against a realistic, persistent related-data read shape rather than only described as a design idea?

This is still a technical preview. It is not a claim that KoutenDB replaces PostgreSQL, Redis, or every document store. The goal of this release is to make the narrower claim measurable and reproducible.

The Request Shape: One Entity, Several Bounded Collections

Many web application detail pages need more than one record. A user page, for example, may need a profile, a few addresses, recent employment entries, preferences, recent orders, and notifications. Each collection has its own limit and sort order.

In a relational database, this can be expressed with several indexed queries or with a JSON aggregate query built from limited subqueries. Both are valid approaches. The question KoutenDB explores is different: if these collections are already known to belong to one user, can that relationship be represented as bounded nearby data from the start?

v0.9.0 models the example as subrings below a user ring:

kouten get --ring=users/<id> \
  --subring=profile,addresses,career,preferences,orders,notifications \
  --subring-limit=profile:1,addresses:3,career:2,preferences:1,orders:10,notifications:5 \
  --subring-rsort=orders:time,notifications:time

This says exactly what the endpoint needs: retrieve six nearby collections, with independent bounds. It is not a general query language and it is not intended to be one. The useful property is that the caller supplies the locality boundary before the read begins.

What Changed in v0.9.0

The release adds a benchmark for that heterogeneous related-data bundle and strengthens the corresponding read path.

readStellar now prepares projection state once and reuses it across subrings.
Stellar reads validate per-subring limits and descending time sorts.
Simple embedded reads with an empty filter, a positive limit, and id or time ordering can use a bounded ring-window read path.
Disk-backed segment reads reuse cached streams instead of repeatedly opening the same segment files.
import-jsonl supports chunked commits through --batch-size=N for larger data imports.

The point of these changes is not an isolated micro-optimization. A locality model only helps if its normal persistent read path remains bounded when the dataset grows.

A Reproducible Related-Data Comparison

The repository now includes a helper that builds fresh temporary KoutenDB and PostgreSQL data directories, loads the same logical user dataset, and measures the related-data bundle:

N=10000 READS=1000 examples/subring_bundle_postgres_bench.sh

One local run on 2026-07-21 used an AMD Ryzen 5 5600H, Linux 6.8, Nim 2.2.10, and PostgreSQL 14.23.

Users	Logical records	System and query shape	Returned records	Read latency
10,000	1,050,000	KoutenDB `users/<id>/*` stellar read with per-subring bounds	22 across 6 rings	196.859 us
10,000	1,050,000	PostgreSQL: six indexed `SELECT` statements	22	515 us
10,000	1,050,000	PostgreSQL: JSON aggregate over indexed limited subqueries	1 JSON bundle	236 us

These numbers describe one machine and one workload; they are not a universal database ranking. PostgreSQL's aggregate form is close to the measured KoutenDB result, and it expresses the result through a different but perfectly
reasonable abstraction. The value of the comparison is to keep the claim honest: KoutenDB's ring and subring model should be judged on workloads where the application can name a local working set.

Testing the Larger Locality Claim

v0.9.0 also records the completed disk-backed effect-validation run on generated data. The validation imports deterministic JSONL corpora and compares broad retrieval with ring-routed retrieval. It reports import time, records scanned, estimated token volume, and retrieval latency.

The standard scale-1000 matrix completed locally with a largest case of 13,500,000 documents.

Workload	Broad scan	Ring-routed scan	Estimated tokens: broad -> routed
`small-balanced`	168,000	24,000	692 -> 260
`near-distractors`	1,860,000	120,000	1,730 -> 433
`medium-noisy`	13,500,000	500,000	2,595 -> 692

The token column is an estimate for the generated retrieval payloads, not a benchmark of any model. More generally, this validation is not limited to AI workloads. Scanned records, transferred data, candidate memory, and downstream application work all increase when a request must inspect unrelated data.

The important result is that the test makes the expected trade-off visible:
locality can reduce the candidate set when the application supplies a valid ring boundary. It cannot invent a useful boundary when the application has none.

Offline Validation Before Production Traffic

The release also adds an offline JSONL path:

KOUTEN_REAL_JSONL=/path/to/export.jsonl \
QUERY_RING=docs/japan \
examples/offline_effect_validation.sh

This is intended for copied or exported data rather than a production service.
It gives a team a way to test whether a proposed ring layout narrows its own workload before asking the database to serve live traffic.

That boundary matters. A database experiment should be reproducible without requiring production credentials, traffic capture, or an unbounded benchmark environment.

Verification Included in the Release

The v0.9.0 release state was checked with Nim module checks, the public API test program, the smoke suite, package validation, and whitespace validation:

nim check src/kouten/store.nim
nim check src/koutendb.nim
nim check src/koutencli.nim
nim check tests/tapi.nim
nim c --nimcache:/tmp/nimcache_kouten_tapi -r tests/tapi.nim
scripts/test_all_smoke.sh
nimble check
git diff --check

The largest generated run remains an explicit manual validation rather than a default CI job. It is valuable precisely because it exercises a scale that a quick smoke test should not pretend to cover.

What Comes Next

v0.9.0 makes the locality hypothesis easier to test with persistent data and a concrete web-style read shape. The next work is about making that evaluation more operable: verification commands, controlled drain and snapshot workflows, backup checks, audit records, and safer topology transitions.

Those are v0.10 development goals, not claims included in v0.9.0.

KoutenDB remains an early project, but the direction is now more concrete:
data locality should be a testable part of a database read path, not only a diagram or a benchmark headline.

Source and release notes:

https://github.com/puffball1567/koutendb

Introducing Clay Board Style System: CSS-Inspired Native UI for Nim

puffball1567 — Sun, 02 Aug 2026 10:59:52 +0000

I am releasing the first developer preview of Clay Board Style System, a
CSS-inspired primitive engine for native GUI toolkits written in Nim.

For brevity, this article refers to Clay Board Style System as CBSS after
this point. CBSS is only an abbreviation used in writing; the project's
official name is Clay Board Style System.

Clay Board Style System starts from a simple observation: Web frontend work is
productive partly because CSS gives many frameworks a shared vocabulary for
layout and visual design. Native GUI development often has to rebuild that
foundation separately for every toolkit.

The project is an attempt to provide that primitive layer for native
applications without embedding a browser or recreating the Web platform.

Repository: https://github.com/puffball1567/clay-board-style-system

What Clay Board Style System is

Clay Board Style System provides a tree of primitive Box, Text, and Image
nodes, along with typed styles, layout, paint commands, hit testing, focus,
input dispatch, retained scrolling, and accessibility semantics.

The authoring model intentionally uses familiar CSS ideas:

spacing, borders, radii, colors, and typography;
row and column layout, alignment, sizing, clipping, and scrolling;
absolute and overlay positioning;
state styling for hover, active, focus, and disabled states.

The aim is that someone who can read a Web design article or a CSS example can
bring much of its layout and visual reasoning to a native application.

ui.box(uiStyle([
  decl("padding", px(24)),
  decl("gap", px(12)),
  decl("background-color", colorValue(rgb(0.10, 0.12, 0.16))),
  decl("border-radius", px(16))
])):
  ui.text("A native UI, styled with familiar ideas")

What it is not

Clay Board Style System is not a browser, a WebView, an HTML renderer, or a
full CSS implementation. It does not aim for DOM compatibility, JavaScript,
browser quirks, or bug-for-bug CSS behavior.

It is also not a finished component library. Buttons, dialogs, menus, inputs,
and editors should be built as compositions on top of the primitive layer.

This boundary matters. The goal is to reuse the useful language of Web UI
design while keeping a native rendering, input, and application model.

Why Nim

The first implementation target is Nim because it can produce small native
applications while remaining expressive enough for UI composition. Clay Board
Style System also exposes a versioned C ABI with opaque handles and fixed-size
value structs. That lets an existing C, C++, Rust, Zig, or Swift core be paired
with a Nim frontend, rather than requiring a rewrite before the UI can be
explored.

My hope is that this creates a practical entry point to Nim: keep the existing
engine or product core, build the frontend in Nim, and adopt Nim more broadly
only where it proves useful.

Current developer preview

Version 0.2 targets Linux x86_64 with SDL3. It includes the primitive
style/layout pipeline, text shaping through cosmic-text, event routing, focus,
text input and IME integration, retained scrolling, navigation state, a
semantic accessibility model, and the C ABI.

The project is still early. Windows and macOS require contributor validation;
platform accessibility transports, animation timelines, full rich-text layout,
and some windowing behavior are not complete. Public APIs may change before
1.0.

This is therefore a release for people who want to evaluate the approach,
build experiments, and help shape the primitive layer—not a claim that every
desktop application should use it today.

Try it

Clay Board Style System requires Nim 2.2 or newer, Rust/Cargo for the native
text and image bridges, and SDL3 development files on the current Linux target.

git clone https://github.com/puffball1567/clay-board-style-system.git
cd clay-board-style-system
nimble setupBundled
nimble test
nimble sdl3Demo

The next work is to make the project's visual promise easier to judge: richer
design demos, practical composite controls, and broader platform validation.
Issues, experiments, and feedback are welcome in the repository.

RAG Database Design: SQL, Full-Text Search, Vector Search, and Context Retrieval

puffball1567 — Thu, 23 Jul 2026 14:38:14 +0000

Retrieval-augmented generation is often introduced with a compact diagram:

documents -> embeddings -> vector database -> LLM

That is a useful starting point, but it can make one part of the architecture
look like the whole system.

Good RAG database design has to cover more than vector storage.

A production RAG application may also need to store original documents,
enforce tenant boundaries, find exact product names, filter by date, remember a conversation, invalidate stale content, combine several data sources, and keep the final prompt within a token budget.

No database becomes a "RAG database" merely by storing embeddings. Different database models solve different parts of retrieval.

This article looks at the roles of relational databases, document databases, caches, full-text search engines, and vector databases in RAG. It then examines a retrieval problem that is harder to express with similarity alone: finding a useful neighborhood of context when the application knows the center, but not the exact boundary of the answer.

Among practical AI use cases, this pattern appears in customer support, business-efficiency AI tools for sales, internal knowledge assistants, AI programming automation, and AI agent development. The same retrieval questions also apply to AI assistant development with internal data, where access control and source freshness matter as much as the generated answer.

How RAG works—and how it differs from fine-tuning

RAG retrieves external information when a request is made, then places selected evidence into the model's context. Fine-tuning changes model weights. The practical RAG vs fine-tuning differences are therefore about when and where knowledge is introduced.

RAG is often easier to update, cite, filter, or remove because the knowledge remains outside the model. Fine-tuning can change behavior, style, or learned patterns, but it is not a convenient replacement for a frequently changing document store.

The main RAG pros and cons follow from that separation. RAG can use current and private sources without retraining the model, but it adds ingestion, retrieval, permissions, evaluation, latency, and operations work. Understanding how AI generation works is not enough: a fluent model can still answer from weak, stale, or unauthorized evidence.

Database types and how to build RAG

A more complete RAG pipeline looks like this:

source data
  -> ingestion and normalization
  -> document and metadata storage
  -> candidate selection
  -> lexical or vector ranking
  -> reranking and context assembly
  -> LLM
  -> answer and citations

Databases can participate at several stages.

Responsibility	Typical data
Source of truth	Documents, records, revisions, users, permissions
Ingestion state	Jobs, checksums, parser versions, failures
Exact filtering	Tenant, category, status, language, date
Lexical retrieval	Keywords, names, identifiers, error codes
Semantic retrieval	Embeddings and similarity scores
Session state	Recent messages, selected sources, cached results
Context assembly	Related records, projections, token-bounded payloads

One database can perform several of these jobs. A small application does not need six databases just because six rows appear in the table. The point is to identify the retrieval job before choosing the storage engine. A useful database comparison begins with those jobs, not with a feature-count table.

When teams research how to build RAG, they often begin with an embedding model or a RAG vector database. A safer order is to identify the source of truth, access rules, update process, exact filters, candidate retrieval method, and evaluation plan first. The vector index is one component of that design.

To make the differences concrete, imagine that we are building a travel assistant for Kyoto. It stores information about landmarks, restaurants, cultural facilities, events, transit, opening hours, reservations, and user itineraries.

PostgreSQL and MySQL: structured truth and exact relationships

Relational databases are often the least surprising place to keep the authoritative application state around a RAG system.

For the travel assistant, a relational database could store:

users and accounts;
bookings and reservations;
saved places and itineraries;
tenant or organization boundaries;
document versions and ingestion status;
permissions and visibility rules;
structured opening hours and accessibility fields.

SQL is especially useful when correctness depends on exact relationships. If a private itinerary belongs to one user, or a document must not cross a tenant boundary, that rule should not depend on an embedding score.

A relational query can select the allowed document IDs first. The RAG pipeline can then rank only those documents.

SELECT document_id
FROM travel_documents
WHERE city = 'Kyoto'
  AND language = 'en'
  AND published = true
  AND valid_from <= CURRENT_TIMESTAMP
  AND valid_until > CURRENT_TIMESTAMP;

PostgreSQL also has JSON, full-text search, and extension support. The pgvector extension can keep vectors beside relational data, which is often enough for an early or moderately sized RAG application.

The important distinction is that adding a vector column does not remove the need to design permissions, document lifecycle, metadata, candidate selection, and prompt construction.

MongoDB and document databases: flexible content records

A document database is a natural fit when an ingested item already looks like a JSON object and different item types have different fields.

A landmark record might contain history, images, coordinates, and opening hours. A restaurant may have cuisine, price range, reservation rules, and dietary options. An event may have a start time, venue, ticket URL, and weather policy.

{
  "kind": "restaurant",
  "name": "Example Cafe",
  "area": "Higashiyama",
  "cuisine": ["Japanese", "Cafe"],
  "features": {
    "indoor": true,
    "vegetarianOptions": true
  }
}

MongoDB documents can
represent that shape without forcing every item type into the same set of columns. Metadata and source text can live together, and nested fields can be indexed when their access patterns are known.

Flexibility does not eliminate retrieval design, however. The application still has to decide which collection, fields, filters, and records should become candidates for the model.

The familiar RDB vs NoSQL differences still matter here. Relational systems center exact relationships and constraints; document databases center flexible record shapes and aggregate-oriented access. For business systems, NoSQL database selection criteria should include consistency, authorization, operational skills, backup, and migration—not only whether incoming data is JSON.

Redis: short-lived state and repeated work

Redis is useful around RAG even when it is not the primary document store.

Common uses include:

conversation session state;
cached retrieval results;
rate limits;
short-lived agent state;
job queues;
deduplication keys;
frequently reused context fragments.

If many users ask similar questions about the same attraction, caching a validated retrieval result may be cheaper than running the complete pipeline again.

Redis also supports persistence and several data structures, but a cache still needs an invalidation policy. Travel information makes this obvious: an answer about an event, temporary closure, or opening time can become wrong even when the cached text is internally consistent.

Redis documentation
describes it as more than a simple key/value cache, but its role in a RAG architecture should still be chosen explicitly.

OpenSearch: words, fields, and filters still matter

Embedding search is not a replacement for lexical search.

Users search for exact place names, train lines, product codes, legal terms, error messages, and quoted phrases. A model may produce similar embeddings for related text, but an exact identifier often deserves a lexical match.

OpenSearch can combine:

full-text queries;
BM25-style lexical relevance;
field filters;
aggregations;
geographic queries;
vector search;
search-oriented dashboards and operational tooling.

For the travel assistant, it could answer questions such as:

Find pages containing "Kiyomizu-dera" in English,
published in the Kyoto corpus, with an opening-hours field updated this month.

That query contains exact text, metadata, and freshness requirements. Those signals should not be reduced to one vector distance.

The OpenSearch documentation
covers both lexical and vector-oriented search, making it a possible single-engine choice when those retrieval modes need to work together.

Qdrant and vector databases: semantic similarity

Vector databases are designed around a different question:

Which stored items are closest to this query in embedding space?

That is valuable when the user and document do not share the same words.

For example, a visitor might ask:

Where can I spend a quiet indoor afternoon learning about local history?

A useful museum description may not contain that exact sentence. Semantic retrieval can still bring it into the candidate set.

Qdrant stores vectors with payload metadata and supports filtered similarity search. A typical request first restricts the eligible points, then ranks the remaining points by vector distance.

metadata filter
  -> approximate nearest-neighbor search
  -> top candidates
  -> optional reranker
  -> LLM context

This is an effective pattern, but it leaves an architectural question: how does the application decide the right filter and search namespace before similarity ranking begins?

Vector database similarity search algorithms, index type, filtering behavior, memory use, and update patterns all affect the result. A RAG vector DB comparison among Pinecone, Qdrant, pgvector, or OpenSearch should use the application's own corpus and filters. Lists of recommended vector databases cannot replace workload-specific evaluation.

RAG hybrid search: BM25, vectors, and reranking

Keyword and semantic retrieval are complementary. A RAG hybrid search pipeline can combine BM25 and vector scores so that exact names and identifiers remain visible while semantically related documents can still be discovered.

metadata and permission filters
  -> BM25 lexical candidates
  + vector similarity candidates
  -> merge and deduplicate
  -> reranker
  -> token-bounded context

For RAG accuracy improvement, rerank integration is often more useful than simply increasing the first-stage result count. A reranker can inspect a small candidate set more carefully, while the first stage remains optimized for recall.

RAG chunk size optimization techniques matter too. Small chunks can improve matching precision but lose surrounding meaning; large chunks preserve context but consume more tokens and may dilute relevance. The right chunk size depends on document structure, retrieval method, and the unit that must be cited.

RAG frameworks such as LangChain and LlamaIndex can connect loaders, embeddings, retrievers, and models. A LangChain RAG setup sample can demonstrate the wiring, but a framework cannot decide the correct authorization boundary, freshness rule, or candidate scope for an application.

An open-source RAG setup for internal search can combine PostgreSQL, OpenSearch or Qdrant, a local model, and an orchestration framework. Self-hosting changes the operations and privacy boundary, but it does not remove the need for the same retrieval and evaluation decisions.

RAG accuracy improvement and evaluation metrics

Retrieval quality and answer quality should be measured separately. Useful RAG evaluation metrics include:

recall and precision at k;
mean reciprocal rank or nDCG;
context relevance and context precision;
groundedness or faithfulness;
citation correctness;
answer completeness;
retrieved tokens, latency, and cost per request.

Tools such as Ragas can help automate parts of evaluation, but no RAG evaluation tool or tutorial can define the correct expected answer for every business workflow. A small, reviewed test set from real questions is still valuable.

RAG hallucination prevention measures should include more than adding extra documents. Stronger measures include current sources, explicit citations, permission checks, conflict detection, an option to abstain, and tests for questions whose answers are not present in the knowledge base.

RAG knowledge base automated updates also need versioning and invalidation. A pipeline should know which source revision produced a chunk, when an embedding must be rebuilt, and how deleted or expired information leaves caches and indexes.

RAG security measures and implementation cost

RAG security measures begin before retrieval. Tenant isolation, source authorization, secret removal, encryption, audit trails, and defenses against prompt injection in retrieved documents all belong in the design. General database security measures still apply even when the database is described as an AI component.

Organizations may also need to align collection and use of data with privacy rules, sector obligations, AI legal regulations, and their own AI ethics guidelines. Retrieval makes data available to a model; it does not create permission to use that data.

A complete RAG cost comparison includes ingestion, embedding generation, database storage, cache, network transfer, reranking, LLM input tokens, monitoring, backup, and engineering time. For a small business, the average AI implementation cost cannot be inferred from the model API price alone.

RAG cost reduction and token savings are closely related to candidate scope.

Passing fewer irrelevant records through retrieval and reranking can reduce database work and LLM input together. For local deployments, local LLM setup and PC requirements are a separate capacity decision, but smaller retrieved contexts can still reduce downstream memory and compute pressure.

Many AI implementation failures happen because a demo is evaluated only on a few successful questions. The practical AI implementation benefits appear when the system also handles stale data, missing answers, access boundaries, cost limits, and failure recovery.

The difficult part is often candidate scope

Consider this question:

I am at Kiyomizu-dera. It is raining, and I have two hours. What should I do
next?

Several retrieval methods can help:

lexical search can find pages that mention Kiyomizu-dera;
geographic search can find coordinates within a radius;
vector search can find semantically similar travel advice;
SQL can enforce opening hours, permissions, and reservation constraints;
a reranker can reorder the resulting candidates.

The challenge is not that any one of these methods is bad. The challenge is deciding what should enter the candidate set at all.

A useful answer may need:

nearby landmarks;
an indoor museum or gallery;
a restaurant that is currently open;
a temporary event happening today;
transit information;
accessibility information;
places already present in the user's itinerary.

The application knows the center: the visitor, Kiyomizu-dera, the current day, and the current situation. It does not yet know the exact boundary of the useful answer.

Semantic similarity is not operational relevance

The most semantically similar document is not necessarily the most useful document for the current request.

A beautifully written article about another Japanese temple may be close in embedding space but unusable within two hours. A short transit notice may be semantically distant from a tourism question but essential to the answer. A restaurant record may contain almost none of the words in the user's request, yet be relevant because it is nearby, open, and compatible with the itinerary.

RAG systems commonly address this with metadata filters:

{
  "city": "Kyoto",
  "area": "Higashiyama",
  "openNow": true,
  "weather": "rain",
  "categories": ["restaurant", "museum", "landmark"]
}

This is reasonable. But as the product grows, the filter becomes an
application-level retrieval plan. Someone must maintain the rules, namespaces, joins, and fan-out reads that reconstruct the context for every request.

Geographic distance is not the complete boundary either

If the question is simply "what is within one kilometer?", a geographic index such as PostGIS or OpenSearch geo search is the direct solution.

Useful travel context is not always a circle, however.

A river, hill, or railway can make two close points inconvenient to reach.
Rain changes which facilities are useful.
A family and a solo traveler may need different neighborhoods.
Opening hours and event dates change the available context.
Editorial groupings such as "rainy-day Kyoto" are not geographic regions.
One place can belong to an area guide, a station guide, and a personal itinerary at the same time.

Distance remains an important signal. It is simply not the only relationship that defines the field of view.

The application keeps rebuilding context

A conventional stack can solve this problem. It may use SQL joins, geographic indexes, search filters, a graph, application-maintained ID lists, and vector ranking.

The repeated work is reconstructing the same locality:

start from the current place
  -> identify the area
  -> find allowed categories
  -> join today's events
  -> add relevant transit
  -> add itinerary items
  -> build candidates
  -> rank candidates

The missing operation here is not another scoring algorithm. It is a reusable way to answer this earlier question:

What belongs in the neighborhood of this known center for this kind of
request?

Treat the reusable neighborhood as data

One design option is to store that neighborhood instead of reconstructing it from scratch for every request.

This does not automatically require a new database. An application can model the relationship with SQL junction tables, graph edges, materialized views, search documents, or maintained ID lists. The important change is conceptual:
candidate scope becomes data with its own lifecycle, rather than temporary query logic assembled immediately before ranking.

For the tourism example, a reusable neighborhood needs a few properties:

each place should keep one canonical record;
the same place should be able to appear in several contexts;
adding a place to a guide should not copy its payload;
reads should be bounded by category, depth, filters, limits, and token cost;
lexical or vector ranking should remain available after the neighborhood is selected.

If this relationship is occasional, application code may be enough. If the same pattern appears across users, places, projects, products, or time windows, it can be useful to make locality a first-class retrieval primitive.

How KoutenDB represents this retrieval pattern

KoutenDB is one database designed
around that pattern. It uses coordinate-like ring placement for canonical records and stellar visibility lenses for reusable views across those coordinates.

An importer or application can keep tourism data in canonical rings:

landmarks/kyoto/kiyomizudera
landmarks/kyoto/yasaka-pagoda
restaurants/kyoto/gion
culture/kyoto/national-museum
events/kyoto/2026-07-23
transit/kyoto/higashiyama

It can then attach useful coordinates to an area-centered view:

kouten stellar attach \
  --stellar=travel/kyoto/higashiyama \
  --ring=landmarks/kyoto/kiyomizudera

kouten stellar attach \
  --stellar=travel/kyoto/higashiyama \
  --ring=restaurants/kyoto/gion

kouten stellar attach \
  --stellar=travel/kyoto/higashiyama \
  --ring=culture/kyoto/national-museum

kouten stellar attach \
  --stellar=travel/kyoto/higashiyama \
  --ring=events/kyoto/2026-07-23

The application can read the surrounding context from the known center:

kouten get --stellar=travel/kyoto/higashiyama

Or narrow the same field of view to one category:

kouten get --stellar=travel/kyoto/higashiyama --subring=restaurants
kouten get --stellar=travel/kyoto/higashiyama --subring=culture

The same restaurant can also be visible from a rainy-day guide, a
station-centered guide, or a user's itinerary without copying its canonical payload.

KoutenDB does not infer geographic or editorial relevance from a path name.

The application, importer, or curation process still decides which coordinates belong in each neighborhood. The database preserves that decision so later reads do not have to reconstruct it from scratch.

This is more than a path prefix

A ring name deliberately looks like a readable path. Hierarchy is useful when parent and child data naturally belong together.

A path prefix alone, however, represents one tree. Real records participate in several views at once:

canonical catalog: restaurants/kyoto/gion/example-cafe
area guide:        travel/kyoto/higashiyama
weather guide:     travel/kyoto/rainy-day
personal plan:     travel/users/123/today

Copying the restaurant under every path creates synchronization work. Moving it to one path weakens the other views. A stellar lens instead stores a visibility relationship across existing coordinates.

The caller supplies a center and a cost boundary. Depth, branch budget, subrings, filters, projections, sorting, and limits control how much context is returned. The caller does not have to enumerate every individual record in the answer.

Locality can precede vector ranking

Locality-aware retrieval and vector search are not mutually exclusive.

A combined pipeline can look like this:

known center: user + place + time
  -> retrieve the configured local neighborhood
  -> apply current constraints
  -> vector-rank the smaller candidate set if needed
  -> rerank and enforce a token budget
  -> LLM

The vector database asks which candidates are semantically close. The locality layer asks which part of the data should be considered first.

This can reduce the number of payloads loaded, vectors compared, records reranked, bytes transferred, and tokens considered downstream. It does not guarantee that every query becomes faster. The model is useful only when the application can express meaningful locality.

Match the database to the retrieval question

The database decision becomes clearer when it starts from the question rather than the current RAG trend.

Retrieval question	Natural starting point
Which exact record or relationship is valid?	Relational query and indexes
Which JSON documents match known fields?	Document database or SQL/JSON
Which result can be reused briefly?	Cache or session store
Which documents contain these words?	Full-text search engine
Which documents are semantically similar?	Vector search
Which places are inside a literal radius?	Geographic index
Which context is useful around this known user, place, project, or time?	Locality-aware neighborhood retrieval

These are not exclusive choices. PostgreSQL may remain the source of truth while OpenSearch handles lexical retrieval. A vector database may rank semantically similar chunks. KoutenDB may be used as a locality-aware document and retrieval store, or as an earlier candidate-selection stage.

A smaller system may choose one database that covers enough of these roles. A larger system may separate them. Additional infrastructure is justified only when it removes a measured retrieval, correctness, or operations problem.

Practical database design steps for RAG are therefore:

identify the authoritative source and update owner;
define tenant, user, and document access boundaries;
list exact, lexical, semantic, geographic, and locality-aware queries;
choose the smallest set of database types that covers those queries;
measure retrieval quality, tokens, latency, and failure cases;
document database backup methods and database migration steps for the data that cannot be rebuilt;
use database performance tuning only after the candidate and access model are correct.

The final question is what the application already knows

RAG architecture often begins by asking which embedding model or vector database to use.

Another useful question comes first:

What does the application already know before retrieval begins?

It may already know the authenticated user, tenant, order, project, document group, current place, time window, or active task. That information can define a much smaller and more relevant starting region than the complete corpus.

If the request begins with no meaningful center, global lexical or vector search may be exactly right. If the request begins with a known center but an uncertain context boundary, preserving locality can keep the retrieval problem smaller before ranking and prompt construction begin.

The important choice is not SQL versus vectors, or one database product versus another. It is deciding whether the application needs exact truth, lexical matching, semantic similarity, geographic proximity, or a reusable neighborhood of context—and then giving each retrieval stage the data model it actually needs.

That is the foundation of a RAG database design that can grow beyond a demo.

Why KoutenDB Exists: You Know the Center, Not the Boundary

puffball1567 — Wed, 22 Jul 2026 08:04:06 +0000

Most database queries begin by describing the answer.

You provide a primary key, write a SQL predicate, select a collection, enter
search terms, or send an embedding to a vector database. The database then
finds the records that match.

That model works extremely well, but it is not the shape of every application
request.

Sometimes the application knows where the answer should begin, while the
complete boundary of useful information is still unclear.

For example:

a support request starts with a customer, but may need their profile, recent orders, the product involved, previous tickets, and a refund policy.
an incident investigation starts with a service and a point in time, but may need a nearby deployment, error logs, metrics, and related service events.
a RAG request starts with a project or topic, but the useful context may span documents, summaries, source metadata, and attached knowledge.
a game request starts with a player or place, but may need nearby entities, inventory, quests, and regional state.

In each case, the center is known. The exact context boundary is not.

That is the problem KoutenDB was created to explore.

Exact lookup is not the same as context retrieval

Suppose an API receives an order ID.

Finding the order itself is easy. Almost any database can do that efficiently.
The harder question is what else should travel with it.

Should the response include the customer? The shop? Recent payment events? The specific product documentation? An unresolved support ticket? The answer may depend on the request, product, current state, and amount of context the caller can afford to process.

A conventional application can assemble that context with:

SQL joins;
several document lookups;
application-maintained ID lists;
a graph traversal;
metadata filters;
full-text or vector search;
a cache built for one response shape.

These are all valid solutions. But they usually require the application to reconstruct the neighborhood every time it reads.

KoutenDB asks a different question:

What if the database preserved useful locality when the data was written, so
a later read could begin from that locality directly?

A ring is a coordinate, not just a container

KoutenDB stores a record with a coordinate-like ring.

users/123
users/123/orders
users/123/support
shops/1123
orders/A-001
products/sku-9/docs

The names are readable, but readability is not their main purpose. A ring is part of data placement and part of the read path.

If the application starts from users/123, KoutenDB can inspect that local hierarchy without beginning with an unrelated global collection. If it starts from orders/A-001, it can use an order-centered view instead.

This turns locality known by the application into retrieval information the database can retain.

The same data can be visible from another center

Related context does not always fit one permanent hierarchy. An order belongs to a customer, a shop, and a product at the same time.

KoutenDB therefore has a stellar lens. It can attach existing rings to a useful center without copying their payloads.

kouten put --ring=users/123 \
  --payload='{"kind":"user","name":"Alice"}' --codec=json

kouten put --ring=shops/1123 \
  --payload='{"kind":"shop","name":"Orbit Store"}' --codec=json

kouten put --ring=orders/A-001 \
  --payload='{"kind":"order","orderNo":"A-001","total":42}' --codec=json

kouten stellar attach \
  --stellar=commerce/order/A-001 --ring=users/123

kouten stellar attach \
  --stellar=commerce/order/A-001 --ring=shops/1123

kouten stellar attach \
  --stellar=commerce/order/A-001 --ring=orders/A-001

Now the application can read from the order as the center:

kouten get --stellar=commerce/order/A-001

The original records remain in their original rings. Attaching or detaching a ring changes what the lens can see; it does not duplicate or delete the payload.

This is useful when the same information needs to participate in several context views without turning each view into another copied document.

Example: retrieving useful information around a tourist destination

Travel information makes the center-versus-boundary problem easier to see.

Imagine that a visitor is looking at Kiyomizu-dera in Kyoto. The application knows the starting point, but a useful response probably should not contain only the temple record.

The visitor may also need:

other nearby landmarks;
restaurants and cafes;
museums, galleries, and cultural facilities;
events taking place that day;
transit information;
accessibility or opening-hour information.

There is no single permanent boundary for that context. A family, a solo traveler, and someone planning a rainy afternoon may need different parts of the same surrounding information.

An importer or application can keep each place in a canonical ring:

landmarks/kyoto/kiyomizudera
landmarks/kyoto/yasaka-pagoda
restaurants/kyoto/gion
culture/kyoto/national-museum
events/kyoto/2026-07-22
transit/kyoto/higashiyama

It can then attach the useful coordinates to an area-centered stellar lens:

kouten stellar attach \
  --stellar=travel/kyoto/higashiyama \
  --ring=landmarks/kyoto/kiyomizudera

kouten stellar attach \
  --stellar=travel/kyoto/higashiyama \
  --ring=landmarks/kyoto/yasaka-pagoda

kouten stellar attach \
  --stellar=travel/kyoto/higashiyama \
  --ring=restaurants/kyoto/gion

kouten stellar attach \
  --stellar=travel/kyoto/higashiyama \
  --ring=culture/kyoto/national-museum

kouten stellar attach \
  --stellar=travel/kyoto/higashiyama \
  --ring=events/kyoto/2026-07-22

The application can now begin with the Higashiyama area and retrieve the attached tourism context in one read:

kouten get --stellar=travel/kyoto/higashiyama

Or it can narrow the same view to restaurants or cultural facilities:

kouten get --stellar=travel/kyoto/higashiyama --subring=restaurants
kouten get --stellar=travel/kyoto/higashiyama --subring=culture

The same restaurant can also be attached to a travel/kyoto/rainy-day lens, a station-centered lens, or a day-plan lens without moving or copying its canonical payload.

KoutenDB does not automatically infer geographic relevance from these names.
The application, importer, or curation process still decides which coordinates belong in the neighborhood. The database's role is to preserve that decision and make it directly readable later.

This is the intended meaning of surrounding information: the caller starts from a useful place, receives several categories of nearby context together, and can narrow the field of view without having to list every individual place in the request.

Is this just a path?

A ring such as landmarks/kyoto/kiyomizudera certainly looks like an ordinary path. KoutenDB deliberately uses readable hierarchical names because paths are a useful way to express parent and child locality.

But a path by itself provides only hierarchy.

If all tourism data lived under one tree, an application could read a prefix
such as travel/kyoto/higashiyama/*. That works when every useful relationship fits the same permanent hierarchy and the caller already knows which branches to request.

The difficulty is that real data belongs to several contexts at once:

a restaurant belongs to its canonical restaurant catalog;
geographically, it belongs to an area;
operationally, it may belong to a currently open list;
editorially, it may belong to a rainy-day guide;
personally, it may belong to one visitor's itinerary.

Duplicating the same record under every path creates synchronization and consistency work. Moving it into one path makes the other views harder to express. Storing only path strings leaves the application responsible for maintaining and joining all of those relationships.

KoutenDB adds two things beyond the path syntax:

the ring hierarchy is understood by the retrieval engine and can be traversed with explicit depth, branch, filter, limit, sort, and projection controls.
stellar metadata can connect rings from different hierarchies into another coordinate-centered view without changing their canonical paths or copying their payloads.

The stellar lens is therefore not another directory. It is a stored visibility relationship across existing coordinates.

The same ring coordinate can also act as a boundary for authorization, dump/import, placement, and synchronization. A plain path string has none of those database semantics unless the application builds them separately.

So the path is the readable address. The KoutenDB-specific part is how that address participates in placement, neighborhood retrieval, and reusable views across otherwise separate branches.

Not knowing the boundary does not mean an unbounded query

"The caller does not know the exact boundary" should not mean "read
everything."

The caller still controls the field of view. KoutenDB can narrow a read with:

a subring;
hierarchy depth;
branch budget;
per-ring limits and sorting;
filters;
field selection;
an overall result limit.

For example, the same order-centered view can request only shop data or only a small projection:

kouten get --stellar=commerce/order/A-001 --subring=shops

kouten get --stellar=commerce/order/A-001 \
  --selection='{ kind name orderNo total }'

The distinction is important:

The application supplies a meaningful center and a cost boundary. It does
not have to enumerate every record relationship that may form the answer.

This is why KoutenDB is intended for surrounding information rather than only exactly bounded queries.

Why not use a collection or partition key?

Collections, namespaces, partitions, and shards can already reduce the amount of data a database examines.

The difference is what they represent.

A partition key commonly answers questions such as:

Where is this record stored?
Which node owns it?
Which partition can be pruned?

KoutenDB also wants locality to answer:

What information is likely to be useful around this record?
Which nearby rings should enter the current field of view?
Can the same existing ring become visible from another useful center?

That makes placement an application-visible retrieval concept, not only an internal scaling decision.

Why not begin with vector search?

Vector search answers a valuable question:

Which items are semantically similar to this query?

But similarity is not the same as operational relevance.

A customer's latest order, a permission record, and the applicable refund policy may be necessary context even when their embeddings are not the closest matches in a global corpus. Systems often compensate by adding namespaces and metadata filters before vector ranking.

KoutenDB makes that first narrowing step the main data model:

begin from a known user, order, tenant, topic, place, or time coordinate;
open the useful local neighborhood;
apply filters, vector ranking, reranking, or an LLM only to that smaller working set when needed.

Vectors remain useful. They are optional rather than the only way to discover related context.

For local AI and RAG systems, that distinction can matter. The expensive work is not limited to the final LLM call. Loading candidates, comparing vectors, reranking chunks, transferring payloads, and filling a context window all have a cost. A smaller first-stage working set can reduce work throughout the pipeline.

Why not keep this entirely in application code?

An application can maintain arrays of related IDs, write join tables, perform fan-out reads, and build cached response documents. Many applications already do.

The tradeoff is that locality becomes scattered across code and supporting infrastructure. One rule determines reads, another determines authorization, another controls exports, and another decides how data should be copied or synchronized.

KoutenDB's larger design goal is to let the same coordinate participate in:

retrieval scope;
authorization scope;
dump and import boundaries;
placement and compaction;
delayed synchronization between topologies.

The value is not just fewer lines in one query. It is keeping the application's concept of locality visible across the data lifecycle.

The database cost should follow the working set

Let the complete corpus contain N records, while the useful neighborhood for one request contains k records.

Many systems eventually introduce a first-stage index or filter so later work does not operate over all N. KoutenDB's central bet is that meaningful placement can provide part of that first stage directly.

The goal is to make more of the request cost follow k, the useful local working set, instead of repeatedly rediscovering locality from the full corpus.

This is not an automatic performance guarantee. It depends on whether the application can express useful locality. If all queries are arbitrary and global, there may be no meaningful center to exploit.

But many applications already begin with one:

an authenticated user;
a tenant;
an order;
a product;
a project;
a document group;
a region;
a service and time window;
an entity in a simulation.

KoutenDB makes that starting knowledge part of the database query model.

When this model is useful

KoutenDB is worth examining when all three of these statements are true:

requests naturally begin from a known coordinate;
the application frequently needs surrounding context, not only one exact record;
the useful context can be placed or attached meaningfully before it is requested.

It is less relevant when the main workload is arbitrary full-corpus analytics, global full-text search, independent lookup on every field, or strict relational constraints. Those problems already have databases built around them.

The reason for another database is not that existing databases forgot how to look up records.

It is that a growing class of application and AI requests does not begin with a complete description of the answer. It begins with a center.

KoutenDB exists to make that center useful.

The implementation, data-model demo, and reproducible benchmarks are available in the KoutenDB repository.