How SmartKey Keeps Two Alphabets Alive Until Context Decides

The input event does not arrive with a language tag.

Press the same two physical keys on a Bulgarian phonetic keyboard and one interpretation is li; the other is ли. Both are plausible prefixes. One can be the beginning of like. The other can already be a complete Bulgarian word.

Most desktop input systems avoid this ambiguity by asking the user to maintain a global mode: English now, Bulgarian later. The engineering is simple. The human cost appears when the mode and the thought diverge. A forgotten toggle turns a sentence into cleanup work.

SmartKey explores a different contract: keep both alphabets alive long enough for evidence to decide.

Its North Star is:

Write the thought, not the keyboard layout.

This article is the technical companion to the project's narrative overview. It focuses on what an input engine designer can learn from the architecture, its current evidence, and the failures that refuse to fit under one convenient label.

1. Start before autocorrect

Autocorrect begins with committed text and asks whether another string was intended. A dual-alphabet input engine has an earlier problem: what characters should the physical keys become in the first place?

SmartKey maintains two candidate readings for the current word. Conceptually, the loop looks like this:

for each eligible physical key:
    extend the English candidate
    extend the Bulgarian candidate
    score both using available local evidence

    if the winner changes before the decision is locked:
        update the visible composing prefix

    if the evidence crosses a lock boundary:
        lock the current winner for further scoring
        keep the word in preedit until the boundary

This is pseudocode, not a copy of the Rust implementation. The useful design idea is delayed commitment. The engine can expose a current best answer in composing preedit without pretending that early evidence is final.

That distinction matters because the shape of evidence changes during a word. At the first character, context may be more informative than corpus frequency. After several characters, a prefix can become distinctive. At a delimiter, the token is no longer merely a prefix; exact-word and phrase evidence become available.

One scoring rule should not be assumed optimal at all three moments.

2. Prefix evidence is not word evidence

The smallest reproducible example is li versus ли.

In the audited corpus snapshot behind this draft, the strongest English completion for li… was represented about 2.57 million times. The strongest Bulgarian completion for ли… was close, at about 2.51 million. A prefix-based contest is therefore nearly balanced and can tip toward English.

But at the exact-word boundary, the evidence is radically different: standalone Bulgarian ли appeared about 2.51 million times, while standalone English li appeared about 18 thousand times.

The numbers are corpus-specific and rounded. Their value is not the apparent precision. Their value is the diagnostic question they expose:

• During the token: “Which candidate has the strongest continuation?”
• At the delimiter: “Which candidate is the word the user just completed?”

If the engine keeps asking the first question after Space arrives, it is solving the wrong problem correctly.

This suggests a general rule for predictive interfaces: boundaries are semantic events. A delimiter, submit action, focus change, or explicit acceptance can reveal evidence that did not exist one event earlier. Test boundary behavior separately instead of treating it as the last iteration of the same loop.

3. Context is an input contract, not application telepathy

It is tempting to summarize context handling as “the keyboard knows it is in a terminal.” SmartKey does not make that claim.

The engine can use three bounded signals:

1. surrounding text that the client exposes;
2. the ContentType the client declares;
3. an inferred typing regime derived from available input evidence.

In the FastCoding regime, technical vocabulary can influence prediction. It is ranking evidence, not a guarantee that every identifier or command will remain untouched.

Sensitive fields have the same contractual boundary. SmartKey bypasses fields whose clients correctly publish a sensitive ContentType. It cannot promise a universal password shield when a client fails to declare that property.

This is an important architectural habit: name the source of context. “The application tells me X” and “I inferred Y from text” have different failure modes, security properties, and test fixtures.

4. Acceptance is a state transition

A visible completion creates another decision boundary. SmartKey supports deliberate acceptance paths such as Tab and Right Arrow, with Escape available to reject.

Space is attractive because it is already the natural end-of-word gesture. It is also dangerous. An input method may have to replace a partial token, commit a completion, preserve its alphabet, and deliver exactly one delimiter without leaking an intermediate state.

The Space-accept path is therefore still experimental. Its feature flag is off, and a newly discovered Unicode-eligibility issue is a release blocker. It is not a shipped capability in this article.

The design lesson is broader than this feature: never model acceptance as a convenient key binding. Model it as a transaction with eligibility, commit, cancellation, and fallback semantics. If eligibility fails, Space must remain ordinary Space.

5. Local prediction still uses a model

SmartKey keeps corpus scoring, contextual ranking, and personal adaptation on the machine. There is no LLM or remote model in the prediction loop.

That wording is deliberate. “No model” would be false: ranking candidates is a model, even when the implementation is compact and inspectable. “No data” would also be misleading: local corpus and personal adaptation are data.

Feedback needs the same precision. Accepted candidates can reinforce useful behavior. Repeated rejections can suppress a suggestion. If an accepted completion is followed by Backspace within 500 milliseconds, the event is recorded as negative feedback. The system does not need to label the user's emotion; it needs a bounded observation that a test can reproduce.

For input software, local processing is not only a privacy feature. It also makes behavior easier to replay against a fixed corpus and exact revision.

6. Debug the decision without logging the sentence

An input engine can be wrong at every keystroke. Plaintext logs would make diagnosis easy and create an unacceptable record of what the user typed.

SmartKey's compromise is a structural per-keystroke receipt without plaintext. In structural mode, it records the input/script class, consume-or-forward verdict, dual-buffer and lock/hypothesis state, action name, payload or commit length, and—on commits—the typed and committed script classes.

These fields are enough to inspect state transitions and dispatched action kinds without saving plaintext. They do not by themselves reveal candidate text or corpus support.

At the combined revision behind this draft, the recorded verification gate passed 496 core tests, focused suites of 9, 5, and 20 cases, and seven native scenarios. Those tests describe one revision; they are not an accuracy measurement and do not erase the Unicode blocker in the experimental Space path.

7. One symptom, three failure lanes

In one diagnostic session, nine tokens in a roughly 45-word Bulgarian instruction surfaced in the wrong form. That is one diagnostic session, not a benchmark or an accuracy percentage.

Two isolated examples teach different lessons:

• no, intended as Bulgarian но, reaches a delimiter while the engine still has a short-word ambiguity.
• statiq, intended as статия, can be pulled toward Latin by strong stat… prefix evidence before the Bulgarian word is complete.

Other observations involve loanwords, product names, abbreviations, out-of-vocabulary forms, punctuation, or ordinary typos. A surface string cannot prove which mechanism produced it. That requires the receipt and an exact replay.

The next design work should therefore be split rather than hidden inside a mega-fix.

Lane A — short words at the delimiter

For ambiguous two- and three-character tokens, evaluate exact-word frequency, language prior, and phrase context when the delimiter arrives.

RED tests should include:

• Bulgarian short words after Bulgarian context;
• genuine English short words after English context;
• a language switch immediately before the token;
• punctuation and end-of-input as boundaries, not only Space.

Lane B — premature prefix lock and unsupported inheritance

Test when a strong completion prefix can lock the wrong alphabet before the intended word has enough evidence. Separately test the case in which the nominal winner has zero support: it should not inherit a previous choice merely because a state variable already has a value.

The exact fallback policy still needs evidence. The important step is to make “no support” an explicit state rather than silently treating it as confidence.

Lane C — OOV, brands, abbreviations, and typos

Loanwords and brand names cross the boundary between preservation and correction. A typo can make both corpus candidates unsupported. Punctuation can belong inside a technical token rather than terminate it.

These cases need their own fixtures and success criteria. Solving Lane A must not be advertised as solving Lane C.

8. What to take into another input engine

SmartKey's validated integration here is Linux with IBus, Bulgarian and English, in the author's own daily use. It is not a Windows or macOS delivery claim, a multi-user field study, or evidence of perfect language selection.

The reusable engineering principles are more important than the platform boundary:

1. Preserve competing interpretations until an irreversible decision is justified.
2. Re-evaluate at semantic boundaries; a complete word is not merely a longer prefix.
3. Distinguish declared context from inferred context.
4. Treat completion acceptance as a transaction.
5. Collect falsifiable structural evidence without storing plaintext.
6. Split failures by mechanism before designing the fix.

A predictive system earns trust when its behavior is not only fast or often correct, but bounded, reversible, and explainable when wrong.

The SmartKey repository is here: https://github.com/RMANOV/smartkey

Canonical project story: https://www.linkedin.com/pulse/fractured-console-two-alphabets-one-stream-thought-ruslan-manov-m3jvf/

Bring a reproducible boundary case. For this kind of engine, the most valuable input is the word that forces the scoring model to reveal what question it was really asking.