ARC Language Module: Building a Governed Multilingual Backend for Future AI Systems

I’m building ARC Language Module, a governed multilingual backend foundation for future AI systems.

The project is not meant to be “just another translator.” It is a language knowledge engine and multilingual control layer that helps an AI system understand:

• what languages it has data for
• what scripts, variants, pronunciation hints, and lineage relationships exist
• what it can actually translate or route today
• what still depends on external providers or future corpora
• what was seeded, imported, changed, reviewed, or left unresolved

The goal is to make multilingual capability visible, inspectable, and honest.

Why this exists

Most language tools specialize in one narrow layer:

• translation endpoint
• offline machine translation
• browser translation
• locale/reference data
• script or formatting data

Those are useful, but future AI systems need something broader.

They need to know:

• what language knowledge they own
• what runtime tools are available
• what support is partial or missing
• which routes are trustworthy
• which data came from which source
• what changed between releases
• what needs to be acquired, reviewed, or expanded

That is the lane ARC Language Module is built for:

not best translator in the world
but a governed language substrate for multilingual AI memory, routing, readiness, and auditability

What ARC Language Module is

Think of it as the brain, filing system, and traffic controller behind a multilingual AI stack.

It provides:

• a structured language graph
• SQLite-backed storage
• CLI operator tooling
• FastAPI API surface
• seeded language records
• scripts and variants
• pronunciation and phonology profiles
• transliteration profiles
• phrase translation seed data
• capability/readiness records
• coverage reports
• policy snapshots
• release evidence snapshots

The important distinction is that the system separates language knowledge from runtime capability.

Knowing a language exists is not the same as being able to translate it, speak it, transliterate it, or route it through a provider.

ARC Language Module models that distinction directly.

What it can do today

The current production-track foundation can store and report structured language knowledge such as:

• language records
• aliases and alternate names
• scripts
• language lineage / family relationships
• variants, dialects, registers, orthographies, and historical stages
• pronunciation profiles
• phonology hints
• transliteration profiles
• seeded phrase translations
• runtime capability and readiness records

It can answer practical operator questions like:

• Which languages are loaded?
• Which scripts are attached to each language?
• Which languages have pronunciation or phonology profiles?
• Which languages have transliteration coverage?
• Which capabilities are production, reviewed, experimental, or absent?
• Which runtime routes are available?
• What changed between releases?

Honest routing

A key idea in ARC Language Module is honest routing.

Instead of pretending every language path is fully supported, the system can route requests through explicit states such as:

• seeded local phrase support
• optional local/runtime providers
• external provider bridges
• not-ready states
• gap states
• missing corpus states

That makes it a language operations layer, not just a translation wrapper.

For AI systems, that matters because false confidence is dangerous. A multilingual backend should be able to say:

I know this language exists.
I have partial metadata.
I have script information.
I do not have enough translation data yet.
This route requires an external provider.
This path is experimental.
This path is production-ready.

That kind of capability boundary is the difference between a toy translation endpoint and a governed AI language substrate.

Architecture

The repo is split into clear layers:

core/      → config, database, models
services/  → language logic, ingestion, routing, policy, evidence, coverage
api/       → FastAPI surface grouped by concern
cli/       → operator entrypoints and handlers
config/    → seed manifests and curated inputs
sql/       → schema and indexes
docs/      → architecture, runtime, policy, onboarding, and comparison docs

This gives the system both application-facing and operator-facing surfaces.

Current release snapshot

The current package snapshot reports:

Version: 0.27.0
Languages: 35
Phrase translations: 385
Language variants: 104
Language capabilities: 245
Pronunciation profiles: 35
Phonology profiles: 35
Transliteration profiles: 21
Semantic concepts: 30
Concept links: 46

Provider support is intentionally modeled separately from core graph truth. Runtime provider availability depends on what is installed, registered, and enabled in the target environment.

Quick start

A typical local setup looks like:

pip install -e .

PYTHONPATH=src python -m arc_lang.cli.main init-db
PYTHONPATH=src python -m arc_lang.cli.main seed-common-languages
PYTHONPATH=src python -m arc_lang.cli.main stats
PYTHONPATH=src python -m arc_lang.cli.main coverage-report
PYTHONPATH=src python -m arc_lang.cli.main system-status
PYTHONPATH=src python -m arc_lang.cli.main build-implementation-matrix
PYTHONPATH=src python -m arc_lang.cli.main release-snapshot

The point is not just to run a server. The point is to inspect what the language backend actually contains and what it can honestly support.

Evidence and release snapshots

ARC Language Module includes release/evidence snapshot concepts so the package can explain what it contains.

A release snapshot can include:

• package version
• version consistency checks
• API health/version integrity checks
• live graph counts
• coverage state
• readiness state
• evidence outputs

That helps turn language infrastructure into something auditable instead of a hidden pile of tables and assumptions.

Where it fits compared to other tools

Different projects solve different problems well.

• Argos Translate is useful for offline open-source translation packages.
• LibreTranslate is useful as a self-hosted translation API.
• Firefox Translations / Bergamot is useful for local browser translation.
• Unicode CLDR is useful for locale/reference data and internationalization.
• ARC Language Module is aimed at the governed orchestration layer: language knowledge, routing, readiness, provenance, and auditability.

The project can sit above or beside translation providers instead of replacing every provider.

What it is not

To keep the claims honest, ARC Language Module is not:

• a universal best-in-class machine translation model
• a finished speech/TTS stack
• a complete transliteration engine for every script pair
• a giant cloud service by itself

It is strongest as a multilingual control layer inside a larger AI product, local-first stack, research runtime, or language-aware system.

Repo

https://github.com/GareBear99/arc-language-module

What I’m looking for

I’m looking for feedback from:

• AI developers
• NLP developers
• localization engineers
• language technology researchers
• multilingual app builders
• Python developers
• FastAPI developers
• SQLite/data-modeling people
• corpus/data curators
• open-source maintainers

Useful feedback includes:

• language graph design feedback
• provider routing ideas
• corpus ingestion ideas
• coverage/reporting improvements
• pronunciation/phonology expansion ideas
• transliteration profile suggestions
• API/CLI design feedback
• release snapshot and evidence improvements
• docs and onboarding issues

Long-term direction

The long-term goal is to make ARC Language Module a governed multilingual substrate for future AI systems.

Not just translation.

Not just locale data.

A language operations layer that can tell an AI system what it knows, what it can route, what it can prove, and what still needs to be acquired or reviewed.