milo zhang

Posted on Sep 24 Originally published at jevaimodel.net Fully Autonomous

The Laya Model: Python, Local APIs, and How It Compares with Jev AI

#ai #opensource #machinelearning #python

Laya is an open-weight decision model that turns a piece of state and a set of typed questions into structured answers, such as a category, an ordered score, or a yes-or-no probability.

That makes it different from the chat models most people use. Laya is designed to make bounded judgments for software; it does not write a paragraph explaining every result.

The project has grown beyond one checkpoint. It now includes English, multilingual, and decision-task-tuned variants, plus a router and a self-hostable HTTP server. This guide explains those parts, walks through a local Python example, and compares the tradeoffs with Jev AI.

What Laya is
Three checkpoints, three different jobs
How a decision is represented
Install and run Laya in Python
Expose a local HTTP endpoint
Laya and Jev AI compared
What the benchmark evidence says
Limits to test before production
Further reading

What Laya is

Consider an application receiving thousands of support tickets. For each ticket, its code may need to know the right team, how urgent the issue is, and whether the customer explicitly asked for a refund. A chat model can answer those questions in prose, but the application then has to parse the prose and check whether the result fits its own categories.

Laya starts with the answer shape instead. You give it a state—the ticket and other relevant context—and one or more typed questions. It scores the possible answers and returns structured results with probabilities. Application code can then make the final routing or escalation decision.

In model-card terminology, Laya is a non-autoregressive “System 1” decision model. “Non-autoregressive” means it scores an answer space in a model pass instead of generating a long answer token by token. It is a model for classification, scoring, and routing, not a general chat assistant. The project and its weights are published through Convai Innovations on Hugging Face; the open Python implementation is documented in the Laya repository.

Three checkpoints, three different jobs

“Laya” can refer to a family of checkpoints rather than one interchangeable model. The model card currently describes three variants with different backbones, context sizes, and intended uses:

Checkpoint	Approximate size	Intended use
`convaiinnovations/laya`	421 million parameters	English text; ModernBERT-large backbone; 512-token context
`convaiinnovations/laya-multilingual`	322 million parameters	Multilingual text; mmBERT-base backbone; 1,024-token default context, with longer-context options documented by the project
`convaiinnovations/laya-typed-decisions`	421 million parameters	A ModernBERT-large variant fine-tuned for the typed-decisions benchmark; 1,024-token context

The Python Router is the easiest starting point for general use. It detects language and chooses between its English and multilingual checkpoints. The task-tuned checkpoint is a separate option for its particular workload; do not assume the router automatically selects it for every request. See the current model card for changes to the available checkpoints and configuration.

Open weights make local inference possible, but local does not mean costless. You still need to download the checkpoint, install its runtime dependencies, and supply enough memory and compute for the traffic you expect. The English checkpoint’s weights are listed as Apache-2.0 on Hugging Face; check the license for each artifact and dependency you use rather than assuming every part of an application has the same terms.

How a decision is represented

A typed question tells the model what kind of answer your program expects. Laya’s main primitives are:

choice selects from named options, such as billing, technical, or other.
score estimates a position on ordered criteria, such as routine, time-sensitive, or blocking.
noul returns the probability that a focused yes-or-no proposition is true.

The probabilities describe the model’s output. They are not proof that a decision is correct. Before a threshold controls a customer-facing action, compare results with labeled examples and decide what should happen to uncertain cases.

All questions in one call read the same state. For example, your program can ask for the ticket’s department, urgency, and refund intent together. It can then route the ticket using hard business rules around those judgments.

Install and run Laya in Python

The Python package supports Python 3.10 and newer. Create a virtual environment and install the package using the same interpreter that will run your program. The first prediction may download a checkpoint from Hugging Face; later runs can reuse the local cache.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install laya

On Windows, create and activate a virtual environment with the Python launcher instead. If your machine needs a particular CPU-only or GPU-enabled PyTorch build, follow PyTorch’s installation instructions for your platform before installing Laya. The project’s installation guide keeps the platform-specific commands up to date.

Now define a short support ticket and three separate decisions:

from laya import Router

router = Router()  # Load the needed language checkpoint on first use.

state = {
    "subject": "Duplicate charge",
    "body": "I was billed twice. Please refund the extra payment.",
}

questions = {
    "department": {
        "type": "choice",
        "instructions": "Which team should handle this ticket?",
        "criteria": {
            "billing": "Payments, charges, and refunds",
            "technical": "Bugs and product errors",
            "other": "A different kind of request",
        },
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this request?",
        "criteria": ["routine", "time-sensitive", "blocking"],
    },
    "refund_requested": {
        "type": "noul",
        "instructions": "Does the customer explicitly request a refund?",
    },
}

result = router.predict(state, questions)
answers = result["answers"]
print(answers["department"]["choice"])
print(answers["urgency"]["score"])
print(answers["refund_requested"]["noul"])

The first argument to predict is the context shared by all three questions. Each question has its own type and instructions, and the answer object is keyed by the question IDs you supplied. The output shown here is illustrative: test the model’s actual answers on your own examples before connecting them to actions.

For a known, single-language pipeline, you can load a checkpoint directly instead of using Router. For a backlog of similar records, the SDK also documents batch prediction. Those choices let a team control which checkpoint is loaded and how it uses available memory; they also make it your responsibility to choose an appropriate variant.

Expose a local HTTP endpoint

If other services need to call Laya, the optional serve extra provides a self-hosted HTTP server. Its POST /v1/systemone request format is compatible with the Jev wire protocol, which can make it easier to repoint some existing clients. Protocol compatibility does not make the models, predictions, service policies, or commercial terms the same.

Install the server extra, bind it to your own machine while experimenting, and configure an API key before starting it:

python -m pip install "laya[serve]"
export LAYA_HOST="127.0.0.1"
export LAYA_API_KEY="replace-with-a-private-random-key"
export LAYA_DEVICE="cpu"
laya-serve

Then send the same state and typed question shape to the local endpoint:

curl http://127.0.0.1:8000/v1/systemone \
  -H "Authorization: Bearer $LAYA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "state": {"body": "We were billed twice. Please refund the extra charge."},
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {"billing": "Payments and refunds", "other": "Everything else"}
      }
    }
  }'

The bearer key shown above is a placeholder; set a private value in your own environment. The project’s server example binds to 0.0.0.0:8000, which can make it reachable beyond your machine. Keep a development server on loopback, or configure authentication, network rules, and a proper deployment setup before exposing it. See the self-hosting documentation for the current server options.

Laya and Jev AI compared

Laya and Jev share a decision-shaped interface: state in, typed answers out. Their practical difference is how you obtain and operate the model.

	Laya	Jev through Jev AI Model
Model access	Downloadable open weights; selected artifacts list Apache-2.0	Hosted Jev model; model weights are not offered by this service
Running it	Your laptop, server, or other infrastructure after setup	Managed web playground and API integration
Main setup	Python/PyTorch dependencies, checkpoint download, runtime capacity	Account and API key; no local model server to maintain
Web experimentation	Community demo or a UI you deploy	Sign in and run decisions in the browser for free
API billing	No hosted inference bill when self-hosted; hardware, storage, and operations still cost money	API requests use paid credits; the signed-in web playground remains free
Control	Choose checkpoint and host the service yourself	Let the hosted service manage inference

The hosted option can be convenient when you want to evaluate a workflow before arranging local dependencies. Jev model free lets you try Jev AI Model in the signed-in browser playground; calls from your own application use paid API credits. Jev AI Model is an independent service for accessing Jev, not the model developer’s account or a claim that Jev and Laya are the same product.

Choose Laya when downloadable weights, checkpoint selection, or operating the inference path yourself fits your team. Choose a hosted Jev workflow when you would rather start with a browser tool and API than maintain model-serving infrastructure. In either case, the important engineering work remains: define the answer space, test representative examples, set review thresholds, and keep consequential application rules in your own code.

What the benchmark evidence says

There are useful comparisons, but they answer different questions and should not be collapsed into one ranking.

The Laya project’s benchmark report

The Laya repository reports results on a typed-decisions benchmark. In the project’s reported evaluation, the task-tuned laya-typed-decisions checkpoint scored 0.766 accuracy, while the English base checkpoint scored 0.362 and the multilingual base checkpoint scored 0.352. The same report gives a majority-class baseline of 0.461. In other words, those numbers point to task specialization as a major factor: the tuned checkpoint and the two base checkpoints are not interchangeable evidence for “Laya accuracy.”

The repository also places a published Jev result beside its measurements, but explicitly says it did not measure Jev itself and that sample sizes and prompts differ. Treat that row as context from separately reported evaluations, not a controlled head-to-head. The benchmark report and its limitations describe which checkpoint produced each result and how the tests were run.

A small community comparison

The Hugging Face discussion you referenced describes a separate test using 100 English states and 310 decisions. The author says the questions were byte-identical, and reports results for the base English Laya checkpoint on local CPU versus Jev 1.13 through its API. Across the three small suites, Laya scored 0.800 on triage, 0.883 on guardrails, and 0.833 on moderation; the reported Jev scores were 0.894, 0.950, and 0.989.

That discussion also gives important qualifications: it was one author’s hand-labeled English set, used preset subsets, and did not run Laya’s recommended Router. The author notes wide confidence intervals and that at least one observed gap was within the noise. The reported five-question latency ranges—375–476 ms for local CPU Laya and 885–1,017 ms for hosted Jev—also include different execution environments, so they are not a neutral hardware speed test.

Taken together, these reports are starting points for questions, not a universal winner. The task-tuned checkpoint performs very differently from base Laya on one reported test, while the small community test favors Jev on its particular labeled cases. Your own language, candidate labels, checkpoint, network, hardware, and error costs can change the outcome.

For a useful evaluation, run the same held-out examples and same question definitions through the exact checkpoints or endpoints you plan to deploy. Track incorrect routes, calibration and confidence, latency at your traffic level, infrastructure cost, and how often a person must intervene. Include ambiguous and out-of-scope cases, not only easy examples.

Limits to test before production

A typed answer can still be wrong. A model that returns a valid category or a probability has met the output contract; that does not prove it understood your policy. Preserve deterministic permission checks and review paths around important actions.

A probability needs local validation. The model card discusses temperature fitting and other calibration limits. Measure probability quality on your own labeled examples before using a confidence threshold to approve, block, refund, or escalate work.

Large choice sets need extra attention. The project’s report calls out performance issues when a question has many labels, including a Banking77 comparison. If your category list is large, test it directly or split the choice into a coarse step followed by a smaller one.

Language routing is not magic. Laya’s router uses a lightweight language and script detector. The repository describes cases where short Latin-script inputs may be hard to classify. For multilingual production traffic, verify both the selected checkpoint and its results by language.

Noul labels deserve a sanity check. The model card reports sensitivity to the built-in false and true option labels on some examples. Test clear positives and negatives for the exact wording you intend to ship.

Local hosting moves work to your team. You control the server and can choose what data reaches it, but you must plan authentication, network exposure, model downloads, runtime updates, monitoring, and capacity. A public endpoint without an authentication and network policy can be abused.

Conclusion

Laya is an open-weight family of structured decision models with a Python SDK, language-aware routing, and a self-hosted HTTP option. Jev AI offers a hosted way to try a similar kind of decision workflow. Decide between them by testing the checkpoint and deployment mode you would actually use against your own labeled cases.

Sources checked: 2026-09-24. Product and model details can change; confirm current commands, checkpoint names, and benchmark notes in the linked documentation before deploying.

DEV Community