Hey đ, Emmanuel here. Unless you've been living under a rock, youâve probably heard of Jev (a new AI decision model from TypeSafe AI). It was launched in September, and although my feeds were packed with news about it, I still had no idea what exactly it was or how it differed from standard Large Language Models (LLMs) like ChatGPT and Claude.
Well, until now, that is. Yep, Iâve done some research on TypeSafe AIâs "Jev," focusing mainly on what makes it different from standard LLMs. Could this actually change how we build AI-powered applications, or is it just another model being hyped?
What is Jev?
According to TypeSafe AI, Jev is a specialized non-generative AI decision model. Note that I refer to it as ânon-generative,â which is due to the fact that it doesnât write prose, chat, or generate text. Rather, Jev is a decision model that simply evaluates unstructured input data against predefined questions and returns structured probabilistic decisions. Unlike the traditional LLMs we are used to, Jev is a pure System One model.
System One is a conceptual method of thinking that was introduced by psychologist Daniel Kahneman in his book Thinking, Fast and Slow. Rather than slow, step-by-step reasoning, System One refers to a mode of thinking thatâs fast, intuitive, effortless, and automatic.
With Jev, you don't get the usual back-and-forth conversation or a lengthy explanation containing the information you need. Instead, you get structured decisions, probability distributions, and confidence scores, all in a structured format/payload.
Jev vs. LLMs: What's the Difference?
Now, as Iâm sure youâre wondering, whatâs the difference? I mean, Jev being a System One model sounds cool and all, but how does that compare to the standard LLMs we know?
Firstly, letâs talk about the primary output. With regular LLMs, you get open-ended natural language, responses, code, or something similar. The response is in natural-language text and is generated sequentially by the model token-by-token.
With Jev, however, you get zero natural-language text. Instead, the decision model returns fixed, type-safe (no wonder the company name) structures. These may be predefined choices, numerical scores from 0 to 1, or simply Yes/No probabilities.
Next, the speed. LLMs usually operate autoregressively, as each generated token depends on the previous ones in a loop-like sequence. As expected, this method of token generation is relatively slow, taking from a fraction of a second to several seconds or minutes, depending on the operation.
On the other hand, Jev is non-autoregressive and evaluates independent decisions in parallel in a single pass. Instead of writing out a paragraph or parsing JSON, Jev directly maps the input state against predefined choices, scores, or probabilities in one unified evaluation. Hence, itâs quite fast (40x to 200x faster), taking around 70-500 ms (at least according to reports from TypeSafe AI).
Lastly, and perhaps most importantly, cost. Standard LLMs charge for both input tokens that are processed and output tokens that are generated. For instance, a lightweight model like GPT-4o Mini offers a pricing structure of about $0.15 per 1M input tokens and $0.60 per 1M output tokens ($0.15 / $0.60).
Now, as Iâve mentioned quite a few times already, Jev doesnât generate output tokens, so thatâs a part of the pricing you donât have to worry about. For inputs, you get charged $0.042 per 1M tokens processed, making the final pricing structure about ($0.042 / $0.00).
Let's Put It to Work
One great way to truly understand how Jev works is by an example I found online: classifying customer support tickets. Now, forgive me for being unoriginal, but if it works, it works.
Imagine you're building a customer support system for an e-commerce store. Customers submit complaints every day, and someone has to read each one, figure out what it's about, determine its urgency, and route it to the right team. Now, you decide to automate the whole classification process using an AI model to be more efficient as your business grows.
Consider this ticket:
"I've been trying to pay for my order since yesterday, but checkout keeps failing. I've also been charged twice, and nobody has responded to my email!"
With a standard LLM, you send it this ticket alongside a prompt like âYou must return valid JSON. Do not include markdown formattingâŚ" or something similar. Then you might need to add a lot more output constraints and validation to get the required information. Plus, the AI might hallucinate more ticket categories or mess up the exact wording, like using âBilling Errorâ instead of the predefined category âPayment Issue."
Now with Jev, you get direct classification from a decision model. During setup, you configure Jev with a strictly-typed question schema (using question types such as Choice, Score, or Noul). For our example, we might go with:
Category (Choice): âPayment Issueâ - âTechnical Issueâ - âShipping Delayâ - âAccount Lockoutâ - âOtherâ
Urgency Score (Score): 1 to 5 (Routine to Critical)
Sentiment (Choice): âPositiveâ - âNeutralâ - âAngryâ
Note that this is just a rough template from the top of my head (Donât judge me)
With that, Jev returns structured results that fit the schema we've defined. Since the available choices are predefined, Jev can't return a new category outside the options we've specified for that question. It can still make the wrong classification, though, which is why we need to consider its probabilities and confidence level rather than blindly trusting every result.
Now, Iâm not writing this because normal LLMs canât stick to strict schemas or always hallucinate. The difference is that Jev is designed around predefined decisions from the start. You're defining the possible answers beforehand, and Jev returns its decision alongside probability information that can help your application handle uncertainty.
Are LLMs Becoming Obsolete?
So, are Large Language Models done? Truth be told, thatâs what I was thinking about when I first heard of a pure System One AI model thatâs significantly faster and cheaper. However, thatâs not actually the case.
Instead, the major shift here is how we separate generative tasks from decision tasks. Using a standard LLM for evaluating data and making decisions from a predefined list can be overkill, especially when we consider speed, cost, and accuracy. As a decision model, Jev is faster, cheaper, and simply better at classification and routing according to strictly set schemas.
However, Jev isnât a generative model, so you donât get anything new like text, prose, or anything like that. For instance, it canât be used to automatically draft an email to send to customers whose tickets have been analyzed. Obviously, we need standard LLMs for such generative tasks.
Therefore, the future might be hybrid systems. Decision models can be used to efficiently filter and sort data in a pipeline. A model like Jev could classify incoming requests, filter spam, or determine which workflow should handle a particular task.
When a request actually needs a human-readable response, a traditional LLM could take over and generate one. It can also be used as some sort of router to determine which tool is best suited to a task, depending on the input provided and the range of available options.
Rather than treating Jev and traditional LLMs as direct replacements for one another, we can think of them as different components that may work better together.
What I Learned
This is probably my longest piece yet, though I consider it worth the effort. What are my key takeaways? Well, now I understand the differences between generative and decision tasks, as Iâve been naively mashing them together. What truly separates Jev from regular models is that itâs configured around our predefined decisions, so we get structured decisions, classifications, and probability scores.
Keep in mind, though, most of these claims are based on information from TypeSafe AIâs reports. Itâs probably best to take them with a pinch of salt until independent benchmarks are available.
Although AI models keep getting better, itâs quite clear that the brute force approach has its limits. Building specialized models for specific tasks and combining them into a single pipeline could be a more efficient approach, depending on the use case.
Top comments (1)
Thanks for reading đ