DEV Community: Abhishek Mishra

Jev is Not a Small LLM: A Technical Guide to System One Models

Abhishek Mishra — Tue, 22 Sep 2026 13:20:49 +0000

TL;DR

Model: Jev, from TypeSafe AI. The first "System One" model, launched September 2026.

What it is architecturally: Not an autoregressive generator. You declare a constrained output space and it returns the probability mass over that space. There are no output tokens in the generative sense, which is why they're free.

Output surface: Three primitives. Noul (a probability), Score (a position on your rubric plus its distribution), Choice (one of up to 255 options plus its distribution). No text, no coordinates, no pixels.

The property that changes your design: Every question in a request is evaluated in parallel against one shared read of the state. Eight questions cost roughly what one costs.

Trained with: RLCD (Reinforcement Learning for Calibrated Decisions), optimising against outcomes rather than human preference. This is why calibration is even a coherent claim here.

What I built: Semantic Microscope. Labels every sentence of a document with five or six calibrated probabilities and renders the whole thing as a heatmap.

My numbers (from Bengaluru): five documents, roughly 68,000 judgments, 31 cents, zero errors. p50 356ms round trip, of which only 98ms is the model and 247ms is the Pacific.

Measured calibration (n=59, hand-labelled): discrimination is excellent and monotonic, but the probabilities sit below the diagonal in every bucket. The cause turned out to be a disagreement about what the question meant, which is the most useful thing I learned.

Repo: github.com/abhishekmishragithub/semantic-microscope

A few days after Jev launched, a demo went round my timeline. Someone holds their hand up to a webcam, says "give me a ring", and a ring appears on their finger. Then sunglasses. Then a rainbow over their head. Credited to Jev. It did numbers.

I got access that week and started building, and about an hour in I realised something: Jev could not have drawn any of that. Not "it would be difficult". Structurally could not. There is no request you can make to Jev that produces a coordinate, an SVG path, or a pixel, because Jev's output space is something you declare in advance and it only ever returns a distribution over what you declared.

Whatever produced that ring was almost certainly MediaPipe finding the finger joint and a pre-made sprite being dropped on it. Jev's largest possible role was picking the string "ring" out of a list.

I'm not writing this to dunk on anyone (the demo is fun, and launch week is chaos for everybody). I'm writing it because the misunderstanding in that demo is the same one that will make your first Jev project disappointing. People are reaching for Jev as though it were a fast, cheap LLM. It's a different class of component, and once the architecture clicks you start seeing places for it that an LLM was never right for in the first place.

This post covers what Jev actually is under the hood, why its economics are shaped the way they are, the five patterns that work, where it fails, and real latency and calibration numbers I measured myself from India.

What kind of model is this, actually

Start here, because everything downstream follows from it.

An LLM is an autoregressive next-token predictor. You give it a context, it produces a probability distribution over the vocabulary, samples one token, appends it, and repeats. If you want a decision out of that process you have to convince it to write the decision down as tokens, then parse those tokens back into a decision. Structured output modes (JSON schema, grammar-constrained decoding) make the parsing more reliable, but the machinery underneath is unchanged: it's still generating a sequence, and you're still paying per token generated.

Jev inverts this. You declare the output space up front, as a Noul (binary), a Score (an ordered rubric of 2 to 10 levels), or a Choice (up to 255 options). The model does a forward pass over your state, and rather than sampling a sequence, it returns the probability mass over the space you declared.

TypeSafe hasn't published the architecture, so some of this is inference rather than documented fact. But the evidence is strong. openjev (166 stars) reproduces the primitive on a 4B open model by reading option logits directly, without generating anything, and gets something recognisably similar. That tells you the mechanism is a constrained read rather than a constrained generation.

Three consequences follow immediately, and they explain most of the strange things about Jev's pricing and API:

Output tokens are free because there aren't any. TypeSafe charges $0.042 per million input tokens and nothing for output. That's not a pricing promotion, it's a description of the architecture. Nothing is being generated, so there's nothing to meter.

Questions can be parallel because they don't share a sequence. Every question in a request is an independent read against the same forward pass over the same state. Question 4 cannot be influenced by question 2's answer, because there's no sequence for them to sit in. This is also why adding a question doesn't cause the context degradation you'd get from stuffing more into an LLM prompt.

Schema violations are structurally impossible, not merely unlikely. With an LLM in JSON mode you're constraining the sampler and hoping. With Jev the output space is your schema, so there is no state the model can be in that produces an invalid value. This is a real guarantee and it is also narrower than the marketing suggests, which I'll come back to.

Why calibration is a coherent claim here

The other half of the architecture story is the training objective. TypeSafe calls it RLCD, Reinforcement Learning for Calibrated Decisions: probabilities are optimised against observed outcomes rather than against human preference.

That distinction matters more than it sounds, and there's a well-known precedent. The GPT-4 technical report includes a pair of calibration plots showing the pre-trained base model was well calibrated on multiple-choice questions, and the post-RLHF model was substantially less calibrated. Training on human preference teaches a model that confident-sounding answers are rewarded, which is precisely the pressure that destroys calibration.

So when a chat model tells you it's "85% confident", that number is a linguistic artifact and means approximately nothing. When Jev returns 0.85, the training objective was at least pointed at making that number track reality.

"At least pointed at" is doing real work in that sentence. Calibration is a population-level property: if Jev says 0.8 across a thousand cases, roughly eight hundred should be true. It says nothing about whether this particular 0.8 is correct. And TypeSafe publishes no reliability diagram and no ECE, so this remains a claim about their training objective rather than a measured guarantee. I show you how to check it yourself later in this post, and I show you what I found.

The API, concretely

One endpoint. You send a state (a string or JSON object describing the situation) and a set of questions.


import httpx, os

r = httpx.post(
    "https://api.typesafe.ai/v1/systemone",
    headers={"Authorization": f"Bearer {os.environ['TYPESAFE_API_KEY']}"},
    json={
        "model": "jev-1.13.0",
        "state": {
            "sentence": "Either party may terminate upon thirty days written notice.",
            "document_kind": "commercial contract",
        },
        "questions": {
            "obligation": {
                "type": "noul",
                "instructions": "This sentence creates a binding obligation on one of the parties.",
            },
            "risk": {
                "type": "score",
                "instructions": "How legally load-bearing and unusual is this sentence?",
                "criteria": [
                    "Pure boilerplate",
                    "Standard market terms",
                    "Materially binding",
                    "Unusual or one-sided",
                ],
            },
            "clause_type": {
                "type": "choice",
                "instructions": "Which category does this sentence belong to?",
                "criteria": {
                    "termination": "Ending, suspending or renewing the agreement",
                    "payment": "Pricing, invoicing, fees or timing",
                    "liability": "Indemnity, warranty, limitation of liability",
                    "other": "None of the above",
                },
            },
        },
    },
)

Note the flat "type" form. I tried the nested {"noul": {...}} version first and got a 422.

What comes back:

Noul returns a single float, the probability the proposition is true. No confidence field, because the probability is the belief.
Score returns a continuous value that can land between levels (2.14), the full per-level distribution, a confidence, and a legend mapping indices back to your level text. The continuous value is a probability-weighted mean over the rubric, which is why it isn't an integer.
Choice returns the winning option, the full distribution across options, and a confidence derived from how peaked that distribution is.

Pin the model version (jev-1.13.0) rather than using jev-latest, and log the model field that comes back in every response. If you're tuning thresholds you really do not want the model moving under you mid-project.

A few operational limits worth knowing before you design anything: the state plus all questions share roughly a 64k token budget, Choice caps at 255 options, Score takes 2 to 10 levels, and the published rate limits are 250k tokens/sec and 1,200 requests/min. TypeSafe notes those limits are moving while capacity lands.

The mental model: a sense, not a reasoner

Kahneman's Thinking, Fast and Slow splits cognition in two. System 2 is slow, deliberate, sequential, the thing you use to work through a proof. System 1 is fast, automatic, parallel, and it does not produce arguments. It produces impressions. You walk into a room and know something is off before you could explain why.

Every LLM you've used is System 2 cosplay. It produces a chain of words, and if you want a decision you have to ask it to write one down and then read the words back.

Jev is shaped like System 1, and the architecture section above is why: parallel, non-sequential, no trace, output is an impression rather than an argument. There's no reasoning trace because there's no reasoning. That's the design, not a missing feature.

The practical version:

Stop asking what Jev can replace. Start asking where your system currently has no instincts at all.

Almost every LLM pipeline I've seen is a brain with no reflexes. It thinks carefully about everything, including things that should have been felt instantly. Those gaps are where Jev goes.

The demos are (mostly) misattributed

The ring isn't the only one. Once I started looking, a pattern showed up across the 180+ community projects that appeared in the first week. Almost all of them are real and clever. Almost all of them are credited to the wrong component.

"Jev drew captions and effects on my webcam." There's a Japanese project, jev-telop-live, that overlays variety-show captions and manga effects on a live camera feed. Its README is the most honest thing I've read about this model: it states outright that Jev does not write text, it answers typed questions with probabilities, and everything shown or played is picked from a closed set. The effects are hardcoded. Jev only selects which preset fires. That is exactly right, and exactly what the ring demo didn't say.

"Jev plays Doom / Mario / StarCraft / Pac-Man." Several of these exist and they do work. But Jev isn't seeing the screen. Something else serialises game state into text and Jev returns a Choice from a fixed action set each tick. Legitimate and interesting. The framing just makes people picture something that isn't happening.

"Jev booked a flight in 7.1 seconds for $0.0039." browser-use/jev-ultrafast, 641 stars, genuinely impressive. But browser-use did the browsing. Jev picked which element to click. The record belongs to the pair.

Published latency numbers from code that was never run. I found a Jev voice-agent repo quoting figures where the author notes, in the repo itself, that it had never been executed against the real API. Launch week produces a lot of this. Check whether numbers came from a real call before quoting them, including mine.

And the one that cuts the other way: openjev reproducing the primitive off a 4B model's logits, which I mentioned above. Someone rebuilt the shape of this in a weekend on open weights. That's genuinely useful information when you're deciding how much to build on the hosted version: the interface is copyable, and what TypeSafe is actually selling is the calibration training plus the hosting.

None of this means Jev is overhyped. It means the thing it's actually good at is less flashy and more useful than the demos suggest.

Four properties, and what each one buys

1. The output is a calibrated float, not a token stream. This is a category change, not a convenience. You get a number you can threshold, rank by, interpolate, animate, or send to a DAC. And you get gradations: not "risky" but 0.73. You can sort four hundred clauses by how risky they are. You cannot sort by the word "risky".

2. Questions are parallel and output is free. TypeSafe's cookbook reports batching 13 questions into one call was about 12× cheaper and 10× faster than asking them serially, with identical answers. My own runs match: going from 3 to 6 questions barely moved latency. So ask everything up front, including things you probably won't use. The LLM instinct of asking the cheap question first and escalating is exactly backwards here.

3. Under 100ms of model time. Fast enough to sit inside a loop. A render frame, a game tick, a conversation turn. Classification stops being a batch job and becomes a primitive you call like a function.

4. Effectively free. Five documents including two full novels, the GDPR and an IETF specification: roughly 68,000 judgments for 31 cents.

Jev vs an LLM vs a fine-tuned classifier

This is the comparison most posts dodge, because a fine-tuned BERT has been faster and cheaper than Jev at inference for years. The honest answer is that the difference isn't inference cost at all. It's what it costs to define a question.

A fine-tuned classifier needs labelled training data, hundreds to thousands of examples per class. You train, deploy, and have a model that answers exactly one question. Want a seventh dimension? Collect labels, retrain, redeploy. Reword a criterion? Retrain. Every question is a small project.

With Jev a question is a sentence of prose, defined at call time. I reworded one of my criteria three times in an afternoon and re-ran the whole document after each change, for a few cents total.

	Fine-tuned classifier	Jev	LLM
Cost to add a question	Days	Seconds	Seconds
Needs training data	Yes	No	No
Inference latency	~5-20ms	~100ms server	0.5-5s
Calibrated by design	No	Claimed	No (RLHF degrades it)
Can generate text	No	No	Yes
Cost at 10M/day	Lowest	Moderate	Prohibitive
Runs offline	Yes	No	Sometimes

Use a fine-tune when the taxonomy is frozen, volume is enormous, you need single-digit milliseconds, or you must run on-prem. And note the natural progression: if a Jev question of yours stabilises and volume explodes, distilling it into a small classifier is a perfectly good endgame. You'll already have the labelled dataset, because Jev generated it.

Use Jev when questions change often, you have many of them, you have no labelled data, or you want probabilities rather than labels.

Use an LLM when you need words.

The one-line rule

If you are writing a prompt that ends with "respond only with one of: A, B, C", that is a Jev call, and you should stop writing that prompt.

Every JSON-mode enum. Every LLM-as-judge returning a 1 to 5. Every "is this safe, yes or no" guardrail. Every router picking a model. These are decisions wearing a text costume. You're paying generation prices and generation latency for something that was never text, then writing parsing code to undo it.

Five patterns that work

The gate. One Noul in front of an expensive or irreversible action. 100ms and a hundredth of a cent to avoid a bad outcome is trivially worth it, and unlike an LLM guardrail it cannot talk itself into a wrong answer through a long chain of reasoning.

The router. One Choice before an expensive model. The classic cascade, except the router is now three orders of magnitude cheaper than the thing it routes to.

The fan-out. Twenty Nouls in one call covering every safety, intent and policy flag you care about. Because the marginal question is free, you can afford to check for things that almost never happen. Most under-used pattern right now.

The reflex. Jev inside a real-time loop, running concurrently with something slow. In a voice agent that's the gap between STT finishing and TTS starting: your LLM drafts a reply while Jev simultaneously decides whether the user is frustrated, whether this is in scope, and whether a human should take over. The fast path lands before the slow path does.

The dense labeller. Nobody is building this one.

The pattern nobody is using: label everything

Put the four properties together. Many questions, free. Per-item cost near zero. Fast. Calibrated floats instead of labels.

That means you can afford to ask every question about every item in a corpus, and then treat the resulting probabilities not as decisions but as a field you can look at.

I built Semantic Microscope to test this. It splits a document into sentences and asks Jev five or six questions about every single one, then renders the whole document as parallel vertical strips, one row per sentence, coloured by probability. You don't read the output. You look at it. A forty-page contract becomes a picture and the three clauses that matter glow. Point it at a novel with a different question set and the three-act structure shows up as bands.

The core:

DIMENSIONS = {
    "obligation": ("noul", "This sentence creates a binding obligation."),
    "ambiguous":  ("noul", "A reasonable professional could read this two incompatible ways."),
    "financial":  ("noul", "This sentence has a direct financial consequence."),
    "liability":  ("noul", "This limits, excludes, caps or assigns liability."),
}

async def label(sentence, prev, nxt):
    state = {
        "previous": prev,
        "sentence": sentence,
        "next": nxt,
        "document_kind": "commercial contract",
    }
    return await jev.ask(state, build_questions(DIMENSIONS))   # one call, all dimensions

Two things here are easy to get wrong and both cost badly:

One request per item, all questions inside it. If you're looping over questions you've made the run 6× slower, 6× more likely to hit the rate limit, and thrown away the exact architectural property the approach depends on.

Send the item, not the document. One sentence plus one of context each side. Stuffing the whole document into state degrades accuracy as irrelevant material crowds it out, and burns input tokens (the ones you do pay for) for nothing. Filtering context to what the question needs is your job, not the model's.

Every existing Jev project I could find treats the output as a decision that triggers an action. Browser agents, trading bots, game players, routers, guardrails. Almost nobody treats the probability distribution as an artifact in its own right. That's open ground, and it's the thing this architecture makes possible that genuinely wasn't possible last month.

Two more worked examples

The interrupt gate

I run a daemon that speaks my coding agent's lifecycle events aloud. The failure mode of that whole genre is spam. One Noul fixes it:

questions = {
    "worth_interrupting": {
        "type": "noul",
        "instructions": "A developer in deep focus would want to be told this immediately.",
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgently does this need attention?",
        "criteria": ["Ignore", "Mention later", "Say now", "Stop everything"],
    },
}

Speak only above the bar, and pick voice and speed from the score. Turned a noisy toy into something I leave running.

Routing before the expensive model

questions = {
    "skill": {
        "type": "choice",
        "instructions": "Which specialist should handle this request?",
        "criteria": {
            "code": "Writing or debugging software",
            "research": "Needs current external information",
            "chat": "Conversational, no tools needed",
            "refuse": "Out of scope or unsafe",
        },
    },
    "needs_frontier_model": {
        "type": "noul",
        "instructions": "This request requires the most capable model rather than a fast one.",
    },
}

Route on choice, and use confidence to decide when to fall back to the expensive path rather than trusting the route. A flat distribution across four options is the model telling you it doesn't know, and that's actionable information you don't get from an LLM router.

Where it fails

No arithmetic, no counting, no date ordering. Dates are text to it. Ask judgments, never computations.
It reads literally, and that's mostly a feature. Negations and scoping are taken at face value. On RFC 9110 it read "non-compliant" as a MUST question and separated MUST (median 0.75) from SHOULD (0.33) almost cleanly. The cost is that you own the question wording completely: there's no prose in the response to reveal that it interpreted you differently.
State is not treated as hostile. If user input goes into state, prompt injection is your threat model, not the model's.
Context rot is real. Accuracy degrades as state fills with irrelevant material. The parallelism protects questions from each other, not from a bloated state.
Served from US West only. Geography is a first-class latency cost if you're not in North America.
"Cannot hallucinate" is narrower than it sounds. The guarantee is structural: it cannot return a value outside your declared space. It can absolutely return the wrong value from inside that space, with high confidence. TypeSafe's CEO said as much on Hacker News. Schema safety is not correctness.
The headline benchmarks are vendor self-reported. "20 to 200× faster, 40 to 400× cheaper" and the accuracy figures are TypeSafe's own and unreproduced. The one independent hands-on test I've seen (Every) confirmed speed and cost, found accuracy "good but not perfect", and explicitly did not audit calibration.

My numbers, measured from Bengaluru

Vendor latency figures assume you're sitting near the inference pool. I'm 13,000km from it. Five documents, cold cache, 15 requests/s with 12 in flight, nothing rounded away:

run	preset	sentences (live)	wall	p50	p95	p99	cost
sample-contract (synthetic MSA)	contract	484 (482)	33.2s	385ms	506ms	617ms	$0.0141
gdpr-contract (GDPR, EUR-Lex)	contract	1,977 (1,630)	108.9s	357ms	454ms	578ms	$0.0481
pride-and-prejudice (Gutenberg #1342)	prose	4,533 (4,433)	295.9s	347ms	439ms	619ms	$0.1017
rfc9110 (HTTP Semantics)	rfc	3,566 (3,001)	200.6s	356ms	446ms	570ms	$0.0826
frankenstein (Gutenberg #84)	prose	3,087 (2,970)	198.8s	410ms	515ms	636ms	$0.0669

That's roughly 68,000 judgments across five documents for 31 cents, including two full novels, the GDPR, and an IETF specification. Zero errors and zero 429s across all of it.

Doing the same with a frontier model would have been a budget conversation rather than a rounding error.

Where the time actually goes

TypeSafe returns an x-envoy-upstream-service-time header, so you can separate inference from network. On the RFC 9110 run:

component	p50	p95	p99
server (from the envoy header)	98ms	162ms	209ms
network and TLS (client minus server)	247ms	322ms	367ms

Roughly two thirds of every request is the Pacific, not the model. Jev itself answers in under 100ms. Two things follow:

Reuse your connections. A cold TLS handshake across that distance costs more than the inference. One persistent client for the whole run is not an optimisation, it's the difference between three minutes and most of an hour. I found this the annoying way.

Geography decides which patterns work for you. Dense labelling and gating are comfortable at 356ms. A reflex inside a live conversation turn, fighting for an 800ms end-to-end budget, is marginal from India and fine from Oregon. Measure before you design.

One more detail worth reading off that table: every run sat pinned at my own 15 rps limiter for its entire duration, so wall time is just sentences ÷ 15. The bottleneck was my rate limiting, not Jev. The documented ceiling is 1,200 requests/min and I was deliberately running under it.

And the cache matters more than I expected. Re-running the GDPR document with a warm cache took 2.9 seconds for 1,977 sentences and made zero requests. That turns "iterate on the viewer while re-running the pipeline" from a cost decision into a non-decision, which changed how I worked on this more than any other single thing.

Does the model actually track the question? A free check first

Before hand-labelling anything, there's a cheap sanity check worth running, and I haven't seen anyone else do it.

One of my six dimensions was deliberately lexical:

"normative_keyword": {
    "type": "noul",
    "instructions": "This sentence contains an RFC 2119 keyword: MUST, MUST NOT, SHALL, "
                    "SHOULD, SHOULD NOT, REQUIRED, RECOMMENDED, MAY, OPTIONAL.",
}

That question has objective ground truth. A regex answers it perfectly. So I ran the regex over all 3,001 sentences and compared.

Recall 1.00. Precision 0.79.

Jev found every single one of the 413 keyword sentences, and flagged 107 more that contain no uppercase keyword.

Look at what those 107 are, though. They're overwhelmingly sentences stating a requirement in lowercase prose: "a server must", "clients should". Which means Jev is doing exactly what a careful human reader does before being told the rules, and getting it wrong for exactly the reason the IETF had to publish RFC 8174, a whole RFC clarifying that only the uppercase forms carry normative force, because everyone kept conflating them.

Two things I'd take from that. First, a Noul does genuinely track its question rather than returning vibes. Second, "reads literally" is a real property and it cuts both ways: here it over-fires because the question said "contains a keyword" and Jev reasoned about meaning instead of characters.

If you're building on Jev, find the one question in your set that has objective ground truth and check it this way. It costs nothing and it's the fastest signal you'll get.

Measured calibration: the part nobody publishes

TypeSafe says Jev's probabilities are calibrated. They publish no reliability diagram and no ECE. So I measured it.

Method

The dimension:

"requirement": {
    "type": "noul",
    "instructions": "Ignoring this sentence would make an implementation non-compliant.",
}

The labelling rule, written before I saw any sentences and not revised during the run:

Work down the list, stop at the first match.

contains MUST, MUST NOT, SHALL, SHALL NOT, REQUIRED → yes

contains SHOULD, SHOULD NOT, RECOMMENDED → no

contains MAY, OPTIONAL → no

states an absolute requirement in plain prose, no keyword → yes

only describes, defines or explains → no

points at another section instead of stating a rule → no

SHOULD is a "no" because RFC 2119 defines it as permitting valid reasons to deviate. An implementation that ignores a SHOULD with good cause is still compliant, and the question asks about compliance.

Then: a stratified sample across the five probability buckets, with Jev's prediction hidden while labelling, in two sittings of 30. That last detail is the one people skip, and skipping it invalidates the whole exercise. If you see the number first, you're measuring your own anchoring.

Result (n=59, single annotator)

bucket	n	predicted	observed
0.0–0.2	12	0.15	0.00
0.2–0.4	12	0.31	0.00
0.4–0.6	12	0.46	0.08
0.6–0.8	11	0.71	0.45
0.8–1.0	12	0.82	0.75

Brier score: 0.165

Three findings, in order of how much they'd change your design.

1. Discrimination is excellent. Ranking is trustworthy.

The observed column rises monotonically: 0.00, 0.00, 0.08, 0.45, 0.75. No inversions anywhere. If you sort by the score, the sort is meaningful. For most real uses (triage, review queues, "show me the 30 that matter") this is the property you actually depend on, and it holds.

2. The probabilities sit below the diagonal in every bucket.

All five, same direction. Under a null of no bias that's about a 1-in-32 coincidence, and two buckets are individually significant: 0/12 against a predicted 0.31 has roughly a 1.2% binomial probability, and 1/12 against 0.46 about 0.8%.

Concretely, 0.71 meant about 0.45 here. But be precise about what's established: with 11 to 12 items per bucket the intervals are wide, and the two end buckets are individually consistent with calibration. It's the middle that isn't. The sign test across all five plus the two significant mid-buckets is the evidence, not any single row.

At n=29 after my first sitting I could see the shape but couldn't call it. The second sitting is what made it a finding rather than a hunch. If you run this yourself, do the second sitting.

This has an identifiable cause rather than being diffuse noise. I dig into it two sections down: the mid-range is almost entirely SHOULD sentences, which Jev scores at a median of 0.33 while my labelling rule scored them as hard no's.

3. The triage number, which is the one I'd actually put in a pitch deck.

Nothing below 0.4 was a real requirement. Zero out of 24 labelled items, across two buckets that together hold 2,059 of 3,001 sentences, or 68.6% of RFC 9110.

So the honest operational claim is: this discards roughly two-thirds of the specification, with no false negatives observed at this sample size, and you read the rest. That's a much better claim than "AI reads your spec for you", and unlike that one, it has a number behind it.

The rule this gives you

For this question on this document: threshold at 0.6. Treat below 0.4 as safely discardable. Trust the ordering, discount the absolute values.

What I am not claiming

One dimension, one document, one annotator, n=59, eleven to twelve items per bucket. The question's wording does real work, as the next section shows. I am not saying Jev is miscalibrated in general. I'm saying that on this task I measured a gap, that you should measure your own, and that the tooling to do it is in the repo:

uv run microscope validate <run-id> --dimension requirement
uv run microscope validate <run-id> --dimension requirement --add 30

Do it before you put a probability in front of anything consequential. And publish what you find either way. Right now almost nobody has, which is the only reason this section is interesting.

What the overconfidence actually was

The calibration gap above has a mechanism, and finding it was the most useful thing in this whole exercise.

RFC 2119 gives you three tiers of obligation. So I grouped every sentence by which uppercase keyword it contains and looked at what Jev gave them under the requirement question:

keyword	n	min	p25	median	p75	max	share ≥ 0.5
MUST / MUST NOT	201	0.23	0.67	0.75	0.79	0.90	96%
SHOULD / SHOULD NOT	119	0.13	0.25	0.33	0.42	0.65	8%
MAY	104	0.12	0.21	0.27	0.36	0.74	6%

Only four sentences contain both a MUST and a SHOULD, so the separation isn't an artefact of mixed sentences.

Jev is reading the question correctly. It asked itself whether ignoring this makes you non-compliant, concluded that's a MUST question, and separated the tiers almost cleanly: 96% of MUST sentences land above 0.5 against 8% of SHOULD. Threshold at 0.5 and you have, in effect, a MUST detector that also catches requirements stated in plain prose without any keyword at all.

But look at where SHOULD sits. Median 0.33, not 0.05. Jev treats SHOULD as a weak yes rather than a no. My labelling rule, written from RFC 2119, treated it as a hard no.

That single disagreement explains most of the calibration gap. The 0.2 to 0.4 bucket, where I measured a predicted 0.31 against an observed 0.00, is the SHOULD population almost exactly. The model isn't badly calibrated there so much as it's holding a slightly different theory of the question than I am. A lawyer would side with me. Plenty of engineers would side with Jev, because in practice ignoring a SHOULD does often mean your implementation is wrong.

I'd call that overconfidence, because under the question as I defined it those probabilities are too high. But it's overconfidence with a cause you can see and correct for, not noise.

The general lesson, which is the thing I'd actually hand someone building on this:

Your question wording isn't prompt engineering. It's the schema.

With an LLM, a disagreement about what you meant surfaces in the prose. You read the answer and think, that's not what I asked. Jev returns a float. There is nowhere for the disagreement to appear, so it emerges downstream as a calibration gap and you blame the model.

So before you trust a Noul: write down what a "yes" means precisely enough that a stranger could label from it, then group your results by some feature you already understand and check the distributions. If a class of items you expected near 0.1 is sitting at 0.33, that isn't noise. That's the model telling you it read the question differently, and it's the cheapest debugging signal you'll get.

Worth noticing that both of my useful findings came from the same move: group the results by something you already have ground truth for, and look at the distribution. The regex check told me a Noul tracks its question. The keyword grouping told me which question it thought it was answering. Neither needed a single hand-label.

Where I think this lands

The obvious commercial fit is anywhere the work is "check N criteria against M documents", because that cost used to scale as N×M and now it basically doesn't. Compliance review, contract triage, policy audit, and increasingly, auditing AI agent transcripts against a policy, continuously, at volume.

With one discipline attached: triage, not verdict. The honest pitch is "this takes a reviewer from four hundred clauses to the thirty they must actually read". It is not "this decides compliance". Given calibration is an unverified claim, anyone selling the second version is selling something they can't back.

The less obvious fit is the dense labelling section. Treating probabilities as a field rather than a trigger. Visualisation, instruments, art. Nobody's there yet, and I think that's where the interesting work is.

Jev's real contribution isn't that it's cheap. It's that it makes asking cheap, and when asking is free you ask about everything, and you find out that documents and conversations and codebases have shapes you were never able to see.

Go build something and tell me about it :)

Sources & References

TypeSafe AI docs - primitives, limits, and their own guidance on confidence thresholds. The confidence page is more careful than the marketing and worth reading first.
Semantic Microscope - the tool from this post. Python pipeline, vanilla JS viewer, no framework, bring your own key.
openjev - reproduces the primitive by reading option logits off a 4B open model. The best available evidence for how the mechanism actually works.
jev-telop-live - the most honest README in the ecosystem about what Jev does and doesn't do.
browser-use/jev-ultrafast - the flight booking demo. Good example of Jev as one component rather than the whole system.
jevmeter - a live probability meter over a debate video, 1,191 calls for about 5 cents. Prior art if you're doing anything with probability as a visual.
RFC 9110 - HTTP Semantics. The document everything in the measurement sections was run against.
RFC 2119 - defines MUST, SHOULD, MAY and the rest. The reason SHOULD counts as "no" in my labelling rule.
RFC 8174 - the clarification that only uppercase keywords are normative. Jev makes the same mistake this RFC exists to correct.
GPT-4 Technical Report - see the calibration plots showing RLHF degrading the base model's calibration. Useful background for why RLCD is a different bet.

Build Voice AI in Python: Complete Speech-to-Text Developer Guide (2026)

Abhishek Mishra — Tue, 10 Feb 2026 15:04:38 +0000

TL;DR – Quick Integration Overview

API Platform: Pulse STT by Smallest AI – a state-of-the-art speech-to-text API supporting real-time streaming and batch audio transcription.

Key Features:

Transcribes in 32+ languages with automatic language detection
Ultra-low latency: ~64ms time-to-first-transcript for streaming
Rich metadata: word timestamps, speaker diarization, emotion detection, age/gender estimation, PII redaction

Integration Methods:

Pre-Recorded Audio: POST https://waves-api.smallest.ai/api/v1/pulse/get_text – upload files for batch processing
Real-Time Streaming: wss://waves-api.smallest.ai/api/v1/pulse/get_text – WebSocket for live transcription

Developer Experience: Use any HTTP/WebSocket client or official SDKs (Python, Node.js). Authentication via a single API key.

Why Pulse STT? Compared to other providers, Pulse offers faster response (64ms vs 200-500ms for typical cloud STT) and all-in-one features (no need for separate services for speaker ID, sentiment, or PII masking).

Quick Links:

API Console – Get your API key
Documentation – Full API reference
Python SDK – Official client

Introduction: Why Voice Integration Matters

Voice is becoming the next frontier for user interaction. From virtual assistants and voice bots to real-time transcription in meetings, speech interfaces are making software more accessible and user-friendly. Developers today have access to Automatic Speech Recognition (ASR) APIs that convert voice to text, opening up possibilities for hands-free control, live captions, voice search, and more.

However, integrating voice AI is more than just getting raw text from audio. Modern use cases demand speed and accuracy – a voice assistant needs to transcribe commands almost instantly, and a call center analytics tool might need not just the transcript but also who spoke when and how they said it.

Latency is critical. A delay of even a second feels laggy in conversation. Traditional cloud speech APIs often have 500–1200ms latency for live transcription, with better ones hovering around 200–250ms. This has pushed the industry toward ultra-low latency – under 300ms – to enable seamless real-time interactions.

In this guide, we'll walk through how to integrate an AI voice & speech API that meets these modern demands using Smallest AI's Pulse STT. By the end, you'll know how to:

Transcribe audio files (WAV/MP3) to text using a simple HTTP API
Stream live audio for instantaneous transcripts via WebSockets
Leverage advanced features like timestamps, speaker diarization, and emotion detection
Use both Python and Node.js to integrate voice capabilities

Understanding Pulse STT

Pulse is the speech-to-text/ ASR(automatic speech recognition) model from Smallest AI's "Waves" platform. It's designed for fast, accurate, and rich transcription with industry-leading latency – around 64 milliseconds to first transcribed word TTFT for streaming audio. This is an order of magnitude faster than many alternatives.

Highlight Features

Feature	Description
Real-Time & Batch Modes	Stream live audio via WebSocket or upload files via HTTP POST
32+ Languages	English, Spanish, Hindi, French, German, Arabic, Japanese, and more with auto-detection
Word/Sentence Timestamps	Know exactly when each word was spoken (great for subtitles)
Speaker Diarization	Differentiate speakers: "Speaker A said X, Speaker B said Y"
Emotion Detection	Tag segments with emotions: happy, angry, neutral, etc.
Age/Gender Estimation	Infer speaker demographics for analytics
PII/PCI Redaction	Automatically mask credit cards, SSNs, and personal info
64ms Latency	Time-to-first-transcript in streaming mode

Getting Started: Authentication

Step 1: Get Your API Key

Step 2: Test Your Key

curl -H "Authorization: Bearer $SMALLEST_API_KEY" \
  https://waves-api.smallest.ai/api/v1/lightning-v3.1/get_voices

Authentication Header

All requests require this header:

Authorization: Bearer <YOUR_API_KEY>

Part 1: Transcribing Audio Files (REST API)

The Pre-Recorded API is perfect for batch processing voicemails, podcasts, meeting recordings, or any existing audio files.

Endpoint

POST https://waves-api.smallest.ai/api/v1/pulse/get_text

Query Parameters

Parameter	Type	Description
`model`	string	Model identifier: `pulse` (required)
`language`	string	ISO code (`en`, `es`, `hi`) or `multi` for auto-detect
`word_timestamps`	boolean	Include word-level timing data
`diarize`	boolean	Enable speaker diarization
`emotion_detection`	boolean	Detect speaker emotions
`age_detection`	boolean	Estimate speaker age group
`gender_detection`	boolean	Estimate speaker gender

Supported Languages (32+)

Italian, Spanish, English, Portuguese, Hindi, German, French, Ukrainian, Russian, Kannada, Malayalam, Polish, Marathi, Gujarati, Czech, Slovak, Telugu, Odia, Dutch, Bengali, Latvian, Estonian, Romanian, Punjabi, Finnish, Swedish, Bulgarian, Tamil, Hungarian, Danish, Lithuanian, Maltese, and auto-detection (multi).

cURL Example

curl --request POST \
  --url "https://waves-api.smallest.ai/api/v1/pulse/get_text?model=pulse&language=en&diarize=true&word_timestamps=true&emotion_detection=true" \
  --header "Authorization: Bearer $SMALLEST_API_KEY" \
  --header "Content-Type: audio/wav" \
  --data-binary "@/path/to/audio.wav"

Python Example

import os
import requests

API_KEY = os.getenv("SMALLEST_API_KEY")
audio_file = "meeting_recording.wav"

url = "https://waves-api.smallest.ai/api/v1/pulse/get_text"
params = {
    "model": "pulse",
    "language": "en",
    "word_timestamps": "true",
    "diarize": "true",
    "emotion_detection": "true"
}
headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "audio/wav"
}

with open(audio_file, "rb") as f:
    audio_data = f.read()

response = requests.post(url, params=params, headers=headers, data=audio_data)
result = response.json()

# Print transcription
print("Transcription:", result.get("transcription"))

# Print word-level details with speaker info
for word in result.get("words", []):
    speaker = word.get("speaker", "N/A")
    print(f"  [Speaker {speaker}] [{word['start']:.2f}s - {word['end']:.2f}s] {word['word']}")

# Check emotions
if "emotions" in result:
    print("\nEmotions detected:")
    for emotion, score in result["emotions"].items():
        if score > 0.1:
            print(f"  {emotion}: {score:.1%}")

Node.js Example

const fs = require('fs');
const axios = require('axios');

const API_KEY = process.env.SMALLEST_API_KEY;
const audioFile = 'meeting_recording.wav';

const url = 'https://waves-api.smallest.ai/api/v1/pulse/get_text';
const params = new URLSearchParams({
  model: 'pulse',
  language: 'en',
  word_timestamps: 'true',
  diarize: 'true',
  emotion_detection: 'true'
});

const audioData = fs.readFileSync(audioFile);

axios.post(`${url}?${params}`, audioData, {
  headers: {
    'Authorization': `Bearer ${API_KEY}`,
    'Content-Type': 'audio/wav'
  }
})
.then(res => {
  console.log('Transcription:', res.data.transcription);

  // Print words with speaker info
  res.data.words?.forEach(word => {
    console.log(`  [Speaker ${word.speaker}] [${word.start}s - ${word.end}s] ${word.word}`);
  });
})
.catch(err => {
  console.error('Error:', err.response?.data || err.message);
});

Example Response

{
  "status": "success",
  "transcription": "Hello, this is a test transcription.",
  "words": [
    {"start": 0.0, "end": 0.88, "word": "Hello,", "confidence": 0.82, "speaker": 0, "speaker_confidence": 0.61},
    {"start": 0.88, "end": 1.04, "word": "this", "confidence": 1.0, "speaker": 0, "speaker_confidence": 0.76},
    {"start": 1.04, "end": 1.20, "word": "is", "confidence": 1.0, "speaker": 0, "speaker_confidence": 0.99},
    {"start": 1.20, "end": 1.36, "word": "a", "confidence": 1.0, "speaker": 0, "speaker_confidence": 0.99},
    {"start": 1.36, "end": 1.68, "word": "test", "confidence": 0.99, "speaker": 0, "speaker_confidence": 0.99},
    {"start": 1.68, "end": 2.16, "word": "transcription.", "confidence": 0.99, "speaker": 0, "speaker_confidence": 0.99}
  ],
  "utterances": [
    {"start": 0.0, "end": 2.16, "text": "Hello, this is a test transcription.", "speaker": 0}
  ],
  "age": "adult",
  "gender": "female",
  "emotions": {
    "happiness": 0.28,
    "sadness": 0.0,
    "anger": 0.0,
    "fear": 0.0,
    "disgust": 0.0
  },
  "metadata": {
    "duration": 1.97,
    "fileSize": 63236
  }
}

Part 2: Real-Time Streaming (WebSocket API)

For live audio – voice assistants, live captioning, call center analytics – use the WebSocket API for sub-second latency with partial results as audio streams in.

WebSocket Endpoint

wss://waves-api.smallest.ai/api/v1/pulse/get_text

Query Parameters

Parameter	Type	Default	Description
`language`	string	`en`	Language code or `multi` for auto-detect
`encoding`	string	`linear16`	Audio format: `linear16`, `linear32`, `alaw`, `mulaw`, `opus`
`sample_rate`	string	`16000`	Sample rate: `8000`, `16000`, `22050`, `24000`, `44100`, `48000`
`word_timestamps`	string	`true`	Include word-level timestamps
`full_transcript`	string	`false`	Include cumulative transcript
`sentence_timestamps`	string	`false`	Include sentence-level timestamps
`redact_pii`	string	`false`	Redact personal information
`redact_pci`	string	`false`	Redact payment card information
`diarize`	string	`false`	Enable speaker diarization

Python Streaming Example

From the official cookbook:

import asyncio
import json
import os
import numpy as np
import websockets
import librosa
from urllib.parse import urlencode

WS_URL = "wss://waves-api.smallest.ai/api/v1/pulse/get_text"

# Configurable features
LANGUAGE = "en"
ENCODING = "linear16"
SAMPLE_RATE = 16000
WORD_TIMESTAMPS = False
FULL_TRANSCRIPT = True
SENTENCE_TIMESTAMPS = False
DIARIZE = False
REDACT_PII = False
REDACT_PCI = False

async def transcribe(audio_file: str, api_key: str):
    params = {
        "language": LANGUAGE,
        "encoding": ENCODING,
        "sample_rate": SAMPLE_RATE,
        "word_timestamps": str(WORD_TIMESTAMPS).lower(),
        "full_transcript": str(FULL_TRANSCRIPT).lower(),
        "sentence_timestamps": str(SENTENCE_TIMESTAMPS).lower(),
        "diarize": str(DIARIZE).lower(),
        "redact_pii": str(REDACT_PII).lower(),
        "redact_pci": str(REDACT_PCI).lower(),
    }

    url = f"{WS_URL}?{urlencode(params)}"
    headers = {"Authorization": f"Bearer {api_key}"}

    # Load audio with librosa (handles any format)
    audio, _ = librosa.load(audio_file, sr=SAMPLE_RATE, mono=True)
    chunk_duration = 0.1  # 100ms chunks
    chunk_size = int(chunk_duration * SAMPLE_RATE)

    async with websockets.connect(url, additional_headers=headers) as ws:
        print("✅ Connected to Pulse STT WebSocket")

        async def send_audio():
            for i in range(0, len(audio), chunk_size):
                chunk = audio[i:i + chunk_size]
                pcm16 = (chunk * 32768.0).astype(np.int16).tobytes()
                await ws.send(pcm16)
                await asyncio.sleep(chunk_duration)
            await ws.send(json.dumps({"type": "end"}))
            print("📤 Sent end signal")

        async def receive_responses():
            async for message in ws:
                result = json.loads(message)

                if result.get("is_final"):
                    print(f"✓ {result.get('transcript')}")

                    if result.get("is_last"):
                        if result.get("full_transcript"):
                            print(f"\n{'='*60}")
                            print("FULL TRANSCRIPT")
                            print(f"{'='*60}")
                            print(result.get("full_transcript"))
                        break

        await asyncio.gather(send_audio(), receive_responses())

# Usage
if __name__ == "__main__":
    api_key = os.environ.get("SMALLEST_API_KEY")
    asyncio.run(transcribe("recording.wav", api_key))

Install dependencies:

pip install websockets librosa numpy

Run:

export SMALLEST_API_KEY="your-api-key"
python transcribe.py recording.wav

Node.js Streaming Example

From the official cookbook:

const fs = require("fs");
const WebSocket = require("ws");
const wav = require("wav");

const WS_URL = "wss://waves-api.smallest.ai/api/v1/pulse/get_text";

// Configurable features
const LANGUAGE = "en";
const ENCODING = "linear16";
const SAMPLE_RATE = 16000;
const WORD_TIMESTAMPS = false;
const FULL_TRANSCRIPT = true;
const DIARIZE = false;
const REDACT_PII = false;
const REDACT_PCI = false;

async function loadAudio(audioFile) {
  return new Promise((resolve, reject) => {
    const reader = new wav.Reader();
    const chunks = [];

    reader.on("format", (format) => {
      reader.on("data", (chunk) => chunks.push(chunk));
      reader.on("end", () => {
        const buffer = Buffer.concat(chunks);
        const samples = new Int16Array(buffer.buffer, buffer.byteOffset, buffer.length / 2);
        resolve(samples);
      });
    });

    reader.on("error", reject);
    fs.createReadStream(audioFile).pipe(reader);
  });
}

async function transcribe(audioFile, apiKey) {
  const params = new URLSearchParams({
    language: LANGUAGE,
    encoding: ENCODING,
    sample_rate: SAMPLE_RATE,
    word_timestamps: WORD_TIMESTAMPS,
    full_transcript: FULL_TRANSCRIPT,
    diarize: DIARIZE,
    redact_pii: REDACT_PII,
    redact_pci: REDACT_PCI,
  });

  const url = `${WS_URL}?${params}`;
  const audio = await loadAudio(audioFile);
  const chunkDuration = 0.1; // 100ms
  const chunkSize = Math.floor(chunkDuration * SAMPLE_RATE);

  return new Promise((resolve, reject) => {
    const ws = new WebSocket(url, {
      headers: { Authorization: `Bearer ${apiKey}` },
    });

    ws.on("open", async () => {
      console.log("✅ Connected to Pulse STT WebSocket");

      for (let i = 0; i < audio.length; i += chunkSize) {
        const chunk = audio.slice(i, i + chunkSize);
        ws.send(Buffer.from(chunk.buffer, chunk.byteOffset, chunk.byteLength));
        await new Promise((r) => setTimeout(r, chunkDuration * 1000));
      }
      ws.send(JSON.stringify({ type: "end" }));
      console.log("📤 Sent end signal");
    });

    ws.on("message", (data) => {
      const result = JSON.parse(data.toString());

      if (result.is_final) {
        console.log(`✓ ${result.transcript}`);

        if (result.is_last) {
          if (result.full_transcript) {
            console.log("\n" + "=".repeat(60));
            console.log("FULL TRANSCRIPT");
            console.log("=".repeat(60));
            console.log(result.full_transcript);
          }
          ws.close();
        }
      }
    });

    ws.on("close", resolve);
    ws.on("error", reject);
  });
}

// Usage
const apiKey = process.env.SMALLEST_API_KEY;
transcribe("recording.wav", apiKey).then(() => console.log("Done!"));

Install dependencies:

npm install ws wav

Run:

export SMALLEST_API_KEY="your-api-key"
node transcribe.js recording.wav

WebSocket Response Format

{
  "session_id": "sess_12345abcde",
  "transcript": "Hello, how are you?",
  "full_transcript": "Hello, how are you?",
  "is_final": true,
  "is_last": false,
  "language": "en",
  "words": [
    {"word": "Hello,", "start": 0.0, "end": 0.5, "confidence": 0.98, "speaker": 0},
    {"word": "how", "start": 0.5, "end": 0.7, "confidence": 0.99, "speaker": 0},
    {"word": "are", "start": 0.7, "end": 0.9, "confidence": 0.97, "speaker": 0},
    {"word": "you?", "start": 0.9, "end": 1.2, "confidence": 0.99, "speaker": 0}
  ]
}

Key Response Fields

Field	Description
`is_final`	`false` = partial/interim transcript; `true` = finalized segment
`is_last`	`true` when the entire session is complete
`transcript`	Current segment text
`full_transcript`	Accumulated text from entire session (if enabled)
`words`	Word-level timestamps (if enabled)

Part 3: Advanced Features

Speaker Diarization

Enable diarize=true to identify different speakers:

params = {"model": "pulse", "language": "en", "diarize": "true"}

Response includes speaker labels:

{
  "words": [
    {"word": "Hello", "speaker": 0, "speaker_confidence": 0.95},
    {"word": "Hi", "speaker": 1, "speaker_confidence": 0.92}
  ],
  "utterances": [
    {"text": "Hello, how can I help?", "speaker": 0},
    {"text": "I have a question.", "speaker": 1}
  ]
}

Emotion Detection

Enable emotion_detection=true to analyze speaker sentiment:

{
  "emotions": {
    "happiness": 0.28,
    "sadness": 0.0,
    "anger": 0.0,
    "fear": 0.0,
    "disgust": 0.0
  }
}

PII/PCI Redaction

For compliance (HIPAA, PCI-DSS), enable redact_pii=true or redact_pci=true:

{
  "transcript": "My credit card is [CREDITCARD_1] and SSN is [SSN_1]",
  "redacted_entities": ["[CREDITCARD_1]", "[SSN_1]"]
}

Age and Gender Detection

Enable age_detection=true and gender_detection=true:

{
  "age": "adult",
  "gender": "female"
}

Comparing STT Providers

Provider	Latency	Languages	Diarization	Emotion	PII Redaction	Price (per 1000 min)
Pulse STT	~64ms	32+	✅	✅	✅	Competitive
Google Cloud STT	200-300ms	125+	✅	❌	❌	~$16
Deepgram	100-200ms	36+	✅	❌	✅	~$4-5
AssemblyAI	200-400ms	30+	✅	✅	✅	~$3.50
OpenAI Whisper	Batch only	99+	❌	❌	❌	~$6

Why Pulse STT stands out:

Fastest time-to-first-transcript (64ms)
All-in-one features (no separate services needed)
Competitive accuracy across diverse accents
Built for real-time voice AI applications

Best Practices

Audio Quality

Use 16kHz, mono, 16-bit PCM for best results
WAV or FLAC formats are ideal
Minimize background noise when possible

Error Handling

try:
    response = requests.post(url, params=params, headers=headers, data=audio_data, timeout=120)
    response.raise_for_status()
except requests.exceptions.HTTPError as e:
    if e.response.status_code == 429:
        # Rate limited - implement exponential backoff
        time.sleep(2 ** retry_count)
    elif e.response.status_code == 401:
        # Invalid API key
        raise ValueError("Invalid API key")

Rate Limiting

Add 500ms+ delay between batch requests
Use webhooks for long audio files
Implement exponential backoff for 429 errors

Bonus: Full Demo Application

Want to see everything working together? Check out the demo app in the code samples repository — a complete Next.js web application featuring:

File upload transcription with word-level timestamps (hover to see timing)
Real-time microphone streaming with live transcript display
Secure WebSocket proxy that keeps your API key server-side
Modern UI with Smallest AI brand colors
Language selection (English, Hindi, Spanish, French, German, Portuguese, Auto-detect)
Emotion detection and speaker diarization display

Quick Start

cd demo-app
npm install

Create a .env.local file with your API key:

echo 'SMALLEST_API_KEY=your-api-key' > .env.local

Start both servers (Next.js + WebSocket proxy):

npm run dev:all

Then open http://localhost:3000 in Chrome or Safari (for microphone access).

How It Works

The demo runs two servers:

Next.js (port 3000) — Serves the React UI and handles file upload via /api/transcribe
WebSocket Proxy (port 3001) — Securely proxies audio from browser to Pulse STT WebSocket API

Browser → WebSocket Proxy (3001) → Pulse STT (wss://waves-api.smallest.ai)
Browser → Next.js API (3000) → Pulse STT (REST API)

This architecture keeps your API key secure on the server while enabling real-time streaming.

Project Structure

demo-app/
├── src/
│   └── app/
│       ├── api/
│       │   └── transcribe/
│       │       └── route.ts    # REST API for file upload
│       ├── page.tsx            # Main UI
│       └── layout.tsx
├── ws-server.js                # WebSocket proxy server
├── .env.local                  # Your API key (create this)
└── package.json

Scripts

Command	Description
`npm run dev`	Start Next.js only
`npm run dev:ws`	Start WebSocket proxy only
`npm run dev:all`	Start both (recommended)

This architecture pattern is recommended for production apps — API keys stay server-side while the React frontend provides a smooth user experience with both file upload and real-time microphone transcription.

Conclusion

Integrating voice and speech capabilities into your workflow and apps can greatly enhance user experience. With Pulse STT, developers can achieve high-accuracy, low-latency transcription with just a few API calls.

When to use REST API:

Podcast transcription
Meeting recordings
Voicemail processing
Batch analytics

When to use WebSocket API:

Live captioning
Voice assistants
Call center real-time analytics
Interactive voice applications

The code patterns in this guide translate directly to production. Start with the REST API for prototyping, then add WebSocket streaming when real-time interaction becomes a requirement.

Resources

Smallest AI Console — API key management
Waves Documentation — Full API reference
Discord Community — Developer support

How to do efficient fine-Tuning for LLMs using SLoRA

Abhishek Mishra — Wed, 07 Aug 2024 08:23:17 +0000

You're a passionate AI developer, eager to try to utilize the power of large language models (LLMs) for your latest project. You've got brilliant ideas, but there's a catch – fine-tuning these massive models feels like trying to parallel park a cruise ship in a crowded marina. It's resource-intensive, time-consuming, and frankly, a bit overwhelming.
Sound familiar? You're not alone in this boat.

Before we dive deeper, let's clarify what we mean by fine-tuning:

Fine-tuning is the process of further training a pre-trained model on a specific task or dataset to adapt its knowledge for a particular application.

The world of AI has been grappling with a significant challenge: how to efficiently fine-tune LLMs without breaking the bank or melting your hardware – we all are GPU poor except NVIDIA.
Traditional fine-tuning methods are like using a sledgehammer to crack a nut, – they get the job done, but at what cost?
Enter SLoRA – Sparse Low-Rank Adaptation, – the unsung hero for efficient model fine-tuning.

Fine-tuning large language models (LLMs) has become a crucial step in achieving state-of-the-art results in various natural language processing (NLP) tasks. However, this process often comes with significant computational costs, memory requirements, and time constraints. In this article, we'll explore SLoRA, a novel approach to efficient model fine-tuning that promises to revolutionize the way we work with LLMs. By leveraging sparse low-rank adaptation, SLoRA offers a faster, more cost-effective, and more sustainable way to fine-tune LLMs without sacrificing performance.

What is SLoRA?

To understand SLoRA, it's essential to first grasp the concept of Low-Rank Adaptation (LoRA). LoRA is a parameter-efficient fine-tuning method that updates only a small subset of model parameters while keeping the rest frozen. This approach has shown remarkable success in reducing computational costs and memory requirements for model fine-tuning.

However, LoRA still updates a relatively large number of parameters, which can be computationally expensive and memory-intensive. This is where SLoRA comes in – by introducing sparsity into the LoRA framework, SLoRA further reduces the number of updated parameters to approximately 1% of the original model's parameters. This drastic reduction in updated parameters leads to significant computational savings and faster convergence rates
SLoRA stands for Sparse Low-Rank Adaptation, a method designed to enhance the efficiency of fine-tuning LLMs. It builds on the concept of LoRA, which constrains the update of pre-trained weights using low-rank decomposition. SLoRA introduces sparsity into this approach, focusing only on a subset of parameters that significantly impact the model's performance.

Here's a simple analogy -- Imagine you have a giant jigsaw puzzle. LoRA would focus on updating a specific section of the puzzle, while SLoRA would pinpoint only the most important pieces within that section.

How SLoRA Works?

SLoRA employs a sparse matrix approach where the weight updates are constrained to a low-rank format and further sparsified. This involves decomposing the weight matrix into a product of two smaller matrices and applying updates only to a sparse subset of the original parameters. This method reduces the density of updates to about 1%, significantly cutting down on the resources needed for training.

Think of a weight matrix in an LLM as a giant grid of numbers. These numbers represent the connections between different parts of the model. SLoRA uses a technique called sparse matrix decomposition to break down this grid into smaller, more manageable pieces.

Imagine slicing a pizza into smaller triangles. SLoRA only updates the toppings on a select few of those triangles, leaving the rest untouched. This dramatically reduces the amount of data we need to process and store.

Comparison with Traditional Fine-Tuning Methods

Traditional fine-tuning involves updating all model parameters, which is resource-intensive and time-consuming. In contrast, SLoRA updates only a small, significant subset of parameters, achieving similar performance with much lower computational overhead.

SLoRA in the nutshell:

SLoRA builds on the foundation of LoRA (Low-Rank Adaptation), which already constrains updates to a small subset of parameters.
It takes this a step further by introducing sparsity – focusing on an even smaller, more crucial set of parameters.
The result? You're updating only about 1% of the model's parameters, dramatically reducing computational load and memory requirements.

You can read more here

Benefits:

Reduced Computational Requirements: By updating only a sparse subset of parameters, SLoRA dramatically lowers the computational load.
Faster Fine-Tuning Process: The sparse updates enable quicker convergence, speeding up the fine-tuning process.
Lower Memory Usage: The reduced number of updates translates to lower memory requirements, making it feasible to deploy models on devices with limited memory.
Potential for Improved Model Performance: Efficient parameter updates can lead to models that are not only faster but also potentially more robust and adaptable to specific tasks.

How to use SLoRA in Your Projects?

Initialize the Sparse Matrix: Begin by setting up the sparse matrix with low-rank decomposition.
Apply Sparse Updates: Update only the critical parameters as identified by the sparsity constraints.
Train the Model: Proceed with the fine-tuning process, leveraging the computational efficiency of SLoRA.

For more detailed guide checkout this paper: https://arxiv.org/pdf/2308.06522

SLoRA's Secret Sauce:

Here's where SLoRA really flexes its muscles. By dramatically reducing the number of parameters updated during fine-tuning, SLoRA processes fewer tokens. It's like having a car that can drive the same distance using only a fraction of the fuel.

Fewer tokens processed = Lower costs
Efficient updates = More bang for your buck
Optimized token usage = Stretch your budget further

Let's say you're fine-tuning a model on a specific task. Traditional methods might process millions of tokens, updating every parameter. With SLoRA, you're looking at a fraction of that - potentially cutting your token usage (and thus, your costs)!

Conclusion

SLoRA represents a significant leap forward in efficient model fine-tuning. It empowers developers to unlock the full potential of LLMs while being mindful of resources and costs.

Ready to give SLoRA a try? At Tune AI, we implemented this approach to, make models mode accessible and provide an efficient way to fine-tune and serve LLMs.
Head over to Tune Studio and start exploring! Share your experiences and let's build a more accessible and sustainable future for AI together.

Don't Be That Dev: Avoiding the XY Problem Trap

Abhishek Mishra — Tue, 20 Feb 2024 19:55:52 +0000

Have you ever found yourself in a situation where you're trying to solve a problem, but you're so focused on your attempted solution that you're not even sure what the actual problem is? This is a common pitfall known as the XY problem.

What is the XY Problem?

The XY problem is a communication issue that often arises in software development. It occurs when someone seeking help focuses on their attempted solution (X) rather than the actual problem they're trying to solve (Y). This can lead to misunderstandings, wasted time, and ineffective solutions.

Here's a classic example:

Imagine a developer named Abhishek is working on a web application that displays product information. He encounters an issue where product images take a long time to load, impacting the user experience.

The XY Problem Approach:

Instead of clearly stating the problem of slow image loading, Abhishek asks a question focusing on his chosen solution:

"How can I implement caching for product images in my application?"

Why it's a problem:

While caching might seem like a logical solution, focusing on it without explaining the context can lead to issues:

Misdiagnosis: The person providing help might suggest specific caching techniques without understanding the root cause of slow loading. This could be inefficient if the issue lies elsewhere, like slow image optimization or network problems.
Limited solutions: Exploring only caching might overlook alternative solutions like image compression or a Content Delivery Network (CDN) that could be more effective.
Wasted time: Both Abhishek and the other dev spend time discussing caching, which might not be the real solution.

Better approach:

Abhishek should rephrase his question to focus on the actual problem:

"I'm experiencing slow loading times for product images in my web application. This is affecting the user experience. Can you suggest ways to improve the loading speed?"

Benefits:

Clearer communication: The helper understands the desired outcome (faster image loading) and can suggest various solutions, including caching, optimization, or CDN, depending on the root cause.
Effective solutions: Exploring a wider range of options increases the chances of finding the most efficient and suitable solution.
Saved time: Focusing on the actual problem avoids unnecessary discussions about potentially irrelevant solutions.

Avoiding the XY Problem

Here are some tips to avoid falling into the XY problem trap:

Clearly define the problem you're facing: What are you trying to achieve? What specific issue are you encountering?
Provide context and relevant details: What have you tried so far? What is the expected outcome?
Be open to exploring different solutions: Don't get fixated on your initial idea.

By following these tips, you can ensure effective communication and collaboration when seeking help with software development problems.

Remember: It's always better to focus on the Y (the actual problem) before jumping to the X (solutions).

Reference:

For more information on the XY problem, you can refer to the Wikipedia page: https://en.wikipedia.org/wiki/XY_problem

Understanding Strings in Rust: String vs str

Abhishek Mishra — Mon, 10 Jul 2023 18:01:05 +0000

If you're a Python developer exploring the Rust language, you'll find that working with strings in Rust is quite different from Python. Rust has two main types to represent strings: String and str.

Let's dive into what they are and how they differ.

What are `String` and `str`?

String is a growable, heap-allocated data structure that allows you to store a sequence of UTF-8 characters. This is similar to Python's str type, but with one key difference - String in Rust is mutable and can be modified (Rust Doc: String).

On the other hand, str (pronounced 'string slice') is an immutable reference or 'view' into a string. It can point to the entire string or a subsection of it (Rust Doc: str).
You will most commonly encounter str in its borrowed form, represented as &str.
Also, &mut str is a mutable reference to a str. You can use it to perform in-place modifications to the string, as long as those modifications maintain valid UTF-8.

In Rust, str itself is a type that represents a sequence of valid UTF-8 characters, while &str and &mut str are references to such a sequence. The difference is that &str is an immutable reference (meaning you can't use it to change the underlying string), while &mut str is a mutable reference (so you can use it to change the underlying string, as long as you maintain valid UTF-8).

Why does Rust have two string types?

The two string types in Rust cater to different use cases.

String is used when you need to own a string and change its contents, like appending characters or changing a character at a certain position.

str is used for more 'read-only' operations. Since it's an immutable reference, it does not own the underlying memory and cannot alter it.

&mut str allows you to perform some in-place modifications to a string, as long as you maintain valid UTF-8. However, its use is less common and usually requires more care.

However, the mutable operations that you can do on &mut str are limited. For example, as the user mentioned, you can use the make_ascii_lowercase method to change all ASCII alphabetic characters to lowercase in place, but you can't use it to append to the string or change its length, because that could invalidate the UTF-8.

This is a different from Python, where you're used to working with just one main string type (str), which is always immutable.

Example

Let's look at an example:

fn main() {
    let mut s = String::from("hello");
    s.push_str(", world");  // This works fine
    println!("{}", s);
}

In this Rust program, we create a String and then append to it using the push_str method, something Python developers might be surprised to see, given that strings in Python are immutable.

Now, let's see what happens when we try to modify (change a specific character) a str:

fn main() {
    let mut s = "hello";
    s[0] = 'H';  // This will cause an error
    println!("{}", s);
}

In this Rust program, we're trying to replace the first character of the str s with 'H'. If s were a mutable String, we could do this with the replace_range method. But because s is a str, we cannot change its contents, and this program will not compile because str is immutable.

If we want to append a character to the string, we first have to convert it to a String with s.to_string(). Then, we can use the push method to append a character to the String.

For example:

fn main() {
    let s = "hello";
    let mut t = s.to_string();  // Convert to String to allow changes
    t.push('!');  // Append a character to the string
    println!("{}", t);
}

This program will compile and print "hello!".

Example of using &mut str

fn main() {
    let mut s = String::from("HELLO");
    let s_slice = &mut s[..];  // Obtain a &mut str from the String
    s_slice.make_ascii_lowercase();  // This works fine
    println!("{}", s);
}

In this program, we create a String and then obtain a &mut str slice from it using &mut s[..]. We then call the make_ascii_lowercase method on the &mut str to change all ASCII alphabetic characters to lowercase. The program will print "hello".

Conclusion

String and str serve different purposes. Use String when you need to own and modify string data, and use str when you need a temporary, read-only view into a string.

Rust string types give you more control over memory allocation and can help prevent bugs related to memory access. This design choice aligns with Rust's core goals of performance and safety. It's one of the ways Rust differs significantly from Python and many other languages, and it's something you'll get used to as you write more Rust code.

For more detailed information, refer to the official Rust documentation for String and str.

Creating a Low-Latency Cache with Dozer and PostgreSQL

Abhishek Mishra — Tue, 30 May 2023 05:56:29 +0000

In the realm of software development and data management, the efficiency of data retrieval plays a crucial role in the overall performance of an application. One effective way to enhance this efficiency is by implementing a cache. A cache is a high-speed data storage layer that stores a subset of data, typically transient in nature so that future requests for that data are served up faster than is possible by accessing the data’s primary storage location.

This blog post explores how to enhance these aspects by implementing a low-latency cache using Dozer and PostgreSQL. We delve into the importance of caching in data management, the limitations of not having a cache, and the unique advantages of using Dozer as a cache layer. We also provide a guide on how to use Dozer for real-time data ingestion from PostgreSQL. This post is a must-read for developers seeking to improve their application's performance and responsiveness, and for those looking to simplify their data infrastructure.

Why Do We Need a Cache on Top of PostgreSQL?

PostgreSQL is a powerful, open-source object-relational database system. It is highly extensible and enables high performance on complex queries and heavy loads. However, like any other database, it has its limitations.

When a database grows in size and complexity, the time taken to retrieve data can increase significantly. This is especially true for complex queries that involve multiple joins and aggregations. In such scenarios, every millisecond counts, and the latency can add up quickly, leading to a slow and unresponsive application.

This is where a cache comes into play. A cache is a high-speed data storage layer that stores a subset of data so that future requests for that data are served up faster than is possible by accessing the data’s primary storage location. By storing frequently accessed data in a cache, we can significantly reduce the time taken to access this data.

What's the Problem with Not Having a Cache?

Without a cache, every time a client requests data, the application has to query the database. This can be a time-consuming operation, especially if the database is large or the query is complex. This can lead to increased latency and a poor user experience.

Moreover, without a cache, the database can become a bottleneck, especially under heavy load. Every read and write operation involves a significant amount of work, and as the number of operations increases, the time taken to complete these operations also increases. This can lead to increased CPU and memory usage, and in extreme cases, it can even lead to database crashes.

Why Not PostgreSQL Read Replica?

While a PostgreSQL read replica can certainly help offload some of the read operations from your primary database, it's not without its limitations. The process of querying a read replica still involves a network round trip and the overhead of a database query. These factors can contribute to latency, especially when dealing with large volumes of data or complex queries. A cache like the one provided by Dozer, on the other hand, can serve data much faster than a read replica.

In terms of using a read replica instead of a cache layer like Dozer, it depends on your specific use case and requirements. A read replica can help improve read performance by distributing read traffic across multiple instances, but it may not provide the same level of performance improvement as a cache layer like Dozer.

Moreover, Dozer doesn't make a distinction between types of data sources. Developers can get a seamless experience building products with application databases such as Postgres and MySQL, data warehouses such as SnowFlake and cloud storage such as S3 and Deltalake. Dozer can also consume real-time events and Ethereum data.

Why Dozer?

In the context of database management, most OLTP workloads involve random disk I/O usage. Given that disks, including SSDs, are slower in performance than RAM, database systems use caching to increase performance. Caching is all about storing data in memory (RAM) for faster access at a later point in time.

One approach to database caching involves replicating data using Change Data Capture (CDC) to an alternative database optimized for your queries. This method is increasingly being adopted for operations that require extensive data reading. However, it comes with its own set of challenges such as ensuring real-time data, managing indexing, guaranteeing availability, handling schema changes, and considering costs. Dozer addresses these pain points by providing an end-to-end system that takes care of the complexity of caching logic, allowing developers to focus on building their applications.

Dozer, an open-source data API backend, provides a solution to this. It connects to any of your data sources, transforms and stores the data in an embedded cache powered by LMDB, automatically creates secondary indexes, and instantly generates low-latency REST and gRPC APIs. Dozer is implemented fully in Rust for fast processing of data and is less resource intensive.

Without a cache layer, data retrieval would rely solely on the performance of the PostgreSQL database and its internal caching mechanisms. This can result in slower response times and increased load on the database server. By using a cache layer like Dozer, you can improve the performance of data retrieval and reduce the load on the database server.

How to Use Dozer as a Cache

Architecture Diagram

Here's a simple architecture diagram showing how Dozer can be used as a cache layer for a PostgreSQL database:

In a nutshell:

Dozer begins by ingesting data in real-time from the PostgreSQL database using the PostgreSQL connector and stores this data in its cache.
When a client sends a request for data, Dozer processes the request using the low-latency APIs it has generated, and retrieves the required data from its cache to serve the client.
For write requests, these are facilitated through the backend API, which directly communicates with the PostgreSQL database. Once the write operation is confirmed, the PostgreSQL database sends a write confirmation back to the backend API.

The key point here is that data ingestion into Dozer's cache and the serving of data to the client through Dozer's APIs occur in parallel. This setup ensures quick and efficient access to data, enhancing the performance of your application."

Dozer provides a PostgreSQL connector that allows it to ingest data in real-time from your PostgreSQL database. The configuration can be as simple as the following:

app_name: movie-booking
connections:
  - config: !Postgres
      user: postgres
      password: postgres
      host: 0.0.0.0
      port: 5438
      database: database_name
    name: connection_name

sql: |
  -- transformation and
  -- analytical queries here.

sources:
  - name: source_name_1
    table_name: table_name
    columns:
    connection: !Ref connection_name

  ...

endpoints:
  - name: endpoint_name
    path: /endpoint_path
    table_name: table_name
    index:
      primary_key:
        - primary_key_column_name

This configuration allows Dozer to ingest data in real-time from your PostgreSQL database and store it in a high-performance cache. This cache can then be used to serve low-latency data APIs to your application.

Conclusion

Implementing Dozer with its ability to ingest data in real-time from PostgreSQL can significantly improve the performance of your application by reducing the time taken to access frequently used data. This approach can significantly enhance your application's performance, making it more efficient and responsive in today's demanding tech scenario. By leveraging Dozer's real-time SQL engine, automatic secondary indexing, and instant API generation, developers can drastically lower the cost, complexity, and effort involved in putting together the data infrastructure necessary to build data applications.

Happy coding, Happy Data APIng! 🚀👩‍💻👨‍💻

Secure Your Data APIs with Dozer: Adding Authorization to APIs

Abhishek Mishra — Thu, 25 May 2023 14:35:50 +0000

In today's world of API-driven applications, securing access to data is very important. As the amount of data generated and consumed continues to grow, the need for reliable, and efficient data access management becomes increasingly critical. One essential aspect of data security is ensuring proper authorization for API calls, allowing only the right people, applications, and services to access the required data. Managing access control for your APIs is crucial to guarantee that only authorized users can access and perform specific actions. With the increasing complexity of API ecosystems, secure and efficient authentication and authorization mechanisms are necessary.

Dozer addresses this challenge by adding authorization layer to your data APIs in an efficient way. One of the ways Dozer provides authorization is through the use of JSON Web Tokens (JWT). JWT is a widely used industry-standard method for representing claims securely between two parties. You can learn more about JWT here. Using JWT tokens simplifies the process of adding authorization to your data APIs.

To demonstrate this, we'll build a simple movie ticket booking app that has two roles - user (public) and admin. The user can only book tickets and view their bookings, whereas the admin has full access to all the APIs. We will use @dozerjs/dozer and dozer-react to enhance the functionality of our app. @dozerjs/dozer is a TypeScript wrapper over gRPC APIs generated by Dozer, facilitating seamless communication with Dozer, while dozer-react provides a set of React helpers for using Dozer as a data provider. The sample application's repository can be found here, you can aslo find the instructions to run the application directly.

By following the step-by-step guide we've provided, you can safeguard your application from unauthorized access, ensuring data integrity and user trust.

TL;DR

Clone the repo and change to the project directory:

git clone git@github.com:getdozer/dozer-samples.git && cd dozer-samples/usecases/api-auth/

Start the PostgreSQL database:

cd database && docker-compose up -d && cd ..

Generate and export a master token for auth:

dozer api generate-token
export MASTER_TOKEN=your_token_here

Start the Dozer app:
```
dozer
```
In a new terminal, install server dependencies and run the server app:
```
cd server && yarn install && node index.js
```
In a new terminal, navigate to the client directory, install client dependencies and start the app:

    cd ../client && yarn install && yarn start

Visit localhost:3000 to use the app. For the admin dashboard, go to localhost:3000/admin/dashboard with "admin" as username and password.

User Page

Admin dashboard

Introduction

In this blog, we'll guide you through the process of constructing a movie ticket booking application using Dozer, React, and PostgreSQL. The PostgreSQL database is utilized directly in our server-side application to manage user creation and ticket bookings.

To elaborate, the server app is specifically designed to handle user login, create new user entries, and register ticket bookings. It interacts directly with the PostgreSQL database to handle these tasks. The information required for booking tickets, such as booking details, is sent from the React frontend application to the Node.js server app, which then inserts these details into the PostgreSQL database.

However, when it comes to consuming real-time data like the movie list, dashboard updates, or ongoing bookings, we use Dozer, which fetches the data from PostgreSQL using the PostgreSQL connector. This strategy allows us to get the most recent booking updates and display them on the dashboard in real-time.

The PostgreSQL connector in Dozer bridges our database with the Dozer server and enables the seamless transfer of data in real-time. You can learn more about this connector at Dozer's documentation page here: PostgreSQL Connector.

On the client side, we're using a React app to display the movie data and booking interface, pulling data from Dozer. We employ the @dozerjs/dozer and dozer-react libraries to simplify interactions with Dozer and access real-time data without having to hardcode API endpoints in the code.

The @dozerjs/dozer library manages gRPC requests between the frontend and the Dozer server, allowing effective communication between the two. In contrast, the dozer-react library provides a set of React helpers, making it simpler to use Dozer as a data provider in our React app. Find more about these libraries at their respective GitHub repositories: @dozerjs/dozer and dozer-react.

Let's dive into the process of building this application, demonstrating how you can leverage Dozer to amplify your app development experience.

Server-Side Setup (Express.js)

First, let's set up the server-side code using Express.js. The code is available in server/index.js:

const express = require('express')
const app = express()
const {ApiClient, AuthClient} = require("@dozerjs/dozer");
const axios = require('axios');
const cors = require('cors');
const { faker } = require('@faker-js/faker');

//... Rest of the code

We import the required libraries, including express, @dozerjs/dozer, axios, cors, and @faker-js/faker.

Database Connection for Server-Side

Setting up a connection with a PostgreSQL database is crucial to perform operations like user creation and booking entries. We accomplish this using the pg client.

const { Client } = require('pg')

const client = new Client({
    host: '127.0.0.1',
    port: 5432,
    database: 'omdb',
    user: 'postgres',
    password: 'postgres',
});

client.connect();

In the configuration object, we specify the parameters like host, port, database name, user, and password. With the successful connection established via client.connect(), we can execute queries to our database in our Express.js routes.

This setup enables two primary functionalities in our application:

User creation: With the database connection ready, we can insert new user details into the users table. This is used in the /public/new_user endpoint, where a new user is created with a unique UUID and a randomly generated name.
Booking entries: The connection also allows us to create new entries in the bookings table. This functionality is used in the /public/book_movie endpoint, where a user can book a movie, resulting in a new booking record in the database with the user's id and the provided movie id.

Admin Login Endpoint

This endpoint validates the provided username and password. If valid, a post request is made to the authentication server to get a token. The token is then sent back to the client. Error handling is also set up to send a 500 status code in case of a server error and a 400 status code for incorrect credentials.

app.post('/admin/login', function(req, res){
    // Check if the provided credentials are valid
    // Generate JWT token with full access to all API endpoints
    // Send the JWT token to the client
})

Public New User Endpoint

The new user endpoint generates a random username and UUID using the faker library. It then inserts this new user into the database. After successful insertion, a JWT token is generated for the new user with limited access. Error handling is set up in case the server encounters an error while processing the request.

app.get('/public/new_user', function(req, response){
    // Generate random username and UUID
    // Insert new user into the database
    // Generate JWT token with limited access for the new user
    // Handle server errors
})

Public Book Movie Endpoint

The book movie endpoint allows users to book a movie by sending their JWT token in the headers. The server first validates the user's token. If valid, it inserts a new record in the bookings table in the database with the user's id and the provided movie id. The response includes error handling as well.

app.post('/public/book_movie', function(req, response){
    // Validate the user's token
    // Insert a new record in the bookings table
    // Handle server errors
})

Error Handling Middleware

Lastly, we also add a middleware function to handle any errors that may slip through our route handlers. This middleware logs the error and sends a generic error message to the client.

app.use((err, request, response, next) => {
    // log the error, for now just console.log
    console.log(err)
    response.status(500).send('Something broke!')
})

After setting up these routes, the Express.js application is ready to listen for requests on port 4000.

app.listen(4000)

Our error handling middleware ensures that any errors that occur during these database operations are effectively captured and appropriately communicated to the user.
In this way, our server is set up to handle the creation of new users, allow users to book movies, and allow an admin to log in, all while ensuring errors are handled appropriately. .

Database Setup For Movie data (PostgreSQL)

We use PostgreSQL as our database, and the configuration is available in the docker-compose.yml file. The file defines a postgres service, which includes the container name, image, volumes, and environment variables.

version: '3.8'
services:
  postgres:
    container_name: omdb-postgres
    image: debezium/postgres:13
    volumes:
      - ./pg_hba.conf:/var/lib/foo/pg_hba.conf
      - ./data/init.sql:/docker-entrypoint-initdb.d/init.sql
      - ./data:/data
    command: postgres -c hba_file=/var/lib/foo/pg_hba.conf
    environment:
      POSTGRES_DB: omdb
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      ALLOW_IP_RANGE: 0.0.0.0/0
    ports:
      - '5438:5432'
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U postgres -d omdb" ]
      interval: 5s
      timeout: 5s
      retries: 5

The database is populated with the Open Movie Database (OMDB) dataset, which provides a comprehensive list of movie-related data. For further details about this dataset, you can visit the official OMDB repository here: OMDB on GitHub

Front-End Setup (React)

The frontend of our application is designed with React, a popular JavaScript library used for building interactive UIs. The frontend setup for our movie booking application involves a series of React components that interact with our backend server using HTTP requests. These components handle user authentication, data fetching, and real-time updates to ensure users have an interactive and up-to-date interface.

Let's take a look at the structure of our project:

├── README.md
├── package.json
├── src
│   ├── App.css
│   ├── App.js
│   ├── App.test.js
│   ├── components
│   │   ├── admin
│   │   │   └── dashboard.js
│   │   ├── login.js
│   │   └── public
│   │       ├── movies.js
│   │       └── moviesList.js
│   ├── index.css
│   ├── index.js
│   ...
└── yarn.lock

Our React components are placed in the src/components directory, separated by user type (public users and admins).

Login Component

The login.js component facilitates the login functionality for the admin users. It includes text fields for the admin to enter their username and password, and a login button to submit the login details.

On submission, a POST request is made to the http://localhost:4000/admin/login endpoint with the username and password as the body of the request. If the credentials are valid, the response will contain a token, which is stored in the browser's local storage and used to authenticate future requests.

Admin Dashboard Component

The dashboard.js component acts as the main interface for admin users. On load, it checks if a token exists in local storage. If not, it redirects the user to the login page.

Using the movies_with_bookings query, it fetches the list of movies with their booking counts and displays them in a table. This component uses real-time updates to ensure the admin always has the most current booking data.

Movie List Component

The moviesList.js component displays the list of movies and their current booking counts to public users. It allows users to "buy" a ticket for a movie, which is simulated by making a POST request to the http://localhost:4000/public/book_movie endpoint with the selected movie's id. The state of this component is updated in real-time as bookings are made.

Movies Component

The movies.js component acts as a wrapper for the moviesList.js component. On load, it checks for a token in local storage. If a token doesn't exist, it makes a GET request to the http://localhost:4000/public/new_user endpoint to generate a new token. Once a token is obtained, it passes the token as a prop to the MoviesList component.

Integrating Dozer JWT in the Application

Having established the foundation of our application in the previous sections, it's time to delve into the secure integration of JWT authentication and Role-Based Access Control (RBAC) using Dozer. Our focus will be on illustrating how this can be applied in our movie ticket booking application.

Outlined below are the key steps to enable secure API authorization in your application:

Generate a master token using Dozer's API server.
Using the master token, generate custom access tokens for each set of APIs to establish granular-level access.
Incorporate the generated custom access tokens in subsequent client-side API calls.

1. Master JWT Token Generation

Kick off the process by running the following command to generate a master JWT token:

dozer api generate-token

This command triggers the generation of the master token, which is outputted to stdout. The master token has the privilege of accessing all API endpoints specified in the config file. To access the APIs, include the token in your request header as demonstrated below:

MASTER_TOKEN = 'your_master_token_here'

2. Custom Access Token Generation

For a more granular access control, Dozer allows for the creation of custom access tokens for specific API calls. These tokens can be generated through a POST request to the /auth/token endpoint, providing an access filter and the previously generated MASTER_TOKEN in the request header.

This affords you the opportunity to designate specific access controls for different users or roles within your application. For a more detailed guide on this, you may refer to the Dozer documentation page on Authentication & Authorization.

In the context of our movie ticket booking app, we generate a JWT token comprising the necessary access controls whenever an admin logs in or a new user is registered. This token is then dispatched to the client and utilized in future API calls.

Here is an illustration of the token generation process in the index.js file located in the server directory:

//...other code
const MASTER_TOKEN = process.env.MASTER_TOKEN;
app.post('/admin/login', function(req, res){
    //...other code
    const data = JSON.stringify("All");
    const config = {
        method: 'post',
        url: 'http://127.0.0.1:8080/auth/token',
        headers: {
            'Authorization': 'Bearer ' + MASTER_TOKEN,
            'Content-Type': 'application/json'
        },
        data : data
    };
//...other code

Furthermore, we can customize the access token to limit access based on the user's role:

//...other code
// Generate token with limited access based on user role
const data = JSON.stringify({
   //...other code
});
//...other code

In the code snippet above, the custom access token restricts access to only the need_to_Add_endpoint_name API endpoint. The access filter can be tweaked to include other endpoints and set specific permissions anchored on user roles.

3. Incorporating Tokens in Client-Side API Calls

Once the JWT tokens are generated, they can be utilized client-side for secure API access. This can be seen in the movies.js file and other components housed in the public directory.

//...other code
function Movies() {
    let [token, setToken] = useState(null);

    useEffect(() => {
        if (localStorage.getItem('public_jwt')) {
            setToken(localStorage.getItem('public_jwt'));
        } else {
            //...other code
        }
    }, []);
//...other code

Whenever the client makes an API call, the server validates the JWT token and checks if the user has the necessary permissions to access the requested endpoint. If the token is valid and the user will get the valid response.

Wrapping Up:

Using Dozer's powerful security capabilities, you can easily add authorization to your data APIs, ensuring that your application remains secure and accessible only to authorized users. Whether you're building a movie ticket app or any other application, Dozer provides a flexible and straightforward way to manage access control for your APIs.

The beauty of Dozer lies in its flexibility—it moulds itself according to your needs, allowing you to craft custom access controls for different user roles, thereby ensuring the right balance between functionality and security.

Through our movie ticket booking application, we've demonstrated just a fragment of the potential that Dozer holds. Regardless of your application's domain—be it e-commerce, social media, healthcare, or any other field—you can leverage the power of Dozer to secure your data APIs.

In conclusion, Dozer presents a robust, scalable, and straightforward solution to facilitate API security and data management. It's a testament to the flexibility of modern software tools and their ability to adapt to a wide range of application needs. We hope this guide has clarified the process of secure API management and inspired you to integrate Dozer into your next project.

Happy coding, Happy Data APIng! 🚀👩‍💻👨‍💻

ES Modules or CommonJS: What's the Deal and Why Should I Care ?

Abhishek Mishra — Wed, 10 May 2023 10:53:12 +0000

Today, I'm on a quest to untangle a little confusion that's been bothering me (as a JS noob 👶)- and maybe you, too.

It's all about JavaScript module systems: ES Modules and CommonJS. Funny enough, I stumbled upon this while trying out a library, @dozerjs/dozer, a JavaScript library to interact with Dozer, a data API backend.

Alright, here's the deal. Back in the day, JavaScript didn't have a built-in module system. So, dev's couldn't easily separate their code into different files or manage dependencies. It was kind of a mess. But then, Node.js stepped up with its own module system, CommonJS. And later, ES6 (ES2015) introduced ES Modules.

Now, you might be thinking, "Why should I, as a web dev, even care about this? 🤔" Well, it's simple. The module system you choose can affect your code's structure, how fast it runs, and even its compatibility. Trust me, I learned this the hard way when I faced some import issues while running a node app with @dozerjs/dozer. Let's dive in!

CommonJS: This is like the godfather of JavaScript module systems. It uses the require() function to import modules and module.exports to export them.

The cool part? You can use require pretty much anywhere - inside functions, loops, or based on some conditions.
The not-so-cool part? Imports are synchronous, meaning they could slow things down a bit.

Here's a little example:

// Importing a module
const express = require('express');

// Exporting a module
module.exports = someFunction;

ES Modules: These are the new kids on the block, coming in with ES6. Instead of require() and module.exports, they use import and export. These modules are statically analyzed, which means the imports and exports are determined before the code runs. This can make your code run faster and gives you some extra tooling perks.

Here's how it looks:

// Importing a module
import express from 'express';

// Exporting a module
export default someFunction;

So, which one do you choose? 🤹‍♀️**

If you're writing code for the browser, ES Modules are a great choice because modern browsers support them natively. For Node.js, it does support ES Modules, but you might still find yourself using CommonJS for compatibility reasons.

Got some older browsers or Node.js versions to support? They might not be fans of ES Modules. In that case, you could stick with CommonJS or use a tool like Babel to compile your ES Modules to CommonJS.

The trend is definitely leaning towards ES Modules, but there's still a ton of CommonJS code out there. So, it's a good idea to know your way around both.

Remember, the choice is yours. Pick the module system that fits your project and team the best.

A Quick Example: Switching to CommonJS in Node.js

I thought it might be handy to wrap this up with a little example. So, let's say you're working on a Node.js app and you've been using ES Modules. And if want to switch back to CommonJS. How do you do it?

Well, first, you'll need to change your import and export statements. Here's an example:

// ES Module import
import express from 'express';

// Change to CommonJS require
const express = require('express')

But wait, there's another step! You also need to update the type in your package.json file. If you have "type": "module", which allows you to use ES Modules, you'll want to remove it to switch back to CommonJS.

Here is the package.json example:

{
  "name": "dozer_lib_js_test",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "type": "commonjs",
  "dependencies": {
    "@dozerjs/dozer": "^0.0.6"
  }
}

I hope this helps!

Announcing the Dozer Appathon!

Abhishek Mishra — Wed, 03 May 2023 05:59:55 +0000

👋🏼 Are you looking for a platform that simplifies the process of building and deploying data-driven applications? Look no further than Dozer ! 💻 is a powerful Data API backend that simplifies the process of building and deploying data-driven applications & APIs for real-time data processing and analysis, predictive analytics, data caching, and much more.

And now, you have the chance to showcase your coding skills and creativity by participating in the Dozer Appathon! 🚀 This exciting event is open to everyone who is interested in building applications with Dozer. The submission period runs from May 3 to May 9, 2023; so you still have time to get your app ready! ⏰

Dozer Appathon Hacker Guide 👩‍💻👨‍💻

Introduction 🚀

Welcome to the Dozer Appathon! This is your chance to showcase your coding skills and creativity by building amazing applications using our platform. Dozer is a powerful Data API backend that simplifies the process of building and deploying data-driven applications & APIs.

Sample use cases📝

To help you get started, we have prepared a list of sample use cases and ideas for your app. You can use these as inspiration for your project, or come up with your own unique ideas.

Some sample use cases are

- Real-time data processing and analysis
- Data aggregation and consolidation from multiple sources
- Machine learning and predictive analytics
- Data caching and optimization for faster performance
- IoT data processing and analysis

Some wonderful project ideas💡

Real-time Eth layer 2 crypto dashboard📈
NodeJS-based Twilio notifications with real-time alerts using Dozer📲
Real-time ML predictions using ONNX model🧠
React Native push notifications📱
Klarna clone with user profiles and transactions in React Native💳
Real-time vehicle tracking on Google Maps with traffic density alerts (Web or React Native)🚗
Real-time inventory dashboard for a retail chain with multiple warehouses🛍️
Fraud detection for credit card transactions using ML models on Python Lambda💳🕵️‍♂️
Model training and scoring with data injection through Python Lambda 🧑‍💻📈

Prizes 🎁

We have some exciting prizes up for grabs!

The top 5 submissions will win an amazing Mechanical Keyboard and a Dozer branded Swagpack 🏆
The next 10 successful submissions win Dozer Branded swagpack which includes an awesome tee and stickers. 🎉
Additionally, all participants will receive a certificate of participation.📜

Resources 📚

To help you build your app, we have some resources for you:

Website: https://getdozer.io/ 🌐
Docs: https://getdozer.io/docs/dozer 📖
Blogs: https://getdozer.io/blog 📰
Blog Tutorial: https://getdozer.io/blog/building-real-time-data-app-using-dozer-react-postgresql/ 📚
Video Tutorial: https://youtu.be/pVfyMfNrlD4 📹
Discord Community support: If you have any questions or need help with your project, you can reach out to #dozer-appathon channel of the Dozer Discord for support. 💬

Judging Criteria 🧐

We will rate as per optimal utilisation of Dozer in your app on a scale of 0-10, based on factors like real-time data, multiple data sources, and caching requirements. We'll also consider the problem you're solving and the code quality of your app.

Submission Method 📩

Submit a PR following the Contribution Guidelines with your project detail.
Fill out this form post PR submission.
Winners will be announced in the Discord Server.💪🏆

Stargazers’ Raffle✨

Star our repository and get a chance to win an amazing Swagpack!!🎁🎉
We will announce the winners in our Discord server. So, don’t forget to join.🤗

We're excited to see what you can create with Dozer!
Good luck and Happy Hacking!, Happy Data APIng 🚀 👩‍💻👨‍💻

For more details, checkout: https://getdozer.io/blog/dozer-appathon

Unleashing the Power of Dozer Lambda Runtime for Real-time and Event-driven Data Apps

Abhishek Mishra — Tue, 02 May 2023 14:21:50 +0000

Real-time data processing is essential for modern applications, as it ensures that insights are timely and accurate. The open-source project Dozer now provides a runtime environment that allows developers to execute lambda functions or custom user-defined functions based on specific data conditions.
This new functionality allows for more dynamic and event-driven applications, as Lambda functions can be triggered based on specific data change conditions. Just imagine triggering a Twilio notification based on a SQL statement (data change condition)!

In this blog post, we will discuss how you can use Dozer to respond to real-time data changes by triggering functions based on data conditions.
In this example, we will create a React app that generates a real-time graph based on room temperature data (room temperature monitoring). We will use the pydozer_log module to react to data change conditions and trigger a Twilio message if the room temperature goes above 22.9°C.

Dozer Lambda Runtime: A Glimpse

The Dozer Lambda runtime is a major enhancement to the Dozer project, enabling developers to write Lambda functions in JavaScript and Python that react to Dozer events.

To help you better understand the Dozer Lambda runtime, we will examine the following state diagram:

DCC: Data Change Condition

Data Source: Your application's data source, such as a database, sends data to the Dozer pipeline.
Dozer Pipeline: The data is processed through the Dozer pipeline, which can apply various transformations and aggregations based on the configuration you provide.
Lambda Runtime: The Dozer Lambda runtime monitors the pipeline for specific data change conditions that you define. If a data change condition is met, the corresponding Lambda function is triggered.
Action/Trigger: The triggered Lambda function performs the desired action, such as sending a Twilio notification or updating a database record.

The reponse of trigger can be used to write back top dozer, for example using gRPC service.

JavaScript and Python Bindings for Reading Dozer Logs

Dozer now provides bindings for both JavaScript and Python to read logs, making it easier for developers to work with the languages they are most comfortable with. These bindings are essential components of the Dozer Lambda runtime, as they enable developers to create Lambda functions that respond to
events.

React App Example: Room Temperature Monitoring

In this example, we will create a React app that generates a real-time graph based on room temperature data. We will use the pydozer_log module to react to data change conditions and trigger a Twilio message if the room temperature goes above 22.9°C.

Here are the steps to run the app:

Clone the Dozer Samples repository and install dependencies:

git clone https://github.com/getdozer/dozer-samples.git
cd dozer-samples/usecases/lambda/
pip install -r requirements.txt

Remove any existing Dozer configuration and start the Dozer:

rm -rf ./.dozer && dozer -c config/dozer-config.yaml

Run the Python scripts to read data from the pipeline:

Note: make sure to replace the Twilio configuration in reader.py, with your account id and auth token. To learn more about Twilio SMS setup checkout this getting started guide.

python reader.py
python main.py

Navigate to the React app folder, install the necessary packages, and start the development server:

cd react
yarn install && yarn start

Once the app is running, you should see a real-time graph of the room temperature data. If the temperature exceeds 22.9°C, a Twilio message will be triggered, notifying the relevant parties of the situation.

Conclusion 🎯

By using the Dozer Lambda runtime, you can create event-driven applications that automatically respond to changes in your data, making your applications more efficient, scalable, and responsive to user needs. By leveraging the Python and JavaScript libraries provided by Dozer, developers can easily build applications that respond to real-time data changes, making it an invaluable resource for modern applications.

Happy coding, Happy Data APIng! 🚀👩‍💻👨‍💻

🚀 Getting Started with Dozer with Docker: A Step-by-Step Tutorial

Abhishek Mishra — Mon, 24 Apr 2023 14:00:00 +0000

In the world of data-driven applications, accessing and manipulating data in real-time is crucial. Dozer is a powerful tool that helps you create low-latency data APIs (gRPC and REST) from any data source, enabling seamless integration with frontend applications. In this tutorial, we'll walk you through setting up Dozer with Docker and connecting it to PostgreSQL using Docker.

🌟 Why Dozer?

Dozer addresses the challenges of accessing and manipulating data in real-time by providing an easy-to-use solution to create efficient data APIs. With Dozer, you can quickly set up APIs that can be easily integrated with your frontend applications. Furthermore, using Docker allows you to containerize your application, making it portable and easy to manage.

In this step-by-step tutorial, we'll guide you through connecting PostgreSQL with Dozer using Docker. Let's get started!

🔧 Prerequisites:

Docker/Docker desktop installed on your machine
Basic knowledge of Docker and PostgreSQL

📝 Overall steps involved:

Create a docker-compose file for PostgreSQL
Create a Dozer config with PostgreSQL connector and API configuration
Run PostgreSQL (via Docker)
Run Dozer (via Docker)
Query your API

🔹 Step 1: Create a `docker-compose.yml` file

First, create a docker-compose.yml file and copy the following configuration to it or download the file here:

version: '3.9'
services:
  postgres:
    platform: linux/amd64
    container_name: quick-start-postgres
    image: public.ecr.aws/getdozer/dozer-samples-pg-stocks:latest
    command: postgres -c hba_file=/var/lib/stock-sample/pg_hba.conf
    environment:
      POSTGRES_DB: stocks
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      ALLOW_IP_RANGE: 0.0.0.0/0
    ports:
      - '5434:5432'
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U postgres  -h 0.0.0.0 -d stocks" ]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  sample-data:

🔹 Step 2: Create a Dozer config file

Next, create a dozer-config.yaml file with the following content:

app_name: 1-hypercharge-postgres-sample
connections:
  - config: !Postgres
      user: postgres
      password: postgres
      host: host.docker.internal
      port: 5434
      database: stocks
    name: stocks
sources:
  - name: stocks
    table_name: stocks
    columns:
      - id
      - ticker
      - date
      - open
      - high
      - low
      - close
      - adj_close
      - volume
    connection: !Ref stocks
  - name: stocks_meta
    table_name: stocks_meta
    columns:
      - symbol
      - security_name
    connection: !Ref stocks
endpoints:
  - name: stocks
    path: /stocks
    table_name: stocks
    index:
      primary_key:
        - id
  - id: null
    name: stocks_meta
    path: /stocks-meta
    table_name: stocks_meta
    index:
      primary_key:
        - symbol

🔹 Step 3: Run PostgreSQL with sample stock data

Open a terminal or command prompt and navigate to the directory containing the docker-compose.yml file. Run the following command to start PostgreSQL with sample stock data:

docker-compose up

🔹 Step 4: Run Dozer with PostgreSQL connector (via Docker)

Open another terminal or command prompt and navigate to the directory containing the dozer-config.yaml file. To run Dozer as a Docker container, execute the following command:

docker run -it \
       -v "$PWD":/usr/dozer \
       -p 8080:8080 \
       -p 50051:50051 \
       --platform linux/amd64 \
       public.ecr.aws/k7k6x1d4/dozer \
       dozer

Congratulations 🥳 , you should see Dozer up and running!

🎉 Query your API

Dozer automatically generates APIs in both REST and gRPC formats, along with documentation, OpenAPI, and proto files. In the following sections, we'll showcase some possible queries. Refer to the APIs section for more details.

🔸 Query Using gRPC:

You can use gRPCurl or Postman to interact with gRPC APIs.

gRPC query example:

grpcurl -plaintext localhost:50051 dozer.generated.stocks_meta.StocksMetas/query

🔸 Query Using REST:

You can use curl or Postman to interact with REST APIs.

REST query example:

curl --location --request GET 'localhost:8080/stocks-meta'

That's it! You've successfully set up Dozer with Docker and connected it to PostgreSQL. Now you can easily create APIs for your data and integrate them into your frontend applications.

If you're interested in learning more about how to use Dozer with applications, we recommend reading our previous blog post: "Building a Real-Time Data App with Dozer, React, and PostgreSQL". In this blog, you'll find a detailed tutorial that demonstrates how to integrate Dozer with a React frontend application and PostgreSQL as a data source. This comprehensive guide will give you a better understanding of how to build powerful, real-time data-driven applications using Dozer.

Happy coding, Happy Data APIng! 🚀👩‍💻👨‍💻

Dozer Goes Open Source: Empowering the Community to Build Real-time Data Apps

Abhishek Mishra — Fri, 21 Apr 2023 15:56:07 +0000

We are excited to announce that Dozer, a powerful platform for building low-latency data APIs (gRPC and REST) from any data source, is now open source! The Apache 2.0 license 🎉. With this move, we aim to empower the community to build and scale real-time data applications more effectively.

Dozer simplifies the process of connecting applications to various data sources, such as PostgreSQL, Kafka, or other databases & sources, enabling developers to easily create and manage real-time data APIs. By making Dozer open source, we are inviting developers to contribute to its growth and help shape the future of real-time data applications.

What is Dozer?

Dozer is a platform that enables developers to build low-latency data APIs (gRPC and REST) from any data source. It simplifies the process of connecting applications to data sources like databases, APIs, or other services, and automatically manages data streaming and synchronization. This allows developers to focus on building powerful applications that utilize real-time data without worrying about the complexities of managing data connections and APIs.
With a few lines of SQL and a simple YAML configuration, you can build, deploy and maintain full data backends. It also has support for client libraries in JS and React to easily integrate with front-end applications.

Why Open Source?

There are several reasons why we decided to make Dozer open source:

Community-driven innovation: We want developers from all over the world to contribute their ideas,and provide valuable insights, improvements, and fixes, leading to a more innovative and robust platform, and faster development cycles.
Transparency and trust: We want to enable users to view and understand the underlying code, fostering trust in the platform and ensuring that it meets their needs and expectations. Especially how data is being stored and processed.
Collaboration and learning: We encourage developers to collaborate, share ideas, and learn from one another. We want to foster a strong community that helps developers grow their skills and expertise in the real-time data space.

Getting Started with Dozer

To help you get started with Dozer, we've created documentation and tutorial blogs that guide you through the process of setting up, configuring, and using Dozer for your data applications. You can find these resources on our official documentation site and can also follow Dozer on dev.to or additional resources on getting to know & use Dozer better!

How to Contribute?

We welcome contributions from the community! If you're interested in contributing to Dozer, please check out our GitHub repository for the guidelines. Whether you want to submit a bug report, suggest a new feature, or contribute code, we appreciate your help in making Dozer even better.

We also encourage you to check out the following resources to know other ways of contributions:

Show and tell GitHub discussion forum: We have setup a GitHub discussion forum, where you can share your Dozer projects, and experience, ask questions, do a feature request, and connect with fellow Dozer community members. Feel free to join the conversation and share your own projects & ideas! Additionally, we have also created a discord channel to give developers an opportunity to showcase and talk about their projects. Join our discord channel, If you’ve built something using Dozer, we’d love to see it!
Dozer Samples Repository: We have created a Dozer Samples repository on GitHub, which contains sample applications built using Dozer. This repository can be a good resource for learning how to use Dozer and getting inspiration for your projects. You can also contribute to the dozer-sample repository by submitting a pull request with your projects. This not only helps showcase your work but also provides good examples for other developers to learn from and take inspiration to build their own projects.

You can also join our community discord to see what we are cooking at Dozer :female-cook: :male-cook:

Conclusion

We believe that by making Dozer open source, we are empowering the developer community to build amazing real-time data applications. We look forward to seeing the project and ideas that arise from this collaborative effort, and we're excited to work together to shape the future of real-time data!
Together, we can shape the future of real-time data and contribute to the ongoing success of open source software. 🚀

Happy coding, Happy Data APIng! 🚀👩‍💻👨‍💻

DEV Community: Abhishek Mishra

Jev is Not a Small LLM: A Technical Guide to System One Models

TL;DR

What kind of model is this, actually

Why calibration is a coherent claim here

The API, concretely

The mental model: a sense, not a reasoner

The demos are (mostly) misattributed

Four properties, and what each one buys

Jev vs an LLM vs a fine-tuned classifier

The one-line rule

Five patterns that work

The pattern nobody is using: label everything

Two more worked examples

The interrupt gate

Routing before the expensive model

Where it fails

My numbers, measured from Bengaluru

Where the time actually goes

Does the model actually track the question? A free check first

Measured calibration: the part nobody publishes

Method

Result (n=59, single annotator)

The rule this gives you

What I am not claiming

What the overconfidence actually was

Where I think this lands

Sources & References

Build Voice AI in Python: Complete Speech-to-Text Developer Guide (2026)

TL;DR – Quick Integration Overview

Introduction: Why Voice Integration Matters

Understanding Pulse STT

Highlight Features

Getting Started: Authentication

Step 1: Get Your API Key

Step 2: Test Your Key

Authentication Header

Part 1: Transcribing Audio Files (REST API)

Endpoint

Query Parameters

Supported Languages (32+)

cURL Example

Python Example

Node.js Example

Example Response

Part 2: Real-Time Streaming (WebSocket API)

WebSocket Endpoint

Query Parameters

Python Streaming Example

Node.js Streaming Example

WebSocket Response Format

Key Response Fields

Part 3: Advanced Features

Speaker Diarization

Emotion Detection

PII/PCI Redaction

Age and Gender Detection

Comparing STT Providers

Best Practices

Audio Quality

Error Handling

Rate Limiting

Bonus: Full Demo Application

Quick Start

How It Works

Project Structure

Scripts

Conclusion

Resources

How to do efficient fine-Tuning for LLMs using SLoRA

What is SLoRA?

How SLoRA Works?

Comparison with Traditional Fine-Tuning Methods

Benefits:

How to use SLoRA in Your Projects?

SLoRA's Secret Sauce:

Conclusion

Don't Be That Dev: Avoiding the XY Problem Trap

Understanding Strings in Rust: String vs str

What are String and str?

What are `String` and `str`?

🔹 Step 1: Create a `docker-compose.yml` file