DEV Community: Shivkrishna Shah

Offline-First in React Native: Building an Auto-Sync Engine That Users Never Think About

Shivkrishna Shah — Thu, 27 Aug 2026 06:13:44 +0000

By Shivkrishna Shah · Engineer Philosophy — @shivkrishnashah · @engineerphilosophy

Your app shouldn't have a "no internet" screen. Here's the architecture I use to make mobile apps write locally, sync automatically, and survive the messy reality of field connectivity.

Every mobile developer has shipped this screen at least once: a sad cloud icon and the words "No internet connection. Please try again."

For consumer apps, that's an annoyance. For enterprise field apps — sales reps in hospital basements, auditors in warehouses, technicians in rural areas — it's a dealbreaker. If the app stops working when the signal drops, people stop trusting it. And once field users stop trusting an app, they go back to paper and WhatsApp.

I spent the last few years building and maintaining an offline-first React Native platform used daily by field teams across multiple countries. This post is the architecture I wish someone had handed me on day one: how to structure local storage, detect connectivity, queue writes, auto-sync in the background, and avoid the two bugs that will absolutely bite you (duplicates and conflicts).

Everything here is generic — I'll use Realm DB and NetInfo in the examples, but the pattern maps cleanly onto WatermelonDB, SQLite, or MMKV-backed queues.

The one rule that changes everything

The local database is the source of truth. The server is just a replica you happen to reconcile with.

Most apps are built the other way around: the server is the truth, and the app is a thin cache over fetch(). Offline-first inverts this. Every read comes from the local DB. Every write goes to the local DB first. The network is an implementation detail that a background service worries about — never the UI.

This single inversion gives you three things for free:

Zero-latency UX. Saves are instant because they're local writes. No spinners on submit.
Airplane-mode parity. The app behaves identically online and offline, because the UI never talks to the network.
Crash safety. Data is durable the moment the user taps "Save" — even if the app is killed a second later.

┌──────────────┐   write    ┌───────────────────┐  reconcile  ┌──────────────┐
│  UI/Screens  │ ─────────► │  Local DB (Realm) │ ◄─────────► │  Sync Engine │
│              │ ◄───────── │  SOURCE OF TRUTH  │             │  (background)│
└──────────────┘  live query└───────────────────┘             └──────┬───────┘
                                                                     │ when online
        The UI never touches the network. Ever.               ┌──────▼───────┐
                                              NetInfo ───────►│   REST API   │
                                           (wake / pause)     │   (replica)  │
                                                              └──────────────┘

Step 1 — Give every record a sync passport

Offline-first lives or dies on per-record sync metadata. Every table that can be written on-device carries the same extra fields:

// Every offline-writable schema carries the same sync metadata
const VisitLogSchema = {
  name: 'VisitLog',
  primaryKey: 'localId',
  properties: {
    localId:     'string',   // UUID generated on-device
    serverId:    'int?',     // assigned by the server after first sync
    // ...domain fields (accountId, notes, timestamp, ...)

    syncStatus:  'string',   // 'pending' | 'syncing' | 'synced' | 'failed'
    updatedAt:   'date',     // device clock, for conflict resolution
    syncedAt:    'date?',    // last successful server ack
    retryCount:  'int',      // exponential backoff counter
  },
};

Three deliberate choices here:

Client-generated primary keys (UUIDs). The device must be able to create records — and relate them to each other — without asking the server for an ID. The server ID arrives later and is stored alongside, never as the primary key.
syncStatus is data, not app state. It survives restarts, it's queryable (syncStatus == 'pending' is your sync queue), and you can surface it in the UI as a per-record badge.
retryCount lives on the record. Backoff shouldn't reset because the user relaunched the app.

💡 Design note: You don't need a separate "outbox" table if your DB is queryable — the pending queue is simply a live query over syncStatus IN ('pending','failed') ordered by updatedAt. One source of truth, no queue/table drift.

Step 2 — The write path: local first, always

Every save in the app goes through one door. No screen ever calls the API directly on submit:

export function saveVisitLog(data) {
  const realm = getRealm();
  realm.write(() => {
    realm.create('VisitLog', {
      ...data,
      localId: uuid(),
      syncStatus: 'pending',   // ← queued by definition
      updatedAt: new Date(),
      retryCount: 0,
    });
  });

  requestSync('write');  // nudge the engine — fire and forget
}

Notice what's missing: no await fetch(), no try/catch around a network call, no "are we online?" check. The save is complete the moment the Realm transaction commits. The UI can navigate away immediately and show a PENDING badge on the record.

User taps "Save" ──► Realm write (status: pending) ──┬──► UI updates instantly ✓
                                                     └╌╌► Sync engine nudged ╌╌► Server (eventually)

Step 3 — Connectivity: react to events, verify before flushing

@react-native-community/netinfo gives you connectivity events, but two gotchas matter in production:

isConnected means "has a network interface", not "can reach your API". Captive portals and dead corporate Wi-Fi will lie to you. Use isInternetReachable, and treat even that as a hint.
Connectivity flaps. Walking through a building can fire a dozen transitions per minute — debounce before triggering a flush.

import NetInfo from '@react-native-community/netinfo';

let online = false;

NetInfo.addEventListener(state => {
  const next = !!state.isConnected && state.isInternetReachable !== false;
  if (next && !online) requestSync('network-restored');  // debounced inside
  online = next;
});

⚠️ Hard-won lesson: Never gate the save on connectivity — only gate the flush. The moment you write if (online) api.post() else saveLocally() you have two write paths, and they will drift apart. One door: local write, then sync.

Step 4 — The sync engine: a state machine, not a loop

The engine is a single background service that drains the pending queue in batches. Every record moves through an explicit lifecycle:

             batch picked            2xx + ack
 ┌─────────┐ ──────────► ┌─────────┐ ──────────► ┌──────────┐
 │ PENDING │             │ SYNCING │             │ SYNCED ✓ │
 └─────────┘ ◄╌╌╌╌╌╌╌╌╌╌ └─────────┘ ──────────► ┌──────────┐
       ▲   retry after        error / timeout    │  FAILED  │
       └╌╌ backoff: 2^retryCount × 30s (capped)  └──────────┘
                                                      │ retryCount > N
                                                      ▼ flag for support

async function flush() {
  if (flushing || !online) return;      // single-flight guard
  flushing = true;
  try {
    const batch = realm.objects('VisitLog')
      .filtered('syncStatus == "pending" AND nextRetryAt <= $0', new Date())
      .sorted('updatedAt')
      .slice(0, BATCH_SIZE);            // e.g. 50 records per request

    if (batch.length === 0) return;

    markStatus(batch, 'syncing');
    const res = await api.post('/sync', { records: serialize(batch) });

    // Server acks per record — never all-or-nothing
    realm.write(() => {
      for (const ack of res.acks) {
        const rec = realm.objectForPrimaryKey('VisitLog', ack.localId);
        if (ack.ok) {
          rec.serverId = ack.serverId;
          rec.syncStatus = 'synced';
          rec.syncedAt = new Date();
        } else {
          rec.syncStatus = 'failed';
          rec.retryCount += 1;
          rec.nextRetryAt = backoff(rec.retryCount);  // 2^n × 30s, capped at 1h
        }
      }
    });

    if (moreRemaining()) requestSync('drain');  // keep draining
  } finally {
    flushing = false;
  }
}

The details that matter more than they look:

Single-flight guard. Multiple triggers (timer + network event + fresh write) must never produce two concurrent flushes. One boolean saves you from the nastiest class of duplicate bug.
Batching. Field users come back from a no-signal day with hundreds of pending records. One request per record melts your server and their battery; one giant request times out on 2G. Batch (~50) and drain iteratively.
Per-record acks. If record 37 of 50 fails validation, the other 49 must still succeed. All-or-nothing batches turn one bad record into a permanently stuck queue.

Step 5 — Auto-sync: triggers, not polling

"Auto" sync is just wiring the same requestSync() nudge to every moment connectivity or data might have changed:

Trigger	Source	Why it matters
After every local write	`saveX()` helpers	Online users sync within seconds — feels real-time
Network restored	NetInfo listener (debounced)	The classic "walked out of the basement" moment
App → foreground	AppState listener	OS may have suspended timers while backgrounded
Periodic timer	~every 2–5 min while active	Safety net for missed events; also pulls server changes down
Manual pull-to-refresh	User	Trust: users want a button even if they never need it

All five funnel into one debounced entry point — which is exactly why the single-flight guard in Step 4 exists.

Step 6 — The duplicate problem (this one will get you)

Here's the failure sequence that produces duplicate records in every naive sync implementation:

Device sends batch → server inserts the rows…
…but the response is lost (timeout, tunnel, app killed mid-request).
Device never got the ack → records stay PENDING.
Next flush re-sends them → server inserts them again.

The network being unreliable in both directions is the whole premise of offline-first, so retries are guaranteed. The fix is idempotency, enforced on both sides:

Client: every record carries its device-generated localId (UUID) in the payload — the same one on every retry.
Server: a unique constraint on (device_id, local_id) and upsert semantics: "if I've seen this localId, return the previous ack instead of inserting."

-- Server-side: retries become no-ops, not duplicates
INSERT INTO visit_logs (local_id, device_id, ...)
VALUES (?, ?, ...)
ON DUPLICATE KEY UPDATE id = LAST_INSERT_ID(id);  -- return existing row's id

⚠️ Also on the client: If you have two possible senders — say, a foreground "submit now" path and a background flush service — they can race and double-send the same rows before either ack lands. Either collapse them into one sender, or add a mutex so only one path can flush a given record type at a time. We learned this from a production duplicate-insert bug that only reproduced on slow networks.

Step 7 — Conflicts: pick a policy before you need one

Downstream sync (server → device) eventually meets a record edited in both places. There is no universally correct answer — there is only a policy chosen per table, on purpose:

Strategy	Rule	Use for
Last-write-wins	Higher `updatedAt` wins	Single-owner data (a rep's own notes) — simple, predictable
Server-wins	Server copy always replaces local	Reference/master data the device merely displays
Client-wins	Device copy survives until explicitly synced	In-progress work the user is actively editing
Field-level merge	Compare per column, merge non-overlapping edits	High-value shared records — costs real complexity
Manual resolution	Park both versions, ask a human	Rare, high-stakes conflicts (approvals, financial data)

In a field-team context, last-write-wins with per-record ownership covers ~90% of cases, because most offline-written records have exactly one author — the device that created them. Design your data model so this stays true and you may never need the expensive strategies.

💡 Clock warning: LWW compares timestamps, and device clocks lie. Record the device's updatedAt but let the server stamp arrival time and sanity-check drift (e.g., reject client timestamps from the future). Never resolve conflicts with unvetted device time alone.

What I'd tell you before you build it

Show sync state honestly. A tiny PENDING / SYNCED badge per record, plus a "3 records waiting to sync" strip, converts anxiety into trust. Users forgive a delayed sync; they never forgive silent data loss.
Sync your error logs too. When a device misbehaves offline, the evidence is trapped on it. Treat error logs as just another offline-writable table that flushes with everything else — production debugging becomes possible again.
Test with the network chaos monkey. Airplane mode mid-flush, killed app mid-request, captive-portal Wi-Fi, 48 hours offline then 400 pending records. Every one of these is a Tuesday for a field user.
Never let a stuck record block the queue. Cap retries, quarantine poison records, flag them for support — and keep draining the rest.
Design the server API for sync from day one. Batched upserts, per-record acks, idempotency keys. Retrofitting idempotency onto a plain CRUD API is far more painful than building it in.

The payoff

The architecture above is maybe 500 lines of engine code plus a schema convention — small enough to own completely, with no black-box sync SaaS in the middle. What it buys you:

Saves that complete in milliseconds, every time, regardless of signal.
A field workforce that can work a full offline day and reconcile automatically over lunch Wi-Fi.
Zero duplicate submissions — idempotency by construction, not by luck.
One write path, one queue, one state machine: an architecture a new team member can hold in their head.

Offline-first isn't a feature you add. It's a decision about where the truth lives — and everything else follows from it.

If you're building something similar — or wrestling with sync bugs in production — I'm happy to compare notes. Drop a comment or reach out.

#ReactNative #OfflineFirst #MobileArchitecture #RealmDB #SyncEngine #MobileDevelopment

Shipping Whisper-Powered Voice Input in a Mobile App: What Actually Works

Shivkrishna Shah — Thu, 27 Aug 2026 06:03:19 +0000

Voice input demos beautifully and ships painfully.

At Engineer Philosophy, my firm, we recently built a voice-first data entry flow for a field-facing mobile app: a user taps a mic, talks through a long structured form naturally, and the app transcribes their speech, interprets it, and fills the form — with a confirmation step before anything is saved. Under the hood it runs on OpenAI's Whisper family of speech-to-text models.

Getting a transcript back from an API took an afternoon. Getting voice input that field users trust took weeks — and almost none of that time was spent on the API call itself. Here's what actually moved the needle, in three parts: architecture, voice detection and recording, and turning transcripts into responses you can use.

![The Voice → Form Pipeline]

Part 1: Using Whisper in an application — two paths, not one

Most tutorials show one flow: record an audio file, POST it to /v1/audio/transcriptions, read the text. That works, but a production app benefits from two transcription paths running as a pair:

1. Batch (record → upload). The recorder captures compressed audio (AAC), and on stop the file goes to the transcriptions endpoint. Models like whisper-1 or the newer gpt-4o-mini-transcribe (~$0.003/min) work here. It's simple, robust, and handles long takes.

2. Realtime (streaming). For live "words appear as you speak" feedback, we stream raw PCM (16-bit, 24 kHz, mono) chunks from a native microphone tap over a WebSocket to the Realtime API and render transcription deltas as they arrive. Seeing your words appear live builds enormous trust — users stop wondering whether the app heard them.

The crucial design decision: realtime is an enhancement, batch is the contract. If the WebSocket fails to open, or the native audio streamer is unavailable, the app silently falls back to record-then-upload. The user never sees a difference except the live transcript. Every layer of the feature follows this "fail open to the simpler path" rule — including an env-var kill switch that can restore the old behavior without touching a line of code.

Two gotchas worth knowing up front:

Models are endpoint-specific. The realtime transcription model only exists on the WebSocket API — sending it to the file endpoint 404s. Verify each model against the endpoint you're actually calling.
The file endpoint has a 25 MB hard limit. Check the size client-side and fail fast rather than uploading for minutes just to receive a 413. At ~128 kbps AAC that's roughly 26 minutes of audio — which directly informed our recording cap (more below).

Part 2: Improving voice detection and recording

This is where the real engineering lived. Three lessons stand out.

Measure loudness in dBFS, and use ONE scale everywhere

Our first silence detector compared the mic level against a threshold of 0.08. The problem: one capture path reported a logarithmic dBFS-derived level and the other reported linear amplitude. The same threshold meant "−46 dBFS" on one path and "amplitude 0.016" on the other — so the same speech produced different meter readings and different auto-stop behavior depending on which path happened to be active.

The fix: both paths now convert amplitude → dBFS → the same normalization, and every threshold in the system is expressed in dBFS. Decibels are how audio actually behaves; a linear 0–1 "level" is a lie that will eventually bite you.

Detect silence relative to the speaker, not against a fixed floor

The naive auto-stop rule — "stop when the level drops below X for N seconds" — fails in the field, and the reason is subtle: automatic gain control. In a noisy clinic or a car, the phone's AGC lifts the noise floor between utterances, so a fixed silence threshold simply never triggers.

Our detector uses layered rules instead:

A speech gate (−35 dBFS): above this, a sample is definitely speech. Room tone sits below it; a person talking does not.
Relative silence: quiet is judged 18 dB below the speaker's own loudest speech, not against an absolute floor. Speech runs 20–30 dB above ambient noise in almost any room, so this survives noisy environments where a fixed threshold is useless.
2.5 seconds of continuous silence to finish — long enough to survive a mid-sentence breath or a pause to think. Stopping too eagerly is worse than stopping late, because truncated words are simply lost.
A minimum of 800 ms of heard speech before silence can end the recording — otherwise the mic closes during the gap between tapping the button and starting to talk.

One rule we added and removed the same day: a live "you're too quiet" warning based on the average of the last ~20 samples. The average includes the silence between words, so ordinary speech averaged below the threshold and the warning fired at people who were being heard perfectly. Anything you surface live must use a peak or percentile, never a mean.

Keep audio levels out of your UI framework's state

We shipped a regression where users reported "mic performance reduced." The cause: the input level was stored as React state, so every metering sample — 4 to 10 per second — re-rendered the entire recording screen, on the same JS thread that was streaming audio to the transcription socket. A decorative gradient ring being reconciled at metering rate made it worse.

The fix: the level became an animated value that bypasses the framework's render cycle entirely (in React Native terms, an Animated.Value eased on the native driver). Samples never reach React; the UI interpolates. We deliberately removed any numeric level from the API so nobody could reintroduce the problem.

The same discipline applies to the auto-stop countdown: instead of one state write per sample, it runs as two animations per quiet episode. If your meter UI costs more than your audio pipeline, you've built it wrong.

Pause/Resume: three details that matter

Actually stop the mic natively — don't fake it in the UI. Showing "Paused" over a live microphone is a trust violation worth extra code to avoid, so if the native pause call fails, roll back and keep showing "Recording."
Close the silence detector while paused — otherwise the pause itself triggers auto-stop 2.5 seconds in.
Reset the detector on resume — so the pause's own quiet doesn't count toward the silence budget.

Part 3: Transcribing and getting the response you need

Pass a language hint — it's the cheapest accuracy win available

Both Whisper paths accept a language parameter, and for months we weren't passing it — every recording ran on auto-detect even though the app already knew which of its 13 supported languages the user had chosen. Auto-detect is a real accuracy cost: it's the usual cause of a transcript coming back in the wrong script, and of code-switched speech (Hindi-English mixes like "do accounts pending hain") being classified as the wrong language outright.

The fix is one function: map the app's language setting to a base ISO-639-1 code (dropping script tags — zh-Hant becomes zh) and pass it on every request. Unknown language → omit the parameter and let the model auto-detect, exactly as before. Tiny change, outsized effect on multilingual accuracy.

Handle the WebSocket closing cleanly — not just erroring

The nastiest bug we shipped: mid-recording, the live transcript silently stopped growing while the level meter kept moving. Users kept talking; every word after a certain point was lost.

The cause: our streaming client handled onerror but its onclose handler did nothing once the session was established. Sockets can close cleanly — server idle timeouts, session-duration limits, a network blip with no error event — and our feed loop kept pumping audio into a dead socket whose send-failures were deliberately swallowed.

Three fixes: onclose marks the session closed and reports it; the feeder stops pumping a closed socket; and the UI surfaces "live transcription stopped" while recording — not only after the take ends. And note the interaction with another change: when we raised the recording cap from 2 to 15 minutes, long takes started hitting session limits that short takes never reached. Every limit you relax exposes failure modes you've never seen.

The transcript is not the answer — interpret it

For our use case, the transcript is an intermediate artifact. The raw text goes to an LLM agent that maps it onto the form's actual questions — single choice, multi-select, numeric, percentage groups, rankings — producing structured draft answers. Two principles keep this safe:

Confirm before writing. The user reviews a summary of interpreted answers; nothing touches the database until they approve. "Start over" genuinely discards.
Show progress honestly. Interpretation takes seconds; a phased progress view (route → read → act → reply) replaced a spinner and dramatically reduced abandoned sessions.

Don't ship your API key in the app

One security note that belongs in every article like this: an API key baked into a mobile binary is extractable by anyone with the app file — no jailbreak required — and certificate pinning doesn't save you. The only real fix is the key not being on the device: route through your backend proxy, and for the realtime WebSocket use server-minted ephemeral tokens. For a POC, at minimum use a dedicated key with a hard spend limit.

The meta-lesson: test on real hardware

Simulator microphones never produced audio loud enough to cross our −35 dBFS speech gate — so the entire "speech was heard" branch of the system was unverifiable there. The meter zones, the silence countdown, the interpret flow: all of it needed a physical device. If your feature has a threshold, your test environment must be able to cross it.

Voice input isn't a transcription API call. It's a signal-processing problem (dBFS, AGC, relative thresholds), a performance problem (keep samples off the render path), a distributed-systems problem (sockets close cleanly), and a UX problem (live feedback, honest failure states, confirm-before-write). Whisper solves exactly one layer of that stack — brilliantly — and the rest is where your product is actually built.

That layered, measure-first way of working is the whole reason my firm is called Engineer Philosophy: every threshold in this feature earned its number from real field behavior, every new capability fails open to a simpler path, and nothing was declared "done" from a simulator. That's the philosophy; the engineering follows from it.

Building voice features into your app? I'd love to hear what tripped you up — the failure modes above can't be the whole list. And if you want help shipping one, that's exactly what we do at Engineer Philosophy.

#AI #Whisper #SpeechToText #MobileDevelopment #ReactNative #VoiceUI #OpenAI #ProductEngineering

Tokens, Context, Parameters: The Real Mechanics of an LLM

Shivkrishna Shah — Wed, 29 Jul 2026 07:19:52 +0000

Tokens, Context, and Parameters: How Large Language Models Actually Work

Large language models can answer questions, generate software, summarize documents, and produce human-like writing. These abilities emerge from three closely connected elements:

Tokens represent the information entering and leaving the model.
Context contains the information available during the current prediction.
Parameters encode the statistical patterns learned during training.

Together, these components allow an LLM to predict one token at a time.

1. Tokens: The Model’s Units of Language

A language model does not process text directly as sentences or complete words. It first uses a tokenizer to divide text into smaller units called tokens.

A token can represent:

A complete word
Part of a word
Punctuation
A number
Whitespace
A special control symbol

For example:

The model is learning.

might become:

The · model · is · learn · ing · .

Breaking uncommon words into smaller pieces allows the model to represent a very large vocabulary without storing every possible word.

From text to token IDs

Every token is assigned an integer called a token ID:

"The"     → 464
" model"  → 2746
" is"     → 318
" learn"  → 2193
"ing"     → 278
"."        → 13

The exact IDs depend on the tokenizer used by the model.

Token IDs are only labels. Before they can be processed mathematically, the model converts them into vectors.

Token embeddings

The model contains a learned embedding matrix:

Vocabulary size × embedding dimension

If the vocabulary contains 50,000 tokens and the model’s internal dimension is 4,096, the embedding matrix has the shape:

50,000 × 4,096

Each row is a learned numerical representation of one token. Looking up a token ID selects the corresponding row from the matrix.

A token therefore becomes a vector such as:

[0.14, -0.82, 0.37, ..., 0.09]

The individual values usually do not have simple human-readable meanings. Meaning is distributed across many dimensions.

Position information

Transformers process multiple tokens in parallel, so they also need information about token order. The model adds or applies a positional representation to each token embedding.

Conceptually:

Input representation = token embedding + position information

This helps the model distinguish between sentences such as:

The dog chased the cat.

and:

The cat chased the dog.

They contain similar tokens, but their order changes their meaning.

2. Context: The Model’s Temporary Working Information

The context is the complete sequence of tokens available to the model during a prediction.

It can include:

System instructions
The user’s prompt
Previous conversation messages
Retrieved information
Uploaded document content
Tool results
Tokens already generated in the response

A model’s context window defines the maximum number of tokens it can process at one time.

Conceptually:

Instructions + conversation + documents + generated output ≤ context window

If the combined content exceeds the limit, some information must be removed, compressed, summarized, or handled through another retrieval mechanism.

Context is not permanent memory

Context acts more like temporary working memory than permanent storage. Information inside the active window can influence the next prediction. Information outside it normally cannot unless it is supplied again.

This creates an important distinction:

Parameters = patterns learned during training
Context    = information supplied for the current task

The context changes from one conversation to another. The model’s parameters normally remain fixed during ordinary inference.

Causal attention

Most text-generating LLMs use causal attention. When predicting a token, the model may attend to previous tokens but not future tokens.

For a sequence:

The capital of Japan is

the representation at the final position can use information from all preceding positions. It cannot use the answer token because that token has not been generated yet.

A causal mask enforces this direction:

Token 1 can attend to: 1
Token 2 can attend to: 1, 2
Token 3 can attend to: 1, 2, 3
Token 4 can attend to: 1, 2, 3, 4

This is what makes autoregressive generation possible.

3. Parameters: The Model’s Learned Numerical Structure

Parameters are the trainable numerical values inside the neural network. They include large matrices and smaller vectors used throughout the transformer.

The symbol θ, pronounced “theta,” is often used to represent all model parameters collectively.

A model with billions of parameters contains billions of learned floating-point values. These parameters do not form a searchable database of sentences. Instead, information is distributed across many interacting weights.

Where the parameters are located

A transformer language model typically contains parameters in several major components.

Embedding matrix

The embedding matrix converts token IDs into vectors:

token ID → embedding vector

These embeddings are learned during training.

Attention projections

Inside each self-attention layer, the input representation is transformed into queries, keys, and values:

Q = XWQ
K = XWK
V = XWV

Where:

X is the current sequence representation.
WQ is the query projection matrix.
WK is the key projection matrix.
WV is the value projection matrix.

Attention scores are calculated using:

Attention(Q, K, V) = softmax(QKᵀ / √dk)V

The result is commonly passed through another learned matrix, WO.

These matrices let the model determine which previous positions are relevant to the current token.

Multi-head attention

Transformers usually divide attention into multiple attention heads. Each head has separate projection weights and can learn different relationships.

Different heads may become sensitive to patterns involving:

Nearby words
Long-distance dependencies
Grammatical structure
References between names and pronouns
Formatting or positional relationships

The heads are combined and projected back into the model’s main representation space.

Feed-forward network

Each transformer block also contains a feed-forward network, often called an MLP.

A simplified version is:

MLP(X) = activation(XW1 + b1)W2 + b2

The first projection usually expands the representation into a larger intermediate dimension. The second projects it back to the model dimension.

Modern models may use gated structures such as SwiGLU, but the purpose is similar: transform the representation at each token position using learned nonlinear features.

Layer normalization

Layer-normalization components help keep internal activations numerically stable. They may contain learned scaling and shifting parameters.

Output projection

After the final transformer layer, the model converts the hidden representation into one score, or logit, for every possible token:

hidden state → vocabulary logits

A softmax function converts these logits into a probability distribution:

P(next token | previous tokens; θ)

The model can then select or sample the next token from this distribution.

4. Inside a Transformer Block

An LLM contains many transformer blocks stacked on top of one another. Each block gradually refines the representation of every token.

A simplified block contains:

Layer normalization
Multi-head self-attention
A residual connection
Another normalization stage
A feed-forward network
Another residual connection

The general flow is:

Input representations
        ↓
Layer normalization
        ↓
Self-attention
        ↓
Residual addition
        ↓
Layer normalization
        ↓
Feed-forward network
        ↓
Residual addition
        ↓
Output representations

Residual connections

Residual connections add a block’s input to its output:

output = input + transformation(input)

They help information and gradients travel through deep networks. Without them, training models with many layers would be considerably more difficult.

What changes through the layers?

Early layers may identify local or surface-level patterns. Deeper layers can combine information across positions and represent more abstract relationships.

The model’s internal representation of a token therefore depends on both:

The token’s original embedding
The surrounding tokens in the current context

The word “bank,” for example, may develop different contextual representations in:

She deposited money at the bank.

and:

They sat on the river bank.

5. How an LLM Is Trained

Before an LLM can generate useful text, its parameters must be learned from data.

The basic training objective is next-token prediction.

Step 1: Prepare the training data

Training text is cleaned, filtered, and converted into token sequences.

Suppose a training sequence is:

Large language models predict tokens

The model receives shifted input-and-target pairs:

Input:  Large
Target: language

Input:  Large language
Target: models

Input:  Large language models
Target: predict

In practice, many positions are trained simultaneously.

Step 2: Run a forward pass

The token sequence passes through the embedding layer and transformer blocks.

For every position, the model produces a probability distribution over the vocabulary.

For example:

"tokens"      0.61
"text"        0.14
"words"       0.09
"outputs"     0.04
other tokens  0.12

Step 3: Measure the error

The predicted distribution is compared with the actual next token using cross-entropy loss.

For the correct token y, the loss can be written as:

L = −log P(y | previous tokens; θ)

If the model gives the correct token a high probability, the loss is low. If it assigns the correct token a low probability, the loss is high.

Step 4: Backpropagate the error

The training system uses backpropagation to calculate how much each parameter contributed to the error.

These derivatives form the gradient:

∇θL

The gradient indicates which direction would increase the loss. Training moves the parameters in the opposite direction.

Step 5: Update the parameters

An optimizer updates the weights:

θ ← θ − η∇θL

Where:

θ represents the model’s parameters.
L represents the loss.
∇θL represents the loss gradient.
η represents the learning rate.

Real training usually employs optimizers such as Adam or AdamW rather than basic gradient descent, but the underlying idea is the same.

Step 6: Repeat across many batches

This process is repeated across enormous numbers of token sequences:

Forward pass
    ↓
Calculate loss
    ↓
Backpropagation
    ↓
Parameter update
    ↓
Next batch

Over time, the model becomes better at predicting tokens across many types of text.

6. Pretraining, Fine-Tuning, and Alignment

LLM development commonly involves several training stages.

Pretraining

During pretraining, the model learns general language patterns from a large and diverse dataset through next-token prediction.

It may learn patterns involving:

Grammar and syntax
Semantic relationships
Writing styles
Factual associations
Programming languages
Common reasoning structures

Pretraining creates the model’s broad foundational capabilities.

Supervised fine-tuning

The pretrained model may then be trained on curated examples containing prompts and high-quality responses.

This teaches it to follow instructions and produce answers in more useful formats.

Preference optimization

Human or AI-generated preference data can be used to improve helpfulness, safety, and response quality.

Possible approaches include:

Reinforcement learning from human feedback
Direct Preference Optimization
Related preference-learning techniques

These stages still modify parameters, but they use more targeted data and objectives than general pretraining.

7. How Generation Works After Training

During normal use, the model performs inference. Its learned parameters are usually fixed.

Suppose the prompt is:

The capital of Japan is

The generation process is:

The tokenizer converts the text into token IDs.
The embedding layer converts the IDs into vectors.
Position information is incorporated.
Transformer layers process the context.
The output layer produces vocabulary logits.
Softmax converts the logits into probabilities.
A decoding strategy chooses the next token.
The chosen token is added to the context.
The process repeats.

The model might produce:

Tokyo

After generating Tokyo, it predicts the token that should follow it.

Decoding strategies

The model does not always select the single highest-probability token. Generation can be controlled using methods such as:

Greedy decoding: choose the most probable token.
Temperature: adjust how sharp or varied the probability distribution is.
Top-k sampling: sample only from the k most likely tokens.
Top-p sampling: sample from the smallest group whose combined probability reaches a threshold.

These settings affect how the model selects output tokens. They do not retrain its parameters.

8. Parameters Versus Context

Parameters and context influence the same prediction in different ways.

Feature	Parameters	Context
Purpose	Store learned statistical patterns	Supply information for the current prediction
Origin	Learned during training	Provided at inference time
Duration	Persists across prompts	Temporary and task-specific
Typical scale	Millions or billions of values	Thousands or millions of tokens
Changed by normal prompting?	No	Yes
Example	Learned language structure	The current conversation

This distinction explains why a model can adapt to instructions without changing its weights. The instructions alter the context, which changes the model’s internal activations and predictions.

9. What Parameter Count Does—and Does Not—Mean

A parameter count measures the model’s numerical capacity. A larger model can potentially represent more complex patterns, but size alone does not determine performance.

Quality also depends on:

Training-data quality and diversity
Model architecture
Tokenizer design
Training duration
Optimization methods
Context handling
Fine-tuning and alignment
Inference techniques

A smaller model trained efficiently on appropriate data can outperform a larger model on a specialized task.

Parameter count should therefore be treated as one engineering characteristic—not a complete measure of intelligence or quality.

10. Common Misconceptions

“One token equals one word”

Not necessarily. A word may be one token, several tokens, or part of a larger token.

“The context window is permanent memory”

It is temporary information available during the current inference process.

“Parameters contain copies of documents”

Parameters encode distributed numerical patterns. They are not equivalent to files or database records, although models can sometimes reproduce memorized material.

“The model searches its training data for every answer”

During standard inference, the model calculates predictions using its parameters and current context. External search or retrieval occurs only when a separate tool or retrieval system is connected.

“The model learns from every conversation immediately”

Ordinary conversations usually do not update the model’s parameters. Updating parameters requires a separate training process.

“More parameters always produce a better model”

Model size is only one factor. Architecture, data, training, and task fit are equally important.

The Complete Picture

The complete LLM pipeline can be summarized as:

Text
  ↓
Tokenizer
  ↓
Token IDs
  ↓
Token embeddings + position information
  ↓
Transformer layers
  ├─ Self-attention
  ├─ Feed-forward networks
  ├─ Normalization
  └─ Residual connections
  ↓
Vocabulary logits
  ↓
Probability distribution
  ↓
Next token

During training, prediction errors are used to update the parameters:

Training data
  ↓
Next-token prediction
  ↓
Cross-entropy loss
  ↓
Backpropagation
  ↓
Optimizer update
  ↓
Improved parameters

During inference, those learned parameters are reused without normally being changed:

Current tokens + context + learned parameters
                         ↓
              next-token probability

The central idea is simple:

Tokens carry the information, context determines what the model can currently consider, and parameters define the learned transformations used to predict what comes next.

At Engineer Philosophy, our objective is not simply to connect an interface to an LLM. We design the complete information flow around the model—controlling the tokens it receives, constructing useful context, selecting the appropriate model, and validating its output for the intended application.

RAG Explained Simply: How AI Finds the Right Information Before Answering

Shivkrishna Shah — Sat, 18 Jul 2026 21:43:34 +0000

Most AI demos look impressive until you ask a question about your own company.

Ask an ordinary language model:

“Which customers recently complained about product pricing?”

The model may understand the question perfectly. But unless your customer feedback was included in its training data or supplied in the prompt, it does not know the answer.

It may refuse.

It may give a generic response.

Worse, it may produce an answer that sounds convincing but is not supported by your data.

This is the problem Retrieval-Augmented Generation—better known as RAG—is designed to solve.

The simple idea behind RAG

RAG gives an AI model access to relevant external information before asking it to answer.

Instead of using this flow:

Question
   ↓
LLM
   ↓
Answer

RAG uses:

Question
   ↓
Retrieve relevant information
   ↓
Give that information to the LLM
   ↓
Generate a grounded answer

The original RAG research combined a language model’s learned knowledge with external, non-parametric memory retrieved at runtime. This made it possible to update accessible knowledge without retraining the entire language model. citeturn146662search0

The important word here is runtime.

RAG does not automatically train the model on your database. It finds useful information when the user asks a question and temporarily places that information inside the model’s context window.

A practical example

Imagine that your application stores these survey responses:

Customer 101:
The product works well, but its price is too high.

Customer 205:
Availability is inconsistent in smaller cities.

Customer 318:
The product is affordable and easy to access.

The user asks:

“Which customers are concerned about affordability?”

A normal SQL keyword search may look for the exact word affordability.

That could miss:

“Its price is too high.”

A semantic search system can recognize that:

affordability
high price
expensive
cost concern

are related concepts.

That is where embeddings and vector databases enter the system.

How the complete RAG pipeline works

Step 1: Prepare your source data

Your source data may come from:

MySQL or PostgreSQL
Realm
PDFs
Word documents
CRM records
support tickets
survey answers
product documentation
internal policies

Structured database rows are usually converted into consistent text.

For example:

Account: ABC Pharmacy.
City: Indore.
Specialty: Retail pharmacy.
Survey response: Product pricing is too high for many patients.

You usually do not need an LLM to create this sentence. A deterministic template is cheaper, faster and more consistent.

Step 2: Create an embedding

An embedding model converts the text into a vector:

Text
  ↓
Embedding model
  ↓
[0.021, -0.184, 0.537, ...]

The vector is a numerical representation of meaning.

It is not encrypted text, and it cannot normally be decoded back into the original sentence.

Similar meanings produce nearby vectors.

For example:

Cardiologist
Heart specialist
Cardiac physician

should be positioned closer together than:

Cardiologist
Dermatologist

Step 3: Store the searchable representation

A vector database can store:

{
  "id": "FEEDBACK_184",
  "document": "The product is effective, but its price is too high.",
  "metadata": {
    "survey_id": "SURVEY_10",
    "account_id": "ACC_101",
    "entity_type": "survey_feedback"
  },
  "embedding": [0.021, -0.184, 0.537]
}

The important parts are:

ID: connects the vector result to the source record
document: readable text that may be returned during retrieval
metadata: exact fields used for filtering
embedding: used for similarity search

MySQL or Realm should normally remain the source of truth for changing business data. The vector database should act as a semantic index.

Step 4: Embed the user’s question

The user asks:

“Which customers are concerned about affordability?”

The same embedding model converts this question into a vector.

Using the same embedding model is important because the stored documents and the query must exist in the same vector space.

Step 5: Retrieve the nearest matches

The vector database searches for records whose embeddings are closest to the query embedding.

It may return:

1. FEEDBACK_184 — pricing is too high
2. FEEDBACK_422 — customers cannot afford the product
3. FEEDBACK_091 — treatment cost is a concern

This is not ordinary keyword matching. It is meaning-based retrieval.

Step 6: Fetch current records when necessary

For documentation and relatively static content, the document returned by the vector database may be enough.

For live enterprise data, use the returned IDs to fetch current records from Realm or MySQL:

SELECT
    id,
    account_id,
    response,
    updated_at
FROM survey_answers
WHERE id IN (184, 422, 91);

This prevents an old vector index from becoming the final authority.

Step 7: Build the model context

The application constructs a controlled prompt:

System instruction:
Answer only from the supplied evidence.
Do not invent customers or conclusions.

User question:
Which customers are concerned about affordability?

Retrieved evidence:
1. Account 101 said the price is too high.
2. Account 205 said many patients cannot afford the product.
3. Account 318 said treatment cost is a concern.

Step 8: Generate a grounded answer

The language model can now respond:

Three customers expressed affordability concerns. Their feedback focuses on high product prices, limited patient affordability and overall treatment cost.

The model did not independently know this.

The system retrieved the evidence and placed it in the model’s temporary context.

RAG is not only a vector database

A common mistake is to think:

RAG = embeddings + vector database.

That is incomplete.

A reliable RAG system also needs:

document preparation
chunking
metadata design
access control
query understanding
retrieval
filtering
reranking
context construction
answer generation
citations
evaluation
monitoring

The language model is only one component.

SQL search and vector search solve different problems

Use SQL for exact and mathematical questions:

How many surveys are pending?
What is the average call frequency?
Which city has the highest sales?

Use SQL or deterministic tools for those.

Use vector search for semantic questions:

What complaints are related to product affordability?
Find doctors similar to this profile.
Which responses discuss supply problems?

The strongest enterprise architecture combines both:

User request
    ↓
Request router
    ├── Exact analytics → SQL / Realm
    ├── Semantic retrieval → Vector database
    ├── Business action → Trusted application tool
    └── Explanation → LLM

RAG does not eliminate hallucination

RAG can improve factuality, but it does not guarantee truth.

A RAG system can still fail when:

the wrong documents are retrieved
important records are missing
chunks lose necessary context
stale data remains indexed
access filters are incorrect
the prompt allows unsupported conclusions
the model misinterprets good evidence

Broader RAG research describes retrieval as a way to improve the accuracy and credibility of knowledge-intensive generation while helping address outdated knowledge and weak provenance. It is still an engineering system that must be evaluated rather than blindly trusted. citeturn146662search8

What is the future of RAG?

RAG is moving beyond the simple “retrieve once, then answer” pipeline.

1. Hybrid retrieval

Future-ready systems combine:

vector similarity
keyword search
SQL filters
metadata filters
graph relationships

Vector search finds meaning. Keyword search preserves exact names and terminology. Metadata applies strict business boundaries.

2. Reranking

The first search may retrieve 20 candidates.

A reranking model then evaluates those candidates more carefully and selects the strongest evidence for the final prompt.

This improves retrieval quality without filling the context window with weak results.

3. Multimodal RAG

Enterprise knowledge is not limited to text.

RAG systems increasingly retrieve from:

images
screenshots
charts
audio
video
scanned forms
maps and spatial records

A user may eventually ask:

“Compare the issue shown in this site photograph with last month’s inspection report.”

The system will need to retrieve across multiple data types.

4. Graph-based retrieval

Some questions depend on relationships rather than isolated documents:

Customer
  ↓ belongs to
Territory
  ↓ managed by
Representative
  ↓ assigned to
Campaign

Knowledge graphs and graph-aware retrieval can help answer multi-hop questions where information is distributed across related entities.

5. Real-time and event-driven RAG

Instead of rebuilding the entire vector index, production systems can update affected records when business data changes.

MySQL record updated
       ↓
Event published
       ↓
Text regenerated
       ↓
Embedding updated
       ↓
Vector record replaced

This is especially important for CRM, operational and mobile applications.

6. Agentic RAG

Traditional RAG follows a fixed pipeline.

Agentic RAG allows an agent to decide:

whether retrieval is needed
which data source to search
how to rewrite the query
whether evidence is sufficient
whether another search is required
which tool should calculate the answer
whether the response needs human approval

Recent Agentic RAG literature describes systems using planning, reflection, tool use and multi-agent collaboration to adapt retrieval for complex, multi-step work. citeturn146662search5turn146662search10

A future workflow may look like:

User goal
   ↓
Agent creates a plan
   ↓
Search documentation
   ↓
Query SQL
   ↓
Check Realm data
   ↓
Compare evidence
   ↓
Retry weak retrieval
   ↓
Generate answer
   ↓
Request confirmation
   ↓
Execute an approved action

Research published in 2025 and 2026 is actively exploring multi-agent retrieval, hierarchical retrieval interfaces and more efficient agentic search. These approaches are promising, but they also introduce more latency, token consumption and operational complexity. citeturn146662search33turn146662search14turn146662academia56

A practical mobile architecture

For an offline-first React Native application using Realm:

React Native application
        ↓
Chat interface and voice input
        ↓
Chatbot orchestrator
        ├── Realm query tools
        ├── Validation action tools
        ├── What-if simulation tools
        └── Server RAG API
                ├── Embedding model
                ├── Vector database
                ├── MySQL
                └── LLM

Realm stores current offline application data.

MySQL remains the server-side source of truth.

The vector database provides semantic retrieval.

The LLM interprets questions and explains verified data.

Trusted application code performs business actions.

The key lesson

RAG is not a magical database that makes an LLM intelligent.

It is a controlled information pipeline.

The embedding model represents meaning.

The vector database retrieves related information.

MySQL and Realm preserve current business facts.

The LLM turns evidence into language.

Application tools enforce business rules.

The future is not an AI model replacing every database.

The future is an AI system that knows:

which database to use
what information to retrieve
how to verify it
when to ask for approval
how to act safely

That is where RAG becomes more than a chatbot feature.

It becomes the knowledge layer behind an enterprise AI assistant.

At Engineer Philosophy Web Services Pvt. Ltd., we are exploring how RAG, Agentic AI, mobile applications, offline databases, and enterprise systems can work together to create AI assistants that do more than answer questions.

engineerphilosophy.com

AgenticAI #ArtificialIntelligence #AIAgents #GenerativeAI #EnterpriseAI #Automation #Innovation #DigitalTransformation #EngineerPhilosophy #MAPOG #shivkrishnashah

[Boost]

Shivkrishna Shah — Thu, 09 Jul 2026 11:50:09 +0000

Shivkrishna Shah

Jul 9

Building an Enterprise AI Assistant That Can Actually Take Action

#ai #programming #mobile #reactnative

4 min read

Building an Enterprise AI Assistant That Can Actually Take Action

Shivkrishna Shah — Thu, 09 Jul 2026 06:30:35 +0000

Imagine you're a field representative visiting customers all day.

It's 6:30 PM.

You still have 40 survey responses to complete before heading home.

Instead of navigating through multiple screens, wouldn't it be easier if you could simply say:

"Mark Question 3 as Yes for Northside Clinic and tell me what's left before I submit."

Most AI assistants can't do this.

They can answer questions.

They can summarize information.

They can explain how to complete a task.

But they can't actually perform the task.

That's the difference between a chatbot and an Agentic AI system.

What Is Agentic AI?

In the context of mobile applications, Agentic AI means giving an LLM the ability to reason, plan, and safely perform actions inside your application.

Instead of telling users where to click, the assistant actually performs the required operations using your application's tools.

An agent can:

Search application data
Read local state
Validate information
Update records
Execute business workflows
Ask follow-up questions when information is missing

Instead of becoming another search box, AI becomes another user of your application.

Why Most Mobile AI Projects Fail

After building and experimenting with production-grade AI assistants, I noticed two common mistakes.

Mistake #1 — AI Wrapped Around If-Else Statements

Many applications still rely on intent matching.

if(message.includes("survey")){
   openSurvey();
}

An LLM simply generates the response.

The routing is still deterministic.

This works during demos.

It usually breaks in production.

Users never ask questions exactly the way developers expect.

Mistake #2 — Giving the AI Unlimited Access

The opposite mistake is allowing the model to directly modify application data.

Without proper validation, an AI can:

Update the wrong record
Modify unintended data
Hallucinate IDs
Save information the user never approved

Neither approach belongs in a production application.

The Architecture That Worked

After several iterations, I settled on one simple principle.

The model makes decisions. The application guarantees safety.

The AI should handle:

Reasoning
Planning
Selecting tools
Recovering from failures
Asking follow-up questions

Your application should handle:

Validation
Permissions
Confirmations
Database writes
Privacy
Security
Execution limits

Once those responsibilities are separated, the architecture becomes both flexible and reliable.

The Agent Loop

Everything begins with one simple loop.

Receive the user's message
Send conversation context to the LLM
Receive requested tool calls
Execute tools locally
Return tool results
Continue until the task is complete

That's it.

No keyword routing.

No intent classifier.

No hardcoded workflows.

The model decides what happens next.

(Insert simplified agent loop diagram here.)

Skills Instead of Massive Prompts

One of the biggest improvements was replacing one enormous system prompt with small reusable skills.

Each skill contains:

Instructions
Available tools
Validation logic
Optional hooks

Example:

Survey Skill

Available Tools

• Find Survey
• Update Answer
• Save Draft
• Submit Survey

Adding a new capability became as simple as adding another folder.

The engine never changes.

Confirmation Before Every Write

One lesson became obvious very quickly.

Never allow an AI model to write directly into your database.

Every write follows the same process.

Stage Changes
      ↓
Show Confirmation
      ↓
 User Approves
      ↓
   Validate
      ↓
     Save

Even if the AI attempts to skip confirmation, the application blocks it.

The runtime—not the model—controls safety.

Let the Model Recover

One surprising discovery was that AI becomes significantly smarter when you return errors instead of hiding them.

Instead of throwing an exception:

Account not found

Return:

Northside Clinic

2 matches found.

Now the AI naturally responds:

"Did you mean Northside East or Northside West?"

This simple design choice creates much more natural conversations.

Privacy Matters

Enterprise applications often contain sensitive business information.

One rule dramatically improved both privacy and cost.

Raw database rows never enter the prompt.

Instead, tools only return:

IDs
Labels
Counts
Status

Detailed records remain inside the application.

Benefits include:

Better privacy
Lower token usage
Faster responses
Smaller prompts

Mobile Applications Have Different Challenges

Building Agentic AI on mobile introduces challenges that most tutorials ignore.

You must think about:

Offline mode
Background synchronization
Battery consumption
Poor network conditions
API rate limits
Provider failover
Secure API key management

Ignoring these realities usually produces assistants that work during demos but fail in production.

What I Learned

If I were building this architecture again, I'd follow these principles from day one.

Let the model handle reasoning.
Keep deterministic logic only for safety.
Build capabilities as independent skills.
Never bypass confirmation for write operations.
Return errors to the model instead of crashing.
Treat offline mode as a first-class feature.
Reuse existing business logic whenever possible.

Final Thoughts

Agentic AI isn't about replacing your application's business logic.

It's about giving users a completely new way to interact with it.

The model plans.

Your tools execute.

Your application guarantees safety.

When those responsibilities are clearly separated, AI becomes much more than a chatbot.

It becomes a reliable teammate that helps users complete real work.

If you're building Agentic AI for React Native, enterprise applications, or mobile platforms, I'd love to hear about your architecture and the lessons you've learned along the way.

Let's connect and continue the conversation.