DEV Community: Shivkrishna Shah

Tokens, Context, Parameters: The Real Mechanics of an LLM

Shivkrishna Shah — Wed, 29 Jul 2026 07:19:52 +0000

Tokens, Context, and Parameters: How Large Language Models Actually Work

Large language models can answer questions, generate software, summarize documents, and produce human-like writing. These abilities emerge from three closely connected elements:

Tokens represent the information entering and leaving the model.
Context contains the information available during the current prediction.
Parameters encode the statistical patterns learned during training.

Together, these components allow an LLM to predict one token at a time.

1. Tokens: The Model’s Units of Language

A language model does not process text directly as sentences or complete words. It first uses a tokenizer to divide text into smaller units called tokens.

A token can represent:

A complete word
Part of a word
Punctuation
A number
Whitespace
A special control symbol

For example:

The model is learning.

might become:

The · model · is · learn · ing · .

Breaking uncommon words into smaller pieces allows the model to represent a very large vocabulary without storing every possible word.

From text to token IDs

Every token is assigned an integer called a token ID:

"The"     → 464
" model"  → 2746
" is"     → 318
" learn"  → 2193
"ing"     → 278
"."        → 13

The exact IDs depend on the tokenizer used by the model.

Token IDs are only labels. Before they can be processed mathematically, the model converts them into vectors.

Token embeddings

The model contains a learned embedding matrix:

Vocabulary size × embedding dimension

If the vocabulary contains 50,000 tokens and the model’s internal dimension is 4,096, the embedding matrix has the shape:

50,000 × 4,096

Each row is a learned numerical representation of one token. Looking up a token ID selects the corresponding row from the matrix.

A token therefore becomes a vector such as:

[0.14, -0.82, 0.37, ..., 0.09]

The individual values usually do not have simple human-readable meanings. Meaning is distributed across many dimensions.

Position information

Transformers process multiple tokens in parallel, so they also need information about token order. The model adds or applies a positional representation to each token embedding.

Conceptually:

Input representation = token embedding + position information

This helps the model distinguish between sentences such as:

The dog chased the cat.

and:

The cat chased the dog.

They contain similar tokens, but their order changes their meaning.

2. Context: The Model’s Temporary Working Information

The context is the complete sequence of tokens available to the model during a prediction.

It can include:

System instructions
The user’s prompt
Previous conversation messages
Retrieved information
Uploaded document content
Tool results
Tokens already generated in the response

A model’s context window defines the maximum number of tokens it can process at one time.

Conceptually:

Instructions + conversation + documents + generated output ≤ context window

If the combined content exceeds the limit, some information must be removed, compressed, summarized, or handled through another retrieval mechanism.

Context is not permanent memory

Context acts more like temporary working memory than permanent storage. Information inside the active window can influence the next prediction. Information outside it normally cannot unless it is supplied again.

This creates an important distinction:

Parameters = patterns learned during training
Context    = information supplied for the current task

The context changes from one conversation to another. The model’s parameters normally remain fixed during ordinary inference.

Causal attention

Most text-generating LLMs use causal attention. When predicting a token, the model may attend to previous tokens but not future tokens.

For a sequence:

The capital of Japan is

the representation at the final position can use information from all preceding positions. It cannot use the answer token because that token has not been generated yet.

A causal mask enforces this direction:

Token 1 can attend to: 1
Token 2 can attend to: 1, 2
Token 3 can attend to: 1, 2, 3
Token 4 can attend to: 1, 2, 3, 4

This is what makes autoregressive generation possible.

3. Parameters: The Model’s Learned Numerical Structure

Parameters are the trainable numerical values inside the neural network. They include large matrices and smaller vectors used throughout the transformer.

The symbol θ, pronounced “theta,” is often used to represent all model parameters collectively.

A model with billions of parameters contains billions of learned floating-point values. These parameters do not form a searchable database of sentences. Instead, information is distributed across many interacting weights.

Where the parameters are located

A transformer language model typically contains parameters in several major components.

Embedding matrix

The embedding matrix converts token IDs into vectors:

token ID → embedding vector

These embeddings are learned during training.

Attention projections

Inside each self-attention layer, the input representation is transformed into queries, keys, and values:

Q = XWQ
K = XWK
V = XWV

Where:

X is the current sequence representation.
WQ is the query projection matrix.
WK is the key projection matrix.
WV is the value projection matrix.

Attention scores are calculated using:

Attention(Q, K, V) = softmax(QKᵀ / √dk)V

The result is commonly passed through another learned matrix, WO.

These matrices let the model determine which previous positions are relevant to the current token.

Multi-head attention

Transformers usually divide attention into multiple attention heads. Each head has separate projection weights and can learn different relationships.

Different heads may become sensitive to patterns involving:

Nearby words
Long-distance dependencies
Grammatical structure
References between names and pronouns
Formatting or positional relationships

The heads are combined and projected back into the model’s main representation space.

Feed-forward network

Each transformer block also contains a feed-forward network, often called an MLP.

A simplified version is:

MLP(X) = activation(XW1 + b1)W2 + b2

The first projection usually expands the representation into a larger intermediate dimension. The second projects it back to the model dimension.

Modern models may use gated structures such as SwiGLU, but the purpose is similar: transform the representation at each token position using learned nonlinear features.

Layer normalization

Layer-normalization components help keep internal activations numerically stable. They may contain learned scaling and shifting parameters.

Output projection

After the final transformer layer, the model converts the hidden representation into one score, or logit, for every possible token:

hidden state → vocabulary logits

A softmax function converts these logits into a probability distribution:

P(next token | previous tokens; θ)

The model can then select or sample the next token from this distribution.

4. Inside a Transformer Block

An LLM contains many transformer blocks stacked on top of one another. Each block gradually refines the representation of every token.

A simplified block contains:

Layer normalization
Multi-head self-attention
A residual connection
Another normalization stage
A feed-forward network
Another residual connection

The general flow is:

Input representations
        ↓
Layer normalization
        ↓
Self-attention
        ↓
Residual addition
        ↓
Layer normalization
        ↓
Feed-forward network
        ↓
Residual addition
        ↓
Output representations

Residual connections

Residual connections add a block’s input to its output:

output = input + transformation(input)

They help information and gradients travel through deep networks. Without them, training models with many layers would be considerably more difficult.

What changes through the layers?

Early layers may identify local or surface-level patterns. Deeper layers can combine information across positions and represent more abstract relationships.

The model’s internal representation of a token therefore depends on both:

The token’s original embedding
The surrounding tokens in the current context

The word “bank,” for example, may develop different contextual representations in:

She deposited money at the bank.

and:

They sat on the river bank.

5. How an LLM Is Trained

Before an LLM can generate useful text, its parameters must be learned from data.

The basic training objective is next-token prediction.

Step 1: Prepare the training data

Training text is cleaned, filtered, and converted into token sequences.

Suppose a training sequence is:

Large language models predict tokens

The model receives shifted input-and-target pairs:

Input:  Large
Target: language

Input:  Large language
Target: models

Input:  Large language models
Target: predict

In practice, many positions are trained simultaneously.

Step 2: Run a forward pass

The token sequence passes through the embedding layer and transformer blocks.

For every position, the model produces a probability distribution over the vocabulary.

For example:

"tokens"      0.61
"text"        0.14
"words"       0.09
"outputs"     0.04
other tokens  0.12

Step 3: Measure the error

The predicted distribution is compared with the actual next token using cross-entropy loss.

For the correct token y, the loss can be written as:

L = −log P(y | previous tokens; θ)

If the model gives the correct token a high probability, the loss is low. If it assigns the correct token a low probability, the loss is high.

Step 4: Backpropagate the error

The training system uses backpropagation to calculate how much each parameter contributed to the error.

These derivatives form the gradient:

∇θL

The gradient indicates which direction would increase the loss. Training moves the parameters in the opposite direction.

Step 5: Update the parameters

An optimizer updates the weights:

θ ← θ − η∇θL

Where:

θ represents the model’s parameters.
L represents the loss.
∇θL represents the loss gradient.
η represents the learning rate.

Real training usually employs optimizers such as Adam or AdamW rather than basic gradient descent, but the underlying idea is the same.

Step 6: Repeat across many batches

This process is repeated across enormous numbers of token sequences:

Forward pass
    ↓
Calculate loss
    ↓
Backpropagation
    ↓
Parameter update
    ↓
Next batch

Over time, the model becomes better at predicting tokens across many types of text.

6. Pretraining, Fine-Tuning, and Alignment

LLM development commonly involves several training stages.

Pretraining

During pretraining, the model learns general language patterns from a large and diverse dataset through next-token prediction.

It may learn patterns involving:

Grammar and syntax
Semantic relationships
Writing styles
Factual associations
Programming languages
Common reasoning structures

Pretraining creates the model’s broad foundational capabilities.

Supervised fine-tuning

The pretrained model may then be trained on curated examples containing prompts and high-quality responses.

This teaches it to follow instructions and produce answers in more useful formats.

Preference optimization

Human or AI-generated preference data can be used to improve helpfulness, safety, and response quality.

Possible approaches include:

Reinforcement learning from human feedback
Direct Preference Optimization
Related preference-learning techniques

These stages still modify parameters, but they use more targeted data and objectives than general pretraining.

7. How Generation Works After Training

During normal use, the model performs inference. Its learned parameters are usually fixed.

Suppose the prompt is:

The capital of Japan is

The generation process is:

The tokenizer converts the text into token IDs.
The embedding layer converts the IDs into vectors.
Position information is incorporated.
Transformer layers process the context.
The output layer produces vocabulary logits.
Softmax converts the logits into probabilities.
A decoding strategy chooses the next token.
The chosen token is added to the context.
The process repeats.

The model might produce:

Tokyo

After generating Tokyo, it predicts the token that should follow it.

Decoding strategies

The model does not always select the single highest-probability token. Generation can be controlled using methods such as:

Greedy decoding: choose the most probable token.
Temperature: adjust how sharp or varied the probability distribution is.
Top-k sampling: sample only from the k most likely tokens.
Top-p sampling: sample from the smallest group whose combined probability reaches a threshold.

These settings affect how the model selects output tokens. They do not retrain its parameters.

8. Parameters Versus Context

Parameters and context influence the same prediction in different ways.

Feature	Parameters	Context
Purpose	Store learned statistical patterns	Supply information for the current prediction
Origin	Learned during training	Provided at inference time
Duration	Persists across prompts	Temporary and task-specific
Typical scale	Millions or billions of values	Thousands or millions of tokens
Changed by normal prompting?	No	Yes
Example	Learned language structure	The current conversation

This distinction explains why a model can adapt to instructions without changing its weights. The instructions alter the context, which changes the model’s internal activations and predictions.

9. What Parameter Count Does—and Does Not—Mean

A parameter count measures the model’s numerical capacity. A larger model can potentially represent more complex patterns, but size alone does not determine performance.

Quality also depends on:

Training-data quality and diversity
Model architecture
Tokenizer design
Training duration
Optimization methods
Context handling
Fine-tuning and alignment
Inference techniques

A smaller model trained efficiently on appropriate data can outperform a larger model on a specialized task.

Parameter count should therefore be treated as one engineering characteristic—not a complete measure of intelligence or quality.

10. Common Misconceptions

“One token equals one word”

Not necessarily. A word may be one token, several tokens, or part of a larger token.

“The context window is permanent memory”

It is temporary information available during the current inference process.

“Parameters contain copies of documents”

Parameters encode distributed numerical patterns. They are not equivalent to files or database records, although models can sometimes reproduce memorized material.

“The model searches its training data for every answer”

During standard inference, the model calculates predictions using its parameters and current context. External search or retrieval occurs only when a separate tool or retrieval system is connected.

“The model learns from every conversation immediately”

Ordinary conversations usually do not update the model’s parameters. Updating parameters requires a separate training process.

“More parameters always produce a better model”

Model size is only one factor. Architecture, data, training, and task fit are equally important.

The Complete Picture

The complete LLM pipeline can be summarized as:

Text
  ↓
Tokenizer
  ↓
Token IDs
  ↓
Token embeddings + position information
  ↓
Transformer layers
  ├─ Self-attention
  ├─ Feed-forward networks
  ├─ Normalization
  └─ Residual connections
  ↓
Vocabulary logits
  ↓
Probability distribution
  ↓
Next token

During training, prediction errors are used to update the parameters:

Training data
  ↓
Next-token prediction
  ↓
Cross-entropy loss
  ↓
Backpropagation
  ↓
Optimizer update
  ↓
Improved parameters

During inference, those learned parameters are reused without normally being changed:

Current tokens + context + learned parameters
                         ↓
              next-token probability

The central idea is simple:

Tokens carry the information, context determines what the model can currently consider, and parameters define the learned transformations used to predict what comes next.

At Engineer Philosophy, our objective is not simply to connect an interface to an LLM. We design the complete information flow around the model—controlling the tokens it receives, constructing useful context, selecting the appropriate model, and validating its output for the intended application.

RAG Explained Simply: How AI Finds the Right Information Before Answering

Shivkrishna Shah — Sat, 18 Jul 2026 21:43:34 +0000

Most AI demos look impressive until you ask a question about your own company.

Ask an ordinary language model:

“Which customers recently complained about product pricing?”

The model may understand the question perfectly. But unless your customer feedback was included in its training data or supplied in the prompt, it does not know the answer.

It may refuse.

It may give a generic response.

Worse, it may produce an answer that sounds convincing but is not supported by your data.

This is the problem Retrieval-Augmented Generation—better known as RAG—is designed to solve.

The simple idea behind RAG

RAG gives an AI model access to relevant external information before asking it to answer.

Instead of using this flow:

Question
   ↓
LLM
   ↓
Answer

RAG uses:

Question
   ↓
Retrieve relevant information
   ↓
Give that information to the LLM
   ↓
Generate a grounded answer

The original RAG research combined a language model’s learned knowledge with external, non-parametric memory retrieved at runtime. This made it possible to update accessible knowledge without retraining the entire language model. citeturn146662search0

The important word here is runtime.

RAG does not automatically train the model on your database. It finds useful information when the user asks a question and temporarily places that information inside the model’s context window.

A practical example

Imagine that your application stores these survey responses:

Customer 101:
The product works well, but its price is too high.

Customer 205:
Availability is inconsistent in smaller cities.

Customer 318:
The product is affordable and easy to access.

The user asks:

“Which customers are concerned about affordability?”

A normal SQL keyword search may look for the exact word affordability.

That could miss:

“Its price is too high.”

A semantic search system can recognize that:

affordability
high price
expensive
cost concern

are related concepts.

That is where embeddings and vector databases enter the system.

How the complete RAG pipeline works

Step 1: Prepare your source data

Your source data may come from:

MySQL or PostgreSQL
Realm
PDFs
Word documents
CRM records
support tickets
survey answers
product documentation
internal policies

Structured database rows are usually converted into consistent text.

For example:

Account: ABC Pharmacy.
City: Indore.
Specialty: Retail pharmacy.
Survey response: Product pricing is too high for many patients.

You usually do not need an LLM to create this sentence. A deterministic template is cheaper, faster and more consistent.

Step 2: Create an embedding

An embedding model converts the text into a vector:

Text
  ↓
Embedding model
  ↓
[0.021, -0.184, 0.537, ...]

The vector is a numerical representation of meaning.

It is not encrypted text, and it cannot normally be decoded back into the original sentence.

Similar meanings produce nearby vectors.

For example:

Cardiologist
Heart specialist
Cardiac physician

should be positioned closer together than:

Cardiologist
Dermatologist

Step 3: Store the searchable representation

A vector database can store:

{
  "id": "FEEDBACK_184",
  "document": "The product is effective, but its price is too high.",
  "metadata": {
    "survey_id": "SURVEY_10",
    "account_id": "ACC_101",
    "entity_type": "survey_feedback"
  },
  "embedding": [0.021, -0.184, 0.537]
}

The important parts are:

ID: connects the vector result to the source record
document: readable text that may be returned during retrieval
metadata: exact fields used for filtering
embedding: used for similarity search

MySQL or Realm should normally remain the source of truth for changing business data. The vector database should act as a semantic index.

Step 4: Embed the user’s question

The user asks:

“Which customers are concerned about affordability?”

The same embedding model converts this question into a vector.

Using the same embedding model is important because the stored documents and the query must exist in the same vector space.

Step 5: Retrieve the nearest matches

The vector database searches for records whose embeddings are closest to the query embedding.

It may return:

1. FEEDBACK_184 — pricing is too high
2. FEEDBACK_422 — customers cannot afford the product
3. FEEDBACK_091 — treatment cost is a concern

This is not ordinary keyword matching. It is meaning-based retrieval.

Step 6: Fetch current records when necessary

For documentation and relatively static content, the document returned by the vector database may be enough.

For live enterprise data, use the returned IDs to fetch current records from Realm or MySQL:

SELECT
    id,
    account_id,
    response,
    updated_at
FROM survey_answers
WHERE id IN (184, 422, 91);

This prevents an old vector index from becoming the final authority.

Step 7: Build the model context

The application constructs a controlled prompt:

System instruction:
Answer only from the supplied evidence.
Do not invent customers or conclusions.

User question:
Which customers are concerned about affordability?

Retrieved evidence:
1. Account 101 said the price is too high.
2. Account 205 said many patients cannot afford the product.
3. Account 318 said treatment cost is a concern.

Step 8: Generate a grounded answer

The language model can now respond:

Three customers expressed affordability concerns. Their feedback focuses on high product prices, limited patient affordability and overall treatment cost.

The model did not independently know this.

The system retrieved the evidence and placed it in the model’s temporary context.

RAG is not only a vector database

A common mistake is to think:

RAG = embeddings + vector database.

That is incomplete.

A reliable RAG system also needs:

document preparation
chunking
metadata design
access control
query understanding
retrieval
filtering
reranking
context construction
answer generation
citations
evaluation
monitoring

The language model is only one component.

SQL search and vector search solve different problems

Use SQL for exact and mathematical questions:

How many surveys are pending?
What is the average call frequency?
Which city has the highest sales?

Use SQL or deterministic tools for those.

Use vector search for semantic questions:

What complaints are related to product affordability?
Find doctors similar to this profile.
Which responses discuss supply problems?

The strongest enterprise architecture combines both:

User request
    ↓
Request router
    ├── Exact analytics → SQL / Realm
    ├── Semantic retrieval → Vector database
    ├── Business action → Trusted application tool
    └── Explanation → LLM

RAG does not eliminate hallucination

RAG can improve factuality, but it does not guarantee truth.

A RAG system can still fail when:

the wrong documents are retrieved
important records are missing
chunks lose necessary context
stale data remains indexed
access filters are incorrect
the prompt allows unsupported conclusions
the model misinterprets good evidence

Broader RAG research describes retrieval as a way to improve the accuracy and credibility of knowledge-intensive generation while helping address outdated knowledge and weak provenance. It is still an engineering system that must be evaluated rather than blindly trusted. citeturn146662search8

What is the future of RAG?

RAG is moving beyond the simple “retrieve once, then answer” pipeline.

1. Hybrid retrieval

Future-ready systems combine:

vector similarity
keyword search
SQL filters
metadata filters
graph relationships

Vector search finds meaning. Keyword search preserves exact names and terminology. Metadata applies strict business boundaries.

2. Reranking

The first search may retrieve 20 candidates.

A reranking model then evaluates those candidates more carefully and selects the strongest evidence for the final prompt.

This improves retrieval quality without filling the context window with weak results.

3. Multimodal RAG

Enterprise knowledge is not limited to text.

RAG systems increasingly retrieve from:

images
screenshots
charts
audio
video
scanned forms
maps and spatial records

A user may eventually ask:

“Compare the issue shown in this site photograph with last month’s inspection report.”

The system will need to retrieve across multiple data types.

4. Graph-based retrieval

Some questions depend on relationships rather than isolated documents:

Customer
  ↓ belongs to
Territory
  ↓ managed by
Representative
  ↓ assigned to
Campaign

Knowledge graphs and graph-aware retrieval can help answer multi-hop questions where information is distributed across related entities.

5. Real-time and event-driven RAG

Instead of rebuilding the entire vector index, production systems can update affected records when business data changes.

MySQL record updated
       ↓
Event published
       ↓
Text regenerated
       ↓
Embedding updated
       ↓
Vector record replaced

This is especially important for CRM, operational and mobile applications.

6. Agentic RAG

Traditional RAG follows a fixed pipeline.

Agentic RAG allows an agent to decide:

whether retrieval is needed
which data source to search
how to rewrite the query
whether evidence is sufficient
whether another search is required
which tool should calculate the answer
whether the response needs human approval

Recent Agentic RAG literature describes systems using planning, reflection, tool use and multi-agent collaboration to adapt retrieval for complex, multi-step work. citeturn146662search5turn146662search10

A future workflow may look like:

User goal
   ↓
Agent creates a plan
   ↓
Search documentation
   ↓
Query SQL
   ↓
Check Realm data
   ↓
Compare evidence
   ↓
Retry weak retrieval
   ↓
Generate answer
   ↓
Request confirmation
   ↓
Execute an approved action

Research published in 2025 and 2026 is actively exploring multi-agent retrieval, hierarchical retrieval interfaces and more efficient agentic search. These approaches are promising, but they also introduce more latency, token consumption and operational complexity. citeturn146662search33turn146662search14turn146662academia56

A practical mobile architecture

For an offline-first React Native application using Realm:

React Native application
        ↓
Chat interface and voice input
        ↓
Chatbot orchestrator
        ├── Realm query tools
        ├── Validation action tools
        ├── What-if simulation tools
        └── Server RAG API
                ├── Embedding model
                ├── Vector database
                ├── MySQL
                └── LLM

Realm stores current offline application data.

MySQL remains the server-side source of truth.

The vector database provides semantic retrieval.

The LLM interprets questions and explains verified data.

Trusted application code performs business actions.

The key lesson

RAG is not a magical database that makes an LLM intelligent.

It is a controlled information pipeline.

The embedding model represents meaning.

The vector database retrieves related information.

MySQL and Realm preserve current business facts.

The LLM turns evidence into language.

Application tools enforce business rules.

The future is not an AI model replacing every database.

The future is an AI system that knows:

which database to use
what information to retrieve
how to verify it
when to ask for approval
how to act safely

That is where RAG becomes more than a chatbot feature.

It becomes the knowledge layer behind an enterprise AI assistant.

At Engineer Philosophy Web Services Pvt. Ltd., we are exploring how RAG, Agentic AI, mobile applications, offline databases, and enterprise systems can work together to create AI assistants that do more than answer questions.

engineerphilosophy.com

AgenticAI #ArtificialIntelligence #AIAgents #GenerativeAI #EnterpriseAI #Automation #Innovation #DigitalTransformation #EngineerPhilosophy #MAPOG #shivkrishnashah

[Boost]

Shivkrishna Shah — Thu, 09 Jul 2026 11:50:09 +0000

Shivkrishna Shah

Jul 9

Building an Enterprise AI Assistant That Can Actually Take Action

#ai #programming #mobile #reactnative

4 min read

Building an Enterprise AI Assistant That Can Actually Take Action

Shivkrishna Shah — Thu, 09 Jul 2026 06:30:35 +0000

Imagine you're a field representative visiting customers all day.

It's 6:30 PM.

You still have 40 survey responses to complete before heading home.

Instead of navigating through multiple screens, wouldn't it be easier if you could simply say:

"Mark Question 3 as Yes for Northside Clinic and tell me what's left before I submit."

Most AI assistants can't do this.

They can answer questions.

They can summarize information.

They can explain how to complete a task.

But they can't actually perform the task.

That's the difference between a chatbot and an Agentic AI system.

What Is Agentic AI?

In the context of mobile applications, Agentic AI means giving an LLM the ability to reason, plan, and safely perform actions inside your application.

Instead of telling users where to click, the assistant actually performs the required operations using your application's tools.

An agent can:

Search application data
Read local state
Validate information
Update records
Execute business workflows
Ask follow-up questions when information is missing

Instead of becoming another search box, AI becomes another user of your application.

Why Most Mobile AI Projects Fail

After building and experimenting with production-grade AI assistants, I noticed two common mistakes.

Mistake #1 — AI Wrapped Around If-Else Statements

Many applications still rely on intent matching.

if(message.includes("survey")){
   openSurvey();
}

An LLM simply generates the response.

The routing is still deterministic.

This works during demos.

It usually breaks in production.

Users never ask questions exactly the way developers expect.

Mistake #2 — Giving the AI Unlimited Access

The opposite mistake is allowing the model to directly modify application data.

Without proper validation, an AI can:

Update the wrong record
Modify unintended data
Hallucinate IDs
Save information the user never approved

Neither approach belongs in a production application.

The Architecture That Worked

After several iterations, I settled on one simple principle.

The model makes decisions. The application guarantees safety.

The AI should handle:

Reasoning
Planning
Selecting tools
Recovering from failures
Asking follow-up questions

Your application should handle:

Validation
Permissions
Confirmations
Database writes
Privacy
Security
Execution limits

Once those responsibilities are separated, the architecture becomes both flexible and reliable.

The Agent Loop

Everything begins with one simple loop.

Receive the user's message
Send conversation context to the LLM
Receive requested tool calls
Execute tools locally
Return tool results
Continue until the task is complete

That's it.

No keyword routing.

No intent classifier.

No hardcoded workflows.

The model decides what happens next.

(Insert simplified agent loop diagram here.)

Skills Instead of Massive Prompts

One of the biggest improvements was replacing one enormous system prompt with small reusable skills.

Each skill contains:

Instructions
Available tools
Validation logic
Optional hooks

Example:

Survey Skill

Available Tools

• Find Survey
• Update Answer
• Save Draft
• Submit Survey

Adding a new capability became as simple as adding another folder.

The engine never changes.

Confirmation Before Every Write

One lesson became obvious very quickly.

Never allow an AI model to write directly into your database.

Every write follows the same process.

Stage Changes
      ↓
Show Confirmation
      ↓
 User Approves
      ↓
   Validate
      ↓
     Save

Even if the AI attempts to skip confirmation, the application blocks it.

The runtime—not the model—controls safety.

Let the Model Recover

One surprising discovery was that AI becomes significantly smarter when you return errors instead of hiding them.

Instead of throwing an exception:

Account not found

Return:

Northside Clinic

2 matches found.

Now the AI naturally responds:

"Did you mean Northside East or Northside West?"

This simple design choice creates much more natural conversations.

Privacy Matters

Enterprise applications often contain sensitive business information.

One rule dramatically improved both privacy and cost.

Raw database rows never enter the prompt.

Instead, tools only return:

IDs
Labels
Counts
Status

Detailed records remain inside the application.

Benefits include:

Better privacy
Lower token usage
Faster responses
Smaller prompts

Mobile Applications Have Different Challenges

Building Agentic AI on mobile introduces challenges that most tutorials ignore.

You must think about:

Offline mode
Background synchronization
Battery consumption
Poor network conditions
API rate limits
Provider failover
Secure API key management

Ignoring these realities usually produces assistants that work during demos but fail in production.

What I Learned

If I were building this architecture again, I'd follow these principles from day one.

Let the model handle reasoning.
Keep deterministic logic only for safety.
Build capabilities as independent skills.
Never bypass confirmation for write operations.
Return errors to the model instead of crashing.
Treat offline mode as a first-class feature.
Reuse existing business logic whenever possible.

Final Thoughts

Agentic AI isn't about replacing your application's business logic.

It's about giving users a completely new way to interact with it.

The model plans.

Your tools execute.

Your application guarantees safety.

When those responsibilities are clearly separated, AI becomes much more than a chatbot.

It becomes a reliable teammate that helps users complete real work.

If you're building Agentic AI for React Native, enterprise applications, or mobile platforms, I'd love to hear about your architecture and the lessons you've learned along the way.

Let's connect and continue the conversation.