DEV Community: Nishanthan

Core Elements of Large Language Models (LLMs): Tokens, Context Windows, Temperature, and Sampling

Nishanthan — Mon, 26 Oct 2026 11:30:00 +0000

Large Language Models (LLMs) have become a fundamental part of modern AI applications, powering everything from conversational chatbots to code generation and intelligent assistants. While their capabilities may seem complex, understanding the core elements behind how they process and generate text provides a clearer picture of how they work.

In this article, we'll explore some of the fundamental concepts behind LLMs, including tokens, context windows, temperature, Top-K sampling, Top-P sampling, and repetition penalties. Understanding these elements will help you build a stronger foundation in LLMs and understand how different parameters influence their outputs.

What are Tokens?

Tokens are individual units of data that are fed into a model during training. They can be words, phrases, or even entire sentences depending on the type of model being trained.

For example, in NLP, tokens are commonly used to represent words in a text. Consider the sentence "Hello, world!" - it might be tokenized into ["Hello", ",", "world", "!"]. These tokens are then used as inputs for the model to learn patterns and relationships between them.

Tokens can also represent other types of data such as numerical values or images. For instance, in image recognition tasks, each pixel in an image is a token that the model uses to identify and classify objects.

Types of Tokens

Tokens can take many forms depending on the type of data and the task at hand. Let’s look at a brief overview of the common types of tokens:

Word Tokens: Each word is treated as a separate token.
Subword Tokens: Words are broken down into smaller meaningful units to handle out-of-vocabulary words better. E.g. "cats" can be broken down into "cat" and "s".
Phrase Tokens: These consist of multiple words that are grouped together, such as "New York City" or "machine learning".
Character Tokens: These represent individual characters within a word.
Image Tokens: These can include pixels, image segments, or other visual features used in computer vision tasks.
Byte-Pair Encoding (BPE): This is a type of tokenization that uses an algorithm to merge the most frequently occurring character pairs in a given text corpus. BPE is commonly used in speech recognition and natural language processing tasks.

Context Window

The context window (or “context length”) of a large language model (LLM) is the amount of text, in tokens, that the model can consider or “remember” at any one time. A larger context window enables an AI model to process longer inputs and incorporate a greater amount of information into each output.

An LLM’s context window can be thought of as the equivalent of its working memory. It determines how long of a conversation it can carry out without forgetting details from earlier in the exchange. It also determines the maximum size of documents or code samples that it can process at once. When a prompt, conversation, document or code base exceeds an artificial intelligence model’s context window, it must be truncated or summarized for the model to proceed.

Generally speaking, increasing an LLM’s context window size translates to increased accuracy, fewer hallucinations, more coherent model responses, longer conversations and an improved ability to analyze longer sequences of data. However, increasing context length is not without tradeoffs: it often entails increased computational power requirements and therefore increased costs.

Equipping a model with a large context window comes at a cost, both figuratively and literally. Compute requirements scale quadratically with the length of a sequence: for instance, if the number of input tokens doubles, the model needs 4 times as much processing power to handle it.

Temperature

Temperature is a crucial feature for controlling randomness in model performance. It allows users to adjust the LLM output to better suit different real-world applications of text generation. More specifically, this LLM setting allows users to balance coherence and creativity when generating output for a specific use case. For instance, a low temperature might be preferable for tasks requiring precision and factual accuracy, such as technical documentation or conversational replies with chatbots. The lower temperature value helps the LLM to produce more coherent and consistent text and avoid irrelevant responses. By contrast, a high temperature is preferable for creative outputs or creative tasks such as creative writing or concept brainstorming. The temperature setting effectively allows users to fine-tune LLMs and adjust a model's output to their own desired outcome.

Top-K Sampling

Top-K sampling limits the model to choosing from the K most likely tokens at each step. For instance:

If top-K = 20, the model only considers the 20 most probable tokens, ignoring less likely ones. This makes responses more focused and factual.
If top-K = 40, the model has more options, which adds variety but can introduce less predictable results.

Think of top-K as narrowing the model’s “vocabulary” to a shortlist of top contenders. A smaller K value keeps things tight and controlled, while a larger K allows for more creativity.

Top-P (Nucleus Sampling)

Top-P sampling, also called nucleus sampling, selects a group of tokens whose cumulative probability is at least P. For example:

If top-P = 0.9, the model picks the smallest set of tokens whose combined probability reaches 90%. This filters out low-probability tokens, balancing accuracy and creativity.
A lower top-P (e.g., 0.8) makes the output more conservative, while a higher top-P (e.g., 0.99) allows more diversity.

Unlike top-K, which fixes the number of tokens, top-P dynamically adjusts based on probability, making it more flexible for tasks requiring varied outputs.

How They Work Together

Top-K and top-P are often used alongside temperature, which controls the randomness of the model’s choices:

Low temperature (e.g., 0.1) makes the model stick closely to high-probability tokens, ideal for precise tasks.
High temperature (e.g., 0.9) increases randomness, boosting creativity but risking incoherent outputs.

Repetition Penalty

A repetition penalty is an inference-time decoding parameter in Large Language Models (LLMs) that reduces the probability of generating tokens that have already appeared in the text. It helps prevent repetitive phrases and endless loops during text generation.

The penalty is applied to previously encountered tokens by modifying their logits before the next token is selected.

1.0: No penalty is applied.
Greater than 1.0: Repeated tokens are discouraged.
Too high: Can make the output unnatural or reduce clarity.

Repetition penalty is useful in chatbots, content generation, and long-form text generation. However, excessive penalties can discourage necessary word repetition and affect the quality of responses.

Conclusion

Understanding the core elements of LLMs is an important step toward understanding how modern language models work. From breaking text into tokens and processing information within a context window to controlling output generation through temperature, sampling strategies, and repetition penalties, each element plays a role in shaping the model's behavior.

These concepts provide a foundation for exploring more advanced topics, such as Transformer architecture, attention mechanisms, model inference, and LLM optimization. As you continue learning about LLMs, understanding these fundamentals will help you move beyond simply using AI models to understanding how to configure, optimize, and build applications around them.

How Query, Key, and Value (QKV) Work in Transformers

Nishanthan — Thu, 22 Oct 2026 08:17:29 +0000

Every Transformer-based language model, from BERT to GPT, relies on a mechanism called attention to determine how different tokens in a sequence relate to one another. At the heart of this mechanism are three representations: Query (Q), Key (K), and Value (V).

Let's understand what they are, why we need them, and how they work together to help Transformers process language.

1. What Are Query, Key, and Value (QKV)?

To understand QKV, imagine searching for information in a large collection of documents. You have a question you want answered, each document has information that helps you decide whether it is relevant, and each document contains the actual content you might retrieve.

The attention mechanism follows a conceptually similar process: it determines what information a token should gather from other tokens in the sequence.

Query (Q) — What information am I looking for?

A Query represents what a token is looking for in other tokens. It helps determine which tokens contain information relevant to the current token.

Key (K) — What information do I match on?

A Key represents the features used to match a token against a Query. It helps determine how relevant that token is to another token's Query.

Value (V) — What information do I contribute?

A Value represents the information a token contributes when other tokens attend to it. The attention mechanism combines Values according to the attention weights.

In simple terms:

Query determines what a token is looking for.
Key helps determine how relevant another token is.
Value supplies the information that gets incorporated.

These are not three different tokens. They are three different representations of tokens, each serving a distinct purpose in the attention mechanism.

2. Why Do We Need Q, K, and V?

Words get their meaning from context. Consider the word bank in these two sentences:

"I deposited money in the bank."
"We sat on the bank of the river."

The word is the same, but its meaning changes depending on the surrounding words. A Transformer needs a way to understand these relationships.

This is where self-attention comes in. It allows each token to gather relevant information from other tokens in the sequence.

But how does a token decide which other tokens are relevant and what information to gather from them?

That's why we need Query, Key, and Value.

For example, in "The animal was tired because it had been running," the token it can attend to animal and other relevant words to build a context-aware representation.

Without this mechanism, the model would have a much harder time connecting information across a sequence. QKV gives self-attention a flexible, learnable way to determine which tokens should influence one another and how their information should be combined.

In short: Queries and Keys help determine where attention goes, while Values determine what information is gathered.

3. How Are Q, K, and V Generated?

Now that we understand why Q, K, and V are needed, let's look at how a Transformer actually generates them.

It starts with the input representations of the tokens. These are arranged into a matrix called (X), where each row represents a token.

The Transformer then uses three learnable weight matrices to create Query, Key, and Value:

Q = X × W_Q

K = X × W_K

V = X × W_V

Here, (W_Q), (W_K), and (W_V) are learnable weight matrices. Each transforms the input representations into a different representation for its specific role.

Why three matrices? Because the model needs to learn what information is useful for calculating attention scores and what information should be passed along through the Values.

These matrices are learned during training through backpropagation and gradient-based optimization.

The important point is that Q, K, and V are not manually defined. The model learns how to generate them from the input representations.

4. How Do Queries and Keys Calculate Attention?

Once the Transformer generates Q, K, and V, it needs to determine how relevant each token is to the others. This is where Queries and Keys come into play.

Step 1: Calculate Attention Scores

The model multiplies the Query matrix (Q) by the transpose of the Key matrix (K):

S = Q × Kᵀ

Each score represents how strongly a token's Query matches another token's Key. Higher scores generally lead to higher attention weights, although these scores are learned representations rather than direct measures of semantic similarity.

Step 2: Scale the Scores

Before applying softmax, the model divides the scores by the square root of the Key dimension, (d_k):

S_scaled = (Q × Kᵀ) / √dₖ

This scaling helps prevent the dot products from becoming excessively large, which can cause softmax to produce very peaked distributions and make optimization less effective.

Step 3: Apply Softmax

Finally, softmax converts the scaled scores into normalized attention weights:

A = softmax((Q × Kᵀ) / √dₖ)

Each row of (A) contains the attention weights for one token, indicating how its attention is distributed across the available tokens. In causal language models, an attention mask prevents tokens from attending to future positions.

In short: Queries and Keys calculate relevance scores, scaling helps stabilize them, and softmax converts them into attention weights.

5. How Do Values Produce Contextual Information?

After calculating the attention weights, the Transformer uses them to determine how much information to gather from each token. This is where Values (V) come into play.

The attention weights are multiplied by the Value matrix to produce the attention output:

O = A × V

Substituting the attention formula gives us the complete scaled dot-product attention equation:

Attention(Q, K, V) = softmax((Q × Kᵀ) / √dₖ) × V

Here, each token's output is a weighted combination of the Value vectors from the tokens it is allowed to attend to.

For example, when processing "The animal was tired because it had been running," the representation of it can incorporate information from animal and other relevant tokens, helping the model build a context-aware representation.

The model does not simply select the token with the highest attention score. Instead, it generally combines information from multiple tokens according to their attention weights.

In short: Queries and Keys determine how attention is distributed, while Values carry the information that gets combined to produce the attention output.

6. QKV in Multi-Head Attention

So far, we've explored how a single attention mechanism uses Query, Key, and Value to determine relationships between tokens and combine information. But what if a Transformer needs to capture different types of relationships at the same time?

This is where Multi-Head Attention comes in.

Instead of using a single attention mechanism, the Transformer uses multiple attention heads, each with its own learned Query, Key, and Value projections. Different heads can learn to focus on different patterns, such as grammatical relationships or connections between entities.

Each head calculates attention independently:

headᵢ = Attention(X × WᵢQ, X × WᵢK, X × WᵢV)

The outputs from all heads are then concatenated and passed through a final learnable projection matrix:

MultiHead(Q, K, V) = Concat(head₁, ..., headₕ) × Wᴼ

Here, (h) is the number of attention heads, and (W^O) is the output projection matrix.

By allowing multiple attention heads to operate in parallel, Transformers can learn different patterns of relationships within the same sequence and combine them into a single representation.

In short: Each attention head learns its own way of gathering information, and the output projection combines the results into a unified representation.

Conclusion

Query, Key, and Value form the foundation of the attention mechanism in Transformers.

Query determines what information a token is looking for.
Key helps determine which tokens are relevant.
Value carries the information that gets incorporated into the output.

Together, they enable self-attention to calculate relationships between tokens and build context-aware representations. Multi-Head Attention extends this idea by allowing multiple attention mechanisms to learn different patterns in parallel.

Understanding QKV is an important step toward understanding how Transformers process language and how modern large language models work under the hood.

Understanding Transformer: The Architecture Behind Modern AI

Nishanthan — Thu, 15 Oct 2026 03:30:00 +0000

What Is a Transformer?

A Transformer is a neural network architecture introduced by Google researchers in 2017 in the paper Attention Is All You Need.

It uses attention mechanisms to help the model learn relationships between tokens. Unlike recurrent neural networks (RNNs) and long short-term memory networks (LSTMs), which process sequences through recurrent steps, Transformers can process input token representations in parallel during training.

The Transformer architecture combines attention with other components, including feed-forward neural networks, positional information, residual connections, and normalization, to build contextual representations of input tokens.

Why Were Transformers Introduced?

Transformers addressed important limitations of recurrent neural networks (RNNs) in sequence processing.

Two key motivations were:

Learning relationships between tokens: RNNs and LSTMs can struggle to capture dependencies between tokens that are far apart. Transformers use attention mechanisms to let tokens incorporate information from other relevant tokens directly.
Parallel processing: RNNs process tokens sequentially because each hidden state depends on the previous one. Transformers enable parallel computation across input tokens during training, improving computational efficiency.

Although LSTMs improved the ability of recurrent networks to retain information over long sequences, they did not remove the sequential-processing limitation.

What Problem Were Transformers Designed to Solve?

Before Transformers, RNNs and LSTMs were widely used to process sequential data, including natural language. They processed tokens sequentially while maintaining a hidden state that carried information from earlier tokens.

However, these architectures had two important limitations.

1. Difficulty learning long-range dependencies

In recurrent networks, information from earlier tokens must pass through multiple recurrent steps to influence later tokens. During training, gradients can vanish as they propagate through long sequences, making it difficult to learn some long-range relationships. LSTMs mitigate this issue through their cell state and gating mechanisms, but they do not eliminate every difficulty.

2. Limited parallelism

Because each recurrent hidden state depends on the previous state, RNNs and LSTMs cannot compute all hidden states of an input sequence simultaneously. This limits parallel computation during training.

Transformers introduced an attention-based architecture without recurrence. Self-attention enables tokens to incorporate information from other relevant tokens directly, while the absence of recurrent dependencies allows input token representations to be processed in parallel during training.

How RNNs and LSTMs Process Sequences

RNN: A recurrent neural network processes a sequence one token at a time. At each step, it combines the current token representation with the previous hidden state to produce a new hidden state. This state carries a learned representation of information from the sequence processed so far.

Limitation: As sequences become longer, an RNN can struggle to preserve useful information from earlier tokens. During training, vanishing gradients can also make long-range dependencies difficult to learn.

LSTM: Long Short-Term Memory networks extend RNNs with a cell state and three gates: forget, input, and output. These mechanisms regulate which information is retained, added, or exposed. They help preserve relevant information over longer sequences and improve gradient flow.

Remaining limitation: LSTMs still process tokens sequentially because each recurrent step depends on the previous step.

Why Was Sequential Processing a Limitation?

RNNs and LSTMs process sequences through recurrent steps, where each hidden state depends on the previous state. This dependency prevents the model from computing all recurrent hidden states simultaneously.

As a result, these architectures limit parallel computation across sequence positions during training, particularly for long sequences.

Transformers remove recurrence from the core architecture and use self-attention to process input token representations in parallel during training. This enables more efficient use of modern parallel hardware, although Transformer attention also introduces computational and memory costs as sequence length increases.

What Did the 2017 Paper Attention Is All You Need Introduce?

In 2017, Vaswani and colleagues introduced the Transformer architecture in their paper Attention Is All You Need.

The paper proposed an encoder-decoder architecture that relied on attention mechanisms instead of recurrent or convolutional layers for sequence processing.

Its key contributions included:

Self-attention: Allowed tokens to incorporate information from other positions in a sequence.
Multi-head attention: Enabled the model to learn different relationships through multiple attention heads.
Parallelizable computation: Removing recurrence allowed input sequence representations to be processed in parallel during training.

The original Transformer was designed for sequence transduction tasks, particularly machine translation. Its architecture later became the foundation for many language models, including encoder-only, decoder-only, and encoder-decoder models.

How Do Transformers Process Relationships Between Tokens?

Transformers use a mechanism called self-attention to model relationships between tokens in a sequence.

Unlike recurrent networks, which pass information through successive hidden states, self-attention allows each token to incorporate information from other relevant tokens directly.

For each token, the model computes attention scores that determine how much information to gather from other token positions. These scores are converted into weights, which are used to combine information from those tokens and produce updated representations.

As a result, each token can develop a contextual representation that incorporates information from across the sequence. These representations are then processed by other Transformer components, including feed-forward neural networks.

This mechanism helps Transformers learn relationships between distant tokens and provides a flexible way to represent context.

Where Are Transformers Used Today?

Transformers have become a foundational architecture in modern AI, with applications across several domains.

Natural language processing: Large language models use Transformers for text generation, question answering, coding, summarization, and conversational AI. Transformer-based encoder models are also used for classification and information extraction.
Machine translation: Encoder-decoder Transformers learn relationships between source and target sequences to translate between languages.
Computer vision: Vision Transformers process image patches to perform tasks such as image classification and image understanding.
Speech and audio: Transformer-based architectures support speech recognition, transcription, translation, and audio generation.
Multimodal AI: These models combine information from modalities such as text, images, audio, and video to support tasks that require understanding multiple forms of input.

Across these applications, the central idea is to learn useful representations and relationships among elements of the input. The specific architecture and training objective depend on the task.

Conclusion

Transformers marked a significant shift in how neural networks process sequential data. By replacing recurrent processing with attention-based mechanisms, they enabled models to learn relationships between tokens more directly while supporting greater parallelism during training.

The idea introduced in Attention Is All You Need in 2017 has since become foundational to modern AI, powering advances in language processing, computer vision, speech, and multimodal systems.

Understanding Transformers is more than learning a particular neural network architecture. It is understanding how a different approach to representing context and relationships can unlock new capabilities in AI.

And this is only the beginning. To understand how modern language models work under the hood, the next step is to explore self-attention in detail: how Query, Key, and Value representations are created, how attention scores are calculated, and how those scores determine the information each token receives.

Backpropagation in Deep Learning: A Complete, Intuitive, and Practical Guide

Nishanthan — Wed, 17 Dec 2025 03:30:00 +0000

Introduction: Why Backpropagation Matters

Backpropagation is the learning engine behind modern deep learning.
Every time a neural network improves its predictions — from recognizing faces to generating text — the improvement comes from one mechanism: adjusting weights using gradients, and those gradients are computed through backpropagation.

Why this matters:

A neural network is essentially a giant parametric function with millions or billions of weights.
Backpropagation provides an efficient way to compute how much each weight contributed to the final error.
Without backpropagation, training a deep model would be computationally impossible — you’d need to recompute partial derivatives for every weight independently, which would scale exponentially.
Backprop makes learning practical and efficient, even for very large models.
It also forms the foundation for:
- Convolutional networks
- Transformers -Diffusion models
- Reinforcement learning
- Large language models
In short, backpropagation is the backbone of modern AI.

What a Neural Network Really Computes (Forward Pass Overview)

Before understanding backpropagation, we need a clear picture of what the network does in the forward pass because backprop is simply the reverse of this computation.
At its core, a neural network performs three steps repeatedly across layers:
1. Take inputs from the previous layer
2. Apply a linear transformation using weights and biases
3. Pass the result through a non-linear activation function

Mathematically, for a single layer:

z = Wx + b
a = f(z)

Where:
- x = input vector
- W = weight matrix
- b = bias vector
- z = linear combination (pre-activation)
- f(⋅) = activation function (ReLU, Sigmoid, etc.)
- a = output (activation)

A deep network simply repeats this process layer after layer.
Each layer transforms the input into a representation that makes the final task easier — whether that’s classification, detection, language modeling, or something else.

Why this matters for backprop:
- The forward pass creates a computational graph — a chain of operations where each output depends on previous ones.
- Backprop will move in the opposite direction through this graph, calculating how changes in each weight affect the final loss.
- Knowing the exact operations in the forward pass helps us understand how the gradients flow backward.

The Core Idea Behind Backpropagation

How should each weight in the network change to reduce the final error?”

A neural network may have millions of parameters, and each parameter influences the output in a slightly different way. Backpropagation provides a systematic method to compute these influences efficiently
Here’s the core idea in simple terms:
- Calculate the loss: After the forward pass, we compare the model’s prediction with the true label and compute how “wrong” the prediction was.
- Determine how the loss changes with respect to the output of the last layer. This tells us the immediate direction in which the model should adjust.
- Move backward layer by layer
  - Using the chain rule, we figure out how the loss depends on:
    1. each layer’s activations
    2. each layer’s weights
    3. each layer’s inputs
- Accumulate gradients
  - Every weight gets a gradient value that tells us:
    - direction (positive or negative)
    - magnitude (how strong the update should be)
- Update the weights
  - Optimizers like SGD or Adam use these gradients to move the weights to slightly better values.

The key insight:

Backprop does not compute gradients from scratch for each weight.
It reuses intermediate results from the forward pass, which makes the entire process efficient.

Understanding the Computational Graph

A neural network can be viewed as a computational graph — a sequence of operations where each node represents a mathematical function, and each edge represents the flow of data.

Why this graph matters:

Every value produced during the forward pass (like z, a, activations, etc.) becomes part of the graph.
Backpropagation relies on this graph to know how outputs depend on earlier computations.
Deep learning frameworks like PyTorch and TensorFlow automatically build this graph behind the scenes.

Chain Rule: The Mathematical Engine of Backpropagation

Backpropagation is built entirely on one mathematical principle: the chain rule of calculus.
If you understand the chain rule, you understand the heart of backprop.

Why the chain rule is needed

In a neural network, the loss doesn’t depend on any weight directly.
Instead, it depends on:
- the activations of the last layer
- which depend on the previous layer
- which depend on the one before it
This creates a long chain of dependencies.
The chain rule tells us how a change in any early variable (like a weight) affects the final loss.

Applying the chain rule to layers

Consider one layer:

    z = Wx + b
    a = f(z)

And the loss:

    L = Loss(a)

Using the chain rule, the gradient of the loss with respect to the weights is:

    ∂L/∂W = (∂L/∂a) · (∂a/∂z) · (∂z/∂W)

    where
        ∂L/∂a: how the loss changes with the layer’s output
        ∂a/∂z: how the activation reacts to its input
        ∂z/∂W: how the output of this layer depends on its weights

Step-by-Step Backpropagation Through a Single Neuron

Now that the chain rule and computational graph ideas are clear, let’s walk through backpropagaion inside one neuron.
This is the smallest unit of a neural network, and understanding it removes most of the confusion around deeper models.

Forward Pass (Single Neuron)

A neuron computes:

    z = Wx + b
    a = f(z)

    where
        x -> input
        W -> weight
        b -> bias
        f -> activation function
        a -> output of the neuron

    L(a): Loss function

Backward Pass (Goal: compute gradients)

We want

Gradient with respect to the activation output

    ∂L/∂a

Gradient through the activation function

    ∂L/∂z = (∂L/∂a) · f′(z)

Gradient with respect to the weights

    ∂L/∂W = (∂L/∂z) · x

Since   z = Wx + b

        ∂z/∂W = x

So,
        ∂L/∂W = ∂L/∂z * x

Gradient with respect to the bias

        ∂z/∂b = 1
So,
        ∂L/∂b = ∂L/∂z

Gradient with respect to the input

        ∂L/∂x = (∂L/∂z) · W

Conclusion

Backpropagation is more than a mathematical procedure — it’s the core mechanism that allows neural networks to learn.
Once you understand how gradients flow through a network, you gain the ability to reason about:

Why certain architectures train well?
Why others fail?
How initialization, activations, normalization, and residual connections shape gradient behavior?
What makes transformers, CNNs, and deep RNNs stable?
How to debug training issues with confidence?

The deeper your intuition for backpropagation, the more effective you become at designing and improving models.

Neural Network — A Simple, Beginner-Friendly Overview

Nishanthan — Mon, 01 Dec 2025 03:30:00 +0000

Neural networks are at the heart of modern AI. Whether it’s image recognition, chatbots, or recommendation systems, they all rely on these interconnected structures to learn patterns from data.

What is a Neural Network?

A neural network is made up of many interconnected nodes (neurons), each acting like a tiny decision-making unit.
Every connection between neurons carries a weight, which is a parameter the model learns during training.
These weights determine how strongly one neuron influences another, and they are gradually adjusted so the network can make better predictions over time.

How Are the Values Estimated?

When training begins, each neuron starts with random weights.
These weights are updated through backpropagation, a feedback process that adjusts them based on how wrong the model’s prediction was.
The update depends on:
- Loss value: how far the prediction was from correct
- Gradient: the direction the weight should move
- Learning rate: how big each step of adjustment should be
The goal is to reduce the loss and push the gradient closer to zero over time.
Interpreting the gradient:
- Positive gradient → decrease the weight
- Negative gradient → increase the weight

Why Neural Networks?

Traditional machine learning works well when data patterns are linear, meaning they follow straight-line relationships.
Neural networks shine because they use activation functions to introduce non-linearity, enabling them to learn complex patterns — shapes, edges, behaviors, and any relationship that isn’t simple or straight.
Popular activation functions include ReLU, Sigmoid, and Softplus, each influencing how a neuron activates.

Layers in a Neural Network

Input Layer

The network begins with an input layer containing multiple nodes.
These nodes don’t learn anything — they simply pass the raw input into the network.
They are often mistaken for neurons, but their job is mainly to feed data forward.
- They take the raw data.
- They pass those numbers into the network.
- They do not apply weights, biases, or activations.
Because of this, these nodes are often mistaken for “neurons” but they’re better understood as data carriers — a doorway into the network rather than processors.

Hidden Layers

A network may have one or several hidden layers, each filled with neurons.
These are the actual learning units, transforming the input step by step to capture patterns in the data.
- It receives input from the previous layer
- Applies weights and biases
- Passes the result through an activation function
- And sends the output deeper into the network
This repeated transformation allows the model to gradually extract patterns — edges in images, sentiment in text, or trends in numerical data.
Layer by layer, the network builds up a more meaningful internal representation of the input.

Output Layer

The final layer, responsible for producing the network’s output — a prediction, probability, class label, etc.
Depending on the problem, this layer may output:
- A single number
- A vector of probabilities
- A class label
- A multi-dimensional prediction
Like hidden neurons, output neurons also use weights, biases, and sometimes an activation function (e.g., sigmoid or softmax).
The only difference is that this layer’s output is directly interpreted as the model’s answer.

The 5-Step Life Cycle of a Neuron (Applies to Every Layer Except Input)

Every neuron inside a neural network — whether in a hidden layer or the output layer — follows the exact same processing routine. Understanding this cycle is key to building intuition.

1. It receives inputs (activations).

These activations come from either the input layer or the previous hidden layer.

2. It multiplies each input by its corresponding weight.

Weights determine how strongly the neuron should pay attention to each incoming value.

3. It adds a bias.

The bias shifts the weighted sum, giving the neuron flexibility to activate even when inputs are small or zero.

4. It applies an activation function.

The weighted sum plus bias (called the pre-activation, or z) goes through a function like ReLU, sigmoid, or tanh.
This step introduces non-linearity — essential for learning complex patterns.

5. It outputs a new activation.

This final value (a) becomes the input to the next layer, allowing the network to stack transformations and build rich representations.

Final Thoughts

A neural network is essentially a repeated cycle of: input → weighted sum → bias → activation → output layer after layer, until the final prediction is produced.

This is a gentle introduction to neural networks — enough to understand what they are made of and how they learn. From here, you can explore topics like activation functions, optimizers, loss functions, and different neural architectures.

How Neural Networks Learn – A Simple Guide to Machine Learning & Deep Learning

Nishanthan — Fri, 28 Nov 2025 04:30:00 +0000

Machine Learning and Deep Learning are often treated as black boxes filled with complex math and jargon. But at their core, they are built on a few simple ideas: measuring error, understanding direction, and making small improvements over time.

In this article, I break down how a model actually learns — from the moment it makes a mistake, to how that mistake travels backward through the network to update weights and improve future predictions. Starting from a simple equation, we’ll build up to neural networks and the complete training loop, step by step.

1. Intelligence, Patterns, and Models

Human intelligence works by building mental models of the world.

Example:

Black clouds + strong wind → We expect rain
Sometimes this is wrong (it could be a cloud shadow)

Over time, the brain refines these models.

Machine Learning works the same way:

Input → Pattern → Output
The system learns relationships from data, not rules written manually.

A model is simply a mathematical way of representing this pattern.

2. The Core Idea: Model as an Equation

The simplest model looks like this:

y = mX + b

Where:
X = input
y = output
m and b = parameters (values to be learned)

In Machine Learning, this is often written as:

y = wX + b

Here:
w = weight (importance of input)
b = bias (base value added to the output)

The entire goal of training is to find the best values of w and b so the prediction matches reality.

3. Loss Function — Measuring Mistakes

Once the model makes a prediction, it needs to know: “How wrong am I?” That’s the job of the loss function.

A loss function:

Compares actual output and predicted output
Returns a number that shows how bad the prediction was
High loss → very wrong
Low loss → close to correct

This number becomes the main guiding signal for learning.

4. Gradient — Finding the Direction of Improvement

After loss is calculated, the model asks: “In which direction should I change my parameters to reduce this error?” The gradient is that direction.

Intuitively:

A positive gradient means: if we increase this parameter, loss will go up
A negative gradient means: if we increase this parameter, loss will go down
A zero gradient means: we are at a flat point (often a minimum)

5. Learning Rate — Controlling the Speed of Learning

The learning rate controls how much the parameter is allowed to change in each step.

Conceptually, the update rule is:

new_weight = old_weight - (learning_rate × gradient)

If the learning rate is too large → the model jumps wildly, overshoots, and becomes unstable
If the learning rate is too small → the model moves very slowly and takes forever to learn
If the learning rate is reasonable → learning is smooth and stable

So the learning rate is simply the step size of learning.

6. Epoch — Repeating the Learning Process

One epoch means the model has seen the entire training dataset once.
During an epoch, the model would makes predictions on all training data
- Calculates the loss
- Updates the parameters
- Repeats this process for the next epoch
More epochs → more opportunities to learn
Too many epochs → the model may start memorizing instead of generalizing (this is called overfitting)

7. Batch — Dividing the Dataset

Instead of feeding the entire dataset to the model at once, we split it into batches.

Example:

Dataset size: 1000 samples
Batch size: 100
Then per epoch, we have: 1000 / 100 = 10 batches

So the model updates its parameters 10 times in one epoch

Benefits of batching:

Reduces memory usage
Gives more frequent parameter updates
Often leads to better and more stable learning
There is no single “perfect” batch size. It depends on:
- Dataset size
- Available hardware (CPU/GPU, RAM/VRAM)
- Model size and complexity

8. Weights and Bias — Making Real-World Sense

Earlier, we saw:

y = mX + b

In ML terms:

y = wX + b

Where:

w (weight) represents how important an input is
b (bias) is a base value added regardless of the input

Example: Courier Charges

₹100 per kg → weight
₹50 base charge → bias
Even if the parcel weighs 0 kg, you still pay ₹50.
Without the bias term, the model would be forced to predict ₹0 when the input is 0. Bias lets us shift the line up or down and represent real “base” behavior.

So bias is needed so that:

The model is not forced to pass through (0, 0), and can better fit real-world data.

9. Neural Network — From One Equation to Many

A single neuron performs a calculation like:

y = w1·x1 + w2·x2 + ... + wn·xn + b

- It takes many inputs
- Multiplies each by its own weight
- Adds them up and adds a bias

A layer is a group of neurons working in parallel on the same input.
A neural network is a stack of such layers.

Why use neural networks?

A single linear equation is limited to straight-line relationships
Neural networks, combined with activation functions, can model complex, non-linear patterns
This is what enables them to handle images, text, audio, and other complex data

10. Activation Function — The Source of Non-Linearity

After a neuron computes:

z = w·x + b

We apply an activation function to get the final output:

a = activation(z)

Activation functions:

Introduce non-linearity
Allow the network to learn complex patterns and decision boundaries
Enable tasks such as:
- Image classification
- Object detection
- Natural language processing

Without activation functions:

Every neuron is just a linear transformation
Stacking multiple linear layers still gives a linear function
The network would be no more powerful than a single linear model

In short:

Activation functions are what make neural networks more powerful than simple linear models.

11. Forward Propagation — Data Moving Through the Network

Forward propagation is the process of passing data through the network to produce an output.
Steps:
- Input data is fed into the input layer
- Each neuron in the next layer:
- Multiplies inputs by weights
- Adds bias
- Applies activation function
The output of one layer becomes the input to the next layer
This continues until the final output layer produces the prediction
Flow: Input → Hidden layer 1 → Hidden layer 2 → ... → Output layer

Important:

Forward propagation only computes the outputs.
No learning or weight updates happen in this step.

12. Backward Propagation — Learning From Mistakes

After forward propagation, we:
- Compare the prediction with the actual target
- Compute the loss (error)
- Now the model needs to adjust its weights to reduce this loss.
- This is where backward propagation (backprop) comes in.
Backpropagation:
- Sends the error signal backward from the output layer toward the input layer
- Computes, for each weight and bias, how much it contributed to the error (its gradient)
- Once gradients are known, we update parameters using gradient descent:
  - new_weight = old_weight - (learning_rate × gradient)
  - new_bias = old_bias - (learning_rate × gradient)
- So:
  - Backpropagation calculates gradients;
  - Gradient descent + learning rate perform the actual update.
- Backpropagation is the core of how the network learns.

13. The Complete Training Cycle

Putting everything together, a full training loop for a neural network looks like this:

Input data is passed into the network
Forward propagation computes the prediction
Loss function measures how wrong the prediction
Backward propagation computes gradients for all weights and biases
Gradient descent updates the parameters using the learning rate
This is repeated for:
- Every batch
- Across multiple epochs

In one line:

Input → Forward Propagation → Loss → Backpropagation → Update Weights → Repeat

With each cycle:
- The model becomes a bit less wrong
- The loss usually goes down
- The predictions improve
- This is the entire training process in a nutshell.

14. Inference — Using the Trained Model

Once the model is trained:
- We freeze the learned weights and biases
- We give it new input data it has never seen
- It runs only forward propagation to produce predictions
- No backpropagation or weight updates happen during inference.
This is the phase where the model is actually used in production:
- Recommending products
- Classifying images
- Translating text
- Detecting fraud, etc.

Final Thought

From a simple line like y = wX + b to deep neural networks, everything in Machine Learning and Deep Learning is built on a few core ideas:

- Representing patterns with models
- Measuring error with a loss function
- Using gradients to find a better direction
- Controlling change with learning rate
- Repeating the process over batches and epochs

The concepts are simple.
The power comes from scale, data, and iteration.

Understanding Java Constructors: Types, Execution, and Differences from Methods

Nishanthan — Sun, 05 Oct 2025 18:30:00 +0000

Understanding Constructors in Java

A constructor in Java is similar to a method but serves a specific purpose: initializing objects. This article provides a deep dive into constructors, their execution, types, and how they differ from regular methods.

What is a Constructor?

Same name as the class
No return type (not even void)
Primary role: Assign or initialize values to class variables
Automatic invocation: Called automatically when an object is created; cannot be called like a regular method

Execution of Constructors

Compile Time

The Java compiler converts a constructor into a special method called <init> in the .class file.
JVM maintains two special methods for initialization:
- <init> → Contains instance constructors; not callable directly by user code
- <clinit> → Contains static field initializations and static blocks; executed once when the class is loaded
If no constructor is defined by the developer, Java automatically creates a default constructor and stores it in <init>.

Runtime

In the JVM, a constructor is represented as a special method (<init>) and stored once in the Method Area.
Every time an object is created, the JVM executes this stored constructor; no new constructor code is created per object.
Constructors are primarily for initializing object state; heavy business logic should generally be avoided.

Types of Constructors

Default Constructor
- Added automatically by Java when no constructor is defined.
No-Argument Constructor
- Explicitly written by the developer without parameters.
Parameterized Constructor
- Written with parameters to initialize object fields flexibly.
Copy Constructor
- Not built into Java by default, but can be created by the developer to copy values from another object.

// Default constructor (auto-generated by Java)
class A {}

// No-arg constructor
class B {
    B() {
        System.out.println("B created");
    }
}

// Parameterized constructor
class C {
    String name;
    C(String n) {
        name = n;
    }
}

// Copy constructor
class D {
    String name;
    D(D other) {
        this.name = other.name;
    }
}

Constructor vs Method in Java

Feature	Constructor	Method
Name	Must be same as the class	Can have any legal identifier
Return Type	None (not even `void`)	Must declare a return type (or `void`)
Invocation	Called automatically with `new`	Must be explicitly invoked (`obj.method()`)
Purpose	Initialize object state (fresh object setup)	Define behavior of an already created object
Bytecode Representation	Stored as `<init>` in `.class` file	Stored with its given name
Inheritance Behavior	Not inherited, but calls parent constructor (`super()`)	Inherited and can be overridden
Overloading	Can be overloaded	Can be overloaded and overridden
Special Rules	Can call another constructor (`this(...)` or `super(...)`) as first statement	No such restriction

Summary

Constructor → Object birth + initialization, JVM-reserved mechanism (<init>)
Method → Object behavior, regular callable unit of code

Constructors are foundational to object-oriented programming in Java, ensuring objects are properly initialized while maintaining a clean separation of initialization logic from business logic.

Brushing Up on React Hooks

Nishanthan — Mon, 28 Jul 2025 03:33:01 +0000

React Hooks Summary

1. `useState`

memoizedState holds the current state value.
baseState is the initial state value.
dispatch is the setter function to update state.
Triggers re-render after the current call stack is empty.

2. `useEffect`

Allows you to perform side effects (API calls, subscriptions).
Cleanup is handled by returning a function inside the effect.
Cleanup runs:
- Before the component unmounts.
- Before the effect runs again (if dependencies change).

3. `useMemo`

Caches the result of a computation.
Prevents recalculating expensive operations unless dependencies change.
Useful for:
- Filtering, sorting, parsing large data.
- Memoizing derived data for performance.
Can also be used to wrap child components to prevent unnecessary re-renders (in conjunction with React.memo).

4. `React.memo` vs `useMemo`

React.memo: Prevents re-rendering a component unless props change.
useMemo: Prevents re-running a function unless dependencies change.

5. `useCallback`

Memoizes the function definition itself.
Useful when passing callbacks to memoized child components.
Prevents unnecessary re-renders due to new function references on each render.

6. `useRef`

Mutable ref object that persists for the full lifetime of the component.
Doesn't trigger re-renders.
Can be used to access DOM nodes or hold mutable variables.

7. `useReducer`

Alternative to useState for complex state logic.
Accepts a reducer function and an initial state.
Returns [state, dispatch].
Best for managing state transitions or multiple sub-values.

8. `useLayoutEffect`

Like useEffect, but runs after DOM mutation, before paint.
Use for DOM measurements, layout adjustments.
Main use case: e.g., sidebar collapse before layout shift.

9. `useImperativeHandle`

Customizes the instance value exposed by ref.
Used with forwardRef.
Allows parent to trigger child functions imperatively.
Use case: trigger child API call on parent submit button click.

10. `useDebugValue`

Used for debugging custom hooks in React DevTools.
Display custom hook values to help debugging.

How Java Stores Data in Memory: Writing Efficient Code with Primitives and Non-Primitives

Nishanthan — Fri, 16 May 2025 03:30:00 +0000

How Java Stores Data in Memory — Primitives vs Non-Primitives

Java is a statically typed language, which means every variable’s type must be known at compile time. This allows the JVM (Java Virtual Machine) to allocate memory efficiently and enforce type safety during execution.

Primitive Types in Java

Primitive types are the building blocks of data in Java. They have fixed memory sizes and are stored in the stack. The value of a primitive can change, but its size does not.

Type	Size	Description
`byte`	1 byte	Smallest integer (–128 to 127)
`short`	2 bytes	Short integer
`int`	4 bytes	Default integer type
`long`	8 bytes	Large-range integer
`float`	4 bytes	Single-precision float
`double`	8 bytes	Double-precision float
`char`	2 bytes	Unicode character
`boolean`	1 bit*	True or False (handled as 1 byte internally)

Note: Although boolean is conceptually 1 bit, it is usually stored as **1 byte* in memory due to alignment and practical implementation constraints.*

Non-Primitive (Reference) Types

Non-primitives, also called reference types, are more complex and stored in the heap, while their references (pointers) are kept in the stack. These types include:

Type	Description
`String`	Sequence of characters (immutable)
`Arrays`	Ordered collection of elements (of any type, including primitives)
`Class`	Custom-defined objects
`Interface`	Abstract types that other classes implement
`Enum`	A fixed set of constants (internally class-like)
`Wrapper`	Object representations of primitive types (e.g., `Integer`, `Double`)
`Collections`	Data structures like `List`, `Set`, `Map`, etc. (part of Java Collections Framework)

How Java Stores Data in Memory

Java abstracts memory management with automatic garbage collection, but here’s how it works under the hood:

Stack Memory

Stores method calls, local variables, and primitive types.
Fast access due to LIFO structure.
Memory is freed automatically once a method exits.

Heap Memory

Stores objects and non-primitive values.
Accessed via references stored in the stack.
Managed by Garbage Collector (GC).
Larger and slower than stack but can store dynamic and complex structures.

How Primitive Types Are Accessed

Stored directly in the stack.
Access and modification are fast and straightforward.
Since they are fixed-size, the JVM knows exactly how much memory to allocate and retrieve.

How Non-Primitives Are Accessed

When a non-primitive is created, the reference (memory address) is stored in the stack.
The actual object is stored in the heap.
Accessing the object involves following the reference from stack to heap.

Example:

String name = "John";

name (stack) → points to → "John" (heap)

If we change name = "Jack";, now name points to a new "Jack" in heap (because String is immutable).

Memory Alignment and JVM Optimization

Java is designed with the principle: “Write once, run anywhere”, which means JVM must work on both 32-bit and 64-bit CPUs.

JVM Alignment Strategy:

Regardless of declared type, JVM aligns memory into 4-byte or 8-byte chunks depending on CPU.
Operations on smaller types (byte, short) are converted to int internally.

Why is that?

byte takes 25% of a 4-byte block; short takes 50%.
CPU needs extra steps to calculate offsets and align the memory block.
int consumes the entire 4-byte block, making storage and retrieval faster.

This is why:

JVM promotes byte and short to int during calculations and requires explicit casting to store results back.

Misconception: Using `byte`/`short` Saves Memory?

Why `byte` and `short` are less efficient:

JVM treats them as int internally.
Adds extra overhead during arithmetic and access.
Introduces hidden conversions (widening and narrowing).
Slower due to memory misalignment and CPU inefficiency.

For performance and alignment, use int as default for integers.

What About Wrapper Classes?

Wrapper classes (Integer, Double, etc.) are object versions of primitive types.

Stored in heap like any object.
Used in Collections (e.g., List<Integer>, not List<int>)
Include utility methods like parseInt, compareTo, etc.

Note: Auto-boxing and unboxing happen automatically:

int a = 10;
Integer b = a; // auto-boxing
int c = b;     // unboxing

Boxing adds memory and performance overhead, so avoid unnecessary boxing in critical code paths.

Key Takeaways

Primitive types are fixed-size and fast, stored in stack.
Non-primitives are stored in heap with reference in stack.
JVM uses int as the baseline for all smaller integral operations.
Avoid using byte and short for optimization, it backfires due to hidden processing.
Wrapper classes add object overhead; prefer primitives for performance-critical paths.
Java abstracts memory handling, but understanding this helps you write more efficient code.

Understanding Angular Rendering, Re-rendering, and Change Detection for Optimal Performance

Nishanthan — Mon, 07 Apr 2025 04:30:00 +0000

Introduction

If you’re transitioning from React to Angular or starting with Angular as first step, understanding Angular’s rendering process, change detection mechanism, and re-rendering strategy is crucial for writing efficient and high-performance applications. Unlike React, where the virtual DOM helps optimize the re-render process, Angular’s change detection mechanism checks every component to ensure the UI reflects the latest data. This article will break down the concepts of rendering, change detection, and performance optimization in Angular, answering the questions you might have along the way.

Understanding Angular Rendering and Re-rendering

How Does Angular Render and Re-render Components?

Angular, unlike React, uses a real DOM and checks for changes during change detection. Let's break down the process:

Initial Rendering: When a component is created for the first time, Angular processes the component’s template and binds the data from the component to the template.
Change Detection: Once the component is initialized, Angular listens for changes in the data. When any data-bound property changes, Angular triggers the change detection process, checking the whole component tree from the root.
Re-rendering: When data in a component changes, Angular goes through the change detection cycle again. Unlike React, which uses a virtual DOM, Angular re-renders the DOM directly.

This brings up an interesting question: If change detection checks the whole component tree, does Angular re-render everything even if only a small part of the data changes?

Do Angular Re-renders Everything or Just the Changed Part?

Angular does indeed go through all bindings in the component, even if only a single property changes. This is less efficient compared to React’s virtual DOM diffing mechanism, where only the changed elements are re-rendered.
However, Angular uses optimizations such as OnPush change detection strategy and TrackBy in *ngFor to reduce unnecessary checks. Let's dive deeper into these strategies.

Exploring Angular's Change Detection Mechanism

How Does Change Detection Work in Angular?

Change detection in Angular follows a process where Angular traverses the component tree and checks each binding to see if it has changed. If any change is detected, Angular updates the DOM.

The default strategy is CheckAlways, which checks all components in the component tree. But Angular allows you to optimize this using OnPush.

What is OnPush?

When using OnPush change detection, Angular only checks a component for changes when:

One of its Input properties changes.
An event is triggered (e.g., button click).
An observable emits a new value.

This can significantly improve performance in large applications by avoiding unnecessary checks on components that haven’t changed.

@Component({
  selector: 'app-my-component',
  changeDetection: ChangeDetectionStrategy.OnPush,
  templateUrl: './my-component.component.html',
})
export class MyComponent {
  @Input() data: any;
}

By using OnPush, you restrict the component to be checked only when a relevant change occurs, rather than checking it on every cycle.

Why Does Angular Check All Components for Changes?

You might wonder, Why does Angular check the entire component tree? Can it just check the component where the change happened?
Angular’s default change detection works by traversing the whole component tree to check for any data changes. This design ensures that all bindings are up to date, but can become inefficient, especially in large applications.
Unlike React’s virtual DOM, Angular doesn’t maintain a previous DOM snapshot, and so it doesn't have an efficient diffing mechanism that would allow it to detect changes at a granular level.

The Challenge with Functions in Templates

Angular has a two-way binding mechanism, which helps with reactivity. But one performance pitfall is calling functions directly in templates. Functions in templates are re-evaluated on every change detection cycle, which can hurt performance if not used properly.

For example, calling a function in a template like this:

{{ computeValue() }}

Could cause the function to run repeatedly, making it inefficient. To avoid this, use getters or Signals (in Angular 16.2+) to track state more efficiently.

get computedValue(): number {
  return this.data * 10;
}

This way, the value is calculated once, and Angular can optimize re-renders.

Optimizing Angular for Better Performance

To address performance concerns, let’s talk about strategies to optimize rendering and change detection in Angular.

Use OnPush Change Detection

As mentioned earlier, the OnPush strategy significantly reduces unnecessary re-renders. It tells Angular to check the component only when its inputs or observables change, or when an event is triggered.

Use trackBy with *ngFor

When using *ngFor, Angular creates new DOM elements for every item in the list. If the list is large, this can become inefficient. You can optimize the rendering of lists by using trackBy to track which items have changed, rather than recreating the entire DOM for the list every time.

<div *ngFor="let item of items; trackBy: trackById">
  {{ item.name }}
</div>

trackById(index: number, item: any): number {
  return item.id;
}

Avoid Functions in Templates

Instead of calling functions directly in the template, use getters or Signals (in Angular 16.2+) to keep state changes efficient and avoid recomputing values on every change detection cycle.

Signals (Angular 16.2+):

Signals in Angular provide a more efficient way to manage state and trigger updates. With signals, Angular tracks the state and updates the DOM only when the value of the signal changes.

isA = signal(true);
isB = signal(false);

Use ngOnChanges and Input Setters For efficient handling of input changes, Angular provides Input Setters and ngOnChanges. Instead of checking changes manually, use these lifecycle hooks to optimize reactivity.

@Input() set data(value: any) {
  this._data = value;
}

You can use this instead of ngOnChanges to efficiently handle changes in input properties.

Conclusion

Understanding Angular’s rendering, re-rendering, and change detection mechanisms is key to building efficient applications. By using techniques like OnPush change detection, trackBy in *ngFor, avoiding functions in templates, and leveraging Signals (in Angular 16.2+), you can optimize your Angular applications for better performance.

Remember, while Angular doesn’t use a virtual DOM like React, it offers a powerful change detection system that you can fine-tune for optimal performance. Implement these strategies and take control of your app’s reactivity—leading to faster, smoother, and more efficient user experiences.

How MongoDB’s ObjectID and B-Tree Search Make Your Queries Super Fast

Nishanthan — Wed, 19 Feb 2025 03:30:00 +0000

MongoDB uses a unique identifier, called ObjectId, for each document stored in its database. An ObjectId is a 12-byte identifier that is generated in a specific format to ensure it’s unique across distributed systems.

In this article, we will break down the structure of MongoDB's ObjectId, how it is generated, and how MongoDB uses a binary tree-like structure for indexing and searching. We will also discuss how this structure helps MongoDB perform searches efficiently, even with millions of entries.

Structure of MongoDB's ObjectId

MongoDB’s ObjectId is a 12-byte identifier, and its structure is designed to ensure uniqueness while keeping it compact. The 12 bytes are divided into four main parts:

1. Timestamp (4 bytes)

The first 4 bytes represent the timestamp when the ObjectId was generated.
This timestamp is recorded in seconds since the Unix epoch (January 1, 1970).
This ensures that ObjectIds are roughly ordered by creation time.

2. Machine Id (3 bytes)

The next 3 bytes represent a unique identifier for the machine where the ObjectId was generated.
Typically, it’s derived from the machine's IP address or a hash of the machine's network interfaces.
This will be constant for its lifetime untill the hostname and other system details got changed
This ensures that even if two MongoDB instances are running on different machines, they can generate unique ObjectIds.

3. Process Id (2 bytes)

The following 2 bytes represent the process ID of the MongoDB server process.
This helps to distinguish ObjectIds generated by different processes running on the same machine.
This is only constant for that run and it will change when the server was restarted / on re-run.

4. Counter (3 bytes)

The last 3 bytes represent an incrementing counter.
The counter starts at a random value. If multiple documents / objectIds are generated at the same timestamp (same second), the counter ensures each generated ObjectId is unique by incrementing.
If only one Document / ObjectId is generated per second, the counter will reset to another random value for the next second.
Key point: This counter ensures uniqueness when multiple ObjectId values are generated in the same second by the same machine and process.

Example of an ObjectId:

507f191e810c19729de860ea

The first 4 bytes: 507f191e represent the timestamp.
The next 3 bytes: 81 represent the machine ID.
The next 2 bytes: 0c represent the process ID.
The last 3 bytes: 19729de represent the incrementing counter.

MongoDB Index Structure and Page Layout:

MongoDB uses a B-tree-like index structure, not an real tree we are using in data structures but with certain modifications for optimization. Here's a breakdown:

1. Root Node:

The root node of the index tree stores keys (usually indexed fields like ObjectId, or other indexed fields).
This root node acts as the starting point for the search process, guiding MongoDB towards the right child node.
Key Features:
- Contains sorted keys.
- Points to child nodes which continue the search deeper into the tree.

2. Child Pages:

Child nodes (or child pages) store sorted keys and pointers to their leaf nodes or further child nodes.
These nodes are like intermediates, helping narrow down the range of values that need to be searched.
Key Features:
- Contains sorted key-value pairs.
- Contains pointers to the next level (further child pages or leaf pages).
- May be created or split as the index grows.

3. Leaf Pages:

The leaf pages are the actual locations where document references (pointers to documents) are stored.
Each entry in the leaf page contains a key (indexed field value) and a pointer to the actual document (i.e., location in the data store).
These pages are where the final results are stored, which MongoDB returns in a query.
Key Features:
- Contains the actual document references.
- Sorted by the indexed field for fast lookups.

Page Structure:

In MongoDB, index pages are typically designed to be 4 KB in size. If the page contains entries like ObjectId (a 12-byte identifier), each page can store around 317 entries. This size may vary slightly based on the size of indexed fields.

Pages are fetched from the disk into memory during searches, ensuring that queries are handled efficiently.
MongoDB uses a binary search within a page to locate the exact entry, which ensures faster retrieval of documents.

How MongoDB Manages Page Growth:

As MongoDB adds more entries to an index, it needs to split nodes to ensure that the index remains balanced.

When a page reaches its capacity, MongoDB splits the page into two and promotes the middle key to the parent node.
The middle key is the key that divides the entries in half and is used to maintain a balanced index.
As the index continues to grow, this process of splitting and promoting ensures the B-tree remains balanced.
Balancing the tree through splits and promotions ensures that search operations remain efficient and that the index is optimal for query performance.

Example of How the Index Tree Works:

Consider an index on the ObjectId field in MongoDB:

Root Node: Contains keys pointing to child nodes. For example, if you have a collection of documents with ObjectId, the root node will store keys like ObjectId:123, ObjectId:124, etc., with pointers to child nodes that contain more detailed keys.
Child Pages: These child nodes will store ranges of ObjectIds and pointers to further nodes or leaf nodes, helping narrow down the search.
Leaf Pages: Finally, the leaf pages contain the actual document pointers for specific ObjectIds. For example, a leaf page might contain a key like ObjectId:123 and a pointer to the exact location of the document in the storage.

When a query is executed, MongoDB will traverse this tree structure by following the key pointers from the root node, through child nodes, and ultimately to the leaf nodes where it finds the document references. This ensures efficient searching, even with large datasets.

Key Takeaways:

MongoDB uses a modified B-tree index structure for fast lookups.
Pages are crucial for holding index entries and documents. Pages are typically 4 KB in size and contain sorted entries.
Leaf pages contain the actual document references and are where the search ultimately lands.
As the tree grows, node splits and middle key promotions ensure that the index remains balanced and efficient for query processing.

Example: From Scratch to 17 Entries

Let’s consider the process of adding entries to a MongoDB index and how the keys move up in the tree.

Step-by-Step Tree Construction:

Insert 1, 2, 3: Initially, we add three entries, and they all fit in the root node.

[1, 2, 3]

Insert 4: Adding the fourth entry causes the root to split. The middle key (2) gets promoted to a new root.

Root: [2]
Left Child: [1]
Right Child: [3, 4]

Insert 5, 6, 7: The right child now overflows, so the middle key (6) moves up.

Root: [2, 6]
Left Child: [1]
Middle Child: [3, 4, 5]
Right Child: [7]

Insert 8, 9, 10: The right child overflows again, so the middle key (9) moves up.

Root: [2, 6, 9]
Left Child: [1]
Middle Left Child: [3, 4, 5]
Middle Right Child: [7, 8]
Right Child: [10]

Insert 11, 12, 13, 14, 15, 16, 17: We continue inserting entries, and they cause further splits, with the middle keys getting promoted up.

Root: [2, 6, 9, 13]

Left Child: [1]

Middle Left Child: [3, 4, 5]

Middle Child: [7, 8]

Right Child: [10, 11]

Final Child: [12, 14, 15, 16, 17]

Final tree strucutre

                         [6, 9, 13]
                      /    |    |    \
         [1, 2, 3, 4, 5]   [7, 8]  [10, 11]  [12, 14, 15, 16, 17]

Simplified MongoDB Search Process

Here’s a simplified summary of how MongoDB searches for documents using the index:

Start with the Index:
- MongoDB first looks at the index (similar to a table of contents in a book) to figure out where to find documents.
- The index is stored in a B-tree structure, which helps MongoDB quickly find the right spot.
Navigate the Tree:
- The root node of the tree holds pointers to the next pages (child nodes).
- The tree is split into pages, each containing key-value pairs. These keys are the indexed fields (like ObjectId, or any other field you're indexing).
Find the Relevant Range:
- When a query is made, MongoDB starts at the root node and uses binary search to find the right range of keys.
- It keeps following the pointers to the next pages until it narrows down to the correct range of keys for the search.
Determine the Page to Search:
- As MongoDB navigates the tree, it looks for the page number (stored in the index) that contains the matching _id or document key range.
- Once it finds the correct page, it jumps to that page.
Search on the Page:
- The page contains a binary search of documents stored in it. MongoDB uses the pointers in the page to go straight to the document's storage location (like the exact spot where the document is stored on disk).
Fetch the Document:
- Finally, MongoDB uses the storage location found in the index (the position pointer) to go directly to the document on the disk and fetch it.

In short:

MongoDB first uses the index (a tree structure) to find the right range of documents.
It uses binary search to find the correct page.
Then, it jumps to that page and finds the document by looking up the position pointer, directly fetching it from the disk.

This process ensures that MongoDB can quickly locate and fetch documents even if the database grows large.

Real strucuture with 64 level split

                           ┌────────────────── Root Node ───────────────────┐
                           │ Keys: [512, 1024, 1536]                         │
                           │ Child Pointers: [P1, P2, P3, P4]                │
                           └────────────────────────────────────────────────┘
                                         ↓ Choose correct child based on _id

 ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
 │                         Internal Nodes (Level 2)                                                              │
 └───────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
 │ P1 → Keys: [128, 256, 384]  │ P2 → Keys: [640, 768, 896]  │ P3 → Keys: [1152, 1280, 1408]  │ P4 → Keys: [1664, 1792, 1920] │
 │ Child Pointers: [P5-P8]     │ Child Pointers: [P9-P12]    │ Child Pointers: [P13-P16]     │ Child Pointers: [P17-P20]    │

                                         ↓ Choose correct child based on _id

 ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
 │                         Internal Nodes (Level 3)                                                              │
 └───────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
 │ P5 → Keys: [32, 64, 96]  │ P6 → Keys: [160, 192, 224]  │ P7 → Keys: [288, 320, 352]  │ P8 → Keys: [416, 448, 480]  │
 │ P9 → Keys: [544, 576, 608]  │ P10 → Keys: [672, 704, 736]  │ P11 → Keys: [800, 832, 864]  │ P12 → Keys: [928, 960, 992]  │
 │ P13 → Keys: [1056, 1088, 1120]  │ P14 → Keys: [1184, 1216, 1248]  │ P15 → Keys: [1312, 1344, 1376]  │ P16 → Keys: [1440, 1472, 1504]  │
 │ P17 → Keys: [1568, 1600, 1632]  │ P18 → Keys: [1728, 1760, 1792]  │ P19 → Keys: [1856, 1888, 1920]  │ P20 → Keys: [1984, 2000]  │

                                         ↓ Choose correct child based on _id

 ┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
 │                         Leaf Nodes (Level 4) - Actual Document Pointers                                       │
 └───────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
 │ P21 → Keys: [1-64] (D1-D64)  │ P22 → Keys: [65-128] (D65-D128)  │ P23 → Keys: [129-192] (D129-D192)  │ P24 → Keys: [193-256] (D193-D256)  │
 │ P25 → Keys: [257-320] (D257-D320)  │ P26 → Keys: [321-384] (D321-D384)  │ P27 → Keys: [385-448] (D385-D448)  │ P28 → Keys: [449-512] (D449-D512)  │
 │ P29 → Keys: [513-576] (D513-D576)  │ P30 → Keys: [577-640] (D577-D640)  │ P31 → Keys: [641-704] (D641-D704)  │ P32 → Keys: [705-768] (D705-D768)  │
 │ P33 → Keys: [769-832] (D769-D832)  │ P34 → Keys: [833-896] (D833-D896)  │ P35 → Keys: [897-960] (D897-D960)  │ P36 → Keys: [961-1024] (D961-D1024)  │
 │ P37 → Keys: [1025-1088] (D1025-D1088)  │ P38 → Keys: [1089-1152] (D1089-D1152)  │ P39 → Keys: [1153-1216] (D1153-D1216)  │ P40 → Keys: [1217-1280] (D1217-D1280)  │
 │ P41 → Keys: [1281-1344] (D1281-D1344)  │ P42 → Keys: [1345-1408] (D1345-D1408)  │ P43 → Keys: [1409-1472] (D1409-D1472)  │ P44 → Keys: [1473-1536] (D1473-D1536)  │
 │ P45 → Keys: [1537-1600] (D1537-D1600)  │ P46 → Keys: [1601-1664] (D1601-D1664)  │ P47 → Keys: [1665-1728] (D1665-D1728)  │ P48 → Keys: [1729-1792] (D1729-D1792)  │
 │ P49 → Keys: [1793-1856] (D1793-D1856)  │ P50 → Keys: [1857-1920] (D1857-D1920)  │ P51 → Keys: [1921-1984] (D1921-D1984)  │ P52 → Keys: [1985-2000] (D1985-D2000)  │

Search flow for _id 1754

Root Node: _id = 1754 falls in range [1536, 2000], follow P4
Internal Node (L2): _id = 1754 falls in range [1664, 1920], follow P18
Internal Node (L3): _id = 1754 falls in range [1728, 1792], follow P48
Leaf Node (L4): _id = 1754 is found, get Document Pointer D1754
Retrieve document from storage

Searching for an ID in MongoDB (Example with 3.6 Million Entries)

Let’s say we have a collection of 3.6 million entries and you need to find an entry with a specific ObjectId. Here’s how MongoDB would perform the search.
At each level, you have 64 choices (64 child pointers).
So for large values (like the 3.7 millionth record), this structure is efficient.

import math

total_entries = 3600000

branching_factor = 64

num_searches = math.ceil(math.log(total_entries, branching_factor))
print(num_searches)

For searching the 3.6 millionth entry, it would take approximately 4 searches (or 4 levels of traversal in the tree). This is because the B-tree structure narrows down the search space significantly at each level.

Conclusion

MongoDB’s ObjectId provides a compact, efficient way to uniquely identify documents, and its indexing system uses a B-tree-like structure to allow for fast searches. As documents are inserted, keys are promoted to parent nodes, ensuring that MongoDB maintains a balanced and efficient index structure. When searching for a specific ObjectId, MongoDB can quickly narrow down the search space using binary search on the root and child pages, resulting in fast lookups even for large collections.

By combining timestamp-based generation for ObjectIds and a binary search-based indexing structure, MongoDB is able to offer high performance for both data storage and retrieval.

Understanding the Java Execution Process: From Code to Execution

Nishanthan — Thu, 30 Jan 2025 04:00:00 +0000

Java is widely known for its platform independence and efficient execution. This article will walk you through the entire Java execution process, from writing human-readable code to running it across different platforms. We’ll cover the roles of JDK, JVM, and JRE, as well as the steps involved in compiling and executing Java programs.

1. Key Java Components

Before diving into the execution process, it’s essential to understand the three core components in Java:

JDK (Java Development Kit)
- The JDK is a full-fledged software development kit that allows you to write, compile, and execute Java programs.
- It includes the JVM (Java Virtual Machine) and JRE (Java Runtime Environment), as well as essential tools for development, such as the compiler (javac).
- JDK is used by developers to write and compile code, which is later executed by the JVM.
JVM (Java Virtual Machine)
- The JVM is the engine that runs Java bytecode. It makes Java platform-independent by abstracting the underlying hardware and operating system.
- Java programs are compiled into bytecode, which the JVM interprets and executes, allowing the same Java program to run on any machine with a JVM.
JRE (Java Runtime Environment)
- The JRE provides the necessary libraries and resources to run Java programs, including the JVM.
- It contains the core classes like String and Array, which your Java program may depend on.
- The JRE doesn’t include development tools like the compiler, making it suitable for running Java applications but not for development.

2. The Java Execution Process

Step 1: Write the Code
- You start by writing Java code, which is typically saved in .java files. This code is human-readable and follows Java syntax.
Step 2: Compile the Code
- Once the code is ready, the javac compiler is used to convert the human-readable .java code into bytecode (stored in .class files).
- Bytecode is a binary format, which is the same for all operating systems. This bytecode can then be executed on any platform that has a JVM, ensuring platform independence.
Step 3: Execute the Bytecode with JVM
- 3.1 Loading the Bytecode
- When you attempt to execute the Java program, the JVM loads the bytecode (i.e., the .class file) into memory.
- The ClassLoader is responsible for finding and loading the class based on the class name provided by the user.
- If the class cannot be found, a ClassNotFoundException is thrown.
- If the class is found, the JVM loads it into memory. Static methods, variables, and data from the class are stored in the Method Area, a special part of the JVM memory.
- 3.2 Executing the Bytecode
- Once the class is loaded, the JVM looks for the main() method (the entry point of the program) to begin execution.
- If the main() method is found, the execution process starts.

3. Execution Mechanism

There are two main approaches the JVM uses to execute bytecode: Interpreter and Just-In-Time (JIT) Compiler.

Interpreter (Slower)
- In the interpreter approach, the JVM reads and executes the bytecode line by line.
- Every time a method is invoked, the JVM re-interprets the bytecode, which can be slow since the same method may be re-executed multiple times.
JIT (Just-In-Time) Compiler (Faster)
- The JIT Compiler compiles bytecode into native machine code, which is specific to the platform and machine on which the program is running.
- It optimizes performance by using a technique called Hot Spots.
- Hot Spots are frequently used parts of the code (like methods). These are identified by the JIT compiler, and instead of interpreting them each time, the JIT compiles them into native machine code.
- The compiled machine code is cached, so when the same method is needed again, the JVM can use the cached machine code, resulting in faster execution.
Hot Spots
- The Hot Spot technique ensures that the JVM only compiles the frequently used methods, not the entire class. This results in significant performance improvements for long-running applications.
- The JVM uses machine code for execution of these hot spots instead of interpreting the bytecode every time.

4. JVM Memory Areas

The JVM allocates memory for different parts of the program during execution. Some key areas include:
- Method Area: Holds information about classes, methods, and static variables.
- Heap Area: Stores objects created during runtime.
- Stack Area: Stores local variables and method calls.
- Program Counter (PC): A register that points to the current instruction being executed.

5. Summary of the Execution Flow

Write Code: Java code is written in .java files.
Compile: The code is compiled into bytecode (.class files) by the javac compiler.
Load Bytecode: The JVM, using the ClassLoader, loads the bytecode into memory.
Find Entry Point: The JVM looks for the main() method to start execution.
Execution via Interpreter or JIT:
- Interpreter: Executes bytecode line by line (slower).
- JIT Compiler: Compiles hot spots into native machine code for faster execution (faster).

The combination of bytecode, JVM, and the JIT compiler ensures that Java is both platform-independent and efficient. The execution flow uses the Interpreter for simplicity and the JIT Compiler for performance optimization, allowing Java programs to run efficiently across various platforms.

Recap:

The JVM plays a key role in ensuring that Java code is portable and efficient. It first loads the bytecode, then executes it through either an interpreter (slower) or a JIT compiler (faster).
By using JIT and hot spots, the JVM optimizes performance while maintaining the ability to run the same bytecode on different platforms.

DEV Community: Nishanthan

Core Elements of Large Language Models (LLMs): Tokens, Context Windows, Temperature, and Sampling

What are Tokens?

Types of Tokens

Context Window

Temperature

Top-K Sampling

Top-P (Nucleus Sampling)

How They Work Together

Repetition Penalty

Conclusion

How Query, Key, and Value (QKV) Work in Transformers

1. What Are Query, Key, and Value (QKV)?

Query (Q) — What information am I looking for?

Key (K) — What information do I match on?

Value (V) — What information do I contribute?

2. Why Do We Need Q, K, and V?

3. How Are Q, K, and V Generated?

4. How Do Queries and Keys Calculate Attention?

Step 1: Calculate Attention Scores

Step 2: Scale the Scores

Step 3: Apply Softmax

5. How Do Values Produce Contextual Information?

6. QKV in Multi-Head Attention

Conclusion

Understanding Transformer: The Architecture Behind Modern AI

What Is a Transformer?

Why Were Transformers Introduced?

What Problem Were Transformers Designed to Solve?

How RNNs and LSTMs Process Sequences

Why Was Sequential Processing a Limitation?

What Did the 2017 Paper Attention Is All You Need Introduce?

How Do Transformers Process Relationships Between Tokens?

Where Are Transformers Used Today?

Conclusion

Backpropagation in Deep Learning: A Complete, Intuitive, and Practical Guide

Introduction: Why Backpropagation Matters

Why this matters:

What a Neural Network Really Computes (Forward Pass Overview)

Why this matters for backprop:

The Core Idea Behind Backpropagation

How should each weight in the network change to reduce the final error?”

Understanding the Computational Graph

Why this graph matters:

Chain Rule: The Mathematical Engine of Backpropagation

Why the chain rule is needed

Applying the chain rule to layers

Step-by-Step Backpropagation Through a Single Neuron

Forward Pass (Single Neuron)

Backward Pass (Goal: compute gradients)

Conclusion

Neural Network — A Simple, Beginner-Friendly Overview

What is a Neural Network?

How Are the Values Estimated?

Why Neural Networks?

Layers in a Neural Network

Input Layer

Hidden Layers

Output Layer

The 5-Step Life Cycle of a Neuron (Applies to Every Layer Except Input)

Final Thoughts

How Neural Networks Learn – A Simple Guide to Machine Learning & Deep Learning

1. Intelligence, Patterns, and Models

2. The Core Idea: Model as an Equation

3. Loss Function — Measuring Mistakes

A loss function:

4. Gradient — Finding the Direction of Improvement

Intuitively:

5. Learning Rate — Controlling the Speed of Learning

6. Epoch — Repeating the Learning Process

7. Batch — Dividing the Dataset

Benefits of batching:

8. Weights and Bias — Making Real-World Sense

Example: Courier Charges

9. Neural Network — From One Equation to Many

Why use neural networks?

10. Activation Function — The Source of Non-Linearity

Activation functions:

Without activation functions:

11. Forward Propagation — Data Moving Through the Network

1. `useState`

2. `useEffect`

3. `useMemo`

4. `React.memo` vs `useMemo`

5. `useCallback`

6. `useRef`

7. `useReducer`

8. `useLayoutEffect`

9. `useImperativeHandle`

10. `useDebugValue`

Misconception: Using `byte`/`short` Saves Memory?

Why `byte` and `short` are less efficient: