Steps of Building an LLM — An Overview

Fahimkhan9 — Sat, 27 Sep 2025 09:04:12 +0000

LLM (Large Language Model) is one of the most trending topics today.

A few years ago, if you Googled this topic, you would see a very different scenario from what we see today. LLM development was a highly specialized activity for AI research.

But today, if you Google “How to build an LLM from scratch” or “Should I build an LLM”, you will see a much different story.

In general, there are four main steps to building an LLM:

Data Curation / Data Collection
Model Architecture
Training at Scale
Evaluation

1. Data Curation / Data Collection

This is the very first step of building an LLM.

It is also the most important and time-consuming step of the process.

It is said that: “The quality of the model is driven by the quality of the data.”

For example:

GPT-3 was trained on 300 billion tokens
LLaMA 2 was trained on 2 trillion tokens

What is a Token?

A token is nothing but a chunk of text.

For example, "PCIU Computer Club" might become 4 tokens:
["PCI", "U", " Computer", " Club"]

Questions that arise:

What kind of data do we need?

This depends on the objective of the LLM. It may be:

General-purpose
Domain-specific (e.g., medical, financial fields)

Where to collect this data?

Generally, data is collected from:

Books
Scientific papers
Codebases (e.g., GitHub, GitLab)
Websites

Alternatively, public datasets such as Hugging Face datasets and Common Crawl can be used.

Data enrichment:

After collection, the data is cleaned by:

Removing duplicate text
Removing low-quality text

Finally, the text is converted into tokens using a tokenizer.

2. Model Architecture

As the name suggests, in this step we design or choose the architecture of the model.

Most popular models are based on the Transformer architecture.

What is Transformer architecture?

Conceptually, it is a way to read and understand language by paying attention to the relationships between words rather than reading word-by-word.

This is why Transformers power most modern LLMs.

Example:

“I hit the basketball with a bat” (bat = a stick)
“I hit the bat with a basketball” (bat = an animal)

The model won’t memorize word meanings word-by-word.

Instead, it will look for relationships:

Who is doing the action? (He / I)
What is being hit? (basketball / bat)
What tool is used? (bat / basketball)

By paying attention to these links, the Transformer figures out which “bat” is meant (stick vs. animal).

For the simplicity of this article, we won’t go deep into the technical aspects of Transformers and their variants.

3. Training at Scale

In this step, we feed the data to the model and adjust its parameters until it becomes good at predicting and generating text.

The essential challenge of building an LLM is its scale.

When we train on trillions of tokens and billions of parameters, the computational cost is enormous.

It’s basically impossible to train without implementing computational tricks such as:

Mixed precision
Gradient checkpointing
Parallelism

These techniques reduce cost while keeping performance high.

4. Evaluation

In this step, we evaluate whether the model works for the desired use case and whether it generates toxic or biased outputs.

This is where model evaluation comes in.

There are many benchmark datasets available for evaluation, for example:

Open LLM Leaderboard

Evaluation is critical before deploying an LLM to ensure it is safe, reliable, and useful.

Javascript Data Types- IN DEPTH!! -With AutoBoxing Concept,Reference Behavior.

Fahimkhan9 — Wed, 30 Jul 2025 13:33:45 +0000

In JavaScript, there are two types of data types.

Primitive Data Type
Non-Primitive Data Type.

Primitive Data Type

What is a Primitive Data Type?

Primitives are basic data types that are not objects and have no methods. Primitives are immutable (can't be changed), meaning their values cannot be changed once created. We will get to details after a while — let's get to know what primitive data types are.

String

String

const name = "Alice";

Number

const age = 25;

BigInt

const big = 1234567890123456789012345678901234567890n;

Boolean

const isActive = true;

Undefined

let x;
console.log(x); // undefined

Null

const empty = null;

Symbol

const sym = Symbol("id");

You might have one question in mind.

If string is a primitive data type, it means it doesn't have any methods and it's immutable.

But we know strings have methods like toUpperCase(), etc.? Well, you are right. Let's understand in detail.

When we use a method on any primitive (like string), JavaScript automatically wraps it in its corresponding object type so that we can access object-like methods.

Example:

const str = "hello";
const upperStr = str.toUpperCase();
console.log(upperStr); // "HELLO"

What JavaScript does internally (conceptually):

// Step 1: You have a primitive string
let str = "hello"; // primitive string

// Step 2: JavaScript wraps it in a String object (temporary)
let tempStrObject = new String(str); // [object String]

// Step 3: Call the method on that object
let result = tempStrObject.toUpperCase(); // "HELLO"

// Step 4: Discard the object, return result

Important: str is still "hello". The original primitive is unchanged because it's immutable.

Non-Primitive Data Type

Non-primitives are the opposite of primitives. Non-primitives are types that are not primitive.

Important: They are stored by reference, not by value.

Let's understand by an example

Primitive (stored by value)

let a = 10;
let b = a;  // b gets a copy of the value of a (10)

b = 20;

console.log(a); // 10 — ❗ a is unaffected

Non-primitive (stored by reference)

let obj1 = { name: "Alice" };
let obj2 = obj1;  // obj2 references the same object in memory

obj2.name = "Bob";

console.log(obj1.name); // "Bob" — ❗ obj1 is affected

Both obj1 and obj2 point to the same object in memory, so changing it through one affects the other.

DEV Community: Fahimkhan9

Steps of Building an LLM — An Overview

1. Data Curation / Data Collection

What is a Token?

Questions that arise:

What kind of data do we need?

Where to collect this data?

Data enrichment:

2. Model Architecture

What is Transformer architecture?

Example:

3. Training at Scale

4. Evaluation

Javascript Data Types- IN DEPTH!! -With AutoBoxing Concept,Reference Behavior.

Primitive Data Type

You might have one question in mind.

Example:

What JavaScript does internally (conceptually):

Non-Primitive Data Type

Let's understand by an example