DEV Community: Ajay Kumbham

Why "Attention" Changed Everything: A Deep Dive into the Transformer Architecture

Ajay Kumbham — Fri, 02 Jan 2026 17:38:57 +0000

Why "Attention" Changed Everything: A Deep Dive into the Transformer Architecture

In 2017, a team of researchers at Google published a paper with a bold, almost defiant title: "Attention Is All You Need." At the time, few could have predicted that this research paper would fundamentally reshape the entire artificial intelligence landscape.

Fast forward to today, and the "Transformer" architecture introduced in that paper is the engine behind everything from ChatGPT to advanced medical research tools. But how does it actually work? What makes it so much more powerful than the neural networks that came before it?

Let's unpack the Transformer step-by-step, moving past the hype to understand the elegant communication system that makes modern AI possible.

The "Before" Times: The Struggle with Sequences

To appreciate the Transformer, we first have to understand what it replaced.

In machine learning, the goal is usually to map an input to an output—like mapping house features to a price or a series of words to a "spam" or "not spam" label. For simple tasks, standard neural networks work great. But for sequential tasks like language translation, things get tricky.

Before 2017, the kings of sequence processing were RNNs (Recurrent Neural Networks) and LSTMs (Long Short-Term Memory). These models processed text like a human reading a book: one word at a time, from left to right. As they read, they would update an internal "memory" and pass it to the next step.

This approach had two massive flaws:

It was slow: Because it was sequential, you couldn't process the beginning and end of a sentence at the same time. No parallel processing meant training took forever.
It was forgetful: By the time an RNN reached the end of a long paragraph, it often "forgot" the context from the very first sentence. These "long-term dependencies" were the Achilles' heel of AI.

The Breakthrough: Letting Tokens Talk

The Transformer solved both problems by introducing a "smarter" layer called Attention.

If RNNs are like a single person reading a book one word at a time, the Transformer is like a room full of people where everyone is looking at every word simultaneously. In a Transformer, every "token" (a word or piece of a word) can talk to every other token in the sequence directly.

This isn't magic; it's communication. The attention mechanism allows the model to decide which other words are important for understanding the current one, whether those words are two steps away or 200.

Anatomy of a Transformer

A Transformer is essentially a stack of blocks, typically divided into an Encoder (which understands the input) and a Decoder (which generates the output). Each block consists of two primary layers:

The Attention Layer: This is the communication hub where tokens exchange information.
The MLP (Feed-Forward) Layer: Once a token has gathered information from its neighbors, it goes here to "privately" refine its own representation.

Think of it this way: the Attention layer is the group discussion, and the MLP is the individual reflection time.

The "Order" Problem

There’s one catch: because the Transformer looks at all words at once, it has no inherent sense of order. To a Transformer, "Jake learned AI" and "AI learned Jake" would look identical.

To fix this, we use Positional Encoding. We add special mathematical patterns to the word embeddings (the numerical vectors representing words) that tell the model exactly where each word sits in the sequence. This gives the model the context of "order" without sacrificing the speed of parallel processing.

The Secret Sauce: Queries, Keys, and Values

If you look under the hood of the attention layer, you'll find three vectors for every token: Queries (Q), Keys (K), and Values (V). This sounds technical, but it's actually a very intuitive system.

Let’s use the sentence: "Jake learned AI even though it was difficult."

When the model processes the word "it," it needs to know what "it" refers to.

The Query (Q): The word "it" sends out a query: "What concept am I referring to?"
The Keys (K): Every other word in the sentence provides a key describing what information it holds. The word "AI" has a key that says, "I am a subject being learned," while "Jake" says, "I am a person."
The Score: The model calculates a "dot product" between the Query of "it" and the Keys of every other word. It finds a high match with "AI" and a lower match with "Jake."
The Value (V): Finally, "it" updates its own meaning by taking a weighted sum of the Values (the actual content) of the words it matched with. In this case, "it" absorbs the "Value" of "AI," becoming a richer, context-aware representation.

Mathematically, the paper expresses this as a single matrix operation. Instead of looping through words, the model stacks all Qs, Ks, and Vs into matrices and computes everything in one giant, parallel step. This is why Transformers can be trained on the entire internet—they are incredibly efficient.

Why It Matters

At the start of training, all these parameters are random. The model has no idea what a "subject" or a "pronoun" is. But as it sees billions of sentences, it learns. Verbs learn to query their subjects; pronouns learn to look for relevant nouns.

The beauty of the Transformer is its generality. While it was built for translation, the idea of "tokens talking to each other" works for almost anything:

Images: Where pixels or patches are the tokens.
Audio: Where sound snippets are the tokens.
Code: Where characters and functions are the tokens.

The Takeaway

If you remember only one thing about Transformers, remember this: It’s a network that lets its inputs talk to each other.

It’s not some mystical black box; it’s a highly efficient communication system that allows every piece of data to find its own context. By moving from sequential processing to parallel attention, Google didn't just give us better translation—they gave us the blueprint for the modern AI era.

As the paper famously concluded, for a model to truly understand the world, attention really is all you need.

A Complete Guide to REST and RESTful API Design

Ajay Kumbham — Fri, 05 Dec 2025 20:14:44 +0000

A Complete Guide to REST & RESTful API Design (For Beginners to Advanced)

Everything you need to understand REST, its core constraints, and how to design clean, scalable APIs.

Introduction

Today, almost every modern application — from mobile apps to enterprise systems — uses APIs to communicate. Among the most widely used architectural styles for building APIs is REST (Representational State Transfer).

However, REST is often misunderstood. Many developers think that “using JSON over HTTP” automatically makes their API RESTful — but REST is much deeper than that.

In this blog, you’ll learn:

What REST actually is
The 5 mandatory REST constraints
What makes an API truly RESTful
Best practices and naming rules
Common mistakes developers make
Clear examples to help you remember each principle

Let’s begin.

What Is REST?

REST (Representational State Transfer) is an architectural style for designing networked applications. It was introduced by Roy Fielding in his PhD dissertation and is based on a set of rules (constraints) that ensure:

Scalability
Simplicity
Performance
Loose coupling between client and server

REST is not a protocol, not a library, and not tied to JSON or HTTP.
But in practice, RESTful APIs use HTTP as the transport protocol.

REST vs RESTful

REST → Architecture, theory
RESTful API → Implementation that follows REST constraints

If an API does not follow the required constraints, it is simply HTTP API, not REST.

The 5 Mandatory REST Constraints

These are the non-negotiable principles that define REST.
Every RESTful API must follow them.

1. Client–Server Architecture

REST strictly separates:

Client → Responsible for UI
Server → Responsible for data and business logic

They can evolve independently.

Benefits:

Easier scaling
Flexibility in technology choices
Separation of concerns

2. Statelessness

A REST API must be stateless, meaning:

The server does not remember anything from previous requests.
Each request must include all necessary information (auth token, parameters, etc.)

Example:
A banking API must receive account number/token every time, even if it was sent earlier.

Benefits:

Easy horizontal scaling
No server-side sessions
Simpler debugging and reliability

3. Cacheability

REST requires that responses explicitly declare whether they are cacheable.

For example:

Cache-Control: max-age=3600

This means:

“The client may reuse this response for 1 hour.”

Important:
Caching is not forced.
REST only requires the server to mark data as cacheable or non-cacheable.

Why this matters:

Prevents clients from caching sensitive or changing data accidentally
Improves performance for safe, static data

4. Uniform Interface (The Heart of REST)

This is the most important constraint.
It ensures that all parts of the system interact in a standard, predictable way.

Uniform Interface has four rules:

4.1 Resource Identification in URLs

URLs identify resources (nouns), not actions.

/users/12
/getUserDetails

4.2 Manipulation Through Representations

Clients modify resources by sending JSON/XML that represents the updated state.

Example:

PUT /users/12
{
  "name": "Ajay"
}

4.3 Self-Descriptive Messages

Each request must contain enough information to be understood:

HTTP method (GET/POST)
Headers
Body format
Status codes

No hidden meaning or special rules.

4.4 HATEOAS (Hypermedia as the Engine of Application State)

The server may provide links that guide the client.

Example:

{
  "id": 10,
  "links": {
    "self": "/users/10",
    "orders": "/users/10/orders"
  }
}

Although optional in practice, HATEOAS is part of true REST uniform interface.

5. Layered System

The client must not care how many internal layers the system has.

A request may pass through:

API gateway
Proxy
Load balancer
Auth server
Microservice

The client sees all as a single endpoint.

Benefit:
Flexible, scalable architecture.

Optional REST Constraint (Bonus)

Code on Demand

Servers may send executable code to clients (e.g., JavaScript).
This is optional and almost never used in APIs.

RESTful API Design Principles (Must-Follow Best Practices)

Now that we know REST constraints, let’s look at RESTful design rules commonly expected in industry.

Use Proper HTTP Methods

Method	Meaning
GET	Fetch resource
POST	Create resource
PUT	Full update
PATCH	Partial update
DELETE	Remove resource

Use Nouns, Not Verbs, in URLs

/products/10
/fetchProductDetails

Use Plural Resource Names

/users
/users/21/orders

Use Query Parameters for Filtering and Pagination

Example:

GET /products?category=mobile&sort=price&limit=20&page=2

Use Meaningful HTTP Status Codes

200 OK
201 Created
204 No Content
400 Bad Request
401 Unauthorized
403 Forbidden
404 Not Found
409 Conflict
500 Server Error

Common Misconceptions About REST

“If I use JSON, my API is RESTful.”

REST has nothing to do with JSON.

“Using HTTP automatically makes it REST.”

REST requires strict constraints.

“HATEOAS is optional.”

For real REST (as defined by Fielding), it’s mandatory, but most APIs skip it.

“GET requests can have a body.”

They shouldn’t — many proxies will block them.

REST vs Other Approaches

Feature	REST	GraphQL	gRPC
Style	Architecture	Query Language	RPC Framework
Format	JSON/XML	JSON	Protobuf
Best for	Public APIs	Complex queries	High-performance microservices
Type safety	Low	Medium	High

Conclusion

REST is not just “API with JSON.”
It is a well-defined architectural style with strict constraints.

A truly RESTful API must follow:

Client–Server
Statelessness
Cacheability
Uniform Interface
Layered System

And use recommended practices like proper HTTP methods, clear status codes, nouns in URLs, and consistent structure.

By understanding the purpose of each rule — not just memorizing them — you’ll be able to design clean, scalable, maintainable APIs that work well in real-world systems.

What Really Happens After You Enter a URL in Your Browser? A Complete Breakdown

Ajay Kumbham — Mon, 01 Dec 2025 15:34:32 +0000

1. What a URL Actually Represents

A URL acts as an instruction set for the browser. It describes how to reach a resource and where that resource lives. A typical URL can include:

• Scheme (Protocol)
Examples: http://, https://
This tells the browser what communication rules to follow.

• Domain Name
Example: example.com
A readable name that must be translated into a numerical IP address.

• Path
Example: /products/
Shows the location of a section or directory on the server.

• Specific Resource
Examples: index.html, logo.png, main.js
Indicates the exact file the browser should retrieve.

All these elements work together to point the browser to a unique item on a remote machine.

2. DNS Resolution: Mapping the Domain to an IP

Browsers can’t connect using domain names directly. They need an IP address.
This is where DNS (Domain Name System) comes in — it acts like a global address directory.

The lookup sequence generally follows this order:

Browser Cache — checks if the IP is already stored from a previous visit
OS Cache — the operating system may have the mapping saved
DNS Resolver — typically provided by your ISP or a public resolver
DNS Hierarchy

Root servers
TLD servers (.com, .net, .org, etc.)
Authoritative DNS server for the domain

Once the resolver finds the correct IP, the browser can move forward.

3. Creating a TCP Connection

With the IP known, the next step is establishing a communication link.
Most websites rely on TCP, a protocol designed for reliable data transfer.

A TCP connection is created using a three-step negotiation called the three-way handshake:

Browser → Server: SYN
Server → Browser: SYN-ACK
Browser → Server: ACK

Modern browsers often keep these connections alive so future requests reuse the same link, reducing latency.

4. HTTPS: Setting Up Encryption (TLS Handshake)

If the URL uses HTTPS, the browser must secure the connection.
This involves an additional process known as the TLS handshake:

The server presents its SSL/TLS certificate
The browser validates it
Both sides agree on encryption methods
A shared session key is generated

After this, all communication is encrypted end-to-end.

5. Sending an HTTP Request

Once the connection is ready, the browser submits an HTTP request.
The request includes:

Method (GET, POST, etc.)
Target path
Headers (cookies, accepted formats, browser details)
Body (for POST/PUT requests)

Example:

GET /index.html HTTP/1.1
Host: example.com

6. How the Server Handles the Request

The server receives the request and decides how to respond:

Retrieves the requested file or runs backend logic
Prepares the response body (HTML, JSON, images, etc.)
Adds relevant headers
Sends it back to the browser with a status code like

200 OK
404 Not Found
500 Internal Server Error

7. Browser Rendering Pipeline

Once the response arrives, the browser begins turning raw data into a visible webpage.

Step-by-step:

• HTML Parsing → DOM Creation
The browser constructs the Document Object Model.

• Fetching Linked Resources
CSS, JavaScript, images, fonts — each may generate additional requests.

• CSS Parsing → CSSOM Creation
Used to apply styling.

• JavaScript Execution
Scripts can modify the DOM, change styles, or fetch more data.

• Layout and Painting
The browser calculates positions, applies styles, and finally renders the pixels on the screen.

8. The Final Result

What looked like a simple URL entry actually triggered caching checks, DNS lookups, secure handshakes, request/response cycles, multiple resource fetches, and a rendering pipeline — all executed in milliseconds.

Quick Recap

When a URL is entered, the browser:

Interprets the URL components
Resolves the domain name into an IP
Establishes a TCP connection
Sets up encryption (for HTTPS)
Sends an HTTP request
Receives and processes the server’s response
Loads additional resources
Renders everything into a functional webpage

This is the entire behind-the-scenes mechanism that powers the modern web.