DEV Community: Kehinde Giwa

How I built a search engine

Kehinde Giwa — Sat, 13 Jun 2026 09:18:51 +0000

Hello all, I built a search engine from scratch in Java. In a previous course we were given it as a group project, but since I worked on just the document processing module, I didn't really understand (I didn't understand at all actually) how the rest of it worked, so I decided to rebuild it from scratch. I called it Nova Search. Keep in mind this post won't have a lot of code but I'll try to keep it as technical as possible.

Before jumping in — what is a search engine? It's a program that allows users to retrieve information from a source. That source could range from a specific set of documents to the entire internet (e.g. Google). I built mine to cover a specific set of documents, since I don't have the resources to crawl the entire internet lol. The aim was just to understand how search engines work under the hood. So let's get into it.

I'll divide this into two parts: indexing and querying. Indexing covers what happens to a document and how it gets stored. Querying covers how information is retrieved from the stored documents.

The indexing

In a search engine, the main data structure for storing data is something called an inverted index. An inverted index maps words, numbers, or terms directly to the specific documents where they appear — it flips the traditional relationship (i.e. documents to words).

Why do we need it? Imagine searching for the word "dog" in a document with a million words. You'd have to go through every single word and match them — that's O(n) time complexity, meaning the time taken increases with the number of words you have to search through. Now imagine doing that across many documents. It gets inefficient fast.

Let me show how an inverted index fixes this. Start with two documents:

Doc1 → "The dog chased the dog around the yard, the dog barked"
Doc2 → "The fox jumped over the fence and ran away"

To turn this into an inverted index, we create a dictionary — let's call it Index. For each word in each document, we check if it's already in Index. If it's not, we add the word as a key with an array containing the document ID as the value. If it is, we just append the document ID to the existing array.

Which gives us something like this:

Word	Documents
the	[Doc1, Doc2]
dog	[Doc1]
chased	[Doc1]
around	[Doc1]
yard	[Doc1]
barked	[Doc1]
fox	[Doc2]
jumped	[Doc2]
over	[Doc2]
fence	[Doc2]
and	[Doc2]
ran	[Doc2]
away	[Doc2]

So now if we wanted to search for the word "dog", we simply check Index["dog"] and it gives us every document that contains it. That's an O(1) operation — no matter how many words were in the document, finding a word always takes constant time.

Now we could call it a day and be done with our index, but there's a way to make it more efficient: normalizing the text before indexing. Normalizing basically means cleaning the data to remove inconsistencies and bring everything to a single, predictable form. Here's what that looks like:

Strip punctuation (yes, I'm aware there are no punctuations in this example but 🐻 with me)
Lowercase everything
Remove stop words — filler words that don't carry much meaning. You can grab a stop word list online; for our example we'll use ["the", "at", "a", "and"]
Reduce each word to its root. "Jumped" becomes "jump", "barked" becomes "bark" — that way "barking" and "jumping" won't register as different words. You can do this with a stemmer or a lemmatizer; a stemmer is simpler, a lemmatizer is more accurate but uses more memory. I went with a stemmer.

Here's our cleaned-up inverted index:

Word	Documents
dog	[Doc1]
chase	[Doc1]
yard	[Doc1]
bark	[Doc1]
fox	[Doc2]
jump	[Doc2]
fenc	[Doc2]
ran	[Doc2]
away	[Doc2]

The querying

Now that we know how to build our index, how do we retrieve information from it based on a query?

Let's walk through an example. The user searches for "A dog and a fox".

First we go through the same pipeline we used during indexing — normalize the query, strip punctuation, lowercase, remove stop words, and stem.

Step	Result
Original query	A dog and a fox
Strip punctuation	A dog and a fox
Lowercase	a dog and a fox
Remove stop words	dog fox
Stem	dog fox

That gives us "dog fox".

Now for each word, we check the inverted index for the documents that contain it:

"dog" → [Doc1]
"fox" → [Doc2]

We first take the intersection — the documents common to both. Since there's no intersection in this example, we fall back to the union: all documents that contain any of the words, which gives us [Doc1, Doc2].

Now that we have our matching documents, how do we decide which one to show first? How do we determine which is most relevant to the user? This is where a ranking algorithm comes in. There are several to choose from, but for a simple search engine I went with TF-IDF.

TF-IDF stands for term frequency–inverse document frequency. It scores how relevant a word is to a given document.

Term frequency (TF) = number of times the word appears in the document ÷ total number of words in that document
Inverse document frequency (IDF) = log(total number of documents ÷ number of documents that contain the word)
TF-IDF = TF × IDF

For each document, we calculate the TF-IDF of each word in the query:

Doc1 → dog: 0.5 × 0.301 = 0.1505, fox: 0 × 0.301 = 0 → total: 0.1505
Doc2 → dog: 0 × 0.301 = 0, fox: 0.2 × 0.301 = 0.0602 → total: 0.0602

We sort the documents by score and present the results — Doc1 ranks higher, which makes sense, it's the dog document.

There's one more feature I added — autocompletion — but I think I'll save that for the next article.

While I was mid-building this project I realized it was following a software architecture pattern I studied last semester — pipe and filter, where data gets transformed at each stage and passed along. I'd understood it well enough to pass the exam , but seeing it click into place in something I actually built hit different. Funny how that works.

You can check out the code in the repo here, and the live link here also my portfolio is here.

I Built an Online Multiplayer Ludo Game

Kehinde Giwa — Sun, 10 May 2026 14:27:19 +0000

So, this year I made a few resolutions — and sure, no one keeps to resolutions, but I decided to at least keep these two: first was no more tutorials, and the other was to build projects instead.
I was looking to improve my backend skills and practice advanced backend concepts I hadn't touched before, so I decided to get my hands dirty and make an online multiplayer game.
I initially decided on chess, but after a few minutes of planning out the move logic and rules, I quickly realized I'd spend a lot more time trying to get the piece movements to work and lose sight of my actual goal — or even more likely, just give up. The alternative was a chess validation library, which didn't sound as fun as building from scratch.

Hard enough that it gave me just enough headache to build but not so much that i'd give up.

So I decided on Ludo. Let me go over my process.

The Frontend

Building the board

So the first thing I worked on was the ludo board — not the landing page or the auth or player selection, the ludo board. I wanted to get the game working first then just plug and play later. I used Tailwind to first implement the outer parts.

Tracking tiles.

Then I thought about the tiles which the pieces were going to move on. Sure, I could do some CSS magic to make the grid, but how was I going to be able to track where the piece was? So, I did a thing : I made a function that took a position and returned a style for it. The function was long and there was probably a better way to write it that I didn’t know, but it worked. Each tile now represented a certain position on the board. I kept this function in a utils folder and called it getCellPosition.

Pieces, dice, and a happy npm find

After I was done with the tiles, I designed the pieces and set them to their starting positions with another function . I also needed a dice component. After a quick search on npm I saw a 3D dice box package 3d-dice, which completely fit my use case, but it turned out it didn’t have type definitions. I initially planned on writing my own .d.ts file, but I realized that the fact that I thought to do that meant someone else probably did too, so I went through the forks of the project and sure enough, I saw a fully typed version.

GAME LOGIC

Implementing the gameplay

This was arguably the hardest part of the entire project: implementing the gameplay. I wouldn’t want to bore you with the intricate details, but I’ll give a high-level overview.

useReducer as the game engine

I decided to use a useReducer to handle the game state, since I knew a game would have lots of state interacting with each other. I had an initial game state object which contained an array of player objects, each with an array of piece objects.
I had 3 reducer actions: ROLL_DICE, SELECT_NUMBER, and MOVE_PIECE. The way I visualized it was: when you play a game of ludo, you roll the dice, select a number in your head, and move a piece — so I built my reducer actions around that.

Flags → State Machine

Before I get into a summary of what each action did, I’d like to mention an early early decision I made. I decided to manage the piece state using flags — something I had read about earlier, so I just went with them. I had a flag for when a piece was in its home, one for when it’s on the board, one for when it’s in the home stretch, and one for when it’s finished.

This caused a big problem later on, because I now had to reset all the other flags when changing one. I fixed that with a state machine — each piece can only have one state at a time

The reducer actions

So back to the reducer actions: the ROLL_DICE action, which probably sounds misleading now that I think about it, was basically what happened after you clicked roll dice. It didn’t trigger the 3D dice box — that was done by the onClick handler. It just set the roll result to the result of the 3D dice rolling. It also determined whether the current player was able to move based on the result of the dice roll, or whether the player’s turn was to be skipped (Ludo game rules).

Then the main headache was the MOVE_PIECE action. This action handled all piece movement — from getting out of the starting position to capturing pieces and entering the homestretch, even finishing. That was all in the reducer and the game was working fine. Then I proceeded to work on the UI — the home page, the player selection screen, up to the game itself. Then I moved over to the backend.

THE BACKEND

Express, Redis & WebSockets

Working on the backend, I set up my Express app using a new folder structure I saw a friend of mine use. Normally I organize my files into the MVC architecture (model, view, controller), but he organized his into features, where each feature had a controller, service, and router. Then outside the features there was a core folder containing things like errors (custom errors), middlewares, and utils (functions that aren’t scoped to any one feature or service).

Authentication first

I started out by implementing authentication as I normally do — JWT cookies and all. Then I started on the online game.

Reusing the reducer over WebSockets

Remember the big reducer function from earlier? Since a reducer is really just a pure TypeScript function that takes input and returns output, I could reuse the exact same function here using WebSockets.

If you don't know how WebSockets work, check it out here. The analogy I use: email vs. a phone call. An email is sent and you wait for a reply, while a phone call lets both parties communicate in real time. Sockets have events — you emit events and act on them.

I had the event handlers on the server, and each handler received an event from the client, processed it, and emitted its own event back.

The 7 socket events

Event	What it does
`create-room`	Server initializes a new game state, adds the first player, generates a room code and stores it in Redis prefixed with `"game:"` e.g. `"game:001"`.
`join-room`	Receives the room code, checks if the game exists on Redis and is still available to join, then adds the new player.
`start-game`	Confirms that all the players are ready and sets the game state to in-progress.
`ready-player`	Sets the player’s state to ready.
`game-action`	Takes a reducer action emitted from the client, runs it through the reducer function, and updates the new game state to Redis.
`rejoin-game`	Allows a disconnected player to reconnect to an ongoing game.
`disconnect`	Handles cleanup when a player drops.

Deployment & closing thoughts

So after this I was mostly done with the server and went back to the client. I first implemented authentication with my auth context and provider, then I created a socket provider and a game state provider. The socket provider simply connected to the socket on the backend and broadcast to all the components, while the game state provider held the game state and listened for any incoming socket events from the server and handled them. Then all that was left was to finish up the frontend and the app was ready. I created a container with Docker and deployed on Vercel, Render, and Upstash — and that was the end of my project.
So this was a fun project to build and I learned a lot from it, not just because the game works, but because of what breaking it taught me. The flags-to-state-machine refactor was probably the most valuable thing I took away: sometimes the first design decision that seems reasonable becomes the thing that quietly makes everything harder. I’m currently working on my next project which is a search engine (not a web crawler lol), with Java Spring Boot, so see you then!
If you have any questions or feedback, drop them in the comments — I'd love to hear how you'd have approached it differently.

Source code: Github Live demo: https://ludoly.vercel.app