DEV Community: Mateus

Tagging 8,700 exam questions with an LLM, and why the Portuguese and English versions disagreed

Mateus — Tue, 06 Oct 2026 20:30:50 +0000

I'm building NaHero (https://nahero.site) on my own. It's a set of timed practice exams for AWS, Azure and Google Cloud certifications, in Brazilian Portuguese and English. There are 30 exams and about 8,700 questions, and every exam exists in both languages. Each exam has 65 questions, the real exam's time limit (90 minutes for CLF, 130 for SAA) and the passing score from the official exam guide.

I started it because I was studying for an AWS certification and couldn't find anywhere to take practice exams for free.

Three parts turned out to be harder than they looked: tagging every question with its exam domain using an LLM, keeping the two languages consistent, and generating study plans without letting the model do the math.

Stack

Front: Next.js, Tailwind, react-query
Back: Spring Boot, Postgres, Flyway
Payments: Stripe
Study plan text: DeepSeek V4.1 Flash
License: AGPL-3.0, nahero-front and nahero-back

Why domains matter

Official exam guides split each certification into domains. A single score tells you whether you passed. A score per domain tells you where to study. NaHero shows results per official domain on the free tier, and the paid study plan is built on top of those numbers. All of that depends on every one of the 8,700 questions carrying the right domain.

The tagging loop

I tagged the questions with claude -p (Claude Code's non-interactive mode) in a loop:

40 questions per call
the domain list for that specific exam in the prompt
output validated against a fixed list of domains for that exam

The whole bank took about 220 calls.

Putting the exam's own domain list in the prompt turns an open-ended question ("what is this about?") into a choice from a closed set. Validating against the same list means a domain name the model makes up, or paraphrases, never reaches the database.

The consistency problem

Every question exists twice, once in Portuguese and once in English. That gave me a check I didn't need labeled data for: tag both versions independently and see whether they get the same domain.

On Google Workspace Admin, the first pass agreed about 73% of the time.

For the product, that's a bug. If roughly a quarter of questions land in a different domain depending on language, a Brazilian user and an English-speaking user with identical answers get different per-domain results, and per-domain results are what the whole product is built around.

Two changes fixed it:

Per-exam disambiguation lines in the prompt such as "billing and pricing".
Tag English only, then copy the domain to Portuguese. With one source of truth, the two versions agree by construction.

The second change is the less clever one and the one I'd recommend first. The first change still matters, because agreement alone doesn't make the English tag correct.

If you have equivalent inputs (two languages, two phrasings, two orderings), agreement between them is a cheap way to find errors when you have no ground truth.

Study plans: SQL does the math, the LLM writes the text

Premium users get a study plan after each attempt: 3 priorities and an ordered list of topics. The pipeline:

SQL computes hits per domain, missed questions and previous scores.
DeepSeek V4.1 Flash receives those numbers and returns JSON with the text. It doesn't calculate anything.
The output is validated: domain names in the response must match the ones in the input, and links aren't allowed.
The plan is stored once per attempt, so reloading the result page doesn't regenerate it.
Failures are stored too, so the system doesn't keep retrying the same attempt forever.

Matching domain names means the plan can't tell you to study a domain that isn't on your exam. Banning links means it can't send you to a URL nobody checked. Storing once means the plan you read today is the one you read tomorrow. Storing failures keeps a bad attempt from turning into a loop of paid API calls.

Resuming an attempt

A 130-minute exam doesn't always fit in one sitting. Answers are auto-saved as drafts, and reopening an attempt puts you back on the last question. Resume is on the free tier.

What the data says so far

Numbers as of October 6, 2026:

About 95 registered users, about 75% from Brazil.
About 30 signups a week, with no ads running.
Most CLF attempts score between 23 and 47 out of 65. Passing is around 46, so most attempts fail.
Only about 15% of users took a second exam in their first week.

Those last two together changed the product. Someone takes an exam, fails, sees a number and doesn't know what to do next. So I rebuilt the result screen around "what to study next".

On acquisition: I tested Google Ads with R$100. That got 71 clicks at R$0.31 to R$0.90 per click, an 11% click-to-signup rate and 1 annual subscriber. LinkedIn posts bring in 3 times what Google Ads did.

The first paying user came from an ad, left, and came back after the 7-day re-engagement email. They scored 44 out of 65, subscribed 36 minutes after seeing the failing score, and passed with 55 out of 65 three days later.

Pricing

All 30 exams, unlimited attempts, scores, per-domain results and resume are free. Premium is R$10/month or R$90/year and adds explanations per question, the study plan and practice questions on your weakest domain.

What I'd like feedback on

If you've used LLMs for classification across languages, I'd like to know how you measured consistency and what you did about disagreements. The code is open, so you can also read exactly how it's done in the repos linked above.

Data Structures - Hash Tables

Mateus — Wed, 22 May 2024 11:06:38 +0000

Introduction

Data structures are a fundamental part of initial programming studies. Understanding different data structures and knowing how to use them is essential. While studying the course: "Learn What Data Structures and Algorithms Are," the topic of hash tables was covered more superficially than other data structures. I decided to seek more information online. As a good apprentice developer, I googled it and soon found several videos discussing the topic. However, these definitions were also superficial, leading to inevitable frustration. I then tried ChatGPT, but believe it or not, it was down, and I couldn't get any answers. So, I sought help from Copilot (thanks to IFPR and their great help from education packages =D) and managed to understand a bit more than I had from the course. I decided to write this article to learn more and get feedback from those who understand the subject better. So, let's get into the little I understood.

Definition

A hash table is a data structure that associates keys with values, allowing for fast and efficient searches, making it ideal for indexing and data access. Each key-value pair is called a "slot." Hash tables are widely used in user authentication systems to store passwords securely; we will discuss this in more detail later.

An example of using hash tables is:

Associative arrays: Mapping keys (e.g., names) to values (e.g., phone numbers).

Index 0: "Paula" | "7198749874" <- slot 1 Index 1: "Antenor" | "71999999999" <- slot 2 Index 2: <- slot 3 Index 3: <- slot 4 Index 4: "Fernando" | "11999994444" <- slot 5

In this example, we have a hash table that maps a person's name to their phone number.

Sets: Membership verification (e.g., checking if an element is in a set). The set elements are not necessarily ordered.

For example, the set of prime numbers (2, 3, 5, 7, 11). To verify membership, we use the in operation.

Caches: Temporary storage areas for frequently used data. For example, a cache of query results from a database prevents repeated queries to the database since the data will be used again. When a query is made, the cache is checked first. If the data is there, it avoids another query, reducing data flow and speeding up the application.

In summary, hash tables are a way to index data. They are designed to allow quick access to values based on associated keys, even when retrieving information from large data sets. The hash code (index) is calculated and mapped to an index in the array, and the values are stored in the corresponding indices.
Hash Function

A hash function maps the key to an index in the hash table. Let's see an example. Suppose we have a hash table with 10 indices numbered from 0 to 9. We want to store keywords in this table. We will create a basic hash function to store the words: "cat," "dog," "lion," "goat," "jaguar," "bear," "elephant," "bull," "snake," "alligator." To populate the hash table, we will use the following function: the remainder of the division between the sum of the ASCII values of each character in the words by the number of table positions.

1st word: "cat" => (103 + 97 + 116 + 111) % 10
-> 427 % 10 = 7, this is the index (hash) where we will store the word "cat."

2nd word: "dog" => (99 + 97 + 99 + 104 + 111 + 114 + 114 + 111) % 10
-> 849 % 10 = 9, this is the hash for the word "dog."

This process will be followed for all words. It is not uncommon, especially in this case where the number of table indices is limited, for a word to need to be stored at the same index as another word. This is called a "collision." Collisions are common in hash functions.

Collisions

Example of a Hash Function

Consider a simple hash function that computes the hash value by summing the ASCII values of the characters in the key and then taking the modulus with the size of the hashmap.

For example, suppose our hashmap has 10 buckets and our hash function is:

def hash_function(key): return sum(ord(char) for char in key) % 10

Let's compute the hash values for two different keys:

Key: "cat"
    ASCII values: c (99), a (97), t (116)
    Sum: 99 + 97 + 116 = 312
    Hash value: 312 % 10 = 2

Key: "act"
    ASCII values: a (97), c (99), t (116)
    Sum: 97 + 99 + 116 = 312
    Hash value: 312 % 10 = 2

Both "cat" and "act" produce the same hash value of 2 because of their same ASCII values summed up, leading to a collision.

But then the question arises: how to store the words if different words cannot be stored in the same index? There are some ways to resolve collisions.

Resolving Collisions

The most common methods for resolving collisions are: Separate Chaining, Open Addressing, and Perfect Hashing (best-case scenario).

Separate Chaining: In this case, each slot in the table contains a linked list of elements with the same index. When a collision occurs (i.e., two keys are mapped to the same slot), the elements are simply added to the corresponding list. Suppose the words "dog" and "elephant" map to the same slot; then the hash table would look like this:

Index 0: Index 1: Index 2: Index 3: Index 4: Index 5: Index 6: Index 7: 7 | "cat" Index 8: Index 9: 9 | "dog" -> "elephant"

Open Addressing: Another way to resolve collisions is through Open Addressing or linear probing, which involves trying to find another free slot in the table. Linear probing is a common technique where slots are checked sequentially until an empty one is found. In the example of the collision between "dog" and "elephant," the word "elephant" would be stored in the slot with index 0, as it is the next empty slot. If slot 0 were occupied, then slot 1 would be tried, and so on. In this case, a free slot will always be found since there are 10 words for 10 slots.

Perfect Hashing: Perfect hashing is the ideal situation but is not always possible. It means that each key has a unique index in the hash table without collisions. If we had a perfect hash function, each word would be perfectly indexed, and there would be no collisions.

There are strategies aimed at finding a good hash function that ensures maximum spreading across the table. Perfect hashing is when, given a table of n positions and n elements to be inserted, there is no collision. One strategy for choosing the hash function is the Cichelli method, which aims to minimize collisions by making some adjustments to the hash to differentiate it from the already calculated hash. This adjustment can be called a "salt," which is, for example, a constant added to the hash, creating different possible values for collisions. This salt must be stored in some field of the table, so when attempting to access the value, it is added to the hash, and the values match. This may be a bit confusing.

Remember the example where the words "dog" and "elephant" collided. The salt could be the hash divided by 2 or the number 1, for example. But the salt is added only to the second value that caused the collision, so it would only affect the hash of the word "elephant."

For example:

`Index 9: 9 | "dog"

Hash calculated for "elephant" is 9; adding the salt 1, the hash becomes 10 % 10 = 0, so the word "elephant" would be stored in the slot with index 0.

Index 0: 0 | "elephant"
Index 7: 7 | "cat"`

The hash table data structure has a complexity of Big O(1) notation because the search is immediate, making it ideal.

Application

A very common application for hash tables is user authentication systems. The user creates a password, a hash is applied to this password, and the hash of the password is stored in the database, never storing the actual password typed by the user. This ensures the security of such sensitive information. Several hashes are used for this: MD4, MD5, SHA-1, SHA-256, SHA-512.

But if the password is not stored, only the hash of the password, how do you know if the user entered the correct password? Simple, the password entered by the user is hashed again, and the newly generated hash is compared with the stored hash in the database. This is how authentication occurs.

Another use of hashing is file verification. How do hash functions ensure the integrity of files? When the hash of a file is calculated, a compact representation of the data is obtained. If there is any alteration in a file, the resulting hash will be different. Therefore, such a comparison allows you to verify if the file remained unchanged. This comparison can be done before and after downloading a file, for example.

Another application of hash tables is digital signatures. Digital signatures use hash functions to verify the authenticity of documents. The hash of the document is encrypted with the sender's private key, creating a digital signature. The recipient can verify the signature by decrypting the hash with the sender's public key.

An example of using hash tables for a small user authentication system. Here I will add Python code. The user creates a password, and the password is stored in a hash table. At login, the password entered by the user is hashed again and then checked if the hash values are identical. For this example, I will use MD5 hashing. I will use the Python hashlib module to work with hash functions.

import hashlib

# Hash table to store passwords (user -> password_hash)
password_hash_table = {}

def hash_password(password):
    # Create an MD5 object
    md5_hash = hashlib.md5()
    # Update the MD5 object with the password as a hexadecimal string
    md5_hash.update(password.encode('utf-8'))
    # Return the MD5 hash as a hexadecimal string
    return md5_hash.hexdigest()

def register_user(username, password):
    # Hash the password using the hash_password function
    password_hash = hash_password(password)
    # Store the username and password hash in the hash table
    password_hash_table[username] = password_hash
    print(f'User "{username}" registered successfully.')

def login_user(username, password):
    # Hash the entered password using the hash_password function
    entered_password_hash = hash_password(password)
    # Check if the username is in the hash table and if the hashes match
    if username in password_hash_table and password_hash_table[import hashlib

# Hash table to store passwords (user -> password_hash)
password_hash_table = {}

def hash_password(password):
    # Create an MD5 object
    md5_hash = hashlib.md5()
    # Update the MD5 object with the password as a hexadecimal string
    md5_hash.update(password.encode('utf-8'))
    # Return the MD5 hash as a hexadecimal string
    return md5_hash.hexdigest()

def register_user(username, password):
    # Hash the password using the hash_password function
    password_hash = hash_password(password)
    # Store the username and password hash in the hash table
    password_hash_table[username] = password_hash
    print(f'User "{username}" registered successfully.')

def login_user(username, password):
    # Hash the entered password using the hash_password function
    entered_password_hash = hash_password(password)
    # Check if the username is in the hash table and if the hashes match
    if username in password_hash_table and password_hash_table[username] == entered_password_hash:
        print(f'User "{username}" logged in successfully.')
    else:
        print(f'Login failed for user "{username}".')

# Registering users
register_user('Paula', '1234')
register_user('Antenor', 'abcd')

# Attempting logins
login_user('Paula', '1234')
login_user('Antenor', 'abcd')
login_user('Fernando', 'wrong_password')
        print(f'Login failed for user "{username}".')

# Registering users
register_user('Paula', '1234')
register_user('Antenor', 'abcd')

# Attempting logins
login_user('Paula', '1234')
login_user('Antenor', 'abcd')
login_user('Fernando', 'wrong_password')

In the example, there is no method to handle collisions. What if there is a collision between the passwords registered by different users? Let's try to handle collisions. In this example, we will use the chaining technique, where each entry in the hash table is a linked list (i.e., a list of passwords with the same hash value). If there is a collision, we add a new password to the existing list. To add collision handling, we will modify the method for adding a user, as this is where values are inserted into the hash table. The method will look like this:

def add_user(user, password):
    # Calculate the password hash
    password_hash = hash_password(password)

    # Check if the user already exists in the table
    if user in password_hash_table:
        # Add the password to the existing hash list
        password_hash_table[user].append(password_hash)
    else:
        # Create a new list with the new password
        password_hash_table[user] = [password_hash]

Important considerations: For security purposes, MD5 hashing is not recommended; it would be better to use SHA-256 or SHA-512. It is also a good idea to add a salt to the passwords before calculating the hash, which will increase security by making it more difficult for an attacker to use precomputed hash tables (Rainbow tables) to crack passwords.

Thank you for reading this article.

Load balancers and Club Bouncers.

Mateus — Mon, 05 Feb 2024 00:53:51 +0000

Imagine it's 11pm and, as the social butterfly you are, going to a club with all your friends seems a good idea. Now, let's say the club only has capacity for 300 people and over 1000 people are trying to enter, what could happen?

Well, either the party is going to be way too crowded leaving no space to dance, or, firefighters will close up the party due to overcrowding. In such scenario, the owner of the club hires a bouncer to smooth up the entrance.

Now, a bouncer in our club analogy is the load balancer on a server and people are like TCP data packets. That is because load balancers distribute incoming requests and data among multiple servers or resources in a way that ensures each server shares the load fairly.

However, there are many more things a load balancer can do that a bouncer simply can't, namely:

1. Scalability

What if the club becomes Project X? The number of partygoers keeps growing more than ever and it doesn't stop, what can the owner do? Either expand the club by opening additional entrances or risk turning away potential customers.

In server language, this growth corresponds to an increase in users. As the demand for a website or application surges, a single server might struggle to handle the traffic. This is where scalability comes into play. Just like opening new entrances to accommodate more clubbers, load balancers enable the addition of more servers to distribute the growing workload.

2. High Availability

Imagine the night is in full swing, the music is pumping, and everyone is having a great time but something minor happens – maybe a speaker malfunctions or a light goes out. In a good club, these issues are quickly addressed to ensure the party goes on without anyone even noticing;

This also translates to servers, where load balancers contribute to high availability. If one server experiences a malfunction or becomes overloaded, the load balancer redirects incoming traffic to other healthy servers, ensuring optimal user experience. Just as the owner invests in backup systems to keep the party alive, load balancing strategies maintain a consistent and reliable back-end.

3. Types of Load Balancing

Different music genres cater to diverse tastes on the dance floor, load balancers offer various algorithms to handle incoming requests. Remember the bouncer's role in managing the crowd? Here, they might employ different strategies to not let the users go away, such as directing people to the least crowded areas or alternating between multiple entrances.

Similarly, load balancing algorithms dictate how incoming requests are distributed across servers. The 'Round Robin' algorithm, for instance, ensures an equal share of the workload, much like guiding people to various sections of the club. On the other hand, the 'Least Connections' algorithm mimics the bouncer's approach of sending attendees to less crowded spots, ensuring each server remains balanced in handling requests.

4. Digital Security

By 4am, the bouncer still plays a crucial role in maintaining order, ensuring the club remains an enjoyable and secure environment as the night dies.

Just like that, load balancers enhance digital security by preventing server overloads, minimizing the risk of system crashes, and mitigating potential threats like Distributed Denial of Service (DDoS) attacks.

Much like a vigilant bouncer adeptly managing the crowd, load balancers act as the first line of defense against digital chaos. They monitor server health, optimize resource utilization, and swiftly respond to increased demand, guaranteeing a smooth digital dance floor experience.

Just as a skilled bouncer bounces a memorable night at the club, load balancers direct servers, ensuring scalability, high availability, and a secure, uninterrupted performance for users. So, whether you're at the hottest club in town or navigating the online realm, the role of the bouncer – or load balancer – is central to a successful and enjoyable experience.