SQL Taught Me That There Is Always an Easier Way to Do Things

#basic #sql #analytics #luxdev

There is a particular kind of fatigue that only a data analyst knows. It is not simply the result of long hours, but the mental exhaustion of manually reviewing thousands of rows to find a single error, such as a misspelled category or inconsistent entry. This is where inefficient processes can become more draining than the work itself.

If you have experienced this, you already know where this is heading. If you have not, keep reading it may help you avoid hundreds of hours of unnecessary manual work.

The Manual Times

For a long time, my approach to data was largely based on trial and error. I collected the data, reviewed it manually, and spent significant time restructuring it. Filtering meant working through dropdowns one column at a time, cleaning involved manually scanning thousands of rows for errors and duplicates, and aggregation often depended on complex formulas that could easily break when the dataset changed.

I became proficient at working with messy datasets, to the point where people trusted me with the ones others avoided. However, being good at a manual process does not mean it is the most efficient approach. For years, I knew I was capable of handling the work, but I also knew there had to be a better way.

The truth is, I didn't feel the weight of that manual process until I compared it to what came after. You don't always know you're carrying something heavy until someone hands you a lighter version of the same load.

The Moment SQL Stopped Being Intimidating

I remember the first time I wrote a SQL query that solved a real problem. It was not perfect, but it returned exactly the records I needed, using three conditions that would have taken me nearly an hour to apply manually. The query did it in less than a second.

That was the moment I realized: I had been doing things the hard way.

That moment changed how I approached data. Instead of spending hours clicking through spreadsheets, I began opening the SQL editor and focusing on one question: What do I actually want to know? SQL handled the process of getting me there, allowing me to focus less on the mechanics and more on the insight.

Let Me Show You What I Mean

So let me walk you through a real kind of dataset I work with, a Tembo hotel booking records: booking_id, guest
_name, guest_phone, guest_city, guest_nationality, room_no, room _type, room_rate_per_night, check-in and check-out dates, payment method, staff_name, staff_department, staff_salary, booking_status,total_amount, service_used and guest_rating.
A few thousand rows collected over months and entered by different people will inevitably contain inconsistencies. If you work with operational data, you know that real-world datasets are rarely clean they are messy in ordinary but time consuming ways.

Cleaning thousands of records that used to take a full day now took a few well-written lines: standardizing inconsistent entries, catching nulls, flagging duplicates, all in one pass, all repeatable.
Filtering was no longer a scavenger hunt through dropdown menus. A WHERE clause could isolate exactly the records I needed, out of tens of thousands, instantly, and I could refine it as many times as I wanted without starting over.

--cleaning guest_name
update tembo_hotel.tembo_hotel_staging
set guest_name = initcap(trim(guest_name))
where guest_name is not null;

Aggregating stopped being a fragile web of formulas. GROUP BY and a handful of aggregate functions gave me summaries, totals, and averages that didn't break the moment the dataset grew.

select
    room_type,
    count(*) as total_bookings
from tembo_hotel.tembo_hotel_staging
group by room_type
order by total_bookings desc;

Transforming data, joining tables, reshaping records, building the exact structure a report needed, became a conversation with the data instead of a fight against it.

The work didn't get less important. It got less exhausting. And that distinction matters more than people admit.

Cleaning. The "Status" column should have four tidy values: Checked Out, Cancelled, No Show. Instead, some rows say "Checked Out" and others say "checked out," entered by someone in a hurry who didn't hit Shift. On a spreadsheet, you'd catch some of these with a filter, miss others, and never really be sure you got them all. In SQL, it's one statement:

update tembo_hotel.tembo_hotel_staging
set booking_status = 'Checked Out'
where lower(status) = 'checked out';

Every inconsistent entry, fixed, in one pass, no matter how many thousand rows are hiding behind it.

Filtering. Buried in the same dataset are a couple of bookings where the check-out date is recorded as earlier than the check-in date, a guest who apparently left two days before they arrived. Nobody enters that on purpose. It's a typo, a swapped date field, a rushed front-desk moment. On paper, or in a spreadsheet with a few thousand rows, that kind of error can sit there for months, quietly making your average length-of-stay figures wrong. A single filter surfaces it immediately:

SELECT booking_id, guest_name, check_in_date, check_out_date
FROM tembo_hotel.tembo_hotel_staging
WHERE check_out_date < check_in_date;

That query doesn't just find an error. It hands you a punch list, exactly which bookings to go back and fix, by name.

Transforming. The dataset also has a "Nights" column, filled in by hand, that's supposed to match the gap between check-in and check-out. Most of the time it does. But every so often you'll find a booking where the dates span two months and the nights column still says "2," because someone typed the wrong check-out date and the nights field was never recalculated. Instead of eyeballing every row to catch that, you let SQL do the comparison for you:

select booking_id, guest_name, check_in_date, check_out_date,
       nights as recorded_nights,
       (check_out_date - check_in_date) as actual_nights
from tembo_hotel.tembo_hotel_staging
where nights <> (check_out_date - check_in_date);

One query, and every mismatch between what was recorded and what actually happened is sitting right in front of you.

Aggregating. And then there's the question every manager eventually asks: which room type is actually making the money? Doing that by hand means sorting, subtotaling, and re-checking a pivot table every time new bookings come in, and hoping nobody left an amount field blank, because a blank cell breaks a SUM formula fast. SQL handles the gaps for you and gives you the answer in seconds:

select
    room_type,
    count(*) as total_bookings,
    sum(total_amount) as total_revenue,
    round(avg(total_amount), 2) as avg_booking_value
from tembo_hotel.tembo_hotel_staging
group by room_type
order by total_revenue desc;

What It Actually Changed

Learning SQL after years of doing things manually teaches you that it does more than save time it gives you back your attention. By reducing the mental effort spent on repetitive tasks, you have more capacity to focus on the data, identify meaningful patterns, and uncover insights that manual processes can easily hide.

SQL also changed how I viewed mistakes. Manual errors could easily go unnoticed until they were identified downstream, while queries could be tested, rerun, and corrected quickly. My confidence in my work increased not because I became more careful, but because SQL made accuracy and consistency easier to achieve.

The Bigger Lesson

If I'm honest, this isn't really an article about SQL. Not entirely.

It is about realizing that doing something the hard way is not a badge of honor, it simply means you have not yet discovered a better approach. For me, SQL was that turning point. For anyone doing repetitive, high-volume work, there is likely a more efficient tool, shortcut, or way of approaching the problem waiting to be discovered.

If you are still relying on manual filters and checking rows one by one to find errors, you are not slow or behind you simply have not found the more efficient approach yet. Once you discover it, you will not only work faster but also think more clearly, with less energy spent on repetitive tasks.

That's what SQL taught me. Not just how to write a query. How to keep asking, every time something feels unnecessarily hard: is there an easier way to do this?

There usually is a simpler, more efficient way to approach the problem.