Most analysis that goes wrong goes wrong in the first ten minutes, before anyone writes a single chart. A date column was read as text. A survey used minus one to mean no answer and the average quietly dropped. Two rows exist for every customer because of a join nobody checked. None of these are hard problems. They are easy to catch and almost impossible to notice later, once a dashboard is built on top of them and the number already looks reasonable.
The habit that separates analysts who get trusted from those who get corrected is simple. Before building anything, they spend the first hour getting to know the data on its own terms. Not exploring for insights yet, just establishing what they are holding and where it might lie to them. This article is that first hour, as a routine you can run on any dataset, in any tool, every single time.
Step one: get the shape
How many rows, how many columns, and what is one row. That last question is the most important one in analytics and the one beginners skip. Is a row a customer, an order, a customer-month, or a line item on an order? Everything you compute later depends on the answer, and getting it wrong is how a total ends up counted five times.
Start by printing the dimensions and reading the first and last handful of rows. The head shows you the columns and a sample of values. The tail often reveals a surprise the head hides: a stray total row at the bottom, a block of blank rows, a change in format partway down a file that was stitched together from several exports. Then say out loud, in one sentence, what a single row represents. If you cannot, that is the first thing to resolve before anything else.
Step two: check the types
Look at what type each column actually is, not what you assume it is. The classic trap is a date stored as text, which sorts alphabetically so that the tenth of a month lands before the second, and which cannot be used for any time calculation until it is converted. Just as common is a number stored as text because one value somewhere contains a comma, a currency symbol, or the word "null", which forces the whole column to be read as strings.
In pandas, df.dtypes and df.info() tell you this in seconds. A column you expect to be numeric that shows up as object is a flag: something non-numeric is hiding in it. Find that value before you convert, because it usually tells you something real about how the data was collected.
Step three: find what is missing
Count the blanks in every column. A column that is ten percent empty is a different problem from one that is ninety percent empty, and both are different from a column where the missing values are not blank at all but coded. Watch for the sentinel values that pretend to be data: zero where zero is impossible, minus one, 9999, a date like 1900-01-01, or the literal text "N/A". These slip past a blank-count and poison an average silently, because to the tool they look like ordinary numbers.
The question is not only how much is missing but whether the missingness has a pattern. If every blank row comes from one region or one month, that is a data collection story, not random noise, and it will bias any conclusion you draw. The counting comes first, and it comes from you.
Step four: sanity-check the numbers
For every numeric column, look at the minimum, the maximum, and a few points in between. You are not looking for insight yet. You are asking whether the values are physically possible. An age of 200, a negative quantity sold, a percentage above one hundred, a price of zero on a product that clearly sold: each of these is the data telling you how it was really recorded, and each one would quietly distort a chart you built without looking.
A quick summary of each column, the kind df.describe() produces, puts the extremes in front of you. When a maximum looks absurd, chase down the row it came from. Sometimes it is a genuine outlier worth keeping. Often it is a data entry error, a unit mismatch, or a placeholder, and knowing which changes everything you do next.
Step five: look for duplicates
Check whether rows repeat when they should not. This connects straight back to step one: if one row is supposed to be one customer, then two rows with the same customer id is a problem, and it is usually the fingerprint of a join that matched more rows than you expected. Duplicate rows inflate counts and totals in a way that is invisible on the surface, because each duplicated value looks perfectly legitimate on its own.
Count the distinct values of whatever is supposed to be unique and compare that to the row count. If they do not match, you have either duplicates to remove or a wrong idea of what one row means. Resolve that now, because every sum and every average downstream depends on it.
Step six: write down what you found
The last step costs five minutes and saves a week. Write a short note of what you learned: what one row is, which columns you had to convert, where the missing values are and how they are coded, which outliers you decided to keep or drop and why. This is not bureaucracy. It is the difference between an analysis someone can trust and one you cannot even reconstruct yourself in a month.
These notes are also what make your work reviewable. When a stakeholder asks why a number looks the way it does, the answer is in your notes, not in your memory. Analysts who keep this habit build a quiet reputation for being right, because their results come with a paper trail that holds up when someone pushes on them.
The short version
- Get the shape and say in one sentence what a single row is
- Check every column's type, especially dates and numbers read as text
- Count missing values and catch the ones coded as fake numbers
- Sanity-check ranges for values that are not physically possible
- Look for duplicates against whatever should be unique
- Write down what you found so the work is trustworthy and reviewable
In the first hour you are not asking what this data says. You are asking whether you can trust what it says. Those are different jobs, and doing the second one first is what makes the first one safe.
Originally published on the EDUSHARK TRAINING blog.
Top comments (1)
The check on what a single row represents in step one saves so much pain later. We see this constantly in financial disclosure data, where a row looks like a company-year until you notice a restatement or a segment breakdown split across duplicate dates.
One check I would add right between step one and step two for tabular exports is scale and unit multipliers. In financial reports and government data, a table header will quietly say "in thousands" or "in millions", while the column itself just holds bare numbers like 1,450. If you join an unscaled file with a scaled one without an explicit multiplier assertion, every sum downstream is off by three or six orders of magnitude while looking completely plausible on a quick scan. Checking for unit markers in the metadata or verifying that known totals match the printed table scale is well worth five minutes.