<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: WeenAIthDev</title>
    <description>The latest articles on DEV Community by WeenAIthDev (@weenaithdev).</description>
    <link>https://dev.to/weenaithdev</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3910009%2F14b8ecb6-78b7-481d-9dbe-6f55ffcc6b45.jpg</url>
      <title>DEV Community: WeenAIthDev</title>
      <link>https://dev.to/weenaithdev</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/weenaithdev"/>
    <language>en</language>
    <item>
      <title>Before You Train the Model: What My First ML Project Taught Me About Data Cleaning</title>
      <dc:creator>WeenAIthDev</dc:creator>
      <pubDate>Sat, 25 Jul 2026 17:35:14 +0000</pubDate>
      <link>https://dev.to/weenaithdev/before-you-train-the-model-what-my-first-ml-project-taught-me-about-data-cleaning-16i3</link>
      <guid>https://dev.to/weenaithdev/before-you-train-the-model-what-my-first-ml-project-taught-me-about-data-cleaning-16i3</guid>
      <description>&lt;p&gt;When I started my first data analytics and machine learning project, I wanted to get straight to the exciting part: &lt;strong&gt;training a model&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;I downloaded a dataset from Kaggle, chose a basic classification model, trained it, and started making predictions.&lt;/p&gt;

&lt;p&gt;But the results were &lt;em&gt;strange&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;My predictions didn't look right, and metrics such as &lt;strong&gt;accuracy&lt;/strong&gt; and &lt;strong&gt;ROC-AUC&lt;/strong&gt; were much lower than I expected.&lt;/p&gt;

&lt;p&gt;My first thought was:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;Maybe I chose the wrong model.&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;But the problem started &lt;strong&gt;before the model&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;I hadn't properly understood or prepared my data.&lt;/strong&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  &lt;strong&gt;Going Back to the Data&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;Instead of immediately switching algorithms, I went back to the dataset.&lt;/p&gt;

&lt;p&gt;I started with:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;df.info()&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;This gave me an overview of my &lt;strong&gt;columns, their data types, and how many non-null values were present&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Next:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;df.isna().sum()&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;This showed me exactly &lt;strong&gt;where values were missing&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Then came an important lesson:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;&lt;em&gt;Finding missing data and deciding what to do with it are two different things.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Depending on the dataset and what the missing value represents, one option for a numerical column is to fill missing values using the &lt;strong&gt;median&lt;/strong&gt;:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;df["column"].fillna(df["column"].median())&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;The median can be useful because it is &lt;strong&gt;less affected by extreme values than the mean&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;For categorical data, one possible approach is the &lt;strong&gt;mode&lt;/strong&gt;:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;df["column"].fillna(df["column"].mode()[0])&lt;/code&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  &lt;strong&gt;Why &lt;code&gt;[0]&lt;/code&gt;?&lt;/strong&gt;
&lt;/h3&gt;

&lt;p&gt;Because &lt;code&gt;mode()&lt;/code&gt; returns a &lt;strong&gt;Series&lt;/strong&gt;. There can be more than one mode, and &lt;code&gt;[0]&lt;/code&gt; selects the &lt;em&gt;first result&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Sometimes, however, filling isn't appropriate at all.&lt;/p&gt;

&lt;p&gt;I also learned about:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;df.drop()&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;df.dropna()&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;And data cleaning wasn't only about &lt;strong&gt;missing values&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;I needed to make sure my columns had the &lt;strong&gt;appropriate data types&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;That's where tools such as these became useful:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;df.convert_dtypes()&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;df["column"].astype("string")&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;code&gt;pd.to_datetime(df["date"])&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;These commands weren't the exciting machine-learning model I originally wanted to build.&lt;/p&gt;

&lt;p&gt;But I started realizing something:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;&lt;em&gt;This was part of machine learning too.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  &lt;strong&gt;Garbage In, Garbage Out&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;The phrase &lt;strong&gt;&lt;em&gt;“garbage in, garbage out”&lt;/em&gt;&lt;/strong&gt; finally started making sense.&lt;/p&gt;

&lt;p&gt;A machine-learning algorithm doesn't automatically understand that a column has the wrong data type, that missing values have been handled poorly, or that the data doesn't represent what I think it represents.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The model learns from what I give it.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I had been worrying about finding a better algorithm when I hadn't properly prepared the &lt;strong&gt;foundation&lt;/strong&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  &lt;strong&gt;Then Genesis 1 Made Me Think About It Differently&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;While studying &lt;strong&gt;Genesis 1&lt;/strong&gt;, I had written something in my notes:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;&lt;em&gt;“Building up bit by bit.”&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Genesis 1 doesn't move immediately from a &lt;strong&gt;formless and empty earth&lt;/strong&gt; to the finished creation.&lt;/p&gt;

&lt;p&gt;There is progression.&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Light.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Separation.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Land.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Vegetation.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The lights in the sky.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Living creatures.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;And finally, humanity.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;There is &lt;strong&gt;order&lt;/strong&gt; to the chapter.&lt;/p&gt;

&lt;p&gt;Of course, training a machine-learning model isn't comparable to &lt;strong&gt;God's act of creation&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;But Genesis 1 made me reflect on something about &lt;em&gt;how I build&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;I had wanted this:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Dataset → Model → Prediction&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;But my project actually required:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Dataset → Understand → Clean → Prepare → Train → Evaluate&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;I was rushing toward the &lt;strong&gt;result&lt;/strong&gt; while overlooking the work that needed to happen first.&lt;/p&gt;




&lt;h2&gt;
  
  
  &lt;strong&gt;The Lesson I Took Away&lt;/strong&gt;
&lt;/h2&gt;

&lt;p&gt;At first, data cleaning felt like the boring part standing between me and &lt;em&gt;“real” machine learning&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;I don't see it that way anymore.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;&lt;em&gt;Preparation is part of building.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Sometimes progress means &lt;strong&gt;checking one column&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Finding one missing value.&lt;/p&gt;

&lt;p&gt;Correcting one data type.&lt;/p&gt;

&lt;p&gt;Understanding one problem.&lt;/p&gt;

&lt;p&gt;Then moving to the next.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Build carefully.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Build in order.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;em&gt;Build bit by bit.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>datascience</category>
      <category>python</category>
      <category>beginners</category>
    </item>
  </channel>
</rss>
