My 5-Day Journey into Machine Learning: From Data Cleaning to My First ML Models
Learning goal: This is not a collection of code snippets to memorize. It is a beginner-friendly guide to understanding why each step in a typical machine-learning workflow exists, what problem it solves, and how the pieces fit together.
A note about the source material
This guide was built primarily from the four Jupyter notebooks you provided:
11_Aug_mpg.ipynb12-Aug-2.ipynb12th-Aug-Telco-Customer-churn.ipynb13-Aug.ipynb123
Your PDF notes were also provided, but the PDF is image-based and its text could not be reliably extracted. So I have not invented or silently reconstructed material from the PDF. The explanations below are grounded mainly in the notebooks, with general ML knowledge added to explain the concepts behind the code.
1. Overview: What did I actually learn in these five days?
At first glance, machine learning can look like this:
Load dataset
↓
Clean data
↓
Create graphs
↓
Train model
↓
Check accuracy
↓
Done!
But that misses the most important part.
The real ML workflow is closer to:
Business / real-world question
↓
Define what we want to predict
↓
Understand the data
↓
Clean and prepare the data
↓
Explore the data visually
↓
Choose useful features
↓
Convert data into numbers
↓
Split into training and testing data
↓
Choose an appropriate ML algorithm
↓
Train the model
↓
Evaluate it on unseen data
↓
Compare models
↓
Improve preprocessing / model settings
↓
Use the model to make predictions
The important insight is:
Machine learning is not mainly about choosing an algorithm. It is about turning a real-world question and messy data into a reliable prediction process.
Your notebooks actually demonstrate most of this workflow.
2. What is Artificial Intelligence?
Artificial Intelligence (AI) is the broad idea of building systems that can perform tasks that normally require some form of human intelligence.
Examples:
- recognizing an image
- understanding language
- recommending a movie
- detecting fraud
- predicting whether a customer will leave
- predicting the price/value of something
- generating text or images
A useful mental model is:
Artificial Intelligence
│
├── Machine Learning
│ │
│ ├── Supervised Learning
│ ├── Unsupervised Learning
│ └── Reinforcement Learning
│
├── Deep Learning
│
└── Generative AI
This is simplified because these areas overlap, but it is a useful beginner mental model.
3. What is Machine Learning?
Machine learning is a way of building systems where the computer learns patterns from data instead of us manually writing every rule.
Imagine we want to predict whether a telecom customer will churn.
A traditional rule-based approach might look like:
IF customer has a month-to-month contract
AND monthly charges are high
AND tenure is low
THEN churn = yes
The problem is that we have to invent those rules ourselves.
With machine learning, we give the algorithm historical examples:
Customer information → Actual churn result
Customer A → No
Customer B → Yes
Customer C → No
Customer D → Yes
...
The algorithm tries to learn the relationship between the input information and the known outcome.
Then we can give it a new customer:
New customer information
↓
Model
↓
Predicted churn
That is the core idea of supervised machine learning.
4. The most important ML vocabulary
Before learning algorithms, understand these words.
4.1 Dataset
A dataset is a collection of examples.
In your notebooks, a dataset is loaded into a Pandas DataFrame.
For example:
df = pd.read_csv("Telco_Customer_Churn.csv")
Think of the DataFrame as a spreadsheet.
4.2 Row / observation / sample
Each row represents one example.
For the Telco dataset:
one row = one customer
For the Auto MPG dataset:
one row = one car
For the insurance dataset:
one row = one customer/policy record
These words are often used interchangeably:
- row
- observation
- sample
- instance
4.3 Feature
A feature is an input variable that the model can use to make a prediction.
For customer churn:
tenure
MonthlyCharges
Contract
InternetService
PaymentMethod
...
These are features.
A useful question to ask is:
"What information is available to the model before it makes its prediction?"
That information is generally your feature set.
4.4 Target
The target is what we are trying to predict.
Examples from your notebooks:
Telco
Target = Churn
The model predicts:
0 → No churn
1 → Churn
Insurance
Target = Customer Lifetime Value
The model predicts a numerical value.
4.5 X and y
You repeatedly used:
X = df.drop(["Churn"], axis=1)
Y = df["Churn"]
This is one of the most important patterns in supervised ML.
X = inputs/features
Y = target/output
For example:
X
├── tenure
├── MonthlyCharges
├── Contract
├── InternetService
└── ...
Y
└── Churn
Think:
X = what the model knows
Y = what we want the model to learn to predict
5. The three major types of Machine Learning
5.1 Supervised Learning
In supervised learning, we have examples where the correct answer is already known.
Input X → Known answer Y
The model learns the relationship between X and Y.
Your notebooks mainly use supervised learning.
There are two major supervised learning tasks.
Classification
The target is a category.
Examples:
Spam / Not Spam
Churn / No Churn
Fraud / Not Fraud
Disease / No Disease
Your Telco project is a classification problem.
Regression
The target is a numerical value.
Examples:
House price
Temperature
Sales
Customer Lifetime Value
Fuel efficiency
Your Customer Lifetime Value project is a regression problem.
6. Classification vs Regression
This distinction is extremely important.
Classification
Question:
"Which category does this example belong to?"
Example:
Customer → Churn or No Churn
Possible output:
0
1
Common classification algorithms you tried:
- Logistic Regression
- Support Vector Classifier (SVC)
- Decision Tree Classifier
Regression
Question:
"What numerical value should we predict?"
Example:
Customer → Customer Lifetime Value = 6,543.21
Common regression algorithms you tried:
- Linear Regression
- KNN Regressor
- SVR
- Decision Tree Regressor
- Bagging Regressor
- AdaBoost Regressor
- Random Forest Regressor
7. Unsupervised Learning
In unsupervised learning, we do not have a known target.
Instead, the algorithm tries to discover structure in the data.
For example:
Customer data
↓
Find groups
↓
Group 1: high-value customers
Group 2: price-sensitive customers
Group 3: new customers
Common examples:
- K-Means clustering
- Hierarchical clustering
- PCA
You did not build an unsupervised-learning project in these notebooks, but it is important to know where it fits.
8. Reinforcement Learning
Reinforcement learning is different.
An agent interacts with an environment and receives rewards or penalties.
Agent
↓ action
Environment
↓
Reward / penalty
↓
Agent learns
Examples:
- game-playing agents
- robotics
- certain recommendation/control systems
This was not part of your five-day practical work.
9. Different ways to approach an ML problem
A useful way to think about ML is not:
"Which algorithm should I use?"
Instead ask:
Step 1 — What is the question?
For example:
Can I predict whether a telecom customer will churn?
Step 2 — What type of target do I have?
Churn → category → Classification
Customer Lifetime Value → number → Regression
Step 3 — What data is available?
Ask:
- What columns exist?
- Which are numerical?
- Which are categorical?
- Are values missing?
- Are there strange values?
- Are some columns identifiers?
- Is the target balanced?
Step 4 — Explore before modeling
Use statistics and graphs.
Step 5 — Prepare the data
Convert it into a form the algorithm can understand.
Step 6 — Establish a baseline
Train a simple model first.
Step 7 — Compare alternatives
Try other appropriate models.
Step 8 — Evaluate on unseen data
A model is useful only if it works beyond the data it memorized.
10. Google Colab: Why did we use it?
Google Colab gives you a cloud-based Jupyter Notebook environment.
Instead of installing everything locally, you can run Python code in the browser.
A notebook combines:
Explanation
+
Python code
+
Output
+
Graphs
This makes it very useful for learning ML.
Your notebooks use paths such as:
/content/auto-mpg.csv
That is typical of a Colab environment.
A normal learning workflow is:
Upload dataset
↓
Open notebook
↓
Run cells
↓
Inspect output
↓
Change code
↓
Run again
11. The Python libraries you used
11.1 NumPy
import numpy as np
NumPy provides numerical operations and arrays.
You used it for things such as:
np.nan
np.log1p()
np.expm1()
np.nan
Represents a missing numerical value.
np.log1p(x)
Calculates:
log(1 + x)
It is useful when a target is heavily skewed.
np.expm1(x)
Reverses log1p:
exp(x) - 1
You used this later to convert log predictions back to the original CLV scale.
12. Pandas
import pandas as pd
Pandas is one of the most important Python libraries for data work.
Think of a Pandas DataFrame as a spreadsheet that Python can manipulate.
You used it to:
- load CSV files
- inspect data
- remove columns
- convert data types
- detect missing values
- fill missing values
- select X and Y
- create encoded columns
- save predictions
Example:
df = pd.read_csv("Telco_Customer_Churn.csv")
13. Matplotlib
import matplotlib.pyplot as plt
Matplotlib is a general-purpose plotting library.
It provides the foundation for many Python visualizations.
You used it for:
plt.figure()
plt.title()
plt.show()
14. Seaborn
import seaborn as sns
Seaborn makes statistical visualizations easier to create.
You used:
countplotboxplothistplotpairplotscatterplotheatmap
A key lesson:
A graph is not decoration. It is a tool for asking questions about the data.
15. Scikit-learn
Scikit-learn is the main ML library used in your notebooks.
You used it for:
- splitting data
- preprocessing
- encoding
- scaling
- classification
- regression
- evaluation
The basic pattern is:
model = SomeModel()
model.fit(X_train, Y_train)
predictions = model.predict(X_test)
This pattern appears again and again in ML.
16. The most important workflow: EDA
EDA means:
Exploratory Data Analysis
EDA is the process of investigating the dataset before building the final model.
Your notebooks use:
df.head()
df.shape
df.describe()
df.info()
df.sample()
df.isnull().sum()
plus graphs.
These are not random commands.
They answer different questions.
17. df.head() — What does the data look like?
You used:
df.head()
It displays the first few rows.
Why?
Because before doing anything else, you want to see:
- column names
- example values
- obvious data problems
- whether the data loaded correctly
Think of it as opening the box before using what is inside.
18. df.shape — How much data do I have?
You used:
df.shape
For Auto MPG:
(398, 9)
That means:
398 rows
9 columns
Why does this matter?
Because dataset size affects:
- model choice
- computation time
- confidence in results
- risk of overfitting
19. df.info() — What types of data do I have?
You used:
df.info()
This tells you:
- column names
- number of non-null values
- data types
- memory usage
This is extremely important.
For example, in Auto MPG:
horsepower → object
At first glance horsepower sounds numerical.
But Pandas sees it as text.
Why?
Because the dataset contains values such as:
?
A column containing:
130
165
150
?
cannot be treated as a clean numerical column.
So:
df.info()helps you detect data-type problems before modeling.
20. df.describe()
You used:
df.describe()
and sometimes:
df.describe(include="all")
This gives summary statistics.
For numerical data you get things such as:
- count
- mean
- standard deviation
- minimum
- 25th percentile
- median
- 75th percentile
- maximum
This helps you understand the scale and distribution of variables.
For example:
minimum
↓
25%
↓
median
↓
75%
↓
maximum
A large difference between the 75th percentile and maximum can be a clue that there may be extreme values.
21. df.sample()
You used:
df.sample(5)
or similar.
This shows random rows.
Why?
Because the first rows may not represent the whole dataset.
Random samples can expose:
- unusual values
- formatting issues
- unexpected categories
- data-entry problems
22. Missing data
You used:
df.isnull().sum()
This asks:
"How many missing values are there in each column?"
Missing data is important because many ML algorithms cannot directly work with missing values.
But there is a subtle lesson from your Auto MPG notebook.
You first checked:
df.isnull().sum()
while horsepower still contained "?".
Pandas did not count "?" as a missing value because "?" is a string, not a true NaN.
That means:
"?" ≠ NaN
So data cleaning sometimes requires identifying fake missing values first.
23. Auto MPG: cleaning horsepower
Your notebook did:
df["horsepower"] = df["horsepower"].replace("?", np.nan)
This converts the placeholder into a real missing value.
Then:
df["horsepower"] = pd.to_numeric(df["horsepower"])
Now Pandas can treat horsepower as numerical data.
This is a very important data-cleaning pattern:
messy text
↓
identify invalid placeholder
↓
convert to NaN
↓
convert column to numeric
↓
handle missing values
24. Why use the median?
You calculated:
median_horsepower = df["horsepower"].median()
and then filled missing values:
df["horsepower"] = df["horsepower"].replace(
np.nan,
median_horsepower
)
Why median?
Because the median is less affected by extreme values than the mean.
Example:
10, 11, 12, 13, 1000
The mean is pulled heavily upward by 1000.
The median is much more representative of the middle of the data.
A useful rule of thumb:
- fairly symmetric data → mean may work well
- skewed data / outliers → median is often safer
But remember:
There is no universal rule that "median is always correct."
The best treatment depends on the data and the problem.
25. Dropping an identifier
You considered dropping:
df.drop("car name", axis=1)
and actually dropped customerID in the Telco work.
Why?
An identifier is usually not a meaningful predictive feature.
For example:
customerID = 759832
does not inherently mean the customer is more or less likely to churn.
The ID exists to identify the row, not to describe the customer.
Important distinction:
A column can be useful for identifying a record without being useful for predicting the target.
26. But don't blindly drop columns
This is an important improvement to your original reasoning.
A column should not be dropped merely because it is an object/string.
For example:
Contract
InternetService
PaymentMethod
are categorical strings, but they can contain very useful predictive information.
So the correct question is:
"Does this column contain useful information that is available at prediction time?"
not:
"Is this column numeric?"
27. Understanding your graphs
You created many graphs. The important thing is to understand what question each graph answers.
28. Countplot
Example:
sns.countplot(x="Churn", data=df)
A countplot answers:
"How many observations are in each category?"
For churn:
No churn → number of customers
Churn → number of customers
This immediately helps you understand class balance.
If the graph looks like:
No Churn ███████████████████
Churn ███████
the classes are imbalanced.
That matters because a model could achieve high accuracy simply by favoring the majority class.
29. Countplot with hue
You used:
sns.countplot(
x="Contract",
hue="Churn",
data=df
)
This answers:
"How does the target category vary across another category?"
For example:
Contract type
↓
Churn / No churn
If month-to-month customers show much more churn than two-year customers, that is an important pattern worth investigating.
But remember:
A graph showing association does not automatically prove causation.
30. Histogram
You used:
df.hist(figsize=(10,10))
and:
sns.histplot(df["MonthlyCharges"], kde=True)
A histogram shows the distribution of a numerical variable.
It helps answer:
- Where are most values?
- Is the data symmetric?
- Is it skewed?
- Are there multiple groups?
- Are there extreme values?
Example:
frequency
^
| ███
| ███████
| █████████
| ███████████
+-----------------> value
31. Why distributions matter
Suppose a feature looks like:
Most values: 0–100
A few values: 10,000
That may indicate:
- genuine outliers
- a different population
- data-entry errors
- a highly skewed distribution
A model may react differently depending on the algorithm.
This is one reason EDA happens before modeling.
32. Boxplot
You used:
sns.boxplot(y=df["Income"])
A boxplot summarizes a distribution.
Conceptually:
outlier
•
|
┌───────┐
│ │
------│ box │------
│ │
└───────┘
|
outlier
The box represents the middle portion of the data.
The line inside the box is the median.
Points outside the whiskers may be treated as potential outliers.
Important:
An outlier is not automatically an error.
A high-income customer may be perfectly legitimate.
33. Boxplot: target vs category
You used:
sns.boxplot(
x="Coverage",
y="Customer Lifetime Value",
data=df
)
This is a powerful graph.
It asks:
"How does the distribution of Customer Lifetime Value differ between categories?"
For example:
Coverage A → CLV distribution
Coverage B → CLV distribution
Coverage C → CLV distribution
Compare:
- median
- spread
- outliers
- overlap
This can help you identify potentially useful relationships.
34. Scatterplot
You used:
sns.scatterplot(
x="Income",
y="Customer Lifetime Value",
data=df
)
A scatterplot examines the relationship between two numerical variables.
Each dot represents an observation.
It helps answer:
"As X changes, does Y appear to change?"
Possible patterns:
Positive relationship:
•
•
•
•
Negative relationship:
•
•
•
•
No obvious relationship:
• •
•
• •
•
The pattern does not have to be a straight line.
35. Pairplot
You used:
sns.pairplot(...)
A pairplot shows many numerical relationships at once.
It is useful for:
- spotting correlations
- identifying clusters
- seeing distributions
- finding obvious relationships
- detecting possible outliers
The downside is that it becomes difficult to read when there are many columns.
So:
Pairplot is excellent for small-to-medium exploratory datasets, but not something you blindly run on every large dataset.
36. Correlation heatmap
You used:
sns.heatmap(
df[["tenure", "MonthlyCharges", "TotalCharges"]].corr(),
annot=True
)
Correlation measures how two numerical variables move together in a linear relationship.
A correlation close to:
+1 → strong positive linear relationship
0 → little/no linear relationship
-1 → strong negative linear relationship
For example:
tenure ↑
TotalCharges ↑
could result in positive correlation.
But correlation has an important limitation:
Correlation does not prove causation.
Also, zero correlation does not necessarily mean "no relationship" because the relationship might be non-linear.
37. A key lesson from visualization
The graphs answer different questions:
| Graph | Main question |
|---|---|
| Countplot | How many observations are in each category? |
| Countplot + hue | How does one category vary with another? |
| Histogram | What does a numerical distribution look like? |
| Boxplot | What are the median, spread and potential outliers? |
| Boxplot + category | How does a numerical distribution differ by category? |
| Scatterplot | How do two numerical variables relate? |
| Pairplot | What relationships exist among several numerical variables? |
| Heatmap | How strongly are numerical variables linearly correlated? |
The goal is not:
"Create lots of graphs."
The goal is:
"Use the right graph to answer the right question."
38. Encoding: why does ML need it?
Many ML algorithms work with numbers.
But real-world data contains:
Male
Female
Yes
No
Month-to-month
One year
Two year
Fiber optic
DSL
No
The model needs these values represented numerically.
This is called encoding.
39. Label Encoding
You used:
from sklearn.preprocessing import LabelEncoder
and encoded categorical columns.
For a binary column:
No → 0
Yes → 1
This can be reasonable for a binary variable.
For example:
Churn:
No → 0
Yes → 1
That makes intuitive sense.
40. Why LabelEncoder can be dangerous for multi-category variables
Suppose:
Contract:
Month-to-month
One year
Two year
Label encoding might produce:
Month-to-month → 0
One year → 1
Two year → 2
A numerical model may interpret that as:
Two year > One year > Month-to-month
That ordering may not be appropriate for the algorithm.
The numbers are just codes.
They do not automatically mean that category 2 is "twice" category 1.
For nominal categories, one-hot encoding is usually safer.
41. One-hot encoding
You used:
pd.get_dummies(df, drop_first=True)
This transforms categories into separate binary columns.
For:
InternetService:
DSL
Fiber optic
No
we might create columns such as:
InternetService_Fiber optic
InternetService_No
with the remaining category represented by both values being 0.
This avoids inventing a fake numerical ordering.
42. Why drop_first=True?
If a categorical variable has several categories, using all one-hot columns can introduce redundancy in some models.
Dropping one category creates a reference category.
For example:
DSL
Fiber optic
No
could become:
Fiber optic
No
Then:
Fiber optic = 0
No = 0
means the reference category:
DSL
For beginner understanding, the key idea is:
One-hot encoding converts categories into machine-readable indicator variables without pretending the categories have numerical order.
43. Train/test split
You repeatedly used:
train_test_split(
X,
Y,
test_size=0.3
)
This is one of the most important ML concepts.
Suppose we have 1,000 examples.
We might use:
700 → training
300 → testing
The training set is used to learn the model.
The test set is held back to evaluate how the trained model performs on unseen data.
Think of it like an exam:
Training data = practice questions
Test data = unseen exam questions
44. Why not train and test on the same data?
Because the model could simply memorize the training examples.
Suppose:
Training score = 99%
Test score = 65%
That is a warning sign.
The model learned the training data extremely well but does not generalize well.
This is called:
Overfitting
45. Underfitting
The opposite can happen.
If the model is too simple:
Training score = 60%
Test score = 58%
It may not have learned enough from the data.
This is:
Underfitting
A useful mental picture:
Underfitting
Model too simple
↓
misses important patterns
Good fit
Learns useful patterns
↓
works on unseen data
Overfitting
Model learns noise/details
↓
great training performance
poor unseen performance
46. Why random_state matters
In one Telco notebook you used:
train_test_split(X, Y, test_size=0.3)
In the improved notebook you used:
random_state=42
A random state makes the split reproducible.
Without it, the random split can change between runs.
That means your score may change.
With:
random_state=42
you can reproduce the same split.
The number 42 is not magical.
Any fixed integer can serve this purpose.
47. Why stratify=Y is useful for classification
In your improved Telco notebook you used:
stratify=Y
This helps preserve the class distribution between training and testing data.
For example, if the full dataset contains:
73% No Churn
27% Churn
stratification aims to keep roughly the same proportion in both splits.
This is particularly useful when classes are imbalanced.
48. Logistic Regression
You used:
from sklearn.linear_model import LogisticRegression
model_lr = LogisticRegression()
model_lr.fit(X_train, Y_train)
Despite its name, Logistic Regression is commonly used for classification.
Its goal is to estimate the probability of belonging to a class.
For binary classification:
features
↓
Logistic Regression
↓
probability
↓
class
Example:
Probability of churn = 0.82
The model may classify that customer as:
Churn
depending on its decision threshold.
49. Why Logistic Regression is a good beginner model
It is useful because:
- relatively simple
- fast
- often strong as a baseline
- easier to interpret than many complex models
- naturally suited to binary classification
A good ML habit is:
Start with a simple baseline before reaching for complex models.
50. Support Vector Machine / SVC
You used:
from sklearn.svm import SVC
model_lr = SVC()
The variable name model_lr is misleading here.
It is actually an SVC model.
SVC means:
Support Vector Classifier
The basic idea is to find a decision boundary that separates classes.
In simple 2D data:
Class A: ● ● ●
--------- decision boundary ---------
Class B: ▲ ▲ ▲
SVM tries to find a boundary with a useful margin between classes.
51. Why scaling matters for SVM
SVM is sensitive to feature scale.
Suppose one feature ranges from:
0–1
and another ranges from:
0–100,000
The larger-scale feature can dominate distance-related calculations.
That is why scaling is often important for:
- SVM
- KNN
- Logistic Regression in many situations
- neural networks
52. Decision Tree
You used:
DecisionTreeClassifier()
and later:
DecisionTreeRegressor()
A decision tree makes predictions using a sequence of questions.
Imagine:
Is tenure < 12?
/ \
Yes No
/ \
Is charge > X? ...
Eventually the tree reaches a leaf containing a prediction.
This is attractive because it is easy to visualize conceptually.
53. Why trees can overfit
A tree can keep splitting the data until it creates extremely specific rules.
For example:
If feature A > 10
and feature B < 4
and feature C = 7
and feature D > 92
...
Eventually the tree may memorize training examples.
That is why you later tried:
max_depth=6
min_samples_split=10
min_samples_leaf=5
These are ways of controlling tree complexity.
54. Random Forest
You used:
RandomForestRegressor(
n_estimators=200,
max_depth=8,
min_samples_leaf=5,
random_state=42
)
A random forest combines many decision trees.
Instead of relying on one tree:
Tree 1 ─┐
Tree 2 ─┤
Tree 3 ─┤
Tree 4 ─┤ → combined prediction
... ┘
The idea is that many different trees can produce a more robust prediction than a single tree.
This is an example of an ensemble method.
55. Bagging
You used:
BaggingRegressor(
n_estimators=100,
max_samples=0.7,
max_features=0.7
)
Bagging means combining models trained on different samples of the data.
The overall idea:
Dataset
↓
Different samples
↓
Multiple models
↓
Combine predictions
The goal is often to reduce variance and make predictions more stable.
Random Forest can be viewed as a specialized tree-based ensemble that adds additional randomness in how trees are built.
56. AdaBoost
You used:
AdaBoostRegressor()
AdaBoost means:
Adaptive Boosting
Instead of training many independent models and simply averaging them, boosting builds models sequentially.
Conceptually:
Model 1
↓
find mistakes
↓
Model 2 focuses more on difficult cases
↓
Model 3 focuses further
↓
combine models
The key idea:
Later models try to improve on the weaknesses of earlier models.
57. K-Nearest Neighbors (KNN)
You used:
KNeighborsRegressor()
KNN predicts using nearby examples.
Imagine a new customer.
The algorithm asks:
"Which existing customers are most similar to this customer?"
Then it uses those neighbors to estimate the output.
For regression, the prediction is commonly based on the neighbors' target values.
58. Why KNN needs scaling
KNN relies on distance.
Imagine:
Income: 0–100,000
Age: 18–80
If raw values are used, income can dominate the distance calculation.
Scaling puts features onto comparable scales.
That is why you experimented with:
StandardScaler()
59. StandardScaler
You used:
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train_log)
X_test_scaled = scaler.transform(X_test_log)
This is an important pattern.
Notice:
scaler.fit_transform(X_train)
but:
scaler.transform(X_test)
Why?
The scaler learns the training data's statistics.
Then the exact same transformation is applied to the test data.
You should not independently fit the scaler on the test set.
Otherwise information from the test set leaks into preprocessing.
This idea is called avoiding:
Data leakage
60. Data leakage
Data leakage happens when information that should not be available to the model during training accidentally influences the model.
Example:
Train preprocessing
↓
uses information from test data
↓
test is no longer truly unseen
That can make evaluation look better than real-world performance.
The correct principle is:
Fit preprocessing steps using training data, then apply them to validation/test/new data.
A production ML pipeline should ideally bundle preprocessing and modeling together.
61. Linear Regression
You used:
LinearRegression()
for Customer Lifetime Value.
Linear regression tries to model a relationship like:
Y = b0 + b1X1 + b2X2 + ...
For one feature:
Y = intercept + slope × X
The goal is to find coefficients that produce predictions close to the observed values.
62. Regression score: what does .score() mean?
You used:
model.score(X_train, Y_train)
model.score(X_test, Y_test)
For the scikit-learn regression models you used, .score() generally returns R², the coefficient of determination.
Very roughly:
R² = 1
means the model explains the observed variation perfectly on that dataset.
R² = 0
means the model is no better than a simple baseline based on predicting the mean target.
R² can also be negative on unseen data.
Important:
R² is not the same thing as accuracy.
For regression, do not say:
"My regression model has 90% accuracy"
just because R² is 0.90.
Instead say:
"The model achieved an R² of 0.90."
63. Classification accuracy
You used:
accuracy_score(Y_test, y_test_pred)
Accuracy is:
correct predictions
-------------------
total predictions
For example:
90 correct
100 total
gives:
90% accuracy
Accuracy is easy to understand.
But it can be misleading when classes are highly imbalanced.
64. Confusion matrix
Your later notes/workflow refer to concepts such as:
TP
TN
FP
FN
A confusion matrix organizes classification predictions.
Actual
No Yes
Predicted No TN FN
Predicted Yes FP TP
Meaning:
True Positive
Model predicted positive and it was positive.
True Negative
Model predicted negative and it was negative.
False Positive
Model predicted positive but it was negative.
False Negative
Model predicted negative but it was positive.
This is much more informative than accuracy alone when the cost of errors differs.
65. Precision and Recall
For classification:
Precision
Of everything the model predicted as positive:
How many were actually positive?
Precision = TP / (TP + FP)
Recall
Of everything that was actually positive:
How many did the model find?
Recall = TP / (TP + FN)
The right metric depends on the problem.
For churn prediction, for example, missing a customer who is about to churn may be more important than contacting an extra customer who would have stayed.
66. Your Telco Customer Churn project
This is probably the clearest classification example in your notebooks.
The business question is:
Can we predict whether a telecom customer will churn?
The target is:
Y = df["Churn"]
Features are:
X = df.drop("Churn", axis=1)
The workflow becomes:
Customer data
↓
Clean data
↓
Explore data
↓
Encode categories
↓
Split train/test
↓
Train classifier
↓
Predict churn
↓
Evaluate
67. Telco: removing customerID
You did:
df = df.drop(columns=["customerID"])
This is reasonable because the ID is an identifier rather than a customer characteristic.
You also commented that it could be kept separately if you later wanted to map predictions back to customers.
That is a good practical idea.
In production:
customerID
↓
keep separately for business reporting
but:
customerID
X
do not necessarily use as a model feature
68. Telco: fixing TotalCharges
You used:
df["TotalCharges"] = pd.to_numeric(
df["TotalCharges"],
errors="coerce"
)
errors="coerce" means invalid values are converted to NaN.
Then you used:
fillna(0)
in one notebook and median imputation in another.
These are two different decisions.
The important lesson is:
Missing-value handling should be based on the meaning of the data, not simply on whichever replacement is easiest.
For example, if missing TotalCharges occurs because a customer has zero tenure and has just joined, then filling with zero may have a meaningful business interpretation.
If missingness represents an unknown measurement, median imputation may be more appropriate.
69. Telco: the first modeling version
The earlier Telco notebook used:
LabelEncoder()
for many categorical columns.
This is okay as a learning exercise for understanding encoding, but it is not ideal for every categorical variable.
A better version in your later notebook used:
pd.get_dummies(
df,
columns=multi_cols,
drop_first=True
)
That is a meaningful improvement.
70. Telco: your improved preprocessing pipeline
The later notebook did something closer to:
Raw data
↓
Drop customerID
↓
Convert TotalCharges
↓
Encode binary columns
↓
One-hot encode multi-category columns
↓
Separate X and Y
↓
Scale X
↓
Train Logistic Regression
↓
Create Gradio interface
This is much closer to a real ML application.
71. Gradio: turning a model into an application
You used:
import gradio as gr
and built a UI.
This is an important step because it changes the project from:
Notebook experiment
to:
User input
↓
Preprocessing
↓
Model
↓
Prediction
↓
Human-readable result
For example:
Gender: Female
Tenure: 12 months
Contract: Month-to-month
Monthly charges: $70
...
↓
Predict Churn
↓
High Churn Risk
Probability: ...
This demonstrates an important ML engineering concept:
A model is useful only when it can be integrated into a workflow where people or systems can use its predictions.
72. Very important: preprocessing must match training
Your Gradio code contains a good lesson.
During training you:
- encode the categorical values
- create dummy columns
- scale the features
- train the model
During prediction you must do the same thing:
raw user input
↓
same encoding
↓
same columns
↓
same scaling
↓
model
If the training data has:
feature_1
feature_2
feature_3
feature_4
but the application sends:
feature_1
feature_3
feature_4
the model cannot interpret the input correctly.
Your use of:
input_df.reindex(
columns=feature_columns,
fill_value=0
)
is intended to make the input columns match the training feature structure.
That is an important practical idea.
73. A stronger production pattern: Pipeline
A cleaner production approach is often:
Pipeline([
("preprocessor", ...),
("model", LogisticRegression())
])
Then preprocessing and prediction are tied together.
This reduces the risk of accidentally applying different transformations during training and inference.
Your notebook demonstrates the concept manually, which is useful for learning.
74. Your Auto Insurance / Customer Lifetime Value project
The 13-Aug.ipynb notebook moves into regression.
The target is:
Y = df["Customer Lifetime Value"]
This changes the question from:
Will this customer churn?
to:
What numerical Customer Lifetime Value should we predict?
Therefore:
Classification → Churn
Regression → Customer Lifetime Value
75. Insurance dataset: initial exploration
You loaded:
df = pd.read_csv("AutoInsurance.csv")
Then inspected:
df.info()
df.shape
df.describe()
You also removed:
Customer
Effective To Date
from the modeling DataFrame.
Again, the purpose is to separate identifiers / dates that were not being used in the current modeling approach.
However, an important ML lesson is:
Dates are not automatically useless.
A date may contain useful information such as:
- month
- day of week
- season
- year
- time since an event
A stronger project could engineer useful date features instead of simply dropping every date column.
76. Insurance EDA
You explored:
sns.countplot(x="State", data=df)
sns.boxplot(y=df["Income"])
sns.boxplot(
x="Coverage",
y="Customer Lifetime Value",
data=df
)
sns.scatterplot(
x="Income",
y="Customer Lifetime Value",
data=df
)
df["Income"].hist()
This is a good progression:
Categorical distribution
↓
Numerical distribution
↓
Numerical vs categorical
↓
Numerical vs numerical
You are gradually asking:
"What might explain the target?"
77. Automatically finding numerical and categorical columns
You used:
numerical_cols = df.select_dtypes(
include=["int64", "float64"]
).columns
categorical_cols = df.select_dtypes(
include=["object"]
).columns
This is useful because instead of manually listing every column, Python can identify columns based on data type.
Then you looped through them to create graphs.
This is the beginning of writing reusable data-analysis code.
78. Automated boxplots
You created boxplots for all numerical columns.
This is useful because you can quickly inspect:
- distributions
- median
- spread
- potential outliers
However, remember that a large collection of graphs can become overwhelming.
The goal should eventually move from:
"Let's plot everything."
to:
"Let's plot the variables that help answer our question."
That is a sign of growing from beginner EDA toward professional analysis.
79. Binning numerical variables for boxplots
You used:
pd.cut(
df[col],
bins=5
)
This divides a numerical variable into five intervals.
Then you compare CLV across those ranges.
Conceptually:
Income range 1 → CLV distribution
Income range 2 → CLV distribution
Income range 3 → CLV distribution
...
This is useful for visualization because a boxplot expects categories on one axis.
But the number and boundaries of bins can affect the story you see.
So binning is mainly a visualization technique here, not necessarily something you should automatically use as a model feature.
80. Comparing many regression algorithms
One of the biggest learning steps in your final notebook was trying multiple regressors:
Linear Regression
KNN Regressor
SVR
Decision Tree Regressor
Bagging Regressor
AdaBoost Regressor
Random Forest Regressor
This is valuable because it teaches a core ML lesson:
Different algorithms make different assumptions and capture different types of patterns.
There is no universally best algorithm.
81. Linear Regression vs tree-based models
Linear Regression assumes a relationship that can be represented through a linear combination of features.
Tree-based methods can represent more complex non-linear relationships.
For example:
Linear model:
Y
│ /
│ /
│ /
│ /
└──────── X
A tree-based model can approximate more irregular patterns:
Y
│ ┌────
│ │
│ ───┘
│
└──────── X
This is why trying multiple model families can be useful.
82. Why model comparison matters
Suppose you obtain:
Model Train R² Test R²
Linear Regression 0.70 0.68
KNN 0.90 0.62
Decision Tree 0.99 0.55
Random Forest 0.91 0.78
You should not automatically choose the model with the highest training score.
The more important question is:
Which model generalizes well to unseen data?
Here Random Forest would be more interesting than the tree with 0.99 training R².
83. Log transformation of Customer Lifetime Value
You later used:
Y_log = np.log1p(Y)
Why?
Because some target variables are heavily skewed.
Imagine:
Most CLV values: relatively small
A few CLV values: extremely large
The model may have difficulty fitting such a distribution.
A log transformation compresses large values.
Conceptually:
Original:
1
10
100
1000
10000
Log scale:
small differences between large values
This can make the target distribution easier for some models to learn.
84. Reversing the log transformation
After predicting:
predictions_log = model.predict(X)
you used:
predictions_actual = np.expm1(predictions_log)
This is important.
If:
Y_log = log(1 + Y)
then:
Y = exp(Y_log) - 1
np.expm1() performs:
exp(x) - 1
So:
log target
↓
model
↓
log prediction
↓
expm1
↓
original target scale
85. An important issue in the notebook: split consistency
In your log-transform section, you did:
Y_log = np.log1p(Y)
X_train_log, X_test_log, Y_train_log, Y_test_log = train_test_split(
X,
Y_log,
test_size=0.3,
random_state=42
)
This is okay as a new split, but it means your earlier train/test split and later train/test split are not necessarily the same.
For a clean experiment, define the split once and reuse it consistently.
For example:
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.3,
random_state=42
)
y_train_log = np.log1p(y_train)
y_test_log = np.log1p(y_test)
This makes comparisons easier to reason about.
86. Another important issue: predicting on the full dataset
You later did:
predictions_log = model_rf.predict(X)
and saved predictions for the full dataset.
That can be useful for producing a business-facing prediction file.
But these are not unbiased test predictions.
Why?
Because X includes training observations the model already saw.
So:
Predictions on X_test
→ useful for evaluating unseen data
Predictions on X
→ useful for generating predictions for all records,
but not for measuring generalization
This distinction is very important.
87. Hyperparameters
Later you changed models from:
DecisionTreeRegressor()
to:
DecisionTreeRegressor(
max_depth=6,
min_samples_split=10,
min_samples_leaf=5
)
These settings are called hyperparameters.
They are chosen by us before training.
The model learns its internal parameters from the training data.
Parameters
Learned by the algorithm.
Hyperparameters
Set by us.
This distinction is fundamental.
88. Understanding your tree hyperparameters
max_depth
Controls how deep the tree can grow.
Smaller:
simpler tree
less overfitting
Larger:
more complex tree
greater overfitting risk
min_samples_split
Minimum number of samples required before a node can be split.
Higher values make the tree more conservative.
min_samples_leaf
Minimum number of samples allowed in a leaf.
Higher values prevent extremely tiny leaves.
89. Random Forest hyperparameters
You used:
RandomForestRegressor(
n_estimators=200,
max_depth=8,
min_samples_leaf=5,
random_state=42
)
n_estimators=200
Build 200 trees.
More trees can make the ensemble more stable, though computation increases.
max_depth=8
Limits tree complexity.
min_samples_leaf=5
Prevents leaves from becoming too specific.
random_state=42
Makes the experiment reproducible.
90. Bagging hyperparameters
You used:
BaggingRegressor(
n_estimators=100,
max_samples=0.7,
max_features=0.7,
random_state=42
)
This means, conceptually:
- build many estimators
- each estimator sees a sample of the training rows
- each estimator can use a subset of features
- combine their predictions
Again, the goal is to create a robust ensemble rather than rely on one model.
91. Scaling experiment in your notebook
You used:
StandardScaler()
and compared KNN/SVR.
This is a very useful experiment because it demonstrates:
Preprocessing requirements depend on the algorithm.
Tree-based algorithms generally do not need feature scaling in the same way that distance- or margin-based methods do.
KNN and SVM are much more sensitive to feature scale.
92. A subtle issue in your scaling experiment
You created:
X_train_scaled
X_test_scaled
but your KNN code first trained on:
X_train_log
before later looping over scaled KNN.
That is useful as experimentation, but for a clean article you should present the comparison more systematically:
KNN without scaling
vs
KNN with scaling
and explicitly explain why the result changed.
For SVR, you correctly used the scaled data in the shown section.
93. The complete mental model
After these five days, the entire workflow can be remembered as:
REAL-WORLD QUESTION
↓
Define the target
↓
Collect data
↓
Understand data
↓
┌──────────┴──────────┐
↓ ↓
Numerical Categorical
↓ ↓
distributions category counts
└──────────┬──────────┘
↓
EDA
↓
Clean the data
↓
Handle missing values
↓
Encode categories
↓
Select useful features
↓
Split train/test
↓
Scale when appropriate
↓
Train baseline
↓
Evaluate on test data
↓
Try other models
↓
Tune hyperparameters
↓
Select based on validation
↓
Final evaluation
↓
Make predictions
↓
Integrate into an application
94. What each notebook taught you
Day 1 — Auto MPG
Main lessons:
- load a dataset
- inspect rows and columns
- understand shape and data types
- inspect summary statistics
- identify categorical/text columns
- detect messy numerical values
- visualize distributions
- use countplots
- use histograms
- use pairplots
- understand missing values
- convert
"?"toNaN - convert text to numerical data
- use median imputation
The biggest lesson:
Before modeling, understand and clean your data.
Day 2 — Telco Customer Churn
Main lessons:
- identify a classification target
- remove an identifier
- handle
TotalCharges - visualize class balance
- compare categories against churn
- use boxplots and histograms
- use pairplots
- inspect correlations
- encode categorical data
- split into training/testing data
- train Logistic Regression
- train SVC
- train Decision Tree
- compare train/test performance
The biggest lesson:
A classification model predicts categories, but the quality of the prediction depends heavily on preprocessing and evaluation.
Day 3 — Improving the Telco project
Main lessons:
- one-hot encode multi-category variables
- use
random_state - use
stratify - scale features
- train Logistic Regression
- build a prediction interface with Gradio
- reproduce preprocessing at inference time
- output a human-readable prediction
The biggest lesson:
An ML model is only one part of an ML application.
Day 4/5 — Customer Lifetime Value
Main lessons:
- formulate a regression problem
- inspect numerical/categorical relationships
- automate EDA
- encode categories
- split train/test
- compare multiple regressors
- understand R²
- try target transformation
- use log transformation
- reverse the transformation
- experiment with scaling
- tune hyperparameters
- use ensemble models
- generate prediction files
The biggest lesson:
Different models capture different patterns, and model selection should be based on performance on unseen data—not training performance alone.
95. What I would change before publishing the project as "production-quality"
Your notebooks are excellent learning material because they show experimentation. For a public article, however, it is worth separating:
Learning experiments
from:
Recommended ML workflow
Here are the most important improvements.
Improvement 1 — Use clear train/test terminology
Prefer:
X_train, X_test, y_train, y_test
rather than mixing Y and y.
Python convention generally uses lowercase y.
Improvement 2 — Make experiments reproducible
Use:
random_state=42
consistently when appropriate.
Improvement 3 — Use stratification for classification
For churn:
train_test_split(
X,
y,
test_size=0.3,
random_state=42,
stratify=y
)
Improvement 4 — Avoid LabelEncoder for nominal multi-class input features
Use one-hot encoding for categories such as:
PaymentMethod
Contract
InternetService
VehicleClass
Label encoding is much more naturally suited to targets or truly ordinal categories.
Improvement 5 — Fit preprocessing only on training data
For example:
Training data
↓
fit scaler
↓
transform training
Test data
↓
transform using existing scaler
Do not independently fit preprocessing on the test data.
Improvement 6 — Use proper evaluation metrics
For classification, consider:
Accuracy
Precision
Recall
F1
Confusion Matrix
ROC-AUC
PR-AUC
For regression:
R²
MAE
MSE
RMSE
Which metrics matter depends on the business problem.
Improvement 7 — Don't compare only training scores
Always look at:
Training performance
+
Validation/test performance
A very high training score with much lower test performance can indicate overfitting.
Improvement 8 — Keep a validation strategy for model selection
A single test set should ideally be kept for final evaluation.
During experimentation, use cross-validation or a validation set to choose models/hyperparameters.
Conceptually:
Training data
↓
Cross-validation / validation
↓
Choose model
↓
Final test set
↓
Final unbiased-ish estimate
96. A very important distinction: EDA vs ML
EDA asks:
"What does my data look like?"
Machine learning asks:
"Can I learn a useful relationship that lets me make predictions on new data?"
These overlap, but they are not the same.
For example:
sns.scatterplot(...)
does not build a model.
It helps you understand the data before deciding how to model it.
97. Another important distinction: correlation vs prediction
A strong correlation can be useful.
But:
high correlation
does not automatically mean:
excellent predictive model
Prediction depends on:
- multiple features
- data quality
- model type
- noise
- generalization
- preprocessing
- evaluation
So don't judge an ML problem from one graph.
98. Another important distinction: model vs algorithm
It is common to hear:
"I trained a model."
The algorithm is the learning procedure.
For example:
Random Forest = algorithm/model family
After training:
model_rf.fit(X_train, y_train)
you have a fitted model containing learned information.
A useful mental model:
Algorithm
+
Training data
↓
Fitted model
↓
Predictions
99. What "learning" actually means
This is probably the most important concept to understand.
When you run:
model.fit(X_train, y_train)
the model is not magically understanding the world.
It is optimizing internal parameters so that its predictions match the training examples according to the algorithm's objective.
For example, linear regression learns coefficients.
A tree learns split rules.
A neural network learns weights.
So:
Training means finding model parameters that make the model perform well according to a defined objective.
100. What happens during prediction?
Once training is complete:
prediction = model.predict(X_test)
the model does not learn again from the test examples.
It applies what it learned during training.
Conceptually:
Training:
X_train + y_train
↓
model.fit()
↓
learned model
Prediction:
X_test
↓
learned model
↓
prediction
101. The biggest beginner misconception to avoid
Do not think:
"I used Random Forest, so I did machine learning."
The real ML skill is being able to explain:
- What problem are you solving?
- What is the target?
- What features are available?
- What does the data look like?
- What problems did you find?
- How did you clean them?
- How did you encode the data?
- Why did you choose the algorithm?
- How did you evaluate it?
- Does it generalize?
- What would you improve?
If you can explain these, you are learning ML—not just memorizing scikit-learn commands.
102. A simple explanation of your five-day journey
If someone asks:
"What did you actually learn in five days?"
A strong answer is:
I learned that machine learning is not just about selecting an algorithm. I started by understanding datasets using Pandas, cleaning missing and incorrectly formatted values, and using visualization to understand distributions and relationships. Then I learned the difference between classification and regression, converted categorical data into numerical representations, split data into training and testing sets, trained several models using scikit-learn, and compared their performance on unseen data. Finally, I experimented with scaling, transformations, ensemble models, hyperparameters, and even built a simple Gradio interface around a churn model.
That is a much stronger story than:
"I learned Logistic Regression, SVM and Random Forest."
103. Suggested article structure for dev.to
For your final public article, I recommend this structure:
Title
1. Why I started learning ML
2. What I thought ML was vs what I learned
3. AI vs ML
4. Types of ML
5. My five-day learning roadmap
6. Day 1 — Understanding and cleaning data
7. EDA and why graphs matter
8. Day 2 — Classification with customer churn
9. Encoding categorical data
10. Train/test split
11. Logistic Regression, SVM and Decision Trees
12. How I evaluated classification models
13. Day 3 — Turning the model into an application
14. Day 4/5 — Regression and Customer Lifetime Value
15. Linear Regression vs KNN vs SVR vs Trees vs Ensembles
16. Log transformation
17. Scaling
18. Hyperparameters
19. What I got wrong / what I would improve
20. What I learned
21. What I plan to learn next
This will make the article feel like a journey, rather than a textbook.
104. Recommended article narrative
The most interesting story is:
"I started with almost no ML knowledge."
↓
"I learned that data preparation matters."
↓
"I discovered that graphs help me ask questions
before training models."
↓
"I learned classification."
↓
"I learned regression."
↓
"I tried different algorithms and realized
there is no single best model."
↓
"I learned that a high training score
doesn't necessarily mean a good model."
↓
"I turned one model into a small application."
↓
"I now understand the basic ML workflow
and know what I need to learn next."
That is a much more authentic beginner-to-ML story.
105. What to learn next
Based on what your notebooks already cover, I would not recommend immediately jumping into deep learning.
First strengthen these foundations:
1. Statistics
Learn:
- mean
- median
- variance
- standard deviation
- distributions
- probability
- correlation
- conditional probability
2. ML evaluation
Learn:
- confusion matrix
- precision
- recall
- F1
- ROC-AUC
- MAE
- RMSE
- R²
- cross-validation
3. Feature engineering
Learn:
- handling dates
- categorical features
- transformations
- interactions
- outliers
- missing values
4. Model selection
Learn:
- cross-validation
- hyperparameter tuning
- GridSearchCV
- RandomizedSearchCV
- pipelines
5. ML fundamentals
Learn:
- bias vs variance
- underfitting
- overfitting
- regularization
- data leakage
- feature importance
Then move toward:
Classical ML
↓
Feature engineering
↓
Model selection
↓
Cross-validation
↓
ML pipelines
↓
Deployment
↓
Deep Learning
↓
Generative AI / LLMs
106. The one-page cheat sheet
Problem type
Predict a category → Classification
Predict a number → Regression
Find hidden groups → Clustering
Learn through reward → Reinforcement Learning
Data
Rows → observations
Columns → variables/features
X → input features
y → target
First inspection
df.head()
df.shape
df.info()
df.describe()
df.sample()
df.isnull().sum()
Visualization
Countplot → categories
Histogram → numerical distribution
Boxplot → spread/outliers
Scatterplot → two numerical variables
Pairplot → many numerical relationships
Heatmap → correlation matrix
Preparation
Missing values
Data types
Categorical encoding
Feature selection
Scaling when needed
Training
model.fit(X_train, y_train)
Prediction
model.predict(X_test)
Evaluation
Classification:
Accuracy
Precision
Recall
F1
Confusion Matrix
ROC-AUC
Regression:
R²
MAE
MSE
RMSE
Core warning signs
Training score >> Test score
↓
Possible overfitting
Test information used during preprocessing
↓
Possible data leakage
Identifier used as feature
↓
Possible meaningless pattern
Categorical values converted to arbitrary numbers
↓
Possible false ordering
107. Final takeaway
The biggest lesson from these five days is not a particular Python library or algorithm.
It is the workflow:
Understand the problem → understand the data → clean the data → explore the data → prepare features → train → evaluate → improve → deploy.
The algorithms are tools inside that workflow.
You have already touched many of the important building blocks:
- Pandas
- NumPy
- Matplotlib
- Seaborn
- Scikit-learn
- data cleaning
- EDA
- classification
- regression
- encoding
- train/test split
- scaling
- model evaluation
- hyperparameters
- ensemble models
- target transformation
- Gradio
The next step is not to memorize more algorithms.
The next step is to become comfortable answering:
Why am I doing this step?
If you can answer that question for every important line in your notebook, you are moving from "following an ML tutorial" to actually understanding machine learning.
Appendix A — Notebook-by-notebook step map
11_Aug_mpg.ipynb
| Notebook step | Why it was done | ML concept |
|---|---|---|
| Import NumPy/Pandas/Matplotlib/Seaborn | Load tools | Python ML stack |
read_csv() |
Load data | Data ingestion |
nunique() |
Understand uniqueness | Data exploration |
head() |
Inspect examples | EDA |
shape |
Understand dataset size | EDA |
describe() |
Summary statistics | EDA |
info() |
Types/non-null values | Data quality |
countplot() |
Inspect categorical distribution | Visualization |
hist() |
Inspect numerical distributions | Visualization |
isnull() |
Detect actual missing values | Data cleaning |
pairplot() |
Explore relationships | EDA |
sample() |
Inspect random examples | Data quality |
Identify car name
|
Recognize identifier-like field | Feature selection |
Replace ?
|
Convert fake missing values | Data cleaning |
to_numeric() |
Make horsepower numerical | Data preparation |
| Median imputation | Fill missing horsepower | Missing-value handling |
| Train/test split | Separate learning/evaluation data | Generalization |
| Logistic Regression | Start classification experiment | Supervised learning |
Important caveat
The notebook switches from Auto MPG to a loan_prediction.csv dataset near the end. That appears to be a copied/reused modeling experiment rather than a continuation of the Auto MPG workflow. For a polished article, present these as separate experiments rather than one continuous Auto MPG pipeline.
12th-Aug-Telco-Customer-churn.ipynb
| Step | Purpose |
|---|---|
| Load Telco CSV | Get customer data |
head() |
Inspect records |
shape |
Understand size |
describe() |
Statistical overview |
Drop customerID
|
Remove identifier from features |
info() |
Check data types |
sample() |
Inspect random rows |
| Countplot of Churn | Check class distribution |
| Countplot of gender | Explore category distribution |
| Histograms | Explore numerical distributions |
| Pairplot | Explore relationships |
| Convert TotalCharges | Fix numeric type |
| Fill missing TotalCharges | Handle missing data |
| LabelEncoder | Convert categories to numbers |
| Train/test split | Evaluate on unseen data |
| Logistic Regression | Classification model |
| SVC | Alternative classifier |
| Decision Tree | Alternative classifier |
| Gradio | Turn model into interactive app |
Important caveat
The notebook uses LabelEncoder on many categorical input columns. This is useful for learning encoding, but one-hot encoding is generally safer for nominal multi-category features.
12-Aug-2.ipynb
This notebook is a more developed version of the Telco project.
Important improvements include:
- dropping
customerID - converting
TotalCharges - one-hot encoding multi-category variables
- using
random_state=42 - using
stratify=Y - scaling features
- training Logistic Regression
- building a Gradio prediction interface
- preserving training feature columns
- applying the same preprocessing to new input
This is the notebook that most clearly demonstrates the transition from:
ML experiment
to:
small ML application
13-Aug.ipynb
Main steps:
Load AutoInsurance
↓
Inspect dataset
↓
Drop Customer / date columns
↓
EDA
↓
Identify numerical/categorical columns
↓
Generate many visualizations
↓
Encode categorical variables
↓
Set CLV as target
↓
Train/test split
↓
Try multiple regressors
↓
Try log target
↓
Try scaling
↓
Tune tree/ensemble hyperparameters
↓
Generate predictions
↓
Save prediction CSVs
This notebook demonstrates the transition from classification to regression and from a single baseline model to model comparison and tuning.
Appendix B — A cleaner ML template to remember
# 1. Load
df = pd.read_csv("data.csv")
# 2. Understand
df.head()
df.shape
df.info()
df.describe()
# 3. Clean
# handle missing values
# fix data types
# remove inappropriate identifiers
# 4. Explore
# histograms
# boxplots
# countplots
# scatterplots
# correlation
# 5. Prepare
X = df.drop("target", axis=1)
y = df["target"]
# 6. Split
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)
# 7. Preprocess
# encoding / scaling
# 8. Train
model.fit(X_train, y_train)
# 9. Predict
predictions = model.predict(X_test)
# 10. Evaluate
# choose metrics appropriate to the problem
# 11. Compare / improve
# try another model
# tune hyperparameters
# use cross-validation
# 12. Final model
# train using the selected approach
# 13. Predict new data
Closing thought for the article
Five days ago, an ML notebook could easily look like a collection of unfamiliar commands.
Now there is a structure behind those commands.
head() is not just a command — it is a way to understand the data.
isnull() is not just a command — it is part of data quality.
A histogram is not just a graph — it tells you about a distribution.
train_test_split() is not just boilerplate — it protects your evaluation from simply measuring memorization.
fit() is not magic — it is the learning stage.
predict() is the model applying what it learned.
And model comparison is not about finding the fanciest algorithm — it is about finding an approach that generalizes well to data the model has never seen.
That, more than anything else, is what I took away from my first five days of machine learning.
Top comments (0)