After spending 8+ years working in IT, I recently did something that felt surprisingly uncomfortable:
I became a beginner again.
Not because I had to.
Because I wanted to.
With AI changing the way software is built and consumed, I kept asking myself:
“What should I learn next?”
I could have watched videos, completed a course, collected a certificate and moved on.
Instead, I decided to build something.
That decision led me to my first end-to-end Customer Churn Prediction project.
And honestly, the biggest lessons weren't about which ML algorithm performed best.
They were about data, leakage, feature engineering, evaluation, interpretability, and the reality of learning something technical while working full-time.
Why Start ML After 8+ Years?
When you're a student, learning is your primary responsibility.
When you're working, learning happens somewhere between:
- A full workday
- Meetings
- Production issues
- Personal commitments
- Family time
- And whatever energy is left at the end of the day 😅
That's what made this journey interesting.
I wasn't learning Machine Learning for eight hours a day.
Most of the learning happened after work and on weekends.
Some days I was motivated.
Some days I was exhausted.
And some days I spent far too long trying to understand something that looked incredibly simple in a tutorial.
But there was one thing I kept reminding myself:
I don't need to learn everything. I just need to keep moving.
So I decided to stop consuming tutorials and start building.
🎯 The Problem: Customer Churn Prediction
The business problem was straightforward:
Can we predict which customers are at a higher risk of churning?
But I didn't want the project to stop at:
Input → Model → Churn / No Churn
I wanted to understand the complete ML workflow:
Raw Data
↓
Data Cleaning
↓
Feature Engineering
↓
Feature Selection
↓
Leakage Analysis
↓
Model Training
↓
Model Evaluation
↓
Feature Importance
↓
Business Insights
That turned out to be much more educational than simply training a classifier.
📊 78 Columns Looked Like a Good Starting Point
The initial dataset contained 78 columns.
My first instinct was:
“More features should give the model more information.”
But that's not necessarily true.
Some features were redundant.
Some didn't provide useful predictive information.
And some raised a much more serious question:
Would this information actually be available at the moment we need to make the prediction?
That question took me into one of the most important concepts I encountered during this project:
Data Leakage
A model can appear to perform extremely well if it gets access to information that would not actually be available when making a real-world prediction.
That's dangerous.
You can have a model with impressive validation metrics and still have a model that is practically useless.
For example, imagine predicting whether a customer will churn while accidentally including a feature that is only populated after the customer has already initiated the cancellation process.
The model isn't predicting churn.
It's recognizing evidence of churn that has already happened.
That distinction matters.
⏳ Look-Ahead Bias Was Another Wake-Up Call
The same idea becomes even more important when working with time-dependent data.
If you're predicting something at time T, your model should not have access to information from T+1.
Conceptually:
Past ---------------------> Prediction Time
↑
Information
available
│
X
Future information
must not leak back
This sounds obvious.
But when you're working with a real dataset containing dozens of columns, timestamps, transactions and historical records, it's surprisingly easy to introduce this problem accidentally.
This was one of the biggest lessons from the project:
Always ask: “Would I actually know this information at prediction time?”
🔧 Feature Engineering: 78 → 30
After analyzing the available features, I reduced the training dataset from:
78 columns → 30 meaningful features
This wasn't simply about deleting columns because they had low importance.
I looked at things such as:
- Redundant information
- Highly correlated features
- Potential leakage
- Temporal availability
- Business meaning
- Predictive usefulness
The important lesson for me was:
Feature engineering isn't just a preprocessing step. It is part of understanding the problem.
You can't meaningfully engineer features if you don't understand what the data represents.
🌲 Model Experiments
Once the feature set was ready, I experimented with several tree-based approaches.
Random Forest
A useful baseline and a good way to understand ensemble decision trees.
XGBoost
A powerful gradient-boosting approach that performed well in my experiments.
LightGBM
Another gradient-boosting implementation that I included in the comparison.
Stacking Ensemble
Finally, I experimented with combining models rather than relying on a single estimator.
The goal wasn't simply to find the model with the highest number.
I wanted to understand how different approaches behaved on the same feature set.
📏 Accuracy Wasn't Enough
One of the easiest mistakes when starting ML is to focus heavily on accuracy.
For churn prediction, that can be misleading.
Suppose:
95% of customers stay
5% of customers churn
A model that predicts:
Everyone → Stay
would have:
95% accuracy
But it would completely fail at the actual business problem.
That's why I evaluated the models using multiple metrics, including:
- ROC-AUC
- Precision
- Recall
- F1-score
The important lesson wasn't memorizing these metrics.
It was understanding why each metric exists and when it matters.
💡 Then I Asked: “Why Did the Model Predict This?”
This became one of my favourite parts of the project.
Imagine a model tells us:
Customer X has a high probability of churning.
That's useful.
But a business stakeholder will naturally ask:
“Why?”
A prediction without context can be difficult to act upon.
So I explored Permutation Feature Importance to understand which features were contributing most strongly to the model's predictions.
This helped move the project from:
Prediction
towards:
Prediction + Explanation
And that distinction is important.
Machine Learning isn't only about making predictions.
In many real-world scenarios, we need to understand whether those predictions make sense.
🧠 The Most Important ML Lesson I Learned
Before starting this project, I thought the Machine Learning workflow was something like:
Get Data
↓
Train Model
↓
Check Accuracy
↓
Done
Now I see it more like this:
Understand the Problem
↓
Understand the Data
↓
Clean the Data
↓
Engineer Features
↓
Check for Leakage
↓
Split Data Correctly
↓
Train Multiple Models
↓
Evaluate Using Appropriate Metrics
↓
Interpret the Model
↓
Translate Results into Business Meaning
The algorithm is only one part of the solution.
🧑💻 What Learning After 8+ Years Actually Felt Like
This is the part I didn't expect when I started.
After years of professional experience, you're generally comfortable with your domain.
You know how to solve problems.
You know how to debug.
You know how to communicate.
You know how to work under pressure.
Then you start learning something completely new.
Suddenly, you're back to:
Why isn't this working?
What does this metric actually mean?
Why is my model performing differently?
Did I introduce leakage?
Why did changing this feature affect the result?
And sometimes:
Maybe I should just watch another tutorial. 😅
But I eventually realized something.
Being experienced doesn't mean you should already know everything.
It means you should be comfortable learning when you don't.
🤖 Why I Think This Matters in the AI Era
The IT industry is changing quickly.
AI is changing development workflows.
Automation is changing repetitive tasks.
Job descriptions are evolving.
And the skills companies value are changing along with them.
I don't think the answer is:
“Everyone should become an ML engineer.”
That's unrealistic.
The answer, at least for me, is:
Understand the technology. Experiment with it. Learn how it works. Find where it can create value in your domain.
That's why I started this journey.
I didn't want to simply hear about AI every day.
I wanted to understand at least some of what was happening underneath the surface.
🚀 Three Months In
I'm still very much at the beginning.
I haven't learned everything.
I haven't worked with every ML technique.
And I definitely don't consider myself an AI/ML expert.
But three months ago, I couldn't say:
“I've built an end-to-end ML project.”
Now I can.
And that matters to me.
Not because the project is perfect.
It's not.
But because it represents something more important:
I started.
I made time after work.
I learned concepts I didn't understand.
I worked through confusing data.
I questioned my assumptions.
I built.
I tested.
I made mistakes.
And I learned.
🌱 If You're Also Thinking About Learning AI/ML
Maybe you're a developer.
Maybe you're a tester.
Maybe you're a data analyst.
Maybe you're in support, infrastructure, project management, or another part of IT.
Maybe you've been working for 5 years.
Maybe 10.
Maybe 15.
And you've been thinking:
“I should probably start learning AI.”
But you keep postponing it because you're already busy.
I understand.
You don't need to spend eight hours a day learning.
Start small.
Spend an hour.
Build something.
Break it.
Fix it.
Read documentation.
Ask questions.
Repeat.
Don't worry about becoming an expert immediately.
Focus on becoming better than you were yesterday.
🎯 What I Want to Explore Next
This project gave me a foundation, but it also gave me a much longer list of things I want to understand.
I'm particularly interested in going deeper into:
- Better feature engineering
- Model evaluation
- Explainable ML
- Model deployment
- ML pipelines
- Monitoring
- Experiment tracking
- And eventually taking ML projects from notebooks toward production
There is a long way to go.
And that's okay.
Final Thought
The biggest thing I built in these three months wasn't the churn prediction model.
It was the habit of learning again.
After 8+ years in IT, becoming a beginner wasn't easy.
But it was worth it.
Because technology will keep changing.
The tools will change.
The frameworks will change.
The models will change.
The industry will change.
And hopefully, so will I.
Not because I'm afraid of the future.
Because I'm curious about it. 🚀
A note about the project
This project was built as part of my AI/ML learning journey using anonymized information. Company-specific data, business details and sensitive information have intentionally been omitted.
If you're also learning Machine Learning, I'd genuinely love to hear:
What are you building right now?
Top comments (0)