Audrine Marion

Posted on Aug 28

Using Scikit-Learn Pipelines: A Cleaner Way to Build Machine Learning Models

#machinelearning #ai #beginners #datascience

If you've spent some time building machine learning models with Python, you've probably had a notebook that looked something like this:

X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

And then a few cells later, you realize you also need to encode categorical variables.

Then there's imputation.

Then feature selection.

Then maybe PCA.

Before you know it, your notebook has a collection of preprocessing steps that depend on being executed in exactly the right order.

I've been there.

One of the things that made my machine learning workflow much cleaner was learning how to use Scikit-Learn Pipelines.

In this article, I'll walk through what pipelines are, why they matter, and how I use them to make machine learning workflows more reliable and easier to maintain.

What is a Scikit-Learn Pipeline?

A pipeline is essentially a way of connecting multiple machine learning steps together so that they can be treated as one workflow.

For example, suppose we have a dataset where we need to:

Handle missing values
Scale numerical features
Train a machine learning model

Instead of doing everything separately:

X_train = imputer.fit_transform(X_train)
X_train = scaler.fit_transform(X_train)

model.fit(X_train, y_train)

We can put everything into a pipeline:

from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler", StandardScaler()),
    ("model", LogisticRegression())
])

pipeline.fit(X_train, y_train)

Now the entire process is represented by one object.

And that's the part I really like about pipelines.

Instead of thinking:

"First I need to do this, then this, then this..."

you can think:

"This is my machine learning workflow."

Why Should We Use Pipelines?

At first, pipelines can feel like extra syntax.

Why not just preprocess the data manually?

You absolutely can.

But pipelines solve several important problems.

1. They reduce data leakage

This is probably the biggest reason to use them.

Imagine you're scaling your dataset before splitting it:

scaler = StandardScaler()

X_scaled = scaler.fit_transform(X)

X_train, X_test, y_train, y_test = train_test_split(
    X_scaled, y, test_size=0.2, random_state=42
)

It looks harmless.

But there's a problem.

The scaler has seen the entire dataset, including the test set.

That means information from your test data has influenced the preprocessing step.

This is a form of data leakage.

The model hasn't technically seen the test labels, but information about the distribution of the test features has already entered the training process.

A pipeline helps prevent this when used correctly:

pipeline = Pipeline([
    ("scaler", StandardScaler()),
    ("model", LogisticRegression())
])

pipeline.fit(X_train, y_train)

During fit(), the scaler learns its parameters only from X_train.

When we evaluate:

pipeline.predict(X_test)

the same fitted scaler transforms X_test without learning anything new from it.

That separation is extremely important.

2. Pipelines Keep Preprocessing and Modeling Together

Without a pipeline, you might end up with something like:

imputer = SimpleImputer(strategy="median")
scaler = StandardScaler()

X_train = imputer.fit_transform(X_train)
X_train = scaler.fit_transform(X_train)

model = LogisticRegression()
model.fit(X_train, y_train)

Then, when making predictions:

X_test = imputer.transform(X_test)
X_test = scaler.transform(X_test)

predictions = model.predict(X_test)

Notice the problem?

You have to remember the exact preprocessing sequence.

With a pipeline:

pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler", StandardScaler()),
    ("model", LogisticRegression())
])

you simply do:

pipeline.fit(X_train, y_train)

predictions = pipeline.predict(X_test)

Much cleaner.

3. Pipelines Make Cross-Validation Safer

This is another major advantage.

Suppose we want to evaluate different models using cross-validation.

We could create a pipeline:

pipeline = Pipeline([
    ("scaler", StandardScaler()),
    ("model", LogisticRegression())
])

Then:

from sklearn.model_selection import cross_val_score

scores = cross_val_score(
    pipeline,
    X,
    y,
    cv=5,
    scoring="accuracy"
)

print(scores)
print(scores.mean())

Each fold gets its own preprocessing fitted only on that fold's training data.

This is exactly what we want.

Without a pipeline, it is easy to accidentally preprocess the entire dataset before cross-validation and introduce leakage.

Building a Simple Pipeline

Let's create a slightly more realistic example.

Imagine we're building a model to predict whether a customer belongs to a particular class.

Our workflow might be:

Missing values → Scaling → Logistic Regression

from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler", StandardScaler()),
    ("model", LogisticRegression())
])

The names we give each step, such as "imputer" and "scaler", are useful because they allow us to access those individual components later.

Training becomes:

pipeline.fit(X_train, y_train)

Prediction:

y_pred = pipeline.predict(X_test)

And evaluation:

from sklearn.metrics import classification_report

print(classification_report(y_test, y_pred))

That's the whole workflow.

What About Categorical and Numerical Features?

This is where pipelines become even more useful.

Real-world datasets rarely contain only numerical variables.

You might have:

Age          → numerical
Income       → numerical
Education    → categorical
Gender       → categorical

We shouldn't necessarily preprocess all of these columns in the same way.

For example:

Numerical columns → imputation + scaling
Categorical columns → imputation + one-hot encoding

Scikit-Learn gives us ColumnTransformer for exactly this situation.

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.linear_model import LogisticRegression

Let's define our columns:

numerical_features = [
    "age",
    "income"
]

categorical_features = [
    "education",
    "gender"
]

Now we create separate preprocessing pipelines.

Numerical pipeline

numeric_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler", StandardScaler())
])

Categorical pipeline

categorical_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("encoder", OneHotEncoder(handle_unknown="ignore"))
])

Now we combine them using ColumnTransformer:

preprocessor = ColumnTransformer([
    ("num", numeric_pipeline, numerical_features),
    ("cat", categorical_pipeline, categorical_features)
])

Finally, we connect preprocessing to our model:

model_pipeline = Pipeline([
    ("preprocessor", preprocessor),
    ("model", LogisticRegression())
])

Now we have one complete workflow:

Raw Data
   ↓
Numerical preprocessing ──┐
                          ├──→ Model
Categorical preprocessing ┘

And training is still just:

model_pipeline.fit(X_train, y_train)

Prediction:

y_pred = model_pipeline.predict(X_test)

This is much closer to how machine learning workflows look in real projects.

Pipelines and Hyperparameter Tuning

Here's where things get even more interesting.

We can use pipelines with GridSearchCV to tune both preprocessing and model parameters.

For example:

from sklearn.model_selection import GridSearchCV

param_grid = {
    "model__C": [0.01, 0.1, 1, 10],
    "model__max_iter": [100, 200, 500]
}

grid_search = GridSearchCV(
    model_pipeline,
    param_grid,
    cv=5,
    scoring="accuracy"
)

grid_search.fit(X_train, y_train)

Notice this:

"model__C"

The double underscore allows us to access parameters inside pipeline steps.

If our pipeline contains:

("model", LogisticRegression())

then:

model__C

means:

"The C parameter belonging to the model step."

We can then check the best parameters:

print(grid_search.best_params_)

And use the best estimator:

best_model = grid_search.best_estimator_

predictions = best_model.predict(X_test)

This makes experimentation much more systematic.

Pipelines Are Not Just for Classification

Pipelines work across many Scikit-Learn workflows.

For example, regression:

from sklearn.ensemble import RandomForestRegressor

regression_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("model", RandomForestRegressor(random_state=42))
])

Or clustering:

from sklearn.cluster import KMeans

clustering_pipeline = Pipeline([
    ("scaler", StandardScaler()),
    ("model", KMeans(n_clusters=3, random_state=42))
])

The same basic idea applies:

Preprocessing
     ↓
Transformation
     ↓
Model / Algorithm

One Important Thing I Learned

One mistake that's easy to make when learning pipelines is assuming that every model needs every preprocessing step.

It doesn't.

For example, scaling is generally important for algorithms that are sensitive to feature magnitude, such as:

Logistic Regression
KNN
SVM
K-Means
PCA

But tree-based models such as:

Decision Trees
Random Forest
Gradient Boosting

generally don't require feature scaling.

So don't build pipelines mechanically.

Think about what your algorithm actually needs.

Pipelines Make Your Code More Reproducible

There's another benefit that isn't always obvious when you're first learning machine learning.

A pipeline makes your workflow easier for someone else to reproduce.

Imagine handing someone this:

pipeline.fit(X_train, y_train)

Compared with giving them five different preprocessing scripts and telling them:

"Run this first, then this, then remember to use the same scaler when predicting."

The pipeline communicates the workflow much more clearly.

This becomes especially useful when working on:

Team projects
Research projects
Production ML systems
GitHub projects
Machine learning competitions

It also makes your notebooks less cluttered.

And honestly, once you start working with several models, that becomes a big deal.

A Complete Example

Here's a compact example putting everything together:

import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report


#### Load data
df = pd.read_csv("customer_data.csv")

# Separate features and target
X = df.drop("target", axis=1)
y = df["target"]

#### Define feature types
numeric_features = ["age", "income"]
categorical_features = ["gender", "education"]

#### Numerical preprocessing
numeric_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler", StandardScaler())
])

#### Categorical preprocessing
categorical_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("encoder", OneHotEncoder(handle_unknown="ignore"))
])

#### Combine preprocessing
preprocessor = ColumnTransformer([
    ("num", numeric_pipeline, numeric_features),
    ("cat", categorical_pipeline, categorical_features)
])

#### Full pipeline
pipeline = Pipeline([
    ("preprocessor", preprocessor),
    ("model", LogisticRegression(max_iter=1000))
])

#### Split data
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y
)

### Train
pipeline.fit(X_train, y_train)

### Predict
y_pred = pipeline.predict(X_test)

### Evaluate
print(classification_report(y_test, y_pred))

What I like about this structure is that the code tells a story.

You can almost read it from top to bottom:

Load → Separate → Preprocess → Build → Split → Train → Predict → Evaluate

That's what good ML code should do.

When Should You Use Pipelines?

I'd recommend getting comfortable with pipelines as soon as you're moving beyond very simple machine learning exercises.

They're particularly useful when you have:

Multiple preprocessing steps
Missing values
Categorical variables
Feature scaling
Feature selection
Dimensionality reduction
Cross-validation
Hyperparameter tuning
Multiple models to compare

Even when a pipeline isn't strictly necessary, using one can make your workflow cleaner.

Final Thoughts

When I first started working with machine learning workflows, preprocessing felt like a collection of separate tasks.

Clean the data.

Encode it.

Scale it.

Split it.

Train the model.

Then somehow remember to apply the exact same transformations to the test data.

Scikit-Learn Pipelines changed the way I think about that process.

A pipeline isn't just a convenient way to shorten your code. It's a way of defining the entire machine learning workflow as one reproducible object.

And perhaps the biggest lesson is this:

Your model is only one part of a machine learning system.

The preprocessing that happens before the model matters just as much.

Once you start using pipelines, you'll find yourself writing ML code that is cleaner, safer, and much easier to maintain.

And when you eventually move from Jupyter notebooks to real-world machine learning projects, that habit becomes incredibly valuable.

Key Takeaways

Pipeline connects preprocessing and modeling into one workflow.
It helps reduce the risk of data leakage.
It makes cross-validation safer.
ColumnTransformer allows different preprocessing for different feature types.
Pipelines work with GridSearchCV and hyperparameter tuning.
They improve reproducibility and maintainability.
Not every algorithm needs the same preprocessing, so build your pipeline based on the model you're using.

If you're learning Scikit-Learn right now, pipelines are one of those concepts I'd strongly recommend getting comfortable with early. They might seem like extra structure at first, but once your projects become more complicated, you'll be very glad you have them.

DEV Community