DEV Community

Cover image for Stop Building Data Science Notebooks. Start Building End-to-End Projects That Get You Hired.
Michael Nyawade
Michael Nyawade

Posted on

Stop Building Data Science Notebooks. Start Building End-to-End Projects That Get You Hired.

If you're trying to break into data science, chances are you've already built a few machine learning projects.

You downloaded a dataset, cleaned it, explored some columns, trained a model, evaluated its performance, and probably pushed the notebook to GitHub.

And that's great. It's a start.

But here's something worth thinking about: if a hiring manager opens your GitHub repository, what exactly are they supposed to learn about you?

That you know how to call .fit() on a scikit-learn model?

Or that you can take an actual business problem, work with messy data, extract useful insights, build a predictive solution, and make that solution accessible to other people?

There's a big difference between the two.

A strong data science portfolio shouldn't just demonstrate that you understand machine learning algorithms. It should demonstrate that you understand the entire data science workflow.

Let's look at what that actually means using a practical example: a DoorDash Delivery Duration Prediction project.

Complete data science workflow from business problem to deployment

1. Start With a Business Problem, Not a Dataset

One mistake in many data science portfolios is that projects start with a dataset rather than a problem.

Someone finds an interesting dataset on Kaggle, opens a Jupyter notebook, and immediately starts exploring the columns.

There's nothing inherently wrong with that approach when you're learning. But if you're building a portfolio project for potential employers, you need to think differently.

Start by asking:

What problem am I trying to solve, and why should anyone care about solving it?

Consider the DoorDash delivery duration prediction project.

Instead of presenting it as:

"Predicting Delivery Time Using Machine Learning"

Frame it around the actual business problem:

"How Can We Predict Food Delivery Times to Improve Delivery Operations and Customer Experience?"

See the difference?

The second title communicates the business context. It tells the reader what you're trying to accomplish and why the project matters.

Predicting delivery duration isn't just about achieving a low prediction error. More accurate estimates can help delivery platforms improve operational planning, set better customer expectations, and identify factors contributing to delivery delays.

That's the kind of thinking that makes a portfolio project interesting.

2. Think Beyond the Notebook: The Complete Data Science Workflow

A production-oriented data science project involves much more than exploratory analysis and model training.

For the DoorDash example, I'd break the workflow into nine stages:

  1. Business problem definition
  2. Data retrieval using SQL
  3. Data cleaning and preparation using Python
  4. Exploratory Data Analysis (EDA)
  5. Feature engineering
  6. Model development and experimentation
  7. Model evaluation and selection
  8. Model deployment through an API
  9. Building an interactive dashboard

Each stage demonstrates a different skill.

Let's go through them.

3. Define the Business Problem Clearly

Before writing SQL or importing pandas, establish what you're trying to predict.

In this case, the objective is to estimate how long an order will take to reach the customer.

That sounds straightforward, but you need to define the problem more carefully.

For example:

  • What information is available when an order is placed?
  • Which variables could influence delivery duration?
  • What exactly counts as delivery duration?
  • How will prediction accuracy be measured?
  • How could the business use these predictions?

These questions help establish the project's boundaries.

They also prevent a common portfolio problem: building something technically interesting without explaining its practical purpose.

A hiring manager should understand what your project does within the first few minutes of opening your repository.

4. Retrieve the Data Using SQL

SQL is an important part of real-world data science workflows.

Data rarely arrives in one perfectly structured CSV file. In many organizations, information is distributed across different database tables.

For a delivery prediction problem, imagine having access to:

  • Orders
  • Restaurants
  • Customers
  • Delivery partners
  • Delivery timestamps

You may need to join these tables before creating your modeling dataset.

SQL allows you to retrieve the information you need, filter irrelevant records, aggregate data, and perform preliminary transformations.

For example:

SELECT
    order_id,
    restaurant_id,
    customer_id,
    order_timestamp,
    delivery_timestamp,
    delivery_timestamp - order_timestamp AS delivery_duration
FROM orders
WHERE delivery_timestamp IS NOT NULL;
Enter fullscreen mode Exit fullscreen mode

This is a simplified illustration, since timestamp subtraction and interval handling vary between database systems.

The important point is that demonstrating SQL skills in your portfolio shows that you can work with structured data beyond the typical CSV-and-pandas workflow.

5. Clean and Prepare the Data With Python

Once the data has been retrieved, the next step is making sure it's suitable for analysis and modeling.

Real-world datasets are rarely perfect.

You might encounter:

  • Missing values
  • Duplicate records
  • Incorrect data types
  • Invalid timestamps
  • Outliers
  • Inconsistent categorical values Python libraries such as pandas and NumPy make this process much easier.

For example:

import pandas as pd

df = pd.read_csv("delivery_data.csv")

# Remove duplicate records
df = df.drop_duplicates()

# Convert timestamp columns
df["order_timestamp"] = pd.to_datetime(
    df["order_timestamp"]
)

df["delivery_timestamp"] = pd.to_datetime(
    df["delivery_timestamp"]
)

# Calculate delivery duration in minutes
df["delivery_duration"] = (
    df["delivery_timestamp"] - df["order_timestamp"]
).dt.total_seconds() / 60

# Remove records with invalid target values
df = df[
    df["delivery_duration"].notna()
    & (df["delivery_duration"] > 0)
]
Enter fullscreen mode Exit fullscreen mode

Of course, cleaning decisions should depend on the actual dataset.

For example, don't automatically remove every unusually long delivery. Some of those records might represent exactly the operational problems your model should learn to identify.

Document your decisions and explain why you made them.

That is more valuable than simply showing a notebook full of cleaning commands.

6. Explore the Data Before Building a Model

Exploratory Data Analysis (EDA) is where you start understanding what's happening in your dataset.

Before training any model, investigate the relationships between your variables and delivery duration.

Some useful questions include:

  • Do deliveries take longer during certain hours?
  • Does distance influence delivery duration?
  • Are some restaurants consistently associated with longer delivery times?
  • Does weather affect delivery performance, if weather data is available?
  • Are there noticeable differences between weekdays and weekends? Visualizations can help answer these questions.

For example, you could create:

  • Histograms to understand delivery duration distributions.
  • Box plots to identify potential outliers.
  • Scatter plots to investigate relationships between distance and delivery duration.
  • Heatmaps to examine numerical correlations.
  • Time-based visualizations to identify delivery patterns. One important thing: don't turn EDA into a collection of random charts.

Every visualization should help answer a question.

A good portfolio project doesn't just show what the data looks like. It explains what you learned from looking at it.

7. Feature Engineering: Where Things Get Interesting

Feature engineering is one of the most important parts of a predictive modeling project.

Your raw dataset might contain useful information, but that doesn't necessarily mean the information is already in the best form for a machine learning algorithm.

The goal is to transform existing data into features that help the model identify meaningful patterns.

For a delivery duration prediction project, potential engineered features include:

Time-based features

  • Hour of the day
  • Day of the week
  • Weekend indicator
  • Peak-hour indicator Distance-related features
  • Delivery distance
  • Distance categories
  • Distance-to-estimated-duration ratios, where appropriate Order-related features
  • Number of items
  • Total order value
  • Number of distinct items Operational features
  • Restaurant preparation time, if available at prediction time
  • Delivery partner availability, if available
  • Historical restaurant delivery statistics You can also investigate correlations between numerical features and the target variable.

However, remember that correlation doesn't automatically imply causation.

More importantly, make sure that every feature is actually available when the prediction is supposed to happen.

For example, using the actual delivery timestamp to predict delivery duration would introduce target leakage.

The model would effectively be using information that only becomes available after the delivery has already happened.

That's not a useful prediction system.

Use a Scikit-learn Pipeline
One way to make your workflow more reliable is to use a scikit-learn pipeline.

Pipelines help combine preprocessing and model training into one consistent process.

They also reduce the risk of data leakage when preprocessing steps are fitted correctly within the training workflow.

For example:

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.impute import SimpleImputer
from sklearn.ensemble import RandomForestRegressor

numeric_features = [
    "distance",
    "hour",
    "number_of_items"
]

categorical_features = [
    "day_of_week",
    "restaurant_type"
]

numeric_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler", StandardScaler())
])

categorical_pipeline = Pipeline([
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("encoder", OneHotEncoder(handle_unknown="ignore"))
])

preprocessor = ColumnTransformer([
    ("numeric", numeric_pipeline, numeric_features),
    ("categorical", categorical_pipeline, categorical_features)
])

model_pipeline = Pipeline([
    ("preprocessor", preprocessor),
    ("model", RandomForestRegressor(
        n_estimators=100,
        random_state=42
    ))
])
Enter fullscreen mode Exit fullscreen mode

Now preprocessing and model training can be handled together.

This makes experimentation easier and helps ensure that the same transformations are applied during training and prediction.

8. Train Multiple Models Instead of Betting on One

Once your features are ready, it's time to experiment with different models.

Don't immediately jump to the most sophisticated algorithm you know.

Start with a baseline.

For a regression problem like delivery duration prediction, you could experiment with:

  1. Dummy Regressor
  2. Linear Regression
  3. Decision Tree Regressor
  4. Random Forest Regressor
  5. Gradient Boosting
  6. XGBoost

The baseline gives you a reference point.

Without one, you might build a complicated model and celebrate its performance without knowing whether it actually improves on a simple prediction strategy.

Evaluate Models Using RMSE
Since we're predicting a continuous value (delivery duration), regression metrics are appropriate.

One useful metric is Root Mean Squared Error (RMSE).

RMSE measures the typical magnitude of prediction errors while penalizing larger errors more heavily.

Its unit is the same as the target variable.

If delivery duration is measured in minutes, RMSE is also measured in minutes.

For example:

from sklearn.metrics import root_mean_squared_error

predictions = model_pipeline.predict(X_test)

rmse = root_mean_squared_error(
    y_test,
    predictions
)

print(f"RMSE: {rmse:.2f} minutes")
Enter fullscreen mode Exit fullscreen mode

You can compare the performance of different models using a table:

Model RMSE
Dummy Regressor To be measured
Linear Regression To be measured
Decision Tree To be measured
Random Forest To be measured
XGBoost To be measured

The values should come from your actual experiments.

Don't invent performance numbers just to make the project look impressive.

Also consider other metrics, such as MAE, because they can provide additional insight into prediction errors.

The best model isn't necessarily the one with the lowest RMSE alone. Consider its complexity, inference speed, maintainability, and suitability for the intended use case.

9. Deploy the Model as an API

Here's where your project starts becoming more than a machine learning experiment.

A trained model sitting inside a Jupyter notebook isn't particularly useful to someone who wants to interact with it.

You need a way for other applications to send input data and receive predictions.

That's where an API comes in.

You could use FastAPI to expose your trained model.

A simplified example:

from fastapi import FastAPI
from pydantic import BaseModel
import joblib

app = FastAPI()

model = joblib.load("delivery_model.pkl")

class DeliveryRequest(BaseModel):
    distance: float
    hour: int
    number_of_items: int
    day_of_week: str
    restaurant_type: str

@app.post("/predict")
def predict_delivery(request: DeliveryRequest):

    input_data = [[
        request.distance,
        request.hour,
        request.number_of_items,
        request.day_of_week,
        request.restaurant_type
    ]]

    prediction = model.predict(input_data)[0]

    return {
        "predicted_delivery_duration": round(
            float(prediction), 2
        )
    }
Enter fullscreen mode Exit fullscreen mode

This is an illustrative starting point. In a complete implementation, the API should use the exact feature schema and preprocessing pipeline expected by the trained model.

You should also validate inputs, handle errors, and document the endpoint.

Now your model can be accessed by an external application rather than being restricted to a notebook.

10. Build an Interactive Dashboard

![Interactive food delivery duration prediction dashboard(https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/p5rgpfwt8ozvglocs0gl.png)

The final piece is making the solution accessible to people who aren't necessarily developers.

Imagine a simple dashboard where someone can enter:

  • Delivery distance
  • Order time
  • Number of items
  • Restaurant category
  • Day of the week After submitting the information, the dashboard sends a request to your API and displays the estimated delivery duration.

You could build this interface using Streamlit.

The dashboard should make it easy to understand:

  • What information is required.
  • What the predicted delivery duration is.
  • How the prediction might be useful operationally. This is a small but important distinction.

You're no longer just showing someone that you trained a model.

You're showing them how that model could fit into an actual business workflow.

11. What This Means for Your Data Science Portfolio

Let's step back for a moment.

Look at everything this single project demonstrates:

Project Stage Skills Demonstrated
Business problem definition Business understanding
Data retrieval SQL
Data cleaning Python, pandas
Exploratory analysis Statistics, visualization
Feature engineering Domain knowledge, analytical thinking
Model development Machine learning
Model evaluation Statistical reasoning
API deployment Backend development
Interactive dashboard Data communication and product thinking

That's a considerably stronger demonstration of your abilities than a notebook containing five different algorithms and a few evaluation metrics.

It also gives you more interesting things to discuss during interviews.

You can explain why you selected certain features, how you handled missing values, why one model performed better than another, and how you made the final solution accessible.

Those conversations demonstrate problem-solving ability, not just familiarity with Python libraries.

Final Thoughts

If you're building a data science portfolio, don't focus exclusively on how many machine learning algorithms you can implement.

Focus on how effectively you can solve problems.

A good project should tell a complete story:

  • What was the problem?
  • Why did it matter?
  • Where did the data come from?
  • How did you prepare and understand it?
  • What features did you engineer?
  • Which models did you experiment with?
  • How did you evaluate the results?
  • How can someone actually use the solution? You don't need ten unfinished notebooks in your GitHub repository.

A few well-executed, end-to-end projects can communicate far more about your abilities.

Build projects that demonstrate how you think, how you solve problems, and how you turn data into something useful.

That's what makes a data science portfolio stand out.

Top comments (0)