part_1_customer_churn_prediction_mlopszoomcamp

🌐 Web Application Service - Architecture & Design

Customer Churn Prediction by Abdelrahman-Adnan

---

� What You'll Learn

This tutorial teaches you how to build a production-ready web application for machine learning predictions. By the end, you'll understand:

• 🎯 How to create APIs for ML model serving
• 📊 How to build interactive dashboards for business users
• 🔄 How to integrate models into web applications
• 🛡️ How to handle errors and validate user inputs
• 📈 How to display ML results in user-friendly formats

🤔 Why Do We Need a Web Application Service?

Imagine you've built an amazing machine learning model that predicts customer churn. But how do people actually use it?

The Problem:

• Data scientists can run models in Jupyter notebooks
• But business users need simple interfaces
• Developers need API endpoints to integrate into other systems
• Models sitting on servers are useless without access points

Our Solution:
The Web Application Service acts as a bridge between your ML models and real users. Think of it like a restaurant:

• 🤖 The Model = The kitchen (where the magic happens)
• 🌐 The Web App = The restaurant interface (menu, waiters, tables)
• 👥 Users = Customers who want to enjoy the food

🎯 What This Service Does

For Business Users:

• Provides a simple dashboard where they can input customer data
• Shows churn probability in easy-to-understand formats
• Gives actionable recommendations for customer retention

For Developers:

• Offers REST API endpoints for system integration
• Provides automatic documentation
• Handles authentication and rate limiting

For the System:

• Manages model loading and caching for fast responses
• Handles multiple concurrent requests
• Logs all predictions for monitoring

🏗️ Understanding the Architecture

Let's break down how our web application is structured. Think of it like building a house:

📐 The Blueprint: System Architecture

🏠 Web Application House
├── 🚪 Front Door (API Gateway)
├── 🛋️ Living Room (Streamlit Dashboard) 
├── 🍴 Kitchen (FastAPI Backend)
├── 📚 Library (Model Storage)
└── 🔧 Utility Room (Shared Components)

Detailed Architecture Explanation:

1. 🚪 The Front Door (Entry Points)

• Streamlit Dashboard (Port 8501): Where business users enter
• FastAPI Backend (Port 8000): Where developers and systems connect
• Both doors lead to the same house but serve different visitors

2. 🛋️ Living Room (User Interface)

• Comfortable space for business users
• Easy-to-use forms and buttons
• Visual charts and graphs
• No technical knowledge required

3. 🍴 Kitchen (Processing Engine)

• Where the real work happens
• Takes raw ingredients (customer data)
• Follows recipes (ML models)
• Produces meals (predictions)

4. 📚 Library (Knowledge Storage)

• Stores trained models
• Keeps data preprocessing tools
• Maintains configuration settings

5. 🔧 Utility Room (Shared Services)

• Logging and monitoring
• Error handling
• Security features
• Performance optimization

📁 Project File Structure (What Goes Where)

services/web-app/
├── 🎯 fastapi_app.py          # Main kitchen (API server)
├── 📊 streamlit_app.py        # Living room (dashboard)
├── 🐳 Dockerfile             # House blueprint for containers
├── 📋 requirements.txt        # Shopping list for dependencies
├── 📂 utils/                  # Utility room contents
│   ├── 🤖 model_loader.py     # Robot that fetches models
│   ├── 🔧 preprocessing.py    # Data cleaning tools
│   └── ✅ validators.py       # Quality control inspectors
└── 📂 config/                 # House rules and settings
    ├── ⚙️ settings.py         # Master control panel
    └── 📝 logging.conf        # Communication system setup

🎓 Tutorial: Building the FastAPI Backend

Let's build our API step by step, like learning to cook:

Step 1: Setting Up the Kitchen (Basic FastAPI Setup)

What we're doing: Creating the foundation for our API server.

# fastapi_app.py - Our main kitchen setup
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import logging

# 🏗️ Create our restaurant (FastAPI app)
app = FastAPI(
    title="Churn Prediction Kitchen",          # Restaurant name
    description="Serves fresh ML predictions",  # What we do
    version="1.0.0"                            # Our recipe book version
)

# 📝 Setup the communication system (logging)
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# 🎯 A simple welcome message (like a restaurant sign)
@app.get("/")
async def welcome():
    return {"message": "Welcome to the Churn Prediction Kitchen! 🍳"}

Why this works:

• FastAPI() creates our web server - like opening a restaurant
• title and description help people understand what we do
• logging helps us track what's happening - like a kitchen log book
• The @app.get("/") creates a simple welcome page

Step 2: Defining Our Menu (Data Models)

What we're doing: Defining what kind of orders (data) we accept.

# 📋 Our menu - what customers can order
class CustomerData(BaseModel):
    """
    This is like a restaurant order form.
    Customers must fill out all required fields.
    """

    # 👥 Customer Demographics (required info)
    gender: str                    # "Male" or "Female"
    SeniorCitizen: int            # 0 or 1 (like a checkbox)
    Partner: str                  # "Yes" or "No"
    Dependents: str               # "Yes" or "No"

    # 📞 Service Information
    tenure: int                   # How many months they've been a customer
    PhoneService: str             # "Yes" or "No"
    InternetService: str          # "DSL", "Fiber optic", or "No"

    # 💰 Billing Information
    MonthlyCharges: float         # How much they pay per month
    TotalCharges: str             # Total amount paid (comes as text)
    PaymentMethod: str            # How they pay
    Contract: str                 # Contract type

    # ✅ Built-in validation (quality control)
    @validator('tenure')
    def check_tenure(cls, v):
        if v < 0 or v > 100:
            raise ValueError('Tenure must be between 0 and 100 months')
        return v

    @validator('MonthlyCharges')
    def check_charges(cls, v):
        if v <= 0:
            raise ValueError('Monthly charges must be positive')
        return v

# 📊 What we serve back (our prediction results)
class PredictionResponse(BaseModel):
    """
    This is what customers get back - like a receipt with results
    """
    churn_probability: float      # Chance of churning (0 to 1)
    risk_level: str              # "Low", "Medium", or "High"
    confidence: float            # How sure we are (0 to 1)
    recommendations: list        # What to do about it
    model_version: str           # Which recipe we used

Why this is important:

• BaseModel ensures data quality - like checking IDs at a club
• @validator functions catch bad data before it causes problems
• Clear field names help users understand what to provide
• Response models guarantee consistent output format

Step 3: Loading Our Secret Recipe (Model Loading)

What we're doing: Getting our trained ML model ready to make predictions.

# 🤖 Our robot chef that loads and manages models
class ModelManager:
    """
    Think of this as our head chef who:
    - Knows where all recipes are stored
    - Can quickly access any cooking tool
    - Ensures everything is fresh and ready
    """

    def __init__(self):
        self.model = None           # Our main recipe
        self.scaler = None          # Tool for standardizing ingredients
        self.encoder = None         # Tool for converting text to numbers
        self.model_version = None   # Which version of recipe we're using

        # 🚀 Load everything when the kitchen opens
        self._load_models()

    def _load_models(self):
        """Load our cooking tools and recipes"""
        try:
            # 📚 Try to get the latest recipe from our recipe book (MLflow)
            import mlflow

            # 🔍 Look for the production recipe
            model_uri = "models:/churn-prediction/Production"
            self.model = mlflow.sklearn.load_model(model_uri)

            # 🛠️ Load our cooking tools
            import joblib
            self.scaler = joblib.load('artifacts/scaler.pkl')
            self.encoder = joblib.load('artifacts/encoder.pkl')

            logger.info("✅ Kitchen is ready! All tools and recipes loaded.")

        except Exception as e:
            logger.error(f"❌ Kitchen setup failed: {str(e)}")
            # 🚨 If something goes wrong, try backup recipes
            self._load_backup_models()

    def _load_backup_models(self):
        """If our main recipes fail, use backup ones"""
        try:
            import joblib
            self.model = joblib.load('models/backup_model.pkl')
            self.scaler = joblib.load('models/backup_scaler.pkl')
            self.encoder = joblib.load('models/backup_encoder.pkl')
            self.model_version = "backup"

            logger.info("📁 Using backup recipes - kitchen operational!")

        except Exception as e:
            logger.critical(f"💥 Complete kitchen failure: {str(e)}")
            raise

# 🏭 Create our model manager (hire our head chef)
model_manager = ModelManager()

Key concepts explained:

• Model Manager: Like a head chef who manages all cooking tools
• MLflow integration: Like having a digital recipe book that updates automatically
• Backup system: Always have Plan B when technology fails
• Error handling: Graceful failure - the show must go on

Step 4: The Main Cooking Process (Prediction Endpoint)

What we're doing: Creating the actual service that makes predictions.

@app.post("/predict")
async def predict_churn(customer_data: CustomerData):
    """
    This is our main cooking station where we:
    1. Take a customer order (input data)
    2. Prepare the ingredients (preprocess)
    3. Cook the meal (run prediction)
    4. Serve the result (return response)
    """

    try:
        # 🕐 Start timing (for performance monitoring)
        import time
        start_time = time.time()

        # 🥕 Step 1: Prepare the ingredients (preprocess data)
        logger.info("🥕 Preparing customer data...")
        processed_data = prepare_customer_data(customer_data)

        # 🍳 Step 2: Cook the prediction (run model)
        logger.info("🍳 Making prediction...")
        churn_probability = model_manager.model.predict_proba(processed_data)[0][1]

        # 🎨 Step 3: Determine risk level (business logic)
        risk_level = determine_risk_level(churn_probability)

        # 💡 Step 4: Generate recommendations (business value)
        recommendations = generate_recommendations(churn_probability, customer_data)

        # ⏱️ Step 5: Calculate how long it took
        processing_time = time.time() - start_time

        # 📝 Step 6: Log what happened (for monitoring)
        logger.info(f"✅ Prediction complete: {churn_probability:.4f} in {processing_time:.3f}s")

        # 🍽️ Step 7: Serve the result
        return PredictionResponse(
            churn_probability=round(churn_probability, 4),
            risk_level=risk_level,
            confidence=calculate_confidence(churn_probability),
            recommendations=recommendations,
            model_version=model_manager.model_version
        )

    except Exception as e:
        # 🚨 If something goes wrong, handle it gracefully
        logger.error(f"❌ Prediction failed: {str(e)}")
        raise HTTPException(status_code=500, detail=f"Prediction error: {str(e)}")

def prepare_customer_data(customer_data: CustomerData):
    """
    Turn raw customer data into model-ready format
    Like washing and chopping vegetables before cooking
    """

    # 📊 Convert to pandas DataFrame (model's favorite format)
    import pandas as pd
    data_dict = customer_data.dict()
    df = pd.DataFrame([data_dict])

    # 🔧 Fix TotalCharges (sometimes it comes as text)
    df['TotalCharges'] = pd.to_numeric(df['TotalCharges'], errors='coerce')
    df['TotalCharges'].fillna(0, inplace=True)

    # 🎯 Add calculated features (like making sauce from basic ingredients)
    df['AvgMonthlyCharges'] = df['TotalCharges'] / (df['tenure'] + 1)
    df['ChargesRatio'] = df['MonthlyCharges'] / (df['TotalCharges'] + 1)

    # 🏷️ Convert categories to numbers (models speak math, not English)
    categorical_columns = ['gender', 'Partner', 'Dependents', 'Contract', 'PaymentMethod']
    for col in categorical_columns:
        if col in df.columns:
            df[col] = model_manager.encoder.transform(df[col].astype(str))

    # 📏 Scale numerical features (make all numbers play nicely together)
    numerical_columns = ['tenure', 'MonthlyCharges', 'TotalCharges']
    df[numerical_columns] = model_manager.scaler.transform(df[numerical_columns])

    return df.values

def determine_risk_level(probability):
    """Convert probability to business-friendly risk levels"""
    if probability > 0.7:
        return "High Risk 🔴"
    elif probability > 0.4:
        return "Medium Risk 🟡"
    else:
        return "Low Risk 🟢"

def generate_recommendations(probability, customer_data):
    """Provide actionable business recommendations"""
    recommendations = []

    if probability > 0.7:  # High risk customers
        recommendations.extend([
            "🎯 Contact customer immediately with retention offer",
            "💰 Provide significant discount or service upgrade",
            "📞 Schedule call with customer success manager"
        ])

        # Specific recommendations based on customer profile
        if customer_data.Contract == "Month-to-month":
            recommendations.append("📝 Offer annual contract with 20% discount")

    elif probability > 0.4:  # Medium risk
        recommendations.extend([
            "📧 Send targeted retention email campaign",
            "🎁 Offer service upgrade or add-on discount",
            "📊 Monitor closely for changes in usage"
        ])
    else:  # Low risk
        recommendations.extend([
            "😊 Continue standard customer engagement",
            "⭐ Consider for premium service upselling"
        ])

    return recommendations

What each part does:

1. @app.post("/predict"): Creates an endpoint that accepts POST requests
2. Data preprocessing: Converts human-readable data to model format
3. Prediction: Uses our trained model to calculate churn probability
4. Business logic: Converts probability to actionable insights
5. Error handling: Ensures the service stays running even when things go wrong
6. Logging: Tracks performance and helps with debugging

🔧 Configuration Management

What we're doing: Setting up a central control panel for our application.

# config/settings.py - Our master control panel
from pydantic import BaseSettings

class Settings(BaseSettings):
    """
    This is like the control room of our kitchen.
    All the important settings are here in one place.
    """

    # 🏷️ Basic application info
    APP_NAME: str = "Churn Prediction API"
    APP_VERSION: str = "1.0.0"
    DEBUG: bool = False                    # Turn on/off detailed error messages

    # 🌐 Network settings (where and how to serve)
    API_HOST: str = "0.0.0.0"             # Listen on all network interfaces
    API_PORT: int = 8000                  # Which door customers use

    # 📚 Model settings (where to find our recipes)
    MLFLOW_TRACKING_URI: str = "http://localhost:5000"
    MODEL_NAME: str = "churn-prediction"
    MODEL_STAGE: str = "Production"        # Which version to use

    # 📁 File locations (where we keep our tools)
    MODEL_PATH: str = "models/"
    ARTIFACTS_PATH: str = "artifacts/"

    # 📝 Logging settings (how much detail to record)
    LOG_LEVEL: str = "INFO"
    LOG_FILE: str = "logs/api.log"

    # 🔒 Security settings
    CORS_ORIGINS: list = ["*"]            # Which websites can use our API
    API_KEY_REQUIRED: bool = False        # Do users need special access?

    class Config:
        env_file = ".env"                 # Load settings from environment file
        case_sensitive = True             # Settings names must match exactly

# 🌍 Create our global settings object
settings = Settings()

Why configuration matters:

• Centralized control: Change settings in one place
• Environment flexibility: Different settings for development vs production
• Security: Keep sensitive information in environment variables
• Maintainability: Easy to update without changing code

This architecture tutorial shows you how to build a robust, production-ready web application for ML model serving. Each component has a specific purpose and works together to create a reliable service that business users love and developers can easily maintain.

mlopszoomcamp