cell-03-COUNTRY TARGET ENCODING

# ============================================================
# COUNTRY TARGET ENCODING
# ============================================================

def fit_country_target_encoding(country_series, y):

    temp = pd.DataFrame({
        "Country": country_series,
        "Target": np.asarray(y)
    })

    global_mean = temp["Target"].mean()

    country_means = (
        temp.groupby("Country")["Target"].mean()
    )

    return country_means, global_mean


def apply_country_target_encoding(
    country_series,
    country_means,
    global_mean
):
    return (
        pd.Series(country_series)
        .map(country_means)
        .fillna(global_mean)
        .to_numpy()
    )

Cell Purpose

This part of the notebook defines two functions for Country Target Encoding:

1. fit_country_target_encoding() → learns the country-to-PCF mapping.
2. apply_country_target_encoding() → applies that learned mapping to another dataset.

Think of it simply as:

Training data
     ↓
fit_country_target_encoding()
     ↓
Country → Mean PCF mapping
     ↓
apply_country_target_encoding()
     ↓
Numerical country feature

The important distinction is:

Fit = learn the mapping. Apply = use the mapping.

---

Function 1 — fit_country_target_encoding()

def fit_country_target_encoding(country_series, y):

What does this line do?

This defines a function named:

fit_country_target_encoding

It takes two inputs:

country_series → Country values
y              → PCF target values

For example:

Country        PCF
Germany        20
USA            40
Germany        30
UK             10

Here:

country_series

contains:

Germany
USA
Germany
UK

and:

y

contains:

20
40
30
10

Presentation

“This function takes the country feature and the corresponding PCF target values as inputs and learns a target-encoding mapping.”

---

Creating the temporary DataFrame

temp = pd.DataFrame({
    "Country": country_series,
    "Target": np.asarray(y)
})

This creates a temporary dataframe with two columns:

Country
Target

For example:

Country	Target
Germany	20
USA	40
Germany	30
UK	10

Why create this dataframe?

Because it makes it easy to calculate the target mean for each country using:

groupby()

What does np.asarray(y) do?

It converts y into a NumPy array.

For example:

y
↓
[20, 40, 30, 10]

This makes sure the target is in a suitable array format for constructing the dataframe.

Presentation

“I create a temporary dataframe containing country and its corresponding target values. I convert the target to a NumPy array to ensure a consistent numerical array representation.”

---

Calculating the global mean

global_mean = temp["Target"].mean()

This calculates the overall average PCF across all observations.

Using the example:

20 + 40 + 30 + 10
----------------- = 25
        4

So:

global_mean = 25

Why do we need the global mean?

It acts as a fallback value when a country does not have a learned country-specific mean.

For example, if later we encounter:

France

but France wasn't present in the mapping, we can use:

global_mean = 25

instead.

Presentation

“I calculate the overall target mean, which will later be used as a fallback when an unseen country does not have a country-specific encoding.”

---

Calculating country-specific means

country_means = (
    temp.groupby("Country")["Target"].mean()
)

This is the main target-encoding operation.

First:

temp.groupby("Country")

groups all observations according to country.

Then:

["Target"]

selects the PCF target values.

Finally:

.mean()

calculates the average PCF for each country.

For example:

Country	Target values	Mean
Germany	20, 30	25
USA	40	40
UK	10	10

So country_means becomes approximately:

Germany → 25
USA     → 40
UK      → 10

This is the mapping that will be used later.

Presentation

“Here I group the observations by country and calculate the mean PCF for each country. This creates the numerical mapping used for target encoding.”

---

Returning the results

return country_means, global_mean

The function returns two outputs:

1. country_means
2. global_mean

So conceptually:

country_means:
Germany → 25
USA     → 40
UK      → 10

global_mean:
25

Presentation

“The function returns both the country-specific means and the overall mean, because I need the country mapping for known countries and the global mean as a fallback for unseen countries.”

---

Complete Function 1

def fit_country_target_encoding(country_series, y):

    temp = pd.DataFrame({
        "Country": country_series,
        "Target": np.asarray(y)
    })

    global_mean = temp["Target"].mean()

    country_means = (
        temp.groupby("Country")["Target"].mean()
    )

    return country_means, global_mean

Simple way to remember it

Country + PCF
     ↓
Temporary DataFrame
     ↓
Calculate overall PCF mean
     ↓
Calculate PCF mean per country
     ↓
Return both

---

Function 2 — apply_country_target_encoding()

def apply_country_target_encoding(
    country_series,
    country_means,
    global_mean
):

What does this function do?

This function does not learn a new mapping.

It takes the mapping created by the previous function and applies it to country values.

Inputs:

country_series → countries that need encoding
country_means  → learned country → mean PCF mapping
global_mean    → fallback value

Important distinction

Function 1:

learns

Function 2:

applies

This distinction is very important in your viva.

---

Converting country to Series

pd.Series(country_series)

This ensures country_series is treated as a pandas Series.

For example:

Germany
USA
UK
France

becomes a pandas Series that can use .map().

---

Mapping country to target mean

.map(country_means)

This looks up each country in the previously learned mapping.

Suppose:

country_means:

Germany → 25
USA     → 40
UK      → 10

And the new data contains:

Germany
USA
UK
Germany

After .map(country_means):

25
40
10
25

So the categorical country values have now been converted into numerical values.

Presentation

“I use the learned country-to-mean mapping to convert each country into a numerical target-encoded value.”

---

Handling unseen countries

.fillna(global_mean)

Suppose the new data contains:

France

but France wasn't present in the learned mapping.

Then:

.map(country_means)

produces:

NaN

Instead of keeping NaN, we replace it with:

global_mean

For example:

France → 25

if the global mean is 25.

Why?

Because the model still needs a numerical feature.

Presentation

“If a country is not present in the learned mapping, the mapping returns NaN. I replace that with the overall training mean so unseen categories can still be represented numerically.”

---

Convert to NumPy array

.to_numpy()

This converts the pandas Series into a NumPy array.

For example:

Pandas Series
     ↓
NumPy array
[25, 40, 10, 25]

Why?

Later in your code, you use:

np.hstack()

to combine this country feature with other numerical features.

So a NumPy array is convenient for that operation.

Presentation

“Finally, I convert the encoded values into a NumPy array so they can be combined with the other numerical feature matrices later.”

---

Complete Function 2

def apply_country_target_encoding(
    country_series,
    country_means,
    global_mean
):
    return (
        pd.Series(country_series)
        .map(country_means)
        .fillna(global_mean)
        .to_numpy()
    )

Simple way to remember it

Country
   ↓
Look up learned country mean
   ↓
Known country → country mean
Unseen country → global mean
   ↓
NumPy array

---

Function 1 + Function 2 Together

This is the most important concept.

Function 1

fit_country_target_encoding()

Learns:

Germany → 25
USA     → 40
UK      → 10

Function 2

apply_country_target_encoding()

Uses:

Germany → 25
USA     → 40
UK      → 10

to transform new observations.

So:

                TRAINING DATA
                     ↓
        fit_country_target_encoding()
                     ↓
             Country Mapping
                     ↓
        apply_country_target_encoding()
                     ↓
              Numerical Feature

---

Important Viva Question — Target Leakage

Teacher: “Is this target encoding leakage-free?”

Don't automatically say yes.

The basic function:

fit_country_target_encoding()

calculates country means using the target values of the data passed to it.

If you use it directly on the same training observations to create their own features, an observation's own target can contribute to its encoded value.

That can cause target leakage.

That's why your notebook later has:

fit_country_target_encoding_oof()

which uses Out-of-Fold encoding for the training observations.

Strong answer

“Basic target encoding can introduce target leakage if each observation contributes its own target to its encoded feature. That's why for the training data I use the OOF version later, where the encoding for each observation is calculated from other folds.”

This is a very important point to understand before your viva.

---

🎤 How to Present These Two Functions

Start:

“These two functions implement the basic country target-encoding mechanism. The first function learns the mapping, and the second function applies that mapping.”

Then Function 1:

“First, I combine country and the corresponding PCF values into a temporary dataframe. I calculate the global PCF mean and then calculate the mean PCF for each country. These two values are returned.”

Then Function 2:

“The second function takes the learned country means and maps each country to its corresponding numerical value. For an unseen country, I use the global mean as a fallback, and finally convert the result to a NumPy array.”

Then connect to the next function:

“However, because this is target encoding, using the target directly can create leakage for training observations. Therefore, later in the notebook I use out-of-fold target encoding to address this issue.”

That last sentence is excellent for your viva because it shows you understand not only what your code does, but why the next function exists.

For this encoding cell

• Trace one row through both functions