# ============================================================
# OOF COUNTRY TARGET ENCODING
# ============================================================
from sklearn.model_selection import KFold
def fit_country_target_encoding_oof(
country_series,
y,
n_splits=5,
random_state=42
):
country_series = pd.Series(
country_series
).reset_index(drop=True)
y = np.asarray(y)
oof_encoded = np.zeros(len(country_series))
kf = KFold(
n_splits=n_splits,
shuffle=True,
random_state=random_state
)
for train_idx, valid_idx in kf.split(country_series):
temp_df = pd.DataFrame({
"Country": country_series.iloc[train_idx],
"Target": y[train_idx]
})
country_means = (
temp_df.groupby("Country")["Target"].mean()
)
global_mean = temp_df["Target"].mean()
oof_encoded[valid_idx] = (
country_series.iloc[valid_idx]
.map(country_means)
.fillna(global_mean)
)
# Mapping for outer validation/test data
full_train_df = pd.DataFrame({
"Country": country_series,
"Target": y
})
country_means = (
full_train_df.groupby("Country")["Target"].mean()
)
global_mean = full_train_df["Target"].mean()
return oof_encoded, country_means, global_mean
Cell Purpose
This function implements Out-of-Fold (OOF) Country Target Encoding.
The main reason for using OOF encoding is to reduce target leakage.
Normal target encoding calculates:
Country → Mean PCF
But if an observation's own PCF contributes to the mean used to encode that same observation, the feature contains information from its own target.
OOF encoding avoids this by making sure that each training observation is encoded using a mapping learned from other folds, not from its own fold.
The overall idea is:
Training data
↓
Split into 5 folds
↓
4 folds → learn country means
1 fold → receive encoding
↓
Repeat for every fold
↓
OOF encoded training feature
1. Import KFold
from sklearn.model_selection import KFold
What does it do?
This imports KFold from Scikit-learn.
KFold is used to divide the training data into multiple folds.
You use:
n_splits=5
so the data is divided into 5 folds.
Presentation
“I use Scikit-learn's KFold to create the folds required for out-of-fold target encoding.”
2. Define the function
def fit_country_target_encoding_oof(
country_series,
y,
n_splits=5,
random_state=42
):
This defines the OOF target-encoding function.
It takes four inputs:
country_series → Country values
y → PCF target values
n_splits → Number of folds
random_state → Reproducibility
The defaults are:
n_splits = 5
random_state = 42
Presentation
“The function takes country values and the target PCF values, and uses five-fold cross-validation to generate leakage-aware target encodings.”
3. Convert country to Series and reset index
country_series = pd.Series(
country_series
).reset_index(drop=True)
There are two operations here.
pd.Series(country_series)
This ensures the country data is represented as a pandas Series.
.reset_index(drop=True)
This gives the Series a clean sequential index:
0
1
2
3
4
...
The old index is discarded because of:
drop=True
Why is this useful?
Later you use:
country_series.iloc[train_idx]
and:
country_series.iloc[valid_idx]
so having clean, sequential indices makes the fold indexing consistent.
Presentation
“I convert the country input into a pandas Series and reset its index so that the fold indices can be applied consistently.”
4. Convert target to NumPy array
y = np.asarray(y)
This converts the target values into a NumPy array.
For example:
PCF values
↓
[10.5, 20.2, 15.7, 30.1, ...]
Why?
Later the code uses:
y[train_idx]
and:
y[valid_idx]
to select target values according to the fold indexes.
Presentation
“I convert the target into a NumPy array so that the fold indices can be used directly to select the corresponding target observations.”
5. Create an empty OOF array
oof_encoded = np.zeros(len(country_series))
This creates an array of zeros with the same number of observations as the country data.
For example, if there are 1,000 training observations:
oof_encoded
↓
[0, 0, 0, 0, ...] # 1,000 values
Later, each zero will be replaced by the appropriate OOF country encoding.
Why?
We need somewhere to store the encoded value for every training observation.
Presentation
“I initialise an array to store the out-of-fold encoded value for every training observation.”
6. Create the K-Fold splitter
kf = KFold(
n_splits=n_splits,
shuffle=True,
random_state=random_state
)
This creates the five-fold splitting strategy.
n_splits=n_splits
Since the default is:
5
the training data is divided into 5 folds.
Conceptually:
Fold 1 | Fold 2 | Fold 3 | Fold 4 | Fold 5
shuffle=True
Before creating the folds, the observations are shuffled.
This helps avoid folds being determined simply by the original row order.
random_state=random_state
The shuffling becomes reproducible.
If the same data and random state are used, the same fold assignment is produced.
Presentation
“I use five folds with shuffling and a fixed random state. This gives me a reproducible partition while reducing dependence on the original row ordering.”
7. Start the fold loop
for train_idx, valid_idx in kf.split(country_series):
This is the core of OOF encoding.
For every iteration, KFold gives two sets of indexes:
train_idx
valid_idx
For example, in one iteration:
Fold 1 → validation
Folds 2–5 → training
Then in the next iteration:
Fold 2 → validation
Folds 1,3,4,5 → training
And so on.
So every observation eventually becomes part of a validation fold exactly once.
Presentation
“For each fold, I use the other folds to learn the target encoding and use the held-out fold to receive the encoding.”
This sentence is very important.
8. Create the fold-training dataframe
temp_df = pd.DataFrame({
"Country": country_series.iloc[train_idx],
"Target": y[train_idx]
})
This creates a temporary dataframe using only the observations in train_idx.
For example:
Country Target
Germany 20
USA 40
Germany 30
UK 10
The important point is:
The observations in
valid_idxare NOT included here.
Therefore their target values are not used to calculate the encoding.
Presentation
“For each fold, I create the mapping only from the fold-training observations. The held-out fold is excluded from this calculation, which is what prevents its target values from influencing its own encoding.”
9. Calculate country means for the fold
country_means = (
temp_df.groupby("Country")["Target"].mean()
)
This calculates the mean PCF for each country using only the fold-training data.
For example:
Germany → 25
USA → 40
UK → 10
But these values were calculated only from the four training folds in that iteration.
Presentation
“I calculate the country-specific mean PCF using only the fold-training data.”
10. Calculate fold global mean
global_mean = temp_df["Target"].mean()
This calculates the overall mean PCF using the fold-training observations.
This will be used as the fallback if the validation fold contains a country that wasn't present in the fold-training data.
For example:
Training folds:
Germany
USA
UK
Validation fold:
France
There is no learned France mapping.
Therefore:
France → fold global mean
Presentation
“I also calculate the global mean from the fold-training data so that unseen countries can receive a fallback value.”
11. Generate the OOF encoding
oof_encoded[valid_idx] = (
country_series.iloc[valid_idx]
.map(country_means)
.fillna(global_mean)
)
This is the most important line in the function.
Let's break it down.
Select validation countries
country_series.iloc[valid_idx]
This selects only the countries belonging to the held-out fold.
Apply the learned mapping
.map(country_means)
Each validation country is mapped to the country mean calculated from the other folds.
For example:
Validation country:
Germany
Mapping learned from other folds:
Germany → 25
Result:
25
Handle unseen country
.fillna(global_mean)
If a country doesn't exist in the mapping:
France → NaN
it becomes:
France → global_mean
Store the result
oof_encoded[valid_idx] = ...
The encoded values are placed into the correct positions in the OOF array.
After all five iterations, every observation has received an encoding generated without using its own fold's target values.
Presentation
“This is the key OOF step. I take the held-out fold, map its countries using the country means learned from the other folds, use the fold global mean for unseen countries, and store those values in their original positions.”
12. Create the full training dataframe
After the OOF loop finishes:
full_train_df = pd.DataFrame({
"Country": country_series,
"Target": y
})
Now we create a dataframe containing the entire development/training data.
This is different from temp_df.
During the OOF loop:
temp_df
contains only the fold-training observations.
Now:
full_train_df
contains all training observations.
Why?
We need a final country mapping that can later be applied to outer validation/test data.
Presentation
“After generating the leakage-safe OOF features for the training observations, I use the full development data to learn the final country mapping that will be applied to outer validation or test data.”
13. Calculate final country means
country_means = (
full_train_df.groupby("Country")["Target"].mean()
)
Now country means are calculated using all development/training observations.
For example:
Germany → 27.4
USA → 41.2
UK → 13.8
These are the final mappings for external data.
Important distinction
This mapping is not used to generate the OOF training values.
The OOF values were already generated using fold-specific mappings.
This full-data mapping is for:
Validation / Test data
14. Calculate final global mean
global_mean = full_train_df["Target"].mean()
This calculates the overall mean PCF across the entire development/training dataset.
It becomes the fallback value for an unseen country in outer validation/test data.
15. Return the three outputs
return oof_encoded, country_means, global_mean
The function returns three things:
1. oof_encoded
Leakage-aware country encoding for the training observations.
2. country_means
Final country-to-PCF mapping learned from the full development data.
3. global_mean
Final overall PCF mean used as a fallback for unseen countries.
So:
fit_country_target_encoding_oof()
↓
┌───────────┼──────────────┐
↓ ↓ ↓
OOF Country Global
encoding mapping mean
↓ ↓ ↓
Train Validation/ Unseen
features test country
Why is this function important?
The key difference from your first target-encoding function is:
Normal Target Encoding
↓
Country mean may include own target
↓
Potential leakage
Whereas:
OOF Target Encoding
↓
Each observation encoded using other folds
↓
Reduces target leakage
Very Important Viva Question
Teacher: “Why are you using OOF target encoding?”
Answer:
“Because country encoding uses the target variable. If I calculate the country mean using the same observation that I am encoding, its own target can influence the feature and cause target leakage. OOF encoding avoids this by calculating each training observation's encoding from other folds.”
Teacher: “Why do you fit the final mapping on the full training data?”
Answer:
“Once the leakage-safe OOF training features have been generated, I can use all available development data to learn the final mapping. This mapping is then applied to validation or test data without using their target values.”
Teacher: “Does OOF completely eliminate every possible source of leakage?”
A careful answer:
“It prevents the direct self-target leakage associated with target encoding of the training observations. Other preprocessing steps must also be fitted only on the development data to maintain a leakage-free pipeline.”
Teacher: “Why five folds?”
Answer:
“Five folds is a practical choice that provides multiple out-of-fold estimates while keeping enough observations in each fold to estimate the country means. It is a design choice rather than a universally optimal value.”
Teacher: “Why shuffle=True?”
Answer:
“To randomise the fold assignment rather than relying on the original ordering of the observations.”
Teacher: “Why random_state=42?”
Answer:
“To make the shuffled fold assignment reproducible.”
Teacher: “What happens if a country appears in validation but not in the training folds?”
Answer:
“Its country-specific mapping will be unavailable, so
.fillna(global_mean)assigns the global mean calculated from the fold-training data.”
Teacher: “Why not calculate the global mean using the validation fold?”
Answer:
“Because that would use information from the held-out fold. The fallback must also be learned from the training portion to avoid leakage.”
Presentation Script
“This function implements out-of-fold country target encoding to address the leakage risk of ordinary target encoding. I first standardise the country and target inputs and initialise an array for the OOF values. I then create five shuffled folds. For each iteration, I calculate country-specific PCF means using only the fold-training observations and apply those mappings to the held-out fold. If a country is unseen, I use the training-fold global mean. After every observation has received an out-of-fold encoding, I fit one final country mapping using the full development data. This final mapping and its global mean are returned for transforming outer validation or test data.”
The Most Important Line
If the teacher asks you to point out the core line, point here:
oof_encoded[valid_idx] = (
country_series.iloc[valid_idx]
.map(country_means)
.fillna(global_mean)
)
And say:
“This is where the held-out fold receives an encoding learned without using its own target values. That is the key mechanism that reduces target leakage.”
One Simple Example to Memorise
Suppose there are 5 observations:
| Observation | Country | PCF |
|---|---|---|
| 1 | Germany | 10 |
| 2 | Germany | 20 |
| 3 | USA | 30 |
| 4 | USA | 40 |
| 5 | Germany | 30 |
If observation 5 is in the validation fold, its encoding should not be calculated using its own PCF of 30.
Instead, Germany's mean is calculated from the training folds, for example:
Germany → (10 + 20) / 2 = 15
So observation 5 receives:
Germany → 15
rather than a mean that includes its own target.
That is the core idea of OOF target encoding.
For this OOF encoding cell
- Trace a row through all five folds
- Explain the final mapping boundary
Top comments (0)