# ============================================================
# STRUCTURED FEATURE BUILDER
# ============================================================
def build_structured_features(
X_train,
y_train,
X_valid
):
train_part = X_train.copy()
valid_part = X_valid.copy()
# Product Weight → Winsorization
lower_limit, upper_limit = fit_winsorization(
train_part,
WEIGHT_COL
)
train_part = apply_winsorization(
train_part,
WEIGHT_COL,
lower_limit,
upper_limit
)
valid_part = apply_winsorization(
valid_part,
WEIGHT_COL,
lower_limit,
upper_limit
)
# Numeric features
numeric_cols = [
YEAR_COL,
WEIGHT_COL
]
train_numeric = train_part[numeric_cols].apply(
pd.to_numeric,
errors="coerce"
)
valid_numeric = valid_part[numeric_cols].apply(
pd.to_numeric,
errors="coerce"
)
# Categorical features → One-Hot Encoding
other_ohe = OneHotEncoder(
handle_unknown="ignore",
sparse_output=False
)
train_other_ohe = other_ohe.fit_transform(
train_part[OTHER_CATEGORICAL_FEATURES]
)
valid_other_ohe = other_ohe.transform(
valid_part[OTHER_CATEGORICAL_FEATURES]
)
# Country → OOF Target Encoding
(
train_country,
country_means,
global_mean
) = fit_country_target_encoding_oof(
train_part[COUNTRY_COL],
y_train
)
train_country = train_country.reshape(-1, 1)
valid_country = apply_country_target_encoding(
valid_part[COUNTRY_COL],
country_means,
global_mean
).reshape(-1, 1)
# Combine features
X_train_structured = np.hstack([
train_numeric.to_numpy(),
train_other_ohe,
train_country
])
X_valid_structured = np.hstack([
valid_numeric.to_numpy(),
valid_other_ohe,
valid_country
])
return X_train_structured, X_valid_structured
# ============================================================
# HYBRID FEATURE FUSION
# ============================================================
def fuse_features(
sbert_features,
structured_features
):
return np.hstack([
sbert_features,
structured_features
])
Cell Purpose
This cell has two functions:
-
build_structured_features()→ prepares and combines the structured features. -
fuse_features()→ combines the structured features with the SBERT text features to create the final hybrid representation.
The overall flow is:
X_train / X_valid
↓
Product Weight → Winsorization
↓
Numeric Features
↓
One-Hot Encoding
↓
Country → OOF Target Encoding
↓
Structured Feature Matrix
↓
+
SBERT Features
↓
Hybrid Features
Function 1 — build_structured_features()
1. Function definition
def build_structured_features(
X_train,
y_train,
X_valid
):
This defines the function that builds the structured feature matrix.
It takes three inputs:
X_train → training features
y_train → training PCF target
X_valid → validation features
Why does it need y_train?
Because Country uses target encoding.
The country encoding needs the PCF target values to calculate country-level mean PCF.
Importantly, it uses y_train only, not y_valid.
Presentation
“This function builds the structured feature representation from the training and validation data. It receives the training target as well because country target encoding requires the target values.”
2. Copy the training data
train_part = X_train.copy()
This creates a separate copy of X_train.
Why?
So that the preprocessing operations inside this function don't directly modify the original X_train.
Presentation
“I create a copy of the training data so that the preprocessing inside this function does not modify the original dataframe.”
3. Copy the validation data
valid_part = X_valid.copy()
Same idea for validation data.
X_valid
↓
copy
↓
valid_part
Presentation
“I do the same for the validation data so that I can apply the learned transformations without changing the original validation dataframe.”
Product Weight — Winsorization
4. Learn winsorization limits from training data
lower_limit, upper_limit = fit_winsorization(
train_part,
WEIGHT_COL
)
This calls your previously defined fit_winsorization() function.
It determines:
lower_limit
upper_limit
for the product-weight feature.
For example, conceptually:
Lower limit = 0.1 kg
Upper limit = 100 kg
Values beyond those limits can later be capped.
Important point
The limits are calculated using:
train_part
not:
valid_part
This is important for avoiding data leakage.
Presentation
“First, I learn the winsorization limits from the training data only. This is important because the validation data should not influence the preprocessing parameters.”
5. Apply winsorization to training data
train_part = apply_winsorization(
train_part,
WEIGHT_COL,
lower_limit,
upper_limit
)
This applies the previously learned limits to the training data.
If an observation is above the upper limit, it is capped at the upper limit.
Conceptually:
Before:
1
2
5
100
5000
After:
1
2
5
100
100
if the upper limit were 100.
Why winsorization?
It reduces the influence of extreme values without deleting the observations.
Presentation
“I then apply the learned limits to the training data. Winsorization reduces the influence of extreme product-weight values while retaining the observations.”
6. Apply the same limits to validation data
valid_part = apply_winsorization(
valid_part,
WEIGHT_COL,
lower_limit,
upper_limit
)
Notice something important:
You are not fitting new limits on validation data.
You use:
training limits
↓
training data
+
validation data
This is the correct preprocessing boundary.
Presentation
“I apply the same training-derived limits to the validation data. I do not calculate new limits from validation data because that would allow information from the validation set to influence preprocessing.”
Numeric Features
7. Define numeric columns
numeric_cols = [
YEAR_COL,
WEIGHT_COL
]
This creates a list containing your two numerical features:
Year of reporting
Product weight
Presentation
“Next, I define the numerical features used in the structured representation: reporting year and product weight.”
8. Convert training numeric features
train_numeric = train_part[numeric_cols].apply(
pd.to_numeric,
errors="coerce"
)
First:
train_part[numeric_cols]
selects only the numerical columns.
Then:
.apply(pd.to_numeric)
converts their values into numeric format.
And:
errors="coerce"
means invalid values are converted to NaN.
For example:
"2023" → 2023
"2.5" → 2.5
"abc" → NaN
Presentation
“I convert the numerical features into numeric format, and invalid values are converted to NaN so they can be handled systematically.”
9. Convert validation numeric features
valid_numeric = valid_part[numeric_cols].apply(
pd.to_numeric,
errors="coerce"
)
Exactly the same operation is applied to validation data.
The result is:
train_numeric
valid_numeric
Categorical Features — One-Hot Encoding
10. Create the encoder
other_ohe = OneHotEncoder(
handle_unknown="ignore",
sparse_output=False
)
This creates a One-Hot Encoder.
It is used for:
OTHER_CATEGORICAL_FEATURES
These are the categorical features other than Country.
What does One-Hot Encoding do?
Suppose Industry has:
Food
Energy
Technology
It can become:
Food → [1,0,0]
Energy → [0,1,0]
Technology → [0,0,1]
handle_unknown="ignore"
Suppose training contains:
Food
Energy
Technology
but validation contains:
Healthcare
which was not present during training.
handle_unknown="ignore" prevents the encoder from crashing when it sees this unseen category.
Presentation
“For the other categorical features, I use one-hot encoding. I set
handle_unknownto ignore so that an unseen category in validation data does not cause the transformation to fail.”
11. Fit and transform training categories
train_other_ohe = other_ohe.fit_transform(
train_part[OTHER_CATEGORICAL_FEATURES]
)
This does two things.
fit
The encoder learns the categories from the training data.
transform
It converts the training categories into one-hot numerical vectors.
So:
Training categorical data
↓
fit
↓
Learn categories
↓
transform
↓
One-hot matrix
Presentation
“I fit the encoder only on the training categorical data and transform that data into numerical one-hot features.”
12. Transform validation categories
valid_other_ohe = other_ohe.transform(
valid_part[OTHER_CATEGORICAL_FEATURES]
)
Here you only use:
transform()
You do not use:
fit_transform()
on validation.
Why?
Because the encoder has already learned its category structure from training.
This is a very likely viva question.
Teacher: “Why don't you fit the encoder on validation data?”
Answer:
“Because validation data should remain unseen during fitting. I learn the categorical representation from the training data and only apply that learned representation to validation data.”
This is another form of data leakage prevention.
Country — OOF Target Encoding
13. Call the OOF encoder
(
train_country,
country_means,
global_mean
) = fit_country_target_encoding_oof(
train_part[COUNTRY_COL],
y_train
)
This calls the function you just studied.
It receives:
Country from training data
+
PCF target from training data
and returns three things:
train_country
country_means
global_mean
train_country
These are the OOF target-encoded country values for training observations.
country_means
This is the final country-to-PCF mapping learned from the full training data.
global_mean
This is the overall training PCF mean used as a fallback for unseen countries.
Presentation
“For Country, I use OOF target encoding rather than ordinary one-hot encoding. The training countries are encoded using out-of-fold estimates to reduce target leakage, while the returned mapping is retained for transforming validation data.”
Why is Country treated differently?
This is a likely examiner question.
You have:
Industry → One-Hot Encoding
Protocol → One-Hot Encoding
Stage-level → One-Hot Encoding
Country → Target Encoding
Why?
Because Country is handled as a target-encoded feature.
You can say:
“I treat Country separately because I want a compact numerical representation based on its relationship with PCF. Since target encoding uses the target variable, I use OOF encoding for the training observations to reduce leakage.”
14. Reshape training country feature
train_country = train_country.reshape(-1, 1)
Before reshaping, the array is approximately:
(n,)
After:
(n, 1)
For example:
Before:
[25, 40, 30, 15]
Shape:
(4,)
After:
[[25],
[40],
[30],
[15]]
Shape:
(4, 1)
Why?
Because later you want to horizontally concatenate it with other feature matrices.
Presentation
“I reshape the country encoding into a two-dimensional column so it has the correct matrix shape for feature concatenation.”
15. Encode validation country
valid_country = apply_country_target_encoding(
valid_part[COUNTRY_COL],
country_means,
global_mean
).reshape(-1, 1)
This applies the mapping learned from the training data to validation countries.
Notice:
country_means
global_mean
come from the training process.
The validation target is not provided.
This is important.
Example
Training mapping:
Germany → 25
USA → 40
UK → 15
Validation:
Germany
USA
France
becomes:
25
40
global_mean
if France was unseen.
Then .reshape(-1, 1) turns it into a column.
Presentation
“For validation, I apply the mapping learned from the full training data. I don't use validation targets. If a validation country is unseen, the global training mean is used as the fallback.”
Combining the Structured Features
16. Build training structured matrix
X_train_structured = np.hstack([
train_numeric.to_numpy(),
train_other_ohe,
train_country
])
This is where the different structured feature types are combined.
You have:
train_numeric
+
train_other_ohe
+
train_country
↓
X_train_structured
np.hstack()
Means horizontal stacking.
It puts the feature columns next to each other.
For example:
Numeric OHE Country
[2023, 5] + [1,0,0] + [25]
becomes:
[2023, 5, 1, 0, 0, 25]
Why .to_numpy()?
train_numeric is a pandas DataFrame.
.to_numpy() converts it into a NumPy array so it can be combined consistently with the other NumPy matrices.
Presentation
“I horizontally concatenate the numerical features, one-hot encoded categorical features, and country target encoding to create the final structured training matrix.”
17. Build validation structured matrix
X_valid_structured = np.hstack([
valid_numeric.to_numpy(),
valid_other_ohe,
valid_country
])
Exactly the same process for validation.
So now we have:
X_train_structured
X_valid_structured
with the same feature structure.
18. Return structured features
return X_train_structured, X_valid_structured
The function returns two matrices:
Training structured features
Validation structured features
Presentation
“Finally, the function returns the structured feature matrices for training and validation.”
Function 2 — fuse_features()
Now we move to the second function.
def fuse_features(
sbert_features,
structured_features
):
This function takes two feature representations:
sbert_features
structured_features
What is sbert_features?
These are numerical embeddings generated from your text using SBERT.
They represent semantic information from the textual data.
What is structured_features?
These are the engineered numerical features we just created:
Year
Weight
One-hot categorical variables
Country target encoding
19. Combine SBERT and structured features
return np.hstack([
sbert_features,
structured_features
])
Again, np.hstack() horizontally concatenates the two matrices.
Conceptually:
SBERT embeddings
+
Structured features
↓
Hybrid feature representation
For example:
SBERT:
[0.12, 0.55, 0.31, ...]
Structured:
[2023, 5.2, 1, 0, 25]
↓
Hybrid:
[0.12, 0.55, 0.31, ..., 2023, 5.2, 1, 0, 25]
Why fuse them?
Because they provide different types of information.
SBERT captures:
semantic information from text
Structured features capture:
explicit numerical and categorical information
So the ML model receives both.
Presentation
“The final function performs feature fusion. It horizontally combines the SBERT embeddings with the structured features, producing a hybrid representation that contains both semantic text information and explicit structured information.”
Complete Function Flow
build_structured_features()
X_train + y_train + X_valid
↓
Product Weight
Winsorization
↓
Numeric Features
↓
One-Hot Encoding
↓
Country OOF Encoding
↓
┌─────────┴─────────┐
↓ ↓
X_train_structured X_valid_structured
fuse_features()
SBERT Features
+
Structured Features
↓
Hybrid Features
↓
Machine Learning Model
🎤 Presentation Script for the Whole Cell
“This function builds my structured feature representation. First, I learn the winsorization limits for product weight from the training data and apply the same limits to both training and validation data. I then prepare the numerical features and one-hot encode the other categorical variables, fitting the encoder only on the training data.
For Country, I use out-of-fold target encoding because target encoding can introduce leakage if the target is used to encode the same observation. The OOF encoded training values and the training-derived mapping for validation are then created.
Finally, I concatenate the numerical, one-hot, and country features into structured feature matrices. The second function then combines these structured features with the SBERT embeddings to create the final hybrid representation for the machine-learning models.”
Key Viva Questions
Why do you need y_train in build_structured_features()?
Because Country target encoding uses the training PCF values to calculate the country-level target relationship.
Why is Country not one-hot encoded?
To create a compact numerical representation based on its relationship with PCF, while using OOF encoding to reduce leakage.
Why fit winsorization only on training?
To prevent validation information from influencing the preprocessing limits.
Why fit_transform() for training but transform() for validation?
The encoder learns its categories from training only; validation is only transformed using that learned representation.
Why handle_unknown="ignore"?
To handle categories appearing in validation that were not present during training without producing an error.
Why OOF encoding?
To reduce target leakage when creating encoded training features.
What is the difference between train_country and valid_country?
train_countryis generated using OOF encoding, whilevalid_countryis generated using the final mapping learned from the full training data.
Why reshape(-1, 1)?
To convert the country encoding into a two-dimensional column compatible with the other feature matrices.
What does np.hstack() do?
It horizontally concatenates feature matrices.
Why combine SBERT and structured features?
SBERT captures semantic information from text, while structured features provide explicit numerical and categorical information. Combining them creates a hybrid representation.
The Most Important Concept in This Cell
Remember this distinction:
Training
↓
Fit / Learn
↓
Transform training using leakage-aware methods
versus:
Validation
↓
DO NOT FIT
↓
Only TRANSFORM using training-learned parameters
That is the data-leakage boundary your examiner is most likely to test.
For this feature-builder cell
- Trace one row through all five folds
Top comments (0)