Shapley Addictive Explanation

A practical, beginner-friendly guide to SHapley Additive exPlanations with a simple example and Python implementation

Imagine applying for a loan and getting rejected by a machine learning model. You ask, “Why was my application rejected?” and someone simply tells you, “The algorithm decided.”

That answer is not very useful.

Modern machine learning models such as gradient boosting, random forests, and neural networks can make highly accurate predictions, but understanding why they made those predictions can be difficult.

This is where SHAP, short for SHapley Additive exPlanations, becomes useful. SHAP explains a prediction by showing how much each feature contributed to moving the prediction up or down.

In this article, we will understand the basic idea behind SHAP, work through a small example by hand, and then use the Python shap library to explain a real machine learning model.

What Is SHAP?

SHAP is based on Shapley values, a concept from cooperative game theory introduced by mathematician Lloyd Shapley in 1953.

The basic idea is simple: when several people work together to earn a reward, how should the reward be divided fairly among them?

Imagine three friends working together on a project. Each person may contribute differently, and some contributions may become more valuable when combined with another person's contribution.

Shapley's approach is to consider the different possible orders in which the participants could join the project. For each order, we calculate how much additional value a participant brings when they join. The average of those contributions becomes that participant's fair share.

Now replace:

Friends → Features
Project reward → Model prediction
Contribution → Feature's effect on the prediction

That is the basic idea behind SHAP.

SHAP asks:

How much did each feature contribute to this particular prediction?

A Simple SHAP Example

Let's consider a fictional loan prediction model with three features:

Income
Debt
Credit history

Suppose our simplified model starts with a score of 0 when it knows nothing about the applicant.

The model behaves like this:

High income adds 30 points
Good credit history adds 20 points
Debt subtracts 10 points
Income and credit history together add an additional 10-point bonus

Therefore, when all three features are known:

30 + 20 − 10 + 10 = 50

The final score is 50.

But how much of that score should be attributed to each feature?

The interaction between income and credit history makes this less obvious

Looking at Every Possible Order

There are six possible orders in which the three features can be added.

Order	Income	Debt	History
Income → Debt → History	+30	−10	+30
Income → History → Debt	+30	−10	+30
Debt → Income → History	+30	−10	+30
Debt → History → Income	+40	−10	+20
History → Income → Debt	+40	−10	+20
History → Debt → Income	+40	−10	+20
Average	+35	−10	+25

The final SHAP contributions are therefore:

Income: +35
Debt: −10
Credit history: +25

Adding them together:

35 − 10 + 25 = 50

So the contributions exactly reconstruct the model's prediction.

The extra 10-point interaction bonus is effectively shared between income and credit history.

This is the basic idea of Shapley fairness.

The SHAP Formula

In general, the contribution of feature i can be written as:

text
φᵢ = Σ [ |S|! (n − |S| − 1)! / n! ] × [f(S ∪ {i}) − f(S)]

You do not need to memorize this formula to use SHAP.

In simple terms, it calculates the feature's average marginal contribution across different subsets of the other features, with appropriate weights so that the possible feature orderings are treated fairly.

What Does "Additive" Mean in SHAP?

The A in SHAP stands for Additive.

One of the most useful properties of SHAP is that the individual feature contributions add up to the model's output.

In simplified form:

text
Prediction =
Base value

• SHAP(feature 1)
• SHAP(feature 2)
• ...
• SHAP(feature n)

Thebase value represents the model's expected output before considering the specific features of the case being explained.

Each SHAP value then moves the prediction higher or lower.

In our loan example:

text
Base value = 0

Income = +35
Debt = −10
Credit history = +25

Final prediction = 50

Why Are SHAP Values Useful?

SHAP is popular because its approach is based on several desirable properties.

1. Local Accuracy

The feature contributions add up to the model's output.

1. Missingness

A feature that is missing does not receive a contribution.

1. Consistency

If a feature becomes more important in the model, its SHAP contribution should not decrease under the SHAP framework.

These properties make SHAP particularly useful when we need explanations that are mathematically grounded.

SHAP in Practice

Calculating exact Shapley values can become computationally expensive because the number of possible feature combinations grows rapidly as the number of features increases.

For example, with 30 features, there are already more than a billion possible subsets.

The Python shap library therefore provides specialized explainers.

Explainer	Common Use	Speed
TreeExplainer	Random forests, XGBoost, LightGBM, gradient boosting	Fast
LinearExplainer	Linear and logistic regression	Very fast
DeepExplainer	Deep neural networks	Fast/approximate
KernelExplainer	General black-box models	Slower

For tree-based models, TreeExplainer is usually the most convenient choice.

Hands-On: Explaining a Real Model with SHAP

Let's train a gradient boosting classifier using scikit-learn's built-in breast cancer dataset.

This dataset is being used only as a teaching example. It should not be used to make real medical decisions.

First, install the required libraries:

bash
pip install shap scikit-learn matplotlib

Then use the following code:

python
import shap
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split

data = load_breast_cancer(as_frame=True)
X, y = data.data, data.target

X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)

model = GradientBoostingClassifier(random_state=42)
model.fit(X_train, y_train)

print("Test accuracy:", round(model.score(X_test, y_test), 3))

explainer = shap.TreeExplainer(model)
shap_values = explainer(X_test)

i = 0
print("Base value:", round(float(shap_values.base_values[i]), 3))

top = sorted(
zip(X_test.columns, shap_values.values[i]),
key=lambda x: abs(x[1]),
reverse=True
)[:5]

for name, value in top:
print(f"{name:25s} {value:+.3f}")

You can also verify SHAP's additive property:

python
total = shap_values.base_values[i] + shap_values.values[i].sum()

print("Base + SHAP values:", round(float(total), 3))

raw = model.decision_function(X_test.iloc[[i]])[0]

print("Model raw output:", round(float(raw), 3))

Example Output

One example run produces:

text
Test accuracy: 0.956

mean concave points +1.140
worst concave points +1.119
worst perimeter +0.820
worst radius +0.582
worst area +0.566

Base + sum of SHAP : 7.055
Model raw output : 7.055

The important part is the final two values:

text
Base + sum of SHAP : 7.055
Model raw output : 7.055

They match exactly, demonstrating SHAP's additive property.

Understanding SHAP Units

One important detail is that SHAP values do not always represent probabilities.

For this classifier, the values are expressed in log-odds.

The raw model output of approximately 7.055 log-odds corresponds to a probability of roughly 99.9% for the benign class.

Positive SHAP values push the prediction toward the benign class, while negative values push it toward the malignant class.

Reading SHAP Visualizations

SHAP provides several useful plots.

1. Waterfall Plot

A waterfall plot explains one prediction.

It starts with the base value and shows how individual features push the prediction higher or lower until the final model output is reached.

python
shap.plots.waterfall(shap_values[0], max_display=8)

This is useful when you want to answer:

Why did the model make this particular prediction?

1. SHAP Bar Plot

A bar plot shows which features have the largest average absolute SHAP values across the dataset.

python
shap.plots.bar(shap_values, max_display=8)

Longer bars indicate features with larger average effects on predictions.

This is useful when you want to understand:

Which features does my model rely on most overall?

1. SHAP Beeswarm Plot

A beeswarm plot provides more detail by showing both the importance and direction of feature effects.

python
shap.plots.beeswarm(shap_values, max_display=8)

It is particularly useful for seeing how feature values influence predictions across many observations.

Things to Be Careful About

SHAP is powerful, but it should not be interpreted as a perfect explanation of reality.

Correlated Features

Correlated features can share credit.

For example, worst radius, worst perimeter, and worst area measure related characteristics. SHAP may distribute their contribution among them.

SHAP Explains the Model, Not Reality

If the model learned an unusual pattern from the training data, SHAP will explain that pattern faithfully.

It does not prove that a feature causes an outcome.

Computational Cost

Model-agnostic explainers such as KernelExplainer can become slow on large datasets.

Background Data Matters

The base value depends on the reference/background data used by the explainer. Therefore, that data should reasonably represent the population you want to study.

Don't Overinterpret Tiny Values

A very small SHAP value may not be practically important. Focus on features showing clearer effects.

Final Thoughts

SHAP gives machine learning practitioners a practical way to look inside model predictions.

Instead of simply saying:

"The model predicted this."

you can ask:

"Which features pushed the prediction in this direction, and by how much?"

The key idea is simple:

text
Model prediction =
Base value + contribution of each feature

For individual predictions, waterfall plots are especially useful. For understanding overall feature importance, bar plots and beeswarm plots can provide a broader view.

If you are working with a tree-based model, trying SHAP can be a great first step toward making your machine learning model easier to understand.

Have you used SHAP on a real project? What did it reveal about your model? Share your experience in the comments.