1. Expansion of the Covariance Matrix (Pixel-wise Intuition)
For a dataset of ( N ) images, each flattened into a vector of ( D ) pixels, the centered data matrix ( X ) (size ( N \times D )) is:
[
X = \begin{bmatrix}
x_{11} - \mu_1 & x_{12} - \mu_2 & \cdots & x_{1D} - \mu_D \
x_{21} - \mu_1 & x_{22} - \mu_2 & \cdots & x_{2D} - \mu_D \
\vdots & \vdots & \ddots & \vdots \
x_{N1} - \mu_1 & x_{N2} - \mu_2 & \cdots & x_{ND} - \mu_D
\end{bmatrix},
]
where ( \mu_j = \frac{1}{N} \sum_{i=1}^N x_{ij} ) is the mean of pixel ( j ).
The covariance matrix ( C ) (size ( D \times D )) is computed as:
[
C = \frac{1}{N-1} X^T X.
]
Expanding ( X^T X ) pixel-wise:
[
X^T X = \begin{bmatrix}
\sum_{i=1}^N (x_{i1} - \mu_1)^2 & \sum_{i=1}^N (x_{i1} - \mu_1)(x_{i2} - \mu_2) & \cdots & \sum_{i=1}^N (x_{i1} - \mu_1)(x_{iD} - \mu_D) \
\sum_{i=1}^N (x_{i2} - \mu_2)(x_{i1} - \mu_1) & \sum_{i=1}^N (x_{i2} - \mu_2)^2 & \cdots & \sum_{i=1}^N (x_{i2} - \mu_2)(x_{iD} - \mu_D) \
\vdots & \vdots & \ddots & \vdots \
\sum_{i=1}^N (x_{iD} - \mu_D)(x_{i1} - \mu_1) & \sum_{i=1}^N (x_{iD} - \mu_D)(x_{i2} - \mu_2) & \cdots & \sum_{i=1}^N (x_{iD} - \mu_D)^2
\end{bmatrix}.
]
Interpretation:
- Diagonal entries ( C_{jj} ): Variance of pixel ( j ) across all images.
- Off-diagonal entries ( C_{jk} ): Covariance between pixels ( j ) and ( k ). High values indicate pixels ( j ) and ( k ) vary together (e.g., edges or textures).
Top comments (0)