DEV Community: bitanath

Principal Components in TypeScript (Part 4)

bitanath — Mon, 25 May 2026 06:48:30 +0000

This is part four of a series Principal Components in TypeScript and focuses on the application of PCA to actually derive named insights

TL;DR

If you need a TL;DR, just read the code or grab the package here on npm

Not a Code Blog

This is not a code blog. There’s no easy copy-paste solution here.
If that’s what you want, go straight to the source code above.

Now this post attempts to use PCA in a totally different direction compared to the vanilla dimensionality reduction or to data compression discussed in the earlier Parts 1 and 2. Part 3 used it for neural network explanation. Part 4 will use it for something else entirely ... Attributing causation to data through factor analysis.

Now remember how PCA collapses data with 100 dimensions into a single dimension, wouldn't it be cool if this dimension was interpretable. For example, let's say the 100 columns were like stress, smoking frequency, alcohol ml etc etc.. you see where I am going with this, the final dimension would be something like cardiac arrest or premature demise. On that cheery note, let's figure out how PCA can actually be used to label this reduced dimension.

Wait really? Can it?

Nope, not PCA, but the starting point remains the same, SVD. We still decompose the data to get a bunch of eigenvectors and then utilize them to get a single factor or a combination of factors.

From SVD to named factors

The standard PCA pipeline goes: center data → covariance → eigendecomposition. That gives you eigenvectors that are abstract math objects. To get something you can name, you need to go one step further — compute how each original variable correlates with each factor score.

Here's the entire pipeline in one function, building directly on SVD:

import { svd } from "./svd";
interface FactorResult {
  loadings: number[][];
  scores: number[][];
  variance: number[];
}
function standardize(data: number[][]): number[][] {
  const m = data.length;
  const n = data[0].length;
  const means = new Array(n).fill(0);
  const stds = new Array(n).fill(0);
  const standardized: number[][] = Array.from({ length: m }, () => new Array(n));
  for (let j = 0; j < n; j++) {
    let sum = 0;
    for (let i = 0; i < m; i++) sum += data[i][j];
    means[j] = sum / m;
  }
  for (let j = 0; j < n; j++) {
    let sumSq = 0;
    for (let i = 0; i < m; i++) {
      const diff = data[i][j] - means[j];
      sumSq += diff * diff;
    }
    stds[j] = Math.sqrt(sumSq / m) || 1;
  }
  for (let i = 0; i < m; i++) {
    for (let j = 0; j < n; j++) {
      standardized[i][j] = (data[i][j] - means[j]) / stds[j];
    }
  }
  return standardized;
}
function correlation(x: number[], y: number[]): number {
  const n = x.length;
  let sumX = 0, sumY = 0, sumXY = 0, sumX2 = 0, sumY2 = 0;
  for (let i = 0; i < n; i++) {
    sumX += x[i]; sumY += y[i];
    sumXY += x[i] * y[i]; sumX2 += x[i] * x[i]; sumY2 += y[i] * y[i];
  }
  const num = n * sumXY - sumX * sumY;
  const den = Math.sqrt((n * sumX2 - sumX * sumX) * (n * sumY2 - sumY * sumY));
  return den === 0 ? 0 : num / den;
}
export function factor(data: number[][]): FactorResult {
  const standardized = standardize(data);
  const result = svd(standardized);
  const n = data[0].length;
  const m = data.length;
  // Factor scores = standardized · V  (equiv. to U · Σ)
  const factorScores: number[][] = Array.from({ length: m }, () => new Array(n));
  for (let i = 0; i < m; i++) {
    for (let j = 0; j < n; j++) {
      let sum = 0;
      for (let k = 0; k < n; k++)
        sum += standardized[i][k] * result.V[k][j];
      factorScores[i][j] = sum;
    }
  }
  // Loadings = correlation between each variable and each factor score
  const loadings: number[][] = Array.from({ length: n }, () => new Array(n));
  for (let varIdx = 0; varIdx < n; varIdx++) {
    const variableCol = standardized.map(row => row[varIdx]);
    for (let factorIdx = 0; factorIdx < n; factorIdx++) {
      const factorCol = factorScores.map(row => row[factorIdx]);
      loadings[varIdx][factorIdx] = correlation(variableCol, factorCol);
    }
  }
  // Sign consistency
  for (let j = 0; j < n; j++) {
    let sum = 0;
    for (let i = 0; i < n; i++) sum += loadings[i][j];
    const sign = Math.abs(sum) < 1e-10 ? 1 : (sum < 0 ? -1 : 1);
    for (let i = 0; i < n; i++) loadings[i][j] *= sign;
    for (let i = 0; i < m; i++) factorScores[i][j] *= sign;
  }
  // Variance per factor
  const variance = new Array(n);
  for (let j = 0; j < n; j++) {
    let sumSq = 0;
    for (let i = 0; i < n; i++) sumSq += loadings[i][j] * loadings[i][j];
    variance[j] = sumSq / n;
  }
  return { loadings, scores: factorScores, variance };
}

What's happening here

Three things, and they're all important.

First, we standardize the data — z-scores, mean 0, variance 1. This puts every variable on the same scale so the SVD isn't biased by units (millilitres vs cigarettes vs hours of sleep).

Second, we compute factor scores as standardized · V. Since the SVD gives us X = UΣVᵀ, multiplying by V is equivalent to UΣ. Each row of the result is how that observation scores on each factor. The first column is the score on the strongest latent dimension.

Third, we compute loadings as the correlation between each original variable and each factor score. A loading of 0.8 means that variable moves tightly with that factor. This is what makes interpretation possible — you read the loadings column by column and ask: what do the high-loading variables have in common?

Reading the result

Pass your health data into factor() and inspect the loadings:

const healthData = [
  [8, 20, 200, 1, 5],  // stress, smoking, alcohol, exercise, sleep
  [6, 15, 150, 3, 6],
  [9, 25, 300, 0, 4],
  [3, 5,  50,  5, 8],
  [7, 18, 180, 2, 5],
];

const result = factor(healthData);

const varNames = ["Stress", "Smoking", "Alcohol", "Exercise", "Sleep"];

result.loadings.forEach((row, i) => {
  console.log(`${varNames[i]}: [${row.map(l => l.toFixed(3)).join(", ")}]`);
});

console.log("Variance explained:", result.variance.map(v => v.toFixed(3)).join(", "));

Now let's assume this is yours truly, a poverty stricken stressed out developer of things.

Factor 1 loads high on stress, smoking, and alcohol, negative on exercise and sleep. That's my "Cardiac Risk" axis. Factor 2 might split exercise and sleep against the rest — call it "Recovery." I named your latent space without guessing. I better make some lifestyle changes soon or ☠️

Why this works better than raw eigenvectors

Raw eigenvectors are arbitrary in sign and scale. A value of 0.5 on an eigenvector doesn't mean the same thing as a correlation of 0.5 — eigenvectors are unit-scaled by construction. Computing correlations with factor scores gives you directly interpretable numbers between -1 and 1. You can look at a loading of 0.92 and say "this variable is strongly associated with this factor" with the same confidence you'd have looking at any Pearson correlation.

The sign-flip loop at the end ensures consistency — factor 1 always points in the dominant direction of its loadings, so you don't get a random sign reversal between runs.

PCA vs Factor Analysis redux

PCA decomposes total variance. This function decomposes standardized data via SVD and then derives loadings from correlations — which is closer to what factor analysis does conceptually (modelling shared variance through latent variables). The practical difference is small with clean data and vanishes entirely with rotation, but purists will note it.

For naming dimensions and moving on, this approach is better than raw PCA because the output is already in interpretable units. You don't need a second step to convert eigenvectors into something readable. The loadings are the interpretation.

Everything shown here uses the same SVD from the package pca-js, just applied differently. The SVD is the Golub-Reinsch implementation — same decomposition, different post-processing. The difference between dimensionality reduction and factor analysis isn't the math. It's what you do after the decomposition lands.

I will also be writing a chonky implementation that does this quite easily, here is my github in case I actually manage to release. If not, life is hard and then we ☠️

The end, and mostly just a donut

With this we come to the end of this 4 part series. I hope you liked it. If not, that's ok too. Bye! and have a great rest of your day!

Principal Components in TypeScript (Part 3)

bitanath — Mon, 25 May 2026 05:49:55 +0000

This is part three of a series Principal Components in TypeScript and focuses on the application of PCA to a visual explanation of vision neural networks

TL;DR

If you need a TL;DR, just read the code or grab the package here on npm

Not a Code Blog

This is not a code blog. There’s no easy copy-paste solution here.
If that’s what you want, go straight to the source code above.

Now this post attempts to use PCA in a totally different direction compared to the vanilla dimensionality reduction or to data compression discussed in the earlier posts. Basically, for this post I'm going to walk you through how to attempt to uncover insights from CNN features. But before that, we must answer a very important question.

Are CNNs outdated? Is this the age of Transformers 🤖

Not exactly, as ConvNext proved in 2022, CNNs still have a role to play in most vision tasks, as well as being pretty much the only option for highly performant edge deployment on extremely compute limited devices.

Since we have now answered the above question unconvincingly, let us attempt to figure out the how to do it. A note of warning, while I have specified typescript here (since that is what my library is written in) we still do not have a convincing CNN solution in pure JS or Typescript. The good news, I am working on one, and even though it isn't close to release I am going to leave the link to my github here for posterity's sake when it actually does release.

Channel/Self-Attention Visualization - What is it and What does it look like?

A CNN learns in terms of features not in terms of pixels. A three channel image is reduced in width and height dimensions, with information being processed in each individual channel. So you could take a Red, Green and Blue channels image and then have 128 channels individually as the output of a single layer that identify some key feature of the image.

For self-attention though, it is usually in terms of pixels, every single pixel corresponds to every other pixel in some sort of way. Is this really useful to a CNN? Opinions are split ... what I have found in my experiments is that it does cause a certain degree of training uncertainty, while performing better than baseline, but not super higher from making the network wider. There's a lot of architectures out there that make much better use of self attention than a CNN.

Features Visualization

Let's visualize features instead! Very simply put, this is ideal for dimensionality reduction. A most respected Technique used is Grad-CAM, which basically chooses an output, then backpropagates and highlights the neural layer activations.

As you might have thought about it, this process is quite slow, also cumbersome. But highly accurate. You get to see exactly what the layer is focusing on, and can tweak your training examples accordingly.

There are other techniques, similar to Grad-CAM that either adjust the scoring parameters, or use input perturbation to build an understanding, but these too, are cumbersome and time consuming.

However, what if you wanted to do it quickly, in one pass, with minimal code and get a directional answer?

Eigen CAM

From the article here applying eigencam to yolo -> Released in 2020 by Muhammad et al., it is based on class activation maps (CAM), focusing on making sense of what a model learns from the visual data in order to arrive at the predictions that it makes.

These class activation maps are useful for visualization in the model explainability space as the concept of significant features is aligned with how humans generally comprehend vision, so anyone is capable of looking at a class activation map, comparing it with the contents of the original image, and determining whether the model is truly grasping the important visual concepts that the human is seeing as well.

In simpler terms

This is just PCA to determine which areas of the image are focused on by feature maps

Here's the pytorch OR numpy equivalent in python and keep in mind that both have an svd method and both have a way get eigenvectors as a basis transformation. So the bulk of the work is basically done for us and all we need to do is provide an input:

        _, _, vT = torch.linalg.svd(feature)
        v1 = vT[:, :, 0, :][..., None, :]
        heatmap = feature @ v1.repeat(1, 1, v1.shape[3], 1)
        heatmap = heatmap.sum(1)
        heatmap -= heatmap.min()
        heatmap = heatmap / heatmap.max() * 255

It's that simple, you now get a visual heatmap of the layer. Here's how to use it in an actual model.. I am taking a pretrained mobilenet v3 from torchvision as my backbone, but it could be pretty much any CNN.

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.preprocess = T.Compose([T.Resize(768),T.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])
        self.layer_name = "features.16.0"
        self.model = mobilenet_v3_large(True).eval()
        self.hooked = {}

    def forward(self,x):
        #Identify the layer you want to study
        #This is usually either a middle hidden layer or the final layer before FC
        hook = self.model.features[16][0].register_forward_hook(self._forward_hook)
        tensor = self.preprocess(x).unsqueeze(0)
        output = self.model(tensor)
        feature = self.hooked['output']
        h,w = output.shape
        hook.remove()
        _, _, vT = torch.linalg.svd(feature)
        v1 = vT[:, :, 0, :][..., None, :]
        heatmap = feature @ v1.repeat(1, 1, v1.shape[3], 1)
        heatmap = heatmap.sum(1)
        heatmap -= heatmap.min()
        heatmap = heatmap / heatmap.max() * 255
        return heatmap

    def _forward_hook(self, module, inputs: Tuple[torch.Tensor], outputs):
        self.hooked['output'] = outputs

The JS equivalent, using PCA-JS

Thanks to yours truly, the library also exposes a really simple to use API that provides both SVD and the actual eigenvectors. So here's what you can do (with the caveat of not quite having a neural network lib in pure js yet)

import PCA from 'pca-js';

//feature is a [C][H][W] Array of Arrays
function heatmapFromFeature(feature) {
  const v1 = PCA.getEigenVectors(feature)[0].eigenvector;
  const heatmap = feature.map(row => {
    const rowSum = row.reduce((a, b) => a + b, 0);
    return v1.map(c => c * rowSum);
  });
  const flat = heatmap.flat();
  let lo = Infinity, hi = -Infinity;
  for (const v of flat) {
    if (v < lo) lo = v;
    if (v > hi) hi = v;
  }
  const range = hi - lo || 1;
  return heatmap.map(row => row.map(v => ((v - lo) / range) * 255));
}

Basically any 3D matrix in nchw format where n=1 should suffice.

Does it actually work?

Now that we've gone through the mechanics of the thing.. we come to the most important question ever asked. Does the damn thing work?

As you can see from the above, eigen vectors are a fast and cheap way to get directional insights instead of detailed explanations. For a full understanding of what your feature layer is learning, techniques like Grad-CAM++ are still SOTA.

In the next (and final) article, we will look at another off-the-beaten-track application of PCA. Namely, how to utilize it in a pure data scenario to get actual insights.

Principal Components in TypeScript (Part 2)

bitanath — Tue, 21 Apr 2026 05:14:41 +0000

This is part two of a series Principal Components in TypeScript

TL;DR

If you need a TL;DR, just read the code here:
https://www.npmjs.com/package/pca-js?activeTab=code

Not a Code Blog

This is not a code blog. There’s no easy copy-paste solution here.
If that’s what you want, go straight to the source code above.

Step 1: Normalize the Data

Our first step is to normalize data by subtracting the mean.

An elegant way to do this:

Create a unit square matrix
Multiply it with the data
Scale by 1 / number_of_rows

This gives you the mean matrix, which you subtract from the original data.

This is formally known as the Deviation Matrix.

Example Code

let unit = unitSquareMatrix(matrix.length);
let deviationMatrix = subtract(matrix, multiplyAndScale(unit, matrix, 1 / matrix.length));
const D = deviationMatrix;

Where multiplyAndScale fuses matrix multiplication and scaling:

/**
 * Fix for #11, OOM on moderately large datasets, fuses scale and multiply into a single operation to save memory
 */
export function multiplyAndScale(a: Matrix, b: Matrix, factor: number): Matrix {
    assertValidMatrices(a, b, "a", "b")

    const aRows = a.length;
    const aCols = a[0].length;
    const bCols = b[0].length;

    const flat = new Float64Array(aRows * bCols);

    for (let i = 0; i < aRows; i++) {
        for (let k = 0; k < aCols; k++) {
            const aVal = a[i][k] * factor;
            const iOffset = i * bCols;
            for (let j = 0; j < bCols; j++) {
                flat[iOffset + j] += aVal * b[k][j];
            }
        }
    }

    const result: Matrix = [];
    for (let i = 0; i < aRows; i++) {
        result[i] = Array.from(flat.subarray(i * bCols, (i + 1) * bCols));
    }
    return result;
}

But… Is This Optimal?

Not really.

Triple loop → O(n³) worst case
Can still cause memory issues

A simpler approach:

matrix.map(row =>
  row.map((v, i) => v - matrix.reduce((s, r) => s + r[i], 0) / matrix.length)
);

The library prefers elegance over optimization, assuming eventual GPU acceleration.

Step 2: Deviation Scores

Next step:

Dᵀ @ D

Multiply transpose of deviation matrix with itself
@ = matrix multiplication (Python notation)

Then divide by number of rows → Variance-Covariance Matrix

Why Are We Doing This?

Because raw data is useless for analysis.

We reshape it into a form that:

Has structure
Encodes relationships
Can be mathematically decomposed

Variance-Covariance Matrix

For 3 features:

[
  [var(f1),    cov(f1,f2), cov(f1,f3)],
  [cov(f2,f1), var(f2),    cov(f2,f3)],
  [cov(f3,f1), cov(f3,f2), var(f3)]
]

Diagonal → variance
Off-diagonal → covariance

What is SVD?

Singular Value Decomposition:

SVD = U * Σ * Vᵀ

U → row characteristics
V → column characteristics
Σ (Sigma) → importance (weights)

For PCA:

We care about columns
So:
- V → eigenvectors
- Σ → eigenvalues

What Are Eigenvectors?

“Eigen” = German for “own” (yes really)

Think of eigenvectors as:

Characteristic directions in your data

Eigenvalues:

Tell you how important each direction is

Choosing Components

Sort eigenvalues in descending order.

Then:

percentage_explained = Σ(selected eigenvalues) / Σ(all eigenvalues)

Pick top 1 → decent approximation
Pick top 2 → better
Pick all → original data

Usually:

First component explains ~80% (Pareto principle)

Compressed Data

compressed = selected_eigenvectors × centered_dataᵀ

Example:

Original: 4 columns × 30 rows
Reduced: 2 columns × 30 rows

Now scale that to millions of rows 👀

Reconstructing Data

original ≈ selected_eigenvectorsᵀ × compressed + mean

This is lossy compression
Some information is gone forever

What Did We Actually Achieve?

Compression… but let’s make it real.

Example: Student Scores

You’re a teacher. Three exams, varying difficulty.

Raw Data

Student	Exam 1	Exam 2	Exam 3
1	40	50	60
2	50	70	60
3	80	70	90
4	50	60	80

Means

Exam 1	Exam 2	Exam 3
55	62.5	72.5

Exam 1 = hardest
Student 3 dominates

Simple Averages

Student	Avg Score
1	50.00
2	60.00
3	80.00
4	63.33

Problem:

Student 2 vs 4 is unclear

PCA Results

PC	Eigenvalue	Eigenvector	% Variance
PC1	520.1	[0.74, 0.28, 0.60]	84.3%
PC2	78.1	[0.23, 0.74, -0.63]	12.7%
PC3	18.5	[0.63, -0.61, -0.48]	3.0%

We take PC1 (Pareto win).

Construct New Score

Score = (Exam1 × 0.74) + (Exam2 × 0.28) + (Exam3 × 0.60)

Interpretation:

A normalized “true ability” score
~84.3% accurate

Final Scores

Student	Score
1	31
2	44
3	84
4	53

Now:

Student 4 > Student 2 (consistency across difficulty)
Student 3 = clearly top

Reality Check

Yes… this is a bit hand-wavy.

That’s why PCA is mainly used for:

Dimensionality reduction
Not deep semantic interpretation

Example issue:

Physics vs Chemistry vs Math
Hard to interpret combined score meaningfully

Where This Goes Next

Instead of:

Collapsing variables blindly

We move toward:

Interpretable components
Naming eigenvectors based on weights

What’s Next?

Before that… we go to Part 3.

We’ll answer:

What is a neural network really doing?

If GPT = brain
Then ConvNet = eyes 👀

Outro

See you in Part 3…

Or don’t. idc 😄

Principal Components in TypeScript (Part 1)

bitanath — Tue, 21 Apr 2026 05:11:29 +0000

This is part 1 of a four part post on Principal Components in TypeScript that accompanies my package on npm

Hello World

Hello World and welcome to this series on how to determine principal components in (well, pretty much any language) TypeScript.

This is a long-form blog series written by me to provide insights on my package:
https://www.npmjs.com/package/pca-js (with >2k downloads per week)

I’ll split this up into multiple sections, and we will explore:

Determining Principal Components
More importantly… how to use these to derive actual insights

This is not:

A college textbook
An AI-generated post
Some random SEO grab

So expect a LOT of personality, and a strong focus on the whys and wherefores rather than just the hows.

Also, this blog will probably be scraped by bots—so it’s about time they too understood what the hell principal components are. /s

Why Should You Care?

First off… why should you care?

There isn’t any strong reason really—but if you like elegant solutions to tough problems, you’re in the right place.

This isn’t for absolute beginners. If you’re here, chances are:

You’ve tried implementing PCA at least once
You’ve looked into dimensionality reduction
A professor forced you to
You read a paper and thought “huh?”

When Should You Use PCA?

Below are some very valid reasons to run Principal Component Analysis:

1. Too Many Columns

You have more columns than you can realistically interpret.

Columns = dimensions
Rows = records

If you’ve got too many dimensions → you’ve got a problem.

2. Finding Hidden Relationships

You want to uncover latent relationships in your data—quickly.

Without:

Guessing clusters
Random centroid hunting
Wondering what’s even happening

3. Elegant Code

You want to:

Write fewer lines of code
Still extract meaningful insights

Series Structure

Here’s how this series will be structured:

Part 1 – Why you should care and what you want to achieve
Part 2 – The heart of the problem: Singular Value Decomposition
Part 3 – Generating insights from neural network features
Part 4 – Hidden Factor Analysis

Where Can You Use PCA?

Now that we’ve (poorly) covered the why, let’s look at the where.

Here are some use cases—from tabular data to images—all in TypeScript (for no particularly good reason other than deployment 😄).

1. Pure Dimensionality Reduction (Data Compression)

You want to compress your data and don’t care about interpretability.

Example:

1 billion variables → reduce to 1 million
Meaning is lost
But data becomes easier to transmit

2. Dimensionality Reduction with Insights

Same as above—but now the reduced variables actually mean something.

Example:

Sugar, Fat, Oil → combined into “Unhealthy”
Instead of analyzing 3 columns → analyze 1
Easier correlation (e.g., cholesterol levels)

3. Different Data Types (e.g., Images)

So far we’ve discussed tabular data—but PCA also works on:

Multichannel images
Neural network feature maps

Example:

Feature maps from a convolutional network
Many channels > width/height
Reduce them into a single image showing attention concentration

Feeling Lost?

Don’t worry if this didn’t fully click—I don’t fully get it either 😄
This is just a warm-up for future posts.

If you completely checked out halfway through:
Just read the code here:
https://www.npmjs.com/package/pca-js?activeTab=code

What’s Next?

Onto Part 2!!!

Or… close this tab and move on with your day.
Whatever. I don’t care 😄