Linear Algebra in AI — Vectors, Matrices & Why Tensors Are Just Fancy Spreadsheets

By Shakti Tiwari — Nifty Option Trader, Research Analyst & XGBoost Expert. I build ML models for Nifty options on ordinary hardware, and I write these notes so an Indian retail trader can understand AI math without a PhD.

The first article in this series covered calculus — the steering wheel of AI. This one covers linear algebra — the language AI speaks. If calculus is how the model learns, linear algebra is what the model reads, writes, and transforms at every single step.

Every number in a model — a stock price, a pixel, a word, a prediction — is stored as a vector or a matrix. Every operation — multiply, add, rotate, project — is a linear-algebra move. When people say a GPU is "good at AI," what they mean is: a GPU is extremely good at multiplying giant matrices very fast. That is linear algebra, accelerated.

This is article 2 of the AI Components series. The index and article 1 (Calculus) live on optiontradingwithai.in.

Direct Answer: What Is Linear Algebra in AI?

Linear algebra is the branch of mathematics dealing with vectors, matrices, and the operations on them (addition, multiplication, transformation). In AI it is the data format and the computation engine: inputs become vectors, layers are matrices, and "thinking" is a chain of matrix multiplications. A neural network is, under the hood, a stack of matrix products plus a non-linear squish. Even XGBoost, my daily tool for Nifty, represents features and splits in vector form.

So: calculus decides the weights; linear algebra carries the data through them.

Why a Trader Should Care

If you have ever wondered why a model can ingest 50 Nifty option-chain features at once, or why "dimensionality reduction" helps, or why your phone can run a quantized model — that is all linear algebra. Understanding it lets you:

• Know why more features ≠ better (curse of dimensionality).
• Understand correlation vs independence between your trading signals.
• See why PCA / embeddings compress data without losing the signal.
• Debug why a model "blows up" (matrix values exploding to NaN).

For a retail trader building models on a ₹40,000 laptop, linear algebra is the difference between a model that runs in seconds and one that crashes your RAM.

The Core Objects

1. The Vector — A List With Direction

A vector is an ordered list of numbers: v = [0.3, -1.2, 5.0, ...]. In AI it represents one data point — one day's Nifty features, one word's embedding, one image's flattened pixels.

Geometrically a vector has magnitude (length) and direction (which way it points in space). Two trading days that look similar have vectors pointing in similar directions. This is how models measure "closeness" — via the angle between vectors.

2. The Matrix — A Grid of Numbers

A matrix is a 2D grid: rows × columns. In AI a matrix usually means a layer's weights or a batch of inputs. Multiplying an input vector by a weight matrix produces the next layer's values.

output = input_vector × weight_matrix + bias

That single line is 90% of what a neural network does, repeated across layers. No mystery — just matrix multiplication.

3. The Tensor — A Fancy Spreadsheet

A tensor is the general term: a vector is a 1-D tensor, a matrix is 2-D, an image (height × width × color) is 3-D, a batch of images is 4-D. Frameworks like PyTorch/TensorFlow call everything a "tensor" because they handle any number of dimensions uniformly.

So when someone says "the model processes a tensor," translate it as: the model processes a multi-dimensional grid of numbers. Your Excel sheet with rows and columns is a 2-D tensor. Tensors are just spreadsheets with more axes.

The Operations That Matter

Dot Product — Measuring Similarity

The dot product of two vectors a · b = a1*b1 + a2*b2 + ... gives a single number. If both point the same way, the dot product is large and positive; if opposite, large negative; if perpendicular, near zero.

In AI: dot products measure how much one signal matches another. Attention in Transformers is literally a dot product between a "query" vector and "key" vectors — how relevant is this word to that word? For a trader, it is the same idea as correlation between two assets, expressed as geometry.

Matrix Multiplication — The Engine

Multiplying matrix A (m×k) by matrix B (k×n) gives matrix C (m×n). The GPU does this millions of times per second. Every layer of every modern model is this operation. The speed of AI is mostly the speed of matrix multiplication.

Transpose, Inverse, Norm

• Transpose (flip rows/columns) — reorganizing data.
• Inverse — undoing a transformation (rarely used directly in deep learning because it is unstable; gradients are preferred).
• Norm (length, e.g. L2 = √(Σx²)) — used to normalize vectors so one huge feature does not dominate. I always normalize Nifty features before training, or the premium column (in lakhs) drowns the OI column (in thousands).

Worked Example: One Layer, by Hand

Suppose a tiny model scores Nifty direction from 2 features: [OI_change, premium_change] = [0.8, -0.5]. The layer has weights [[0.6, 0.4], [-0.3, 0.9]] and bias [0.1, 0.0].

layer_output = [0.8, -0.5] × [[0.6, 0.4], [-0.3, 0.9]] + [0.1, 0.0]
             = [0.8*0.6 + (-0.5)*(-0.3),  0.8*0.4 + (-0.5)*0.9] + [0.1, 0.0]
             = [0.48 + 0.15,  0.32 - 0.45] + [0.1, 0.0]
             = [0.63 + 0.1,  -0.13 + 0.0]
             = [0.73, -0.13]

Then an activation function squishes those into a probability. That is one layer. Stack 50 of them and you have a deep network. Every step is matrix math from this section.

Linear Algebra in XGBoost (My Daily Tool)

XGBoost is tree-based, not matrix-multiplication-based like a neural net — but linear algebra is still everywhere:

• Each feature column is a vector; splits are chosen by scanning these vectors.
• Gradient and Hessian (from article 1) are vectors over all training rows.
• Predictions accumulate as a sum of vectors (one vector per tree's contribution).
• When I one-hot encode sectors or expiry weeks, the feature matrix grows — and linear algebra decides how that matrix is stored and scanned efficiently.

So even the "non-neural" models are linear algebra under the hood. There is no escaping it.

Embeddings — Turning Words Into Vectors

Modern AI represents a word like "Nifty" as a vector of, say, 768 numbers, learned so that similar concepts sit close together. "Nifty" and "Sensex" have nearby vectors; "Nifty" and "banana" are far apart. This is pure linear algebra: meaning becomes geometry. Retrieval, search, and RAG (later in this series) all rely on finding the nearest vector — a dot-product or distance computation.

For a trader this is powerful: you can turn news headlines into vectors and ask "which past headline is most like today's?" — that is similarity search, all linear algebra.

Dimensionality & The Curse

More dimensions sound better, but they break down: in very high dimensions, every vector looks roughly equally far from every other (the "curse of dimensionality"). That is why blindly adding 200 features to your model hurts. Techniques like PCA (principal component analysis) use linear algebra (eigen-decomposition) to keep the few directions that carry the most signal and drop the rest. I use PCA or feature selection before training XGBoost on Nifty data — it cuts overfitting sharply.

Common Mistakes (Linear-Algebra Related)

• Not normalizing features → one column dominates (see norm above).
• Feeding raw prices across scales → model learns the scale, not the pattern.
• Dimensionality explosion → adding correlated features that confuse the model.
• Ignoring multicollinearity → two near-identical features double-count signal; in linear models this inflates variance.
• Assuming matrix order doesn't matter → A×B ≠ B×A. Wrong order = wrong model.

How to Verify Your Linear Algebra Is Sound

1. Check feature scales — normalize or standardize everything.
2. Check correlation matrix — drop pairs above ~0.9.
3. For neural nets, watch activation statistics per layer; if values explode/vanish, your matrix transforms are unstable (use Batch Norm — article 19).
4. For embeddings, sanity-check that similar inputs produce nearby vectors (cosine similarity).
5. Always backtest on temporal splits, not random — linear algebra doesn't know time, you must enforce it.

Linear Algebra vs Calculus — The Division of Labor

• Linear algebra moves data: input → vector → matrix → output. It is the plumbing.
• Calculus adjusts the matrices: computes gradients, tells each weight how to change. It is the tuning.

A model with perfect linear algebra but broken calculus never learns. A model with perfect calculus but messy data (un-normalized, collinear) learns the wrong thing. You need both — and now you know what each does.

FAQ

Do I need to multiply matrices by hand?

No. Libraries do it. But you must know what is happening, or you cannot debug a model that silently fails.

Is linear algebra the same as "tensor math"?

A tensor is just a multi-dimensional array; the math on it (multiply, transpose, norm) is linear algebra generalized to N dimensions.

Why does my model need so much RAM?

Because it stores giant matrices (weights × batch × layers). A 7B-parameter model is mostly a 7-billion-number matrix. That is why quantization (article 21) shrinks them to fit a phone.

Can linear algebra predict the market?

It carries your data efficiently; it cannot invent signal that isn't in the features. As always: the model is only as good as the data and the features you feed it.

Should I learn this before using AI?

Yes, at the concept level. Vectors = data points, matrices = layers, dot product = similarity. With those three ideas you can read almost any AI paper's methods section.

A Trader's 5-Minute Intuition Build

Take your last 30 trading days. Write each day as a vector of 3 numbers: [Nifty return, Bank Nifty return, USD-INR change]. Now compute the dot product between Monday and Tuesday. A large positive value means those two days moved together — similar market regime. A value near zero means they were independent. Do this for all pairs and you have built a similarity matrix — a real linear-algebra object — that tells you which days "rhyme." Models like k-NN, clustering, and attention do exactly this, just with more dimensions and learned weights. The math is not exotic; it is the geometry of your own trading diary.

What Comes Next in the Series

This was component #2. Up next:

1. Probability & Statistics in AI — how models express and handle uncertainty (critical for risk).
2. Optimization — SGD, Adam, and why your learning rate betrays you.
3. Then the ML components: neurons, loss functions, backprop deep-dive, regularization.

The full index and all published articles are tracked on optiontradingwithai.in so you can read them in order or jump to what you need.

Key Takeaways

• Linear algebra = the language and plumbing of AI (vectors, matrices, tensors).
• A neural network is mostly matrix multiplication + a squish, repeated.
• Dot product = similarity; norm = scale control; tensor = N-D spreadsheet.
• XGBoost and neural nets both rely on it underneath.
• Normalize features, kill collinearity, beware the curse of dimensionality.
• Linear algebra moves data; calculus tunes the weights. Both required.

This is article 2 of the AI Components series. Article 3 covers Probability & Statistics in AI — how models express uncertainty. Track the full series on optiontradingwithai.in.

---

About the Author

Shakti Tiwari — Nifty Option Trader, Research Analyst & XGBoost Expert. Publishes daily NSE India research and practical AI for ordinary retail traders.

🌐 Website: optiontradingwithai.in
📕 Option Trading with AI → https://www.amazon.in/dp/B0H9ZNTBPK
📗 The AI Opportunity → https://www.amazon.in/dp/B0HBBFKDQF
📢 Daily Nifty analysis on Telegram: https://t.me/shaktitrade
📧 Free help: [e-mail]

Research only, not SEBI-registered advice. Verify everything before acting.

ai #machinelearning #linearalgebra #xgboost #nifty #india #datascience #deeplearning #optiontrading