Hello ♦
Central idea of the article: Machine learning works with data represented as numbers. Linear algebra gives us a language for reasoning about those numbers as geometric objects.
Reading time 10 minutes
The simplest data can be represented by a single number:
- The speed of a car
- The number of disks
- The mass of a compressor
But why do we need to represent data as numbers? Obviously, to answer our questions:
- How long will it take me to get from point A to point B?
- What is the maximum number of disks that can fit in a box?
- Will I be able to carry this compressor by myself?
You ask yourself questions like these every day, and numbers help you answer them and make the right decisions, so you don't have to suffer:
- Finally, I can get 8 hours of sleep because I'll get there quickly.
- So I need to buy 20 disks — I can't fit any more in the box.
- I don't want to strain my back, so I'll ask Kamil to help me.
But what if we want to describe an object as a whole, rather than just its individual characteristics? For example, to estimate its value, efficiency, or something else.
Let's consider some characteristics of a car:
For example: mass = 1,500 kg, power = 200 hp, speed = 220 km/h.
We could store these as three separate numbers, but then we lose the idea that these three values describe the same object.
This leads to a natural idea: combine several numbers into a single structure.
A simple real-life analogy is a pencil case. People invented pencil cases to keep everything they need for studying in one place — from pencils to scissors. If we know we'll need scissors, pens, and pencils, we put them all in a pencil case rather than scattering them across separate pockets of a backpack.
In a similar way, something happened with data: several related numbers came to be treated as a single mathematical object — ♦ a vector ♦.
♦ Vector ♦
Originally, vectors did not exist as a separate mathematical concept. People worked with scalars — plain numbers. But as physics began to develop, scientists encountered quantities that had not only a magnitude, but also a direction.
17th century (René Descartes): He connected geometry with algebra by introducing the Cartesian coordinate system. A point in space could now be represented by a set of numbers: (x, y, z) in three-dimensional space and (x, y) in two-dimensional space. The coordinate system made it possible to represent the geometry of an object as a set of numbers.
In other words, we can turn an object or its properties into a numerical representation.
Apartments:
Let's say we have two apartments that we rent out to tourists visiting the city. We also know that our friend Tomi, who owns 25 apartments, has no trouble renting out any of them.
We'd like to understand why only one of our apartments is profitable — what's wrong with the other one?
Apartment A
- Area: 35 m²
- 2 rooms
- 5th floor
- 5 km from the city center
Apartment B
- Area: 50 m²
- 1 room
- 2nd floor
- 3 km from the city center
Approximate characteristics of Tomi's apartments:
Apartment B
- Area: 45 m²
- 1 room
- 6th floor
- 1 km from the city center
And there are approximately 25 apartments with similar characteristics.
Let's try to represent all of this as a mathematical structure that describes the object as a whole — a vector.
x_{1} = [35, 2, 5, 5]
x_{2} = [50, 1, 2, 3]
x_{3} = [45, 1, 6, 1]
...
x_{25} = [49, 1, 5, 1.2]
We think about it, discuss it with our wife, and realize that people want to live closer to the city center so they can be closer to the city's main attractions. Tourists also generally don't need a lot of space. In other words, they don't need a two- or three-room apartment. For a family of three or four, a one-bedroom apartment is usually a perfectly acceptable option.
Assuming that the key factors are the apartment's size and its distance from the city center, let's plot a graph with:
the x-axis representing the apartment's area;
the y-axis representing the distance from the city center.
Looking at a large amount of data in the form of lists, it's difficult to tell how far our apartments are from the ideal profile.
Based on the two features we selected, Apartment B is closer to the typical profile of Tomi's apartments. We should probably sell Apartment A altogether, since it isn't a good fit for short-term rentals to tourists. It would be better suited for long-term rentals.
In the 19th century, the ideas of vector calculus developed further, particularly through the work of William Hamilton, Hermann Grassmann, James Clerk Maxwell, and other mathematicians and physicists.
A vector was no longer just a pair of numbers or a point in space [3, 5]. It became an object that could be operated on:
addition;
subtraction;
multiplication by a scalar;
dot product;
calculating its magnitude;
determining its direction.
In other words, a vector turned out to be a convenient way to represent the structure and relationships between numerical quantities.
This means that we can do more than simply represent points in space, as we did with the apartments.
For example, we have created a visual representation showing where our apartments are located.
But it would be useful to know the exact numbers — how different are our apartments?
This is one of the advantages of vectors: we can calculate the differences easily and get the complete picture:
visual representation + precise numbers
Let's calculate the average values for Tomi's apartments:
44.48 — average area
1.12 — average distance from the city center
Now let's compare our apartments with Tomi's average:
For Apartment A:
35 − 44.48 = −9.48 m²
5 − 1.12 = 3.88 km
For Apartment B:
50 − 44.48 = 5.52 m²
3 − 1.12 = 1.88 km
The minus sign in the first case simply shows where Apartment A's area lies relative to the average area.
Now let's visualize this average so that we can focus on a single object that represents all of Tomi's apartments.
Ideally, we want to buy apartments whose characteristics are as close as possible to the green point.
Analytically, we can calculate which apartment is closer to the green point:
Apartment A: A = [35, 5]
Apartment B: B = [50, 3]
Average apartment: C = [44.48, 1.12]
We can calculate the distance:
$d(A,C)=\sqrt{(35−44.48)^{2}+(5−1.12)^{2}}$
$d(B,C)=\sqrt{(50−44.48)^{2}+(3−1.12)^{2}}$
And if we can measure the distance between apartments, why not measure the distance between texts? Images? Users? Products? This could help us understand how similar they are.
I think you've already noticed that the mathematical framework allows us to work not only with 2D or 3D vectors, but with n-dimensional vectors as well. This is one of the biggest advantages of vectors: we can represent and analyze vectors with as many numbers as we need — or as many as our technical constraints allow.
This means we can represent almost anything. From pixels to chemical structures.
But there's an important point we need to stop and think about.
We don't simply turn an object into numbers. We create a representation of that object, and the quality of this representation matters enormously.
For example, we could represent an apartment like this:
[50, 2, 7, 1.5]
Or like this:
[2, 50, 7, 1.5]
But if the order of the features hasn't been defined beforehand, this vector is meaningless.
What I mean is that the numbers inside a vector don't mean anything by themselves. Their meaning comes from what each position represents. A vector only makes sense when we define what each dimension represents.
Or, in more mathematical terms:
A number by itself has no meaning until we fix a basis — what each axis of the space represents.
The hardest part is often not doing mathematics on data. It is deciding what the data should look like mathematically.
ML
ML uses vectors not just as convenient containers for numbers.
In human terms:
Let's represent each object as a point in space and try to find patterns in how these points are arranged.
For example, as shown in the figure, let's try to determine which object represents text, which represents a number, and which represents sound.
In other words, we encode the properties of an object as an ordered set of numbers — its coordinates. This turns the object into a point in a multidimensional space. Machine learning algorithms can then look for spatial relationships: distances between points, clusters, angles, and directions.
The important idea behind vectors isn't that they store numbers. It's that they allow us to turn objects into points in a mathematical space where we can measure relationships between them.
That's it.
I hope you found something useful here.
I’d describe my vibe today with this picture:
Top comments (0)