Juan Carlos

Posted on

# Wat?

• Whats Scalar?.
• Whats Vector?.
• Whats Matrix?.
• Whats Tensor?.

This is a quick and short intro to Tensors for busy people...
I will keep it KISS and to the minimum possible for easy reading.

Scalar is the minimum possible unit we are working with, like an integer.

Vector is a bunch of Scalars in a particular axis, can be drawn as a line of boxes or items, lets say is a list or array collection of items.

Matrix is a 2 dimensions arrays, usually represented as a table.

Tensor is a multi-dimensional array, usually can be represented graphically as a cube.

# Rank & Dimension

• Whats Rank?.
• Whats Dimension?.

You can imagine lists of integers, accommodated on different layouts, on a 3D space, or 2D space, etc thats the Rank.

How it looks like on Code?.

# Scalar

``````let myscalar = 42
``````

Scalar can be a variety of things, usually numeric values, to keep things simple and easy to understand we will use an integer here, `42` is our Scalar.

# Vector

``````let myvector = [1, 2, 3]
``````

Vector is a collection of items, we continue using integers, it can be seen on the code as an array or list, you can draw it as a Rank 1 Vector.

# Matrix

``````let mymatrix = [
[1, 2, 3],
[4, 5, 6],
]
``````

We continue adding dimensions then we end up with the Matrix, a 2D Tensor, can be simplified on code as a list with lists inside.

# Tensor

``````let mytensor = [
[
[1, 2, 3],
[4, 5, 6],
],
[
[7,   8,  9],
[10, 11, 12],
],
]
``````

Wow, we reached the crazy cube, a multiple dimensions array of integers,
we need to convert this jam of lists into a `Tensor` object!.

# Tensor Arraymancer

``````import arraymancer

let mytensor = [
[
[1, 2, 3],
[4, 5, 6],
],
[
[7,   8,  9],
[10, 11, 12],
],
].toTensor
``````

Done, congrats you coded your first `Tensor`!.

# Playing with Tensors

What can I do with it?.

``````echo mytensor is Tensor  # Is really a Tensor type?
echo sizeOf(mytensor)    # Whats the Size of it?
echo mytensor.rank       # Whats the Rank?
echo mytensor.shape      # Whats the Shape of my Tensor?
echo mytensor.strides    # Strides?
echo mytensor.offset     # Offset? (if any)
``````

Print a human readable representation using:

``````echo mytensor
``````

This should print on the standard output or REPL:

``````Tensor[int] of shape [2, 2, 3] of type "int" on backend "CPU":
| |     1       2       3 |     7       8       9 |
| |     4       5       6 |     10      11      12|

``````

On this basic example I am using `CPU` as Backend, for compatibility,
yours can also say `CUDA` or `OpenCL` or `OpenMP` (if you have the hardware),
because GPUs are comfy massaging numbers.

# Tensor Slicing

You can read or write a value from the Tensor:

``````import arraymancer

var mytensor = [
[1, 2, 3, 4, 5] # 0
#0  1  2  3  4
].toTensor

echo mytensor[0, 2]  # Read the value "3"
mytensor[0, 2] = 42  # Write the value "42"
``````

Remember that stuff declared with `var` is mutable, `let` is immutable,
a Tensor can also be Compile-Time pre-computed declaring it with `const`.

We can overwrite that Scalar value, lets see if we can get the whole Column?.

(Comments added to explain indexes, not required, we will take it away soon)

``````import arraymancer

var mytensor = [
[1,   2,  3,  4,  5],   # 0
[6,   7,  8,  9, 10],   # 1
[11, 12, 13, 14, 15],   # 2
[16, 17, 18, 19, 20],   # 3
[21, 22, 23, 24, 25],   # 4
# 0   1   2   3   4
].toTensor

echo mytensor[_, 2]
``````

The only change is the `_` on the index to select the whole Column.

Lets try get an entire Row now, without comments this time...

``````import arraymancer

var mytensor = [
[1,   2,  3,  4,  5],
[6,   7,  8,  9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
].toTensor

echo mytensor[3, _]
``````

I hope was not too complicated, just the `_` for the Row.

Now lets grab just the center without touching the rest of it.

``````import arraymancer

var mytensor = [
[1,   2,  3,  4,  5],
[6,   7,  8,  9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
].toTensor

echo mytensor[2, 2]
``````

# Going 3D

Lets chop off a piece of this pie...

``````import arraymancer

var mytensor = [
[
[ 1,  2,  3,  4,  5],      # 1st Matrix.
[ 6,  7,  8,  9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
],
[
[26, 27, 28, 29, 30],      # 2nd Matrix.
[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40],
[41, 42, 43, 44, 45],
[46, 47, 48, 49, 50],
],
[
[51, 52, 53, 54, 55],      # 3rd Matrix.
[56, 57, 58, 59, 60],
[61, 62, 63, 64, 65],
[66, 67, 68, 69, 70],
[71, 72, 73, 74, 75],
],
[
[76, 77, 78, 79,  80],     # 4th Matrix.
[81, 82, 83, 84,  85],
[86, 87, 88, 89,  90],
[91, 92, 93, 94,  95],
[96, 97, 98, 99, 100],
],
[
[101, 102, 103, 104, 105], # 5th Matrix.
[106, 107, 108, 109, 110],
[111, 112, 113, 114, 115],
[116, 117, 118, 119, 120],
[121, 122, 123, 124, 125],
]
].toTensor

echo mytensor[3, _, _]
``````

What if we want the corner slice?.

``````import arraymancer

var mytensor = [
[
[ 1,  2,  3,  4,  5],      # 1st Matrix.
[ 6,  7,  8,  9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
],
[
[26, 27, 28, 29, 30],      # 2nd Matrix.
[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40],
[41, 42, 43, 44, 45],
[46, 47, 48, 49, 50],
],
[
[51, 52, 53, 54, 55],      # 3rd Matrix.
[56, 57, 58, 59, 60],
[61, 62, 63, 64, 65],
[66, 67, 68, 69, 70],
[71, 72, 73, 74, 75],
],
[
[76, 77, 78, 79,  80],     # 4th Matrix.
[81, 82, 83, 84,  85],
[86, 87, 88, 89,  90],
[91, 92, 93, 94,  95],
[96, 97, 98, 99, 100],
],
[
[101, 102, 103, 104, 105], # 5th Matrix.
[106, 107, 108, 109, 110],
[111, 112, 113, 114, 115],
[116, 117, 118, 119, 120],
[121, 122, 123, 124, 125],
]
].toTensor

echo mytensor[_, 1, 4]
``````

A last one slicing...

``````import arraymancer

var mytensor = [
[
[ 1,  2,  3,  4,  5],      # 1st Matrix.
[ 6,  7,  8,  9, 10],
[11, 12, 13, 14, 15],
[16, 17, 18, 19, 20],
[21, 22, 23, 24, 25],
],
[
[26, 27, 28, 29, 30],      # 2nd Matrix.
[31, 32, 33, 34, 35],
[36, 37, 38, 39, 40],
[41, 42, 43, 44, 45],
[46, 47, 48, 49, 50],
],
[
[51, 52, 53, 54, 55],      # 3rd Matrix.
[56, 57, 58, 59, 60],
[61, 62, 63, 64, 65],
[66, 67, 68, 69, 70],
[71, 72, 73, 74, 75],
],
[
[76, 77, 78, 79,  80],     # 4th Matrix.
[81, 82, 83, 84,  85],
[86, 87, 88, 89,  90],
[91, 92, 93, 94,  95],
[96, 97, 98, 99, 100],
],
[
[101, 102, 103, 104, 105], # 5th Matrix.
[106, 107, 108, 109, 110],
[111, 112, 113, 114, 115],
[116, 117, 118, 119, 120],
[121, 122, 123, 124, 125],
]
].toTensor

echo mytensor[2, 0, 2]
``````

# Tensor Exercises

To reduce repetition lets assume this base Tensor:

``````import arraymancer

var mytensor = [
[1, 2],
[3, 4],
].toTensor
``````

``````echo mytensor + mytensor
``````

# Tensor Substraction

``````echo mytensor - mytensor
``````
• Multiplication is left to the reader as exercise.

# Tensor Iterator

``````# Values only.
for value in mytensor:
echo value

# Coordinates and Values.
for coordinates, value in mytensor:
echo coordinates
echo value
``````
• Iteration of `pairs` and `items` is left to the reader as exercise.

``````# Broadcasting with operations are made beginning with a dot "."
echo mytensor .+ mytensor    # Addition
echo mytensor .- mytensor    # Substraction
``````

# Tensor from list

``````# Tensor from vanilla std lib sequence.
let mytensor = toSeq(1..24).toTensor.reshape(2, 3, 4)
``````
• Try reshaping your Tensors to new shapes!

# Why Arraymancer?

• C Speed.
• Python like syntax, Numpy like ergonomics.
• Accelerated with Intel MKL/OpenBLAS/CUDA/OpenCL/OpenMP.
• Dependency free distribution.
• 1 Language for the whole stack from experimental quick prototypes to Performance critical Production.

# This is just the beginning, is not the end

It really wont take more than a `nimble install arraymancer` to get started on CPU!.