PyTorch :: Understanding Tensors (Part 1)

In this post, you will start from the very basics of deep learning frameworks. TENSORS

So basically a neural network is the bunch of nodes that take one or more input, processes them and returns one or more output.

Table of Content

1. What are Tensors
2. Creating Tensors
3. Tensor Types and Their Conversions
4. Shapes and Dimensions
   • Reshaping Tensors

What are Tensors

Tensors are the building blocks of a neural network. A PyTorch tensor is the generalized form of arrays in $n$ dimensions to run arbitrary computations on GPU. [READ MORE]

A tensor can be of any dimension. In this course you will learn about the following dimensions only

• 0D -> Scalar numbers like 0, 10, 20 etc. are 0-Dimensional tensors
• 1D -> Array of numbers like [1, 2, 3] or [20, 30, 10] etc.
• 2D -> Matrix of numbers like [[1, 2, 3], [2, 3, 4]]

Note During the process of learning, the network applies some tensor methods on these tensor values in order to gain the desired output.

Before creating and using tensors in PyTorch, install it by following the instructions from here: https://pytorch.org/get-started/locally/

Creating Tensors

You can create the tensors by importing torch and using torch.tensor(data) method.

import torch

t0 = torch.tensor(10)                     # 0-D tensor
t1 = torch.tensor([1, 2, 3, 4])           # 1-D tensor
t2 = torch.tensor([[1, 2, 3], [2, 3, 4]]) # 2-D tensor

print(t0)
print(t1)
print(t2)

tensor(10)
tensor([1, 2, 3, 4])
tensor([[1, 2, 3],
        [2, 3, 4]])

To convert it into numpy array, simply use Tensor.numpy() method and use it as you wish

print(t2.numpy())

array([[1, 2, 3],
       [2, 3, 4]])

Tensor Types and Their Conversions

PyTorch supports the following data-types of the tensors

To get the data type use Tensor.dtype property and Tensor.type() to get the PyTorch tensor type

print(t2.dtype)
print(t2.type())

torch.int64
torch.LongTensor

Note Difference between .dtype and .type() is better explained here

To explicitly create a tensor of specific type, either pass dtype while defining the tensor in torch.tensor method or use tensor constructor.

For example, here I am creating a float tensor by providing int32 data

tf0 = torch.tensor([1, 2, 3, 4], dtype=torch.float32)
print(tf0)
print(tf0.dtype)

tf1 = torch.FloatTensor([5, 6, 7, 8])
print(tf1)
print(tf1.dtype)

tensor([1., 2., 3., 4.])
torch.float32
tensor([5., 6., 7., 8.])
torch.float32

You can pass-in the the datatype in the Tensor.type([type]) method, in order to change the data-type of the existing tensor.

For example, changing float tensor to int32

tf0 = tf0.type(torch.int32) # after changing, it returns a new tensor object of that type
print(tf0.dtype)

torch.int32

Shapes and Dimensions

While training the model, you will deal with higher dimensions and neural network only accept the dimension which is defined at input layer while architecting the model.

The dimension basically tells whether the tensor is 0-D or 1-D or 2-D or even higher than that. Dimension of tensor is also called the rank of the tensor.

To get the size you can use Tensor.size() method or Tensor.shape property and to get the dimension of the tensor, use Tensor.ndimension() method or Tensor.ndim property.

print(t1.size())
print(t1.ndim)

print(t2.shape)
print(t2.ndimension())

torch.Size([4])
1
torch.Size([2, 3])
2

Reshaping Tensors

Sometimes working with neural network, you will face a situation when you are supposed to change the dimension of the tensor. This can be done by using Tensor.view(nrows, ncols) method of the tensor

For example, I have a tensor [[1, 2, 3], [2, 3, 4]] and I want to convert it to 2D with 1 row and 6 columns

tx = t2.view(1, 6)
print(tx)

tensor([[1, 2, 3, 2, 3, 4]])

Note The number of parameters in .view method is the total number of dimensions

# Converting 2D to 1D tensor
tx = t2.view(6)
print(tx)

tensor([1, 2, 3, 2, 3, 4])

There are special values for the view method. Suppose you don't know how many rows would be good while resizing your matrix, you can set it to -1 it will automatically adjust the rows for you. Same goes with columns.

# Telling PyTorch to keep the max 2 elements in one column and make as many rows as you want
tr = t2.view(-1, 2)
print(tr)

# Telling PyTorch to keep max 2 rows and adjust columns automatically
tc = t2.view(2, -1)
tc

tensor([[1, 2],
        [3, 2],
        [3, 4]])
tensor([[1, 2, 3],
        [2, 3, 4]])

Note Both the rows and columns can't be set to -1 at same time.

I hope you have liked this post. Please share it with your friends and colleagues and help them learn the concepts of PyTorch. If you have doubts or any idea reach me out via following sources

• Email: [e-mail] (Recommended)
• Twitter: @tbhaxor
• GitHub: @tbhaxor
• LinkedIn: @gurkirat--singh
• Instagram: @_tbhaxor_