Device conversion, from_numpy() and numpy() in PyTorch

*Memos:

• My post explains how to check PyTorch version, CPU and GPU(CUDA).
• My post explains how to create a tensor.
• My post explains how to access a tensor.
• My post explains is_tensor(), numel() and device().
• My post explains type conversion with type(), to() and a tensor.
• My post explains type promotion, result_type(), promote_types() and can_cast().
• My post explains set_default_dtype(), set_default_device() and set_printoptions().
• My post explains manual_seed(), initial_seed() and seed().

to() can do device conversion as shown below:

*Memos:

• to() can be used with a tensor but not with torch.
• The 1st argument(str, int or device()) with a tensor is device(Required).
• cpu, cuda, ipu, xpu, mkldnn, opengl, opencl, ideep, hip, ve, fpga, ort, xla, lazy, vulkan, mps, meta, hpu, mtia or privateuseone can be set to device.
• cuda is selected if setting 0 to device.
• A copied tensor can be created.

import torch

cpu_tensor = torch.tensor([0, 1, 2])

cpu_tensor.device
# device(type='cpu')

gpu_tensor = cpu_tensor.to(device='cuda:0')
gpu_tensor = cpu_tensor.to(device='cuda')
gpu_tensor = cpu_tensor.to(device=0)
gpu_tensor = cpu_tensor.to(device=torch.device(device='cuda:0'))
gpu_tensor = cpu_tensor.to(device=torch.device(device='cuda'))
gpu_tensor = cpu_tensor.to(device=torch.device(device=0))
gpu_tensor = cpu_tensor.to(device=torch.device(type='cuda', index=0))
gpu_tensor = cpu_tensor.to(device=torch.device(type='cuda'))

gpu_tensor.device
# device(type='cuda', index=0)

cpu_tensor = gpu_tensor.to(device='cpu')

cpu_tensor.device
# device(type='cpu')

cuda() and cpu() can change the device of a tensor to GPU(CUDA) and CPU respectively as shwon below:

*Memos:

• cuda() or cpu() can be used with a tensor but not with torch.
• For cuda(), - the 1st argument(str, int or device()) with a tensor is device(Optional). *Memos:
  • The default is the current GPU(CUDA).
  • cpu, cuda, ipu, xpu, mkldnn, opengl, opencl, ideep, hip, ve, fpga, ort, xla, lazy, vulkan, mps, meta, hpu, mtia or privateuseone can be set to device.
• cuda is selected if setting 0 to device.
• A copied tensor is created.

import torch

cpu_tensor = torch.tensor([0, 1, 2])

cpu_tensor.device
# device(type='cpu')

gpu_tensor = cpu_tensor.cuda(device='cuda:0')
gpu_tensor = cpu_tensor.cuda(device='cuda')
gpu_tensor = cpu_tensor.cuda(device=0)
gpu_tensor = cpu_tensor.cuda(device=torch.device(device='cuda:0'))
gpu_tensor = cpu_tensor.cuda(device=torch.device(device='cuda'))
gpu_tensor = cpu_tensor.cuda(device=torch.device(device=0))
gpu_tensor = cpu_tensor.cuda(device=torch.device(type='cuda', index=0))
gpu_tensor = cpu_tensor.cuda(device=torch.device(type='cuda'))

gpu_tensor.device
# device(type='cuda', index=0)

cpu_tensor = gpu_tensor.cpu(device='cpu')

cpu_tensor.device
# device(type='cpu')

from_numpy() can convert a NumPy array to a PyTorch tensor as shown below:

*Memos:

• from_numpy() can be used with torch but not with a tensor.
• The 1st argument(ndarray) with torch is required. *There is no keyword argument.
• The type of a NumPy array is also inherited to a PyTorch tensor.

import torch

my_array = np.array([0., 1., 2.])

my_array.dtype
# dtype('float64')

my_tensor = torch.from_numpy(my_array)

my_tensor
# tensor([0., 1., 2.], dtype=torch.float64)

numpy() can convert a PyTorch tensor to a NumPy array as shown below:

*Memos:

• numpy() can be used with a tensor but not with torch.
• There is force argument(bool) (Optional-Default:False) with a tensor. *Memos:
  • If it's True, a GPU(CUDA) PyTorch tensor can be converted to a NumPy array which may be a copy.
  • force= must be used.
• The type of a PyTorch tensor is also inherited to a NumPy array.

import torch

my_tensor = torch.tensor([0., 1., 2.])

my_tensor.dtype
# torch.float32

my_array = my_tensor.numpy()

my_array
# array([0., 1., 2.], dtype=float32)

my_tensor = torch.tensor([0., 1., 2.], device='cuda:0')

my_tensor.numpy(force=True)
# array([0., 1., 2.], dtype=float32)

my_tensor = torch.tensor([0., 1., 2.], device='cuda:0')

my_tensor.numpy()
my_tensor.numpy(force=False)
# Error