DEV Community

Super Kai (Kazuya Ito)
Super Kai (Kazuya Ito)

Posted on • Edited on

Dropout in PyTorch

Buy Me a Coffee

*Memos:

Dropout() can get the 0D or more D tensor of the zero or more elements randomly zeroed or multiplied from the 0D or more D tensor of zero or more elements as shown below:

*Memos:

  • The 1st argument for initialization is p(Optional-Default:0.5-Type:float): *Memos:
    • It's the probability of an element to be zeroed.
    • It must be 0 <= x <= 1.
  • The 2nd argument for initialization is inplace(Optional-Default:False-Type:bool):
    • It does in-place operation.
    • Keep it False because it's problematic with True.
  • The 1st argument is input(Required-Type:tensor of float): *Memos:
    • It must be the 0D or more D tensor of zero or more elements.
    • The tensor's requires_grad which is False by default is not set to True by Dropout().
import torch
from torch import nn

tensor1 = torch.tensor([8., -3., 0., 1., 5., -2.])

tensor1.requires_grad
# False

torch.manual_seed(7)

dropout1 = nn.Dropout()
tensor2 = dropout1(input=tensor1)
tensor2
# tensor([16., -0., 0., 2., 10., -4.])

tensor2.requires_grad
# False

dropout1
# Dropout(p=0.5, inplace=False)

dropout1.p
# 0.5

dropout1.inplace
# False

torch.manual_seed(7)

dropout2 = nn.Dropout()
dropout2(input=tensor2)
# tensor([32., -0., 0., 4., 20., -8.])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.5, inplace=False)
dropout(input=tensor1)
# tensor([16., -0., 0., 2., 10., -4.])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.8)
dropout(input=tensor1)
# tensor([40., -0., 0., 0., 25., -0.])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.3)
dropout(input=tensor1)
# tensor([11.4286, -0.0000, 0.0000, 1.4286, 7.1429, -2.8571])

my_tensor = torch.tensor([[8., -3., 0.],
                          [1., 5., -2.]])
torch.manual_seed(7)

dropout = nn.Dropout()
dropout(input=my_tensor)
# tensor([[16., -0., 0.],
#         [2., 10., -4.]])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.8)
dropout(input=my_tensor)
# tensor([[40., -0., 0.],
#         [0., 25., -0.]])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.3)
dropout(input=my_tensor)
# tensor([[11.4286, -0.0000, 0.0000],
#         [1.4286, 7.1429, -2.8571]])

my_tensor = torch.tensor([[8.], [-3.], [0.],
                          [1.], [5.], [-2.]])
torch.manual_seed(7)

dropout = nn.Dropout()
dropout(input=my_tensor)
# tensor([[16.], [-0.], [0.], [2.], [10.], [-4.]])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.8)
dropout(input=my_tensor)
# tensor([[40.], [-0.], [0.], [0.], [25.], [-0.]])

torch.manual_seed(7)

dropout = nn.Dropout(p=0.3)
dropout(input=my_tensor)
# tensor([[11.4286], [-0.0000], [0.0000], [1.4286], [7.1429], [-2.8571]])

my_tensor = torch.tensor([[[8., -3., 0.],
                           [1., 5., -2.]]])
torch.manual_seed(7)

dropout = nn.Dropout()
dropout(input=my_tensor)
# tensor([[[16., -0., 0.],
#          [2., 10., -4.]]])
Enter fullscreen mode Exit fullscreen mode

Image of Timescale

Timescale – the developer's data platform for modern apps, built on PostgreSQL

Timescale Cloud is PostgreSQL optimized for speed, scale, and performance. Over 3 million IoT, AI, crypto, and dev tool apps are powered by Timescale. Try it free today! No credit card required.

Try free

Top comments (0)

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more