espnet2.asr.state_spaces.components.DropoutNd

source

class espnet2.asr.state_spaces.components.DropoutNd(p: float = 0.5, tie=True, transposed=True)

Bases: Module

N-dimensional dropout module.

This module applies dropout in N dimensions, allowing for flexible application of dropout masks across multiple axes. It supports options to tie the dropout mask across sequence lengths and to apply dropout in a transposed manner.

p

Probability of an element being zeroed. Must be in the range [0, 1).

• Type: float

tie

If True, tie the dropout mask across sequence lengths.

• Type: bool

transposed

If True, apply dropout in a transposed manner.

• Type: bool

binomial

A binomial distribution used to sample dropout masks.

• Type: Binomial

• Parameters:

  • p (float) – Dropout probability (default: 0.5).
  • tie (bool) – Whether to tie dropout mask across sequence lengths (default: True).
  • transposed (bool) – Whether to apply dropout in a transposed manner (default: True).

• Raises:ValueError – If p is not in the range [0, 1).

####### Examples

>>> dropout = DropoutNd(p=0.3)
>>> input_tensor = torch.randn(2, 3, 4)  # Example input
>>> output_tensor = dropout(input_tensor)
>>> print(output_tensor.shape)
torch.Size([2, 3, 4])  # Shape remains the same, but values are dropped

NOTE

This module is particularly useful in sequence models where you may want to apply dropout independently to each sequence step.

Initialize dropout module.

tie: tie dropout mask across sequence lengths (Dropout1d/2d/3d)

forward(X)

Perform the forward pass of the DropoutNd module.

This method applies dropout to the input tensor during training. It randomly zeroes elements of the input tensor based on the specified dropout probability p. The behavior can be controlled with the tie and transposed attributes, which determine how the dropout mask is applied.

• Parameters:input (Tensor) – The input tensor of shape (batch, dim, lengths…). The dimensions beyond the second are treated as additional lengths over which the dropout is applied.
• Returns: The output tensor with dropout applied. If the module is : not in training mode, the input tensor is returned unchanged.
• Return type: Tensor

####### Examples

>>> dropout = DropoutNd(p=0.5, tie=True, transposed=True)
>>> input_tensor = torch.rand(10, 3, 5)  # Example input
>>> output_tensor = dropout(input_tensor)
>>> print(output_tensor.shape)
torch.Size([10, 3, 5])  # Shape remains the same, but values are zeroed out

NOTE

When tie is set to True, the same dropout mask is applied across all lengths, while if tie is False, each length has its own independent dropout mask.