espnet2.asr.state_spaces.components.StochasticDepth

source

class espnet2.asr.state_spaces.components.StochasticDepth(p: float, mode: str)

Bases: Module

Stochastic depth module for randomly dropping residual branches.

This module implements stochastic depth as described in the paper “Deep Networks with Stochastic Depth” (https://arxiv.org/abs/1603.09382). It randomly drops entire residual connections during training, which helps to improve model robustness and reduces overfitting.

p

The probability of dropping a residual branch.

• Type: float

mode

The mode of operation, either “batch” or “row”.

• Type: str

• Parameters:

  • p (float) – Probability of dropping the residual branch.
  • mode (str) – Mode of operation; can be “batch” or “row”.

####### Examples

>>> import torch
>>> sd = StochasticDepth(p=0.5, mode='batch')
>>> input_tensor = torch.randn(10, 3, 32, 32)  # Example input
>>> output_tensor = sd(input_tensor)  # Apply stochastic depth

NOTE

Ensure that the model is in training mode when applying this module to observe the effects of stochastic depth.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(input)

Perform the forward pass of the Stochastic Depth module.

This method applies stochastic depth to the input tensor, which randomly drops residual branches in the architecture during training, as described in the paper “Deep Networks with Stochastic Depth”. The behavior of the module is controlled by the drop probability p and the specified mode (“batch” or “row”).

• Parameters:input (Tensor) – The input tensor of shape (N, …), where N is the batch size and … represents any number of additional dimensions.
• Returns: The output tensor after applying stochastic depth. The shape : of the output tensor will match the input tensor.
• Return type: Tensor

####### Examples

>>> import torch
>>> stochastic_depth_layer = StochasticDepth(p=0.5, mode="row")
>>> input_tensor = torch.randn(10, 5)  # Batch of 10 samples, 5 features
>>> output_tensor = stochastic_depth_layer(input_tensor)
>>> output_tensor.shape
torch.Size([10, 5])  # Output shape matches input shape

NOTE

Stochastic depth is only applied during training. If the module is in evaluation mode, the input is returned unchanged.

• Raises:ValueError – If the input tensor is not of type torch.Tensor.