espnet2.asr.state_spaces.components.stochastic_depth

source

espnet2.asr.state_spaces.components.stochastic_depth(input: tensor, p: float, mode: str, training: bool = True)

Apply stochastic depth.

Implements the Stochastic Depth from “Deep Networks with Stochastic Depth” used for randomly dropping residual branches of residual architectures.

Stochastic depth helps in training deep networks by randomly skipping layers during training, which can improve generalization and reduce overfitting.

• Parameters:
  • input (Tensor *[*N , ... ]) – The input tensor or arbitrary dimensions with the first one being its batch i.e. a batch with N rows.
  • p (float) – Probability of the input to be zeroed, where 0 means no dropout and 1 means dropping the entire input.
  • mode (str) – Specifies the mode of operation, can be either:
    • “batch”: Randomly zeroes the entire input tensor.
    • “row”: Randomly zeroes selected rows from the batch.
  • training (bool) – Indicates whether to apply stochastic depth. If False, the input is returned unmodified. Default: True.
• Returns: The randomly zeroed tensor, with the same shape as the input.
• Return type: Tensor[N, …]
• Raises:
  • ValueError – If p is not in the range [0, 1], or if mode is not
  • "batch"` or "row" –

Examples

>>> input_tensor = torch.randn(4, 3)  # A batch of 4 samples with 3 features
>>> output_tensor = stochastic_depth(input_tensor, p=0.5, mode="row")
>>> output_tensor.shape
torch.Size([4, 3])  # Output shape remains the same as input

>>> output_tensor = stochastic_depth(input_tensor, p=0.2, mode="batch")
>>> output_tensor.shape
torch.Size([4, 3])  # Output shape remains the same as input