espnet2.enh.layers.ncsnpp_utils.normalization.VarianceNorm2d

source

class espnet2.enh.layers.ncsnpp_utils.normalization.VarianceNorm2d(num_features, bias=False)

Bases: Module

Variance Normalization layer for 2D inputs.

This layer normalizes the input tensor by its variance. It scales the normalized output using learnable parameters. This can help stabilize training and improve model performance by controlling the variance of the activations.

num_features

The number of input features (channels).

• Type: int

bias

If True, includes a bias term. Not currently used in this implementation.

• Type: bool

alpha

Learnable scaling parameter initialized from a normal distribution with mean 1 and standard deviation 0.02.

• Type: nn.Parameter

• Parameters:

  • num_features (int) – Number of features (channels) in the input.
  • bias (bool , optional) – Whether to include a bias term. Defaults to False.

• Returns: The scaled and normalized output tensor.

• Return type: Tensor

####### Examples

>>> variance_norm = VarianceNorm2d(num_features=64)
>>> input_tensor = torch.randn(10, 64, 32, 32)  # Batch size of 10
>>> output_tensor = variance_norm(input_tensor)
>>> print(output_tensor.shape)
torch.Size([10, 64, 32, 32])

NOTE

The variance is computed over the spatial dimensions (height and width) of the input tensor. A small constant (1e-5) is added to avoid division by zero.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)

Applies the variance normalization to the input tensor.

This method computes the variance of the input tensor x along the spatial dimensions (height and width) and normalizes the input by dividing it by the square root of the variance (plus a small epsilon for numerical stability). It then scales the normalized output using the learnable parameter alpha.

• Parameters:x (torch.Tensor) – Input tensor of shape (N, C, H, W) where: N = batch size, C = number of channels, H = height of the feature map, W = width of the feature map.
• Returns: The normalized output tensor of the same shape as the input.
• Return type: torch.Tensor

####### Examples

>>> model = VarianceNorm2d(num_features=3)
>>> input_tensor = torch.randn(2, 3, 4, 4)  # Batch size of 2, 3 channels
>>> output_tensor = model(input_tensor)
>>> output_tensor.shape
torch.Size([2, 3, 4, 4])

NOTE

The normalization is performed per channel independently.