espnet2.gan_svs.visinger2.visinger2_vocoder.LayerNorm

source

class espnet2.gan_svs.visinger2.visinger2_vocoder.LayerNorm(channels, eps=1e-05)

Bases: Module

Layer normalization module.

This module applies layer normalization to the input tensor, normalizing across the specified number of channels. Layer normalization is useful in stabilizing the learning process and improving convergence in deep learning models.

channels

The number of channels to normalize.

• Type: int

eps

A small constant added for numerical stability.

• Type: float

gamma

Scale parameter for the normalized output.

• Type: torch.nn.Parameter

beta

Shift parameter for the normalized output.

• Type: torch.nn.Parameter

• Parameters:

  • channels (int) – Number of channels to normalize.
  • eps (float) – A small constant to prevent division by zero. Default is 1e-5.

• Returns: Normalized output tensor.

• Return type: Tensor

####### Examples

>>> layer_norm = LayerNorm(channels=10)
>>> input_tensor = torch.randn(2, 10, 5)  # Batch size 2, 10 channels, 5 time steps
>>> output_tensor = layer_norm(input_tensor)
>>> print(output_tensor.shape)
torch.Size([2, 10, 5])  # Output shape is the same as input shape

NOTE

The normalization is performed over the last dimension (channels).

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)

Calculate forward propagation through the LayerNorm layer.

This method applies layer normalization to the input tensor, which is useful for stabilizing the learning process by normalizing the inputs to each layer.

• Parameters:x (Tensor) – Input tensor of shape (B, C, T), where B is the batch size, C is the number of channels, and T is the sequence length.
• Returns: Normalized output tensor of the same shape as input x.
• Return type: Tensor

####### Examples

>>> layer_norm = LayerNorm(channels=10)
>>> input_tensor = torch.randn(5, 10, 20)  # Batch size of 5, 10 channels
>>> output_tensor = layer_norm(input_tensor)
>>> output_tensor.shape
torch.Size([5, 10, 20])

NOTE

The layer normalization is applied independently to each channel, which means that for each input tensor, the mean and variance are computed along the channel dimension.