espnet2.enh.layers.adapt_layers.MulAddAdaptLayer

source

class espnet2.enh.layers.adapt_layers.MulAddAdaptLayer(indim, enrolldim, ninputs=1, do_addition=True)

Bases: Module

Layer that performs multiplication and addition adaptation.

This layer adapts the main neural network activations by either multiplying or adding the enrollment embeddings, depending on the specified mode. The layer can be configured to perform addition or only multiplication based on the do_addition flag.

ninputs

The number of input tensors to be processed.

• Type: int

do_addition

A flag to indicate whether to perform addition along with multiplication. If True, the enrollment dimension must be twice the input dimension.

• Type: bool

• Parameters:

  • indim (int) – The dimension of the input activations.
  • enrolldim (int) – The dimension of the enrollment embeddings.
  • ninputs (int , optional) – The number of input tensors (default is 1).
  • do_addition (bool , optional) – Flag to perform addition (default is True).

• Raises:AssertionError – If do_addition is True and enrolldim is not equal to twice indim, or if do_addition is False and enrolldim is not equal to indim.

####### Examples

>>> layer = MulAddAdaptLayer(indim=128, enrolldim=256, ninputs=2)
>>> main_input = torch.randn(10, 128)
>>> enroll_input = torch.randn(10, 256)
>>> output = layer(main_input, enroll_input)
>>> print(output[0].shape)
torch.Size([10, 128])

>>> layer_no_add = MulAddAdaptLayer(indim=128, enrolldim=128,
...                                   ninputs=2, do_addition=False)
>>> output_no_add = layer_no_add(main_input, enroll_input)
>>> print(output_no_add[0].shape)
torch.Size([10, 128])

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(main, enroll)

A layer that performs multiplication and addition for adaptation.

This layer adapts the activations of a neural network using an enrollment embedding. Depending on the configuration, it can either perform both multiplication and addition or just multiplication.

ninputs

The number of inputs to the layer.

• Type: int

do_addition

A flag to indicate if addition should be performed. If True, the enrollment dimension must be twice the input dimension. If False, they must be equal.

• Type: bool

• Parameters:

  • indim (int) – The input dimension of the main activations.
  • enrolldim (int) – The dimension of the enrollment embeddings.
  • ninputs (int , optional) – The number of input tensors. Defaults to 1.
  • do_addition (bool , optional) – Whether to perform addition. Defaults to True.

• Raises:AssertionError – If the enrollment dimension does not match the expected dimension based on the configuration.

####### Examples

>>> layer = MulAddAdaptLayer(indim=64, enrolldim=128, ninputs=2)
>>> main_input = (torch.randn(10, 64), torch.randn(10, 64))
>>> enroll_input = (torch.randn(10, 128), torch.randn(10, 128))
>>> output = layer(main_input, enroll_input)
>>> print(output[0].shape)  # Output shape should match (10, 64)

NOTE

The input tensors can be provided as a tuple or list. This is useful for cases where both normal and skip connections are being adapted simultaneously. The forward method ensures that the main and enroll inputs are of the same type and length as specified by ninputs.