espnet2.asr.state_spaces.components.TransposedLinear

source

class espnet2.asr.state_spaces.components.TransposedLinear(d_input, d_output, bias=True)

Bases: Module

Transposed linear module.

This module performs a linear transformation on the second-to-last dimension of the input tensor. It assumes that the input tensor has the shape (B, D, L), where B is the batch size, D is the number of features, and L can be one or more dimensions.

weight

The learnable weight parameter of the module.

• Type: torch.Parameter

bias

The learnable bias parameter or 0.0 if bias is not used.

• Type: torch.Parameter or float

• Parameters:

  • d_input (int) – The number of input features (D).
  • d_output (int) – The number of output features.
  • bias (bool , optional) – Whether to include a bias term. Defaults to True.

• Returns: The output tensor after applying the linear transformation.

• Return type: Tensor

####### Examples

>>> import torch
>>> layer = TransposedLinear(d_input=4, d_output=2)
>>> x = torch.randn(3, 4, 5)  # Batch size of 3, 4 features, 5 lengths
>>> output = layer(x)
>>> output.shape
torch.Size([3, 2, 5])  # Output has 2 features

NOTE

The weight is initialized using Kaiming uniform initialization, which is suitable for layers with ReLU activations. If bias is used, it is initialized uniformly within a specific bound based on the input dimensions.

Initialize internal Module state, shared by both nn.Module and ScriptModule.

forward(x)

Transposed linear module.

This module applies a linear transformation on the second-to-last dimension of the input tensor. It assumes that the input tensor has the shape (B, D, L), where B is the batch size, D is the number of features, and L can be one or more additional dimensions.

weight

The learnable weights of the module.

• Type: torch.Tensor

bias

The learnable bias of the module, or 0.0 if no bias is used.

• Type: torch.Tensor or float

• Parameters:

  • d_input (int) – Number of input features.
  • d_output (int) – Number of output features.
  • bias (bool , optional) – Whether to include a bias term. Defaults to True.

• Returns: The output tensor after applying the linear transformation.

• Return type: torch.Tensor

####### Examples

>>> layer = TransposedLinear(10, 5)
>>> input_tensor = torch.randn(2, 10, 3)  # Batch size of 2, 10 features, 3 lengths
>>> output_tensor = layer(input_tensor)
>>> output_tensor.shape
torch.Size([2, 5, 3])  # Output shape should be (B, D_out, L)

NOTE

The linear transformation is performed using the einsum contraction to handle the multi-dimensional input tensor efficiently.