espnet2.asr.state_spaces.residual.Feedforward

source

class espnet2.asr.state_spaces.residual.Feedforward(*args)

Bases: Residual

Feedforward residual connection that bypasses input features.

This class implements a feedforward residual connection where the input features are directly passed to the output without any scaling. The main purpose is to allow for a direct pathway in the network, effectively bypassing the residual computation.

i_layer

The index of the current layer.

• Type: int

d_input

The dimensionality of the input features.

• Type: int

d_model

The dimensionality of the model output.

• Type: int

alpha

Scaling factor for the input features, defaults to 0.0.

• Type: float

beta

Scaling factor for the output features, defaults to 1.0.

• Type: float

• Parameters:*args – Variable length argument list passed to the parent class.

Examples

>>> feedforward_layer = Feedforward(i_layer=1, d_input=256, d_model=256)
>>> x = torch.randn(10, 256)  # Batch of 10, input dimension 256
>>> y = torch.randn(10, 256)  # Batch of 10, output dimension 256
>>> output = feedforward_layer(x, y, transposed=False)
>>> print(output.shape)
torch.Size([10, 256])  # Output shape matches the input shape

NOTE

This implementation sets alpha to 0.0, which means the input features will not be scaled, effectively making it a bypass connection for the input features.

Initialize internal Module state, shared by both nn.Module and ScriptModule.