espnet2.lm.abs_model.AbsLM

source

class espnet2.lm.abs_model.AbsLM(*args, **kwargs)

Bases: Module, BatchScorerInterface, ABC

The abstract base class for Language Models (LMs) in ESPnet.

This class defines the interface for language models and shares the loss calculation method among different model implementations. It employs the delegate pattern, where an instance of this class is passed to the LanguageModel.

Example

>>> from espnet2.lm.abs_model import AbsLM
>>> lm = AbsLM()  # This will raise NotImplementedError
>>> model = LanguageESPnetModel(lm=lm)

NOTE

This class cannot be instantiated directly since it is an abstract class. Subclasses must implement the forward method.

None

forward(input

torch.Tensor, hidden: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]: Abstract method that must be implemented by subclasses to define the forward pass of the language model.

• Raises:
  • NotImplementedError – If the forward method is called without being
  • implemented in a subclass. –

Initialize internal Module state, shared by both nn.Module and ScriptModule.

abstract forward(input: Tensor, hidden: Tensor) → Tuple[Tensor, Tensor]

Computes the forward pass of the language model.

This method takes an input tensor and a hidden state tensor, processes them through the language model, and returns the output tensor along with the updated hidden state. It must be implemented by subclasses of the abstract class AbsLM.

• Parameters:
  • input (torch.Tensor) – The input tensor containing the data for the language model.
  • hidden (torch.Tensor) – The hidden state tensor from the previous time step, which is updated during the forward pass.
• Returns: A tuple containing: : - output (torch.Tensor): The output tensor after processing the input through the model.
  • hidden (torch.Tensor): The updated hidden state tensor.
• Return type: Tuple[torch.Tensor, torch.Tensor]
• Raises:NotImplementedError – If this method is called directly on an instance of AbsLM, since it must be implemented in subclasses.

##

Example

s

>>> model = MyLanguageModel()
>>> input_tensor = torch.randn(10, 32)  # (sequence_length, batch_size)
>>> hidden_state = torch.zeros(1, 32, 256)  # (num_layers, batch_size, hidden_size)
>>> output, new_hidden = model.forward(input_tensor, hidden_state)

NOTE

This method is essential for the functioning of the language model and should be properly defined in any subclass.