Skip to main content

espnet2.tasks.lm.LMTask

About 4 min

espnet2.tasks.lm.LMTask

source

class espnet2.tasks.lm.LMTask

Bases: AbsTask

LMTask is a class that handles the language modeling tasks in the ESPnet2

framework. It extends the AbsTask class and provides methods for argument parsing, model building, data preprocessing, and collation of data for training and evaluation.

num_optimizers

The number of optimizers to be used (default is 1).

  • Type: int

class_choices_list

A list of ClassChoices instances for models and language models.

  • Type: list

trainer

The Trainer class to be used for training and evaluation.

  • Type:Trainer

  • Parameters:parser (argparse.ArgumentParser) – The argument parser to which task related arguments will be added.

  • Returns: The updated argument parser with task related : arguments.

  • Return type: argparse.ArgumentParser

  • Yields:Callable – A function that collates data for training or evaluation.

  • Raises:RuntimeError – If the token_list is not of type str or dict.

################# Examples

To add task arguments to a parser

parser = argparse.ArgumentParser() LMTask.add_task_arguments(parser)

To build a model

args = parser.parse_args() model = LMTask.build_model(args)

########## NOTE If you need to modify the training or evaluation procedures, you can change the Trainer class assigned to the ‘trainer’ attribute.

classmethod add_task_arguments(parser: ArgumentParser)

Adds command-line arguments specific to the language modeling task.

This method configures the argument parser with options relevant to the language modeling task, including token lists, initialization methods, and preprocessing options. It allows users to specify various parameters that control the behavior of the task during execution.

  • Parameters:parser (argparse.ArgumentParser) – The argument parser to which the task-related arguments will be added.
  • Returns: The updated argument parser with task : arguments included.
  • Return type: argparse.ArgumentParser

################# Examples

>>> import argparse
>>> parser = argparse.ArgumentParser()
>>> LMTask.add_task_arguments(parser)
>>> args = parser.parse_args(["--token_list", "path/to/token_list.txt"])
>>> print(args.token_list)
path/to/token_list.txt

########## NOTE The –token_list argument is crucial for mapping integer IDs to tokens. The initialization method can be chosen from a predefined set of options. Additionally, preprocessing options can be enabled or disabled.

classmethod build_collate_fn(args: Namespace, train: bool) → Callable[[Collection[Tuple[str, Dict[str, ndarray]]]], Tuple[List[str], Dict[str, Tensor]]]

Builds a collate function for batching data.

This method constructs a callable that collates a batch of data samples into a single tensor, ensuring that the sequences are properly padded to the same length. The resulting batch will include both the tokenized text and corresponding feature tensors.

  • Parameters:
    • args (argparse.Namespace) – The command-line arguments containing configuration options for the task.
    • train (bool) – A flag indicating whether the collate function is being built for training or evaluation.
  • Returns: Callable[[Collection[Tuple[str, Dict[str, np.ndarray]]]], : > Tuple[List[str], Dict[str, torch.Tensor]]]: <br/> A function that takes a collection of samples and returns a tuple containing a list of texts and a dictionary of tensors.

################# Examples

>>> collate_fn = LMTask.build_collate_fn(args, train=True)
>>> batch_samples = [
...     ("sample1", {"feature": np.array([1, 2, 3])}),
...     ("sample2", {"feature": np.array([4, 5])}),
... ]
>>> texts, features = collate_fn(batch_samples)
>>> print(texts)
['sample1', 'sample2']
>>> print(features)
{'feature': tensor([[1, 2, 3],
                     [4, 5, 0]])}  # Example padded tensor

########## NOTE This method uses the CommonCollateFn for the actual implementation of the collate functionality.

classmethod build_model(args: Namespace) → ESPnetLanguageModel | ESPnetMultitaskLanguageModel

Builds and initializes the language model based on the provided arguments.

This method constructs a language model using the specified model type and configuration parameters. It handles the creation of both the language model and the ESPnet model, initializing them with the appropriate parameters.

  • Parameters:args (argparse.Namespace) – The parsed command line arguments containing model configuration options, including the token list and model type.
  • Returns: An instance of the selected language model class.
  • Return type: Union[ESPnetLanguageModel, ESPnetMultitaskLanguageModel]
  • Raises:RuntimeError – If the token_list is not provided as a string or list.

################# Examples

To build a model with a token list specified:

import argparse

parser = argparse.ArgumentParser() parser.add_argument(’–token_list’, type=str, required=True) parser.add_argument(’–lm’, type=str, default=’seq_rnn’) parser.add_argument(’–model’, type=str, default=’lm’) args = parser.parse_args()

model = LMTask.build_model(args)

########## NOTE This method assumes that the token list is either a path to a file containing the tokens or a list of tokens. The vocabulary size is derived from the token list, which is essential for initializing the language model correctly.

classmethod build_preprocess_fn(args: Namespace, train: bool) → Callable[[str, Dict[str, array]], Dict[str, ndarray]] | None

Builds a preprocessing function based on the provided arguments.

This method checks if preprocessing is enabled and returns a CommonPreprocessor instance configured with the specified parameters. If preprocessing is not enabled, it returns None.

  • Parameters:
    • cls – The class reference (usually the class itself).
    • args (argparse.Namespace) – The namespace containing command-line arguments.
    • train (bool) – A flag indicating whether the preprocessing is for training or evaluation.
  • Returns: A callable preprocessing function if preprocessing is enabled, otherwise None.
  • Return type: Optional[Callable[[str, Dict[str, np.array]], Dict[str, np.ndarray]]]

################# Examples

To create a preprocessing function for training:

>>> args = argparse.Namespace(use_preprocessor=True, token_type='bpe',
... token_list='tokens.txt', bpemodel='model.bpe',
... cleaner='tacotron', g2p=None, non_linguistic_symbols=None)
>>> preprocess_fn = LMTask.build_preprocess_fn(args, train=True)
>>> output = preprocess_fn("sample text", {"key": np.array([1, 2, 3])})

If preprocessing is disabled:

>>> args.use_preprocessor = False
>>> preprocess_fn = LMTask.build_preprocess_fn(args, train=True)
>>> assert preprocess_fn is None

########## NOTE This method relies on the CommonPreprocessor class for preprocessing tasks, which should be defined elsewhere in the codebase.

class_choices_list

*: List[[ClassChoices](../train/ClassChoices.md#espnet2.train.class_choices.ClassChoices)]* *= [<espnet2.train.class_choices.ClassChoices object>, <espnet2.train.class_choices.ClassChoices object>]*

num_optimizers

*: int* *= 1*

classmethod optional_data_names(train: bool = True, inference: bool = False) → Tuple[str, ...]

Returns the optional data names required by the task.

This method returns a tuple of optional data names that the task may use during training or inference. By default, it returns an empty tuple, indicating that there are no optional data names.

  • Parameters:
    • train (bool) – Indicates whether the data is for training. Default is True.
    • inference (bool) – Indicates whether the data is for inference. Default is False.
  • Returns: A tuple containing the names of optional data.
  • Return type: Tuple[str, …]

################# Examples

>>> optional_names = LMTask.optional_data_names()
>>> print(optional_names)
()  # Returns an empty tuple by default

########## NOTE This method can be overridden in subclasses to specify additional optional data names.

classmethod required_data_names(train: bool = True, inference: bool = False) → Tuple[str, ...]

Returns the required data names for the task.

This method specifies the data names that are essential for the training or inference of the language model task. By default, it returns a tuple containing the string “text”, which indicates that text data is required.

  • Parameters:
    • train (bool , optional) – A flag indicating whether the method is called for training. Defaults to True.
    • inference (bool , optional) – A flag indicating whether the method is called for inference. Defaults to False.
  • Returns: A tuple of required data names. In this case, it will return (“text”,).
  • Return type: Tuple[str, …]

################# Examples

>>> LMTask.required_data_names(train=True)
('text',)
>>> LMTask.required_data_names(inference=True)
('text',)

trainer

alias of Trainer

Copyright © 2024 ESPnet Community. All rights reserved.