espnet2.tasks.abs_task.IteratorOptions

source

class espnet2.tasks.abs_task.IteratorOptions(preprocess_fn: callable, collate_fn: callable, data_path_and_name_and_type: list, shape_files: list, batch_size: int, batch_bins: int, batch_type: str, max_cache_size: float, max_cache_fd: int, allow_multi_rates: bool, distributed: bool, num_batches: int | None, num_iters_per_epoch: int | None, train: bool)

Bases: object

IteratorOptions holds the configuration options for the iterator.

preprocess_fn

A function to preprocess the data.

• Type: callable

collate_fn

A function to collate data samples into a batch.

• Type: callable

data_path_and_name_and_type

A list of tuples specifying the data paths, names, and types.

• Type: list

shape_files

A list of shape files for the dataset.

• Type: list

batch_size

The size of each mini-batch.

• Type: int

batch_bins

The number of bins for batching.

• Type: int

batch_type

The type of batching to use.

• Type: str

max_cache_size

The maximum cache size for data loading.

• Type: float

max_cache_fd

The maximum number of file descriptors for opened files.

• Type: int

allow_multi_rates

Whether to allow multiple sampling rates.

• Type: bool

distributed

Whether the data loading is distributed across multiple devices.

• Type: bool

num_batches

The number of batches to process.

• Type: Optional[int]

num_iters_per_epoch

The number of iterations per epoch.

• Type: Optional[int]

train

A flag indicating if the iterator is for training.

• Type: bool

Examples

>>> options = IteratorOptions(
...     preprocess_fn=my_preprocess_fn,
...     collate_fn=my_collate_fn,
...     data_path_and_name_and_type=[("data/path", "name", "type")],
...     shape_files=["shape/file"],
...     batch_size=32,
...     batch_bins=1000,
...     batch_type="unsorted",
...     max_cache_size=0.1,
...     max_cache_fd=10,
...     allow_multi_rates=True,
...     distributed=False,
...     num_batches=100,
...     num_iters_per_epoch=10,
...     train=True
... )

allow_multi_rates

*: bool*

batch_bins

*: int*

batch_size

*: int*

batch_type

*: str*

collate_fn

*: callable*

data_path_and_name_and_type

*: list*

distributed

*: bool*

max_cache_fd

*: int*

max_cache_size

*: float*

num_batches

*: int | [None](../asr/AbsDecoder.md#espnet2.asr.decoder.abs_decoder.AbsDecoder.None)*

num_iters_per_epoch

*: int | [None](../asr/AbsDecoder.md#espnet2.asr.decoder.abs_decoder.AbsDecoder.None)*

preprocess_fn

*: callable*

shape_files

*: list*

train

*: bool*