espnet3.systems.base.system.BaseSystem

source

class espnet3.systems.base.system.BaseSystem(, configs: Mapping[str, DictConfig | None], exp_dir: Path | None = None, stages_to_run: Sequence[str] | None = None)

Bases: object

Base class for all ESPnet3 systems.

A system declares its stages as the class attribute stages: a tuple of StageSpec, in the order “all” runs them. Every public method a system defines (class-defined, not starting with _) must be one of these stage names - a system has no other public surface. See espnet3.contract.stages.check_stage_contract for the checks run when a subclass is defined, and espnet3.contract.stages.stage_log_dir for how a stage’s log directory is resolved.

Class Attributes: : DATASET_BUILDER_CLASS_NAME: Name of the builder class expected in each : dataset module (default "DatasetBuilder"). <br/> DATASET_CLASS_NAME: Name of the dataset class expected in each dataset : module (default "Dataset"). Used by subclasses that instantiate datasets directly. <br/> stages: The declared stage order (see above).

All behavior is config-driven.

################# Examples

>>> system = BaseSystem(
...     configs={"training": OmegaConf.create({"exp_dir": "./exp/x"})}
... )
>>> [s.name for s in system.stages[:3]]
['create_dataset', 'collect_stats', 'train']

Build each requested stage’s config and hold it, by stage name.

• Parameters:
  • configs – Each config role’s own config (its default plus whatever the caller loaded), not yet resolved, keyed by role name ("training", "inference", …). A role this system has no stage for is ignored; a role it does have a stage for but that is missing here is treated as empty. A stage that needs its own recipe_dir reads it from its own config (self.stage_configs[stage].recipe_dir), not from a separate constructor argument.
  • exp_dir – The experiment directory to bake stage configs under and read earlier baked configs from. When None, derived from the first given config whose own exp_dir resolves by itself (see _exp_dir_from); if none does, raises.
  • stages_to_run – The stages this run actually requests. Only these (and the earlier stages they inherit from) have their config built and resolved; a later stage’s missing identity never stops construction. None builds every declared stage.
• Raises:ConfigError – exp_dir is not given and no config in configs has a self-resolving exp_dir.

Notes

Each stage’s config is its own DictConfig object - the earlier stages’ own keys (this run’s, in memory, or an earlier run’s baked file), then this stage’s own config on top, then resolved. No two stages share one config object, so one stage popping or overwriting a key never affects another.

DATASET_BUILDER_CLASS_NAME = 'DatasetBuilder'

DATASET_CLASS_NAME = 'Dataset'

collect_stats(*args, **kwargs)

Collect statistics needed for training.

Reads the same dataset: block as train. By default the DataOrganizer is built with train_mode=False, so the train preprocessor’s augmentation (data_aug_effects, speed perturbation and so on) is not applied, as in ESPnet2: the statistics and the shape files the sampler batches by are then the same on every run. train still augments. To collect statistics with the augmentation applied, set it in the training config:

collect_stats_train_mode: true

################# Examples

``

`

python system = ASRSystem(

configs={“training”: training_config}, exp_dir=exp_dir

) system.collect_stats()

Writes stats under the stage config’s stats_dir.

``

`

create_dataset(*args, **kwargs)

Create datasets from dataset references.

################# Examples

>>> system = BaseSystem(configs={"training": training_config})
>>> system.create_dataset()

infer(*args, **kwargs)

Run inference on the configured datasets.

################# Examples

``

`

python system = ASRSystem(

configs={“inference”: inference_config}, exp_dir=exp_dir

) system.infer()

Writes decoded hypotheses under the stage config’s inference_dir.

``

`

measure(*args, **kwargs)

Compute evaluation metrics from hypothesis/reference outputs.

################# Examples

``

`

python system = ASRSystem(

configs={“metrics”: metrics_config}, exp_dir=exp_dir

) result = system.measure()

result holds the computed metric values.

``

`

pack_demo(*args, **kwargs)

Pack demo assets into a runnable demo directory.

pack_model(*args, **kwargs)

Pack model artifacts into an espnet3 bundle.

################# Examples

``

`

python system = ASRSystem(

configs={ : “training”: training_config, “publication”: publication_config,

}, exp_dir=exp_dir,

) system.pack_model()

Packs model artifacts under the stage config’s pack_model.out_dir.

``

`

stages : ClassVar[Tuple[StageSpec, ...]] = (StageSpec(name='create_dataset', config='training', log_dir='data_dir', dir_aliases=()), StageSpec(name='collect_stats', config='training', log_dir='stats_dir', dir_aliases=()), StageSpec(name='train', config='training', log_dir='exp_dir', dir_aliases=()), StageSpec(name='infer', config='inference', log_dir='inference_dir', dir_aliases=()), StageSpec(name='measure', config='metrics', log_dir='inference_dir', dir_aliases=()), StageSpec(name='pack_model', config=('training', 'publication'), log_dir=None, dir_aliases=()), StageSpec(name='upload_model', config='publication', log_dir=None, dir_aliases=()), StageSpec(name='pack_demo', config='demo', log_dir='pack.out_dir', dir_aliases=()), StageSpec(name='upload_demo', config='demo', log_dir='pack.out_dir', dir_aliases=()))

train(*args, **kwargs)

Train the system model.

################# Examples

``

`

python system = ASRSystem(

configs={“training”: training_config}, exp_dir=exp_dir

) system.train()

Runs model.fit() and saves the resolved config under exp_dir.

``

`

upload_demo(*args, **kwargs)

Upload demo bundle to HuggingFace.

upload_model(*args, **kwargs)

Upload model bundle to HuggingFace.