ESPnet3 Components (espnet3/components)

This page is a developer-oriented map of the reusable building blocks under espnet3/components/. Components are instantiated from YAML (Hydra/OmegaConf) and wired by Systems and stage entrypoints. In ESPnet3, a "component" is any reusable unit that is shared across systems/stages (data, modeling, training, optimization, metrics)

If you can answer "which config key selects which component?", you can extend ESPnet3 without changing the stage driver.

Directory map

The espnet3/components/ directory is currently organized into:

espnet3/components/
├── callbacks/   # Lightning callbacks and related helpers
├── data/        # datasets, organizers, dataloader builders, stats
├── metrics/     # AbsMetrics interfaces and metric implementations
├── modeling/    # model wrappers (LightningModule) and model utilities
├── optim/       # optimizer/scheduler wiring (incl. multi-optim)
└── training/    # trainer wrappers and training loop helpers

What you customize (cheat sheet)

Goal	You change	Where it is used
Build train/valid/test splits	dataset (usually DataOrganizer)	collect_stats, train, infer, measure
Change batching/sharding/collate	dataloader	collect_stats, train
Swap the training wrapper (LightningModule/Trainer)	code: espnet3/components/modeling/*, espnet3/components/training/*	collect_stats, train
Add training behaviors	trainer.callbacks	train
Change optimization	optim/scheduler (or multi-optim variants)	train
Add new evaluation metrics	metrics.metric + metrics.inputs	measure

Key interfaces (contracts)

These are the "sharp edges" to document and keep stable. When something feels confusing in components, it usually comes back to one of these contracts.

Callbacks (training-time hooks)

Callbacks are Lightning objects that run during training (checkpointing, logging, averaging, progress bars, etc.). ESPnet3 provides a default stack, and you can extend or override it via trainer.callbacks in train.yaml.

• Component doc: Callbacks
• Config doc: Train configuration

DataOrganizer and dataset items

ESPnet3 typically builds datasets through DataOrganizer. Each dataset item is an arbitrary dict-like object produced by your dataset class (wrapped by DatasetWithTransform / CombinedDataset when configured).

If you plan to implement a custom organizer, see the "Writing a custom DataOrganizer" section on that page.

• Component doc: DataOrganizer and dataset pipeline
• Config docs: Train configuration, Inference configuration, Measure configuration, Config overview

Metrics (AbsMetrics)

Metrics are implemented as AbsMetrics subclasses and called by the measure stage with aligned SCP fields.

• Stage doc: Measure stage
• Component doc: Metrics
• Config doc: Measure configuration

Modeling / training wrappers

If you want to change how training is wired (LightningModule/trainer wrappers), you will typically modify code under espnet3/components/modeling/ and espnet3/components/training/ and keep the YAML-facing interface stable.

• Component docs: Lightning module, Trainer
• Config doc: Train configuration

Optimizers and schedulers

Optimizer/scheduler wiring is config-driven but has strict consistency rules (especially for multi-optimizer setups).

• Config doc: Optimizer configuration
• Component docs: Multiple optimizers & schedulers

"Source of truth": tests

For developer-facing behavior, the tests are the most precise specification. If you are unsure what a component accepts or returns, check:

• test/espnet3/components/*
• test/espnet3/systems/base/* (stage wiring behavior)