ESPnet3 Components (espnet3/components)

This page is a developer-oriented map of the reusable building blocks under espnet3/components/.

Components are instantiated from YAML and wired by Systems and stage entrypoints.

In ESPnet3, a "component" is any reusable unit shared across systems or stages:

• data
• modeling
• training
• optimization
• metrics
• callbacks

If you can answer "which config key selects which component?", you can extend ESPnet3 without changing the stage driver.

Directory map

The espnet3/components/ directory is currently organized into:

espnet3/components/
├── callbacks/   # Lightning callbacks and related helpers
├── data/        # datasets, organizers, dataloader builders, stats
├── metrics/     # BaseMetric interfaces and metric implementations
├── modeling/    # LightningModule and optimization wiring
└── trainers/    # trainer wrappers and training loop helpers

Current optimizer and scheduler wiring lives under modeling/, not under a separate optimizer/ package.

What you customize

Goal	You change	Where it is used
Build train/valid/test splits	dataset (usually DataOrganizer)	collect_stats, train, infer, measure
Change batching, collate, iterator setup	dataloader	collect_stats, train
Swap the training wrapper	code under espnet3/components/modeling/* and espnet3/components/trainers/*	collect_stats, train
Add training behavior	trainer.callbacks	train
Change optimization	optimizer / scheduler or optimizers / schedulers	train
Add new evaluation metrics	metrics.metric + metrics.inputs	measure

Key interfaces

These are the main component contracts.

Callbacks

Callbacks are Lightning objects that run during training.

Typical roles:

• checkpointing
• logging
• progress bars
• metric summaries

ESPnet3 provides a default stack and lets recipes extend or override it through trainer.callbacks in training.yaml.

• Component doc: Callbacks
• Config doc: Training configuration

DataOrganizer and dataset items

ESPnet3 usually builds datasets through DataOrganizer. Each dataset item is an arbitrary dict-like object produced by your dataset class.

If you plan to implement a custom organizer, see the custom organizer section on that page.

• Component docs:
  • DataOrganizer
  • Dataset references and builders
• Config docs:
  • Training configuration
  • Inference configuration
  • Measure configuration
  • Config overview

Metrics (BaseMetric)

Metrics are implemented as BaseMetric subclasses and called by the measure stage with path-based inputs.

• Stage doc: Measure stage
• Component doc: Metrics
• Config doc: Measure configuration

Modeling and training wrappers

If you want to change how training is wired, you will usually modify code under:

• espnet3/components/modeling/
• espnet3/components/trainers/

and keep the YAML-facing interface stable.

• Component docs:
  • Model
  • Trainer
• Config doc: Training configuration

Optimizers and schedulers

Optimizer and scheduler wiring is config-driven. This includes:

• the single-optimizer path
• the named multi-optimizer path

The multi-optimizer path has stricter rules, especially for parameter routing, gradient clipping, and scheduler naming.

• Component docs:
  • Optimizer configuration
  • Multiple optimizers and schedulers
• Config doc: Training configuration

Useful references

• Callbacks
• Dataset references and builders
• DataOrganizer
• Metrics
• Model
• Optimizer configuration
• Multiple optimizers and schedulers
• Trainer

Source of truth

For developer-facing behavior, tests are the most precise specification.

If a component contract feels unclear, check:

• test/espnet3/components/*
• test/espnet3/systems/base/*