ESPnet3 Dataset References And Builders

This page is the main reference for how ESPnet3 resolves datasets from YAML and how the create_dataset stage interacts with recipe dataset modules.

Key implementations:

• espnet3.components.data.dataset_module
• espnet3.components.data.dataset_builder.DatasetBuilder
• espnet3.systems.base.system.BaseSystem.create_dataset

Dataset resolution from YAML

Each dataset entry is a small reference object. The important keys are:

• data_src
• data_src_args

Example:

dataset:
  recipe_dir: ${recipe_dir}
  train:
    - data_src: mini_an4/asr
      data_src_args:
        split: train

You usually do not need to write _target_ here. dataset already defaults to DataOrganizer.

ESPnet3 parses that entry and then instantiates:

Dataset(**data_src_args)

from the resolved dataset module.

The three supported dataset reference forms

1. Dataset tag

data_src: mini_an4/asr

This resolves to:

egs3.mini_an4.asr.dataset

Tags are convenient for reusing another recipe's dataset module.

1. Explicit module path

data_src: egs3.mini_an4.asr.dataset

This imports the given module directly.

1. Omit data_src

dataset:
  recipe_dir: ${recipe_dir}
  train:
    - data_src_args:
        split: train

If data_src is omitted, ESPnet3 loads:

${recipe_dir}/dataset/__init__.py

This is the normal pattern for recipe-local datasets.

What data_src_args means

data_src_args is passed directly to the exported Dataset class.

Example:

data_src_args:
  split: test-clean
  recipe_dir: ${recipe_dir}
  source_dir: ${dataset_dir}

becomes:

Dataset(split="test-clean", recipe_dir=recipe_dir, source_dir=dataset_dir)

Top-level dataset entry fields such as:

• name
• transform

are not part of Dataset.__init__. They stay in organizer space.

Expected dataset module structure

A dataset module should export:

from .builder import MyBuilder as DatasetBuilder
from .dataset import MyDataset as Dataset

This lets the same module support:

• normal dataset instantiation through Dataset
• create_dataset through DatasetBuilder

create_dataset call flow

BaseSystem.create_dataset() loops over all dataset entries in:

• dataset.train
• dataset.valid
• dataset.test

For each unique dataset source, it instantiates DatasetBuilder() and runs:

if not builder.is_source_prepared(**builder_kwargs):
    builder.prepare_source(**builder_kwargs)

if not builder.is_built(**builder_kwargs):
    builder.build(**builder_kwargs)

The builder_kwargs come from:

create_dataset:
  ...

in training.yaml.

For a task-oriented walkthrough, start with the custom dataset guide. For lifecycle diagrams and parallel preparation patterns, use the lifecycle pages below.

Custom Dataset Guide

See the practical recipe-local Dataset and DatasetBuilder workflow.

Data Pipeline Migration

See how local/data.sh maps to builder.py, dataset.py, and config.

Parallel Data Preparation

See the DatasetBuilder lifecycle diagram and parallel download pattern.

Builder lifecycle in detail

is_source_prepared()

Cheap check for raw source availability.

Examples:

• extracted corpus exists
• source directory exists
• archive was already unpacked

Returns:

• True: raw source is available
• False: prepare_source() should run

prepare_source()

Makes raw source available.

Typical work:

• download archives
• extract corpora
• normalize incoming directory layout
• verify that required source directories exist

is_built()

Cheap check for task-ready artifacts.

Examples:

• manifest TSVs already exist
• precomputed files are already written
• for raw-directory-backed recipes, this may simply mirror source readiness

Returns:

• True: build() can be skipped
• False: build() should run

build()

Creates the files consumed by the dataset class or later stages.

Typical work:

• manifest generation
• audio conversion
• metadata generation
• derived split preparation

Example 1: mini_an4

Local mode config:

dataset:
  recipe_dir: ${recipe_dir}
  train:
    - data_src_args:
        split: train
  valid:
    - data_src_args:
        split: valid
  test:
    - name: test
      data_src_args:
        split: test

create_dataset:
  recipe_dir: ${recipe_dir}

What happens:

1. local dataset module ${recipe_dir}/dataset/__init__.py is loaded
2. MiniAn4Builder is instantiated
3. prepare_source() extracts AN4 if needed
4. build() converts audio and writes manifest TSVs
5. MiniAn4Dataset(split=...) reads those manifest files

Example 2: librispeech_100

Local mode config:

dataset:
  recipe_dir: ${recipe_dir}
  train:
    - data_src_args:
        split: train-clean-100
  valid:
    - data_src_args:
        split: dev-clean

create_dataset:
  recipe_dir: ${recipe_dir}
  source_dir: ${dataset_dir}

What happens:

1. local dataset module is loaded
2. LibriSpeech100Builder validates source availability
3. build() does not generate manifests; it only preserves the ready state
4. LibriSpeech100Dataset(split=..., source_dir=...) scans the original corpus tree directly

This is the useful contrast with mini_an4:

• mini_an4: source preparation plus manifest build
• librispeech_100: source validation with direct raw-tree reading

Recommended practical rules

• Put recipe-local dataset code under dataset/, not src/.
• Keep DatasetBuilder idempotent.
• Keep is_source_prepared() and is_built() cheap.
• Put recipe-specific constructor args in data_src_args.
• Put builder-stage args in create_dataset.

Related pages

DataOrganizer

See how dataset references are organized into train, valid, and test splits.

Dataloader

See how datasets connect to collate functions and iterators.

Create dataset stage

Return to the stage-level dataset preparation flow.

Custom Dataset Guide

Read the end-user guide for Dataset, DatasetBuilder, and dataset config.

Data Pipeline Migration

Read the lifecycle-focused guide for porting old data preparation.

Parallel Data Preparation

See lifecycle diagrams and provider/runner use for heavy data prep.