Datasets

This page is the entry point for dataset-related internals in ESPnet3.

Check here first

• espnet3/components/data/dataset_module.py
• espnet3/components/data/data_organizer.py
• espnet3/components/data/dataset.py
• espnet3/components/data/dataset_builder.py
• espnet3/components/data/dataloader.py

Main concepts

• dataset reference resolution
• recipe-local dataset modules under egs3/<recipe>/<task>/dataset/
• DataOrganizer split management
• dataset wrappers such as combined and transformed datasets
• dataloader construction for training and inference

Related pages

System and stages

Read how dataset creation fits into `run.py` and stage dispatch.

Create dataset stage

See the stage-level flow that calls recipe-local builders.

Train stage

See how built datasets are consumed during training.

Custom dataset guide

Read the end-user guide for finetuning with a custom dataset.