espnet3.components.data.collect_stats.CollectStatsRunner

source

class espnet3.components.data.collect_stats.CollectStatsRunner(provider: EnvironmentProvider, output_dir: str | Path, mode: str, write_collected_feats: bool = False, **kwargs)

Bases: BaseRunner

Runner that executes collect-stats over batches of indices.

Initialize CollectStatsRunner object.

static close_writers(writers: Dict[str, Any], state: Dict[str, Any], **env) → Dict[str, Any] | None

Close shard writers and flush shard summary files.

static forward(batch_indices: Iterable[int] | int, dataset, model, collate_fn, device, write_collected_feats: bool = False, collect_stats_kwargs: Dict[str, Any] | None = None, **env)

Process a batch of dataset indices and compute feature statistics.

merge(shard_dirs: List[Path]) → Dict[str, Any]

Merge shard outputs into aggregated statistics for one split.

static open_writers(shard_dir: Path | None, write_collected_feats: bool = False, **env) → Dict[str, Any]

Create per-shard writer state.

static write_record(writers: Dict[str, Any], result, state: Dict[str, Any], **env) → None

Merge per-batch stats and persist shapes/features.