espnet3.components.dataloader.update_shard

source

espnet3.components.dataloader.update_shard(config: dict | list, shard_idx: int) → dict | list

Recursively replace all string occurrences of the placeholder “{shard_idx}” in a nested configuration structure with the actual shard index.

This function is typically used when a config dictionary or list (e.g., from OmegaConf.to_container) includes placeholders like “{shard_idx}” in file paths (e.g., shape_files), which need to be resolved dynamically based on the current shard index before initializing a data loader or training process.

For example, a config entry like: : shape_files: : - stats/train/split.12/speech_shape.

will be updated to: : shape_files: : - stats/train/split.12/speech_shape.3

when initializing shard_idx=3.

• Parameters:
  • config (Union *[*dict , list ]) – A nested configuration structure containing dictionaries, lists, or strings possibly including “{shard_idx}”.
  • shard_idx (int) – The shard index to be substituted into all relevant strings within the config.
• Returns: A new config structure with all “{shard_idx}” : placeholders replaced by the given shard index.
• Return type: Union[dict, list]