espnet3.systems.tts.remove_long_short_runner.RemoveLongShortRunner

source

class espnet3.systems.tts.remove_long_short_runner.RemoveLongShortRunner(provider: EnvironmentProvider, batch_size: int | None = None, output_dir: str | Path | None = None, shard_subdir: str = '', resume: bool = True)

Bases: BaseRunner

Runner for filtering utterances by audio duration in parallel.

Each shard appends its status dicts to a shard-local results.jsonl file, and merge() reads every shard file back and re-sorts by idx, so callers receive results in manifest order regardless of shard completion order.

Initialize BaseRunner object.

static forward(idx: int | Iterable[int], entries: List[Tuple[str, str, str]], min_duration: float, max_duration: float, **env) → Dict[str, Any] | list

Check duration bounds for the given index or batch.

• Parameters:
  • idx – Single index or iterable of indices into entries.
  • entries – List of (utt_id, wav_path, line) tuples from the manifest.
  • min_duration – Minimum allowed duration in seconds (exclusive).
  • max_duration – Maximum allowed duration in seconds (exclusive).
  • **env – Additional environment entries.
• Returns: A status dict for an int index, or a list of status dicts for an iterable. Each entry is {"idx": int, "utt_id": str, "keep": bool}.

merge(shard_dirs: List[Path]) → List[Dict[str, Any]]

Concatenate shard results and restore manifest (idx) order.

Each shard’s results.jsonl holds one JSON object per line, e.g.:

{"idx": 0, "utt_id": "103_1241_000000_000001", "keep": true}
{"idx": 1, "utt_id": "103_1241_000000_000002", "keep": false}
{"idx": 2, "utt_id": "103_1241_000001_000000", "keep": true}

static open_writers(shard_dir: Path | None, **env) → Dict[str, Any]

Open the shard-local JSONL results file.

static write_record(writers: Dict[str, Any], result: Any, state: Dict[str, Any], **env) → None

Append one forward result (or batch of results) to the shard file.