ESPnet3 Metrics Configuration

This page explains the metrics.yaml schema used by the measure stage.

Important

Metrics run after inference and read SCP outputs from inference_dir.

Minimum required keys

Typical metrics runs need:

• metrics
• inference_dir

dataset.test is optional. If it is omitted, ESPnet3 scans subdirectories under inference_dir and uses those names as test sets.

Basic shape

recipe_dir: .
exp_tag:
exp_dir: ${recipe_dir}/exp/${exp_tag}

metrics:
  - metric:
      _target_: espnet3.systems.asr.metrics.wer.WER
      ref_key: ref
      hyp_key: hyp

Optional explicit test-set definitions:

dataset:
  _target_: espnet3.components.data.data_organizer.DataOrganizer
  recipe_dir: ${recipe_dir}
  test:
    - name: test-clean
      data_src_args:
        split: test-clean

Main sections

Section	Description
inference_dir	Root directory containing per-test-set SCP outputs
dataset.test	Optional explicit test-set names
metrics	Metric definitions and optional SCP input mappings

metrics

Each list entry contains:

• metric
• optional inputs

Example:

metrics:
  - metric:
      _target_: espnet3.systems.asr.metrics.wer.WER
      clean_types: ["whisper_basic"]
    inputs:
      ref: ref
      hyp: hyp

If inputs is omitted, ESPnet3 falls back to the metric's ref_key and hyp_key attributes.

SCP input mapping

Each inputs entry maps to a file under:

${inference_dir}/${test_name}/

Example:

• inputs.ref: ref -> ${inference_dir}/${test_name}/ref.scp
• inputs.hyp: hyp -> ${inference_dir}/${test_name}/hyp.scp

Output

The metrics stage writes:

${inference_dir}/metrics.json

Each metric produces one result block keyed by its class path.

Related pages

Inference Config

See where `inference_dir` and SCP outputs come from.

Metrics Stage

Read the stage-level behavior behind `measure`.

Custom Metrics

See the reusable metric layer for implementing your own metrics.