espnet3.contract.kinds.audio.Audio

source

class espnet3.contract.kinds.audio.Audio(array: ndarray, rate: int)

Bases: object

A float32 waveform and the rate it is sampled at.

The one shape audio has once it is inside the API. Anything a caller is likely to hold - a path, the (rate, samples) pair gr.Audio returns, a NumPy array, a torch tensor - becomes one of these through coerce(), resampled to the rate the model wants, so a hook only ever sees audio.array at audio.rate.

Every channel is kept: array is (samples,) for mono and (channels, samples) otherwise. Field.channels can constrain the count; explicit normalizers select or average channels.

• Parameters:
  • array – The samples. Integer PCM is scaled to [-1, 1] using its bit-depth midpoint (signed int16 is divided by 32768). Floating samples are not rescaled or clipped. A 2-D array is stored channels-first, taking the shorter axis as channels ((samples, channels) from soundfile and Gradio, (channels, samples) from torchaudio), and one channel is mono. Ambiguous one/two-frame mono/stereo arrays keep their channels-first layout. Samples must be real and finite.
  • rate – Samples per second.
• Raises:ValueError – If the array is not 1-D or 2-D, or the rate is not positive.

##################### Examples

>>> Audio(np.zeros(16000, dtype=np.float32), 16000).duration
1.0
>>> stereo = Audio(np.zeros((8000, 2), dtype=np.int16), 8000)
>>> stereo.array.shape, stereo.channels
((2, 8000), 2)
>>> stereo.mono().array.shape           # the reference channel
(8000,)
>>> Audio.read("utt.wav", rate=16000).rate
16000

array : ndarray

property channels : int

1 for a 1-D array.

• Returns: The number of channels.

##################### Examples

>>> Audio(np.zeros((2, 8000), dtype=np.float32), 8000).channels
2

• Type: Count the channels

classmethod coerce(value: Any, rate: int | None) → Audio

Turn whatever a caller holds into an Audio at rate.

• Parameters:
  • value – One of: an Audio; a path, which is read; a (rate, samples) pair, which is what gr.Audio returns; a NumPy array or a torch tensor, taken to be at rate already, because nothing says otherwise.
  • rate – The rate the result is at; None keeps each value at its own rate, for a model that takes any, and then an array or tensor - which carries none - is refused.
• Returns: The audio, resampled when its own rate differs, every channel kept.
• Raises:TypeError – If value is none of those, or carries no rate when none is fixed.

##################### Examples

>>> Audio.coerce("utt.wav", 16000).rate
16000
>>> Audio.coerce((44100, samples), 16000).rate    # from gr.Audio
16000
>>> Audio.coerce(torch.zeros(2, 16000), 16000).channels
2

classmethod concat(pieces: Sequence[Audio]) → Audio

Join consecutive pieces of one signal in time.

• Parameters:pieces – The consecutive pieces, all at one rate with one channel count.
• Returns: One Audio holding the pieces back to back.
• Raises:ValueError – If the pieces differ in rate or in channel count.

##################### Examples

>>> piece = Audio(np.zeros(8000, dtype=np.float32), 8000)
>>> Audio.concat([piece, piece]).duration
2.0

property duration : float

Measure the duration in seconds.

• Returns: The number of samples divided by the sample rate.

##################### Examples

>>> Audio(np.zeros(8000, dtype=np.float32), 8000).duration
1.0

mono(channel: int = 0) → Audio

Return one channel as a 1-D Audio; itself when already mono.

The reference channel, not the mean: a microphone array’s channels differ in delay, and their mean cancels what one channel keeps.

• Parameters:channel – Which channel, 0 by default.
• Returns: The selected channel as a 1-D Audio, or self when mono.

##################### Examples

>>> Audio(np.zeros((2, 8000), dtype=np.float32), 8000).mono().array.shape
(8000,)

multichannel() → Audio

Return this audio as (channels, samples); mono becomes (1, samples).

For a hook that wants one rank whatever came in.

• Returns: The audio as (channels, samples); self when already 2-D.

##################### Examples

>>> Audio(np.zeros(8000, dtype=np.float32), 8000).multichannel().array.shape
(1, 8000)

rate : int

classmethod read(path: str | Path, rate: int | None = None) → Audio

Read an audio file, at its own rate or resampled to rate.

• Parameters:
  • path – Any file soundfile reads (WAV, FLAC, OGG, …).
  • rate – When given, the result is resampled to this rate.
• Returns: The file as float32, every channel kept.

##################### Examples

>>> Audio.read("utt.flac").rate       # whatever the file holds
44100
>>> Audio.read("utt.flac", 16000).rate
16000

resample(rate: int) → Audio

Return this audio at rate; itself when already there.

• Parameters:rate – The target rate in samples per second.
• Returns: A new Audio at rate, every channel resampled, or this one when it already is.

##################### Examples

>>> Audio(np.zeros(16000, dtype=np.float32), 16000).resample(8000).duration
1.0
>>> audio.resample(audio.rate) is audio
True