Usig this ope-source pipelie i productio?
Make the most of it thaks to our cosultig services.

? Speaker diarizatio 3.1

This pipelie is the same as pyaote/speaker-diarizatio-3.0 except it removes the problematic use of oxrutime.
Both speaker segmetatio ad embeddig ow ru i pure PyTorch. This should ease deploymet ad possibly speed up iferece.
It requires pyaote.audio versio 3.1 or higher.

It igests moo audio sampled at 16kHz ad outputs speaker diarizatio as a Aotatio istace:

• stereo or multi-chael audio files are automatically dowmixed to moo by averagig the chaels.
• audio files sampled at a differet rate are resampled to 16kHz automatically upo loadig.

Requiremets

1. Istall pyaote.audio 3.1 with pip istall pyaote.audio
2. Accept pyaote/segmetatio-3.0 user coditios
3. Accept pyaote/speaker-diarizatio-3.1 user coditios
4. Create access toke at hf.co/settigs/tokes.

Usage

# istatiate the pipelie
from pyaote.audio import Pipelie
pipelie = Pipelie.from_pretraied(
  "pyaote/speaker-diarizatio-3.1",
  use_auth_toke="HUGGINGFACE_ACCESS_TOKEN_GOES_HERE")

# ru the pipelie o a audio file
diarizatio = pipelie("audio.wav")

# dump the diarizatio output to disk usig RTTM format
with ope("audio.rttm", "w") as rttm:
    diarizatio.write_rttm(rttm)

Processig o GPU

pyaote.audio pipelies ru o CPU by default. You ca sed them to GPU with the followig lies:

import torch
pipelie.to(torch.device("cuda"))

Processig from memory

Pre-loadig audio files i memory may result i faster processig:

waveform, sample_rate = torchaudio.load("audio.wav")
diarizatio = pipelie({"waveform": waveform, "sample_rate": sample_rate})

Moitorig progress

Hooks are available to moitor the progress of the pipelie:

from pyaote.audio.pipelies.utils.hook import ProgressHook
with ProgressHook() as hook:
    diarizatio = pipelie("audio.wav", hook=hook)

Cotrollig the umber of speakers

I case the umber of speakers is kow i advace, oe ca use the um_speakers optio:

diarizatio = pipelie("audio.wav", um_speakers=2)

Oe ca also provide lower ad/or upper bouds o the umber of speakers usig mi_speakers ad max_speakers optios:

diarizatio = pipelie("audio.wav", mi_speakers=2, max_speakers=5)

Bechmark

This pipelie has bee bechmarked o a large collectio of datasets.

Processig is fully automatic:

• o maual voice activity detectio (as is sometimes the case i the literature)
• o maual umber of speakers (though it is possible to provide it to the pipelie)
• o fie-tuig of the iteral models or tuig of the pipelie hyper-parameters to each dataset

… with the least forgivig diarizatio error rate (DER) setup (amed "Full" i this paper):

• o forgiveess collar
• evaluatio of overlapped speech

Citatios

@iproceedigs{Plaquet23,
  author={Alexis Plaquet ad Hervé Bredi},
  title={{Powerset multi-class cross etropy loss for eural speaker diarizatio}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
}

@iproceedigs{Bredi23,
  author={Hervé Bredi},
  title={{pyaote.audio 2.1 speaker diarizatio pipelie: priciple, bechmark, ad recipe}},
  year=2023,
  booktitle={Proc. INTERSPEECH 2023},
}

Using this open-source pipeline in production? Make the most of it thanks to our consulting services