OpenAI’s Whisper: 7 must-know libraries and add-ons built on top of it

Community improvements over OpenAI’s Whisper

Photo by Bogomil Mihaylov on Unsplash

In Sept 2022, OpenAI released Whisper, the world’s most accurate speech recognition (ASR) that can transcribe and translate speech audio from 97 languages!

While it touted state-of-the-art accuracies in speech-to-text of many languages, it lacked several key aspects of speech-to-text systems like word level time stamps, speaker diarisation, etc.

Here are the top 7 community enhancements on top of Whisper that address some of the shortcomings!

1. WhisperX — Word-level time stamps with Whisper

Whisper’s transcription accuracy was great but lacked fine-grained word-level timestamps.

This repo combines Whisper with Phoneme-based ASR to deliver word-level timestamps using forced alignment!

GitHub - m-bain/whisperX: WhisperX: Automatic Speech Recognition with Word-level Timestamps (&…

2. Buzz

Transcribe and translate audio on your personal computer using OpenAI’s Whisper.