Introducing Moondream Parakeet Redux and Parakeet Ultra

We've spent a lot of time making the Moondream VLM smaller and faster. A model that fits on your laptop, or costs less to run on a server, opens up more things you can build with it. The same is true for speech.

Today we're introducing Moondream Parakeet Redux and Moondream Parakeet Ultra, two speech-to-text models based on NVIDIA's Parakeet. Both support automatic speech recognition (ASR) in 25 languages.

Parakeet Redux compresses the model weights from 1.2 GB to 178 MB, and is designed for fast inference on CPUs and Macs. Parakeet Ultra keeps the full-precision weights and improves transcription accuracy through further training. It's built for running on GPUs.

Making Parakeet smaller

Running a model requires repeatedly moving its weights from memory to the processor. On CPUs and Apple Silicon, memory bandwidth and not computation is often the limiter of inference speed. Smaller weights reduce that traffic, so compressing the model can make transcription faster as well as reduce its memory footprint.

For Parakeet Redux, we reduced the encoder's weights to just three possible values: −1, 0, and +1. This is called ternary quantization. Those weights take much less space to store, and Photon, our inference engine, has specialized kernels that compute directly from the packed representation.

Here's how fast Parakeet Redux runs in Photon:

On the AMD CPU, Parakeet Redux delivers 2.5 times the throughput of parakeet.cpp, the fastest alternative we measured on the same machine. See the model card for comparisons with other engines and full test details.

One user has already moved transcription to the CPU to free up GPU memory for LLMs. In their tests, Parakeet Redux on an AMD Ryzen 9 9950X3D CPU beat 8-bit parakeet.cpp on an NVIDIA RTX PRO 6000 GPU for shorter clips and nearly matched it for longer ones.

Of course, making a model this small comes with a question: how much accuracy do you lose?

We measure accuracy using word error rate, which counts missed, incorrect, and extra words against a reference transcript. Lower is better.

On our seven English test sets, average word error rate rises from 6.26% to 6.55%. Across the 25-language FLEURS evaluation, Parakeet Redux actually improves the average, from 11.62% to 10.56%. The biggest tradeoff is background noise, where Parakeet Redux makes more mistakes than the original. We'd use Parakeet Ultra when noise is a concern and a GPU is available.

Making Parakeet more accurate

With Parakeet Ultra, we kept the original model's size and focused on improving its transcriptions. Post-training of the model brought down the average error rate in every evaluation group we tested, including English, multilingual speech, business recordings, and speech mixed with background noise.

Here's how the two models compare with the original, measured by word error rate:

Parakeet Ultra's multilingual result represents 18% lower average word error rate than the original on that evaluation.

Knowing where to pause

Transcribing a long recording means breaking it into smaller pieces. Where you make those cuts matters: splitting in the middle of a word can make it harder to get the transcription right.

We built voice activity detection (VAD) into both models. It identifies where speech is happening, and Photon uses it to find pauses where it can split the recording into chunks of at most 30 seconds. You can pass in the whole recording without setting up a separate VAD model or cutting the audio yourself.

On our test of 11 complete TED-LIUM talks, each 10–20 minutes long, Parakeet Redux brings word error rate down from the original's 2.71% to 2.51%. Parakeet Ultra gets it down to 1.94%, a 28% reduction.

Give them a try

Both models support files, live audio, and timestamps through the Moondream Python package:

pip install --upgrade "moondream>=2.4.1"

Transcribe a recording

Here's all you need to run Parakeet Redux on your CPU. WAV, MP3, FLAC, and M4A files are supported, among other formats. Long recordings are split automatically.

import moondream as md

with md.photon("moondream/parakeet-redux", device="cpu") as speech:
    result = speech.transcribe(audio="meeting.m4a")
    print(result["text"])

For the Apple GPU, use device="mps". The same examples work with Parakeet Ultra on an NVIDIA GPU by changing the model to "moondream/parakeet-ultra" and the device to "cuda". Both models choose the language automatically and preserve punctuation and capitalization.

Find when each word was spoken

Use timestamps="word" to get a transcript with sentence segments and word timings. All times are in seconds.

import moondream as md

with md.photon("moondream/parakeet-redux", device="cpu") as speech:
    result = speech.transcribe(audio="interview.wav", timestamps="word")

    for segment in result["segments"]:
        print(segment["start"], segment["end"], segment["text"])
        for word in segment["words"]:
            print(word["start"], word["end"], word["word"])

Use timestamps="segment" for sentence timings alone, or "none" if you only need the text.

Read the transcript as it develops

You don't have to wait for a whole recording to finish. With stream=True, Photon returns updated transcripts as it works through the audio:

import moondream as md

with md.photon("moondream/parakeet-redux", device="cpu") as speech:
    updates = speech.transcribe(
        audio="meeting.m4a",
        timestamps="segment",
        stream=True,
    )
    for update in updates:
        # Each update replaces the previous transcript; don't append it.
        print(update["text"])

    final = updates.result()
    print("Final transcript:", final["text"])

Transcribe live audio

You can also feed audio in as it arrives from a microphone or a network stream. Pass an asynchronous iterator of mono audio chunks to atranscribe, along with their sample rate. Each chunk should be a nonempty, one-dimensional NumPy array or CPU Torch tensor containing raw PCM samples.

This function connects your application's audio source to the transcriber:

import moondream as md


async def transcribe_live(audio_chunks, sample_rate):
    with md.photon("moondream/parakeet-redux", device="cpu") as speech:
        updates = await speech.atranscribe(
            audio=audio_chunks,
            sample_rate=sample_rate,
            timestamps="word",
            stream=True,
        )
        async for update in updates:
            print(update["text"])

        final = await updates.aresult()
        return final

Call it with await transcribe_live(audio_chunks, sample_rate=48_000) for a source producing 48 kHz audio. End the iterator when the recording stops to receive the final transcript. Live previews begin after four seconds of audio and are scheduled every two seconds after that; earlier text can change as more context arrives. As with files, each update is a replacement snapshot.

Work with part of a file or audio already in memory

You can select a time range without editing the source file, or pass encoded audio bytes directly:

from pathlib import Path
import moondream as md

with md.photon("moondream/parakeet-redux", device="cpu") as speech:
    # Transcribe the second minute of a recording.
    clip = speech.transcribe(
        audio="interview.mp3",
        clip_start_seconds=60,
        clip_end_seconds=120,
    )
    print(clip["text"])

    # Encoded audio bytes, such as a file uploaded to your application.
    result = speech.transcribe(audio=Path("speech.wav").read_bytes())
    print(result["text"])

For raw audio already in memory, pass a one-dimensional mono NumPy array or CPU Torch tensor as audio and supply sample_rate, for example speech.transcribe(audio=samples, sample_rate=48_000). Encoded files and bytes carry their own sample rate. Clip ranges apply to recorded audio, not live streams.

Weights are available under the CC-BY-4.0 license, same as NVIDIA's original model. The Parakeet Redux and Parakeet Ultra model cards have more detailed benchmarks and examples. Give them a try, and let us know what you're building.

Happy Moondreamin'.