Blog

Up to 70% lower speech startup latency than vLLM-Omni. Qwen3-TTS and Kokoro are now available in Photon.

Meet Moondream Parakeet Redux and Parakeet Ultra: speech-to-text models for local transcription in 25 languages, on CPUs and GPUs.

Photon 2.2 adds five NVIDIA GPUs, Ampere to Blackwell. Parakeet runs 3.1x to 4.8x NeMo's throughput. Qwen3-ASR beats vLLM in 23 of 24 benchmarked configurations.

Photon 2.1 adds automatic speech recognition for Whisper, Qwen3-ASR, and Parakeet, beating each model's reference engine in every matched test, and brings the full model catalog to NVIDIA B200.

A 13× throughput collapse, minute-long cold boots, memory that never comes back: why datacenter inference engines break on live video, with receipts.

Photon compiles models into GPU programs optimized for the chip and the job. The first release supports Moondream, Qwen, and Gemma on NVIDIA H100, and wins every matched throughput test against vLLM and SGLang.

Moondream 3.1 launches with best-in-class benchmark scores, a new fine-tuning motion that transfers to your tasks, and a partnership with Cloudflare.

Photon 1.3.0 makes Moondream faster across NVIDIA, Mac, and Windows, runs finetunes on far more hardware, fixes an accuracy issue on older GPUs — and running Moondream locally is now completely free.

Photon, Moondream's inference engine, achieves near-realtime VLM inference (~33ms on NVIDIA B200). This is a peek into how it delivers up to 35% higher decode throughput by optimizing how the GPU works.