Photon 2.2 adds five NVIDIA GPUs: A10/A10G, A100, RTX 3090, L4, and RTX PRO 6000 Blackwell. With H100 and B200, Photon now runs on seven GPUs across four architectures: Ampere, Ada, Hopper, and Blackwell.
We benchmarked speech on L4, A10/A10G, and A100 at concurrency 1, 2, 4, and 8. Photon beats the reference engine in 35 of 36 configurations and ties in one.
- Parakeet TDT 0.6B v3: 3.1x to 4.8x the throughput of NVIDIA NeMo
- Qwen3-ASR 0.6B: 1.1x to 2.2x the throughput of qwen-asr on vLLM
- Qwen3-ASR 1.7B: 1.0x to 1.2x the throughput of qwen-asr on vLLM
Vision, language, and Whisper numbers on the new GPUs will follow.
Also in 2.2
- Whisper transcription now runs on L4 and RTX 3090.
- Lower transcription latency and less host overhead for Qwen3-ASR and Parakeet. The largest gains are at concurrency 1.
- Lower latency and higher throughput for Moondream and Qwen models.
Speech benchmarks
Photon throughput divided by reference-engine throughput. Higher is better. C is concurrent requests.
| GPU | Model | C=1 | C=8 |
|---|---|---|---|
| L4 | Parakeet TDT 0.6B v3 | 4.83x | 4.04x |
| L4 | Qwen3-ASR 0.6B | 1.06x | 1.41x |
| L4 | Qwen3-ASR 1.7B | 1.04x | 1.14x |
| A10/A10G | Parakeet TDT 0.6B v3 | 3.58x | 3.13x |
| A10/A10G | Qwen3-ASR 0.6B | 1.06x | 1.19x |
| A10/A10G | Qwen3-ASR 1.7B | 1.12x | 1.11x |
| A100 SXM 80GB | Parakeet TDT 0.6B v3 | 3.44x | 3.74x |
| A100 SXM 80GB | Qwen3-ASR 0.6B | 1.45x | 2.17x |
| A100 SXM 80GB | Qwen3-ASR 1.7B | tied | 1.23x |
Reference engines: NVIDIA NeMo for Parakeet, qwen-asr on vLLM for Qwen3-ASR.
Parakeet gains are large on all three GPUs. Qwen3-ASR gains are smaller. On smaller GPUs, some workloads already saturate compute or memory bandwidth, which leaves less launch and scheduling overhead for Photon to remove.
The compiler keeps compounding
Photon compiles each model into one GPU program, a megakernel. All model execution happens inside it.
When we started, getting one megakernel to compile at all was hard. Making it beat other engines was harder. But each compiler improvement applies to every model and every GPU it targets, so coverage compounds:
- Photon 2.0, August 3: Qwen, Gemma, and Moondream on H100. That took months.
- Photon 2.1, August 26: B200, plus Whisper, Qwen3-ASR, and Parakeet. Three weeks.
- Photon 2.2, September 9: five more GPUs. Two weeks.
Try Photon 2.2
pip install --upgrade moondreamPhoton is free. Local inference needs no API key.
Have a model, GPU, or realtime workload that needs to be faster? Email contact@moondream.ai with the model, the GPU, and the latency or throughput you need. We will compile it on Photon and send you numbers.