Photon performance.
Measured throughput, latency, startup, memory, power, and energy use on H100. Results include matched vLLM and SGLang runs, with setup and reproduction commands below. The launch post provides more context.
Runtime performance, not model quality. The tested Qwen and Gemma configurations score below 5% ChartQA exact-match and are included as matched runtime controls.
- Engines
- Photon 2.0 · vLLM 0.25.1 · SGLang 0.5.3
- Workload
- ChartQA 512-request slice
- Concurrency
- 1, 2, 4, 8
- Updated
- Aug 3, 2026
- H100 SXM5 80GB HBM3 · driver 590.42.06 · CUDA 13.1
- Identical 512-request stream replayed to each engine pair
- SGLang request identity, prompt, ground truth, and input-token counts match Photon in all 24 cells
- SGLang 0.5.3 does not support Moondream, so its comparison covers Qwen 3.5 and Gemma 4 only
- Matched model revisions across engines · one physical GPU per engine pair · sequential runs
- Prefix cache disabled on compared engines
- Photon measured in-process; vLLM and SGLang measured over HTTP — their native deployment forms
- Photon output-token totals ranged from 9.6% lower to 5.2% higher than SGLang across cells; throughput is completed requests per second
- SGLang resource telemetry was not captured and is not included
Throughput by concurrency
52 matched runtime pairs across 7 models and concurrency 1, 2, 4, 8. Values show Photon throughput relative to each matched runtime.
python -m moondream.bench --suite h100-chartqa-512 --engines photon,vllm,sglang --seed 20260803 --chart throughputRequest completion latency
Client-observed p50, p95, and p99 at concurrency 1.
python -m moondream.bench --suite h100-chartqa-512 --engines photon,vllm,sglang --seed 20260803 --chart latencyTime to first response
Total time from process start to first response. Results cover seven models for Photon and vLLM, and six for SGLang.
Photon
vLLM
SGLangShared scale across models · lower is better36.0s42.1s34.8s101s77.8s35.7s98.6s79.4s37.4s105s79.4s38.8s112s79.4s67.6s180s72.6s74.0s185s74.3spython -m moondream.bench --suite h100-chartqa-512 --engines photon,vllm,sglang --seed 20260803 --chart cold-startMemory, power, and energy per token
Measured GPU memory, power, temperature, and energy per output token.
| Model | VRAM | Power | Temp | J / output token |
|---|---|---|---|---|
| Moondream 3 | 21.4 GB | 342 W | 61 °C | 0.42 |
| Qwen 3.5 0.8B | 12.8 GB | 385 W | 64 °C | 0.55 |
| Qwen 3.5 2B | 14.6 GB | 391 W | 65 °C | 0.61 |
| Qwen 3.5 4B | 17.9 GB | 402 W | 66 °C | 0.72 |
| Qwen 3.5 9B | 24.2 GB | 418 W | 67 °C | 0.98 |
| Gemma 4 E2B | 15.1 GB | 388 W | 64 °C | 0.63 |
| Gemma 4 E4B | 18.4 GB | 397 W | 65 °C | 0.75 |
python -m moondream.bench --suite h100-chartqa-512 --engines photon --seed 20260803 --chart resourcesTest Photon on your stack.
Use the published command as a starting point, or talk to us about your model, hardware, and latency target.