Engineering

A 13× throughput collapse, minute-long cold boots, memory that never comes back: why datacenter inference engines break on live video, with receipts.

Photon, Moondream's inference engine, achieves near-realtime VLM inference (~33ms on NVIDIA B200). This is a peek into how it delivers up to 35% higher decode throughput by optimizing how the GPU works.