Moondream 3.1: Beyond Benchmarks

Today we're excited to announce the launch of Moondream 3.1 and a partnership with Cloudflare. The benchmarks on this new model are strong. Many are best-in-class, and we'll get to them. But your use cases aren't benchmarks. This launch is about making Moondream the best model for what you need.

In this post:

• Benchmarks
• Our partnership with Cloudflare
• New license
• New cloud pricing

But first, obligatory benchmarks

All figures below are like-for-like against the same baselines under the same settings: Qwen3.5 9B, SAM 3, LocateAnything, and Gemma 4 12B. Detection metrics are F1@0.5. The bold value marks the best score in each chart. An "n/a" means the model cannot do that task. SAM 3 and LocateAnything are detection/segmentation specialists, not general VLMs. Setup details in the benchmark methodology footer.

Open-vocabulary detectiondetect() · F1@0.5

Dense detectiondetect() · crowded scenes · F1@0.5

Referring detectiondetect() · phrase-grounded · F1@0.5

Aerial detectiondetect() · oriented / small objects · F1@0.5

Countingpoint() · % correct

Document understandingquery() · % / ANLS

State of the art on COCO and ODinW-13, on DOTA-v2, on ChartQA and PixMo-Count. But look at the dense detection numbers. Crowded shelves, crowds, bins of parts: that's where 3.1 pulls the furthest ahead, both over other models and over the last Moondream. It's the biggest jump in this release. Full methodology in the footer.

The scores are good. How we got them is the better story.

What benchmarks don't measure

A benchmark is a fixed set of questions and images that someone else assembled, scored against answers they chose in advance. That makes it good for one thing: comparing models on equal footing, which is why we publish ours.

But your vision tasks never look like the benchmarks. Every customer who builds on Moondream brings a task specific to them, and they're delightfully different, even weird:

A factory points a camera at a weld seam and asks: is this weld clean? The answer that matters isn't a paragraph. It's "No. Porosity at top edge."

A warehouse runs detect: crushed pallets across aisle footage and gets a bounding box on the one pallet to pull before it takes a rack down.

A broadcast team runs point: the ball on a live feed and gets a coordinate that tracks the ball and the player on it, frame after frame, so the camera operator doesn't have to.

There is no WeldBench. So with 3.1 we spent less effort chasing benchmark numbers and more on the workflow that adapts Moondream to a specific task.

How we made it

No special research pipeline produced these numbers. We used Lens, our fine-tuning product: the same hosted API customers use to tune Moondream with SFT and RL.

We treated each benchmark like a customer task. We used Lens to improve Moondream 3.1 on each task, using the same API our customers do. These fine-tuned LoRAs achieved state-of-the-art results on the key benchmarks listed above.

Then one extra step, the only one a customer doesn't need: on-policy distillation, which folds what every LoRA learned back into the base weights. That's why the downloaded model scores at the top with no tuning at all.

The distilled base lands just behind the individual LoRAs, so if you only care about one task you can stop early: train the LoRA on Lens and ship it.

These numbers came out of the same workflow we ship to customers. We just ran it on public benchmarks instead of private data, and every step is available to you in Lens.

The right answer, on time

On a live feed, latency is a hard limit. A camera watching a checkout line, a weld seam, or a loading dock can't wait several seconds for each result, so speed counts as much as accuracy. Here's how Moondream compares on throughput against the models benchmarked above:

The vertical axis averages each model's scores on the benchmarks above. The horizontal axis is throughput: average requests (inferences) per second, measured across our full benchmark suite on the same setup. You want the top-right corner: high score, high throughput. Moondream is the only model there, at 34 requests per second. That's seven times faster than the next fastest model here, and over twenty times faster than the closest model on accuracy.

Part of that is the architecture: 2B active parameters means less work per frame. Part of it is Photon, our free inference engine, which squeezes real-time vision out of whatever hardware you point it at. Together they put a state-of-the-art vision model on a live feed within reach, without standing up a dedicated infrastructure project.

Announcing our partnership with Cloudflare

Moondream already runs on-prem, on desktops, and in our cloud (Fal too!). Today we're adding one more option, and it's the second announcement of the day: we're partnering with Cloudflare to bring Moondream 3.1 to Workers AI, their serverless global inference platform, as @cf/moondream/moondream3.1-9B-A2B.

Vision workloads are large and latency-sensitive. Frames arrive constantly, and every millisecond of round trip to a distant data center eats the speed advantage of running a small model in the first place. So take the fastest model (Moondream) and run it on the network with the most edge presence (Cloudflare). Requests run close to your users and your cameras, and responses stream: in Cloudflare's testing, first tokens came back in roughly 20 to 30 ms, with point at ~145 ms and detect at ~160 ms end to end on a simple image.

That's fast enough to call the model inline while handling a request instead of pushing work to a background queue: moderate an upload before it's stored, drive a live overlay from a video frame, pull fields from a document during a form submission, or let an agent read a screenshot and pick its next step in a single turn.

Workers AI supports query, caption, point, and detect, and Cloudflare offers Moondream at the same price as Moondream Cloud: $0.30 per million input tokens and $1.00 per million output tokens.

New license

We've adopted a new license for Moondream 3.1: the Moondream Model License. It's the same underlying theme as before, but hopefully simpler to understand. The weights are open and source-available: use them commercially, self-host them, fine-tune, quantize, and redistribute them, no strings on any of that. The single restriction is that you can't turn around and offer general-purpose Moondream inference or fine-tuning as a hosted service to others. A narrow, domain-specific app built on Moondream is fine. In short: build whatever you want on it, just don't resell Moondream itself as an API.

New cloud pricing

Moondream's inference efficiency keeps going up, and we're passing the savings on to you. Cloud pricing for Moondream 3.1 drops from $2.50 per million output tokens to $1.00 per million. Input tokens stay at $0.30 per million. The Batch API keeps its 50% discount on top of that.

Ready when you are

Moondream 3.1 is live everywhere, today. Try it in the playground, download the weights on Hugging Face, make your first call on Moondream Cloud, or run it at the edge on Workers AI.

And it launches with full support across the platform: Lens can fine-tune it on your task right now, and Photon runs it on whatever hardware you have, from a rack of H100s to the laptop in your bag.

Moondream 3.1. State of the art on the benchmarks. State of the art on your tasks.

Benchmark methodology

Models were evaluated on a single H100 SXM5 with a batch size of 16. The highest accuracy and reasoning settings were chosen for each model. LocateAnything was evaluated in slow mode, maximizing accuracy. vLLM was used to evaluate LocateAnything, Qwen3.5, and Gemma 4. Photon was used to evaluate Moondream. Hugging Face Transformers was used to evaluate SAM 3.