VISIONROUTE

Guide · Pillar: Choose

Choose the approach, not just the model.

Choose the solution family before any model name. If the output is free text or a judgment over the whole scene, start with a prompted vision-language model (VLM). If you need locations or counts of fixed classes under tight latency, start with a detector. If a specialist API covers exactly your task, test it first. Custom training is a later decision that data and economics must justify. Then test your top two families on about 100 representative images.

Published August 31, 2026 · Pillar: Choose · By the VisionRoute team

Definitions

Model vs stack

A computer-vision model turns an image into a prediction. A vision stack is everything that turns that prediction into a product decision: the model or API, prompts or training data, preprocessing, output schema, thresholds, review rules, hosting, and monitoring. This guide chooses the family at the center of the stack — the rest of the stack still has to be designed around it.

The four solution families

Prompted VLMs (a general multimodal model steered by instructions), pretrained or open-vocabulary detectors (models that locate objects, promptable by class name), specialist APIs (hosted services for one fixed task), and custom-trained models (you supply labeled data and own the result). Most production stacks are one of these, or two composed.

Who this guide is for

Software teams adding a narrow visual decision to a product — compliance checks, attribute extraction, classification and routing — without a computer-vision specialist. It assumes still images, not video, robotics, medical, or biometric work.

What the tree returns

A starting hypothesis, deliberately. The tree orders what to try first; it does not certify what will work. Roughly 100 representative images and an agreed definition of correct settle the question — documentation does not.

The decision tree

Four questions, in order, before any vendor name

Answer with your 12-month volume and your hardest images in mind, not the demo. Terms are defined in the glossary.

Assumes a still-image decision inside a software product and no dedicated vision team. A specialist team may weigh custom training earlier.
StepQuestion to askIf yesIf no
1Is the output a small, fixed set of answers — labels, fields, or pass/fail verdicts?Continue to step 2.Prompted vision-language model (VLM) first. Free-text description, variable attributes, and judgment against written rules sit where language models are strong. Enforce an output schema from day one.
2Does a specialist API already cover exactly this task (OCR, barcodes, explicit content, face blurring)?Specialist API first. It is the fastest path when the task matches — but test it on your hard cases before committing; "exactly" is doing a lot of work in this question.Continue to step 3.
3Do you need object locations or counts, not just an image-level answer?Pretrained or open-vocabulary detector first. Fine-tune only if your evaluation shows specific slices failing.Continue to step 4.
4Will per-image cost or a strict latency budget bind at your 12-month production volume?Small custom classifier (or a fine-tuned open model) — if you can label roughly 500–5,000 images. Low marginal cost and stable latency are what you are buying.Prompted VLM first. Revisit this step when volume grows or the latency budget hardens — the answer is allowed to change.

Worked example

Walking the tree: marketplace listing photos

A concrete, illustrative walk-through — the marketplace is fictional and nothing here is a measured result. The value is seeing each question do its work.

  1. The product need, in product terms

    A marketplace lets sellers upload listing photos. Three rules decide acceptance: plain background, no watermark or overlay text, product fully in frame. About 5,000 photos arrive per day, checked asynchronously at upload.
  2. Step 1 — fixed answers? Yes.

    Each photo needs three pass/fail verdicts. The output set is small and fixed, so the free-text branch does not apply.
  3. Step 2 — an exact specialist API? No.

    Text-overlay detection exists as a service, but no single API covers all three rules as this marketplace defines them (its own background and framing standards). Stitching three vendors for one decision adds cost and three failure modes.
  4. Step 3 — locations needed? No.

    The verdicts are image-level. Framing can be judged without boxes ("does the product touch the image border?"), so a detector is not required by the output.
  5. Step 4 — cost or latency binding? Not yet.

    Checks run asynchronously, so seconds-scale latency is acceptable. At 5,000 images/day, per-image API pricing is material but not decisive. The tree lands on: prompted VLM, one call per photo, returning JSON verdicts with a confidence per rule.
  6. The first experiment

    Collect 100 representative photos — including about 30 known rejects and 20 borderline cases — and measure per-rule false accepts and false rejects against the team’s definitions. If volume grows 10×, step 4 flips: move the highest-volume rule to a small classifier and keep the VLM for the long tail.

When this answer changes

A hard latency floor appears

If the check moves into a synchronous user flow with a strict budget, seconds-scale VLM calls stop being acceptable and step 4 dominates: detector or small classifier, served close to your backend.

Images cannot leave your infrastructure

Privacy or residency constraints remove hosted APIs from steps 2 and 4. The same tree still works, restricted to self-hosted open models — but budget for serving and upgrades you now own.

The rules change often

If the definition of acceptable changes monthly, prompt edits beat retraining cycles. That weighs the tree toward a VLM even when a classifier would be cheaper per image.

Volume grows an order of magnitude

Per-call pricing that was noise at 5,000 images/day can dominate at 50,000. Re-run step 4 at every volume milestone; migrating one high-volume rule to a small model is a normal second step, not a failure of the first.

Limitations

What this tree cannot decide

  • It covers narrow decisions on still images. Video, robotics, edge hardware, medical, and biometric applications have constraints this tree does not model.
  • It returns a starting family, not a verdict. Two families that both survive the tree are separated by testing on your images — not by further reading.
  • It contains no benchmark numbers by design: latency and cost claims here are directional, and the honest figures are the ones you measure under your own serving path and volume.

In short

Pick a computer-vision approach with four ordered questions: Is the output a fixed set of answers? Does a specialist API cover exactly this task? Are locations or counts required? Will cost or latency bind at production volume? Start with the family the tree returns, and let roughly 100 representative images make the final call.

Related: VLM vs object detection · custom model vs hosted API · feasibility testing