Most teams that bolt “AI” onto a product end up building the same Rube Goldberg machine: a prompt that begs a chat model to answer in JSON, a regex parser, a retry loop for the responses wrapped in markdown fences, and a validation layer that quietly rejects the rest. It works — until latency spikes, or the model answers a slightly different question, or a downstream service acts on a number nobody can trace. A decision API exists to delete that machine. Instead of generating text and hoping it parses, you send state and a question defined in code, and you get back a typed, schema-valid answer with a calibrated confidence attached. That single shift changes what is possible in production systems, because the output is no longer something you interpret — it is something your code can branch on.
What a Decision API Actually Is — and Why “Reply in JSON” Isn’t One
A decision API is an HTTPS endpoint whose output space is deliberately constrained. You are not asking a model to produce language; you are asking it to resolve a question that already exists in your codebase, against state you supply, and to return a value that conforms to a schema you defined. The practical difference is enormous. A generative call returns a token stream that might contain the answer somewhere inside it. A decision call returns the answer itself: one of your labels, a level on an ordered scale, a probability, a value on a grid, or a coordinate.
State is flexible. It can be plain text, structured JSON, or up to 16 images or video frames in a single request — which means the same endpoint can triage a support ticket, inspect a photo of a damaged pallet, and read a shipping label without three different vendor integrations. The questions themselves live in code, so they are versioned, peer-reviewed in pull requests, unit-tested, and diffable. When someone asks six months later why a specific order was routed to manual review, the answer is a commit, not a screenshot of a prompt.
A concrete implementation of this pattern is DecisionNode, a decision engine and decision API at decisionnode.com that exposes a single endpoint, POST /v1/decide, and returns typed answers in one pass. There is no chain of prompts, no tool-calling loop, no second model to repair the first model’s formatting. One request in, one structured decision out. That one-pass design matters more than it sounds: every additional round trip in an agentic loop is a chance for state to drift and a latency multiplier you cannot control.
The reason teams reach for this instead of hand-written rules is not novelty — it is coverage. Hand-written conditionals handle the cases you thought of. A decision API handles the long tail of messy real-world inputs that would otherwise fall through to a default branch: the ticket with no subject line, the address with three typos, the photo shot at an angle under bad warehouse lighting. You keep the determinism and reviewability of rules while gaining the pattern recognition you cannot express as if/else.
The Typed Primitives: choice, score, truth, number, point, and box
A decision API is only as useful as the answer shapes it can return, because each shape maps to a different kind of downstream action. The six core primitives cover most operational logic.
Choice returns one of the labels you defined — billing, shipping, technical, refund. This is the workhorse for ticket routing and triage, intent classification, and content moderation categories. Score returns a level on an ordered scale, which is what you want for severity, priority, quality grades, or risk tiers where the ranking carries meaning and the exact numeric gap does not. Truth returns a calibrated probability between 0.00 and 1.00 — the right primitive for fraud checks, promo-abuse detection, and any gate where you need a number to compare against a policy threshold rather than a label to trust blindly.
Number returns a value on a grid, useful for quantities, counts, and estimates where free-form generation would invite hallucination. Point and box return locations inside an image, which is what turns the endpoint into a visual inspection tool: where is the defect on the panel, where is the parcel on the conveyor, where is the face in the frame.
The glue across all six is calibrated confidence. Every answer carries it, and calibration is a testable property — an answer reported at 0.80 should be correct roughly 80% of the time across a large sample. That lets you write routing logic that reads like ordinary engineering: act instantly above a threshold, re-check or fall back to a human queue below it. You are no longer choosing between “trust the model” and “block on a human”; you are setting a confidence budget per workflow and measuring it. Boosters extend the primitives for specialized inputs — human age estimation, AI-generated image detection, face detection and recognition, OCR, depth, and segmentation — so the same request shape handles a document scan or a camera frame without a separate pipeline.
Production Realities: Latency, Determinism, and Cost at Scale
A decision API lives or dies by whether it fits inside an existing request budget. The numbers here are the interesting part. DecisionNode offers two models: DecisionNode Flash at roughly 5 ms server-side and 19 ms end to end, and DecisionNode MoE at roughly 9 ms server-side. For context, that is faster than most teams’ existing database round trips — fast enough to sit inline in a checkout flow, a page render, a webhook handler, or a robot’s control loop rather than being pushed into an async queue nobody monitors.
Determinism is the property that makes this debuggable. Same request, same answer. That means decisions can be cached, replayed from a logged payload when a customer disputes an outcome, and asserted against expected values in CI. A support engineer can reproduce the exact decision that flagged an order three weeks ago, because the state and the question are both artifacts you already store. Try doing that with a sampling-based chat model.
Cost shape matters too, because cheap decisions change system design. Pricing is per input token — $0.035 per 1M input tokens on Flash, $0.042 per 1M input tokens on MoE — and output tokens are always free, which is the logical consequence of returning typed values rather than generated prose. Batch discounts and a free daily tier make it practical to backfill a dataset with decisions before you ship, or to run offline re-scoring over historical records. When a decision costs a fraction of a cent, you stop rationing it and start calling it in the places that actually need judgment.
For high-frequency loops, per-request HTTP overhead becomes the bottleneck rather than inference. WebSocket sessions handle workloads above roughly 10 decisions per second — drone and robot control, simulators, continuous machine monitoring on a production line — where the persistent connection removes handshake cost from the control cycle. Official SDKs for TypeScript, Python, Go, and Ruby (plus curl), MCP tools for agent builders, and webhooks for event-driven pipelines mean integration is usually an afternoon, not a quarter. The operational use cases map cleanly onto the primitives: order and shipping decisions, fraud and promo-abuse checks, moderation and gating, warehouse and traffic monitoring, quality control on production lines, and AI agents that decide before they act instead of narrating what they might do.
Kuala Lumpur civil engineer residing in Reykjavik for geothermal start-ups. Noor explains glacier tunneling, Malaysian batik economics, and habit-stacking tactics. She designs snow-resistant hijab clips and ice-skates during brainstorming breaks.
Leave a Reply