Structured outputs: JSON you can trust

The short version

Structured outputs make the model return valid JSON matching your schema — every time, guaranteed by the provider, not by hoping. If you've ever written a regex to rescue a malformed response at 2am, this is the feature that ends that era. For any LLM output that feeds code rather than eyeballs, it's non-negotiable.

How the guarantee works

Providers implement this with constrained decoding: the model's token probabilities are masked at each step so only tokens that keep the output schema-valid can be sampled. It's not a post-processing check — invalid JSON is impossible by construction, because the model literally cannot emit the characters that would break it.

You define the schema (JSON Schema, Zod, or a Pydantic model depending on your stack), and the API enforces it. Enums become hard constraints. Required fields are always present. Nested objects nest correctly.

Patterns that work

• Extraction. Pull entities, dates, and amounts from messy text into typed records. The classic, and still the highest-volume use case.
• Classification with evidence. Return {label, confidence, rationale} — the rationale keeps humans in the loop while the label drives automation.
• Multi-step agents. Each tool call's arguments are a structured output. Constrain them and an entire class of "the agent passed garbage to the tool" bugs disappears.
• Routing. One small model classifies intent into an enum; the router dispatches. Cheap, fast, and now type-safe.

Sharp edges

• Schemas must be reasonable. Enormous schemas with dozens of optional fields degrade quality — the model spreads attention thin. Split big extractions into stages.
• "Valid" isn't "correct." The JSON will parse; the values can still be wrong. You still need evals on the content, not just the shape.
• Refusals get structured too. When the model declines, you get a schema-valid refusal object, not an exception. Handle that case explicitly instead of assuming success.
• Not all providers are equal. Constraint support varies — verify enum handling and nested required fields on your provider before relying on them.

Rule of thumb: if an LLM's output is consumed by code, constrain it. Parsing free text with regex is a bug you're choosing to ship later.