What Is Structured Output and How Does It Work?

It is requiring the model to return an answer in a specific structured format such as JSON, as opposed to free text. This way, the response is parsed reliably by the software rather than guessed at.

You can do that, but prompting alone will not ensure valid output, since the model can still produce invalid JSON. The more reliable way to enforce validity is through function calling or constrained decoding.

Constrained decoding constrains the possible tokens the model can generate, ensuring that the output adheres to the schema. Since invalid tokens cannot be produced, the output is guaranteed to be structurally correct.

Tool and function calling depend on structured output, because the model produces arguments that conform to a tool's schema. It is that structure that enables an agent to call tools and APIs correctly, rather than passing free text.

In cases where the reply is meant to be read by a person and not processed by a program. Conversational answers are more natural written freely, without any particular structure; structure is needed only when software consumes the output.