OpenAI ChatCompletion JSON Mode

A OpenAI ChatCompletion JSON Mode is an API Mode that constrains OpenAI's Chat Completions API to generate outputs that strictly conform to the JSON format.

• Context:
  • It can (typically) be activated by setting the `response_format` parameter to `{"type": "json_object"}` in the API request, ensuring that the model's response is a valid JSON object.
  • It can (often) be used in conjunction with specific instructions within the system message to direct the model to produce output in JSON format, enhancing the reliability of structured data generation.
  • It can (often) require explicit inclusion of JSON-related instructions in the conversation context to prevent errors or excessive whitespace generation, with the API enforcing this requirement by checking for the presence of "JSON" in the context.
  • It can (typically) result in outputs that may be partially formed JSON objects if the generation exceeds the specified `max_tokens` limit or the overall token capacity for a conversation, necessitating careful checking of the `finish_reason` before parsing.
  • It can (often) not guarantee that the output adheres to any specific schema, only ensuring that the output is valid JSON that can be parsed without errors, allowing for flexibility in handling diverse response structures.
  • ...
• Example(s):
  • A request to the Chat Completions API using `gpt-3.5-turbo-0125` with `response_format` set to `{"type": "json_object"}` and a system message instructing the model to output JSON. The response includes a JSON formatted string containing the text: `{"winner": "Los Angeles Dodgers"}`, which provides structured information about the winner of the 2020 World Series.
  • ...
• Counter-Example(s):
  • Standard text output mode in the Chat Completions API without specifying JSON format, resulting in plain text responses.
  • Use of the `response_format` parameter in a non-API context or without specifying it as `{"type": "json_object"}`, leading to outputs not constrained to the JSON format.
• See: API Mode, Chat Completions API, System Message, JSON.

---

---

References

2024

• https://platform.openai.com/docs/guides/text-generation/json-mode
  • QUOTE: A common way to use Chat Completions is to instruct the model to always return a JSON object that makes sense for your use case, by specifying this in the system message. While this does work in some cases, occasionally the models may generate output that does not parse to valid JSON objects.

    To prevent these errors and improve model performance, when calling gpt-4-turbo-preview or gpt-3.5-turbo-0125, you can set response_format to { "type": "json_object" } to enable JSON mode. When JSON mode is enabled, the model is constrained to only generate strings that parse into valid JSON object.

---