Structured prompts

This recipe asks an OpenAI model for three questions about Rust and reads them from a Vec<String> inside your own struct. A second call gets the same answer through a trait object. The code comes from the structured-prompt example in the swiftide repository.

Setup

Add Swiftide with the openai feature flag. Add schemars, serde and serde_json for the response type, and tokio and anyhow for main:

cargo add swiftide@0.34 --features openai
cargo add schemars@1 serde serde_json --features serde/derive
cargo add anyhow
cargo add tokio --features macros,rt-multi-thread

Set OPENAI_API_KEY and run the example from the root of the swiftide repository:

cargo run -p swiftide-examples --example structured-prompt

Imports and client

The example needs no extra setup inside the swiftide repository. Your own binary starts like this:

use anyhow::Result;
use schemars::JsonSchema;
use serde::Deserialize;
use swiftide::{integrations, traits::StructuredPrompt as _};

#[tokio::main]
async fn main() -> Result<()> {
    let client = integrations::openai::OpenAI::builder()
        .default_prompt_model("gpt-5-mini")
        .build()?;

Source: snippets/src/cookbook/structured_prompts.rs#L5-L14

structured_prompt::<T> is a method of the StructuredPrompt trait. The import StructuredPrompt as _ brings the method into scope without the name. Leave it out and the call fails to compile with no method named structured_prompt.

OpenAI is a type alias of GenericOpenAI for the OpenAI endpoint. A structured prompt uses the default prompt model, so set one with default_prompt_model. Without it the call fails with LanguageModelError::PermanentError("Model not set").

Response type

// Note that deny unknown fields is required. If you get an error on 'additionalProperties' to
// be required, and false, this is what is missing.
#[derive(Deserialize, JsonSchema, Serialize, Debug)]
#[serde(deny_unknown_fields)]
struct MyResponse {
    questions: Vec<String>,
}

Source: examples/structured_prompt.rs#L15-L21

Swiftide turns the struct into a JavaScript Object Notation (JSON) Schema with schemars::schema_for! and sends that schema with the prompt. OpenAI checks the schema in strict mode, which is why the struct carries #[serde(deny_unknown_fields)]. Schema rules lists what strict mode rejects, including Option<T> fields.

Typed call

let response = client
    .structured_prompt::<MyResponse>(
        "List three interesting questions about the Rust programming language.".into(),
    )
    .await?;

println!("Response: {:?}", response.questions);

Source: examples/structured_prompt.rs#L25-L31

The method takes a Prompt. .into() turns a &'static str or a String into one.

The client sends the rendered prompt as one user message, with no system message. It then deserializes the JSON the model returns into MyResponse. A reply that does not fit the type fails with LanguageModelError::PermanentError.

Swiftide never retries a structured_prompt call, including transient and rate-limit errors. LanguageModelWithBackOff does not implement StructuredPrompt, so you call the inner client and handle retries yourself. Errors, backoff, usage and tokens covers the decorator.

Templated prompts

The docs-qa showcase grades answers with a structured prompt. It renders a template and wraps the call in its own retry helper:

let contexts = answer.contexts().collect::<Vec<_>>().join("\n---\n");
let prompt = Prompt::from(JUDGE_PROMPT)
    .with_context_value("question", answer.question())
    .with_context_value("ground_truth", ground_truth)
    .with_context_value("answer", answer.text())
    .with_context_value("contexts", contexts);

// The JSON schema of `Judgement` is sent along; the reply is parsed into it
let judgement: Judgement =
    retry_transient(|| self.llm.structured_prompt(prompt.clone())).await?;

Source: examples/docs-qa/src/judge.rs#L154-L163

Prompt::from takes the template, and each with_context_value fills one variable. Prompts and templates covers the syntax. The prompt is cloned on every attempt because structured_prompt takes it by value.

The response type nests structs, and each one carries #[serde(deny_unknown_fields)]:

#[derive(Debug, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
struct Judgement {
    reference_claims: Vec<ReferenceClaim>,
    answer_claims: Vec<AnswerClaim>,
}

#[derive(Debug, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
struct ReferenceClaim {
    // Never read; asking for the claim first makes the model commit to it before judging it.
    #[expect(dead_code)]
    claim: String,
    in_answer: bool,
    in_context: bool,
}

#[derive(Debug, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
struct AnswerClaim {
    #[expect(dead_code)]
    claim: String,
    supported_by_context: bool,
}

Source: examples/docs-qa/src/judge.rs#L91-L114

Strict mode applies to every object in the schema, not only the outer one. A nested struct without the attribute makes OpenAI reject the request.

Trait objects

// Because we use generics, structured_prompt is not dyn safe. However, there is an
// alternative:

let client: Box<dyn DynStructuredPrompt> = Box::new(client);

let response: serde_json::Value = client
    .structured_prompt_dyn(
        "List three interesting questions about the Rust programming language.".into(),
        schemars::schema_for!(MyResponse),
    )
    .await?;

let parsed: MyResponse = serde_json::from_value(response)?;

println!("Response: {:?}", parsed);

Source: examples/structured_prompt.rs#L35-L49

structured_prompt is generic over T, so StructuredPrompt is not dyn safe. You cannot hold a Box<dyn StructuredPrompt>.

DynStructuredPrompt is the object-safe variant. Its structured_prompt_dyn takes the prompt and a schemars::Schema, and returns a serde_json::Value. You pass the schema and decode the value yourself.

Use the trait object when the client is chosen at runtime, for example OpenAI in one deployment and Amazon Web Services (AWS) Bedrock in another.

Variations

Other providers

The OpenAI-compatible clients and AwsBedrock support structured prompts, each behind its own feature flag, and Anthropic does not. Provider support lists the clients and their flags.

Responses API

With use_responses_api(true) on the OpenAI builder, structured_prompt goes through the Responses application programming interface (API) instead of Chat Completions. The schema rules stay the same. OpenAI Responses API and reasoning shows that client.

Agents

An agent talks to its model through chat completion, not structured_prompt. For a typed final answer, replace the default stop tool with StopWithArgs, behind the swiftide-agents feature flag; Agents shows how. Call without_default_stop_tool() or register it with add_tool, because tools() keeps the default stop tool and drops yours.

Reference

Describing images