OpenAI Responses API and reasoning

This recipe gives you an OpenAI client that sends every kind of request through the OpenAI Responses application programming interface (API): prompts, structured prompts, chat completions, streams and tool calls. A second recipe gives you an agent that prints the reasoning summary of a reasoning model. Both recipes are runnable examples in the swiftide repository: responses_api.rs and responses_api_reasoning.rs.

Setup

Add Swiftide with the openai and swiftide-agents feature flags. The first example also streams, defines a tool argument type and parses tool arguments, so it needs futures-util, schemars, serde and serde_json.

cargo add swiftide@0.34 --features openai,swiftide-agents
cargo add futures-util schemars@1 serde_json
cargo add serde --features derive
cargo add tokio --features full
cargo add anyhow

The examples are on Rust edition 2024. The agent snippet below uses a let-chain, which does not compile on edition 2021.

Set OPENAI_API_KEY and run the examples from the root of the swiftide repository:

export OPENAI_API_KEY=sk-...
cargo run -p swiftide-examples --example responses-api
cargo run -p swiftide-examples --example responses-api-reasoning

Imports and types

The anchored excerpts from responses_api.rs leave out its imports and its two argument types. Here they are:

use anyhow::{Context, Result};
use futures_util::StreamExt as _;
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
use swiftide::{
    chat_completion::{ChatCompletionRequest, ChatMessage, ToolOutput, errors::ToolError},
    integrations::openai::{OpenAI, Options},
    traits::{AgentContext, ChatCompletion, SimplePrompt, StructuredPrompt},
};

/// The type `structured_prompt` deserializes the model's answer into.
#[derive(Debug, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
struct WeatherSummary {
    description: String,
}

/// The single parameter of the echo tool.
#[derive(Debug, Serialize, Deserialize, JsonSchema)]
#[serde(deny_unknown_fields)]
struct EchoArgs {
    message: String,
}

Source: snippets/src/cookbook/responses_api.rs#L6-L28

ChatCompletion brings complete and complete_stream into scope. StreamExt gives you next() on the stream. anyhow::Context gives you .context(...) in the tool call code.

Client

let openai = OpenAI::builder()
    .default_prompt_model("gpt-4.1-mini")
    .default_options(Options::builder().temperature(0.2))
    .use_responses_api(true)
    .build()?;

Source: examples/responses_api.rs#L49-L53

use_responses_api(true) is the only change from a Chat Completions client. The flag defaults to false. With it set, complete, complete_stream, prompt and structured_prompt all go through the Responses API. Embeddings still use the embeddings endpoint.

default_options takes anything that converts into Options, so you can pass an OptionsBuilder without calling build(). On the Responses path, the client drops some options instead of sending them. It logs a warning for seed and presence_penalty. It drops user, also set by for_end_user, without a warning. It skips metadata that is not a flat object, with a warning.

Prompts

let greeting = openai
    .prompt("Say hello in one short sentence".into())
    .await?;
println!("Prompt result: {greeting}");

let structured: WeatherSummary = openai
    .structured_prompt("Summarise today's weather in Amsterdam as JSON".into())
    .await?;
println!("Structured result: {structured:?}");

Source: examples/responses_api.rs#L57-L65

prompt returns the model’s text. structured_prompt returns a WeatherSummary, built from the JavaScript Object Notation (JSON) Schema that schemars derives for the type. The struct carries #[serde(deny_unknown_fields)] because OpenAI’s strict mode requires additionalProperties: false on every object. The schema rules list the other constraints. SimplePrompt, StructuredPrompt and ChatCompletion from swiftide::traits must be in scope, and futures_util::StreamExt for the stream.

Chat completion

let chat_request = ChatCompletionRequest::builder()
    .messages(vec![
        ChatMessage::new_system("You are a concise assistant."),
        ChatMessage::new_user("Share one fun fact about Amsterdam."),
    ])
    .build()?;

let completion = openai.complete(&chat_request).await?;
println!(
    "Complete result: {}",
    completion.message().unwrap_or("<no message>")
);

Source: examples/responses_api.rs#L69-L80

A ChatCompletionRequest holds the messages and, optionally, tool specs. build() returns a Result. complete returns a ChatCompletionResponse, and message() gives you the assistant text if there is any.

Streaming

let mut stream = openai.complete_stream(&chat_request).await;
print!("Streaming result: ");
let mut streamed_message = String::new();
while let Some(chunk) = stream.next().await {
    let chunk = chunk?;
    if let Some(delta) = chunk
        .delta
        .as_ref()
        .and_then(|delta| delta.message_chunk.as_deref())
    {
        print!("{delta}");
        std::io::stdout().flush().ok();
    }

    if let Some(message) = chunk.message() {
        streamed_message = message.to_string();
    }
}
println!();
if streamed_message.is_empty() {
    println!("Full streamed result: <no message>");
} else {
    println!("Full streamed result: {streamed_message}");
}

Source: examples/responses_api.rs#L84-L107

complete_stream returns a stream of ChatCompletionResponse values. The new text of each chunk is in delta.message_chunk. The client’s stream_full setting defaults to true, so every chunk also carries the message accumulated so far in message(). The example prints the deltas and keeps the last accumulated message. With stream_full(false) on the builder, every chunk but the last carries only the delta. The last one still holds the complete response, including usage, so the same logic works.

Tool calls

/// Minimal echo tool used to demonstrate tool calling with the Responses API.
/// The macro implements the `Tool` trait, derives the JSON schema, and generates
/// a helper constructor (`echo_tool()`) that returns a boxed tool ready for use.
#[swiftide::tool(
    description = "Echos the provided message back to the caller.",
    param(name = "payload", description = "Text to echo back")
)]
async fn echo_tool(
    _context: &dyn AgentContext,
    payload: EchoArgs,
) -> Result<ToolOutput, ToolError> {
    Ok(ToolOutput::text(format!("Echo: {}", payload.message)))
}

Source: examples/responses_api.rs#L27-L39

#[swiftide::tool] turns the function into a tool. It generates a constructor named after the function, echo_tool(), which returns a boxed tool. It also generates an argument struct named after the tool in PascalCase with an Args suffix, here EchoToolArgs, with one field per parameter. The tool name defaults to the function name, and name = "..." in the attribute changes both. The only parameter, payload, is the EchoArgs struct from the imports excerpt.

let tool_request = ChatCompletionRequest::builder()
    .messages(vec![
        ChatMessage::new_system(
            "You are a precise assistant. Use available tools before replying directly.",
        ),
        ChatMessage::new_user(
            "Call the echo tool with the phrase \"Hello Responses API\" and then summarise the result.",
        ),
    ])
    .tool(echo_tool())
    .build()?;

let tool_completion = openai.complete(&tool_request).await?;

if let Some(tool_call) = tool_completion
    .tool_calls()
    .and_then(|calls| calls.first())
    .cloned()
{
    println!(
        "Assistant requested tool `{}` with arguments {}",
        tool_call.name(),
        tool_call.args().unwrap_or("<missing arguments>")
    );

    let args_json = tool_call
        .args()
        .context("echo tool call missing arguments")?;
    let args: EchoToolArgs = serde_json::from_str(args_json)?;
    let tool_output = format!("Echo: {}", args.payload.message);

    let mut follow_up_messages = tool_request.messages().to_vec();
    follow_up_messages.push(ChatMessage::new_assistant(
        None::<String>,
        Some(vec![tool_call.clone()]),
    ));
    follow_up_messages.push(ChatMessage::new_tool_output(
        tool_call.clone(),
        ToolOutput::text(tool_output),
    ));

    let follow_up_request = ChatCompletionRequest::builder()
        .messages(follow_up_messages)
        .tool(echo_tool())
        .build()?;

    let final_completion = openai.complete(&follow_up_request).await?;
    println!(
        "Final response after tool call: {}",
        final_completion.message().unwrap_or("<no message>")
    );
} else {
    println!(
        "Assistant responded without tool calls: {}",
        tool_completion.message().unwrap_or("<no message>")
    );
}

Source: examples/responses_api.rs#L111-L167

ChatCompletionRequest::builder().tool(...) adds only the tool’s spec to the request. Nothing runs the tool for you. The example reads the first tool call, parses its arguments into EchoToolArgs and builds the output itself. It then appends the assistant’s tool call with ChatMessage::new_assistant and the result with ChatMessage::new_tool_output, and sends a follow-up request. An agent runs this loop for you, as the next recipe shows.

Reasoning summaries

// Reasoning models require the Responses API. Enabling reasoning effort also asks for a
// summary and encrypted reasoning content (enabled by default). If your OpenAI org is not
// verified for reasoning access, summaries may be absent. Disable with
// `reasoning_features(false)` if desired.
let openai = OpenAI::builder()
    .default_prompt_model("o3-mini")
    .default_options(Options::builder().reasoning_effort(ReasoningEffort::Low))
    .use_responses_api(true)
    .build()?;

Source: examples/responses_api_reasoning.rs#L34-L42

Setting reasoning_effort on the options turns on reasoning for the request. With the Responses API, the client then also asks for an automatic reasoning summary and for the encrypted reasoning content. reasoning_features(false) on the options turns both off. If your OpenAI organization is not verified for reasoning access, the response may hold no summary.

fn reasoning_summary(reasoning: Option<&[ReasoningItem]>) -> Option<String> {
    let summary = reasoning
        .unwrap_or(&[])
        .iter()
        .flat_map(|item| item.summary.iter())
        .cloned()
        .collect::<Vec<_>>()
        .join("\n");

    if summary.is_empty() {
        None
    } else {
        Some(summary)
    }
}

Source: examples/responses_api_reasoning.rs#L10-L24

The helper joins the summaries of all ReasoningItem values into one string.

let mut agent = Agent::builder()
    .llm(&openai)
    .on_new_message(|_, message| {
        if let ChatMessage::Assistant(content, _) = message
            && let Some(content) = content.as_deref()
        {
            println!("Assistant: {content}");
        }
        Box::pin(async move { Ok(()) })
    })
    .after_completion(|_, response| {
        if let Some(summary) = reasoning_summary(response.reasoning.as_deref()) {
            println!("Reasoning summary:\n{summary}");
        }

        let has_encrypted = response
            .reasoning
            .as_ref()
            .is_some_and(|items| items.iter().any(|item| item.encrypted_content.is_some()));
        println!("Encrypted reasoning content present: {has_encrypted}");
        Box::pin(async move { Ok(()) })
    })
    .build()?;

Source: examples/responses_api_reasoning.rs#L46-L68

Two hooks print what happens. after_completion receives the ChatCompletionResponse before the agent takes the reasoning out of it, so response.reasoning is still there. on_new_message fires for the messages the agent adds itself: reasoning items, assistant messages and tool outputs. The system prompt and your query go into the context directly and skip it. The example prints only assistant messages.

agent
    .query("Explain why the sky is blue in one short paragraph.")
    .await?;

Source: examples/responses_api_reasoning.rs#L72-L74

Reasoning items explains how the agent stores the items and replays them on the next request.

Variations

ReasoningEffort has the variants None, Minimal, Low, Medium, High and Xhigh. Which ones a model accepts depends on the model.
Without an agent, call complete on a client with use_responses_api(true) and reasoning_effort set, and read response.reasoning yourself. On the Chat Completions path, response.reasoning stays None.
The on_new_message hook uses a let-chain, which needs edition = "2024". On edition 2021, nest the two if let statements.
Responses API and reasoning covers how other providers return reasoning.

Reference

Agents on AWS Bedrock