# Streaming

> Turn on streaming with on_stream, read the text and the tool call arguments in each stream chunk, and switch OpenAI-compatible clients to deltas only.

You end up with an agent that prints the model's text, and the arguments of its tool calls, while the model is still writing them. Streaming changes only how a completion arrives.

The agent needs the `swiftide-agents` feature flag, and each model client needs its own: `openai`, `anthropic` or `aws-bedrock`. None of them is on by default.

```sh
cargo add swiftide@0.34 --features swiftide-agents,openai
```

## Streaming switch

```rust
agents::Agent::builder()
    .llm(&openai)
    .on_stream(|_agent, response| {
        // We print the message chunk if it exists. Streamed responses also include
        // the full response (without tool calls) in `message` and an `id` to map them to
        // previous chunks for convenience.
        //
        // The agent uses the full assembled response at the end of the stream.
        if let Some(delta) = &response.delta {
            print!(
                "{}",
                delta
                    .message_chunk
                    .as_deref()
                    .map(str::to_string)
                    .unwrap_or_default()
            );
        };
        // If `stream_full` is disabled, response.message() will be the accumulated response
        // response.message()

        Box::pin(async move { Ok(()) })
    })
    // Every message added by the agent will be printed to stdout
    .on_new_message(move |_, msg| {
        let msg = msg.to_string();
        Box::pin(async move {
            println!("\n---\nFinal message:\n {msg}");
            Ok(())
        })
    })
    .limit(5)
    .build()?
    .query("Why is the rust programming language so good?")
    .await?;
```

Source: [`examples/streaming_agents.rs#L24-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/streaming_agents.rs#L24-L58)

`openai` is an `OpenAI` client built with `stream_full(false)`; [Deltas only](#deltas-only) shows it.

Registering `on_stream` turns streaming on. The agent then calls `complete_stream` on the model instead of `complete`, and runs the hook once for every stream chunk. Without a hook, `streaming(true)` on the builder streams as well.

The hook receives `&Agent` and the chunk, a `ChatCompletionResponse`. The example prints `delta.message_chunk`, the text that arrived with this chunk. It never flushes stdout, so the text can arrive in bursts.

The `on_new_message` hook prints every message the agent adds to the message history. For the assistant, that is the assembled message after the stream has ended.

## Stream chunks

Each stream chunk is a whole `ChatCompletionResponse`. With the default settings it carries:

- `id`, the same for every chunk of one completion.
- `message`, the text accumulated so far.
- `tool_calls`, `usage` and `reasoning`, the tool calls rebuilt, the token counts and the reasoning items so far.
- `delta.message_chunk`, the text that arrived with this chunk.
- `delta.tool_calls_chunk`, a map from a per-client index to a `ToolCallAccum` with `id`, `name` and `arguments`. `arguments` holds everything received so far, not only the newest fragment. The map stays on every later chunk of the completion, so remember what you already showed.

The key differs by client. OpenAI chat completions use the tool call's index, the Responses API the output item index. Anthropic and Bedrock use the content block index, which also counts text blocks. Match calls on `id` or `name`, not on the key.

When the stream ends, the agent keeps the last chunk and uses it as the completion. It records that chunk's message in the message history and runs its tool calls. A stream that yields nothing fails with `AgentError::EmptyStream`. A client without its own `complete_stream` sends one chunk with the whole response and no `delta`.

## Tool argument previews

The release-notes-agent showcase streams its draft. The release notes are the `markdown` argument of the `write_changelog` tool, so they arrive as tool call arguments, not as text:

```rust title="examples/release-notes-agent/src/progress.rs"
/// Streams the model's text, and the release notes while they are being drafted.
pub fn stream_draft(builder: &mut AgentBuilder) {
    let preview = Arc::new(Mutex::new(DraftPreview::default()));

    // Registering `on_stream` switches the agent to streaming completions.
    builder.on_stream(move |_agent, response| {
        let Some(delta) = &response.delta else {
            return Box::pin(async { Ok(()) });
        };
        let mut out = stdout().lock();
        if let Some(text) = &delta.message_chunk {
            let _ = write!(out, "{text}");
        }
        // The notes arrive as the (JSON) argument of `write_changelog`; show them as they
        // are written.
        let drafts = delta.tool_calls_chunk.iter().flat_map(HashMap::values);
        for call in drafts.filter(|call| call.name.as_deref() == Some(WRITE_CHANGELOG)) {
            if let (Some(id), Some(args)) = (&call.id, &call.arguments) {
                let mut preview = preview.lock().unwrap_or_else(PoisonError::into_inner);
                if let Some(text) = preview.advance(id, args) {
                    let _ = write!(out, "{text}");
                }
            }
        }
        let _ = out.flush();
        Box::pin(async { Ok(()) })
    });
}
```

Source: [`examples/release-notes-agent/src/progress.rs#L31-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/progress.rs#L31-L58)

The hook writes `message_chunk` to stdout. It then walks the entries of `tool_calls_chunk`, keeps the `write_changelog` call, and hands its id and accumulated arguments to `DraftPreview`. It flushes stdout after every chunk.

[`DraftPreview`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/draft.rs) lives in `draft.rs`. Every chunk repeats the whole argument text, so the preview tracks how much it has shown. It decodes the complete part of the `markdown` value in the growing JSON (JavaScript Object Notation) text, stopping before an unfinished escape sequence. It returns only the text it has not returned before. A new call id starts a fresh preview.

The hook is a `Fn` closure, so it cannot change what it captures, and it must be `Clone`. State that changes from chunk to chunk goes behind an `Arc<Mutex<_>>`, as `preview` does here.

## Captured stream

These lines come from the showcase's `OUTPUT.md`, a real run with `claude-sonnet-5-5` through an OpenAI-compatible gateway. `OUTPUT.md` cuts the draft after its first lines with `...`:

```text
  git_diff ✓ 17 lines
## 0.34.0

This release adds the opt-in `swiftide-agents-contrib` crate and Perl support for tree-sitter. It also fixes duplicated text in OpenAI streaming and resolves security advisories, which removes AWS Console login from the Bedrock client.
...
  → write_changelog <32 lines>
  write_changelog ⏸ waiting for approval
```

The draft appears before the `→ write_changelog` line. `on_stream` runs while the completion arrives, and the `before_tool` hook runs only after it is complete. After the approval, the agent runs the pending tool call without a new completion, so the draft streams once.

## Deltas only

By default, the OpenAI-compatible clients clone the accumulated response, which grows with the answer, into every chunk. Turn that off with `stream_full(false)`:

```rust
let openai = swiftide::integrations::openai::OpenAI::builder()
    .default_embed_model("text-embeddings-3-small")
    .default_prompt_model("gpt-4o-mini")
    // Only streams the delta, leave this out to stream the full response
    .stream_full(false)
    .build()?;
```

Source: [`examples/streaming_agents.rs#L11-L16`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/streaming_agents.rs#L11-L16)

The last chunk is still the complete response, so the agent behaves the same. `delta.tool_calls_chunk` carries the same accumulated arguments with either setting, so the showcase's preview works with the default too. [Deltas only](/models/overview/#deltas-only) lists the clients that have the setting and the fields each chunk drops.

## Gotchas

- Before 0.34, the `GenericOpenAI` chat completions stream (OpenAI, Ollama, Groq, Gemini, OpenRouter and Dashscope) repeated the previous `message_chunk` on tool call fragments, the stop and usage frames, and the final chunk, so a hook that printed it printed text more than once. [#1216](https://github.com/bosun-ai/swiftide/pull/1216) fixed this.
- The fix covers only the chat completions stream. The OpenAI Responses API (`use_responses_api(true)`, which the showcase's `--responses-api` flag turns on), `Anthropic` and `AwsBedrock` still keep the previous `message_chunk` on tool call events. With them, both examples print the last text again for every tool call event. Their chunks carry the accumulated `message`, so printing the part of `message` beyond what you already printed avoids repeats.
- The agent awaits the hook for every chunk before it reads the next one. A slow hook slows the stream. Hand slow work to a channel or a spawned task.
- An error returned from `on_stream` is logged with `tracing` and does not stop the agent.
- `on_stream` receives `&Agent`, so it cannot stop the agent or change its tools. Do that from `after_completion`.
- Rust's stdout is line buffered. `print!` without a newline may show nothing until a line ends or the buffer fills, so flush after each chunk. The first example does not flush; the showcase does.

## Reference

- [`AgentBuilder::on_stream`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.on_stream)
- [`OnStreamFn`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/trait.OnStreamFn.html)
- [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html)
- [`ChatCompletionResponseDelta`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponseDelta.html)
- [`ToolCallAccum`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCallAccum.html)
- [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html)
- [`GenericOpenAIBuilder::stream_full`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.stream_full)
- [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html)
- [Feature flags](/reference/feature-flags/)

## Next

[MCP tools](/agents/mcp/)