This is the full developer documentation for Swiftide # LLM applications in Rust: swift to build, built to ship. > An agent harness, typed task graphs and streaming RAG pipelines that work together, with retries, streaming, human approval, caching and tracing already there. [Get started](/start/what-is-swiftide/) Add swiftide to a project ```sh cargo add swiftide --features swiftide-agents,openai ``` [API reference](https://docs.rs/swiftide/0.34/swiftide/) · [GitHub](https://github.com/bosun-ai/swiftide) · [Discord](https://discord.gg/3jjXYen9UY) · [llms.txt](/llms.txt) · [For coding agents](/start/for-coding-agents/) $release-notes-agent --since v0.33.0 --version 0.34.0 ``` → git_log_since v0.33.0 git_log_since 346 lines → git_diff swiftide-integrations/Cargo.toml 3b1ad22e^..3b1ad22e → git_diff …/src/chat_completion/chat_completion_response.rs 90bcadc7^..90bcadc7 → git_diff swiftide-agents-contrib/Cargo.toml 71fa4587^..71fa4587 git_diff 15 lines git_diff 72 lines git_diff 75 lines → write_changelog <32 lines> write_changelog ⏸ waiting for approval Write these release notes? [y/N] y write_changelog wrote RELEASE_NOTES.md → stop (version 0.34.0: highlights 2, breaking 1, fixes 1) stop ■ done ``` examples/release-notes-agent/src/agent.rs ```rust /// Builds the agent. All tools run through `executor`, whose working directory is the repo. pub fn build( llm: &(impl ChatCompletion + Clone + 'static), executor: Arc, task: &ReleaseTask, github_tools: Option, ) -> Result { let mut builder = Agent::builder(); builder .llm(llm) .context(DefaultContext::from_executor(executor)) .system_prompt(task.system_prompt(github_tools.is_some())) // Our own `stop` tool replaces the default one; call this before `tools`. .without_default_stop_tool() .tools([ git_log_since(), git_diff(), read_file(), // Drafts that describe a change twice go back to the model before anyone reviews them. OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(&task.notes_file))) .into(), ReleaseNotes::stop_tool().into(), ]) .limit(STEP_LIMIT); if let Some(github_tools) = github_tools { builder.add_toolbox(github_tools); } progress::report_tool_calls(&mut builder); progress::stream_draft(&mut builder); Ok(builder.build()?) } ``` Source: [`examples/release-notes-agent/src/agent.rs#L104-L136`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/agent.rs#L104-L136) ## [Release notes agent](/showcases/release-notes-agent/) Reads git, drafts the notes, and writes them only after you say yes. `→ stop (version 0.34.0: highlights 2, breaking 1, fixes 1)` [Agents guide →](/agents/overview/) examples/workspace-architect/src/graph.rs ```rust /// Builds the task: `WorkspaceRoot` in, the written ARCHITECTURE.md out. pub fn architect_task( llm: &L, settings: ArchitectSettings, progress: &Progress, ) -> Result, TaskError> where L: ChatCompletion + SimplePrompt + Clone + 'static, { let mut task = Task::new(); let discover = task.register_node_fn({ let progress = progress.clone(); move |root: &WorkspaceRoot| { let members = workspace::discover(root, settings.selection)?; progress.step("discover", format_args!("{members} in {root}")); Ok::<_, workspace::DiscoveryError>(members) } }); let analyse = task.register_node(CrateAnalyst::new(llm.clone(), progress.clone())); let join = task.register_node_fn({ let progress = progress.clone(); move |summaries: &JoinInput| { progress.joined(summaries.len()); architecture_prompt(summaries) } }); // `Arc` is a task node out of the box: `Prompt` in, `String` out. let render = task.register_node(Arc::new(llm.clone()) as Arc); let review = task.register_node(ApprovalGate::new(settings.pre_approved, progress.clone())); let write = task.register_node_async_fn({ let output = settings.output; let progress = progress.clone(); move |draft: &ArchitectureDraft| { let output = output.clone(); let progress = progress.clone(); Box::pin(async move { let written = draft.write_to(&output).await?; progress.step( "write", format_args!("{} ({} bytes)", written.path.display(), written.bytes), ); Ok::<_, std::io::Error>(written) }) } }); task.starts_with(discover); // One branch per crate, all evaluating the same analyst node with a different input. task.register_transition(discover, move |members: WorkspaceMembers| { let mut crates = members.crates.into_iter(); let Some(first) = crates.next() else { return Transition::error("the workspace has no crates to document"); }; crates .fold(Transition::fan_out(&analyse, first), |fan_out, member| { fan_out.and(&analyse, member) }) .concurrency_model(ConcurrencyModel::Parallel) .join_with(join.join()) })?; // Each branch hands its `CrateSummary` to the join; the join runs once all have arrived. task.register_transition(analyse, join.join())?; task.register_transition(join, { let progress = progress.clone(); move |prompt| { progress.step("render", "drafting ARCHITECTURE.md"); render.transitions_with(prompt) } })?; task.register_transition(render, move |markdown: String| { review.transitions_with(ArchitectureDraft::from_model_output(&markdown)) })?; task.register_transition(review, move |decision| match decision { Review::Approved(draft) => write.transitions_with(draft).into(), Review::AwaitingApproval => Transition::pause(), })?; task.register_transition(write, task.transitions_to_finish())?; Ok(task) } ``` Source: [`examples/workspace-architect/src/graph.rs#L35-L134`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L35-L134) ## [Workspace architect](/showcases/workspace-architect/) One agent per crate, run in parallel and joined into ARCHITECTURE.md. `[ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x)` [Tasks guide →](/tasks/overview/) examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` Source: [`examples/ask-your-codebase/src/index.rs#L89-L115`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L89-L115) ## [Ask your codebase](/showcases/ask-your-codebase/) Indexes a codebase as a stream; unchanged files hit the cache. `Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s` [Pipelines guide →](/indexing/overview/) ## The same parts, three places [Section titled “The same parts, three places”](#the-same-parts-three-places) The same client, executor or store works in an agent, a task graph and a pipeline, so you wire a value once. * **Values**\ an `OpenAI` client, a `LocalExecutor`, a `Qdrant` store * **Traits**\ `ChatCompletion`, `SimplePrompt`, `ToolExecutor`, `Persist`, `Retrieve` * **Runtimes**\ [an agent](/agents/overview/), [a task graph](/tasks/overview/), [a pipeline](/indexing/overview/) ## Integrations * [openai](/integrations/openai/) * [anthropic](/integrations/anthropic/) * [aws-bedrock](/integrations/aws-bedrock/) * [ollama](/integrations/openai-compatible/) * [fastembed](/integrations/fastembed/) * [qdrant](/integrations/qdrant/) * [lancedb](/integrations/lancedb/) * [pgvector](/integrations/pgvector/) * [redis](/indexing/caching/) * [tree-sitter](/integrations/tree-sitter/) * [mcp](/agents/mcp/) * [All 24 →](/integrations/overview/) # Using Swiftide with coding agents > Give Claude Code, Cursor, Codex and other coding agents the Swiftide skill, Context7, llms.txt, per-page Markdown, rustdoc JSON and an AGENTS.md block pinned to the right version. Your coding agent, such as Claude Code, Cursor or Codex, can write Swiftide code that compiles against 0.34. Swiftide is pre-1.0, and minor releases break APIs. Release 0.33 changed system prompts, hook signatures, command output and the AWS Bedrock module. A model that learned an older release writes code that no longer compiles. Each source below gives the agent the current API, and the [AGENTS.md block](#agentsmd-block) at the end pins the rules in your project. | Source | What the agent gets | | -------------- | --------------------------------------------------------- | | Skill | Rules, gotchas and compiled snippets for the current API | | Context7 | Search over the examples, the skill and the crate sources | | llms.txt files | This site as plain text | | Page Markdown | One page as Markdown | | API docs | Exact signatures, as HTML or rustdoc JSON | | Examples | Programs that compile | ## Release status [Section titled “Release status”](#release-status) The skill, `context7.json` and the four showcase crates ship with the 0.34 release. These docs pin the release commit, and the skill describes the APIs at that commit. ## Skill [Section titled “Skill”](#skill) The Swiftide skill is a set of instructions that the agent loads when the task needs it. Install it with the `skills` CLI from npm (needs Node.js): ```sh npx skills add bosun-ai/swiftide ``` Without Node.js, copy the [`skills/swiftide`](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/skills/swiftide) folder from the pinned commit into the skills directory of your agent. `SKILL.md` covers feature flags, OpenAI-compatible endpoints, tools, hooks, structured stops, human approval, MCP (Model Context Protocol) toolboxes, tasks and pipelines, and ends with a checklist. Four reference files go deeper: `agents.md`, `tasks.md`, `pipelines.md` and `integrations.md`. Every snippet in them is copied from compiled code in the swiftide repository, and names its source file. The agent picks the skill up when a Rust project depends on swiftide or one of its crates, when you ask for an agent, task graph or RAG (retrieval-augmented generation) pipeline in Rust, or when it fixes a missing feature flag. ## Context7 [Section titled “Context7”](#context7) [Context7](https://context7.com) indexes the swiftide repository under the library ID `/bosun-ai/swiftide`. With the Context7 MCP server installed, ask the agent to look up Swiftide in Context7, or give it the ID. Context7 indexes the default branch, which can hold APIs that are not released yet. Check what it returns against the version in your `Cargo.lock`. The repository’s [`context7.json`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/context7.json) limits the index to `examples`, `skills` and the source folders of the crates, and adds nine rules that the AGENTS.md block below also covers. ## llms.txt files [Section titled “llms.txt files”](#llmstxt-files) The site publishes plain-text versions of itself for language models. `/llms.txt` states the version these docs describe, 0.34, and links the others. | File | Contents | | ------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------- | | [/llms.txt](https://swiftide.rs/llms.txt) | An index with the pinned version and links to the files below | | [/llms-full.txt](https://swiftide.rs/llms-full.txt) | Every page in one file | | [/llms-small.txt](https://swiftide.rs/llms-small.txt) | The start pages and core guides, without recipes, showcases, integrations or reference | | [/\_llms-txt/agents.txt](https://swiftide.rs/_llms-txt/agents.txt) | Agents: tools, hooks and executors | | [/\_llms-txt/tasks.txt](https://swiftide.rs/_llms-txt/tasks.txt) | Typed task graphs | | [/\_llms-txt/indexing-and-querying.txt](https://swiftide.rs/_llms-txt/indexing-and-querying.txt) | Indexing and query pipelines | | [/\_llms-txt/integrations.txt](https://swiftide.rs/_llms-txt/integrations.txt) | Model providers, vector stores and other integrations | | [/\_llms-txt/showcases.txt](https://swiftide.rs/_llms-txt/showcases.txt) | The four showcase programs and their output | The full and small files start with the home page, followed by the Start pages. None of the files include the changelog. A section file is the smallest useful input when the agent works on one part, such as a task graph. ## Page Markdown [Section titled “Page Markdown”](#page-markdown) Every page is also served as Markdown at its URL plus `.md`, for example . Snippets are expanded into code, so the agent sees the same compiled Rust as you do. The buttons next to each page title do the same by hand. **Copy page** copies the page as Markdown, for pasting into a chat. **View as Markdown** opens the `.md` version. **Open in ChatGPT** and **Open in Claude** start a conversation about the page. ## API docs [Section titled “API docs”](#api-docs) The [API docs on docs.rs](https://docs.rs/swiftide/0.34/swiftide/index.html) are the most reliable source of exact signatures. docs.rs builds swiftide with every feature flag on and labels each item with the flag it needs. docs.rs also serves rustdoc JSON. The `swiftide` crate only re-exports the other crates, so its JSON holds re-export stubs and no signatures. Fetch the JSON of the crate that defines the items, such as `swiftide-agents`, `swiftide-tasks`, `swiftide-core`, `swiftide-indexing`, `swiftide-query` or `swiftide-integrations`. docs.rs sends it zstd-compressed, so decompress it before the agent reads it: ```sh curl -sSL https://docs.rs/crate/swiftide-agents/0.34/json | zstd -d > swiftide-agents.json ``` Pin the version in the URL so the agent does not read a newer release. `latest` in place of the version gives the newest release instead. ## Examples [Section titled “Examples”](#examples) The [examples directory](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples) holds the single-file examples and the four [showcases](/showcases/release-notes-agent/). Each showcase has a README, an `OUTPUT.md` with output from a real run, and source that this site excerpts. From the root of a checkout of the pinned commit, an agent can run them. `hello-agents` uses Gemini and needs `GEMINI_API_KEY`. `ask-your-codebase` needs `OPENAI_API_KEY`, and its first run downloads an embedding model of about 600 MB. ```sh cargo run -p swiftide-examples --example hello-agents cargo run -p ask-your-codebase -- index swiftide-agents ``` ## AGENTS.md block [Section titled “AGENTS.md block”](#agentsmd-block) Paste this block into the `AGENTS.md` or `CLAUDE.md` at the root of your project. It comes from the skill’s gotchas and the rules in `context7.json`. Change the version line when you upgrade. ```md ## Swiftide This project uses swiftide 0.34 (Rust 1.97 or newer, edition 2024). Swiftide is pre-1.0 and minor releases break APIs. Check the version in Cargo.lock, and read https://swiftide.rs/llms.txt or https://docs.rs/swiftide/0.34/swiftide/ before trusting examples written for older releases. - Feature flags: default features are only `macros` (`#[swiftide::tool]`, `#[derive(Tool)]`). Agents need `swiftide-agents`, which also turns on `swiftide-tasks`. Each integration needs its own flag (`openai`, `anthropic`, `gemini`, `qdrant`, `fastembed`, `lancedb`, `duckdb`, `redb`, `mcp`, ...). An unresolved import under `swiftide::agents` or `swiftide::integrations` means a missing flag. `all` turns on only `qdrant`, `redis`, `tree-sitter`, `openai`, `fastembed`, `scraping`, `aws-bedrock`, `groq`, `ollama` and `pgvector`; everything else, including agents and tasks, needs its own flag. - With `default-features = false`, add `macros` back. Tools and structured stops also need `schemars` (with `derive`) and `serde`. - TLS needs no setup: the HTTP clients use rustls with their own crypto provider. Do not add `rustls` or call `install_default()` unless the app wants a specific provider. - Tools: `#[swiftide::tool]` on an async fn that takes `&dyn AgentContext` and returns `Result`. The doc comment is the tool description and a `# Arguments` list describes the arguments. Use `#[derive(Tool)]` for a tool with state. Return `ToolOutput::fail(..)` for errors the model should see and recover from. - Run commands with `context.executor().exec_cmd(&Command::shell(..))`; `AgentContext::exec_cmd` is deprecated. `CommandOutput` records stdout and stderr chunks: `stdout()` and `stderr()` iterate over the `Bytes` chunks of one stream, and `into_string_lossy()` merges both streams into one `String` in the order they arrived. - The system prompt is a `Prompt`: `.system_prompt(..)` takes anything `Into`, such as a `&str`. `SystemPrompt` no longer exists. An agent without a system prompt sends none. - Hooks are closures that return `Box::pin(async move { Ok(()) })`; clone captured values before the `async move` block. Most hooks get `&mut Agent`; `on_new_message` and `on_stream` get `&Agent`. A hook that returns an error is logged, not propagated. - Structured output: call `without_default_stop_tool()` before `.tools(..)`, add `StopWithArgs::with_parameters_schema(schema_for!(T))`, and read `StopReason::RequestedByTool(_, Some(payload))`. `.tools(..)` replaces any earlier list. - `StopReason::NoNewMessages` means the model answered without a tool call or `.limit(n)` was reached. `limit` counts completions per `query`. - Human approval: wrap a tool in `ApprovalRequired::new(tool)`. On `StopReason::FeedbackRequired`, call `agent.context().feedback_received(&tool_call, &ToolFeedback::approved())`, then `agent.run()`. - Tasks: import from `swiftide::tasks`; `swiftide::agents::tasks` is deprecated. Use `Task`, `TaskNode` and `Transition` instead of chaining agents by hand. Give every node one transition and end with `task.transitions_to_finish()`. A fan-out (`Transition::fan_out(..).and(..).join_with(join.join())`) ends in a join node. `task.run(..)` returns `TaskRunOutcome::Completed(output)` or `TaskRunOutcome::Paused`; continue a paused task with `task.resume()`. - OpenAI-compatible endpoints go through the `openai` flag: `OpenAI::builder().api_base("http://127.0.0.1:8080/v1").api_key(key)`. The URL includes the version prefix. `.client(..)` replaces both; whichever is called last wins. - AWS Bedrock is `swiftide::integrations::aws_bedrock_v2` (the Converse API), behind the `aws-bedrock` flag. - MCP tools are named `:`. OpenAI rejects `:` in tool names, so rename them. - Pipelines: use the same embedding model for indexing and querying, and set the vector store's `vector_size` to that model's dimensions. Wrap LLM clients used during indexing in `LanguageModelWithBackOff`. - Before finishing: run `cargo check` with the exact features. ``` ## Verification [Section titled “Verification”](#verification) Ask the agent to finish with the checks from the skill. `cargo check` with the exact feature flags in `Cargo.toml` catches a missing flag as an unresolved import (error E0432). ## Reference [Section titled “Reference”](#reference) * [`swiftide` crate root](https://docs.rs/swiftide/0.34/swiftide/index.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) and [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) and [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) and [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html), [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) and [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) and [`OpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAIBuilder.html) * [`aws_bedrock_v2`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/index.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`skills/swiftide/SKILL.md`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/skills/swiftide/SKILL.md) at the pinned commit * [Feature flags](/reference/feature-flags/) * [Upgrading](/reference/upgrading/) * [TLS](/reference/feature-flags/#tls) ## Next [Section titled “Next”](#next) [Agents overview](/agents/overview/) # Installation > Add swiftide with the feature flags you need and tokio, set your API keys, and run a first program that prints one answer from a model. You end with a Rust binary on swiftide 0.34 that prints one answer from a model. The page also lists the environment variables, feature flags and local servers the rest of the site uses. ## Dependencies [Section titled “Dependencies”](#dependencies) Swiftide 0.34 needs Rust 1.97 or newer, its minimum supported Rust version (MSRV). Create a binary crate and add three dependencies: ```sh cargo new hello-swiftide cd hello-swiftide cargo add swiftide@0.34 --features swiftide-agents,openai cargo add tokio --features macros,rt-multi-thread cargo add anyhow ``` `openai` compiles the OpenAI client. `swiftide-agents` is for [Your first agent](/start/your-first-agent/); this page does not use it yet. tokio runs the async `main`, and anyhow carries its errors. ## First program [Section titled “First program”](#first-program) Replace `src/main.rs` with this program: src/main.rs ```rust use swiftide::{integrations::openai::OpenAI, traits::SimplePrompt}; #[tokio::main] pub async fn main() -> anyhow::Result<()> { // Reads OPENAI_API_KEY from the environment. let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let answer = openai .prompt("Say hello to Swiftide in five words.".into()) .await?; println!("{answer}"); Ok(()) } ``` Source: [`snippets/src/start/installation.rs#L5-L19`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/start/installation.rs#L5-L19) `OpenAI::builder()` without a `client(..)` call reads `OPENAI_API_KEY` from the environment, and `OPENAI_BASE_URL` when it is set. `default_prompt_model` sets the model that `prompt` uses. `prompt` comes from the `SimplePrompt` trait, which the `use` line imports. It takes a `Prompt`, which `.into()` builds from a string. ### TLS [Section titled “TLS”](#tls) Swiftide’s Hypertext Transfer Protocol (HTTP) clients use rustls for Transport Layer Security (TLS), with the HTTP client’s own crypto provider, so the program needs no TLS setup. A provider that your application installs for the whole process takes precedence. [TLS](/reference/feature-flags/#tls) covers the one exception, Redis over `rediss://`. ## API keys [Section titled “API keys”](#api-keys) Export your application programming interface (API) key and run the program: ```sh export OPENAI_API_KEY=sk-... cargo run ``` Cargo builds the crate and the program prints the model’s one-line answer. These environment variables come up across the site and the showcases: | Variable | Read by | Default | | ------------------------------------------------ | --------------------------------------------- | ----------------------------------------------- | | `OPENAI_API_KEY` | the OpenAI client (`openai` flag) | none | | `OPENAI_BASE_URL` | the OpenAI client (`openai` flag) | `https://api.openai.com/v1` | | `OPENAI_MODEL` | the showcase command-line tools, not Swiftide | `gpt-5.4` or `gpt-5.4-mini` | | `QDRANT_URL`, `QDRANT_API_KEY` | the Qdrant builder (`qdrant` flag) | `http://localhost:6334` | | `GROQ_API_KEY` | the Groq client (`groq` flag) | none | | `GEMINI_API_KEY` | the Gemini client (`gemini` flag) | none | | `OPENROUTER_API_KEY` | the OpenRouter client (`open-router` flag) | none | | `QWEN_API_KEY`, then `DASHSCOPE_API_KEY` | the DashScope client (`dashscope` flag) | none | | `GITHUB_PERSONAL_ACCESS_TOKEN` or `GITHUB_TOKEN` | release-notes-agent with `--mcp` | none | | `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY` | the Langfuse tracing layer (`langfuse` flag) | required; the default layer panics without them | | `LANGFUSE_URL` | the Langfuse tracing layer (`langfuse` flag) | `http://localhost:3000` | The Langfuse layer also accepts `LANGFUSE_INIT_PROJECT_PUBLIC_KEY` and `LANGFUSE_INIT_PROJECT_SECRET_KEY` as fallbacks for the two keys. You can point `OPENAI_BASE_URL` at an OpenAI-compatible endpoint, such as a local gateway. The endpoint must serve the model name you pass to `default_prompt_model`, here `gpt-5-mini`. Unset the variable rather than leave it empty: an empty value becomes an empty base URL. [OpenAI-compatible providers](/integrations/openai-compatible/) and [Models](/models/overview/) cover the other clients. ## Feature flags [Section titled “Feature flags”](#feature-flags) Only `macros` is on by default. It provides the `#[tool]` attribute, `#[derive(Tool)]` and `#[indexing_transformer]`. Agents, tasks and each integration sit behind their own feature flag, and each flag adds a module: * `openai` adds `swiftide::integrations::openai`. * `qdrant` adds `swiftide::integrations::qdrant`. * `swiftide-agents` adds `swiftide::agents` and turns on `swiftide-tasks`, which adds `swiftide::tasks`. The showcases name only the flags they use: | Showcase | Flags | | ------------------------------------------------------ | -------------------------------------------------------- | | [release-notes-agent](/showcases/release-notes-agent/) | `macros`, `swiftide-agents`, `openai`, `mcp`, `langfuse` | | [ask-your-codebase](/showcases/ask-your-codebase/) | `tree-sitter`, `fastembed`, `lancedb`, `redb`, `openai` | | [workspace-architect](/showcases/workspace-architect/) | `swiftide-agents`, `swiftide-tasks`, `openai` | | [docs-qa](/showcases/docs-qa/) | `openai`, `fastembed`, `qdrant`, `langfuse` | Three of them turn default features off. With `--no-default-features`, add `macros` back if you use any of its macros. The `all` flag is not a shortcut to everything: it turns on only `qdrant`, `redis`, `tree-sitter`, `openai`, `fastembed`, `scraping`, `aws-bedrock`, `groq`, `ollama` and `pgvector`. [Feature flags](/reference/feature-flags/) lists every flag and what it turns on. ## Local services [Section titled “Local services”](#local-services) LanceDB, DuckDB, FastEmbed and Redb run inside your process, so the ask-your-codebase showcase needs no server. FastEmbed downloads its model on first use. Swiftide’s FastEmbed default is the fastembed crate’s default model; ask-your-codebase picks `jina-embeddings-v2-base-code` instead, which is about 600MB. Qdrant, Redis and PgVector run as servers. Docker starts each one: ```sh # Qdrant (qdrant flag): Swiftide talks gRPC Remote Procedure Call (gRPC) on 6334; # 6333 is the HTTP API and dashboard. docker run -d -p 6333:6333 -p 6334:6334 qdrant/qdrant # Redis (redis flag): the examples read REDIS_URL and default to redis://localhost:6379. docker run -d -p 6379:6379 redis:7-alpine # PgVector (pgvector flag): pass the connection URL to the PgVector builder with db_url. docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=postgres pgvector/pgvector:pg17 ``` ## Common errors [Section titled “Common errors”](#common-errors) * **The program panics with “No rustls crypto provider is configured”**: you are on 0.33 or earlier. See [Crypto provider panic](/reference/troubleshooting-and-faq/#crypto-provider-panic). * **Error E0432, “unresolved import”**: the feature flag behind that module is off. Add it with `cargo add swiftide@0.34 --features `. * **An authentication error (HTTP 401)**: `OPENAI_API_KEY` is missing or empty. The client builds without a key, so the error shows up at the first request. [Troubleshooting and FAQ](/reference/troubleshooting-and-faq/) covers more. ## Reference [Section titled “Reference”](#reference) * [`swiftide` crate root, Cargo Features section](https://docs.rs/swiftide/0.34/swiftide/index.html#cargo-features) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) ## Next [Section titled “Next”](#next) [Your first agent](/start/your-first-agent/) # What is Swiftide > Swiftide is a Rust library with three building blocks, agents, tasks, and indexing and query pipelines, and this page shows each one in a real program and when to pick it. Swiftide is a Rust library for building LLM (large language model) applications. It has three building blocks: agents for tool loops, tasks for typed hand-offs, and pipelines for data flows. This page shows each block in a real program, so you know which one to start with. ## Building blocks [Section titled “Building blocks”](#building-blocks) | Building block | Use it for | Entry point | Feature flag | | -------------- | ------------------------------------------------------------ | ----------------------------------------------------------- | ----------------------------------------------------- | | Agents | A model that calls tools in a loop until it stops | `swiftide::agents::Agent::builder()` | `swiftide-agents` | | Tasks | A typed graph of steps with fan-out, join and pause | `swiftide::tasks::Task::new()` | `swiftide-tasks`, also turned on by `swiftide-agents` | | Pipelines | Streaming data into storage, and answering questions from it | `swiftide::indexing::Pipeline`, `swiftide::query::Pipeline` | always compiled; each integration has its own flag | The three blocks share one set of core traits, such as `SimplePrompt`, `ChatCompletion`, `EmbeddingModel` and `Tool`. The `OpenAI` client implements the first three, so one client can serve all three blocks. It needs the `openai` feature flag. ### Agents [Section titled “Agents”](#agents) ![Agent loop](/diagrams/diagram-agent-loop.svg) The release-notes-agent showcase reads the git history since the last tag and drafts release notes. This function builds its agent: examples/release-notes-agent/src/agent.rs ```rust /// Builds the agent. All tools run through `executor`, whose working directory is the repo. pub fn build( llm: &(impl ChatCompletion + Clone + 'static), executor: Arc, task: &ReleaseTask, github_tools: Option, ) -> Result { let mut builder = Agent::builder(); builder .llm(llm) .context(DefaultContext::from_executor(executor)) .system_prompt(task.system_prompt(github_tools.is_some())) // Our own `stop` tool replaces the default one; call this before `tools`. .without_default_stop_tool() .tools([ git_log_since(), git_diff(), read_file(), // Drafts that describe a change twice go back to the model before anyone reviews them. OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(&task.notes_file))) .into(), ReleaseNotes::stop_tool().into(), ]) .limit(STEP_LIMIT); if let Some(github_tools) = github_tools { builder.add_toolbox(github_tools); } progress::report_tool_calls(&mut builder); progress::stream_draft(&mut builder); Ok(builder.build()?) } ``` Source: [`examples/release-notes-agent/src/agent.rs#L104-L136`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/agent.rs#L104-L136) The agent sends its message history to the model, calls the tools the model asks for, and adds their results to the history. The tools run their commands through the tool executor in the context, here a `DefaultContext` built from the repository’s executor. `progress::report_tool_calls` prints each tool call from `before_tool` and `after_tool` hooks. `progress::stream_draft` registers an `on_stream` hook, which switches the agent to streaming completions. `add_toolbox` adds optional GitHub tools from an MCP (Model Context Protocol) server, which needs the `mcp` feature flag. The loop ends when a tool asks the agent to stop, when a tool needs human feedback (here `ApprovalRequired` around `WriteChangelog`), when the model replies without calling a tool, or when it reaches the step limit. Each case sets a stop reason. Reaching the limit is not an error, so check the stop reason after `query` returns. This agent swaps the default `stop` tool for its own, which requires the full release notes as structured arguments. `limit(STEP_LIMIT)` caps the completions per query at 40. [Your first agent](/start/your-first-agent/) builds the first step of this agent with one tool and one hook. Read more in [Agents overview](/agents/overview/) and [Release notes agent](/showcases/release-notes-agent/). ### Tasks [Section titled “Tasks”](#tasks) ![Task graph](/diagrams/diagram-task-graph.svg) The workspace-architect showcase drafts an `ARCHITECTURE.md` for a Cargo workspace. Its task first discovers the member crates, then fans out to one analyst per crate: examples/workspace-architect/src/graph.rs ```rust // One branch per crate, all evaluating the same analyst node with a different input. task.register_transition(discover, move |members: WorkspaceMembers| { let mut crates = members.crates.into_iter(); let Some(first) = crates.next() else { return Transition::error("the workspace has no crates to document"); }; crates .fold(Transition::fan_out(&analyse, first), |fan_out, member| { fan_out.and(&analyse, member) }) .concurrency_model(ConcurrencyModel::Parallel) .join_with(join.join()) })?; ``` Source: [`examples/workspace-architect/src/graph.rs#L92-L105`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L92-L105) Every task node has an input type and an output type. The compiler checks that each transition hands the next node the input it expects. The transition after `discover` starts one branch per crate on the same `analyse` node and runs the branches with `ConcurrencyModel::Parallel`. `join_with` names the join node that collects the results. The `analyse` node still needs its own transition into the join, `task.register_transition(analyse, join.join())`, or the task fails validation with a missing-transition error. An empty workspace ends the task with `Transition::error`. A run on the swiftide workspace with claude-sonnet-5-5, through an OpenAI-compatible gateway, prints this (trimmed): ```text [ 0.0s] analyse started swiftide-core [ 0.0s] analyse started swiftide-langfuse ... [ 40.5s] analyse finished swiftide-core in 40.5s (15 tool calls) [ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x) ``` [Your first task](/start/your-first-task/) builds a smaller task that starts the same way, with a `discover` node and no fan-out. Read more in [How tasks work](/tasks/overview/) and [Workspace architect](/showcases/workspace-architect/). ### Pipelines [Section titled “Pipelines”](#pipelines) ![Indexing pipeline](/diagrams/diagram-pipeline.svg) An indexing pipeline streams nodes from a loader through transformers, a chunker and an embedder into storage. A query pipeline turns a question into an answer from retrieved chunks. The ask-your-codebase showcase indexes a codebase into LanceDB and answers questions about it with this query pipeline: examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` Source: [`examples/ask-your-codebase/src/query.rs#L36-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L36-L58) Query transformers rewrite and embed the question, the retriever searches the LanceDB table with the search strategy, and the answer generator writes the answer from the retrieved chunks. [Your first pipeline](/start/your-first-pipeline/) runs this showcase and walks through each step, along with the indexing pipeline that fills the table. The showcase needs the `lancedb`, `fastembed`, `tree-sitter` and `openai` feature flags. The docs-qa showcase adds hybrid search in Qdrant (the `qdrant` feature flag), reranking and evaluation. Read more in [Indexing overview](/indexing/overview/), [How querying works](/querying/overview/), [Ask your codebase](/showcases/ask-your-codebase/) and [Docs Q\&A](/showcases/docs-qa/). ## Composition [Section titled “Composition”](#composition) The blocks nest. In workspace-architect, every fan-out branch runs an agent. `CrateAnalyst` implements `TaskNode` with a `WorkspaceCrate` in and a `CrateSummary` out. Each evaluation builds a fresh agent for one crate: examples/workspace-architect/src/analyst.rs ```rust /// How often an agent is told to fix a missing or malformed report before the crate fails. const MAX_CORRECTIONS: usize = 2; #[async_trait] impl TaskNode for CrateAnalyst { type Input = WorkspaceCrate; type Output = CrateSummary; type Error = AnalystError; async fn evaluate( &self, _node_id: &DynNodeId, member: &WorkspaceCrate, ) -> Result { self.analyse(member).await.map_err(|cause| AnalystError { crate_name: member.name.clone(), cause, }) } } impl CrateAnalyst { async fn analyse(&self, member: &WorkspaceCrate) -> Result { let analysis = self.progress.analysis(&member.name); let mut agent = self.agent_for(member, analysis.clone())?; agent.query(brief(member)).await?; // Models occasionally answer in prose or nest the `stop` arguments; say what went wrong. let mut corrections = 0; let report = loop { match submitted_report(&agent) { Ok(report) => break report, Err(problem) if corrections < MAX_CORRECTIONS => { corrections += 1; analysis.corrected(&problem); agent.query(problem.correction()).await?; } Err(problem) => return Err(problem.into()), } }; analysis.finished(&report); Ok(report.reconciled_with(member)) } } /// The report the agent passed to `stop`, checked against the `CrateReport` schema. fn submitted_report(agent: &Agent) -> Result { match agent.stop_reason() { Some(StopReason::RequestedByTool(_, Some(arguments))) => { Ok(CrateReport::deserialize(arguments)?) } other => Err(ReportProblem::Missing { stop_reason: other.cloned(), }), } } ``` Source: [`examples/workspace-architect/src/analyst.rs#L138-L194`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/analyst.rs#L138-L194) Two built-in node types need no impl of your own. `Arc` takes a `Prompt` and returns a `String`; the showcase’s render step is one. `TaskAgent` wraps an `Agent` as a node that takes a `Prompt` and returns `()`. To pass an answer to the next node, write your own `TaskNode`. `TaskAgent` also locks the agent, so branches that share one `TaskAgent` run one at a time. For parallel work, build one agent per branch as `CrateAnalyst` does. A task node or a tool is plain async Rust, so its body can run an indexing pipeline or a query pipeline. ## Block selection [Section titled “Block selection”](#block-selection) | You need | Use | | ------------------------------------------------------------------------------------------- | ----------------------------------------- | | A model that decides which tools to call and when to stop | an agent | | Known steps with typed inputs and outputs, parallel branches, or a pause for human approval | a task | | Documents or code loaded, chunked, embedded and stored, then searched to answer questions | an indexing pipeline and a query pipeline | | Several agents in a fixed order, or one agent per item in parallel | a task with agent nodes | Start with an agent when the model should choose the steps. Move to a task when you know the steps and want the compiler to check the hand-offs between them. ## Project status [Section titled “Project status”](#project-status) Swiftide is pre-1.0. APIs change while the agent and task APIs settle, and breaking changes land in minor releases. Upgrade one minor version at a time with [Upgrading](/reference/upgrading/), and check the [Changelog](/reference/changelog/) for each release. These docs describe swiftide 0.34. Swiftide is MIT licensed and part of the [bosun.ai](https://bosun.ai) project. ## Reference [Section titled “Reference”](#reference) * [`swiftide`](https://docs.rs/swiftide/0.34/swiftide/index.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`TaskAgent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.TaskAgent.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`ConcurrencyModel`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.ConcurrencyModel.html) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`Tool`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Tool.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [Feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [Installation](/start/installation/) # Your first agent > Build an agent with one git tool and one hook that summarizes the commits since a tag, then compare it with the release-notes-agent showcase. This page builds a command-line program that asks a model to summarize the commits since a git tag. It is the first step of the [release-notes-agent](/showcases/release-notes-agent/) showcase: an agent with one tool that runs `git log` and one hook that prints each tool call. ## Dependencies [Section titled “Dependencies”](#dependencies) Start from the crate on [Installation](/start/installation/), or create a new one with the same dependencies plus serde: ```sh cargo new first-agent cd first-agent cargo add swiftide@0.34 --features swiftide-agents,openai cargo add tokio --features macros,rt-multi-thread cargo add anyhow serde ``` `swiftide-agents` adds `swiftide::agents`. The tool attribute comes from `macros`, which is on by default. Its generated code derives serde’s traits for the tool arguments, so your crate depends on serde. ## Program [Section titled “Program”](#program) The tool and the imports go at the top of `src/main.rs`: src/main.rs ```rust use anyhow::Context as _; use swiftide::{ agents::Agent, chat_completion::{ChatMessage, ToolOutput, errors::ToolError}, integrations::openai::OpenAI, traits::{AgentContext, Command}, }; /// Lists the commits since a git tag, newest first, one per line. /// /// # Arguments /// - `tag`: The tag to start from, e.g. `v0.33.0`. #[swiftide::tool] async fn git_log_since(context: &dyn AgentContext, tag: &str) -> Result { // The model chooses the tag; refuse anything that is not a plain tag name. let plain = |c: char| c.is_ascii_alphanumeric() || "._-/".contains(c); if tag.starts_with('-') || !tag.chars().all(plain) { return Ok(ToolOutput::fail(format!("`{tag}` is not a tag name"))); } let log = Command::shell(format!("git log --oneline --end-of-options {tag}..HEAD")); Ok(context.executor().exec_cmd(&log).await?.into()) } ``` Source: [`snippets/src/start/your_first_agent.rs#L4-L26`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/start/your_first_agent.rs#L4-L26) `main` builds the agent and prints its answer: src/main.rs ```rust #[tokio::main] pub async fn main() -> anyhow::Result<()> { let tag = std::env::args() .nth(1) .context("usage: first-agent ")?; // Reads OPENAI_API_KEY from the environment. let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let mut agent = Agent::builder() .llm(&openai) .system_prompt("You summarize git history for release notes. Be brief.") .tools([git_log_since()]) .before_tool(|_agent, tool_call| { println!( " → {} {}", tool_call.name(), tool_call.args().unwrap_or_default() ); Box::pin(async { Ok(()) }) }) .build()?; agent .query(format!("Summarize the commits since {tag}.")) .await?; let history = agent.history().await?; let summary = history.iter().rev().find_map(|message| match message { ChatMessage::Assistant(Some(text), _) => Some(text.as_str()), _ => None, }); println!("{}", summary.unwrap_or("(no summary)")); println!("stopped: {:?}", agent.stop_reason()); Ok(()) } ``` Source: [`snippets/src/start/your_first_agent.rs#L31-L68`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/start/your_first_agent.rs#L31-L68) The tool runs `git` in the directory you start the program from. That directory needs a git repository with the tag you pass. The repository `cargo new` created has no commits and no tags, so run the program from another repository, such as a clone of swiftide: ```sh export OPENAI_API_KEY=sk-... git clone https://github.com/bosun-ai/swiftide cd swiftide cargo run --manifest-path ../first-agent/Cargo.toml -- v0.33.0 ``` The program prints one line per tool call, such as `→ git_log_since {"tag":"v0.33.0"}`. Then it prints the model’s summary. The last line is `stopped: Some(NoNewMessages)` or `stopped: Some(RequestedByTool(..))`, because `stop_reason()` returns an `Option`. ## Tool [Section titled “Tool”](#tool) `#[swiftide::tool]` turns the async function into a tool named `git_log_since`. The macro replaces your function: `git_log_since()` becomes a constructor that returns the tool as a `Box`, and your body moves to a method on a generated struct, so you cannot call the function directly any more. `.tools([git_log_since()])` hands the tool to the agent. The doc comment becomes the tool description the model reads. The `# Arguments` list becomes the description of each argument. Every tool takes `&dyn AgentContext` as its first argument. `context.executor()` returns the tool executor, the part that runs commands. `Command::shell` describes a shell command, and `exec_cmd` runs it. The model chooses `tag`, so the tool checks it before it reaches the shell. A tag that starts with `-`, or holds anything but letters, digits, `.`, `_`, `-` and `/`, gets `ToolOutput::fail`. The model gets that message as the tool’s output and can try again. `--end-of-options` also stops git from reading the tag as a flag. When git exits with a non-zero status, `exec_cmd` returns a `CommandError`, and `?` turns it into a `ToolError`. The agent also sends that error back to the model. When a call with the same arguments fails more than `tool_retry_limit` times (3 by default), the agent stops with `StopReason::ToolCallsOverLimit` and `query` returns the error. On success, `.into()` turns the `CommandOutput` into `ToolOutput::Text`, with stdout and stderr in the order the executor saw them. ## Hook [Section titled “Hook”](#hook) A hook is a closure that runs at a fixed point in the agent loop. `before_tool` runs before each tool call and receives `&mut Agent` and the `ToolCall`. Every hook returns a boxed future, written `Box::pin(async { Ok(()) })`. When a hook returns an error, the agent logs it with tracing and carries on. ## Agent loop [Section titled “Agent loop”](#agent-loop) `Agent::builder()` starts with one tool already registered: `stop`, which the model calls when it is done. `.llm(&openai)` clones the client into the agent; `build` fails without it. `system_prompt` takes anything that converts into a `Prompt`, such as a `&str`. `query` adds your message to the message history and loops. Each completion either calls tools, whose outputs go into the history for the next completion, or answers in text. In a normal run the loop ends in one of two ways: * The model calls `stop`. `stop_reason()` then returns `StopReason::RequestedByTool`. * The model answers without calling a tool. The default context, `DefaultContext`, ends the loop on such an answer, and `stop_reason()` returns `StopReason::NoNewMessages`. Errors, failed tool calls and limits end it with other stop reasons. `history()` returns every `ChatMessage` of the run. The program prints the last `ChatMessage::Assistant` that has text. `DefaultContext` runs commands through a `LocalExecutor` in your working directory, without a sandbox: the tool can reach anything your user can. It sets no timeout either, so a git command that waits for a pager or a credential prompt blocks the agent. ## Showcase version [Section titled “Showcase version”](#showcase-version) The showcase prints its tool calls with the same kind of hook. This is the start of a real run on swiftide’s own history, from its `OUTPUT.md`. It used `claude-sonnet-5-5` through an OpenAI-compatible gateway: ```text Drafting release notes for the 7 commits since v0.33.0 with claude-sonnet-5-5 at http://127.0.0.1:8080/v1 (chat completions) → git_log_since v0.33.0 git_log_since ✓ 346 lines ``` It also adds the timeout your program lacks. Every command runs at the repository root, with a one-minute `default_timeout`: examples/release-notes-agent/src/main.rs ```rust let root = Git::new(&LocalExecutor::new(&cli.repo)) .top_level() .await .with_context(|| format!("{} is not a git repository", cli.repo.display()))?; // Every tool the agent runs, runs with the repository as its working directory. let executor: Arc = Arc::new( LocalExecutor::builder() .workdir(&root) .default_timeout(Some(Duration::from_mins(1))) .build()?, ); ``` Source: [`examples/release-notes-agent/src/main.rs#L93-L103`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/main.rs#L93-L103) The agent gets this executor through `DefaultContext::from_executor`. Use the same pattern to run your program against a fixed repository instead of the current directory. ## Variations [Section titled “Variations”](#variations) To print results too, add `after_tool`. It receives `&mut Result` and can replace what the model sees. ## Common mistakes [Section titled “Common mistakes”](#common-mistakes) * **A second `.tools(..)` call drops the first list.** `.tools(..)` replaces the whole set with the default tools plus the ones you pass. Pass every tool in one call, or call `add_tool` after the last `.tools(..)`. * **Your own stop tool is dropped.** Tools are kept by name, and a typed stop tool such as `StopWithArgs` is also named `stop`. `.tools(..)` inserts the default `stop` first, so yours is discarded and the model sees only the default. Call `without_default_stop_tool()` before `.tools(..)`, as the showcase does. * **The model passes an argument you did not expect.** Check model-chosen arguments before they reach a shell or a path. * **The agent loops for a long time.** `.limit(n)` caps the completions per `query`; the showcase uses 40. ## Related pages [Section titled “Related pages”](#related-pages) * [Tools](/agents/tools/): `#[derive(Tool)]` for tools with state, and argument types. * [Hooks](/agents/hooks/): every hook and what it receives. * [Executors](/agents/executors/): `LocalExecutor`, timeouts and other executors. * [Stopping and structured results](/agents/stopping-and-structured-results/): a stop tool that returns typed data. ## Reference [Section titled “Reference”](#reference) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`tool`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`Command`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.Command.html) * [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`CommandError`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.CommandError.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) ## Next [Section titled “Next”](#next) [Your first pipeline](/start/your-first-pipeline/) # Your first pipeline > Index a directory of Rust code into an embedded LanceDB table with local FastEmbed embeddings, then ask it one question and get an answer that cites files. This page indexes one directory of Rust code into an embedded LanceDB table and answers a question about it, naming the files the answer came from. You run the [ask-your-codebase](/showcases/ask-your-codebase/) showcase and read the parts that make it work. Everything except the LLM (large language model) runs on your machine. Tree-sitter chunks the code, FastEmbed embeds it on the CPU (central processing unit), and LanceDB stores the vectors in a local directory. ## Showcase run [Section titled “Showcase run”](#showcase-run) The showcase lives in `examples/ask-your-codebase/` in the swiftide repository. Clone it. To run exactly the code this site shows, check out the commit it is pinned to: ```sh git clone https://github.com/bosun-ai/swiftide.git cd swiftide git checkout e377c4a2e76fe2559629ae0efdae3c6e0fb4d829 # optional: the commit this site documents export OPENAI_API_KEY=sk-... cargo run -p ask-your-codebase -- index swiftide-agents cargo run -p ask-your-codebase -- ask --path swiftide-agents "How does an agent decide to stop?" ``` ## Dependencies [Section titled “Dependencies”](#dependencies) To build the same pipelines in your own crate, add the showcase’s dependencies: ```sh cargo add swiftide@0.34 --no-default-features --features tree-sitter,fastembed,lancedb,redb,openai cargo add fastembed@5 --no-default-features cargo add tokio --features full cargo add anyhow async-trait ``` Each integration has its own [feature flag](/reference/feature-flags/). The direct `fastembed` dependency picks a model other than the default. Only the showcase’s own chunker needs `async-trait`. ## LLM client [Section titled “LLM client”](#llm-client) examples/ask-your-codebase/src/llm.rs ```rust impl LlmEndpoint { /// Builds a client for this endpoint. /// /// Rate limits and transient errors are retried with exponential backoff, which matters when /// indexing fires many prompts concurrently. After `retry_budget` the error is returned, so /// an unreachable or broken endpoint fails instead of hanging. pub fn client(&self, retry_budget: Duration) -> Result> { let openai = OpenAI::builder() .api_base(&self.base_url) .api_key(&self.api_key) .default_prompt_model(&self.model) .use_responses_api(matches!(self.api, OpenAiApi::Responses)) .build()?; let backoff = BackoffConfiguration { max_elapsed_time_sec: retry_budget.as_secs(), ..BackoffConfiguration::default() }; Ok(Arc::new(LanguageModelWithBackOff::new(openai, backoff))) } } ``` Source: [`examples/ask-your-codebase/src/llm.rs#L79-L100`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/llm.rs#L79-L100) The client is swiftide’s `OpenAI` integration pointed at a base URL with `api_base` and `api_key`. Indexing sends many prompts at once, so `LanguageModelWithBackOff` wraps it and retries transient errors, such as rate limits, with exponential backoff. A wrong key is a permanent error and fails at once. `max_elapsed_time_sec` in `BackoffConfiguration` caps how long one prompt is [retried](/models/errors-backoff-and-usage/); the showcase allows 60 seconds while indexing and 30 while asking. ## Embedder and storage [Section titled “Embedder and storage”](#embedder-and-storage) examples/ask-your-codebase/src/embedding.rs ```rust /// A code-aware embedding model that runs on the CPU and needs no API key. /// /// The model (~600MB) is downloaded on first use. pub fn local_model() -> Result { let model = TextEmbedding::try_new( InitOptions::new(EmbeddingModel::JinaEmbeddingsV2BaseCode).with_max_length(MAX_TOKENS), )?; FastEmbed::builder() .embedding_model(model) .batch_size(BATCH_SIZE) .build() } ``` Source: [`examples/ask-your-codebase/src/embedding.rs#L18-L30`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/embedding.rs#L18-L30) The embedder runs `jina-embeddings-v2-base-code`, a model trained on code, with up to 2048 tokens per text. Its vectors have 768 dimensions, kept in `DIMENSIONS`. `BATCH_SIZE` (32) is the batch size FastEmbed uses inside one embedding call. Keep it equal to the batch size of the pipeline step that calls the embedder. examples/ask-your-codebase/src/codebase.rs ```rust /// Remembers which files were fully indexed, so unchanged files are skipped next time. pub fn node_cache(&self) -> Result { std::fs::create_dir_all(&self.data_dir)?; Redb::builder() .database_path(self.data_dir.join("cache.redb")) .table_name("indexed_files") .build() } /// Embedded vector store with one vector per chunk and the source path next to it. pub fn vector_store(&self) -> Result { let uri = self.lancedb_dir(); LanceDB::builder() .uri(uri.to_str().context("Index path is not valid UTF-8")?) .table_name("code_chunks") .vector_size(embedding::DIMENSIONS) .with_vector(EmbeddedField::Combined) .with_metadata(PATH_FIELD) .build() } ``` Source: [`examples/ask-your-codebase/src/codebase.rs#L56-L77`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/codebase.rs#L56-L77) `Redb` is the node cache, a single embedded file. `LanceDB` is the storage: `uri` is a local directory, and `vector_size` must equal the embedder’s dimensions. `EmbeddedField::Combined` stores one vector per chunk, computed from the chunk and its metadata. `with_metadata(PATH_FIELD)` keeps the file path in its own column, so the answer can cite it. Both live in `/.ask-your-codebase/`, inside the indexed directory. ## Indexing pipeline [Section titled “Indexing pipeline”](#indexing-pipeline) examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` Source: [`examples/ask-your-codebase/src/index.rs#L89-L115`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L89-L115) The indexing pipeline is a stream of nodes. Nothing runs until `run()` is awaited. `run` returns the pipeline statistics; the showcase reports `nodes_stored` as the chunk count. The marked lines are the core: * The loader, `FileLoader`, reads the files under the root with the given extensions. It skips hidden files and what `.gitignore` and `.ignore` files exclude. `.gitignore` only applies inside a git repository. Each file becomes one node. * `filter_cached` drops nodes the node cache has seen before. The cache key is a hash of the path and the content, so an edited file counts as new. * `then_chunk` runs the chunker. `ChunkByLanguage` is a small showcase type that picks a tree-sitter `ChunkCode` per file extension. It splits along syntax boundaries into chunks of 10 to 4096 bytes. * `then` runs a transformer on each chunk. `MetadataQACode` asks the LLM, with a custom prompt, for eight questions the chunk answers and stores them as metadata. `concurrency` caps the prompts in flight. * `then_in_batch` runs a batch transformer. `Embed` sends 32 chunks per call to FastEmbed. * `then_store_with` hands each chunk to the storage, here LanceDB, through the `Persist` trait. The other lines are showcase extras that drop tests, record the relative path and count for the report. Leave them out in your own crate, and replace the showcase helpers: * `FileLoader::new(codebase.root()).with_extensions(&languages.extensions())` becomes `FileLoader::new("./src").with_extensions(&["rs"])`. * `ChunkByLanguage::try_new(languages, CHUNK_SIZE)?` becomes `ChunkCode::try_for_language_and_chunk_size("rust", 10..4096)?`. * `codebase.node_cache()?` and `codebase.vector_store()?` are the builders from the storage excerpt. * `llm` is the client from the LLM client excerpt. Without `prompt_template`, `MetadataQACode` uses its default prompt. ## Query pipeline [Section titled “Query pipeline”](#query-pipeline) examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` Source: [`examples/ask-your-codebase/src/query.rs#L36-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L36-L58) `GenerateSubquestions` rewrites the question into several related ones, to widen what the search can match. The query transformer `Embed` turns them into one vector with the same local model that embedded the chunks. The search strategy, `SimilaritySingleEmbedding`, runs one vector search with that embedding and returns the `top_k` closest chunks. Its default is 10; the showcase passes 20. `then_retrieve` uses the same `LanceDB` store as the retriever. The answer generator, `answers::Simple`, answers from the retrieved chunks; its document template renders each chunk under its file path. [Answering](/querying/answering/) covers other answer generators. `query` consumes the pipeline and returns the answered query. To ask more than once, call `query_mut` or `query_all`. ## Output [Section titled “Output”](#output) This output is from the showcase’s `OUTPUT.md`. The chunk questions came from claude-haiku-5-5 and the answer from claude-sonnet-5-5, both through an OpenAI-compatible gateway. The embeddings came from FastEmbed on the CPU. The first run indexes everything: ```text Indexing Rust files in swiftide-agents ... Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s ``` A second run finds every file in the node cache and sends no prompts: ```text Indexing Rust files in swiftide-agents ... Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s ``` Asking “How does an agent decide to stop?” gives, trimmed: ```text # How an agent decides to stop An agent stops when something calls `Agent::stop` (or `stop_with_error`), which moves the state to `State::Stopped(StopReason)`. The main loop in `run_agent` checks that state after each step. `Agent::stop` is a no-op if the agent is already stopped (`src/agent.rs`). ## The stop mechanism - `Agent::stop(reason)` fires the `OnStop` hooks with the reason, then sets `self.state = State::Stopped(reason)`. It only takes effect at the end of the current loop iteration (`src/agent.rs`). - `Agent::stop_with_error(&AgentError)` fires `OnStop` with `StopReason::Error` and the error, then sets `Stopped(StopReason::Error)` (`src/agent.rs`). - You can inspect the result with `stop_reason()`, `is_stopped()` and `state()` (`src/agent.rs`, `src/state.rs`). ... Sources: - src/agent.rs - src/tools/control.rs - src/lib.rs - src/state.rs - src/errors.rs - src/task_agent.rs - src/hooks.rs - src/tasks.rs - src/default_context.rs ``` ## Variations [Section titled “Variations”](#variations) ### OpenAI embeddings [Section titled “OpenAI embeddings”](#openai-embeddings) The LanceDB example embeds with OpenAI’s `text-embedding-3-small` instead, so its `vector_size` is 1536: ```rust let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-4o-mini") .build()?; let tempdir = TempDir::new().unwrap(); // Configure lancedb with a default vector size, a single embedding // and in addition to embedding the text metadata, also store it in a field let lancedb = LanceDB::builder() .uri(tempdir.child("lancedb").to_str().unwrap()) .vector_size(1536) .with_vector(EmbeddedField::Combined) .with_metadata(METADATA_QA_TEXT_NAME) .table_name("swiftide_test") .build() .unwrap(); ``` Source: [`examples/lancedb.rs#L20-L36`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/lancedb.rs#L20-L36) ### Qdrant with the default FastEmbed model [Section titled “Qdrant with the default FastEmbed model”](#qdrant-with-the-default-fastembed-model) `FastEmbed::builder()` without a model uses the default model, with 384 dimensions. This pipeline stores into Qdrant. It needs the `qdrant` feature flag and a Qdrant server at `QDRANT_URL`, which defaults to `http://localhost:6334`. See [Qdrant](/integrations/qdrant/): ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .then_in_batch(Embed::new(FastEmbed::builder().batch_size(10).build()?)) .then_store_with( Qdrant::try_from_url(qdrant_url)? .batch_size(50) .vector_size(384) .collection_name("swiftide-examples-fastembed".to_string()) .build()?, ) .run() .await?; ``` Source: [`examples/fastembed.rs#L32-L42`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/fastembed.rs#L32-L42) For hybrid search and reranking, see the [docs-qa](/showcases/docs-qa/) showcase. ## Gotchas [Section titled “Gotchas”](#gotchas) * Use the same embedding model for indexing and querying, and set `vector_size` to its dimensions. After you change the model, delete `/.ask-your-codebase/` in the indexed directory and index again. * The first run downloads `jina-embeddings-v2-base-code`, about 600MB, into `.fastembed_cache/` in the current directory. * When a file changes, its new chunks are added, but the chunks of the old version stay in LanceDB. * Files enter the node cache only after `run()` finishes without an error. One failed chunk ends the run, and every file of that run is prompted and embedded again next time. The same holds for a run you interrupt. * A file that produces no chunks is never cached, so it is loaded again on every run. ## Reference [Section titled “Reference”](#reference) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`BackoffConfiguration`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.BackoffConfiguration.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) ## Next [Section titled “Next”](#next) [Your first task](/start/your-first-task/) # Your first task > Build a typed task with two nodes and one transition, run it to completion and print a one-line report of crate names. You end this page with a `Task` that turns a list of crate names into a one-line report. It has two task nodes, one transition between them and a finish. The [workspace-architect](/showcases/workspace-architect/) showcase starts the same way, with a `discover` node, before it fans out to one agent per crate. ## Dependencies [Section titled “Dependencies”](#dependencies) The task needs no model, so the `swiftide-tasks` feature flag is enough. `swiftide-agents` also turns `swiftide-tasks` on. ```sh cargo add swiftide@0.34 --features swiftide-tasks cargo add tokio --features macros,rt-multi-thread cargo add anyhow ``` ## Program [Section titled “Program”](#program) `main` builds the task and runs it: src/main.rs ```rust use swiftide::tasks::{NodeError, Task, TaskRunOutcome}; #[tokio::main] pub async fn main() -> anyhow::Result<()> { // A task from a manifest listing (one crate per line) to a one-line report. let mut task = Task::::new(); let discover = task.register_node_fn(|listing: &String| { let crates: Vec = listing.lines().map(str::trim).map(String::from).collect(); Ok::<_, NodeError>(crates) }); let report = task.register_node_fn(|crates: &Vec| { Ok::<_, NodeError>(format!("{} crates: {}", crates.len(), crates.join(", "))) }); task.starts_with(discover); task.register_transition(discover, move |crates| report.transitions_with(crates))?; task.register_transition(report, task.transitions_to_finish())?; let listing = "swiftide-core\nswiftide-tasks\nswiftide-agents".to_string(); match task.run(listing).await? { TaskRunOutcome::Completed(line) => println!("{line}"), TaskRunOutcome::Paused => println!("paused"), } Ok(()) } ``` Source: [`snippets/src/start/your_first_task.rs#L5-L30`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/start/your_first_task.rs#L5-L30) ## Task nodes [Section titled “Task nodes”](#task-nodes) `Task::::new()` creates an empty task. The first type parameter is the input you pass to `run`. The second is the output it returns. `register_node_fn` turns a synchronous closure into a task node. The closure takes a reference to its input and returns a `Result`. The error type must implement `std::error::Error + Send + Sync + 'static`, so `anyhow::Error` does not fit. The closures never fail, so `Ok::<_, NodeError>(..)` only names a type the compiler can infer. The method returns a typed `NodeId`, which you use to wire transitions. `discover` splits the listing into a `Vec`. `report` takes that vector and formats a line. Every input and output type must be `Clone + Debug + Send + Sync + 'static`. The `NodeArg` trait covers exactly that bound, so plain owned types work. ## Transitions [Section titled “Transitions”](#transitions) `starts_with(discover)` marks the node that receives the task input. `register_transition` connects a node to what runs after it. The closure receives the output of `discover` and returns `report.transitions_with(crates)`. The compiler checks that the input type of `report` matches the output type of `discover`. A mismatch is a compile error, not a runtime surprise. `task.transitions_to_finish()` ends the task with the output of `report`. The finishing node’s output type must equal the task’s output type. Each node gets exactly one transition. A second `register_transition` for the same node returns an error. A node without a transition makes `run` fail with `TaskError::MissingTransition`, and a task without a start node fails with `TaskError::NoSteps`. ## Run outcome [Section titled “Run outcome”](#run-outcome) `task.run(input)` returns `Result, TaskError>`. `TaskRunOutcome::Completed` carries the output. `TaskRunOutcome::Paused` means a node asked the task to pause, which this graph never does. `cargo run` prints: ```text 3 crates: swiftide-core, swiftide-tasks, swiftide-agents ``` A node error stops the run and comes back as `TaskError::NodeError`. Its message gives the numeric id of the node that failed, as in `Node error in node 0: ...`. The id is the order in which you registered the node, not the name of the variable that holds it. To run the same graph again with the last input, call `reset` and then `resume`. ## Fan-out [Section titled “Fan-out”](#fan-out) A transition can also fan out. It schedules one branch per input on the same node, and a join node collects the outputs. The [workspace-architect showcase](/showcases/workspace-architect/) uses this to run one agent per crate. [Fan-out and join](/tasks/fan-out-and-join/) covers the API. ## Common mistakes [Section titled “Common mistakes”](#common-mistakes) * Returning `anyhow::Result` from a `register_node_fn` closure does not compile. Use an error type that implements `std::error::Error`, such as your own `thiserror` enum. * `run` on a task that is still active or paused returns `TaskError::TaskActive`. Continue a paused task with `resume`. * A human approval step is `Transition::pause()` followed by `task.resume()`. See [Pause and resume](/tasks/pause-and-resume/). * For nodes that call a model, see [Nodes and transitions](/tasks/nodes-and-transitions/) and [Agents as nodes](/tasks/agents-as-nodes/). An agent node also needs the `swiftide-agents` feature flag and a provider feature flag such as `openai`. ## Reference [Section titled “Reference”](#reference) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`NodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html) * [`NodeError`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeError.html) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) ## Next [Section titled “Next”](#next) [Using Swiftide with coding agents](/start/for-coding-agents/) # 404: nothing at this path. > The page may have moved when these docs were rebuilt. Search from the header, or start from one of these. * [What is Swiftide?](/start/what-is-swiftide/) * [Installation](/start/installation/) * [Agents](/agents/overview/), [Tasks](/tasks/overview/), [Indexing](/indexing/overview/), [Querying](/querying/overview/) * [Changelog](/reference/changelog/) and [upgrading](/reference/upgrading/) * [All pages for LLMs](/llms.txt) # Context and message history > Store an agent's message history in memory or Redis, resume it in a new process, and rewrite it while the agent runs. An agent’s message history can outlive the process and be rewritten while the agent runs. The `agents_resume.rs` example saves the history of one agent as JSON (JavaScript Object Notation) and loads it into a second one: ```rust // Let's store the messages in a database, retrieve them back, and start a new agent let stored_history = serde_json::to_string(&first_agent.history().await?)?; let retrieved_history: Vec<_> = serde_json::from_str(&stored_history)?; let restored_context = DefaultContext::default() .with_existing_messages(retrieved_history) .await? .to_owned(); ``` Source: [`examples/agents_resume.rs#L23-L30`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_resume.rs#L23-L30) ```rust let mut second_agent = agents::Agent::builder() .llm(&openai) .context(restored_context) // We'll use the one from the first agent, alternatively we could also pop it from the // previous history and add a new one here .no_system_prompt() .build()?; second_agent.query("What did you say?").await?; ``` Source: [`examples/agents_resume.rs#L34-L42`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_resume.rs#L34-L42) `Agent::history()` returns every `ChatMessage` the context holds. `ChatMessage` implements `Serialize` and `Deserialize`, so `serde_json` turns the history into a string you can store anywhere. `DefaultContext::with_existing_messages` loads the messages back into a fresh context. The second agent gets that context through the builder’s `context` method and continues the conversation with `query`. The next completion sends the whole restored history to the model. The first agent sets no system prompt, so `no_system_prompt()` changes nothing in this example. [Prompt removal](/agents/system-prompts/#prompt-removal) explains when it matters. `run` without a new message only continues when the history ends in a user message, a tool output or a pending tool call. When it ends in a plain assistant answer, as it does here, `run` stops at once with `StopReason::NoNewMessages`. ## Default context [Section titled “Default context”](#default-context) The context stores the message history, decides when the next completion runs and what it contains, holds the tool executor and records feedback for tools that wait on a person. Every agent built without `.context(..)` gets a `DefaultContext`: * the message history is an in-memory `Mutex>`; * tools run commands through a `LocalExecutor` in the current directory; * the agent stops when the last message is an assistant answer and no tool feedback is pending. `DefaultContext::from_executor` keeps those defaults and swaps the tool executor. It takes a value that converts into `Arc`. [Tool executors](/agents/executors/) covers the conversion and the tool executors themselves. `with_stop_on_assistant(false)` turns off the last rule. A text answer then no longer ends the run, and the agent asks the model again. Only a tool that stops the agent, the builder’s `limit`, or a model reply with no content and no tool calls ends the loop. An agent has no limit by default, so set one ([Stop reasons](/agents/stopping-and-structured-results/#stop-reasons)). A clone of a `DefaultContext` shares its history and counters. ## Message history [Section titled “Message history”](#message-history) Where the messages live is the job of the `MessageHistory` trait. It has three required methods: * `history` returns all messages in order; * `push_owned` appends one message; * `overwrite` replaces all messages. The trait requires `Send + Sync + Debug` and is declared with `#[async_trait]`, so annotate your impl the same way. `push`, `extend` and `extend_owned` have default implementations on top of `push_owned`. Implement the trait for your database and pass the value to `DefaultContext::with_message_history`. The backend only stores the list. Each completion sends the full history, cut at the latest summary, and the context’s counters only decide whether a new completion runs. ### Custom backend [Section titled “Custom backend”](#custom-backend) This backend keeps one conversation in a JSON file. It uses the `async-trait` crate for the `#[async_trait]` attribute: ```rust /// Keeps the message history of one conversation in a JSON file. #[derive(Debug)] struct JsonFileHistory { path: PathBuf, // Serializes read-modify-write cycles on the file. lock: Mutex<()>, } impl JsonFileHistory { async fn read(&self) -> anyhow::Result> { match tokio::fs::read_to_string(&self.path).await { Ok(json) => Ok(serde_json::from_str(&json)?), Err(err) if err.kind() == std::io::ErrorKind::NotFound => Ok(Vec::new()), Err(err) => Err(err.into()), } } async fn write(&self, messages: &[ChatMessage]) -> anyhow::Result<()> { tokio::fs::write(&self.path, serde_json::to_vec(messages)?).await?; Ok(()) } } #[async_trait] impl MessageHistory for JsonFileHistory { async fn history(&self) -> anyhow::Result> { let _guard = self.lock.lock().await; self.read().await } async fn push_owned(&self, item: ChatMessage) -> anyhow::Result<()> { let _guard = self.lock.lock().await; let mut messages = self.read().await?; messages.push(item); self.write(&messages).await } async fn overwrite(&self, items: Vec) -> anyhow::Result<()> { let _guard = self.lock.lock().await; self.write(&items).await } } ``` Source: [`snippets/src/agents/context_and_history.rs#L71-L112`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/context_and_history.rs#L71-L112) ```rust let mut context = DefaultContext::default(); context.with_message_history(JsonFileHistory { path: "session-42.json".into(), lock: Mutex::new(()), }); let mut agent = Agent::builder().llm(&openai).context(context).build()?; ``` Source: [`snippets/src/agents/context_and_history.rs#L118-L124`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/context_and_history.rs#L118-L124) Every method takes `&self`, so the backend guards its own state; here a `tokio::sync::Mutex` keeps two writes from interleaving. `history` returns an empty list for a file that does not exist yet. `with_message_history` takes the value by ownership and stores it behind an `Arc`. ## Redis storage [Section titled “Redis storage”](#redis-storage) The `redis` feature flag adds a `Redis` integration that implements `MessageHistory`: ```sh cargo add swiftide@0.34 --features swiftide-agents,openai,redis ``` ```rust let redis = Redis::try_from_url("redis://localhost:6379", "session-42")?; let mut context = DefaultContext::default(); context.with_message_history(redis); let mut agent = Agent::builder() .llm(&openai) .context(context) .system_prompt("You help maintain the swiftide repository.") .build()?; agent.query("Which crates changed this week?").await?; ``` Source: [`snippets/src/agents/context_and_history.rs#L17-L28`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/context_and_history.rs#L17-L28) The messages go into a Redis list under `:message_history`, here `session-42:message_history`, one JSON string per message. Use one prefix per conversation. A new process that builds a context with the same prefix sees the whole conversation again. Build that agent without `.system_prompt(..)`, or with `no_system_prompt()`, as in the first example. Otherwise its first run pushes another system message onto the list. The connection opens on first use. If Redis is unreachable, the agent returns `AgentError::MessageHistoryError`. ## Approvals across restarts [Section titled “Approvals across restarts”](#approvals-across-restarts) Approvals are not part of the message history. They live in the context’s memory. When the agent stops with `StopReason::FeedbackRequired`, store the `ToolCall` it carries; the person’s `ToolFeedback` comes later. Store each decision as a `(ToolCall, ToolFeedback)` pair, since `serde_json` cannot use a struct as a map key. Both types implement `Serialize` and `Deserialize`. Collect the pairs into a `HashMap` for `DefaultContext::with_tool_feedback` in the new process. [Human in the loop](/agents/human-in-the-loop/) covers approvals. ```rust // First process: the agent waits for approval; store the person's decision with its call. if let Some(StopReason::FeedbackRequired { tool_call, .. }) = agent.stop_reason() { let decisions = vec![(tool_call.clone(), ToolFeedback::approved())]; std::fs::write("approvals.json", serde_json::to_string(&decisions)?)?; } // Second process: load the decisions next to the restored message history, then resume. let decisions: Vec<(ToolCall, ToolFeedback)> = serde_json::from_str(&std::fs::read_to_string("approvals.json")?)?; let mut context = DefaultContext::default(); context.with_message_history(Redis::try_from_url("redis://localhost:6379", "session-42")?); context.with_tool_feedback(decisions.into_iter().collect::>()); let mut resumed = Agent::builder() .llm(&openai) .context(context) .no_system_prompt() .build()?; resumed.run().await?; ``` Source: [`snippets/src/agents/context_and_history.rs#L133-L152`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/context_and_history.rs#L133-L152) `with_tool_feedback` accepts anything that converts into a `HashMap` and adds the entries to the feedback the context already holds. The restored history still ends in the pending tool call, so `run` calls the tool again, and the tool now finds its approval. ## History rewrites [Section titled “History rewrites”](#history-rewrites) Since 0.33, `AgentContext::replace_history` swaps the whole history while the agent runs. It takes `&self`, so any hook can call it through `agent.context()`. This `after_each` hook shortens old tool outputs once the history passes `MAX_MESSAGES`: ```rust const MAX_MESSAGES: usize = 40; const KEEP_RECENT: usize = 10; let mut agent = Agent::builder() .llm(&openai) .after_each(|agent: &mut Agent| { Box::pin(async move { let history = agent.history().await?; if history.len() <= MAX_MESSAGES { return Ok(()); } let keep_from = history.len() - KEEP_RECENT; let trimmed = history .into_iter() .enumerate() .map(|(index, message)| match message { ChatMessage::ToolOutput(call, ToolOutput::Text(_)) if index < keep_from => { ChatMessage::ToolOutput(call, ToolOutput::text("(output removed)")) } other => other, }) .collect(); agent.context().replace_history(trimmed).await?; Ok(()) }) }) .build()?; ``` Source: [`snippets/src/agents/context_and_history.rs#L36-L64`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/context_and_history.rs#L36-L64) The hook only replaces the text of older tool outputs. Each tool call keeps its output, which the providers require. `DefaultContext` writes the new list through `overwrite` and resets its completion counters. The next completion then runs even if no new message was added since the last one. ## Summaries [Section titled “Summaries”](#summaries) A `ChatMessage::Summary` shortens what the model sees without rewriting storage. When the history holds one, `DefaultContext` sends the system messages plus everything from the latest summary on. Add one with `ChatMessage::new_summary` through `agent.context().add_message(..)`. OpenAI-compatible providers send a summary as an assistant message. `ContextCompression` in agents-contrib writes these summaries for you; see [Contrib](/agents/contrib/). ## Custom contexts [Section titled “Custom contexts”](#custom-contexts) For behavior the message history cannot express, implement `AgentContext` yourself. Wrap a `DefaultContext` and forward the calls you do not change. The [upgrading guide](/reference/upgrading/) has a context that counts added messages: ```rust /// Counts the messages an agent adds and delegates everything else. #[derive(Clone)] pub struct CountingContext { inner: DefaultContext, added: Arc, } #[async_trait] impl AgentContext for CountingContext { async fn next_completion(&self) -> anyhow::Result>> { self.inner.next_completion().await } async fn current_new_messages(&self) -> anyhow::Result> { self.inner.current_new_messages().await } async fn add_messages(&self, items: Vec) -> anyhow::Result<()> { self.added .fetch_add(items.len(), std::sync::atomic::Ordering::Relaxed); self.inner.add_messages(items).await } async fn add_message(&self, item: ChatMessage) -> anyhow::Result<()> { self.added .fetch_add(1, std::sync::atomic::Ordering::Relaxed); self.inner.add_message(item).await } #[allow(deprecated)] async fn exec_cmd(&self, cmd: &Command) -> Result { self.inner.exec_cmd(cmd).await } fn executor(&self) -> &Arc { self.inner.executor() } async fn history(&self) -> anyhow::Result> { self.inner.history().await } async fn replace_history(&self, items: Vec) -> anyhow::Result<()> { self.inner.replace_history(items).await } async fn redrive(&self) -> anyhow::Result<()> { self.inner.redrive().await } async fn has_received_feedback(&self, tool_call: &ToolCall) -> Option { self.inner.has_received_feedback(tool_call).await } async fn feedback_received( &self, tool_call: &ToolCall, feedback: &ToolFeedback, ) -> anyhow::Result<()> { self.inner.feedback_received(tool_call, feedback).await } } ``` Source: [`snippets/src/reference/upgrading.rs#L211-L272`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L211-L272) ## Gotchas [Section titled “Gotchas”](#gotchas) * `with_existing_messages` calls `overwrite` on the backend. With Redis it deletes the list under the key and writes the given messages. * `with_existing_messages` and `with_message_history` return `&mut DefaultContext`. Use a `let mut` binding, or end the chain with `.to_owned()` as the example does. * A builder-made `Redis` uses the key `message_history` unless you set `message_history_key(..)` on the builder or call `with_message_history_key` afterwards. Agents that share that key also share one history. * Messages added through `agent.context()` do not trigger `on_new_message` hooks. Only `Agent::add_message` runs them. ## Reference [Section titled “Reference”](#reference) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`DefaultContext::with_existing_messages`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.with_existing_messages) * [`DefaultContext::with_message_history`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.with_message_history) * [`DefaultContext::with_tool_feedback`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.with_tool_feedback) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`AgentContext::replace_history`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html#tymethod.replace_history) * [`MessageHistory`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.MessageHistory.html) * [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) * [`Redis::try_from_url`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html#method.try_from_url) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ChatMessage::new_summary`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html#method.new_summary) * [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) * [`AgentBuilder::context`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.context) * [`AgentBuilder::no_system_prompt`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.no_system_prompt) * [`Agent::history`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.history) ## Next [Section titled “Next”](#next) [Tool executors](/agents/executors/) # Contrib tools and hooks > Give an agent file tools (read_file, edit, apply_patch), an update_plan tool, hooks that truncate tool output and compress the conversation, and an executor that starts on first use, from swiftide-agents-contrib. With the components on this page, your agent reads and edits files, keeps a plan, and keeps its context small on long runs. They come from swiftide-agents-contrib. ```rust let openai = OpenAI::builder().default_prompt_model("gpt-5.4").build()?; let executor: Arc = Arc::new(LocalExecutor::new("/srv/checkout")); let mut agent = Agent::builder() .llm(&openai) .context(DefaultContext::from_executor(executor)) .system_prompt("You fix failing tests. Keep a plan with update_plan.") .add_tool(ReadFile::default().with_line_numbers()) .add_tool(ExactReplace) .add_tool(ApplyPatch) .add_tool(UpdatePlan::default()) .limit(50) .build()?; agent .query("The test `parses_empty_input` in src/parser.rs fails. Fix it.") .await?; ``` Source: [`snippets/src/agents/contrib.rs#L20-L36`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/contrib.rs#L20-L36) `limit(50)` caps the run at 50 completions, because a fixing loop can run long. The four contrib tools sit next to the `stop` tool that every agent gets. Every file operation runs as a command through the tool executor of the context, here a `LocalExecutor` rooted at the checkout. ## Feature flags [Section titled “Feature flags”](#feature-flags) Nothing in the crate is enabled by default. Pick one feature flag per capability: | Feature flag | Adds | | ------------------------------ | ---------------------------------------------------- | | `agents-contrib-file-tools` | `ReadFile`, `ExactReplace`, `ApplyPatch` | | `agents-contrib-planning` | `UpdatePlan`, `PlanItem`, `PlanStatus`, `PlanMemory` | | `agents-contrib-context` | `ToolSummarizer`, `ContextCompression` | | `agents-contrib-lazy-executor` | `LazyExecutor`, `ToolExecutorExt` | Each one turns on `agents-contrib`, which turns on `swiftide-agents`. The plain `agents-contrib` flag only exposes the crate as `swiftide::agents_contrib`: ```sh cargo add swiftide@0.34 --features openai,agents-contrib-file-tools,agents-contrib-planning cargo add tokio --features full cargo add anyhow ``` A feature flag never registers a tool or a hook. You add each component to the builder yourself. [Feature flags](/reference/feature-flags/) lists every flag. ## File tools [Section titled “File tools”](#file-tools) The model sees these tool names: | Type | Tool name | Arguments | | ----------------------------------------- | ----------------------------- | --------------------------------------------------------------- | | `ReadFile::default()` | `read_file` | `file_name`, optional `offset` and `limit` | | `ReadFile::default().with_line_numbers()` | `read_file_with_line_numbers` | same, output prefixed with `N\|` line numbers | | `ExactReplace` | `edit` | `file_path`, `old_string`, `new_string`, optional `replace_all` | | `ApplyPatch` | `apply_patch` | `patch` | `read_file` returns up to 2,000 lines by default. It cuts single lines at 2,000 characters and refuses a window over 100,000 characters. When it stops early, the output tells the model which offset to read next. `edit` replaces a match only when it is unique, unless the model sets `replace_all`. `apply_patch` takes a patch in this format: ```text *** Begin Patch *** Add File: hello.txt +Hello *** Update File: src/main.rs @@ fn greet() - println!("Hi"); + println!("Hello"); *** End Patch ``` `*** Move to: new/path` after an update header moves a file. `*** Delete File: path` deletes one. Lines starting with a space are context, `-` removes a line and `+` adds one. The file tools need a tool executor with a POSIX (Portable Operating System Interface) shell and the `cat`, `mkdir` and `rm` commands. ## Plans [Section titled “Plans”](#plans) `UpdatePlan` registers `update_plan`. The model sends `items`, each with a `step` and a `status` of `pending`, `in_progress` or `completed`, plus an `explanation` string. Both arguments are required. An empty `items` list returns the current plan without changing it. The tool answers with a `ToolOutput::fail` when a step is empty or spans several lines, or when more than one step is in progress. The stored plan stays as it was. Clones of one `UpdatePlan` share the plan, while two `UpdatePlan::default()` values each have their own. The plan lives in memory and is not persisted. ## Context management [Section titled “Context management”](#context-management) Two hooks keep the message history short: ```rust let small_model = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let summarizer = ToolSummarizer::builder() .llm(Arc::new(small_model.clone()) as Arc) .tools_to_summarize(vec!["run_tests"]) .git_start_sha("HEAD") .build()?; let compression = ContextCompression::builder() .llm(Arc::new(small_model) as Arc) .git_start_sha("HEAD") .git_diff_dir(".") .context_limit(200_000usize) .build()?; builder.after_tool(summarizer.summarize_hook()); compression.register_hooks(builder); ``` Source: [`snippets/src/agents/contrib.rs#L44-L62`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/contrib.rs#L44-L62) Call both on the same `AgentBuilder` before `build()`. `tools_to_summarize` only applies to tools the agent has registered. A listed tool the agent lacks is skipped with a warning. `run_tests` here stands for a tool you define yourself. `ToolSummarizer::summarize_hook` returns an `after_tool` hook. It truncates any tool output over an estimated 10,000 tokens to its first 40 and last 160 lines. It then cuts the result to a character budget of four characters per token, 40,000 by default, so a long line can still end mid-line. Finally, it summarizes the output of the tools named in `tools_to_summarize` with the model you pass to `llm`. Set `summarize_all_at_token_cutoff` to summarize any large output. By default, `skip_summarize_all` exempts the file-read tools from truncation and from `summarize_all_at_token_cutoff`. For the tools in `tools_needing_diff` (default `run_tests` and `run_coverage`), the summary prompt includes `git diff` against `git_start_sha`. `ContextCompression::register_hooks` adds an `after_completion` and an `after_each` hook. They check three triggers in this order: 1. A turn that ends with a file read postpones any summary until the history reaches 90% of `context_limit`. 2. The sixth turn after the last summary triggers a new summary (`num_completions_for_summary`). 3. A history estimated above 70% of `context_limit` triggers one, unless the agent has stopped. The model you pass to `llm` writes the summary from the history, a Git diff and the plan progress. The hook stores the summary as a `ChatMessage::Summary`. A `DefaultContext` then sends only the system messages and everything from the latest summary on ([Summaries](/agents/context-and-history/#summaries)). When the summary interrupts a tool turn, the hook also appends a user message `Continue` to the message history. After the first summary, the hook adds a `read_original_instructions` tool so the model can recover the first user message. It adds the tool only when the history holds a user message. `ContextCompression`, and `ToolSummarizer` for the tools in `tools_needing_diff`, run `git diff` through the agent’s tool executor. Point that tool executor at a Git repository. ## Lazy executor [Section titled “Lazy executor”](#lazy-executor) `LazyExecutor` defers expensive setup, such as cloning a repository or starting a container, until the agent first needs it: ```rust let lazy = LazyExecutor::new(|| async { // Clone a repository, start a container, ... before the first command runs. Ok(Arc::new(LocalExecutor::new("/srv/checkout")) as Arc) }); assert!(!lazy.is_started()); let executor: Arc = Arc::new(lazy); let agent = Agent::builder() .llm(openai) .context(DefaultContext::from_executor(executor)) .add_tool(ReadFile::default()) .build()?; ``` Source: [`snippets/src/agents/contrib.rs#L70-L81`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/contrib.rs#L70-L81) `LazyExecutor::new` takes a closure whose future resolves to an `anyhow::Result>`. The first command, file stream or download calls it. Clones share the started executor. A failed start is retried on the next use. `is_started` returns true once the closure has succeeded. `ToolExecutorExt` adds `is_started`, `cleanup` and `download` to a tool executor, so bring the trait into scope to call them. It is implemented for `LocalExecutor`, for `LazyExecutor` itself and for a `ScopedExecutor` whose inner executor implements `ToolExecutorExt` and `Clone`. `cleanup` before the start does nothing. `download` runs `dd`, `base64` and `sha256sum` in the tool executor. ## Gotchas [Section titled “Gotchas”](#gotchas) * `LocalExecutor` does not isolate the file system. `read_file` and `edit` pass absolute paths to the tool executor unchanged. `apply_patch` only takes relative paths without `..` components, but it follows symlinks like the other tools. Use an isolated executor, such as the separate swiftide-docker-executor crate, for repositories you do not trust. * Patches are not transactions. `apply_patch` runs its operations in order and can partly succeed, so the model has to read the result report. * Tools are unique by name. If you already have a tool called `read_file` or `edit`, `add_tool` replaces it with the contrib tool, or the other way around. * The default `context_limit` is 400,000,000. Set one that matches your model, or compression only triggers on the turn count. * `ContextCompression` keeps its turn counter behind an `Arc`, shared with clones. Build a fresh one for every independent agent. * Summaries lose detail. Keep facts the model must not forget in the system prompt. * Debug logs contain the summary prompts and summaries, including file contents. ## Reference [Section titled “Reference”](#reference) The links point at the source on GitHub. * [swiftide-agents-contrib README](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/README.md) * [`ReadFile`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/tools/read_file.rs), [`ExactReplace`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/tools/exact_replace.rs), [`ApplyPatch`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/tools/apply_patch/mod.rs) * [`UpdatePlan`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/tools/update_plan.rs) * [`ToolSummarizer`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/hooks/tool_summarizer.rs), [`ContextCompression`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/hooks/context_compression.rs) * [`LazyExecutor`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/executors/lazy.rs), [`ToolExecutorExt`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/src/executors/traits.rs) * [contrib examples](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib/examples) * [`AgentBuilder::add_tool`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.add_tool), [`AgentBuilder::after_tool`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.after_tool) * [`DefaultContext::from_executor`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.from_executor) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [Feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [Testing agents](/agents/testing/) # Tool executors > Run agent tool commands through a ToolExecutor: a LocalExecutor with a working directory, timeout and environment, scoped executors, streamed output, and model arguments validated before they reach a shell. Your tools run shell commands and file operations through a tool executor instead of spawning processes themselves. You end up with one that works in a chosen directory, stops slow commands, streams their output, and only receives arguments your code has checked. The local executor needs the `swiftide-agents` feature flag: ```sh cargo add swiftide@0.34 --features swiftide-agents ``` ## Local executor [Section titled “Local executor”](#local-executor) ```rust let executor = LocalExecutor::builder() .workdir("/srv/checkout") .default_timeout(Some(Duration::from_secs(30))) .env_remove(vec!["OPENAI_API_KEY".to_owned(), "GITHUB_TOKEN".to_owned()]) .envs(HashMap::from([( "CARGO_TERM_COLOR".to_owned(), "never".to_owned(), )])) .build()?; ``` Source: [`snippets/src/agents/executors.rs#L27-L35`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/executors.rs#L27-L35) The builder sets where and how commands run: * `workdir` sets the directory commands start in. It defaults to `.`. `LocalExecutor::new` only sets this field. * `default_timeout` takes an `Option` for commands without a timeout of their own. * `env_remove` drops variables, here secrets that the model’s commands do not need. * `envs` sets variables. They apply after `env_clear` and `env_remove`. * `env_clear(true)` (not shown) starts every command with an empty environment. That also drops `PATH` and `HOME`, so pass the ones your commands need back through `envs`. The agent gets the executor through its context. [Context and message history](/agents/context-and-history/) shows how `DefaultContext::from_executor` swaps it in. ## Tool executor trait [Section titled “Tool executor trait”](#tool-executor-trait) `ToolExecutor` has three methods: * `exec_cmd` runs a `Command` and returns its `CommandOutput` or a `CommandError`. * `exec_cmd_streaming` also hands each piece of output to a closure. The default implementation reports all output once the command has finished. * `stream_files` returns the files below a path as an `IndexingStream` for an indexing pipeline; see [Loading](/indexing/loading/). A tool reaches the executor with `context.executor()`. `AgentContext::exec_cmd` is deprecated. ### Custom executor [Section titled “Custom executor”](#custom-executor) You write your own tool executor to run commands somewhere else, such as a remote machine, or to put a policy in front of another executor. This one wraps any executor and refuses file writes: ```rust /// Runs commands through another executor, but refuses to write files. #[derive(Clone)] pub struct ReadOnly { inner: E, } impl ReadOnly { fn check(cmd: &Command) -> Result<(), CommandError> { if matches!(cmd, Command::WriteFile { .. }) { return Err(CommandError::ExecutorError(anyhow::anyhow!( "this executor is read-only" ))); } Ok(()) } } #[async_trait] impl ToolExecutor for ReadOnly { async fn exec_cmd(&self, cmd: &Command) -> Result { Self::check(cmd)?; self.inner.exec_cmd(cmd).await } async fn exec_cmd_streaming( &self, cmd: &Command, output: &mut CommandOutputSink<'_>, ) -> Result { Self::check(cmd)?; self.inner.exec_cmd_streaming(cmd, output).await } async fn stream_files( &self, path: &Path, extensions: Option>, ) -> anyhow::Result> { self.inner.stream_files(path, extensions).await } } ``` Source: [`snippets/src/agents/executors.rs#L142-L182`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/executors.rs#L142-L182) The trait uses `#[async_trait]`, so add the `async-trait` crate. The trait extends `DynClone`, so your type must implement `Clone`. `exec_cmd` and `stream_files` are required. `ReadOnly` forwards `exec_cmd_streaming` as well; without it, the default would run `exec_cmd` and report the output only at the end. `Command` is non-exhaustive, so match it with a wildcard arm or `matches!`. Return `CommandError::ExecutorError` when the executor refuses or fails, and `NonZeroExit` when the command ran and failed. An executor that cannot stream can return `IndexingStream::from(anyhow::Error)`, so a loader reports the error instead of panicking. Give the agent your executor the same way as a local one: ```rust let executor: Arc = Arc::new(ReadOnly { inner: LocalExecutor::new("/srv/checkout"), }); let agent = Agent::builder() .llm(&openai) .context(DefaultContext::from_executor(executor)) .build()?; ``` Source: [`snippets/src/agents/executors.rs#L188-L194`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/executors.rs#L188-L194) `ReadOnly` only blocks `Command::WriteFile`. A shell string such as `echo hi > notes.md` still writes, so a real policy also has to check or forbid `Command::Shell`. ## Output streaming [Section titled “Output streaming”](#output-streaming) ```rust let (chunks, mut received) = tokio::sync::mpsc::unbounded_channel(); let printer = tokio::spawn(async move { while let Some(chunk) = received.recv().await { match chunk { CommandOutputChunk::Stdout(bytes) => print!("{}", String::from_utf8_lossy(&bytes)), CommandOutputChunk::Stderr(bytes) => eprint!("{}", String::from_utf8_lossy(&bytes)), } } }); let cmd = Command::shell("cargo test").with_timeout(Duration::from_secs(600)); let result = executor .exec_cmd_streaming(&cmd, &mut |chunk: &CommandOutputChunk| { // The sink runs inside the executor's read loop: hand the chunk off, never block. let _ = chunks.send(chunk.clone()); }) .await; drop(chunks); printer.await?; let output = result?; println!("{} bytes of output", output.len()); ``` Source: [`snippets/src/agents/executors.rs#L73-L94`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/executors.rs#L73-L94) `exec_cmd_streaming` passes each chunk to a sink while the command runs. The sink is a `CommandOutputSink`: a `FnMut(&CommandOutputChunk) + Send` closure. The executor calls the sink from its read loop, so hand each chunk off instead of doing work in it. The local executor streams shell commands and reports file commands once they finish. The return value matches `exec_cmd`. ## Commands [Section titled “Commands”](#commands) You build a `Command` with `Command::shell`, `Command::read_file` or `Command::write_file`, as in the streaming example above and the tool further down. `Command::shell` runs a string with `sh -c`. The file commands skip the shell, so a path never needs quoting. Two methods set options on one command: * `with_current_dir` sets the working directory of the command. * `with_timeout` sets the timeout of the command. It wins over `default_timeout`, also when it is longer. How a relative directory or path resolves is up to the executor. The local executor joins a relative `current_dir` onto its `workdir` and uses an absolute one as it is. File paths resolve against the result. Its `write_file` creates missing parent directories and replaces the file’s content. ## Command output [Section titled “Command output”](#command-output) ```rust let cmd = Command::shell("cargo test").with_current_dir("crates/core"); match context.executor().exec_cmd(&cmd).await { Ok(output) => { let stdout: Vec = output.stdout().flat_map(|b| b.iter().copied()).collect(); println!("{}", String::from_utf8_lossy(&stdout)); } Err(CommandError::NonZeroExit(output)) => { for chunk in output.stderr() { eprintln!("{}", String::from_utf8_lossy(chunk)); } } Err(other) => return Err(other.into()), } ``` Source: [`snippets/src/reference/upgrading.rs#L87-L99`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L87-L99) `CommandOutput` keeps stdout and stderr apart. It holds `CommandOutputChunk::Stdout` and `CommandOutputChunk::Stderr` chunks in the order the executor read them. `stdout()` and `stderr()` iterate over one stream; `to_string_lossy()` and `Display` join both. A `CommandOutput` converts into a `ToolOutput` with the joined text. A failed command returns an error that still carries its output: * `CommandError::NonZeroExit` holds the output of a command that exited with a non-zero code. An input/output (I/O) error, such as reading a missing file, also becomes `NonZeroExit`, with the error message as output. * `CommandError::TimedOut` holds the timeout and the output read so far. For a shell command, the local executor kills the process (its process group on Unix). A file command that times out returns empty output. * `CommandError::ExecutorError` means the executor itself failed. A tool that returns such an error with `?` sends its text to the model and counts toward the tool retry limit. See [Tool errors and retries](/agents/tools/#tool-errors-and-retries). Match `NonZeroExit` yourself when failing is a normal answer, like a failing test. ## Shebang scripts [Section titled “Shebang scripts”](#shebang-scripts) ```rust let script = "#!/usr/bin/env python3\nimport sys\nprint(sys.version_info.major)"; let output = executor.exec_cmd(&Command::shell(script)).await?; println!("python {}", output.to_string_lossy().trim()); ``` Source: [`snippets/src/agents/executors.rs#L43-L45`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/executors.rs#L43-L45) When a shell command starts with `#!`, the local executor starts the interpreter named on that line and writes the rest of the string to its stdin. Everything after the interpreter on the first line is passed as one argument, so `#!/usr/bin/env python3` runs `/usr/bin/env` with `python3`. A model can send a multi-line script without escaping it for `sh -c`. ## Scoped executors [Section titled “Scoped executors”](#scoped-executors) ```rust let checkout = LocalExecutor::new("/srv/checkout"); // Borrowed: one command in /srv/checkout/docs. let docs = checkout.scoped("docs"); let listing = docs.exec_cmd(&Command::shell("ls")).await?; println!("{listing}"); // Owned: commands without an absolute directory or path run in /srv/checkout/crates/core. let executor: Arc = Arc::new(checkout.scoped_owned("crates/core")); let agent = Agent::builder() .llm(&openai) .context(DefaultContext::from_executor(executor)) .build()?; ``` Source: [`snippets/src/agents/executors.rs#L53-L65`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/executors.rs#L53-L65) `ExecutorExt` adds `scoped`, which borrows the executor, and `scoped_owned`, which takes it so it can live as long as an agent. Both return a `ScopedExecutor`: * A command without a working directory runs in the scope. * A relative `current_dir` joins onto the scope. The local executor then joins the result onto its `workdir`. * An absolute `current_dir` is left alone, and so is an absolute file path. A scope does not keep a tool inside it. * `stream_files` joins a relative path onto the scope. The local executor does not add its `workdir` to that path; see the gotchas below. ## Argument validation [Section titled “Argument validation”](#argument-validation) A working directory is not a sandbox: it only chooses where a command starts. An absolute path, a `..` or a `;` in a shell string still reaches the rest of the machine. Parse model arguments into a type that cannot hold a bad value first. The release-notes-agent showcase validates every argument with `Revision` and `RepoPath` in [`src/git.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/git.rs). The same pattern for your own tool: ```rust /// A path inside the executor's working directory: relative and without `..`. #[derive(Debug, Clone)] pub struct WorkspacePath(PathBuf); impl FromStr for WorkspacePath { type Err = anyhow::Error; fn from_str(raw: &str) -> anyhow::Result { let path = Path::new(raw.trim()); ensure!(!path.as_os_str().is_empty(), "path is empty"); for component in path.components() { match component { Component::Normal(_) | Component::CurDir => {} Component::ParentDir => bail!("`{raw}` must not contain `..`"), Component::RootDir | Component::Prefix(_) => { bail!("`{raw}` must be relative to the workspace root") } } } Ok(Self(path.to_path_buf())) } } /// Counts the lines of a file in the workspace. /// /// # Arguments /// - `path`: File path relative to the workspace root. #[swiftide::tool] pub async fn count_lines(context: &dyn AgentContext, path: &str) -> Result { let path: WorkspacePath = path.parse()?; let content = context .executor() .exec_cmd(&Command::read_file(&path.0)) .await?; Ok(ToolOutput::text( content.to_string_lossy().lines().count().to_string(), )) } ``` Source: [`snippets/src/agents/executors.rs#L100-L138`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/executors.rs#L100-L138) Prefer `Command::read_file` and `Command::write_file` over `cat` or `>` in a shell string: no shell ever sees the path. ## Docker executor [Section titled “Docker executor”](#docker-executor) For isolation, the [Docker executor](https://github.com/bosun-ai/swiftide-docker-executor) runs the commands in a container. It is a separate crate; pass it to `DefaultContext::from_executor`, and check that its swiftide version matches yours. ## Gotchas [Section titled “Gotchas”](#gotchas) * `DefaultContext::from_executor` takes `Into>`. A bare `LocalExecutor` does not convert, so wrap it in an `Arc` annotated as `Arc`, as the scoped example does. * `LocalExecutor::stream_files` uses the path as given, not relative to `workdir`. A relative path resolves against your process’s current directory, also through a scope. Scope it with an absolute path or pass an absolute path. * Symlinks inside the working directory can still point elsewhere. Validated paths do not change that; a container does. ## Reference [Section titled “Reference”](#reference) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`Command`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.Command.html) * [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`CommandOutputChunk`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.CommandOutputChunk.html) * [`CommandOutputSink`](https://docs.rs/swiftide/0.34/swiftide/traits/type.CommandOutputSink.html) * [`CommandError`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.CommandError.html) * [`ExecutorExt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ExecutorExt.html) * [`ScopedExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.ScopedExecutor.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`DefaultContext::from_executor`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.from_executor) * [`AgentContext::executor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html#tymethod.executor) ## Next [Section titled “Next”](#next) [Lifecycle hooks](/agents/hooks/) # Lifecycle hooks > Register agent hooks that log tool calls, change completion requests, rewrite tool output and report why an agent stopped. You end up with hooks that print each tool call, change a completion request before it goes out, trim tool output before the model reads it, and report why an agent stopped. A hook is a closure that the agent calls at a fixed point in its loop. The closure returns a boxed future that the agent awaits. Hooks need the `swiftide-agents` feature flag. The release-notes showcase prints one line per tool call and one per result: examples/release-notes-agent/src/progress.rs ```rust /// Prints each tool call and its result on one line. pub fn report_tool_calls(builder: &mut AgentBuilder) { builder .before_tool(|_agent, tool_call| { println!(" → {}", describe_call(tool_call)); Box::pin(async { Ok(()) }) }) .after_tool(|_agent, tool_call, result| { println!(" {} {}", tool_call.name(), describe_result(result)); Box::pin(async { Ok(()) }) }); } ``` Source: [`examples/release-notes-agent/src/progress.rs#L16-L27`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/progress.rs#L16-L27) ## Hook registration [Section titled “Hook registration”](#hook-registration) You pass a closure to an `AgentBuilder` method such as `before_tool`. The closure returns `Box::pin(async { Ok(()) })`, which resolves to `anyhow::Result<()>`. An `async` closure does not satisfy the hook traits. The closure must be `Clone`, `Send` and `Sync`. Share state through an `Arc`. Capturing a value that is not `Clone`, such as an `mpsc::Receiver`, fails to compile. Hooks of one kind run in the order you register them. `add_hook` appends a `Hook` value, and `hooks` replaces all earlier hooks. This is the start of a real run on the Swiftide repository, with `claude-sonnet-5-5` behind an OpenAI-compatible gateway: ```text Drafting release notes for the 7 commits since v0.33.0 with claude-sonnet-5-5 at http://127.0.0.1:8080/v1 (chat completions) → git_log_since v0.33.0 git_log_since ✓ 346 lines → git_diff swiftide-integrations/Cargo.toml 3b1ad22e^..3b1ad22e → git_diff …/src/chat_completion/chat_completion_response.rs 90bcadc7^..90bcadc7 → git_diff swiftide-agents-contrib/Cargo.toml 71fa4587^..71fa4587 → git_diff Cargo.toml 71fa4587^..71fa4587 → git_diff swiftide-integrations/src/aws_bedrock_v2/mod.rs 3b1ad22e^..3b1ad22e → git_diff swiftide/Cargo.toml 71fa4587^..71fa4587 git_diff ✓ 15 lines git_diff ✓ 72 lines git_diff ✓ 75 lines git_diff ✓ No changes to Cargo.toml in 71fa4587^..71fa4587. git_diff ✓ 18 lines git_diff ✓ 32 lines ``` The `→` lines come from `before_tool` and the `✓` lines from `after_tool`. The agent runs `before_tool` for a call and then starts that call, so the tool calls of one completion run concurrently. An earlier tool can already be running while `before_tool` runs for a later one. It then waits for the results and runs `after_tool` in call order. ## Hook points [Section titled “Hook points”](#hook-points) | Builder method | When it runs | What it receives | | ------------------- | --------------------------------------------------------------------------- | --------------------------------------------------------------- | | `before_all` | Once, on the first run, before the system prompt renders and toolboxes load | `&mut Agent` | | `on_start` | On every `query` or `run`, before its first completion | `&mut Agent` | | `before_completion` | Before each completion request | `&mut Agent`, `&mut ChatCompletionRequest` | | `on_stream` | For each stream chunk, when the agent streams | `&Agent`, `&ChatCompletionResponse` | | `after_completion` | After each completion, before the agent acts on it | `&mut Agent`, `&mut ChatCompletionResponse` | | `on_new_message` | When `Agent::add_message` adds a message | `&Agent`, `&mut ChatMessage` | | `before_tool` | Before each tool call starts | `&mut Agent`, `&ToolCall` | | `after_tool` | After each tool call, before its output is recorded | `&mut Agent`, `&ToolCall`, `&mut Result` | | `after_each` | After a completion and its tool calls | `&mut Agent` | | `on_stop` | When the agent records a stop reason | `&mut Agent`, `StopReason`, `Option<&AgentError>` | Since 0.33, most hooks receive `&mut Agent`, so they can stop the agent, read its history or change its tools: ```rust let mut agent = Agent::builder() .llm(&openai) .after_each(|agent: &mut Agent| { Box::pin(async move { if agent.history().await?.len() > 40 { agent.stop("history too long").await; } Ok(()) }) }) .build()?; ``` Source: [`snippets/src/reference/upgrading.rs#L68-L78`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L68-L78) Only the `on_stream` builder method turns streaming on. A `Hook::OnStream` passed to `add_hook` or `hooks` never fires unless streaming is on. [Streaming](/agents/streaming/) covers stream chunks. ## Completion request and response edits [Section titled “Completion request and response edits”](#completion-request-and-response-edits) ```rust let agent = Agent::builder() .llm(openai) .before_completion(|_agent, request| { println!( "sending {} messages and {} tools", request.messages().len(), request.tools_spec().len() ); // Only this request gets the reminder; the message history stays as it is. request .messages .to_mut() .push(ChatMessage::new_user("Reply in at most three sentences.")); Box::pin(async { Ok(()) }) }) .after_completion(|_agent, response| { if let Some(usage) = &response.usage { println!("used {} tokens", usage.total_tokens); } Box::pin(async { Ok(()) }) }) .build()?; ``` Source: [`snippets/src/agents/hooks.rs#L18-L39`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/hooks.rs#L18-L39) A change to `messages` or `tools_spec` applies to this one request. The reminder reaches the model but never enters the message history. Removing an entry from `tools_spec` hides a tool for one turn. In `after_completion`, changing `response.message` or `response.tool_calls` changes what the agent records and which tools it runs. `response.usage` is `None` when the provider reports no usage. ## Tool output rewrites [Section titled “Tool output rewrites”](#tool-output-rewrites) ```rust const KEEP_LINES: usize = 200; let agent = Agent::builder() .llm(openai) .after_tool(|_agent, tool_call, result| { if let Ok(ToolOutput::Text(text)) = result { let lines: Vec<&str> = text.lines().collect(); if lines.len() > KEEP_LINES { let dropped = lines.len() - KEEP_LINES; let tail = lines[dropped..].join("\n"); *text = format!( "[{dropped} lines of {} output cut]\n{tail}", tool_call.name() ); } } Box::pin(async { Ok(()) }) }) .build()?; ``` Source: [`snippets/src/agents/hooks.rs#L47-L65`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/hooks.rs#L47-L65) Whatever you leave in the result is what the agent records in the message history and its tracing spans. The hook runs before the agent looks at errors. Replacing an `Err` with `Ok(ToolOutput::fail(..))` still tells the model what went wrong, but no longer counts toward `tool_retry_limit`. [Tool errors and retries](/agents/tools/#tool-errors-and-retries) explains the retry limit. ## Stop notifications [Section titled “Stop notifications”](#stop-notifications) ```rust let agent = Agent::builder() .llm(openai) .on_stop(|_agent, reason, error| { match error { Some(error) => eprintln!("agent failed: {error}"), None => println!("agent stopped: {reason:?}"), } Box::pin(async { Ok(()) }) }) .build()?; ``` Source: [`snippets/src/agents/hooks.rs#L73-L82`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/hooks.rs#L73-L82) `on_stop` runs once per stop and before the agent records its new state, so read the reason from the argument. [Stopping and structured results](/agents/stopping-and-structured-results/) lists every stop reason. `on_stop` is not a failure signal. The error is `Some` only when a completion step fails. Errors before the loop, such as a failed toolbox load, skip `on_stop`. A tool over `tool_retry_limit` fires `on_stop` with `StopReason::ToolCallsOverLimit` and no error, yet `query` returns `Err`. Check the result of `query` for failures. ## Hooks from methods [Section titled “Hooks from methods”](#hooks-from-methods) ```rust /// Counts tool calls per tool, shared between the hook and the code that reads it. #[derive(Clone, Default)] pub struct ToolCallCounter { counts: Arc>>, } impl ToolCallCounter { /// `use<>` keeps `&self` out of the returned type, so the hook is `'static`. pub fn hook(&self) -> impl AfterToolFn + use<> { let counts = Arc::clone(&self.counts); move |_agent, tool_call, _result| { let mut counts = counts.lock().unwrap_or_else(PoisonError::into_inner); *counts.entry(tool_call.name().to_owned()).or_default() += 1; Box::pin(async { Ok(()) }) } } pub fn count(&self, tool_name: &str) -> usize { let counts = self.counts.lock().unwrap_or_else(PoisonError::into_inner); counts.get(tool_name).copied().unwrap_or_default() } } pub async fn count_tool_calls(openai: &OpenAI) -> anyhow::Result<()> { let counter = ToolCallCounter::default(); let mut agent = Agent::builder() .llm(openai) .after_tool(counter.hook()) .build()?; agent.query("List the files in this directory").await?; println!("stop was called {} times", counter.count("stop")); Ok(()) } ``` Source: [`snippets/src/agents/hooks.rs#L88-L121`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/hooks.rs#L88-L121) Builder methods only accept `'static` hooks. In Rust 2024, `fn hook(&self) -> impl AfterToolFn` captures the lifetime of `&self`. Clone the `Arc` into the closure and add `use<>` to the return type. ## Gotchas [Section titled “Gotchas”](#gotchas) * The future may only borrow the agent. Touch the request, response, tool call or message before `Box::pin`. The `after_tool` result is the exception. * A hook that returns `Err` is logged with `tracing::error!` and the agent carries on. * `agent.stop(..)` lets the current step finish. A stop in `before_completion` still sends that request, and one in `before_tool` still runs the tool. In `before_all` and `on_start`, the agent fires `on_stop` but runs the loop anyway. * In `before_all`, `agent.tools()` does not yet hold Model Context Protocol (MCP) toolbox tools. It can fill template values through `system_prompt_mut`, as [System prompts](/agents/system-prompts/) shows. * `on_new_message` skips the system prompt and user queries, because the agent writes those to the context directly. * A resumed tool call that waited for approval runs again, so a counter like the one above counts it twice. Its tool hooks run before `on_start` while the agent is still stopped, so a stop from them is ignored. [Human in the loop](/agents/human-in-the-loop/) covers approvals. * A call to a tool the agent does not have is skipped, and no tool hook runs for it. ## Reference [Section titled “Reference”](#reference) * [`hooks` module](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/index.html) * [`Hook`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/enum.Hook.html) * [`AfterToolFn`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/trait.AfterToolFn.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [Feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [System prompts](/agents/system-prompts/) # Human in the loop > Wrap a tool in ApprovalRequired so the agent stops with StopReason::FeedbackRequired, then approve or refuse the call and resume the agent with run() or query(). An agent can stop before a tool runs, wait for a person to approve or refuse the call, and then continue where it stopped. The agent types on this page live in `swiftide::agents`, behind the `swiftide-agents` feature flag. The example also uses the `openai` integration and the default `macros` feature flag: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents ``` ## Approval wrapper [Section titled “Approval wrapper”](#approval-wrapper) ```rust // ApprovalRequired is a simple wrapper. You can also implement your own approval // flows by returning a `ToolOutput::FeedbackRequired` in a tool, // you can then use `has_received_feedback` and `received_feedback` on the context // to build your custom workflow. let guess_with_approval = ApprovalRequired::new(guess_a_number()); ``` Source: [`examples/agents_with_human_in_the_loop.rs#L60-L64`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_with_human_in_the_loop.rs#L60-L64) The comment in the example gets one method name wrong: the method on the context that stores feedback is `feedback_received`. ```rust let mut agent = agents::Agent::builder() .llm(&openai) .tools(vec![guess_with_approval]) // Every message added by the agent will be printed to stdout .on_new_message(move |_, msg| { println!("{msg}"); Box::pin(async move { Ok(()) }) }) .limit(5) .build()?; ``` Source: [`examples/agents_with_human_in_the_loop.rs#L68-L78`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_with_human_in_the_loop.rs#L68-L78) Register the wrapper in place of the bare tool. [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) wraps any tool. It keeps the inner tool’s name and spec, so the model sees the same tool. When the model calls it, the wrapper asks the context for feedback on that tool call: * No feedback yet: it returns `ToolOutput::FeedbackRequired(None)`. The agent stops with `StopReason::FeedbackRequired { tool_call, payload }` and records no tool output, so the tool call stays open. * Approved: it runs the inner tool and returns its output. * Refused: it returns the text `This tool call was refused`, which the model reads as the tool output. `query` returns `Ok(())` when the agent stops this way. Read the stop reason to find the call that waits: ```rust // First query the agent, the agent will stop with a reason that feedback is required agent .query("Guess a number between 0 and 100 using the `guess_a_number` tool") .await?; // The agent stopped, lets get the tool call let Some(StopReason::FeedbackRequired { tool_call, .. }) = agent.stop_reason() else { panic!("expected a tool call to approve") }; // Alternatively, you can also get the stop reason from the agent state // agent.state().stop_reason().unwrap().feedback_required().unwrap() // Register that this tool call is ok. println!("Approving number guessing"); agent .context() .feedback_received(tool_call, &ToolFeedback::approved()) .await .unwrap(); // Run the agent again and it will pick up where it stopped. agent.run().await.unwrap(); ``` Source: [`examples/agents_with_human_in_the_loop.rs#L82-L104`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_with_human_in_the_loop.rs#L82-L104) `feedback_received` on the context stores a [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) for that tool call. `run` adds no user message. Before it asks the model for anything, it invokes the tool calls of the last assistant message again, provided no tool output follows that message. This time the wrapper finds the approval and runs the inner tool, and the loop carries on as usual. ## Approval loop [Section titled “Approval loop”](#approval-loop) The release notes showcase wraps its `write_changelog` tool as `OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(..)))` and loops until a maintainer approves a draft or gives up: examples/release-notes-agent/src/session.rs ```rust /// Asks the agent for release notes and loops through review rounds until they are written or /// rejected. pub async fn draft_release_notes( agent: &mut Agent, task: Prompt, reviewer: &mut impl Reviewer, ) -> Result { agent.query(task).await?; let mut written = false; let mut nudged = false; loop { // `write_changelog` is wrapped in `ApprovalRequired`: the agent stops and waits. let tool_call = match agent.stop_reason() { Some(StopReason::FeedbackRequired { tool_call, .. }) => tool_call.clone(), // The custom `stop` tool carries the structured release notes. Some(StopReason::RequestedByTool(_, Some(notes))) if written => { let notes = ReleaseNotes::deserialize(notes) .context("the agent stopped with malformed release notes")?; return Ok(Outcome::Written { notes: Some(notes) }); } _ if written => return Ok(Outcome::Written { notes: None }), // A turn without a tool call, or the step limit. Some(StopReason::NoNewMessages) => { nudge_once(agent, &mut nudged).await?; continue; } other => bail!("the agent stopped without writing the notes: {other:?}"), }; let proposal: WriteChangelogArgs = serde_json::from_str(tool_call.args().unwrap_or_default())?; match reviewer.review(&proposal)? { Verdict::Approve => { agent .context() .feedback_received(&tool_call, &ToolFeedback::approved()) .await?; // Resumes the pending tool call, which now runs, and continues the loop. agent.run().await?; // The write itself can still fail; the agent then hears why and carries on. written |= succeeded(agent, &tool_call).await?; } Verdict::Revise(feedback) => { agent .context() .feedback_received(&tool_call, &ToolFeedback::refused()) .await?; // The refusal is recorded as the tool's output, followed by the feedback. agent.query(feedback).await?; } Verdict::Reject => return Ok(Outcome::Rejected), } } } ``` Source: [`examples/release-notes-agent/src/session.rs#L41-L96`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/session.rs#L41-L96) The arguments of the waiting call are the proposal. The session parses them into the arguments struct that `#[derive(Tool)]` generates and hands them to a `Reviewer`, a small trait with a terminal implementation in `terminal.rs`. After an approval the write can still fail, so the session checks the history for a `ToolOutput::Text` answering that call before it counts the notes as written. ## Captured approval [Section titled “Captured approval”](#captured-approval) This is the approval excerpt of a real run on the Swiftide repository. The model was `claude-sonnet-5-5` behind an OpenAI-compatible gateway, and `y` was piped into stdin, which the reviewer echoes: ```text → write_changelog <32 lines> write_changelog ⏸ waiting for approval ── proposed: 32 lines into a new file, RELEASE_NOTES.md Write these release notes? [y/N] y → write_changelog <32 lines> write_changelog ✓ wrote RELEASE_NOTES.md → stop (version 0.34.0: highlights 2, breaking 1, fixes 1) stop ■ done ``` The `→` lines come from a `before_tool` hook and the result lines from an `after_tool` hook. `⏸ waiting for approval` is the hook seeing `ToolOutput::FeedbackRequired`. The second `→ write_changelog` is the same tool call, invoked again by `run()` after the approval. ## Refusals and revisions [Section titled “Refusals and revisions”](#refusals-and-revisions) A refusal is an answer, not an error. `ToolFeedback::refused()` makes the wrapper return its refusal text, and the agent records it as the tool output. `query(feedback)` queues the person’s message until that output is recorded, so the history reads: tool call, refusal, feedback. The model then drafts again and calls `write_changelog` with a new tool call, which starts a new approval round. Neither `approved()` nor `refused()` takes an argument. To attach a payload, chain `.with_payload(serde_json::Value)` after the constructor. `ApprovalRequired` ignores it; your own tools can read it. ## Draft checks [Section titled “Draft checks”](#draft-checks) `OneEntryPerChange` in the showcase’s `tools.rs` wraps the approval tool. It parses the draft and returns `ToolOutput::fail` when one pull request or commit appears in more than one entry; otherwise it calls the wrapped tool. The model reads the failure and fixes the draft, and the maintainer only sees drafts that pass. A `ToolOutput::fail` does not count toward the tool retry limit ([Tool errors and retries](/agents/tools/#tool-errors-and-retries)). Put checks outside the approval wrapper, so they run before anyone is asked. ## Custom feedback tools [Section titled “Custom feedback tools”](#custom-feedback-tools) A tool can ask for feedback itself by returning `ToolOutput::feedback_required(Some(payload))`. The payload reaches the caller as the `payload` of `StopReason::FeedbackRequired`. The tool needs the `ToolCall` to look up feedback, so it implements [`Tool`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html) by hand; `#[swiftide::tool]` functions and `#[derive(Tool)]` structs only receive the context and their arguments. ```rust #[derive(Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct RefundArgs { /// The order to refund. order_id: String, /// The amount to refund, in cents. amount_cents: u64, } #[derive(Clone)] struct IssueRefund; #[swiftide::reexports::async_trait::async_trait] impl Tool for IssueRefund { async fn invoke( &self, context: &dyn AgentContext, tool_call: &ToolCall, ) -> Result { let args = tool_call .args() .ok_or_else(|| ToolError::missing_arguments("order_id, amount_cents"))?; let refund: RefundArgs = serde_json::from_str(args)?; match context.has_received_feedback(tool_call).await { // No decision yet: stop the agent and show the reviewer what it asks for. None => Ok(ToolOutput::feedback_required(Some(json!({ "order_id": refund.order_id, "amount_cents": refund.amount_cents, })))), // The reviewer can lower the amount in the approval payload. Some(ToolFeedback::Approved { payload }) => { let amount_cents = payload .as_ref() .and_then(|payload| payload["amount_cents"].as_u64()) .unwrap_or(refund.amount_cents); send_refund(&refund.order_id, amount_cents).await?; Ok(ToolOutput::text(format!( "Refunded {amount_cents} cents for order {}", refund.order_id ))) } Some(ToolFeedback::Refused { payload }) => { let reason = payload .as_ref() .and_then(|payload| payload["reason"].as_str()) .unwrap_or("no reason given"); Ok(ToolOutput::text(format!( "A person refused the refund: {reason}" ))) } } } fn name(&self) -> Cow<'_, str> { Cow::Borrowed("issue_refund") } fn tool_spec(&self) -> ToolSpec { ToolSpec::builder() .name("issue_refund") .description("Refund (part of) an order. A person approves every refund.") .parameters_schema(schema_for!(RefundArgs)) .build() .expect("the refund arguments form a strict schema") } } ``` Source: [`snippets/src/agents/human_in_the_loop.rs#L20-L86`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/human_in_the_loop.rs#L20-L86) `DefaultContext` removes the feedback when `has_received_feedback` returns it, so each decision applies to one invocation. The caller can change what the tool does through the payload of the approval: ```rust if let Some(StopReason::FeedbackRequired { tool_call, payload }) = agent.stop_reason() { println!("approve {payload:?}?"); let tool_call = tool_call.clone(); let feedback = ToolFeedback::approved().with_payload(json!({ "amount_cents": 2_500 })); agent .context() .feedback_received(&tool_call, &feedback) .await?; agent.run().await?; } ``` Source: [`snippets/src/agents/human_in_the_loop.rs#L92-L102`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/human_in_the_loop.rs#L92-L102) ## Gotchas [Section titled “Gotchas”](#gotchas) * Have the model call an approval tool on its own. `run` and `query` only resume open tool calls when no tool output follows the assistant message. If the same completion also called another tool, that tool’s output is recorded and the approved call never runs. The showcase prompt says “Call `write_changelog` with the Markdown, on its own.” * The stop reason names only the first tool call that asked for feedback in a completion. * Calling `run()` or `query()` before giving feedback invokes the wrapper again, which asks again, and the agent returns `AgentError::UnfulfilledToolCalls`. A message passed to that `query` stays queued, and the next successful `run()` or `query()` adds it to the history after the tool output. Retry with `run()` so the message does not appear twice. * Hooks see the resumed call twice ([Lifecycle hooks](/agents/hooks/#gotchas)). * Feedback lives in the context, not the message history. To approve in another process, see [Context and history](/agents/context-and-history/). ## Reference [Section titled “Reference”](#reference) * [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) * [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) (`query`, `run`, `stop_reason`, `context`) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) (`feedback_received`, `has_received_feedback`) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) (`with_tool_feedback`) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`Tool`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html), [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html), [`ToolSpec`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpec.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [Section titled “Next”](#next) [Streaming](/agents/streaming/) # MCP tools > Start a Model Context Protocol (MCP) server as a child process, wrap it in an McpToolbox with a whitelist, and add its tools to a Swiftide agent. Here you give an agent a few tools from a Model Context Protocol (MCP) server that runs as a child process and hide the rest from the model. Swiftide speaks MCP through the [`rmcp`](https://docs.rs/rmcp/2.1) crate. Turn on the `mcp` feature flag, which also enables `swiftide-agents`, and add `rmcp` with a client and the transport you use: ```sh cargo add swiftide@0.34 --features openai,mcp cargo add rmcp@2.1 --features client,transport-child-process cargo add tokio --features macros,rt-multi-thread,process,time ``` ## Child process toolbox [Section titled “Child process toolbox”](#child-process-toolbox) The release-notes-agent showcase reads pull requests through the GitHub MCP server when you pass `--mcp`: examples/release-notes-agent/src/mcp.rs ```rust /// Starts `@modelcontextprotocol/server-github` with `npx` and exposes a few of its tools. /// /// Errors explain what is missing, so the caller can carry on without GitHub. pub async fn github_toolbox() -> Result { let token = env::var("GITHUB_PERSONAL_ACCESS_TOKEN") .or_else(|_| env::var("GITHUB_TOKEN")) .context("set GITHUB_PERSONAL_ACCESS_TOKEN (or GITHUB_TOKEN)")?; let server = Command::new("npx").configure(|cmd| { cmd.args(["-y", "@modelcontextprotocol/server-github"]) .env("GITHUB_PERSONAL_ACCESS_TOKEN", token); }); let (transport, _stderr) = TokioChildProcess::builder(server) .stderr(Stdio::null()) .spawn() .context("could not run `npx`; is Node.js installed?")?; let mut toolbox = timeout( Duration::from_mins(1), McpToolbox::try_from_transport(transport), ) .await .context("the GitHub MCP server did not start within a minute")? .context("the GitHub MCP server failed to start")?; toolbox.with_whitelist(GITHUB_TOOLS); // Swiftide names MCP tools `:`; OpenAI rejects ':', see `OpenAiToolNames` below. let toolbox = OpenAiToolNames(toolbox); // Fail early, rather than mid-run, if the server does not offer the tools we expect. let available = toolbox.available_tools().await?; if available.len() != GITHUB_TOOLS.len() { bail!("the GitHub MCP server does not offer {GITHUB_TOOLS:?}"); } Ok(toolbox) } ``` Source: [`examples/release-notes-agent/src/mcp.rs#L18-L52`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/mcp.rs#L18-L52) `McpToolbox::try_from_transport` takes any `rmcp` client transport, such as a child process or a Transmission Control Protocol (TCP) stream. It performs the MCP handshake with a default client info named `swiftide` and returns the toolbox. The handshake waits for `npx` to download the package on a first run, so the showcase puts a one-minute timeout around it. Calling `available_tools` up front turns a missing tool into a startup error instead of a failure in the middle of a run. `src/agent.rs` adds the toolbox with `builder.add_toolbox(github_tools)`. Run it from the swiftide repository. You need `OPENAI_API_KEY`, Node.js, a token in `GITHUB_PERSONAL_ACCESS_TOKEN` or `GITHUB_TOKEN`, and an `origin` remote on GitHub. Without the remote, the showcase skips the GitHub tools: ```sh cargo run -p release-notes-agent -- --mcp ``` ## Running service toolbox [Section titled “Running service toolbox”](#running-service-toolbox) When you need your own client info or capabilities, start the `rmcp` service yourself and hand it over: ```rust // First set up our client info to identify ourselves to the server let client_info = ClientInfo::new( ClientCapabilities::default(), Implementation::new("swiftide-example", env!("CARGO_PKG_VERSION")), ); // Use `rmcp` to start the server let running_service = client_info .serve(TokioChildProcess::new( tokio::process::Command::new("npx").configure(|cmd| { cmd.args(["-y", "@modelcontextprotocol/server-everything"]); }), )?) .await?; // Create a toolbox from the running server, and only use the `add` tool // // A toolbox reveals it's tools to the swiftide agent the first time it starts (if the state of // the agent was pending). You can add as many toolboxes as you want. MCP services are an // implmenentation of a toolbox. A list of tools is another. let everything_toolbox = McpToolbox::from_running_service(running_service) .with_whitelist(["add"]) .to_owned(); ``` Source: [`examples/agents_mcp_tools.rs#L34-L56`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_mcp_tools.rs#L34-L56) `ClientInfo::serve` comes from `rmcp`’s `ServiceExt` and returns a `RunningService`. `McpToolbox::from_running_service` wraps it without contacting the server. Serve from a `ClientInfo`, not from `()`. The filter methods return `&mut Self`, so finish a chain on a temporary with `.to_owned()`. ```rust agents::Agent::builder() .llm(&openai) // Add the toolbox to the agent .add_toolbox(everything_toolbox) // Every message added by the agent will be printed to stdout .on_new_message(move |_, msg| { let msg = msg.to_string(); let tx = tx.clone(); Box::pin(async move { tx.send(msg).unwrap(); Ok(()) }) }) .build()? .query("Use the add tool to add 1 and 2") .await?; ``` Source: [`examples/agents_mcp_tools.rs#L60-L75`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_mcp_tools.rs#L60-L75) `add_toolbox` accepts any owned `ToolBox + 'static`. You can add as many toolboxes as you like. A `Vec>` is a toolbox too. The agent lists a toolbox’s tools when it first starts, after the `before_all` hooks and the system prompt, not when you call `build`. If listing fails, `query` returns `AgentError::ToolBoxFailedToLoad`. After a successful listing the agent keeps the list, so tools the server adds later do not reach it. A failed listing is retried on the next `query`, and the `before_all` hooks run again. The example needs `OPENAI_API_KEY` and Node.js: ```sh cargo run -p swiftide-examples --example agents-mcp ``` It sends the prefixed name `:add` to OpenAI unchanged. If OpenAI rejects it, wrap the toolbox as described under [Tool names](/agents/mcp/#tool-names). ## Filters [Section titled “Filters”](#filters) A toolbox holds one filter, and each call replaces it: * `with_whitelist(names)` keeps only the named tools. * `with_blacklist(names)` keeps every tool except the named ones. * `with_filter(ToolFilter::Whitelist(..))` or `with_filter(ToolFilter::Blacklist(..))` sets a filter built elsewhere, such as from configuration. `ToolFilter` implements `Serialize` and `Deserialize`. Names match the tool names the server reports, exactly and without the server prefix. A whitelisted name that the server does not offer is skipped without an error. A blacklist suits a server where most tools are safe to expose. This toolbox keeps the GitHub server’s read tools and hides three that write: ```rust // `github` is an `McpToolbox` from `try_from_transport` or `from_running_service`. github .with_name("github") .with_blacklist(["create_or_update_file", "push_files", "merge_pull_request"]); let agent = Agent::builder() .llm(openai) .add_toolbox(github) .build()?; ``` Source: [`snippets/src/agents/mcp.rs#L11-L19`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/mcp.rs#L11-L19) `with_name` and `with_blacklist` both return `&mut Self`, so they chain on a `let mut` binding, and the toolbox itself then moves into `add_toolbox`. A tool that a new version of the server adds passes a blacklist, but not a whitelist. ## Tool names [Section titled “Tool names”](#tool-names) Since 0.32, Swiftide registers each MCP tool as `:`. The server name comes from the handshake, and falls back to `mcp` when it is missing or empty. The model sees the prefixed name, and Swiftide calls the server with the original one. `with_name` only names the toolbox in logs and does not change the prefix. OpenAI accepts tool names made of letters, digits, `_` and `-`, so it rejects the `:`. Swiftide does not rewrite the names. The showcase wraps its toolbox in `OpenAiToolNames`, a `ToolBox` in `src/mcp.rs` that replaces every other character with `_`. The model then sees `github-mcp-server_get_pull_request`. Tool names are unique per agent. When a toolbox tool has the same name as a tool the agent already has, the agent keeps the existing one. ## Results [Section titled “Results”](#results) Since 0.33, Swiftide reads the content blocks of `rmcp` 2 tool results: * Text blocks are joined with newlines into a text tool output. Other block kinds, such as images, are dropped. * Only a result with no content blocks at all uses its `structured_content`, serialized as JSON (JavaScript Object Notation). * A successful result with no text left becomes `Tool executed successfully`. That includes a result with only image blocks, even when it carries `structured_content`. * A result with `is_error` set becomes `ToolError::Unknown` with the server’s text, or `Unknown error` when there is none. The model reads the error as the tool output, and the failure counts toward the tool retry limit (`tool_retry_limit`). See [Tool errors and retries](/agents/tools/#tool-errors-and-retries). * Arguments that are not valid JSON return `ToolError::WrongArguments` before the server is called. ## Shutdown [Section titled “Shutdown”](#shutdown) `McpToolbox::cancel` disconnects from the server and stops the `rmcp` service. A toolbox is `Clone`, and clones share one connection with the tools they listed. Bind a clone with `let mut` before you pass the toolbox to `add_toolbox`, and call `cancel().await` on it once the agent is done. After `cancel`, calls to the server’s tools return a tool error that says the server is not running, and listing the tools fails. A second `cancel` logs a warning and returns `Ok`. ## Gotchas [Section titled “Gotchas”](#gotchas) * Swiftide supports MCP tools only. Prompts, resources and sampling from the server are not used. * Use `rmcp` 2. Another major version brings its own `RunningService` type, which `from_running_service` does not accept. * Every tool that passes the filter needs a description, and its input schema must parse as a JSON Schema that Swiftide accepts. Otherwise `query` returns `AgentError::ToolBoxFailedToLoad` for the whole toolbox, so filter out the tools you do not need. * A whitelist limits what the model can call, not what the server can do. Pick read-only tools when the model’s arguments are untrusted, as the showcase does. ## Reference [Section titled “Reference”](#reference) * [`McpToolbox`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/mcp/struct.McpToolbox.html) * [`ToolFilter`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/mcp/enum.ToolFilter.html) * [`AgentBuilder::add_toolbox`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.add_toolbox) * [`ToolBox`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolBox.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`rmcp` transports](https://docs.rs/rmcp/2.1/rmcp/transport/index.html) ## Next [Section titled “Next”](#next) [Providers and reasoning](/agents/providers-and-reasoning/) # How agents work > Build a Swiftide agent, run its completion and tool-call loop, and read why it stopped. After this page you can build an agent, run it, continue its conversation, and read why it stopped. The larger example is the [release-notes-agent showcase](/showcases/release-notes-agent/), which drafts release notes from git history. Agents live in `swiftide::agents`, behind the `swiftide-agents` feature flag: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents cargo add tokio --features full cargo add anyhow ``` ## A minimal agent [Section titled “A minimal agent”](#a-minimal-agent) ```rust // Reads OPENAI_API_KEY from the environment. let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let mut agent = Agent::builder() .llm(&openai) .system_prompt("You answer questions about Rust crates in two sentences.") .limit(10) .build()?; // A failed run, including a tool that fails too often, returns `Err` here. agent.query("What does the tokio crate do?").await?; assert!(agent.is_stopped()); match agent.stop_reason() { Some(StopReason::RequestedByTool(..)) => println!("a tool stopped the agent"), Some(StopReason::NoNewMessages) => println!("no new messages, or the limit was reached"), other => println!("stopped: {other:?}"), } // A second query appends to the same message history and runs the loop again. agent.query("And how does it compare to async-std?").await?; println!("{} messages in the history", agent.history().await?.len()); ``` Source: [`snippets/src/agents/overview.rs#L11-L34`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/overview.rs#L11-L34) `OpenAI::builder()` reads `OPENAI_API_KEY` from the environment. `llm` takes any `ChatCompletion` that is `Clone` and clones it into the agent. `build` fails without it. `system_prompt` takes anything that converts into a `Prompt`. The agent has no system prompt unless you set one. `limit` caps the completions in one run. `query` adds a user message and loops until the agent stops. A stopped agent keeps its message history, so the second `query` continues the same conversation. `history` returns every message in the context. ## The agent loop [Section titled “The agent loop”](#the-agent-loop) ![Agent loop](/diagrams/diagram-agent-loop.svg) The agent sends the message history and the spec of every tool to the model. When the completion holds tool calls, the agent runs the tools and records their outputs in the history. Then it asks the model again. Hooks run around every step. The loop ends when a tool stops the agent, or when the step limit is reached. With `DefaultContext`, it also ends when the last message is an answer without a tool call. `DefaultContext::with_stop_on_assistant(false)` turns that off. A tool that returns `ToolOutput::FeedbackRequired` stops the agent and leaves its tool call open, without an output. A call to a tool the agent does not have is skipped with a warning and gets no output either. ## State and stop reasons [Section titled “State and stop reasons”](#state-and-stop-reasons) `agent.state()` returns a `State`: `Pending` before the first run, `Running` during one, and `Stopped` with a `StopReason` after it. `agent.stop_reason()` tells you why it stopped: * a tool returned `ToolOutput::Stop` (`RequestedByTool`, which holds that tool call) * an answer without a tool call, the step limit, or the end of a `query_once` or `run_once` step (`NoNewMessages`) * a pending approval (`FeedbackRequired`) * a failure (`ToolCallsOverLimit`, `AgentFailed`, `Error`) For `ToolCallsOverLimit` and `Error`, `query` also returns the error, so the `?` in the snippet returns before the `match`. `StopReason` is marked non-exhaustive, so a `match` on it needs a wildcard arm. [Stopping and structured results](/agents/stopping-and-structured-results/) covers each reason. ## Runs [Section titled “Runs”](#runs) `query_once` adds the message and runs one step. `run` and `run_once` do the same without a new message, which is how you resume after an approval. A run on an agent whose state is `Running` returns `AgentError::AlreadyRunning`. Cloning an agent shares its hooks and context, so both clones write to the same message history. Each clone gets its own copy of the tools, so `add_tool` on one does not change the other. The clone copies the state and starts with an empty tool retry counter. To run an agent as one step of a larger workflow, wrap it in a task node; see [Agents as task nodes](/tasks/agents-as-nodes/). ## A larger builder [Section titled “A larger builder”](#a-larger-builder) The release-notes-agent showcase builds its agent in one function: examples/release-notes-agent/src/agent.rs ```rust /// Builds the agent. All tools run through `executor`, whose working directory is the repo. pub fn build( llm: &(impl ChatCompletion + Clone + 'static), executor: Arc, task: &ReleaseTask, github_tools: Option, ) -> Result { let mut builder = Agent::builder(); builder .llm(llm) .context(DefaultContext::from_executor(executor)) .system_prompt(task.system_prompt(github_tools.is_some())) // Our own `stop` tool replaces the default one; call this before `tools`. .without_default_stop_tool() .tools([ git_log_since(), git_diff(), read_file(), // Drafts that describe a change twice go back to the model before anyone reviews them. OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(&task.notes_file))) .into(), ReleaseNotes::stop_tool().into(), ]) .limit(STEP_LIMIT); if let Some(github_tools) = github_tools { builder.add_toolbox(github_tools); } progress::report_tool_calls(&mut builder); progress::stream_draft(&mut builder); Ok(builder.build()?) } ``` Source: [`examples/release-notes-agent/src/agent.rs#L104-L136`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/agent.rs#L104-L136) The showcase passes an `OpenAI` client pointed at any OpenAI-compatible endpoint, but the function accepts any provider. `context` sets where the message history lives and which tool executor runs commands. `DefaultContext::from_executor` keeps the history in memory and uses the executor that `run` creates: examples/release-notes-agent/src/main.rs ```rust let root = Git::new(&LocalExecutor::new(&cli.repo)) .top_level() .await .with_context(|| format!("{} is not a git repository", cli.repo.display()))?; // Every tool the agent runs, runs with the repository as its working directory. let executor: Arc = Arc::new( LocalExecutor::builder() .workdir(&root) .default_timeout(Some(Duration::from_mins(1))) .build()?, ); ``` Source: [`examples/release-notes-agent/src/main.rs#L93-L103`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/main.rs#L93-L103) `Agent::builder()` starts with one tool, `stop`, which the model calls when it is done. The showcase replaces it with a `stop` tool whose arguments are the release notes, so it calls `without_default_stop_tool` before `tools`. The list mixes tools from `#[swiftide::tool]` functions, a tool wrapped in `ApprovalRequired`, and a `StopWithArgs` built from a JavaScript Object Notation (JSON) schema. `add_toolbox` adds the GitHub tools from a Model Context Protocol (MCP) server when you pass `--mcp` in a repository hosted on GitHub. The MCP toolbox needs the `mcp` feature flag, which the showcase enables. The two `progress` functions take `&mut AgentBuilder` and register hooks, which keeps the builder short. The [hooks page](/agents/hooks/) shows a captured run of this agent with `claude-sonnet-5-5`. ### Agent names [Section titled “Agent names”](#agent-names) `AgentBuilder::name` labels an agent, and `Agent::name()` reads the label back: ```rust let agent = Agent::builder() .llm(openai) .name("release-checker") .build()?; assert_eq!(agent.name(), "release-checker"); // Without `name`, every agent is called "unnamed_agent". let unnamed = Agent::builder().llm(openai).build()?; assert_eq!(unnamed.name(), "unnamed_agent"); ``` Source: [`snippets/src/agents/overview.rs#L42-L50`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/overview.rs#L42-L50) The name defaults to `unnamed_agent`. Swiftide only shows it in the agent’s `Debug` output, so use it in your own logs or to tell agents apart in a multi-agent setup. ## Agent and application responsibilities [Section titled “Agent and application responsibilities”](#agent-and-application-responsibilities) Besides running the loop, the agent: * renders the system prompt on the first run and appends it to the history, after any messages that `BeforeAll` hooks or an existing history put there * removes duplicate keys from the JSON arguments of a tool call * wraps queries, steps, tools and hooks in tracing spans ([Tracing](/observability/tracing/)) * lists toolbox tools when it first starts You provide the model, the tools, the system prompt, the tool executor, the hooks and, when you want a typed result, the schema of the stop tool. ## Gotchas [Section titled “Gotchas”](#gotchas) * A hook that returns an error is logged with tracing. The agent carries on. * When a completion holds only calls to tools the agent does not have, no output is recorded. The next `query` returns `AgentError::UnfulfilledToolCalls`. * Some errors return from `query` without setting a stop reason, for example a message history error or a tool task that fails to join. The agent then stays `Running`, and the next run returns `AgentError::AlreadyRunning`. * A clone of a `Pending` agent renders the system prompt again on its first run and appends it to the shared history. ## Reference [Section titled “Reference”](#reference) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`State`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.State.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) ## Next [Section titled “Next”](#next) [Tools](/agents/tools/) # Providers and reasoning > Run one agent definition on OpenAI, Azure, Anthropic, Ollama, AWS Bedrock or any OpenAI-compatible gateway, keep reasoning items across turns, and know how each provider treats tool schemas. You end up with one agent definition that runs on whichever provider you pick at startup, with the model’s reasoning carried from one turn to the next. ```rust /// Builds the same agent for any chat completion client. fn changelog_agent(llm: &(impl ChatCompletion + Clone + 'static)) -> anyhow::Result { let agent = Agent::builder() .llm(llm) .system_prompt("You summarize git history for release notes.") .limit(20) .build()?; Ok(agent) } ``` Source: [`snippets/src/agents/providers.rs#L14-L22`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/providers.rs#L14-L22) ```rust let mut agent = match std::env::var("LLM_PROVIDER").as_deref() { Ok("anthropic") => changelog_agent( &Anthropic::builder() .default_prompt_model("claude-sonnet-4-5") .build()?, )?, Ok("ollama") => changelog_agent(&Ollama::builder().default_prompt_model("qwen3").build()?)?, Ok("azure") => { let config = AzureConfig::new() .with_api_base("https://my-resource.openai.azure.com") .with_api_version("2024-10-21") .with_deployment_id("gpt-4o-mini") .with_api_key(std::env::var("AZURE_OPENAI_API_KEY")?); let azure: GenericOpenAI = GenericOpenAIBuilder::default() .client(async_openai::Client::with_config(config)) .default_prompt_model("gpt-4o-mini") .build()?; changelog_agent(&azure)? } _ => changelog_agent( &OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?, )?, }; agent .query("Summarize the commits since the last tag.") .await?; ``` Source: [`snippets/src/agents/providers.rs#L28-L56`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/providers.rs#L28-L56) `AgentBuilder::llm` takes a reference to any client that implements `ChatCompletion + Clone + 'static`. It stores a boxed clone, so `Agent` has no type parameter for the provider. The tools, hooks and system prompt stay the same whichever client you pass. The Azure client needs the `async-openai` crate as a direct dependency, because Swiftide does not re-export its client type. `api_base` and `api_key` on the builder only configure the `OpenAI` client, so Azure passes a full client to `client(..)`: ```sh cargo add async-openai@0.41 --no-default-features ``` Use the same version Swiftide depends on, or the client types do not match. ## Chat providers [Section titled “Chat providers”](#chat-providers) Each chat client sits behind its own feature flag. The snippet above needs `swiftide-agents` and three provider flags: ```sh cargo add swiftide@0.34 --features swiftide-agents,openai,anthropic,ollama ``` Every chat client implements `ChatCompletion`. [Providers](/models/overview/#providers) lists each client with its feature flag, and [OpenAI-compatible endpoints](/models/overview/#openai-compatible-endpoints) covers gateways and base URLs. The ask-your-codebase showcase builds its client from a base URL, a key and a flag for the Responses API, so the same binary runs against OpenAI or a gateway. It sets them with `api_base` and `api_key`: examples/ask-your-codebase/src/llm.rs ```rust impl LlmEndpoint { /// Builds a client for this endpoint. /// /// Rate limits and transient errors are retried with exponential backoff, which matters when /// indexing fires many prompts concurrently. After `retry_budget` the error is returned, so /// an unreachable or broken endpoint fails instead of hanging. pub fn client(&self, retry_budget: Duration) -> Result> { let openai = OpenAI::builder() .api_base(&self.base_url) .api_key(&self.api_key) .default_prompt_model(&self.model) .use_responses_api(matches!(self.api, OpenAiApi::Responses)) .build()?; let backoff = BackoffConfiguration { max_elapsed_time_sec: retry_budget.as_secs(), ..BackoffConfiguration::default() }; Ok(Arc::new(LanguageModelWithBackOff::new(openai, backoff))) } } ``` Source: [`examples/ask-your-codebase/src/llm.rs#L79-L100`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/llm.rs#L79-L100) ## Bedrock agents [Section titled “Bedrock agents”](#bedrock-agents) `AwsBedrock` talks to the Bedrock Converse API. It reads AWS credentials and the region from your usual Amazon Web Services (AWS) configuration, such as a command-line interface (CLI) profile or environment variables: ```rust let bedrock = AwsBedrock::builder() .default_prompt_model("global.anthropic.claude-sonnet-4-6") .build()?; ``` Source: [`examples/aws_bedrock_agent.rs#L61-L63`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L61-L63) The model ID here is a global inference profile. The agent itself looks like any other: ```rust let mut agent = agents::Agent::builder() .llm(&bedrock) .tools(vec![current_utc_time(), format_timestamp()]) .on_new_message(|_, msg| { let rendered = msg.to_string(); Box::pin(async move { println!("{rendered}"); Ok(()) }) }) .limit(6) .build()?; ``` Source: [`examples/aws_bedrock_agent.rs#L67-L78`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L67-L78) Setup and credentials are on [AWS Bedrock](/integrations/aws-bedrock/). ## Reasoning items [Section titled “Reasoning items”](#reasoning-items) On `OpenAI`, reasoning models return their reasoning only through the Responses API. `AwsBedrock` returns signed Claude reasoning as `ReasoningItem` values without extra setup. Reasoning on `OpenAI` needs the Responses API and a reasoning effort; [Responses API and reasoning](/models/overview/#responses-api-and-reasoning) covers the client options. ```rust // Reasoning models require the Responses API. Enabling reasoning effort also asks for a // summary and encrypted reasoning content (enabled by default). If your OpenAI org is not // verified for reasoning access, summaries may be absent. Disable with // `reasoning_features(false)` if desired. let openai = OpenAI::builder() .default_prompt_model("o3-mini") .default_options(Options::builder().reasoning_effort(ReasoningEffort::Low)) .use_responses_api(true) .build()?; ``` Source: [`examples/responses_api_reasoning.rs#L34-L42`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api_reasoning.rs#L34-L42) Each `ReasoningItem` has a `summary` list and an optional `encrypted_content`. This helper joins the summaries: ```rust fn reasoning_summary(reasoning: Option<&[ReasoningItem]>) -> Option { let summary = reasoning .unwrap_or(&[]) .iter() .flat_map(|item| item.summary.iter()) .cloned() .collect::>() .join("\n"); if summary.is_empty() { None } else { Some(summary) } } ``` Source: [`examples/responses_api_reasoning.rs#L10-L24`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api_reasoning.rs#L10-L24) The agent reads them in an `after_completion` hook: ```rust let mut agent = Agent::builder() .llm(&openai) .on_new_message(|_, message| { if let ChatMessage::Assistant(content, _) = message && let Some(content) = content.as_deref() { println!("Assistant: {content}"); } Box::pin(async move { Ok(()) }) }) .after_completion(|_, response| { if let Some(summary) = reasoning_summary(response.reasoning.as_deref()) { println!("Reasoning summary:\n{summary}"); } let has_encrypted = response .reasoning .as_ref() .is_some_and(|items| items.iter().any(|item| item.encrypted_content.is_some())); println!("Encrypted reasoning content present: {has_encrypted}"); Box::pin(async move { Ok(()) }) }) .build()?; ``` Source: [`examples/responses_api_reasoning.rs#L46-L68`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api_reasoning.rs#L46-L68) After the `after_completion` hooks run, the agent adds every item in `response.reasoning` to the message history as a `ChatMessage::Reasoning`. It adds them before the assistant message, through `Agent::add_message`, so `on_new_message` hooks see them too. A hook that empties `response.reasoning` keeps the items out of the history. ## Reasoning replay [Section titled “Reasoning replay”](#reasoning-replay) What happens to those items on the next request depends on the client: * The Responses API client replays them only when `reasoning_effort` is set on that client. It then sends back every item with a non-empty `encrypted_content` and sets `store` to false, so the reasoning lives in your message history and survives a saved and restored history (see [Context and history](/agents/context-and-history/)). Without `reasoning_effort`, it drops every `ChatMessage::Reasoning`. * `AwsBedrock` replays an item only when it has non-empty `content` and a non-empty `encrypted_content`, which it sends as the signature. It drops anything else. * The Chat Completions client and `Anthropic` leave `ChatMessage::Reasoning` out of their requests. ## Bedrock reasoning effort [Section titled “Bedrock reasoning effort”](#bedrock-reasoning-effort) `AwsBedrock` has its own `Options` and its own `ReasoningEffort` enum in `swiftide::integrations::aws_bedrock_v2`. Set `reasoning_effort` in those options and pass them to `default_options` on the builder. For the Opus 4.5 beta header and the other fields, see [AWS Bedrock settings](/models/overview/#aws-bedrock-settings). ## Tool schema policies [Section titled “Tool schema policies”](#tool-schema-policies) `ToolSpec::builder().build()` checks every schema against Swiftide’s strict contract (see [Tools](/agents/tools/)) and rewrites what it can: a nullable type such as `["string", "null"]` becomes an `anyOf` with a null branch, a two-entry `oneOf` with a null branch becomes an `anyOf`, annotations next to a `$ref` are dropped, and every object gets `additionalProperties: false`. Each provider then applies its own policy when it builds a request. Only `AwsBedrock` lets you change it. * `OpenAI` and the other `GenericOpenAI` clients remove `$schema` and Rust integer formats such as `int32`, and list every property in `required`. They reject any remaining `oneOf` or array-valued `type`, and send the schema with `strict: true`. * `Anthropic` and `AwsBedrock` send the strict schema unchanged. `AwsBedrock` marks tools strict unless you set `tool_strict` to `false` in its `Options`. ## Gotchas [Section titled “Gotchas”](#gotchas) * The OpenAI schema policy runs when the request is built, not when the tool is built. A schema it rejects fails the first completion with `AgentError::CompletionsFailed`, wrapping a permanent error that says `tool schema must be OpenAI compatible`. * Gateways do not always speak the Responses API. Against the gateway used for the showcase runs, the Responses API fails because the gateway streams events with `"output": null`, which the client cannot parse. Use it with OpenAI itself. * OpenAI rejects the `:` in Model Context Protocol (MCP) tool names; see [Tool names](/agents/mcp/#tool-names). * Reasoning items are tied to the provider that made them. Bedrock items carry a Bedrock signature in `encrypted_content`, and an OpenAI Responses client with `reasoning_effort` set sends that to OpenAI as encrypted reasoning. OpenAI items usually have no `content`, so `AwsBedrock` drops them silently. Remove `ChatMessage::Reasoning` entries from the message history before you switch providers. * `AwsBedrock::builder().build()` loads the AWS configuration with `block_in_place`, so it needs a multi-threaded Tokio runtime. A default `#[tokio::test]` or `flavor = "current_thread"` panics. Pass your own client with `.client(..)` otherwise. * A client’s `on_usage` callback counts every call on that client, including other agents that share it; see [Usage callbacks](/models/errors-backoff-and-usage/#usage-callbacks). ## Reference [Section titled “Reference”](#reference) * [`AgentBuilder::llm`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.llm) * [`AgentBuilder::after_completion`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.after_completion) * [`AgentBuilder::on_new_message`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.on_new_message) * [`Agent::add_message`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.add_message) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`GenericOpenAIBuilder::use_responses_api`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.use_responses_api) * [`Options` (OpenAI)](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`AzureConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.AzureConfig.html) * [`Anthropic`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Anthropic.html) * [`Ollama`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/struct.Ollama.html) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`Options` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.Options.html) * [`ReasoningEffort` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/enum.ReasoningEffort.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ReasoningItem`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ReasoningItem.html) * [`ToolSpec`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpec.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [Section titled “Next”](#next) [Contrib tools and hooks](/agents/contrib/) # Stopping and structured results > Replace the stop tool with StopWithArgs and a JSON schema to get a typed result, report failures with AgentCanFail, cap runs with limit and tool_retry_limit, and handle every StopReason. Your agent ends its run by handing your code a typed value, such as `ReleaseNotes`, or a typed report of why it failed. Your code can also tell why any other run ended. `Agent::builder()` starts with one tool, named `stop`, which takes no arguments and ends the run with `StopReason::RequestedByTool(tool_call, None)`. With the default context, an answer without a tool call also ends the run, with `StopReason::NoNewMessages`. ## Structured results [Section titled “Structured results”](#structured-results) The examples need the `swiftide-agents` feature flag and your own `schemars` 1.x, `serde` and `serde_json`: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents cargo add schemars@1 serde serde_json --features serde/derive ``` The [release-notes-agent showcase](/showcases/release-notes-agent/) returns its notes as data. It describes them as a type and turns the type’s JSON (JavaScript Object Notation) Schema into a stop tool: examples/release-notes-agent/src/notes.rs ```rust /// Structured release notes. The agent hands these over when it calls `stop`. #[derive(Debug, Serialize, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] pub struct ReleaseNotes { /// The version being released, without a leading `v`. pub version: String, /// One or two sentences on what this release is about, written for users. pub summary: String, /// New features and notable improvements. pub highlights: Vec, /// Changes that require users to update their code. Empty if there are none. pub breaking_changes: Vec, /// Bug fixes users may notice. pub fixes: Vec, } #[derive(Debug, Serialize, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] pub struct Item { /// Short, user-facing title. pub title: String, /// What changed and why it matters, in one or two sentences. pub details: String, /// Pull requests or commits as plain text, e.g. `#1207` or `90bcadc7`, not Markdown links. pub references: Vec, } impl ReleaseNotes { /// A `stop` tool whose arguments must match [`ReleaseNotes`]. /// /// With a custom schema the arguments are forwarded as-is, and end up in /// `StopReason::RequestedByTool`. pub fn stop_tool() -> StopWithArgs { StopWithArgs::with_parameters_schema(schema_for!(ReleaseNotes)) } } ``` Source: [`examples/release-notes-agent/src/notes.rs#L12-L47`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/notes.rs#L12-L47) `StopWithArgs::with_parameters_schema` builds a tool that is also named `stop`. The model calls it with arguments that follow the schema. The doc comments on the fields end up in the schema, so the model reads them as instructions. `deny_unknown_fields` gives both objects `additionalProperties: false`. With a custom schema, the tool passes the whole argument object on as the payload. The agent stops with `StopReason::RequestedByTool(tool_call, Some(payload))`, where `payload` is a `serde_json::Value`. Match that arm and turn the payload into your type with `ReleaseNotes::deserialize(payload)` (a `&serde_json::Value` is a deserializer) or `serde_json::from_value(payload.clone())`. [Human in the loop](/agents/human-in-the-loop/) shows the whole loop. `StopWithArgs::default()` is the other form. Its schema has a single `output` string, and the payload is that string. `tools` keeps the default `stop`, so call `without_default_stop_tool` before `tools`, or add yours with `add_tool`. In this example, `stop_tool` is `StopWithArgs::with_parameters_schema(stop_schema())`: ```rust let mut builder = Agent::builder(); builder .llm(&openai) .system_prompt( "You are a workflow finisher. Summarize completed work and recommend next actions. \ When done, call the `stop` tool with the provided JSON schema. Always include \ `details`, using null when there is nothing to add. Never fabricate task status \ values outside the schema.", ) .without_default_stop_tool() .tools([stop_tool.clone()]) .on_stop(|_, reason, _| { Box::pin(async move { if let StopReason::RequestedByTool(_, payload) = reason && let Some(payload) = payload { println!( "agent stopped with structured payload:\n{}", to_string_pretty(&payload).unwrap_or_else(|_| payload.to_string()), ); } Ok(()) }) }); ``` Source: [`examples/stop_with_args_custom_schema.rs#L59-L82`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/stop_with_args_custom_schema.rs#L59-L82) The `on_stop` hook receives the same `StopReason`, so you can read the payload there instead of after `query` returns. The showcase’s output with `claude-sonnet-5-5`, through an OpenAI-compatible gateway, ends with the stop call and the deserialized notes: ```text → stop (version 0.34.0: highlights 2, breaking 1, fixes 1) stop ■ done ✔ RELEASE_NOTES.md 0.34.0 — This release adds the opt-in `swiftide-agents-contrib` crate and Perl support for tree-sitter. It also fixes duplicated text in OpenAI streaming and resolves security advisories, which removes AWS Console login from the Bedrock client. highlights (2) • Agent tools, hooks and executors in `swiftide-agents-contrib` [#1207] • Perl support for tree-sitter [#1198] breaking changes (1) • AWS Console login removed from the Bedrock client [#1215] fixes (1) • Repeated text in OpenAI streaming [#1216] ``` ## Failure reports [Section titled “Failure reports”](#failure-reports) `AgentCanFail` is a second control tool, named `task_failed`. The agent does not add it unless you pass it to `tools`. `AgentCanFail::default()` takes a `reason` string, and the agent stops with `StopReason::AgentFailed(Some(reason))`. `query` still returns `Ok`, so you check the stop reason. `AgentCanFail::with_parameters_schema` accepts your own schema: ```rust /// What the agent reports when it cannot finish. #[derive(Debug, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] struct FailureReport { /// What went wrong, in one sentence. summary: String, /// What a person has to fix before the task can be retried. blocked_by: Vec, } let mut agent = Agent::builder() .llm(&openai) .tools([AgentCanFail::with_parameters_schema(schema_for!( FailureReport ))]) .build()?; agent.query("Restore last night's database backup.").await?; // With a custom schema the arguments arrive as one JSON-encoded string. if let Some(StopReason::AgentFailed(Some(Value::String(json)))) = agent.stop_reason() { let report: FailureReport = serde_json::from_str(json)?; println!("failed: {}", report.summary); for blocker in &report.blocked_by { println!(" blocked by: {blocker}"); } } ``` Source: [`snippets/src/agents/stopping.rs#L57-L83`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/stopping.rs#L57-L83) With a custom schema the tool serializes the arguments back to text. The payload is a `Value::String` that holds JSON, so you parse it a second time with `serde_json::from_str`. ## Stop reasons [Section titled “Stop reasons”](#stop-reasons) This run caps itself at 20 completions, lets one tool call fail twice, adds `task_failed`, and stops itself from a hook when the message history grows too long: ```rust let mut agent = Agent::builder() .llm(&openai) .system_prompt("Check that the release branch builds. Call `stop` when you are done.") .tools([AgentCanFail::default()]) .limit(20) .tool_retry_limit(2) .after_each(|agent| { Box::pin(async move { if agent.history().await?.len() > 200 { agent.stop("the history grew past 200 messages").await; } Ok(()) }) }) .build()?; let result = agent.query("Is the release branch green?").await; match agent.stop_reason() { Some(StopReason::RequestedByTool(_call, payload)) => println!("done: {payload:?}"), Some(StopReason::AgentFailed(reason)) => println!("the model gave up: {reason:?}"), Some(StopReason::ToolCallsOverLimit(call)) => println!("`{}` kept failing", call.name()), Some(StopReason::FeedbackRequired { tool_call, .. }) => { println!("`{}` waits for approval", tool_call.name()); } Some(StopReason::Error) => println!("the run failed"), Some(StopReason::NoNewMessages) => println!("turn ended without a tool call, or limit hit"), Some(StopReason::Other(reason)) => println!("stopped by our hook: {reason}"), Some(other) => println!("stopped: {other:?}"), None => println!("the agent did not stop; see the returned error"), } // `ToolCallsOverLimit` and `Error` also come back as an `Err`. result?; ``` Source: [`snippets/src/agents/stopping.rs#L16-L49`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/stopping.rs#L16-L49) `limit` caps the number of completions in one run. The counter starts at zero with every `query` or `run`. When it is reached, the agent logs a warning and stops with `StopReason::NoNewMessages`, and `query` returns `Ok`. No limit is set by default. `tool_retry_limit` caps how often the same tool call may fail before the agent stops with `StopReason::ToolCallsOverLimit` and `query` returns `AgentError::ToolError`. [Tools](/agents/tools/) explains how failures are counted. Each variant means something different for your code: * `RequestedByTool(tool_call, payload)`: a tool returned `ToolOutput::Stop`, usually a `stop` tool. `payload` is `None` for the default tool, the `output` value for `StopWithArgs::default()`, and the argument object for a custom schema. Treat it as success. * `AgentFailed(payload)`: a tool returned `ToolOutput::AgentFailed`, usually `task_failed`. Show the reason to a person, or retry with more context. * `ToolCallsOverLimit(tool_call)`: one call failed too often. `query` also returns the error. Fix the tool or its description, or raise `tool_retry_limit`. * `FeedbackRequired { tool_call, payload }`: a tool waits for approval. Record a `ToolFeedback` on the context and call `run`, as [Human in the loop](/agents/human-in-the-loop/) shows. * `Error`: a step failed, for example a completion request, and `query` returns the `AgentError`. * `NoNewMessages`: the model answered without a tool call, sent an empty reply, or the run reached `limit`. `query_once` and `run_once` also end with it after one step, unless a tool stopped the agent first. A new `query` continues the same message history. * `Other(text)`: your code stopped the agent. `agent.stop("text")` accepts any string. Hooks that receive `&mut Agent`, such as `after_each` and `after_tool`, can call it. `StopReason` is marked `#[non_exhaustive]`, so a `match` needs a wildcard arm. `agent.stop_reason()` returns `None` while the agent is `Pending` or `Running`. That includes a run that returned an error without recording a stop. Each variant also has an `is_*` method, and most have an `as_*` accessor such as `as_requested_by_tool`. ### Tool outputs [Section titled “Tool outputs”](#tool-outputs) `ToolOutput` has the same kind of accessors. `as_text` and `as_fail` return the text of a `Text` or `Fail` output; `as_stop` and `as_agent_failed` return the payload of a `Stop` or `AgentFailed` output. This `after_tool` hook reads them, and turns a failed `deploy` call into a failed run with `ToolOutput::agent_failed`: ```rust let agent = Agent::builder() .llm(openai) .after_tool(|_agent, tool_call, result| { if let Ok(output) = result { if let Some(text) = output.as_text() { println!("`{}` returned {} bytes", tool_call.name(), text.len()); } else if let Some(error) = output.as_fail().map(str::to_owned) { // A failed deploy ends the run instead of going back to the model. if tool_call.name() == "deploy" { *output = ToolOutput::agent_failed(json!({ "deploy": error })); } } else if let Some(payload) = output.as_stop() { println!("stopping with {payload}"); } else if let Some(reason) = output.as_agent_failed() { println!("giving up: {reason}"); } } Box::pin(async { Ok(()) }) }) .build()?; ``` Source: [`snippets/src/agents/stopping.rs#L91-L110`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/stopping.rs#L91-L110) The agent checks the output after the `after_tool` hooks run, so the rewritten output stops it with `StopReason::AgentFailed(Some(payload))`. `as_stop` and `as_agent_failed` return `None` when the output carries no payload, as with the default `stop` tool; match on `ToolOutput::Stop` or `ToolOutput::AgentFailed` to test the variant itself. ## Gotchas [Section titled “Gotchas”](#gotchas) * To tell the cases of `NoNewMessages` apart, check whether the last assistant message has tool calls, or turn off the text-answer stop with `with_stop_on_assistant(false)` (see [Context and history](/agents/context-and-history/)). * `without_default_stop_tool` takes effect when you call `tools`, so call it first. * A `StopWithArgs` or `AgentCanFail` call whose arguments are not valid JSON is a tool error. It goes back to the model and counts toward `tool_retry_limit`. The default `Stop` ignores its arguments. * `to_string_pretty` on an `AgentCanFail` payload prints a quoted, escaped string; parse it with `serde_json::from_str` first. * Once the agent has stopped, `Agent::stop` does nothing. The first stop reason stays, and `on_stop` fires once. Tools the model called in the same completion as `stop` still run. * The payload is whatever the model sent. Handle a failed deserialization, as the showcase does. ## Reference [Section titled “Reference”](#reference) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`State`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.State.html) * [`Stop`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.Stop.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`AgentCanFail`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.AgentCanFail.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [Section titled “Next”](#next) [Human in the loop](/agents/human-in-the-loop/) # Streaming > Turn on streaming with on_stream, read the text and the tool call arguments in each stream chunk, and switch OpenAI-compatible clients to deltas only. You end up with an agent that prints the model’s text, and the arguments of its tool calls, while the model is still writing them. Streaming changes only how a completion arrives. The agent needs the `swiftide-agents` feature flag, and each model client needs its own: `openai`, `anthropic` or `aws-bedrock`. None of them is on by default. ```sh cargo add swiftide@0.34 --features swiftide-agents,openai ``` ## Streaming switch [Section titled “Streaming switch”](#streaming-switch) ```rust agents::Agent::builder() .llm(&openai) .on_stream(|_agent, response| { // We print the message chunk if it exists. Streamed responses also include // the full response (without tool calls) in `message` and an `id` to map them to // previous chunks for convenience. // // The agent uses the full assembled response at the end of the stream. if let Some(delta) = &response.delta { print!( "{}", delta .message_chunk .as_deref() .map(str::to_string) .unwrap_or_default() ); }; // If `stream_full` is disabled, response.message() will be the accumulated response // response.message() Box::pin(async move { Ok(()) }) }) // Every message added by the agent will be printed to stdout .on_new_message(move |_, msg| { let msg = msg.to_string(); Box::pin(async move { println!("\n---\nFinal message:\n {msg}"); Ok(()) }) }) .limit(5) .build()? .query("Why is the rust programming language so good?") .await?; ``` Source: [`examples/streaming_agents.rs#L24-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/streaming_agents.rs#L24-L58) `openai` is an `OpenAI` client built with `stream_full(false)`; [Deltas only](#deltas-only) shows it. Registering `on_stream` turns streaming on. The agent then calls `complete_stream` on the model instead of `complete`, and runs the hook once for every stream chunk. Without a hook, `streaming(true)` on the builder streams as well. The hook receives `&Agent` and the chunk, a `ChatCompletionResponse`. The example prints `delta.message_chunk`, the text that arrived with this chunk. It never flushes stdout, so the text can arrive in bursts. The `on_new_message` hook prints every message the agent adds to the message history. For the assistant, that is the assembled message after the stream has ended. ## Stream chunks [Section titled “Stream chunks”](#stream-chunks) Each stream chunk is a whole `ChatCompletionResponse`. With the default settings it carries: * `id`, the same for every chunk of one completion. * `message`, the text accumulated so far. * `tool_calls`, `usage` and `reasoning`, the tool calls rebuilt, the token counts and the reasoning items so far. * `delta.message_chunk`, the text that arrived with this chunk. * `delta.tool_calls_chunk`, a map from a per-client index to a `ToolCallAccum` with `id`, `name` and `arguments`. `arguments` holds everything received so far, not only the newest fragment. The map stays on every later chunk of the completion, so remember what you already showed. The key differs by client. OpenAI chat completions use the tool call’s index, the Responses API the output item index. Anthropic and Bedrock use the content block index, which also counts text blocks. Match calls on `id` or `name`, not on the key. When the stream ends, the agent keeps the last chunk and uses it as the completion. It records that chunk’s message in the message history and runs its tool calls. A stream that yields nothing fails with `AgentError::EmptyStream`. A client without its own `complete_stream` sends one chunk with the whole response and no `delta`. ## Tool argument previews [Section titled “Tool argument previews”](#tool-argument-previews) The release-notes-agent showcase streams its draft. The release notes are the `markdown` argument of the `write_changelog` tool, so they arrive as tool call arguments, not as text: examples/release-notes-agent/src/progress.rs ```rust /// Streams the model's text, and the release notes while they are being drafted. pub fn stream_draft(builder: &mut AgentBuilder) { let preview = Arc::new(Mutex::new(DraftPreview::default())); // Registering `on_stream` switches the agent to streaming completions. builder.on_stream(move |_agent, response| { let Some(delta) = &response.delta else { return Box::pin(async { Ok(()) }); }; let mut out = stdout().lock(); if let Some(text) = &delta.message_chunk { let _ = write!(out, "{text}"); } // The notes arrive as the (JSON) argument of `write_changelog`; show them as they // are written. let drafts = delta.tool_calls_chunk.iter().flat_map(HashMap::values); for call in drafts.filter(|call| call.name.as_deref() == Some(WRITE_CHANGELOG)) { if let (Some(id), Some(args)) = (&call.id, &call.arguments) { let mut preview = preview.lock().unwrap_or_else(PoisonError::into_inner); if let Some(text) = preview.advance(id, args) { let _ = write!(out, "{text}"); } } } let _ = out.flush(); Box::pin(async { Ok(()) }) }); } ``` Source: [`examples/release-notes-agent/src/progress.rs#L31-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/progress.rs#L31-L58) The hook writes `message_chunk` to stdout. It then walks the entries of `tool_calls_chunk`, keeps the `write_changelog` call, and hands its id and accumulated arguments to `DraftPreview`. It flushes stdout after every chunk. [`DraftPreview`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/draft.rs) lives in `draft.rs`. Every chunk repeats the whole argument text, so the preview tracks how much it has shown. It decodes the complete part of the `markdown` value in the growing JSON (JavaScript Object Notation) text, stopping before an unfinished escape sequence. It returns only the text it has not returned before. A new call id starts a fresh preview. The hook is a `Fn` closure, so it cannot change what it captures, and it must be `Clone`. State that changes from chunk to chunk goes behind an `Arc>`, as `preview` does here. ## Captured stream [Section titled “Captured stream”](#captured-stream) These lines come from the showcase’s `OUTPUT.md`, a real run with `claude-sonnet-5-5` through an OpenAI-compatible gateway. `OUTPUT.md` cuts the draft after its first lines with `...`: ```text git_diff ✓ 17 lines ## 0.34.0 This release adds the opt-in `swiftide-agents-contrib` crate and Perl support for tree-sitter. It also fixes duplicated text in OpenAI streaming and resolves security advisories, which removes AWS Console login from the Bedrock client. ... → write_changelog <32 lines> write_changelog ⏸ waiting for approval ``` The draft appears before the `→ write_changelog` line. `on_stream` runs while the completion arrives, and the `before_tool` hook runs only after it is complete. After the approval, the agent runs the pending tool call without a new completion, so the draft streams once. ## Deltas only [Section titled “Deltas only”](#deltas-only) By default, the OpenAI-compatible clients clone the accumulated response, which grows with the answer, into every chunk. Turn that off with `stream_full(false)`: ```rust let openai = swiftide::integrations::openai::OpenAI::builder() .default_embed_model("text-embeddings-3-small") .default_prompt_model("gpt-4o-mini") // Only streams the delta, leave this out to stream the full response .stream_full(false) .build()?; ``` Source: [`examples/streaming_agents.rs#L11-L16`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/streaming_agents.rs#L11-L16) The last chunk is still the complete response, so the agent behaves the same. `delta.tool_calls_chunk` carries the same accumulated arguments with either setting, so the showcase’s preview works with the default too. [Deltas only](/models/overview/#deltas-only) lists the clients that have the setting and the fields each chunk drops. ## Gotchas [Section titled “Gotchas”](#gotchas) * Before 0.34, the `GenericOpenAI` chat completions stream (OpenAI, Ollama, Groq, Gemini, OpenRouter and Dashscope) repeated the previous `message_chunk` on tool call fragments, the stop and usage frames, and the final chunk, so a hook that printed it printed text more than once. [#1216](https://github.com/bosun-ai/swiftide/pull/1216) fixed this. * The fix covers only the chat completions stream. The OpenAI Responses API (`use_responses_api(true)`, which the showcase’s `--responses-api` flag turns on), `Anthropic` and `AwsBedrock` still keep the previous `message_chunk` on tool call events. With them, both examples print the last text again for every tool call event. Their chunks carry the accumulated `message`, so printing the part of `message` beyond what you already printed avoids repeats. * The agent awaits the hook for every chunk before it reads the next one. A slow hook slows the stream. Hand slow work to a channel or a spawned task. * An error returned from `on_stream` is logged with `tracing` and does not stop the agent. * `on_stream` receives `&Agent`, so it cannot stop the agent or change its tools. Do that from `after_completion`. * Rust’s stdout is line buffered. `print!` without a newline may show nothing until a line ends or the buffer fills, so flush after each chunk. The first example does not flush; the showcase does. ## Reference [Section titled “Reference”](#reference) * [`AgentBuilder::on_stream`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.on_stream) * [`OnStreamFn`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/trait.OnStreamFn.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletionResponseDelta`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponseDelta.html) * [`ToolCallAccum`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCallAccum.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`GenericOpenAIBuilder::stream_full`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.stream_full) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [Feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [MCP tools](/agents/mcp/) # System prompts > Give an agent a system prompt as a Prompt, fill its Tera template from runtime values, change it before the first run, or leave it out. Here you build an agent whose system prompt is a Tera template, filled with values your program only knows at run time. The example is the system prompt of the [release notes agent](/showcases/release-notes-agent/). The code on this page needs the `swiftide-agents` feature flag, and the `openai` feature flag for the model: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents ``` ## Prompt as system message [Section titled “Prompt as system message”](#prompt-as-system-message) examples/release-notes-agent/src/agent.rs ```rust /// The system prompt is a Tera template; optional values are left undefined when absent. /// /// GitHub tools are only mentioned when the repository is on GitHub, since the prompt tells /// the agent which repository to query. fn system_prompt(&self, github_tools: bool) -> Prompt { let mut prompt = Prompt::from(SYSTEM_PROMPT) .with_context_value("since", self.since.as_str()) .with_context_value("github_tools", github_tools && self.github.is_some()); if let Some(version) = &self.next_version { prompt.insert_context_value("next_version", version.as_str()); } if let Some(github) = &self.github { prompt.insert_context_value("github", github.to_string()); } prompt } ``` Source: [`examples/release-notes-agent/src/agent.rs#L69-L84`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/agent.rs#L69-L84) The system prompt of an agent is a `Prompt`. `AgentBuilder::system_prompt` takes anything that converts into one, so a `&'static str`, a `String` or a finished `Prompt` all work. The showcase passes the result of this method to `system_prompt` in its `build` function. `SYSTEM_PROMPT` is [`src/system_prompt.md`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/system_prompt.md), included at compile time with `include_str!`. It is a Tera template, the Jinja-style template language Swiftide renders prompts with. Its last lines show how it handles optional values: ```text {% if github %} - Reference pull requests as Markdown links, exactly like `[#1207](https://github.com/{{ github }}/pull/1207)`. Reference a commit by its short hash only when it has no pull request.{% else %} - Reference pull requests as `#1207`, or commits by their short hash.{% endif %} {% if next_version %}- The version is `{{ next_version }}`.{% else %}- Choose the version yourself: before 1.0, bump the minor version of `{{ since }}` for new features or breaking changes, and the patch version for fixes only.{% endif %} ``` The template uses these constructs: * `{{ since }}` prints the tag the release starts from. Every run has one, so the method always sets it. * `{% if github_tools and github %}` adds the instructions for the GitHub tools only when the agent has them and the repository is on GitHub. * `{% if github %} ... {% else %} ... {% endif %}` picks the style of pull request references, and `{{ github }}` prints the repository as `owner/name`. * `{% if next_version %}` switches between “use this version” and “choose a version”, and `{{ next_version }}` prints it. * `{{ next_version | default(value="") }}` prints the version in the heading, or a placeholder when the user gave none. The optional values stay undefined when they are absent. Tera treats an undefined value in an `if` as false, and the `default` filter replaces it. That is why the method inserts `next_version` and `github` only when it has them. ## Template values [Section titled “Template values”](#template-values) `with_context_value` takes the prompt by value and returns it, so it chains onto `Prompt::from`. `insert_context_value` takes `&mut self`, which suits values you add conditionally, as the showcase does. Both accept anything that converts into a `tera::Value`: strings, numbers, booleans, and `serde_json::Value`. `with_context` merges a whole `tera::Context` at once. [Prompts and templates](/models/prompts-and-templates/#template-syntax) covers the template syntax, loops and the filters that are available. It also lists the `&str` and `format!` pitfalls. ## Values from a before_all hook [Section titled “Values from a before_all hook”](#values-from-a-before_all-hook) Some values are only known when the agent starts. A `before_all` hook can read them and fill them in: ```rust let mut agent = Agent::builder() .llm(openai) .system_prompt("You review the changes on the branch `{{ branch }}`.") .before_all(|agent: &mut Agent| { Box::pin(async move { let branch = agent .context() .executor() .exec_cmd(&Command::shell("git branch --show-current")) .await? .into_string_lossy(); if let Some(prompt) = agent.system_prompt_mut() { prompt.insert_context_value("branch", branch.trim()); } Ok(()) }) }) .build()?; ``` Source: [`snippets/src/agents/system_prompts.rs#L14-L32`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/system_prompts.rs#L14-L32) The hook asks the tool executor for the current git branch and inserts it as `branch`. With the default local executor, the command runs in the working directory of your program. With a Docker executor, it runs in the container. On the first run, the `before_all` hooks run, then the agent renders the prompt and stores it in the context as a `ChatMessage::System`. A `before_all` hook is therefore the last place where a change still counts. `Agent::system_prompt_mut` returns `None` when the agent has no system prompt, so the hook only touches a prompt that exists. Before you call `build`, `AgentBuilder::system_prompt_mut` gives the same access on the builder. It also returns `None` when no prompt is set. A render error stops the run before the first completion. `query` returns `AgentError::FailedToRenderSystemPrompt`. The agent then stays pending, so the next `query` runs the `before_all` hooks and renders the prompt again. That gives a hook the chance to fill the missing value on a retry. ## Named templates [Section titled “Named templates”](#named-templates) A system prompt that several agents share can live in a named template: ```rust let mut templates = tera::Tera::default(); templates.add_raw_templates([ ( "release_rules", "Name the crate and the feature flag in every entry.", ), ( "release_manager", "You write the release notes for {{ project }}.\n\n{% include \"release_rules\" %}", ), ])?; Prompt::extend(&templates)?; let agent = Agent::builder() .llm(openai) .system_prompt( Prompt::from_compiled_template("release_manager") .with_context_value("project", "swiftide"), ) .build()?; ``` Source: [`snippets/src/agents/system_prompts.rs#L71-L90`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/system_prompts.rs#L71-L90) `Prompt::extend` merges your Tera instance into one template repository for the whole process. `Prompt::from_compiled_template` stores only the template name and looks it up when the agent renders the prompt. Named templates can include each other, so shared rules live in one partial, here `release_rules`. Swiftide does not re-export Tera. Add it to your own crate: ```sh cargo add tera@1.20 --no-default-features ``` ## Prompt removal [Section titled “Prompt removal”](#prompt-removal) `no_system_prompt` removes the system prompt from the builder, including one set earlier: ```rust let system_prompt = "You answer questions about the swiftide repository."; let mut first_agent = Agent::builder() .llm(openai) .system_prompt(system_prompt) .build()?; first_agent.query("Which crates does it have?").await?; // The stored history starts with the rendered system message. let history = first_agent.history().await?; let mut second_agent = Agent::builder() .llm(openai) .context( DefaultContext::default() .with_existing_messages(history) .await? .to_owned(), ) .system_prompt(system_prompt) // Removes the prompt set above, so the history keeps a single system message. .no_system_prompt() .build()?; second_agent.query("Which one holds the agents?").await?; ``` Source: [`snippets/src/agents/system_prompts.rs#L41-L63`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/system_prompts.rs#L41-L63) The second agent continues a stored message history that already holds a system message. Without `no_system_prompt`, its first run appends a second one. The call is useful when a shared builder function sets a prompt you do not want for a resumed run. [Context and history](/agents/context-and-history/) shows how the history is restored. ## Default prompt [Section titled “Default prompt”](#default-prompt) An agent has no system prompt unless you set one. Code written for 0.32 or earlier relied on a built-in default; [Upgrading](/reference/upgrading/#system-prompts) lists what to change. The model only sees what you write. A prompt for an agent with tools usually says what the task is, which tools to use in which order, and when to call `stop`. The release notes prompt does all three in its “How to work” list. ## Gotchas [Section titled “Gotchas”](#gotchas) * A `Prompt::from` prompt without any context value is sent verbatim, without rendering. `{{ since }}` then reaches the model as literal text. Named templates always render. * An undefined value outside an `if` or the `default` filter fails rendering. Guard every optional value. * Changes to the system prompt after the first render do not reach the stored message. To change the instructions of a running conversation, replace the system message with `AgentContext::replace_history`, as [Context and history](/agents/context-and-history/) shows. * Setup errors after the render, such as a toolbox that fails to load, also leave the agent pending. The next `query` then adds the system message a second time. * Filters, functions and partials registered with `Prompt::extend` only work in named templates. A `Prompt::from` prompt renders with a fresh Tera instance. ## Reference [Section titled “Reference”](#reference) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [Tera documentation](https://keats.github.io/tera/docs/) ## Next [Section titled “Next”](#next) [Stopping and structured results](/agents/stopping-and-structured-results/) # Testing agents > Test Swiftide agents without a provider by scripting the model, calling tools directly, or matching exact requests with MockChatCompletion. You can test an agent end to end without provider credentials or network access: a scripted model decides which tools to call, and your test asserts on the tool outputs, the requests and the stop reason. `Agent`, `DefaultContext` and `StopReason` sit behind the `swiftide-agents` feature flag, which is off by default. The tests also need tokio for its test macros, serde_json and async-trait: ```sh cargo add swiftide@0.34 --features swiftide-agents cargo add --dev tokio --features macros,rt-multi-thread cargo add --dev serde_json async-trait anyhow ``` Swiftide re-exports serde_json, async_trait and anyhow under `swiftide::reexports`, so you can use those paths instead of adding the crates. ## Scripted models [Section titled “Scripted models”](#scripted-models) ```rust /// A model that answers with scripted responses and records every request it receives. #[derive(Clone, Default)] pub struct ScriptedLlm { replies: Arc>>, requests: Arc>>>, } impl ScriptedLlm { pub fn new(replies: impl IntoIterator) -> Self { Self { replies: Arc::new(Mutex::new(replies.into_iter().collect())), requests: Arc::default(), } } /// The messages of every request, in the order the agent sent them. pub fn requests(&self) -> Vec> { self.requests.lock().unwrap().clone() } } #[async_trait] impl ChatCompletion for ScriptedLlm { async fn complete( &self, request: &ChatCompletionRequest<'_>, ) -> Result { self.requests .lock() .unwrap() .push(request.messages().to_vec()); self.replies .lock() .unwrap() .pop_front() .ok_or_else(|| LanguageModelError::permanent("the script has no more replies")) } } /// A scripted reply that calls one tool with JSON arguments. pub fn reply_calling( name: &str, args: serde_json::Value, ) -> anyhow::Result { let call = ToolCall::builder() .id(format!("call_{name}")) .name(name) .args(args.to_string()) .build()?; ChatCompletionResponse::builder() .tool_calls(vec![call]) .build() } ``` Source: [`snippets/src/agents/testing.rs#L30-L82`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/testing.rs#L30-L82) An agent talks to its model through the [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) trait. Only `complete` is required. The default `complete_stream` wraps `complete` in a stream of one response, so a scripted model also works for an agent with `on_stream` hooks. `ScriptedLlm` hands out one prepared [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) per completion and records the messages of every request. When the script runs out, it returns a permanent `LanguageModelError`, and `query` fails with `AgentError::CompletionsFailed`. The state sits behind `Arc>` because `AgentBuilder::llm` takes a reference and stores a boxed clone. Your test keeps the original, and both share the same queue and request log. `reply_calling` builds a response that calls one tool. The arguments are a JSON (JavaScript Object Notation) string, as a provider would send them. ## Agent tests [Section titled “Agent tests”](#agent-tests) The tool under test is a plain function tool: ```rust /// Counts the words in a piece of text. /// /// # Arguments /// - `text`: The text to count the words of. #[swiftide::tool] pub async fn count_words(_context: &dyn AgentContext, text: &str) -> Result { Ok(text.split_whitespace().count().to_string().into()) } ``` Source: [`snippets/src/agents/testing.rs#L19-L26`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/testing.rs#L19-L26) The test scripts two turns: the model calls `count_words`, then the `stop` tool. ```rust #[tokio::test] async fn counts_the_words_then_stops() -> anyhow::Result<()> { let llm = ScriptedLlm::new([ reply_calling("count_words", json!({ "text": "one two three" }))?, reply_calling("stop", json!({}))?, ]); let mut agent = Agent::builder().llm(&llm).tools([count_words()]).build()?; agent .query("How many words are in `one two three`?") .await?; let history = agent.history().await?; let counted = history.iter().find_map(|message| match message { ChatMessage::ToolOutput(call, output) if call.name() == "count_words" => { output.as_text() } _ => None, }); assert_eq!(counted, Some("3")); assert!(matches!( agent.stop_reason(), Some(StopReason::RequestedByTool(call, None)) if call.name() == "stop" )); let requests = llm.requests(); assert_eq!(requests.len(), 2); assert!(matches!( requests[1].last(), Some(ChatMessage::ToolOutput(..)) )); Ok(()) } ``` Source: [`snippets/src/agents/testing.rs#L100-L132`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/testing.rs#L100-L132) The agent runs the real tool, records its output in the message history and sends the history back to the model. `stop` ends the run with `StopReason::RequestedByTool` and no payload. The test then checks three things: * the `ChatMessage::ToolOutput` for `count_words` holds `3` * the stop reason names the `stop` tool * the second request ended with the tool output, so the model saw the result Because the agent has no system prompt by default, the first request holds only the user message. To test a system prompt, set one on the builder and assert on the first `ChatMessage::System` in `llm.requests()`. A reply without tool calls, such as `ChatCompletionResponse::builder().message("Done.").build()`, scripts a turn where the model only answers in text. The default context stops the agent after such a turn, with `StopReason::NoNewMessages`. ## Direct tool invocation [Section titled “Direct tool invocation”](#direct-tool-invocation) Most tool logic needs no agent at all. Call `invoke` on the tool with a context and a [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html): ```rust #[tokio::test] async fn count_words_rejects_a_misspelled_argument() -> anyhow::Result<()> { let tool = count_words(); let context = DefaultContext::default(); let call = ToolCall::builder() .id("call_1") .name("count_words") .args(json!({ "text": "one two" }).to_string()) .build()?; let output = tool.invoke(&context, &call).await?; assert_eq!(output.as_text(), Some("2")); let typo = ToolCall::builder() .id("call_2") .name("count_words") .args(json!({ "txt": "one two" }).to_string()) .build()?; let error = tool.invoke(&context, &typo).await.unwrap_err(); assert!(matches!(error, ToolError::WrongArguments(_))); Ok(()) } ``` Source: [`snippets/src/agents/testing.rs#L136-L157`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/testing.rs#L136-L157) [`DefaultContext::default()`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) runs commands through a `LocalExecutor` in the current directory, so tools that call `context.executor()` work too. Point it at a temporary directory with `DefaultContext::from_executor` when the tool touches files. Arguments that do not deserialize into the generated arguments struct return `ToolError::WrongArguments`. In the test above, the misspelled `txt` key leaves the required `text` argument missing. A call without arguments returns `ToolError::MissingArguments`. Extra keys are ignored: `deny_unknown_fields` only applies to the JSON schema the model sees. A misspelled key for an `Option` argument becomes `None` without an error. Inside an agent, both go back to the model as a failed tool output and count toward `tool_retry_limit`. ## MockChatCompletion [Section titled “MockChatCompletion”](#mockchatcompletion) For tests that pin down the exact requests, swiftide-core ships `MockChatCompletion` behind its `test-utils` feature flag. The swiftide crate has a `test-utils` feature too, but it does not re-export the module, so add swiftide-core as a dev dependency and import `swiftide_core::test_utils::MockChatCompletion`: ```sh cargo add --dev swiftide-core@0.34 --features test-utils ``` ```rust #[tokio::test] async fn sends_the_tool_output_back_to_the_model() -> anyhow::Result<()> { let llm = MockChatCompletion::new(); let question = ChatMessage::new_user("How many words are in `one two`?"); let count = ToolCall::builder() .id("call_1") .name("count_words") .args(json!({ "text": "one two" }).to_string()) .build()?; let stop = ToolCall::builder().id("call_2").name("stop").build()?; llm.expect_complete( ChatCompletionRequest::builder() .messages(vec![question.clone()]) .tools(Agent::default_tools()) .tool(count_words()) .build()?, Ok(ChatCompletionResponse::builder() .tool_calls(vec![count.clone()]) .build()?), ); llm.expect_complete( ChatCompletionRequest::builder() .messages(vec![ question, ChatMessage::new_assistant(None::, Some(vec![count.clone()])), ChatMessage::new_tool_output(count, ToolOutput::text("2")), ]) .tools(Agent::default_tools()) .tool(count_words()) .build()?, Ok(ChatCompletionResponse::builder() .tool_calls(vec![stop]) .build()?), ); let mut agent = Agent::builder().llm(&llm).tools([count_words()]).build()?; agent.query("How many words are in `one two`?").await?; assert!(agent.is_stopped()); Ok(()) } ``` Source: [`snippets/src/agents/testing.rs#L161-L202`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/testing.rs#L161-L202) Each `expect_complete` queues an expected [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) and the response to return. Expectations are consumed in the order you add them. A request that differs from the next expectation fails the test with both requests printed. An `Err` response reaches the agent as a permanent `LanguageModelError`. Request equality covers the messages and the full set of tool specs. The agent sends the spec of every tool it has, including the default `stop` tool, so the expected request lists `Agent::default_tools()` next to your own tools. When the last clone of the mock is dropped with expectations left over, it panics. A test that stops early fails even when every request it made matched. ## Showcase tests [Section titled “Showcase tests”](#showcase-tests) The release notes showcase has 27 tests. From the swiftide repository root: ```sh cargo test -p release-notes-agent ``` The session tests exercise the approval loop in `session.rs`: examples/release-notes-agent/src/session.rs ```rust /// Asks the agent for release notes and loops through review rounds until they are written or /// rejected. pub async fn draft_release_notes( agent: &mut Agent, task: Prompt, reviewer: &mut impl Reviewer, ) -> Result { agent.query(task).await?; let mut written = false; let mut nudged = false; loop { // `write_changelog` is wrapped in `ApprovalRequired`: the agent stops and waits. let tool_call = match agent.stop_reason() { Some(StopReason::FeedbackRequired { tool_call, .. }) => tool_call.clone(), // The custom `stop` tool carries the structured release notes. Some(StopReason::RequestedByTool(_, Some(notes))) if written => { let notes = ReleaseNotes::deserialize(notes) .context("the agent stopped with malformed release notes")?; return Ok(Outcome::Written { notes: Some(notes) }); } _ if written => return Ok(Outcome::Written { notes: None }), // A turn without a tool call, or the step limit. Some(StopReason::NoNewMessages) => { nudge_once(agent, &mut nudged).await?; continue; } other => bail!("the agent stopped without writing the notes: {other:?}"), }; let proposal: WriteChangelogArgs = serde_json::from_str(tool_call.args().unwrap_or_default())?; match reviewer.review(&proposal)? { Verdict::Approve => { agent .context() .feedback_received(&tool_call, &ToolFeedback::approved()) .await?; // Resumes the pending tool call, which now runs, and continues the loop. agent.run().await?; // The write itself can still fail; the agent then hears why and carries on. written |= succeeded(agent, &tool_call).await?; } Verdict::Revise(feedback) => { agent .context() .feedback_received(&tool_call, &ToolFeedback::refused()) .await?; // The refusal is recorded as the tool's output, followed by the feedback. agent.query(feedback).await?; } Verdict::Reject => return Ok(Outcome::Rejected), } } } ``` Source: [`examples/release-notes-agent/src/session.rs#L41-L96`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/session.rs#L41-L96) Five of the six session tests drive this loop with a scripted model like the one above. The showcase’s own `ScriptedLlm`, a simpler variant of the one above, takes `(tool name, arguments)` pairs, and an empty name scripts a turn without a tool call. A `ScriptedReviewer` plays the person who approves or asks for a revision, and those tests run against a temporary repository. They cover: * a revision round, then approval and the write * stopping without a write, which is an error * one nudge after a turn without a tool call, and an error after the second one * a draft that lists a pull request twice, which goes back to the model before review The duplicate check also ran against a live model. The other paths ran only in these tests. The sixth session test is a synchronous unit test. It builds a history by hand and checks that reaching the step limit is not mistaken for a turn without a tool call, although both report the same stop reason. The remaining tests are plain unit tests in the other modules. They cover system prompt rendering, decoding a streamed draft however it is chunked, and restarting the draft preview for a new tool call. They also cover finding a pull request listed in two entries, the stop tool schema, rendering the notes, tool names, clipping tool output, shell quoting, GitHub remote parsing, and rejecting unsafe revisions and paths. ## Gotchas [Section titled “Gotchas”](#gotchas) * `swiftide_agents::test_utils` (`MockTool`, `MockHook`, `chat_request!`) is compiled only in test and debug builds, and its macros expand to `swiftide_core` paths. It serves swiftide’s own tests; do not build on it. * `MockChatCompletion::complete_stream` splits the response text on whitespace inside a spawned task. On a response without text that task panics, the stream closes empty, and `query` returns `AgentError::EmptyStream`. A streamed reply that only calls tools never delivers its tool calls. Script a message on every reply when the agent streams. * The agent rewrites JSON object arguments before it stores the assistant message, to drop duplicate keys. Build expected arguments with `serde_json`, as above, rather than as hand-typed strings with their own spacing. * The unit context `()` implements `AgentContext` for tools that never run commands. Its `executor()` panics, and it approves every feedback request. * A tool call that names a tool the agent does not have is skipped with a warning and gets no output. With a typo in a scripted tool name, the run ends with `StopReason::NoNewMessages` instead of the scripted stop. The next `query` or `run` on that agent fails with `AgentError::UnfulfilledToolCalls`. ## Reference [Section titled “Reference”](#reference) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`MockChatCompletion`](https://docs.rs/swiftide-core/0.34/swiftide_core/test_utils/struct.MockChatCompletion.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [Section titled “Next”](#next) [How tasks work](/tasks/overview/) # Tools > Write agent tools as async functions with #[swiftide::tool], as stateful structs with #[derive(Tool)], or by implementing Tool by hand with your own schema, and control how tool errors are retried. You end up with three kinds of tool: async functions the model can call, a tool with its own state, and a hand-written tool with a custom argument schema. The examples come from the release notes agent in `examples/release-notes-agent`, which drafts release notes from git history. Add the dependencies the macros expect: ```sh cargo add swiftide@0.34 --features swiftide-agents,openai cargo add serde --features derive cargo add schemars@1 async-trait ``` examples/release-notes-agent/src/tools.rs ```rust /// Lists every commit since a release tag, oldest first: hash, subject, body and the files it /// changed. /// /// # Arguments /// - `tag`: Tag to start from, e.g. `v0.33.0`. Use null for the latest tag. #[swiftide::tool] pub async fn git_log_since( context: &dyn AgentContext, tag: &Option, ) -> Result { let git = Git::new(context.executor().as_ref()); let since = match tag { Some(tag) => tag.parse()?, None => git.latest_tag().await?, }; let log = git.log_since(&since).await?; Ok(ToolOutput::text(clip( log, MAX_LOG_LINES, "Use git_diff on a specific path for details.", ))) } /// Shows the diff of one file or directory over a revision range. /// /// # Arguments /// - `range`: Revision range, e.g. `v0.33.0..HEAD`, or a single commit hash. /// - `path`: File or directory relative to the repository root. #[swiftide::tool] pub async fn git_diff( context: &dyn AgentContext, range: &str, path: &str, ) -> Result { let (range, path): (Revision, RepoPath) = (range.parse()?, path.parse()?); let diff = Git::new(context.executor().as_ref()) .diff(&range, &path) .await?; if diff.trim().is_empty() { return Ok(ToolOutput::text(format!( "No changes to {path} in {range}." ))); } Ok(ToolOutput::text(clip( diff, MAX_DIFF_LINES, "Diff a narrower path to see the rest.", ))) } /// Reads a file from the repository, e.g. to check a public API or the style of the changelog. /// /// # Arguments /// - `path`: File path relative to the repository root. #[swiftide::tool] pub async fn read_file(context: &dyn AgentContext, path: &str) -> Result { let path: RepoPath = path.parse()?; let content = context .executor() .exec_cmd(&Command::read_file(path.as_path())) .await?; Ok(ToolOutput::text(clip( content.into_string_lossy(), MAX_FILE_LINES, "The file is longer than shown.", ))) } ``` Source: [`examples/release-notes-agent/src/tools.rs#L32-L101`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/tools.rs#L32-L101) ## Function tools [Section titled “Function tools”](#function-tools) `#[swiftide::tool]` turns an async function into a tool. The first argument must be `&dyn AgentContext`. Every other argument becomes a tool argument that the model fills in. The macro generates three items from `git_log_since`: * a struct `GitLogSince` that implements `Tool`, * a function `git_log_since()` that returns it as a `Box`, ready for `AgentBuilder::tools`, * an argument struct `GitLogSinceArgs` that derives serde and `JsonSchema`. Its schema forbids extra properties, but deserialization ignores keys it does not know. The tool name the model sees is the function name, `git_log_since`. Since 0.33, the description is the doc comment text before the first heading. Each item under `# Arguments` describes one argument. A `# Parameters` heading works the same way. The tools parse the model’s strings into `Revision` and `RepoPath` before they call `context.executor()`, because the `LocalExecutor` in this example does not sandbox anything. See [Executors](/agents/executors/). A hook printed the first tool call of a real run with `claude-sonnet-5-5`: ```text Drafting release notes for the 7 commits since v0.33.0 with claude-sonnet-5-5 at http://127.0.0.1:8080/v1 (chat completions) → git_log_since v0.33.0 git_log_since ✓ 346 lines ``` ## Argument types [Section titled “Argument types”](#argument-types) An argument type must implement serde’s `Serialize` and `Deserialize`, `schemars::JsonSchema` and `Debug`, because the generated argument struct derives all four. The macro builds the JavaScript Object Notation (JSON) Schema of the arguments with `schemars::schema_for!`: ```rust /// An inclusive range of lines. #[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)] pub struct LineRange { /// First line, starting at 1 pub start: u32, /// Last line, inclusive pub end: u32, } /// Reads line ranges from files in the workspace. /// /// # Arguments /// - `paths`: Files to read, relative to the workspace root. /// - `range`: Optional line range to limit the output. Use null for the whole file. #[swiftide::tool] async fn read_lines( context: &dyn AgentContext, paths: Vec, range: Option, ) -> Result { let _ = (context, range); Ok(ToolOutput::text(paths.join("\n"))) } ``` Source: [`snippets/src/agents/tools.rs#L57-L79`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/tools.rs#L57-L79) * `&str`, `&[T]`, `&Vec` and `&Option` are deserialized as their owned types. Take any other type, such as `LineRange`, by value. * An `Option` argument is optional. The OpenAI integration marks every property as required, so there the model sends `null` instead of leaving the argument out. The `range` description tells the model so. * The doc comments of a struct argument become field descriptions. * A map type such as `HashMap` compiles, but the generated `tool_spec` panics when the agent builds its first request. Arguments that do not parse never reach your function: the model gets a `ToolError::WrongArguments` back. ## Stateful tools [Section titled “Stateful tools”](#stateful-tools) When a tool needs configuration, derive `Tool` on a struct: examples/release-notes-agent/src/tools.rs ```rust /// Prepends the approved release notes to the release notes file. /// /// The file is fixed by the CLI (`--output`), never chosen by the model. Wrap this tool in /// `ApprovalRequired` so a human signs off before anything is written. #[derive(Clone, Tool)] #[tool( description = "Prepends the finished release notes to the release notes file. A maintainer \ reviews the notes before they are written.", param( name = "markdown", description = "The complete release notes for this version, as GitHub flavored Markdown" ) )] pub struct WriteChangelog { file: PathBuf, } impl WriteChangelog { pub fn new(file: impl Into) -> Self { Self { file: file.into() } } async fn write_changelog( &self, context: &dyn AgentContext, markdown: &str, ) -> Result { let executor = context.executor(); // A missing file is started; any other failure must not overwrite the existing notes. let existing = if file_exists(executor.as_ref(), &self.file).await? { executor .exec_cmd(&Command::read_file(&self.file)) .await .map(CommandOutput::into_string_lossy)? } else { String::new() }; executor .exec_cmd(&Command::write_file( &self.file, prepend_notes(markdown, &existing), )) .await?; Ok(ToolOutput::text(format!("wrote {}", self.file.display()))) } } ``` Source: [`examples/release-notes-agent/src/tools.rs#L105-L152`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/tools.rs#L105-L152) The derive calls the method named after the struct in snake case, here `write_changelog`. The model never chooses the file: it comes from the struct. * The derive does not read doc comments. `description` is required, and each argument needs a `param`. * A `param` without a type is a `String`, so the method takes `&str`. * The struct must be `Clone`. Register the tool with `add_tool(WriteChangelog::new(path))`, or pass `WriteChangelog::new(path).into()` to `tools`. ## Hand-written tools [Section titled “Hand-written tools”](#hand-written-tools) Implement `Tool` yourself to write the tool spec by hand and parse the arguments in your own code. Describe the arguments as a type: ```rust #[derive(Clone)] struct WorkflowTool; #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[schemars( title = "WorkflowInstruction", description = "Choose a workflow action and optional payload", deny_unknown_fields )] struct WorkflowInstruction { #[schemars(description = "Which workflow action to execute")] action: WorkflowAction, #[schemars(description = "Optional payload forwarded to the workflow engine")] #[serde(default, skip_serializing_if = "Option::is_none")] payload: Option, } #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(rename_all = "lowercase")] enum WorkflowAction { Start, Stop, Status, } ``` Source: [`examples/tool_custom_schema.rs#L11-L34`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/tool_custom_schema.rs#L11-L34) Then return that schema from `tool_spec`: ```rust #[swiftide::reexports::async_trait::async_trait] impl Tool for WorkflowTool { async fn invoke( &self, _agent_context: &dyn AgentContext, _tool_call: &ToolCall, ) -> Result { Ok(ToolOutput::text( "Workflow execution not implemented in this example", )) } fn name<'tool>(&'tool self) -> Cow<'tool, str> { Cow::Borrowed("workflow_tool") } fn tool_spec(&self) -> ToolSpec { ToolSpec::builder() .name("workflow_tool") .description("Executes a workflow action with strict input choices") .parameters_schema(workflow_schema()) .build() .expect("tool spec should be valid") } } fn workflow_schema() -> Schema { schema_for!(WorkflowInstruction) } ``` Source: [`examples/tool_custom_schema.rs#L38-L66`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/tool_custom_schema.rs#L38-L66) `invoke` receives the `ToolCall`, whose `args()` is the JSON string of the arguments. The agent drops duplicate keys and re-serializes the JSON first, so the string can differ from what the model sent. `name()` must match the `ToolSpec` name: the agent looks tools up by `name()`, the model calls the spec name. `ToolSpec::builder().build()` returns a `Result`. It checks the schema against Swiftide’s strict contract, so a `HashMap` field fails; see [Tool schema policies](/agents/providers-and-reasoning/#tool-schema-policies). The example calls `expect`, so an invalid schema panics at the first request. Cover it with a unit test that calls `tool_spec()`. A hand-written tool can wrap another one. `OneEntryPerChange` in the same `tools.rs` delegates to `WriteChangelog` and rejects drafts that list a change twice. ## Tool output [Section titled “Tool output”](#tool-output) A tool returns `Result`. `ToolOutput` is non-exhaustive and has these variants: * `Text(String)`: the result, added to the message history. `&str`, `String` and `CommandOutput` convert into it. * `Fail(String)`: a failure the model can fix. * `Stop(Option)`: stops the agent with `StopReason::RequestedByTool`. See [Stopping and structured results](/agents/stopping-and-structured-results/). * `FeedbackRequired(Option)`: stops the agent with `StopReason::FeedbackRequired` and leaves the tool call open. See [Human in the loop](/agents/human-in-the-loop/). * `AgentFailed(Option)`: stops the agent with `StopReason::AgentFailed`. ## Tool errors and retries [Section titled “Tool errors and retries”](#tool-errors-and-retries) `ToolError` has four variants: `WrongArguments`, `MissingArguments`, `ExecutionFailed` and `Unknown`. It converts from `serde_json::Error`, `CommandError` and `anyhow::Error`, so `?` works. ```rust /// Runs the tests of one crate in the workspace and returns their output. /// /// # Arguments /// - `package`: Name of the crate to test, e.g. `swiftide-core`. #[swiftide::tool] async fn run_tests(context: &dyn AgentContext, package: &str) -> Result { let is_crate_name = !package.is_empty() && package .chars() .all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_'); if !is_crate_name { return Ok(ToolOutput::fail(format!( "`{package}` is not a crate name. Use a package name from Cargo.toml." ))); } let command = Command::shell(format!("cargo test --package {package}")); match context.executor().exec_cmd(&command).await { Ok(output) => Ok(output.into()), Err(CommandError::NonZeroExit(output)) => { Ok(ToolOutput::text(format!("Tests failed:\n{output}"))) } Err(error) => Err(ToolError::execution_failed(error)), } } ``` Source: [`snippets/src/agents/tools.rs#L11-L35`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/tools.rs#L11-L35) A bad crate name returns `Ok(ToolOutput::fail(..))`, and the model tries again. An executor error returns `Err`: the agent records it as a `Fail` output and counts the failure. `tool_retry_limit` sets how many failures of the same call the agent feeds back to the model: ```rust let mut agent = Agent::builder() .llm(openai) .tools([run_tests()]) .tool_retry_limit(2) .build()?; if let Err(error) = agent.query("Run the tests of swiftide-core.").await { if let Some(StopReason::ToolCallsOverLimit(call)) = agent.stop_reason() { eprintln!("gave up on `{}`: {error}", call.name()); } } ``` Source: [`snippets/src/agents/tools.rs#L41-L51`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/agents/tools.rs#L41-L51) With a limit of 2, the third identical failure stops the agent. The full rules for `tool_retry_limit` and `limit` are on [Stopping and structured results](/agents/stopping-and-structured-results/#stop-reasons). ## Attribute descriptions [Section titled “Attribute descriptions”](#attribute-descriptions) Instead of doc comments you can pass `description` and `param` attributes. An explicit attribute wins over the doc comment: ```rust #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] struct FormatTimestampRequest { /// Prefix to prepend to the timestamp. prefix: String, /// Timestamp to format. timestamp: String, } #[swiftide::tool(description = "Get the current UTC date and time in RFC3339 format")] async fn current_utc_time(context: &dyn AgentContext) -> Result { let command_output = context .executor() .exec_cmd(&Command::shell("date -u +\"%Y-%m-%dT%H:%M:%SZ\"")) .await?; Ok(command_output.into()) } #[swiftide::tool( description = "Format a timestamp with a caller-provided prefix", param(name = "request", description = "Timestamp formatting input") )] async fn format_timestamp( _context: &dyn AgentContext, request: FormatTimestampRequest, ) -> Result { Ok(ToolOutput::text(format!( "{}{}", request.prefix, request.timestamp ))) } ``` Source: [`examples/aws_bedrock_agent.rs#L23-L53`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L23-L53) `description` also accepts a path to a `const`, such as `description = READ_FILE`. ## Gotchas [Section titled “Gotchas”](#gotchas) * The generated code refers to `::swiftide::` and `::serde` paths. The crate that uses the macros needs `swiftide` with the default `macros` feature flag and `serde` with `derive`. * Every argument after the context must be listed under `# Arguments` or as a `param`. A doc bullet with an empty description is dropped, and a missing argument fails to compile with “parameters are missing from the spec”. * Tools are unique by name. `add_tool` replaces a tool with the same name, and `tools` replaces every tool added before it. To bring your own `stop`, see [Stopping and structured results](/agents/stopping-and-structured-results/). * The tool calls of one completion run concurrently on separate Tokio tasks, so two calls to a tool that writes the same file can race. ## Reference [Section titled “Reference”](#reference) * [`#[swiftide::tool]`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) and [`#[derive(Tool)]`](https://docs.rs/swiftide/0.34/swiftide/derive.Tool.html) * [`Tool`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html) * [`ToolSpec`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpec.html) and [`ToolSpecBuilder`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpecBuilder.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.ToolError.html) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`AgentBuilder::tool_retry_limit`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.tool_retry_limit) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) ## Next [Section titled “Next”](#next) [Context and message history](/agents/context-and-history/) # Contributing and community > Ask questions on Discord, file issues from the bug report and feature request templates, and open pull requests that pass Swiftide's CI checks. By the end of this page you know where to ask for help, how to file an issue, and which checks a pull request must pass before review. ## Help and discussion [Section titled “Help and discussion”](#help-and-discussion) * [Discord](https://discord.gg/3jjXYen9UY) gives you the fastest feedback. Release changelogs are posted there automatically when a GitHub release is published. * [GitHub issues](https://github.com/bosun-ai/swiftide/issues) are the place for design discussion. * The [Bosun blog](https://blog.bosun.ai/) has background posts on tasks, streaming agents, human-in-the-loop flows and Rust performance for artificial intelligence (AI) tools. Swiftide is pre-1.0. The application programming interface (API) can change between releases. The [API docs](https://docs.rs/swiftide/0.34/swiftide/index.html) are the most reliable source for exact signatures. The [changelog](/reference/changelog/) lists what changed in each release. ## Issues [Section titled “Issues”](#issues) The repository has two issue templates: * [Bug report](https://github.com/bosun-ai/swiftide/issues/new?template=bug_report.md): describe the bug, give a minimal example that reproduces it, and state the behavior you expected. * [Feature request](https://github.com/bosun-ai/swiftide/issues/new?template=feature_request.md): describe the problem, the solution you want, and the alternatives you considered. To suggest an enhancement without writing the code, open a feature request instead of a pull request. To find something to work on, browse the [open issues](https://github.com/bosun-ai/swiftide/issues). ## Design principles [Section titled “Design principles”](#design-principles) * Write simple, thin wrappers with sane defaults. * Offer a builder (`derive_builder`) for customization. * Keep Rust complexity such as `Arc`, `Box`, lifetimes and pinning away from library users. * Follow the [Rust API Guidelines on naming](https://rust-lang.github.io/api-guidelines/naming.html). * Watch allocations in indexing and querying. Both are performance-sensitive. The library crates opt in to the workspace lints. These lints forbid `unsafe` code and set Clippy’s `pedantic` and `cargo` lint groups to warn. Continuous integration (CI) turns warnings into errors. ## Development setup [Section titled “Development setup”](#development-setup) * **Rust toolchain.** `rust-toolchain.toml` pins Rust 1.97.1, so `rustup` installs the right version for you. * **Nightly rustfmt.** Formatting needs nightly because `rustfmt.toml` wraps comments. Install it with `rustup component add rustfmt --toolchain nightly`. * **Cargo tools.** The checks use tools that do not ship with `rustup`. Install them with `cargo install cargo-hack cargo-deny cargo-insta typos-cli`. * **protoc and libcurl.** CI installs the Protocol Buffers compiler and the libcurl development headers before it builds with every feature flag. Install both locally if you run `--all-features` builds. * **Docker.** The integration tests start their services with testcontainers. Shared fixtures and container helpers live in `swiftide-test-utils/`. * **Secrets.** Keep credentials in environment variables or `mise`. Never commit them. ## Checks [Section titled “Checks”](#checks) These commands mirror the main CI jobs. Run them before you request a review: ```sh cargo +nightly fmt --all -- --check cargo clippy --workspace --all-targets --all-features -- -D warnings cargo test -j 2 --tests --all-features --no-fail-fast cargo test --doc --all-features --no-fail-fast cargo hack check --each-feature --no-dev-deps typos cargo deny check ``` `cargo hack` checks each [feature flag](/reference/feature-flags/) on its own. `cargo deny check` checks advisories, licenses, banned crates and crate sources against `deny.toml`. CI also checks `swiftide-agents-contrib` once per capability, without default features, and builds its rustdoc with warnings as errors. If you change that crate, run the same steps for the capability you touched, for example `planning`: ```sh cargo check -p swiftide-agents-contrib --lib --no-default-features --features planning cargo test -p swiftide-agents-contrib --no-default-features --features planning RUSTDOCFLAGS="-D warnings" cargo doc -p swiftide-agents-contrib --no-default-features --features planning --no-deps cargo check -p swiftide --lib --no-default-features --features agents-contrib-planning ``` The other capabilities are `context-management`, `file-tools` and `lazy-executor`, with the `swiftide` feature flags `agents-contrib-context`, `agents-contrib-file-tools` and `agents-contrib-lazy-executor`. While you iterate, run focused checks on the crates you changed: ```sh cargo check -p swiftide-agents cargo test -p swiftide-integrations ``` * `RUST_LOG=info` turns on verbose diagnostics. * Some tests are marked `#[ignore]`, and CI skips them. Most download HuggingFace models through the `fastembed` feature flag. One calls AWS Bedrock and bills your account. Run them only on purpose, with the feature flags they need, for example `cargo test -p swiftide-integrations --features fastembed -- --ignored`. * When an insta snapshot is new or no longer matches, the test writes a pending `.snap.new` file. Review it with `cargo insta review`, and commit only after it clears the pending files. * Benchmarks run with `cargo bench -p benchmarks` on every push to master. The job fails and comments on the commit when a benchmark regresses past the 200% alert threshold. ## Commits and pull requests [Section titled “Commits and pull requests”](#commits-and-pull-requests) Commits and pull request (PR) titles follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/). Write a lowercase imperative summary with an optional scope, such as `feat(agents): ...` or `fix(indexing): ...`. * CI checks every PR title against the format. * A `!` before the colon, as in `feat(agents)!:`, marks a breaking change. CI adds the `breaking change` label when the PR title has this marker. The check only accepts a one-word scope, so a title such as `feat(agents-contrib)!:` gets no label. * A PR labeled `do not merge` fails CI on purpose. To open a PR, fork the repository, create a feature branch, commit, push, and open the PR against `master`. The description covers: * what changed and why * the linked GitHub issue, if there is one * any API or schema impact * before and after traces or logs when behavior changes Functional changes come with tests, and with benchmarks where performance matters. ## Documentation [Section titled “Documentation”](#documentation) * Document every public function with rustdoc. * Update the user documentation in [bosun-ai/swiftide-website](https://github.com/bosun-ai/swiftide-website). Each documentation page has an edit link. * The site compiles every line of Rust it shows. Pages excerpt code from source files and never contain hand-written Rust. The website `README.md` explains the snippet directives. * To show code from `examples/` in the swiftide repository, wrap the region in `// ANCHOR: name` and `// ANCHOR_END: name` there first. A page can use the anchor once the website pins a swiftide commit that contains it. ## AI-generated code [Section titled “AI-generated code”](#ai-generated-code) Code written with an AI assistant is welcome. It is reviewed like any other code, so read and test what you submit. Coding agents should read `AGENTS.md` in the swiftide repository for the workspace layout, commands and expectations. ## Core team [Section titled “Core team”](#core-team) * [timonv](https://github.com/timonv) * [tinco](https://github.com/tinco) Swiftide is part of the [bosun.ai](https://bosun.ai) project. The full list of contributors is on the [contributors graph](https://github.com/bosun-ai/swiftide/graphs/contributors). ## License [Section titled “License”](#license) Swiftide is distributed under the MIT License, copyright 2024 Bosun.ai. ## Reference [Section titled “Reference”](#reference) * [swiftide API docs](https://docs.rs/swiftide/0.34/swiftide/index.html) ## Next [Section titled “Next”](#next) [What is Swiftide](/start/what-is-swiftide/) # Caching > Make re-indexing incremental with a Redb, Redis or DuckDB node cache that skips files whose path and content have not changed. Add a node cache to your indexing pipeline and a second run skips every file whose path and content have not changed, before any large language model (LLM) call or embedding is made. ## Redb cache [Section titled “Redb cache”](#redb-cache) The [ask-your-codebase](/showcases/ask-your-codebase/) showcase keeps its node cache in an embedded Redb file next to the code it indexes: examples/ask-your-codebase/src/codebase.rs ```rust /// Remembers which files were fully indexed, so unchanged files are skipped next time. pub fn node_cache(&self) -> Result { std::fs::create_dir_all(&self.data_dir)?; Redb::builder() .database_path(self.data_dir.join("cache.redb")) .table_name("indexed_files") .build() } /// Embedded vector store with one vector per chunk and the source path next to it. pub fn vector_store(&self) -> Result { let uri = self.lancedb_dir(); LanceDB::builder() .uri(uri.to_str().context("Index path is not valid UTF-8")?) .table_name("code_chunks") .vector_size(embedding::DIMENSIONS) .with_vector(EmbeddedField::Combined) .with_metadata(PATH_FIELD) .build() } ``` Source: [`examples/ask-your-codebase/src/codebase.rs#L56-L77`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/codebase.rs#L56-L77) `Redb::builder()` takes a `database_path` and creates the file if it does not exist. It does not create missing parent directories, so the showcase calls `create_dir_all` first. You can pass an open `redb::Database` with `database` instead of a path. `table_name` defaults to `swiftide`. The second function builds the LanceDB storage, covered on [Storage](/indexing/storage/). The pipeline adds the cache with `filter_cached`: examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` Source: [`examples/ask-your-codebase/src/index.rs#L89-L115`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L89-L115) `filter_cached` drops every node the cache knows. Every other node continues to the chunker, the LLM and the embedder. The first run indexes the `swiftide-agents` crate from scratch, with `claude-haiku-5-5` writing the questions and FastEmbed computing the embeddings, as described in [Output](/indexing/overview/#output): ```text $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s ``` The second run, with no file changed: ```text $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s ``` The cache skips all 14 files, so no prompts are sent and nothing is embedded. ## Cache keys [Section titled “Cache keys”](#cache-keys) The cache key is a node id: a UUID (universally unique identifier) computed from the node’s path and chunk, not its metadata. The built-in chunkers set `parent_id` on every chunk to the id of the node that entered the pipeline, the first ancestor, even across several chunkers. Caches use `parent_id` when it is set. One key therefore covers a file and all of its chunks. Each backend puts a prefix in front of the id: | Backend | Key format | Prefix setting | | ------- | ----------------- | ------------------------------------------------------------------------------------------------------- | | Redb | `.` | `cache_key_prefix`, empty by default | | Redis | `:` | second argument of `Redis::try_from_url`, or `cache_key_prefix` on `Redis::builder()`, empty by default | | DuckDB | `.` | `cache_key_prefix`, empty by default | The pipeline records the id a node has when it passes `filter_cached`. At the end it marks the `parent_id` of each finished node, or its own id when `parent_id` is not set. The file is cached only when the two ids match. Three things break the match silently. A step after `filter_cached` that rewrites `chunk` or `path` in place changes the node’s id. The file is then never marked and is processed again on every run, with or without a chunker. Keep content-changing steps in front of `filter_cached`, like the showcase does with `test_code::without_unit_tests`. Steps that only add metadata are safe after it. A custom chunker must keep the parent. The built-in chunkers build chunks with `Node::into_chunks`, which sets `parent_id`. A closure or `ChunkerTransformer` that creates fresh nodes with `TextNode::new` or `Node::builder` leaves `parent_id` empty, so the file is never marked. Derive chunks with `into_chunks` or `Node::build_from_other`. The path is part of the id. The same file loaded through `./src` and through an absolute path has two keys. The showcase canonicalizes the codebase root for this reason. ## When nodes are cached [Section titled “When nodes are cached”](#when-nodes-are-cached) Since 0.33, `run` marks a file in the cache only after the stream has completed and every chunk of that file reached the end. A file with a failed chunk is left out and retried on the next run. Two consequences follow: * If `run` returns an error, nothing from that run is cached. The same holds when the process stops before `run` returns. * A source whose chunker produces zero chunks never reaches the end of the stream. It is never cached, and every step up to the chunker runs for it again on each run. To keep progress when some nodes fail, log and drop the errors so that `run` completes. This snippet needs the `redis`, `qdrant` and `openai` feature flags: ```rust // Change the prefix, for example to "docs-v2", to index every file again let cache = Redis::try_from_url(redis_url, "docs-v1")?; Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .filter_cached(cache) .then_chunk(ChunkMarkdown::from_chunk_range(100..1500)) .then(MetadataQAText::new(openai.clone())) // Log and drop failed chunks so the run completes. Their files stay uncached and are // retried next run; every other file is cached. .log_errors() .filter_errors() .then_in_batch(Embed::new(openai).with_batch_size(32)) .then_store_with( Qdrant::builder() .vector_size(1536) .collection_name("docs") .build()?, ) .run() .await?; ``` Source: [`snippets/src/indexing/caching.rs#L15-L34`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/caching.rs#L15-L34) The file behind a failed chunk stays out of the cache and is retried next run. Every other file is cached. [Errors](/indexing/streams-and-concurrency/#errors) covers where to place `log_errors` and `filter_errors`. ## Backends [Section titled “Backends”](#backends) | Backend | Feature flag | Node type | Notes | | ----------------------------------------------------------------------------------------- | ------------ | -------------- | ------------------------------------------------------- | | [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) | `redb` | `TextNode` | Embedded file, no service to run | | [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) | `redis` | any chunk type | Shared between machines; also a storage | | [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) | `duckdb` | any chunk type | Cache table `swiftide_cache` by default; also a storage | Turn on the feature flag for the backend you pick: ```sh cargo add swiftide@0.34 --features redb ``` `Redis::try_from_url` returns a cache for `String` chunks. For another chunk type, use `Redis::try_build_from_url` or `Redis::builder()`. With Redis, the cache is one line. This example also needs the `qdrant`, `openai` and `tree-sitter` feature flags: examples/index_codebase.rs ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .filter_cached(Redis::try_from_url(redis_url, "swiftide-examples")?) .then(MetadataQACode::new(openai_client.clone())) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples") .build()?, ) .run() .await?; ``` Source: [`examples/index_codebase.rs#L52-L68`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase.rs#L52-L68) `MetadataQACode` runs before chunking here. That is safe, because it only adds metadata and the id does not change. No backend fails the pipeline on a cache error. `get` logs the error and reports the node as not cached, so an unreachable Redis means every node is processed again. ## Custom cache [Section titled “Custom cache”](#custom-cache) To write your own cache, implement [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html). This one keeps the ids in memory: ```rust use std::{ collections::HashSet, sync::{Arc, Mutex}, }; use async_trait::async_trait; use swiftide::{indexing::TextNode, traits::NodeCache}; use uuid::Uuid; /// Remembers indexed sources in memory, for tests or one-off runs. #[derive(Debug, Clone, Default)] pub struct InMemoryCache { indexed: Arc>>, } /// The same key the pipeline marks: the source id, shared by all chunks of a file. fn source_id(node: &TextNode) -> Uuid { node.parent_id().unwrap_or_else(|| node.id()) } #[async_trait] impl NodeCache for InMemoryCache { type Input = String; async fn get(&self, node: &TextNode) -> bool { match self.indexed.lock() { Ok(indexed) => indexed.contains(&source_id(node)), Err(error) => { tracing::error!("node cache lock poisoned: {error}"); false } } } async fn set(&self, node: &TextNode) { self.set_by_id(source_id(node)).await; } async fn set_by_id(&self, id: Uuid) { match self.indexed.lock() { Ok(mut indexed) => { indexed.insert(id); } Err(error) => tracing::error!("node cache lock poisoned: {error}"), } } } ``` Source: [`snippets/src/indexing/caching.rs#L41-L87`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/caching.rs#L41-L87) Implement it with `#[async_trait]`, declare `type Input` as the chunk type, and make the type `Send`, `Sync`, `Debug` and `Clone`. `get` returns `bool`, and `set` and `set_by_id` return nothing, so log errors instead of returning them. Key on `parent_id`, falling back to `id()`, so that `get`, `set` and `set_by_id` agree. The pipeline calls `set_by_id` with the source id when it marks a file. It is a required method since 0.33, so existing custom caches must add it. `clear` is optional, and its default implementation panics. ## Cache invalidation [Section titled “Cache invalidation”](#cache-invalidation) `NodeCache::clear` empties a cache, but each backend does it differently: * Redb deletes its table. * DuckDB drops its cache table and does not recreate it. Every later lookup on the same `Duckdb` logs an error and misses, so build a new `Duckdb` before the next run. * Redis sends `DEL` with the pattern `*`. Redis treats that as a literal key name, so the call removes nothing. With an empty prefix, `clear` returns an error instead. Change the prefix, for example from `docs-v1` to `docs-v2` in the partial-progress snippet. A new prefix works on every backend. Use it to index everything again, for example after you change a prompt or the embedding model. Clearing the cache does not touch the storage. A changed file’s new chunks get new ids and are stored next to the old ones, which stay in LanceDB until you delete the `.ask-your-codebase/` directory. ## Reference [Section titled “Reference”](#reference) * [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html) * [`Pipeline::filter_cached`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.filter_cached) * [`Pipeline::log_errors`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.log_errors) * [`Pipeline::filter_errors`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.filter_errors) * [`Node`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html) * [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) * [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) * [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) * [Feature flags](/reference/feature-flags/) * [ask-your-codebase source](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase) ## Next [Section titled “Next”](#next) [Storage](/indexing/storage/) # Chunking > Split source code with tree-sitter and markdown or plain text by characters, write a chunker as a closure, and keep path and metadata on every chunk. After this page your indexing pipeline splits code along syntax boundaries, markdown and text by characters, and any other format with a closure, with chunks sized for your embedder. A chunker takes one node and returns many; you add it with `then_chunk`. ## Source code [Section titled “Source code”](#source-code) `ChunkCode` needs the `tree-sitter` [feature flag](/reference/feature-flags/): ```sh cargo add swiftide@0.34 --features tree-sitter ``` ```rust // At most 1500 bytes per chunk, the default let python = ChunkCode::try_for_language("python")?; // At most 2000 bytes per chunk; chunks of 200 bytes or less are dropped let rust = ChunkCode::try_for_language_and_chunk_size(SupportedLanguages::Rust, 200..2000)? .with_concurrency(4); Pipeline::from_loader(FileLoader::new("src").with_extensions(&["rs"])) .then_chunk(rust) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/chunking.rs#L17-L28`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/chunking.rs#L17-L28) `ChunkCode` parses each file with tree-sitter and splits it along syntax boundaries such as functions, `impl` blocks and classes. It adds top-level items to a chunk. It closes the chunk when the next item would bring it to the maximum or past it. An item that is too large on its own is split along its children. The size is a `ChunkSize`, and it counts bytes. A number such as `1500` is a maximum. A range such as `200..2000` adds a minimum: a chunk of 200 bytes or less is dropped. `try_for_language` uses a maximum of 1500 bytes. The language is a `SupportedLanguages` value or a name such as `"python"`; an unknown name returns an error. `with_concurrency` sets how many files are chunked at once. Without it the chunker uses the pipeline’s concurrency. When tree-sitter finds a syntax error, `ChunkCode` logs a warning and returns the whole file as one chunk, whatever its size. ## Several languages [Section titled “Several languages”](#several-languages) `ChunkCode` handles one language. The ask-your-codebase showcase indexes several in one pipeline with its own chunker: examples/ask-your-codebase/src/languages.rs ```rust /// Splits each file with a tree-sitter `ChunkCode` for the file's language. /// /// `ChunkCode` is built for a single language; this chunker picks the right one per file from /// its extension, so one pipeline can index a polyglot codebase. #[derive(Clone)] pub struct ChunkByLanguage { chunkers: Vec<(SupportedLanguages, ChunkCode)>, } impl ChunkByLanguage { /// Chunks are between `chunk_size.start` and `chunk_size.end` bytes; smaller ones are dropped. pub fn try_new(languages: &SourceLanguages, chunk_size: Range) -> Result { let chunkers = languages .0 .iter() .map(|&language| { let chunker = ChunkCode::try_for_language_and_chunk_size(language, chunk_size.clone())?; Ok((language, chunker)) }) .collect::>()?; Ok(Self { chunkers }) } fn chunker_for(&self, path: &Path) -> Option<&ChunkCode> { let extension = path.extension()?.to_str()?; self.chunkers .iter() .find(|(language, _)| language.file_extensions().contains(&extension)) .map(|(_, chunker)| chunker) } } #[async_trait] impl ChunkerTransformer for ChunkByLanguage { type Input = String; type Output = String; async fn transform_node(&self, node: TextNode) -> IndexingStream { match self.chunker_for(&node.path) { Some(chunker) => chunker.transform_node(node).await, None => anyhow!("No chunker for {}", node.path.display()).into(), } } } ``` Source: [`examples/ask-your-codebase/src/languages.rs#L39-L85`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/languages.rs#L39-L85) The showcase builds it with a range of `10..4096` bytes. The [tree-sitter integration](/integrations/tree-sitter/#several-languages-in-one-pipeline) explains how it picks a language when extensions overlap. A file without a matching chunker becomes an error in the stream, and the first error stops `run`. Load only the extensions you have chunkers for, as the showcase does, or add `log_errors()` and `filter_errors()` ([Streams and concurrency](/indexing/streams-and-concurrency/)). ## Markdown and text [Section titled “Markdown and text”](#markdown-and-text) ```rust // Aims for 100 characters, at most 1499; chunks under 100 bytes after trimming are dropped let markdown = ChunkMarkdown::from_chunk_range(100..1500); // At most 1000 characters; only blank chunks are dropped let text = ChunkText::from_max_characters(1000); ``` Source: [`snippets/src/indexing/chunking.rs#L41-L45`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/chunking.rs#L41-L45) `ChunkMarkdown` and `ChunkText` use the text-splitter crate and count characters, not bytes. `ChunkMarkdown` prefers to split at headings, then at paragraphs and other blocks. `ChunkText` splits at blank lines and line breaks, then at sentences and words. Both default to a maximum of 2056 characters. With `from_chunk_range`, text-splitter treats the start of the range as the size to aim for and the end, exclusive, as a hard limit. A chunk grows past the start only when that keeps a larger unit, such as a section, in one piece. Chunks that are empty after trimming, or shorter than the start, are dropped. That check uses the byte length of the trimmed text, which equals the character count only for ASCII. The [docs-qa showcase](/showcases/docs-qa/) splits documentation pages with `ChunkMarkdown::from_chunk_range(120..1500)`, because shorter chunks were mostly badges and stray headings. ## Closures [Section titled “Closures”](#closures) ```rust let paragraphs = |node: TextNode| { let chunks: Vec = node .chunk .split("\n\n") .map(str::trim) .filter(|paragraph| !paragraph.is_empty()) .map(str::to_owned) .collect(); // Each chunk keeps the path, metadata and embed mode of `node` IndexingStream::iter(node.into_chunks(chunks).map(Ok)) }; Pipeline::from_loader(FileLoader::new("notes").with_extensions(&["txt"])) .then_chunk(paragraphs) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/chunking.rs#L53-L70`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/chunking.rs#L53-L70) A closure `Fn(TextNode) -> IndexingStream` that is `Clone + Send + Sync` is a chunker. It works on text nodes only. `IndexingStream::iter` turns an iterator of `Result` into a stream. `node.into_chunks` builds one node per chunk from the input node. Every chunk keeps the path, metadata, vectors and embed mode of that node. For anything more involved, or for other node types, implement `ChunkerTransformer` on a struct as the showcase does. ## Node expansion [Section titled “Node expansion”](#node-expansion) ```rust // A JSON Lines file holds one record per line let records = |node: TextNode| { let lines: Vec = node .chunk .lines() .filter(|line| !line.trim().is_empty()) .map(str::to_owned) .collect(); IndexingStream::iter( node.into_chunks(lines) .enumerate() .map(|(index, mut record)| { record.metadata.insert("Record", index); Ok(record) }), ) }; Pipeline::from_loader(FileLoader::new("exports").with_extensions(&["jsonl"])) .then_expand(records) .then_chunk(ChunkText::from_max_characters(1000)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/chunking.rs#L78-L102`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/chunking.rs#L78-L102) `then_expand` runs the same `ChunkerTransformer` as `then_chunk`; only its tracing span is named differently. Use it when one node holds several records, as in this JSON Lines file, and use `then_chunk` when you cut content to size. ## Metadata and parents [Section titled “Metadata and parents”](#metadata-and-parents) Metadata that a node carries before chunking is copied to every chunk. The showcase stores each file’s relative path as metadata before it chunks, so every chunk can cite its file. The same holds for metadata written by a large language model (LLM). Put a transformer such as `MetadataQACode` before the chunker and every chunk of a file carries the questions for the whole file. Put it after the chunker for metadata per chunk, at one request per chunk. `MetadataQACode` also needs the `tree-sitter` feature flag. Since 0.33 every chunk also carries `parent_id`, the id its source node had when it reached the first chunker. The node cache marks that id when the file’s chunks finish. Two consequences matter while you chunk: * A step between `filter_cached` and the chunker that changes the path or the chunk breaks that match, so the file is never cached. Edit the content before `filter_cached`, as the showcase does. * A file whose chunks all fall under the minimum is never cached and is processed again on every run. [Cache keys](/indexing/caching/#cache-keys) explains the match in full. ## Gotchas [Section titled “Gotchas”](#gotchas) * Size chunks for the embedder. With the default embed mode the metadata is embedded together with the chunk, so leave room for it under the model’s input limit. * Chunks come out of `then_chunk` in no particular order, and chunks of different files interleave. `ChunkCode` sets `offset` on each chunk, a running byte count over the chunks before it in the same file. The text chunkers keep the offset of the input node. * Only `ChunkCode` sees syntax. Running `ChunkText` over code cuts through functions at line or sentence boundaries. * `ChunkCode` cannot split a syntax node that has no children. A single token larger than the maximum, such as a long comment or string, is silently left out of the chunks. ## Reference [Section titled “Reference”](#reference) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`ChunkSize`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.ChunkSize.html) * [`SupportedLanguages`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.SupportedLanguages.html) * [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html) * [`ChunkText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkText.html) * [`ChunkerTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChunkerTransformer.html) * [`Pipeline::then_chunk`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_chunk) and [`Pipeline::then_expand`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_expand) * [`Pipeline::filter_errors`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.filter_errors) * [`Node::into_chunks`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html#method.into_chunks) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * The [ask-your-codebase](/showcases/ask-your-codebase/) and [docs-qa](/showcases/docs-qa/) showcases, the [tree-sitter integration](/integrations/tree-sitter/) and [feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [Transformers and metadata](/indexing/transformers-and-metadata/) # Code-aware indexing > Index a codebase with tree-sitter so each code chunk carries a compressed outline of its file, questions about the code, and the symbols it defines and uses. This page builds an indexing pipeline that splits source code along syntax boundaries and gives every chunk the context of its file: an outline trimmed to what the chunk uses, questions the code answers, and the symbols it defines and uses. Tree-sitter is a parser library with one grammar per language. Swiftide wraps it in a chunker and four transformers behind the `tree-sitter` feature flag. The example below also uses OpenAI, a Redis node cache and Qdrant storage: ```sh cargo add swiftide@0.34 --features tree-sitter,openai,redis,qdrant ``` ## Reduced context pipeline [Section titled “Reduced context pipeline”](#reduced-context-pipeline) This example indexes a Rust codebase. It sets up an OpenAI client, the Redis URL for the node cache and a chunk size. It needs `OPENAI_API_KEY`, plus Redis and Qdrant running locally: ```rust let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-3.5-turbo") .build()?; let redis_url = std::env::var("REDIS_URL") .as_deref() .unwrap_or("redis://localhost:6379") .to_owned(); let chunk_size = 2048; ``` Source: [`examples/index_codebase_reduced_context.rs#L36-L46`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase_reduced_context.rs#L36-L46) The pipeline gives every chunk an outline of its file, trimmed to what that chunk uses: ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .filter_cached(Redis::try_from_url( redis_url, "swiftide-examples-codebase-reduced-context", )?) .then( indexing::transformers::OutlineCodeTreeSitter::try_for_language( "rust", Some(chunk_size), )?, ) .then(MetadataQACode::new(openai_client.clone())) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..chunk_size, )?) .then(indexing::transformers::CompressCodeOutline::new( openai_client.clone(), )) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples-codebase-reduced-context") .build()?, ) .run() .await?; ``` Source: [`examples/index_codebase_reduced_context.rs#L50-L78`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase_reduced_context.rs#L50-L78) The steps run in this order: 1. `filter_cached` skips files Redis has already seen, keyed on path and content as [Cache keys](/indexing/caching/#cache-keys) explains. 2. `OutlineCodeTreeSitter` parses the whole file and stores an outline in the `Outline` metadata field. An outline is the file with the bodies of functions and methods removed. Signatures, types, imports and comments stay. The second argument skips files smaller than that many bytes. Here it equals the chunk size, because a file that fits in one chunk needs no outline. 3. `MetadataQACode` asks a large language model (LLM) for questions and answers about the file. Its default prompt includes the outline when the node has one. 4. `ChunkCode` splits the file into chunks of more than 10 and up to 2048 bytes. 5. `CompressCodeOutline` asks the LLM to keep only the outline lines the chunk needs and writes the result back to `Outline`. 6. `Embed` embeds the chunks in batches of 10, and Qdrant stores them with vectors of 1536 dimensions, the size of `text-embedding-3-small`. See [Caching](/indexing/caching/) and [Storage](/indexing/storage/) for the node cache and the storage. Chunking copies the metadata of a file to every chunk, so each chunk starts with the full outline and the questions of its file. The default embed mode embeds metadata together with the chunk. Compressing the outline keeps that embedded context small and specific. ## Cost per chunk [Section titled “Cost per chunk”](#cost-per-chunk) `MetadataQACode` runs before chunking here, so it sends one prompt per file and every chunk shares the answers. Move it after `then_chunk` to get questions per chunk, at one prompt per chunk. `CompressCodeOutline` sends one prompt per chunk of a file larger than the outline threshold. Chunks of smaller files have no outline and cost nothing. On a large codebase that is still many requests. Lower the concurrency or wrap the client in `LanguageModelWithBackOff` to stay within rate limits. ## Question prompt [Section titled “Question prompt”](#question-prompt) `MetadataQACode` writes its reply to `Questions and Answers (code)`. The default prompt asks generic questions. The [ask-your-codebase](/showcases/ask-your-codebase/) showcase replaces it with a prompt about behaviour, which matches the questions developers ask: examples/ask-your-codebase/src/index.rs ````rust /// Asks for the questions a developer would ask that a chunk answers. /// /// The default `MetadataQACode` prompt asks generic questions ("What does this code do?"). /// Questions about behaviour, phrased the way a developer would ask them, match real questions /// much better. const QUESTIONS_PROMPT: &str = r" # Task List the questions a developer could ask about this codebase that the code below answers, each with a short answer. They are stored with the code to help a search find it. Focus on behaviour: when and in which order things happen, what starts or ends a process, which conditions and configuration change the outcome, what happens on errors, and which extension points (traits, hooks, builders, options) exist and when they apply. # Constraints - Write {{ questions }} questions and answers. - Phrase questions the way a developer would ask them, e.g. 'When does the agent stop?'. - Name the relevant types, functions and variants in the answers. - Only use what can be derived from the code. - Respond only in this format: Q1: A1: # Code ``` {{ node.chunk }} ``` "; ```` Source: [`examples/ask-your-codebase/src/index.rs#L39-L70`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L39-L70) The prompt receives `questions` and the node. The default prompt also renders `{{ outline }}`, which `MetadataQACode` sets when the node has an `Outline` field. The showcase prompt leaves it out, because the showcase builds no outline. Add `{{ outline }}` to your own prompt when you use it in the pipeline above, or the outline does not reach the LLM. [Tree-sitter](/integrations/tree-sitter/#questions-per-chunk) shows how to pass a prompt template and the number of questions. ## References and definitions [Section titled “References and definitions”](#references-and-definitions) `MetadataRefsDefsCode` adds the symbols each chunk defines and uses, without an LLM: ```rust indexing::Pipeline::from_loader(FileLoader::new("src").with_extensions(&["rs"])) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) .then(MetadataRefsDefsCode::try_from_language("rust")?) .then_store_with(MemoryStorage::default()) .run() .await?; // The same transformer on one node let refs_defs = MetadataRefsDefsCode::try_from_language("rust")?; let node = refs_defs .transform_node(TextNode::new(r#"fn main() { println!("Hello"); }"#)) .await?; assert_eq!(node.metadata.get(NAME_REFERENCES).unwrap(), "println"); assert_eq!(node.metadata.get(NAME_DEFINITIONS).unwrap(), "main"); ``` Source: [`snippets/src/integrations/tree_sitter.rs#L37-L54`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/tree_sitter.rs#L37-L54) It stores the symbols the chunk defines in `Definitions (code)` and the symbols it uses but does not define in `References (code)`. Both lists are sorted, without duplicates, and joined with commas. A field is left out when nothing is found. The constants `NAME_DEFINITIONS` and `NAME_REFERENCES` in `swiftide::indexing::transformers::metadata_refs_defs_code` hold the field names. Run it after chunking, so each chunk lists its own symbols. ## Languages [Section titled “Languages”](#languages) `ChunkCode`, `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` take a language. `MetadataQACode` and `CompressCodeOutline` take only an LLM client. The constructor for references and definitions is `try_from_language`; the others use `try_for_language`. [Tree-sitter](/integrations/tree-sitter/#supported-languages) lists which languages each transformer supports. Perl needs the `tree-sitter-perl` feature flag. ## Code splitter outside a pipeline [Section titled “Code splitter outside a pipeline”](#code-splitter-outside-a-pipeline) `CodeSplitter` is the splitter inside `ChunkCode`. Use it directly when you need chunks outside an indexing pipeline, for example in a tool: ```rust // Case-insensitive; "c#", "c-sharp", "cpp" and "c++" parse too let language: SupportedLanguages = "python".parse()?; assert_eq!(language.file_extensions(), ["py"]); // Drop chunks of 100 bytes or less, split anything above 1500 bytes let splitter = CodeSplitter::builder() .try_language(language)? .chunk_size(100..1500) .build()?; let chunks = splitter.split(source)?; // Code with a syntax error comes back whole, as one chunk let broken = splitter.split("def main(:\n pass")?; assert_eq!(broken.len(), 1); ``` Source: [`snippets/src/indexing/code.rs#L8-L22`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/code.rs#L8-L22) `SupportedLanguages` parses names without regard to case, and alternative names such as `"c#"`, `"c-sharp"`, `"cpp"` and `"c++"` work. Parsing an unknown name with `parse()` returns a strum parse error that says `Matching variant not found`. The constructors and `try_language` return the error `Treesitter language not supported` instead. The chunk size is a `ChunkSize`. A number is a maximum in bytes; `ChunkSize::default()` is 1500 bytes. A range sets a minimum and a maximum, and a chunk is kept only when it is longer than the minimum. `CodeSplitter::new(language)` uses the default size. ## Gotchas [Section titled “Gotchas”](#gotchas) * The splitter drops chunks at or below the minimum and syntax leaves larger than the maximum, and returns a file with a syntax error as one chunk. [Chunking](/indexing/chunking/) covers these cases. * `OutlineCodeTreeSitter` returns an error for a file with a syntax error. The first error aborts `run`, so add `log_errors()` and `filter_errors()` when your codebase contains files that do not parse. See [Streams and concurrency](/indexing/streams-and-concurrency/). * `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` panic on code in a language they do not handle. In a pipeline the panic happens inside a spawned task, so `run` fails with a join error. * Chunk sizes for code are bytes. `ChunkMarkdown` and `ChunkText` count characters, see [Markdown and text](/indexing/chunking/#markdown-and-text). ## Reference [Section titled “Reference”](#reference) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`OutlineCodeTreeSitter`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.OutlineCodeTreeSitter.html) * [`CompressCodeOutline`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.CompressCodeOutline.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`MetadataRefsDefsCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataRefsDefsCode.html) and the [`metadata_refs_defs_code`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/metadata_refs_defs_code/index.html) module with `NAME_DEFINITIONS` and `NAME_REFERENCES` * [`CodeSplitter`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/struct.CodeSplitter.html) * [`ChunkSize`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.ChunkSize.html) * [`SupportedLanguages`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.SupportedLanguages.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [Tree-sitter integration](/integrations/tree-sitter/) and [feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [How querying works](/querying/overview/) # Embedding > Add dense and sparse embeddings to nodes with Embed and SparseEmbed, choose the embedded text with EmbedMode, and store each embedded field as a named vector in Qdrant. Here every chunk in your indexing pipeline ends up with a dense and a sparse vector, stored in a Qdrant collection that a retriever can search. You add an embedder as a batch transformer, choose which text it embeds, and configure the storage to keep the vectors you need. Add the `fastembed` and `qdrant` feature flags: ```sh cargo add swiftide@0.34 --features fastembed,qdrant ``` ## Dense and sparse embeddings [Section titled “Dense and sparse embeddings”](#dense-and-sparse-embeddings) The docs-qa showcase gives every chunk a dense and a sparse embedding: examples/docs-qa/src/index.rs ```rust indexing::Pipeline::from_loader(loader) .then_chunk(ChunkMarkdown::from_chunk_range( MIN_CHUNK_CHARS..MAX_CHUNK_CHARS, )) // A short title and keywords per chunk; both end up in the embedded text and help the // sparse (keyword) side of hybrid search in particular. .then( MetadataTitle::from_client(llm.clone()) .prompt_template(TITLE_PROMPT) .build()?, ) .then( MetadataKeywords::from_client(llm.clone()) .prompt_template(KEYWORDS_PROMPT) .build()?, ) .then_in_batch(Embed::new(embedders.dense.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_in_batch(SparseEmbed::new(embedders.sparse.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_store_with(qdrant.clone()) .run() .await ``` Source: [`examples/docs-qa/src/index.rs#L50-L70`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/index.rs#L50-L70) `Embed::new` takes any `EmbeddingModel` and fills the node’s `vectors`. `SparseEmbed::new` takes any `SparseEmbeddingModel` and fills `sparse_vectors`. A dense vector captures what the text means. A sparse vector weights the words in it, which helps keyword-like matches in hybrid search. Both are batch transformers, so you add them with `then_in_batch`. Each batch becomes one model call. `with_batch_size` sets the nodes per batch, with a pipeline default of 256. `with_concurrency` sets how many batches run at the same time. Both work on text nodes (`TextNode`) only. A second `Embed` step replaces the dense vectors of the first one instead of adding to them. ## Embed mode [Section titled “Embed mode”](#embed-mode) A node’s embed mode decides which texts the embedder sees, each under an embedded field: | `EmbedMode` | Embedded fields | | ------------------------------ | ----------------------------------------------------- | | `SingleWithMetadata` (default) | `Combined` | | `PerField` | `Chunk`, plus one `Metadata(name)` per metadata field | | `Both` | all of the above | `Combined` is the metadata as `key: value` lines, a newline, then the chunk. Metadata keys come in sorted order. `Chunk` is the chunk alone. `Metadata(name)` is the value of one metadata field. Values that are not strings are embedded as JSON (JavaScript Object Notation). `Pipeline::with_embed_mode` sets the mode on every node that passes that point. Chunkers that use `Node::into_chunks` copy the mode to each chunk, so you can set it right after the loader. Qdrant names each vector after its field: `Combined`, `Chunk` or `Metadata: Title`. Sparse vectors add `_sparse`, as in `Combined_sparse`. LanceDB prefixes the column with `vector_`, as in `vector_combined`. ## Named vectors [Section titled “Named vectors”](#named-vectors) With `EmbedMode::PerField`, every node carries several dense vectors. Qdrant stores each one as a named vector: examples/store_multiple_vectors.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("LICENSE")) .with_concurrency(1) .with_embed_mode(EmbedMode::PerField) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .then(MetadataQAText::new(openai_client.clone())) .then(MetadataSummary::new(openai_client.clone())) .then(MetadataTitle::new(openai_client.clone())) .then(MetadataKeywords::new(openai_client.clone())) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .log_all() .filter_errors() .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-multi-vectors") .with_vector(EmbeddedField::Chunk) .with_vector(EmbeddedField::Metadata(metadata_qa_text::NAME.into())) .with_vector(EmbeddedField::Metadata(metadata_summary::NAME.into())) .with_vector( VectorConfig::builder() .embedded_field(EmbeddedField::Metadata(metadata_title::NAME.into())) .distance(Distance::Manhattan) .build()?, ) .with_vector(EmbeddedField::Metadata(metadata_keywords::NAME.into())) .build()?, ) .run() .await?; ``` Source: [`examples/store_multiple_vectors.rs#L45-L74`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/store_multiple_vectors.rs#L45-L74) The example embeds the chunk and four metadata fields that a large language model (LLM) writes. Each `with_vector` call adds a named vector for one embedded field. Each metadata transformer module has a `NAME` constant, such as `metadata_title::NAME`, so the vector names match. `VectorConfig::builder()` overrides the distance or the size of one vector. When a node has several vectors, Qdrant stores only the fields you configured and skips the rest. Without `with_vector`, the collection has one vector for `Combined`. The first `with_vector` call replaces that default. [Similarity search](/integrations/qdrant/#similarity-search) on a collection with named vectors always queries `Combined`, which the example above lacks. To search it, use `EmbedMode::Both` and add `with_vector(EmbeddedField::Combined)`. ### Single vector collections [Section titled “Single vector collections”](#single-vector-collections) With exactly one dense vector and no sparse vector, Qdrant uses one unnamed vector, both in the collection and in each node that has a single vector. That node skips the field filter. So `with_vector(EmbeddedField::Chunk)` alone breaks in two ways: * `PerField` or `Both` nodes with several vectors arrive as named vectors, and Qdrant rejects the write. * Nodes in the default mode get their `Combined` vector stored without an error. With a single configured vector, configure the field your embed mode produces. ## Hybrid collections [Section titled “Hybrid collections”](#hybrid-collections) Hybrid search queries a dense and a sparse vector. `HybridSearch` uses `Combined` for both by default, and `with_dense_vector_field` and `with_sparse_vector_field` change each one on its own. The hybrid search example loads both models and configures both vectors for `Combined`: examples/hybrid_search.rs ```rust // Ensure all batching is consistent let batch_size = 64; let fastembed_sparse = FastEmbed::try_default_sparse().unwrap().to_owned(); let fastembed = FastEmbed::try_default().unwrap().to_owned(); // Set up openai with the mini model, which is great for indexing let openai = openai::OpenAI::builder() .default_prompt_model("gpt-4o-mini") .build() .unwrap(); // Set up qdrant and use the combined fields (metadata + chunks) for both sparse and dense // vectors let qdrant = Qdrant::builder() .batch_size(batch_size) .vector_size(384) .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name("swiftide-hybrid-example") .build()?; ``` Source: [`examples/hybrid_search.rs#L23-L43`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs#L23-L43) `FastEmbed::try_default()` loads BGE (BAAI General Embedding) small, a dense model with 384 dimensions. `FastEmbed::try_default_sparse()` loads a SPLADE (Sparse Lexical and Expansion) model for sparse vectors. `vector_size` must match the dense model, here 384. Qdrant stores a sparse vector only when its field also has a dense vector configured. So `with_sparse_vector(EmbeddedField::Chunk)` without `with_vector(EmbeddedField::Chunk)` adds a sparse vector that never gets filled. ## Embedding models [Section titled “Embedding models”](#embedding-models) [FastEmbed](/integrations/fastembed/) runs models on your central processing unit (CPU). It is the only integration that produces sparse embeddings. One instance holds one model, and passing a sparse instance to `Embed` fails with “Expected dense model, got sparse”. [FastEmbed](/integrations/fastembed/#model-choice) shows how to load another model, such as a code-trained one. Provider embeddings come from a client with an embedding model. The named vectors example uses OpenAI, which needs the `openai` feature flag: examples/store_multiple_vectors.rs ```rust let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-4o") .build()?; ``` Source: [`examples/store_multiple_vectors.rs#L38-L41`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/store_multiple_vectors.rs#L38-L41) Without `default_embed_model`, every `embed` call fails with “Model not set”. The example sets `vector_size(1536)` to match `text-embedding-3-small`. [Models](/models/overview/) lists which client implements `EmbeddingModel`. Index and query with the same model. A new model needs a new index. ## Batch sizes [Section titled “Batch sizes”](#batch-sizes) Batching happens at three levels: * `with_batch_size` on `Embed` sets the nodes per model call, and `batch_size` on `FastEmbed` the texts per model run, as [FastEmbed](/integrations/fastembed/#batch-size) explains. * `batch_size` on the storage sets the nodes per write. Qdrant defaults to 50. [Batching](/indexing/storage/#batching) covers how the storage receives them. With `PerField`, a node with four metadata fields gives five texts, so a batch of 10 nodes sends 50 texts in one call. ## Gotchas [Section titled “Gotchas”](#gotchas) * A failed model call turns the whole batch into one error. `filter_errors` then drops every node of that batch, see [Errors before a batch step](/indexing/streams-and-concurrency/#errors-before-a-batch-step). * `LanguageModelWithBackOff` does not retry `embed` calls, as [errors and backoff](/models/errors-backoff-and-usage/) explains. * `Combined` puts the metadata before the chunk. A model that truncates long input cuts the end of the chunk first. FastEmbed truncates at its token limit without an error. `fastembed::TextInitOptions::with_max_length` raises that limit, as ask-your-codebase does. * `PerField` embeds every metadata field, including the ones your storage skips. Each one costs embedding time. * A chunker that builds nodes with `TextNode::new` instead of `Node::into_chunks` resets the embed mode to the default. * Clones of a `FastEmbed` share one model behind a lock, so a higher `with_concurrency` does not speed it up. ## Reference [Section titled “Reference”](#reference) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`EmbedMode`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbedMode.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`Pipeline::then_in_batch`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_in_batch) * [`Pipeline::with_embed_mode`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.with_embed_mode) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`QdrantBuilder::with_vector`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.QdrantBuilder.html#method.with_vector) * [`QdrantBuilder::with_sparse_vector`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.QdrantBuilder.html#method.with_sparse_vector) * [`VectorConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.VectorConfig.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OpenAI.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`Node::into_chunks`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html#method.into_chunks) * [Feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [Caching](/indexing/caching/) # Loading data > Start an indexing pipeline from files on disk, nodes you build in code, a tool executor, web pages, Kafka, Fluvio, Parquet or a custom Loader. Every indexing pipeline starts with a stream of nodes. You fill that stream from files, web pages, Kafka, Fluvio, Parquet or your own data, with a loader and `Pipeline::from_loader` or with any stream and `Pipeline::from_stream`. ## Files [Section titled “Files”](#files) ```rust let languages = [SupportedLanguages::Rust, SupportedLanguages::Python]; let extensions: Vec<&str> = languages .iter() .flat_map(|language| language.file_extensions().iter().copied()) .collect(); // Extensions without the leading dot; each call adds to the list let loader = FileLoader::new("src") .with_extensions(&extensions) .with_extensions(&["md"]); Pipeline::from_loader(loader) .then_chunk(ChunkText::from_max_characters(2000)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/loading.rs#L17-L32`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/loading.rs#L17-L32) `FileLoader::new` takes a directory or a single file. Its walk uses the `ignore` crate’s defaults. It skips hidden files and files excluded by `.ignore` files, and by `.gitignore` rules when the root is inside a git repository. `with_extensions` keeps only files with those extensions. Without any call the loader yields every file. `SupportedLanguages::file_extensions` returns the extensions tree-sitter knows for a language. It needs the `tree-sitter` feature flag. `FileLoader` itself needs none. Each file becomes one `TextNode`: * `chunk` holds the file content, with invalid UTF-8 replaced. * `path` is relative when the root is relative. The node id hashes path and chunk, so changing the root from relative to absolute misses the [node cache](/indexing/caching/). * A file that cannot be read becomes an error in the stream. The first error ends `run` unless you filter errors, see [Errors](/indexing/streams-and-concurrency/#errors). ## Nodes in memory [Section titled “Nodes in memory”](#nodes-in-memory) ```rust let mut faq = TextNode::new("A loader turns a data source into a stream of nodes."); faq.path = "faq/loading.md".into(); faq.metadata.insert("Source", "faq"); let nodes = vec![ faq, TextNode::new("An indexing pipeline does nothing until you call run."), ]; let storage = MemoryStorage::default(); Pipeline::from_stream(nodes) .then_store_with(storage.clone()) .run() .await?; println!("Stored {} nodes", storage.get_all_values().await.len()); ``` Source: [`snippets/src/indexing/loading.rs#L40-L55`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/loading.rs#L40-L55) `Pipeline::from_stream` takes anything that converts into an `IndexingStream`. That includes a `Vec`, a boxed `Stream` of results and a Tokio `mpsc` receiver. A channel lets another task produce nodes while the pipeline runs: ```rust let (sender, receiver) = tokio::sync::mpsc::channel(100); tokio::spawn(async move { for line in ["first message", "second message"] { if sender.send(Ok(TextNode::new(line))).await.is_err() { break; } } // Dropping the sender ends the stream, and with it the run }); Pipeline::from_stream(receiver) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/loading.rs#L63-L77`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/loading.rs#L63-L77) The stream ends when every sender is dropped. Until then, `run` keeps waiting for nodes. ## Files from a tool executor [Section titled “Files from a tool executor”](#files-from-a-tool-executor) ```rust let executor = LocalExecutor::new("."); let files = executor .stream_files(Path::new("src"), Some(vec!["rs".to_string()])) .await?; Pipeline::from_stream(files) .then_chunk(ChunkText::from_max_characters(2000)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/loading.rs#L85-L95`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/loading.rs#L85-L95) `ToolExecutor::stream_files` returns the files below a path as an `IndexingStream`. `LocalExecutor` uses `path` as given and does not resolve it against the executor’s working directory. `LocalExecutor` needs the `swiftide-agents` feature flag. ## Web pages [Section titled “Web pages”](#web-pages) examples/scraping_index_to_markdown.rs ```rust indexing::Pipeline::from_loader(ScrapingLoader::from_spider( Website::new("https://www.bosun.ai/") .with_limit(1) .to_owned(), )) .then(HtmlToMarkdownTransformer::default()) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .log_all() .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`examples/scraping_index_to_markdown.rs#L26-L36`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/scraping_index_to_markdown.rs#L26-L36) `ScrapingLoader` crawls with the `spider` crate and needs the `scraping` feature flag. `ScrapingLoader::from_spider` takes a configured `Website`, here limited to one page. To configure it, add `spider` at the version swiftide uses (2.52), or the `Website` types do not match. Each node carries the raw HTML of one page, with the page URL as its `path`. `HtmlToMarkdownTransformer` converts the HTML to markdown and drops `script` and `style` tags. ### Spider Cloud [Section titled “Spider Cloud”](#spider-cloud) Since 0.33, `ScrapingLoader` crawls through Spider Cloud when `SPIDER_CLOUD_API_KEY` is set and not empty. It reads these environment variables once per process: | Variable | Effect | | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | | `SPIDER_CLOUD_API_KEY` | Turns Spider Cloud on | | `SPIDER_CLOUD_MODE` | `proxy`, `api`, `unblocker` or `fallback`. Anything else selects smart mode, which uses the unblocker only against bot protection. | | `SPIDER_CLOUD_API_URL` | Optional API endpoint | Spider Cloud cannot reach hosts that only your machine can route to. A start URL on `localhost`, a `.localhost` or `.local` name, a loopback address, or a private or link-local IPv4 address is crawled directly. ## Kafka and Fluvio [Section titled “Kafka and Fluvio”](#kafka-and-fluvio) examples/kafka.rs ```rust static LOADER_TOPIC: &str = "loader"; static STORAGE_TOPIC: &str = "storage"; let mut client_config = ClientConfig::new(); client_config.set("bootstrap.servers", "localhost:9092"); client_config.set("group.id", "group_id"); client_config.set("auto.offset.reset", "earliest"); let loader = Kafka::builder() .client_config(client_config.clone()) .topic(LOADER_TOPIC) .build() .unwrap(); let storage = Kafka::builder() .client_config(client_config) .topic(STORAGE_TOPIC) .create_topic_if_not_exists(true) .batch_size(2usize) .build() .unwrap(); ``` Source: [`examples/kafka.rs#L27-L47`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/kafka.rs#L27-L47) examples/kafka.rs ```rust indexing::Pipeline::from_loader(loader) .then_in_batch(Embed::new(FastEmbed::try_default().unwrap()).with_batch_size(10)) .then_store_with(storage) .run() .await?; ``` Source: [`examples/kafka.rs#L51-L55`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/kafka.rs#L51-L55) The pipelines embed with `FastEmbed`, which needs the `fastembed` feature flag. `Kafka` needs the `kafka` feature flag, which builds librdkafka and needs cmake and a C toolchain. It takes an `rdkafka` `ClientConfig`, re-exported as `integrations::kafka::ClientConfig`. As a loader it subscribes to `topic` and turns each message payload into a node. Messages without a UTF-8 payload are skipped. The node metadata gets `kafka_topic`, `kafka_partition` and `kafka_offset`, plus `kafka_timestamp` and `kafka_key` when the message has them. `Kafka` is also a storage, here writing to a second topic. examples/fluvio.rs ```rust static TOPIC_NAME: &str = "hello-rust"; static PARTITION_NUM: u32 = 0; let loader = Fluvio::builder() .consumer_config_ext( ConsumerConfigExt::builder() .topic(TOPIC_NAME) .partition(PARTITION_NUM) .offset_start(fluvio::Offset::from_end(1)) .build() .unwrap(), ) .build() .unwrap(); ``` Source: [`examples/fluvio.rs#L36-L49`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/fluvio.rs#L36-L49) `Fluvio` needs the `fluvio` feature flag and is a loader only. It takes a `ConsumerConfigExt` with the topic, partition and start offset. `Offset` comes from the `fluvio` crate, which you add at swiftide’s version (0.50). Each record becomes a node with its key in the `fluvio_key` metadata field, which is null when the record has no key. A Kafka pipeline waits for new messages, so `run` does not return on its own. For both loaders, a receive error ends the run. ## Parquet [Section titled “Parquet”](#parquet) ```rust let loader = Parquet::builder() .path("data.parquet") .column_name("chunk") .build()?; Pipeline::from_loader(loader) .then_chunk(ChunkText::from_max_characters(2000)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/loading.rs#L103-L112`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/loading.rs#L103-L112) `Parquet` needs the `parquet` feature flag. `batch_size` sets the rows read per batch and defaults to 1024. The loader reads one column and turns every non-null value into a node. The column must hold strings. A column of another type yields an error, and a missing column panics. Batches are flattened unordered, so node order can differ from row order. Each load prints the column descriptors to stderr. ## Custom loaders [Section titled “Custom loaders”](#custom-loaders) When no loader fits, implement `Loader`. The [docs-qa](/showcases/docs-qa/) showcase loads markdown and `llms-full.txt` sources as one node per page: examples/docs-qa/src/loader.rs ```rust /// Yields one [`TextNode`] per documentation page, with the page title as metadata. #[derive(Debug, Clone)] pub struct DocsLoader { sources: Vec, } impl DocsLoader { pub fn new(sources: Vec) -> Self { Self { sources } } } impl Loader for DocsLoader { type Output = String; fn into_stream(self) -> IndexingStream { stream::iter(self.sources) .then(DocsSource::pages) .map_ok(|pages| stream::iter(pages).map(|page| Ok(page_node(page)))) .try_flatten() .boxed() .into() } // Only needed when the loader is used as a `Box`. fn into_stream_boxed(self: Box) -> IndexingStream { self.into_stream() } } fn page_node(page: Page) -> TextNode { let mut node = TextNode::new(page.markdown); node.path = page.origin.into(); node.metadata.insert(PAGE_TITLE, page.title.as_str()); node } ``` Source: [`examples/docs-qa/src/loader.rs#L74-L109`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/loader.rs#L74-L109) * `type Output` is the chunk type of the nodes. Built-in transformers expect `String`. * `into_stream` is synchronous and consumes the loader. Put slow work inside the stream, like the page downloads this loader starts through `DocsSource::pages`, so it runs when the pipeline polls it. * A failing source becomes an `Err` item, not a panic. * `into_stream_boxed` is only needed when you use the loader as a `Box`. Its default implementation panics. * `Loader` requires `Send`, `Sync` and `DynClone`, which any `Clone` type implements. ## Gotchas [Section titled “Gotchas”](#gotchas) * `Pipeline::from_loader` calls `into_stream` right away, so work starts when you build the pipeline, not when you call `run`. `Kafka` subscribes to its topic. `Fluvio` connects. `Parquet` opens the file. `ScrapingLoader` starts crawling, and panics outside a Tokio runtime. * `Fluvio` and `Parquet` use `tokio::task::block_in_place`, which panics on a current-thread runtime. `#[tokio::main]` uses a multi-thread runtime by default; `#[tokio::test]` does not, so use `#[tokio::test(flavor = "multi_thread")]`. * `Kafka`, `Fluvio` and `Parquet` panic when they cannot create their consumer or open their file. ## Feature flags [Section titled “Feature flags”](#feature-flags) Install the feature flags you need, for example: ```sh cargo add swiftide@0.34 --features tree-sitter,scraping,kafka,fluvio,parquet,fastembed ``` [Feature flags](/reference/feature-flags/) lists them all. ## Reference [Section titled “Reference”](#reference) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`Pipeline::from_loader`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.from_loader) * [`Pipeline::from_stream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.from_stream) * [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`SupportedLanguages`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.SupportedLanguages.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`ScrapingLoader`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.ScrapingLoader.html) * [`HtmlToMarkdownTransformer`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.HtmlToMarkdownTransformer.html) * [`Kafka`](https://docs.rs/swiftide/0.34/swiftide/integrations/kafka/struct.Kafka.html) * [`Fluvio`](https://docs.rs/swiftide/0.34/swiftide/integrations/fluvio/struct.Fluvio.html) * [`Parquet`](https://docs.rs/swiftide/0.34/swiftide/integrations/parquet/struct.Parquet.html) ## Next [Section titled “Next”](#next) [Chunking](/indexing/chunking/) # How indexing works > An indexing pipeline streams nodes from a loader through filters, transformers, chunkers and embedders into storage, and the pipeline does its work only when you await run(). You can read a Swiftide indexing pipeline line by line and know when each step runs. The example comes from the [ask-your-codebase](/showcases/ask-your-codebase/) showcase. It prepares the source files of a codebase for retrieval-augmented generation (RAG): it splits them into chunks, describes each chunk with a large language model (LLM), embeds it, and stores it in LanceDB. ## The ask-your-codebase pipeline [Section titled “The ask-your-codebase pipeline”](#the-ask-your-codebase-pipeline) examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` Source: [`examples/ask-your-codebase/src/index.rs#L89-L115`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L89-L115) ![Indexing pipeline](/diagrams/diagram-pipeline.svg) Each node passes through the steps from top to bottom until a step drops it: 1. `FileLoader` walks the codebase root and emits one node per file with a matching extension. 2. `filter` drops test files. The closure sees each `Result`, so errors pass through and still surface. 3. `then(test_code::without_unit_tests)` cuts unit tests out of Rust files. 4. `filter_cached` drops files whose path and content did not change since the last successful run. `codebase.node_cache()` is a showcase helper that builds a `Redb` node cache, see [Caching](/indexing/caching/). 5. `then(record_relative_path(..))` stores the file path, relative to the root, as metadata. 6. `then_chunk` splits each file along syntax boundaries with tree-sitter. `ChunkByLanguage` is the showcase’s own wrapper that picks a Swiftide `ChunkCode` per language. One file becomes several nodes. 7. `MetadataQACode` asks the LLM which questions each chunk answers and stores them as metadata. 8. `then_in_batch` with `Embed` embeds each chunk and its metadata with FastEmbed, on the central processing unit (CPU). `with_batch_size` sets 32 nodes per call. Without it, a batch holds 256 nodes. 9. `then_store_with` writes each node to storage. `codebase.vector_store()` builds a `LanceDB` store, see [Storage](/indexing/storage/). 10. `run().await` drives the stream to the end and returns the run’s statistics as `stats`. ## Nodes [Section titled “Nodes”](#nodes) A node is a `Node`. It carries one piece of content and everything known about it. | Field | Holds | | --------------------------- | ------------------------------------------------------------------------------- | | `path` | The source path. | | `chunk` | The content, of type `T`. | | `metadata` | Key and value pairs, sorted by key. | | `vectors`, `sparse_vectors` | Embeddings, one per embedded field. `Embed` and `SparseEmbed` fill them. | | `embed_mode` | Whether the chunk and metadata are embedded together, each on its own, or both. | | `offset` | A running position that `ChunkCode` counts from 0 over the chunks it keeps. | | `parent_id` | The `id()` of the source node when the first chunker split it. | Since 0.31 the pipeline is generic. A `Pipeline` streams `Node`. A chunk is any type that implements the `Chunk` marker trait, which requires `Clone + Send + Sync + Debug + AsRef<[u8]> + 'static`. Most built-in steps take `TextNode`, which is `Node`. `id()` returns a universally unique identifier (UUID) computed from the path and the chunk, so it changes when either changes. The node cache keys on this id, see [Cache keys](/indexing/caching/#cache-keys). ## Steps [Section titled “Steps”](#steps) | Step | Method | Trait | | ----------------- | ------------------------------------------------ | --------------------------------- | | loader | `Pipeline::from_loader`, `Pipeline::from_stream` | `Loader` | | filter | `filter` | a closure over `&Result>` | | node cache | `filter_cached` | `NodeCache` | | transformer | `then` | `Transformer`, or a closure | | chunker | `then_chunk`, `then_expand` | `ChunkerTransformer` | | batch transformer | `then_in_batch` | `BatchableTransformer` | | embedder | `then_in_batch(Embed::new(model))` | `EmbeddingModel` | | storage | `then_store_with` | `Persist` | A chunker returns a stream of nodes for each node. `then_expand` behaves like `then_chunk` with a different name in traces. Use it when you split a node for a reason other than chunking. A step can change the node type. `then` with a transformer whose `Output` is `U` turns a `Pipeline` into a `Pipeline`. Nodes keep flowing after storage. ## Execution [Section titled “Execution”](#execution) The pipeline methods only describe the stream, but their arguments run while you build the chain. In the snippet, `codebase.node_cache()?` creates the Redb file and `embedding::local_model()?` loads the FastEmbed model, which downloads it on first use. `run` takes the pipeline by value and does the work: 1. It calls `setup` on every storage backend. LanceDB creates its table here if it is missing. 2. It pulls nodes through the stream until the stream ends. Some steps spawn Tokio tasks, so `run` needs a Tokio runtime. 3. It marks the source nodes in the node cache, but only after the stream ended without an error. 4. It logs `Pipeline completed` at info level and returns the same numbers as `PipelineStats`, see [Statistics](/indexing/streams-and-concurrency/#statistics). `run` returns `Result`, not the nodes. The showcase reads the number of stored chunks from `stats.nodes_stored`. It counts files with pass-through closures such as `.then(files_found.count())`, because the statistics count nodes at the end of the stream, after chunking. `then`, `then_chunk`, `then_in_batch` and storage work on several nodes at once and pass them on as they finish, see [Streams and concurrency](/indexing/streams-and-concurrency/). The showcase passes `.concurrency(concurrent_requests)` to `MetadataQACode` to bound requests to the LLM. ### Errors and the node cache [Section titled “Errors and the node cache”](#errors-and-the-node-cache) The first error stops the run, and `run` returns it, see [Errors](/indexing/streams-and-concurrency/#errors). It is the error the step failed with, so a `LanguageModelError` can be recovered with `downcast_ref`. A source node is cached only when at least one of its chunks reached the end and none of them failed, and [When nodes are cached](/indexing/caching/#when-nodes-are-cached) covers the consequences. ## Output [Section titled “Output”](#output) Two runs on the `swiftide-agents` crate. `claude-haiku-5-5` wrote the questions, through an OpenAI-compatible gateway to Amazon Web Services (AWS) Bedrock. FastEmbed `jina-embeddings-v2-base-code` computed the embeddings. ```text $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s ``` The second run, with no file changed: ```text $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s ``` The node cache drops all 14 files at step 4, so no prompt is sent. ## Feature flags [Section titled “Feature flags”](#feature-flags) `Pipeline`, `FileLoader` and `Embed` need no feature flag. The showcase turns off the defaults and turns on `tree-sitter`, `fastembed`, `lancedb`, `redb` and `openai`. To add the same features to your crate: ```sh cargo add swiftide@0.34 --no-default-features --features tree-sitter,fastembed,lancedb,redb,openai ``` TLS needs no setup, because swiftide’s HTTP clients bring their own rustls crypto provider ([TLS](/integrations/overview/#tls)). [Feature flags](/reference/feature-flags/) lists every flag. ## Reference [Section titled “Reference”](#reference) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`PipelineStats`](https://docs.rs/swiftide/0.34/swiftide/indexing/statistics/struct.PipelineStats.html) * [`Node`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html) * [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`EmbedMode`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbedMode.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html) * [`Transformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Transformer.html) * [`ChunkerTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChunkerTransformer.html) * [`BatchableTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.BatchableTransformer.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) ## Next [Section titled “Next”](#next) [Loading data](/indexing/loading/) # Storage > Store indexed nodes with then_store_with in LanceDB, Qdrant, PgVector, DuckDB, Redis, Kafka or MemoryStorage, and write your own storage by implementing the Persist trait. `then_store_with` writes every node to a storage backend. You end up with nodes in LanceDB or Qdrant, a clear picture of what each backend keeps of a node, and a storage of your own built on the `Persist` trait. ## LanceDB [Section titled “LanceDB”](#lancedb) The ask-your-codebase showcase keeps its vectors in an embedded LanceDB table, in a directory next to the code. There is no server to run. Enable the `lancedb` feature flag: ```sh cargo add swiftide@0.34 --features lancedb ``` examples/ask-your-codebase/src/codebase.rs ```rust /// Remembers which files were fully indexed, so unchanged files are skipped next time. pub fn node_cache(&self) -> Result { std::fs::create_dir_all(&self.data_dir)?; Redb::builder() .database_path(self.data_dir.join("cache.redb")) .table_name("indexed_files") .build() } /// Embedded vector store with one vector per chunk and the source path next to it. pub fn vector_store(&self) -> Result { let uri = self.lancedb_dir(); LanceDB::builder() .uri(uri.to_str().context("Index path is not valid UTF-8")?) .table_name("code_chunks") .vector_size(embedding::DIMENSIONS) .with_vector(EmbeddedField::Combined) .with_metadata(PATH_FIELD) .build() } ``` Source: [`examples/ask-your-codebase/src/codebase.rs#L56-L77`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/codebase.rs#L56-L77) `vector_store` builds the storage; `node_cache` belongs to [Caching](/indexing/caching/). The showcase pipeline passes the storage to `then_store_with`, after the embedding step. * `uri` is the directory that holds the table. It is required unless you pass a `connection_pool`. * `table_name` defaults to `swiftide`. * `vector_size` must match the output of your embedder. The showcase uses 768, the size of its FastEmbed code model. * `with_vector(EmbeddedField::Combined)` adds a column for the embedding of the chunk together with its metadata. * `with_metadata(PATH_FIELD)` adds a column for the `path` metadata field. LanceDB stores only what you configure: the node id, the chunk, each configured vector and each metadata field named with `with_metadata`. The node’s own `path` is not a column. The showcase stores the path relative to the codebase root as metadata, so answers can cite their sources. Metadata columns hold text, so a value that is not a JSON string is stored as null. Before the first node arrives, setup creates the table if it does not exist. Nodes are written in batches of 256 with a merge on `id`, so storing the same node twice updates its row. See [LanceDB](/integrations/lancedb/) for retrieval and remote storage. ## Qdrant [Section titled “Qdrant”](#qdrant) The docs-qa showcase stores a dense and a sparse vector for every chunk in a Qdrant collection. Enable the `qdrant` feature flag: ```sh cargo add swiftide@0.34 --features qdrant ``` examples/docs-qa/src/store.rs ```rust let qdrant = Qdrant::builder() .batch_size(BATCH_SIZE) .vector_size(DENSE_VECTOR_SIZE) // One dense and one sparse vector, both over the chunk *and* its metadata .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name(&collection.0) .build()?; ``` Source: [`examples/docs-qa/src/store.rs#L38-L45`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/store.rs#L38-L45) Qdrant keeps more of the node than LanceDB. Each point’s payload holds every metadata field, plus `path`, `content` (the chunk) and `last_updated_at`. The point id is the node id. Nodes without vectors fail with “Node without vectors”. Setup creates the collection when it is missing. An existing collection is kept as it is, even when its vector configuration differs from the builder. The client reads `QDRANT_URL` (default `http://localhost:6334`, gRPC) and `QDRANT_API_KEY`. The default batch size is 50; docs-qa raises it to 64. A debug build of docs-qa split four markdown pages into 43 chunks and stored them in Qdrant (metadata from `claude-haiku-5-5`, embeddings from local FastEmbed): ```text Indexed 43 chunks into `docs-qa` in 71.7s ``` [Qdrant](/integrations/qdrant/) covers named vectors, hybrid search and filters. ## Backends [Section titled “Backends”](#backends) Every backend sits behind its own feature flag, except `MemoryStorage`: | Storage | Feature flag | What it stores | Default batch size | Retrieval | | --------------- | ------------ | -------------------------------------------------------------- | ------------------ | --------- | | `LanceDB` | `lancedb` | id, chunk, configured vectors, `with_metadata` fields | 256 | yes | | `Qdrant` | `qdrant` | vectors; payload with all metadata, path, chunk | 50 | yes | | `PgVector` | `pgvector` | id, chunk, configured vectors, `with_metadata` fields as JSONB | 50 | yes | | `Duckdb` | `duckdb` | `uuid`, chunk, path, configured vectors; no metadata | none, node by node | yes | | `Redis` | `redis` | the node as JSON under `:` | 10 | no | | `Kafka` | `kafka` | the node as JSON, keyed `:`, on a topic | 32 | no | | `MemoryStorage` | none | the whole node, in memory | none, node by node | no | PgVector fails to store a node that lacks a field named with `with_metadata`; see [PgVector](/integrations/pgvector/). Kafka’s setup fails when the topic does not exist, unless you set `create_topic_if_not_exists`. Kafka is also a loader; see [Kafka and Fluvio](/indexing/loading/#kafka-and-fluvio). DuckDB stores node by node and ignores its builder’s `batch_size`. It rejects a node that lacks a vector for a configured field. When a row with the same `uuid` exists, DuckDB keeps the old row unless you set `upsert_vectors`, which needs DuckDB 1.2.1 or later. Redis works as a key-value store for nodes. `persist_key_fn` changes the key and `persist_value_fn` the value. Both take a plain `fn` pointer, so a closure that captures variables does not compile. The example writes only the metadata and raises the batch size from 10 to 50: examples/index_into_redis.rs ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) .then_store_with( // By default the value is the full node serialized to JSON. // We can customize this by providing a custom function. Redis::try_build_from_url(&redis_url)? .persist_value_fn(|node| Ok(serde_json::to_string(&node.metadata)?)) .batch_size(50) .build()?, ) .run() .await?; ``` Source: [`examples/index_into_redis.rs#L39-L53`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_into_redis.rs#L39-L53) ### DuckDB [Section titled “DuckDB”](#duckdb) No swiftide example uses DuckDB, so this section describes the builder without code. `Duckdb::builder()` returns a `DuckdbBuilder` with these methods: * `connection` takes anything that converts into a `duckdb::Connection`. Swiftide does not re-export the `duckdb` crate, so add it to your crate yourself. * `with_vector(field, size)` adds a vector column of `size` floats for an `EmbeddedField`, such as `EmbeddedField::Combined`. * `table_name` defaults to `swiftide`, and `cache_table` to `swiftide_cache`. `cache_key_prefix` prefixes the node cache keys, which also lets you invalidate the cache by hand. * `schema` replaces the SQL that loads the extensions and creates the table. The upsert statement is built from the table name and vectors, so keep the column names when you change it. `upsert_vectors` is the flag described above. `Duckdb` implements `Persist` for `then_store_with`, `NodeCache` for `filter_cached`, and `Retrieve` for three search strategies: `SimilaritySingleEmbedding`, `HybridSearch` and `CustomStrategy`. The setup installs and loads the `vss` and `fts` extensions before it creates the table. Retrieved documents carry `id` and `path` as metadata, since no other metadata is stored. See [`DuckdbBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.DuckdbBuilder.html) for every method. ## MemoryStorage [Section titled “MemoryStorage”](#memorystorage) `MemoryStorage` lives in `swiftide::indexing::persist` and needs no feature flag. It suits tests and experiments: ```rust let storage = MemoryStorage::default(); Pipeline::from_stream(vec![ TextNode::new("Swiftide is a Rust library for building LLM applications."), TextNode::new("An indexing pipeline loads, transforms and stores nodes."), ]) .then(|mut node: TextNode| -> anyhow::Result { let words = node.chunk.split_whitespace().count(); node.metadata.insert("Words", words); Ok(node) }) // Clones share the same map, so keep one to read from later .then_store_with(storage.clone()) .run() .await?; for node in storage.get_all_values().await { println!("{:?} words: {}", node.metadata.get("Words"), node.chunk); } ``` Source: [`snippets/src/indexing/storage.rs#L15-L33`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/storage.rs#L15-L33) Its keys are an incrementing counter, not the node id, so storing the same node twice keeps two copies. `get(key)` reads one node, `get_all_values()` returns all nodes, and `get_all()` returns them with their keys. It has no retrieval for query pipelines. ## Batching [Section titled “Batching”](#batching) A storage chooses how it receives nodes through `batch_size`. When it returns `Some(n)`, the pipeline groups up to `n` nodes and calls `batch_store`. When it returns `None`, the pipeline calls `store` for each node. In both cases, up to the pipeline’s concurrency of these calls run at once. Storage has no concurrency setting of its own, and [Concurrency](/indexing/streams-and-concurrency/#concurrency) explains when the pipeline value is read. For how this differs from the embedder’s batch size, see [Batch sizes](/indexing/embedding/#batch-sizes). Stored nodes flow on, so you can chain a second `then_store_with` to write to two backends. If `setup` fails, for example because Qdrant is unreachable, `run` returns that error. To keep going after a failed write, see [Errors](/indexing/streams-and-concurrency/#errors). ## Custom storage [Section titled “Custom storage”](#custom-storage) Implement `Persist` to write nodes anywhere else. This one appends each node as a line of JSON: ```rust /// Appends every node as one line of JSON to a file. #[derive(Debug, Clone)] pub struct JsonLines { path: PathBuf, // Batches are stored concurrently; one writer at a time keeps lines whole write_lock: Arc>, } impl JsonLines { pub fn new(path: impl Into) -> Self { Self { path: path.into(), write_lock: Arc::default(), } } async fn append(&self, nodes: &[TextNode]) -> anyhow::Result<()> { let mut lines = String::new(); for node in nodes { lines.push_str(&serde_json::to_string(node)?); lines.push('\n'); } let _guard = self.write_lock.lock().await; let mut file = OpenOptions::new() .create(true) .append(true) .open(&self.path) .await?; file.write_all(lines.as_bytes()).await?; Ok(()) } } #[async_trait] impl Persist for JsonLines { type Input = String; type Output = String; // Runs once, before the first node arrives async fn setup(&self) -> anyhow::Result<()> { if let Some(dir) = self.path.parent() { tokio::fs::create_dir_all(dir).await?; } Ok(()) } // Used when `batch_size` returns `None` async fn store(&self, node: TextNode) -> anyhow::Result { self.append(std::slice::from_ref(&node)).await?; Ok(node) } // Used when `batch_size` returns `Some`; pass the stored nodes on async fn batch_store(&self, nodes: Vec) -> IndexingStream { self.append(&nodes).await.map(|()| nodes).into() } fn batch_size(&self) -> Option { Some(100) } } ``` Source: [`snippets/src/indexing/storage.rs#L39-L100`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/storage.rs#L39-L100) * `Input` is the chunk type the storage accepts, and `Output` the chunk type it passes on. Both are `String` for text nodes. * `batch_store` returns an `IndexingStream`. Return the stored nodes so later steps receive them, or an error. * `Persist` requires `Debug`, `Clone`, `Send` and `Sync`, so fields like `Rc` or `RefCell` do not compile. Implement it with `#[async_trait]`. Batches are stored concurrently, so guard anything they share. Use the storage like any other: ```rust Pipeline::from_stream(vec![TextNode::new("Stored as a line of JSON.")]) .then_store_with(JsonLines::new("index/nodes.jsonl")) .run() .await?; ``` Source: [`snippets/src/indexing/storage.rs#L106-L109`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/storage.rs#L106-L109) ## Re-indexing [Section titled “Re-indexing”](#re-indexing) A node’s id is computed from its path and its chunk, as [Cache keys](/indexing/caching/#cache-keys) explains. When you index an unchanged chunk again, LanceDB, Qdrant, PgVector and DuckDB find the same id and keep one row. DuckDB keeps the old row as it is unless `upsert_vectors` is set. When a chunk changes, it gets a new id and a new row, and the row of the old version stays. Swiftide does not delete rows for you. docs-qa’s `--recreate` flag drops the Qdrant collection before indexing. For incremental updates, delete a file’s rows yourself before you index its new version, and skip unchanged files with a [node cache](/indexing/caching/). ## Reference [Section titled “Reference”](#reference) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`Pipeline::then_store_with`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_store_with) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`MemoryStorage`](https://docs.rs/swiftide/0.34/swiftide/indexing/persist/struct.MemoryStorage.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html) * [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) * [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) * [`Kafka`](https://docs.rs/swiftide/0.34/swiftide/integrations/kafka/struct.Kafka.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [ask-your-codebase source](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase) * [docs-qa source](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa) ## Next [Section titled “Next”](#next) [Streams and concurrency](/indexing/streams-and-concurrency/) # Streams and concurrency > Set how many nodes an indexing pipeline processes at once, throttle it, split and merge its stream, and drop failed nodes without losing good ones. You set how many nodes each step of an indexing pipeline handles at once, how fast they reach a slow step, which steps they pass through, and what happens to the ones that fail. The pipeline is a stream of nodes, and every step is a stage on that stream. The snippets use `OpenAI`, which needs the `openai` feature flag, and `ChunkCode`, which needs the `tree-sitter` feature flag: ```sh cargo add swiftide@0.34 --features openai,tree-sitter ``` ## Concurrency [Section titled “Concurrency”](#concurrency) ```rust indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) // Applies to every step added after this call .with_concurrency(8) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) // At most one chunk every 50ms enters the next step .throttle(Duration::from_millis(50)) // Its own concurrency wins over the pipeline's 8 .then(MetadataQAText::new(openai.clone()).with_concurrency(2)) .then_in_batch(Embed::new(openai).with_batch_size(64)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/streams.rs#L19-L30`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/streams.rs#L19-L30) Each transformer, chunker and storage step spawns a Tokio task per node, or per batch for batch transformers and batch storage, and keeps up to its concurrency of them running. Results leave a step as they finish, so the order of nodes is not preserved. `with_concurrency` sets the pipeline value. The default is the number of logical cores of the machine. A step reads the value when you add it, so call `with_concurrency` before the steps it should apply to. A transformer, batch transformer or chunker can bring its own limit through `concurrency()`. When that returns a value, it wins over the pipeline’s. The built-in large language model (LLM) transformers and `Embed` set it with `with_concurrency`, as `MetadataQAText` does above. Storage has no such override and always uses the pipeline value. `throttle` lets at most one node through per duration, at the point where you put it. Here it caps the chunks that reach the LLM at 20 per second, whatever the concurrency. ## Split and merge [Section titled “Split and merge”](#split-and-merge) ```rust let (markdown, code) = indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["md", "rs"])) .split_by(|result| { result .as_ref() .is_ok_and(|node| node.path.extension().is_some_and(|ext| ext == "md")) }); let markdown = markdown.then_chunk(ChunkMarkdown::from_chunk_range(100..2048)); let code = code.then_chunk(ChunkCode::try_for_language("rust")?); // One set of statistics for both branches let stats = markdown .merge(code) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/streams.rs#L38-L54`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/streams.rs#L38-L54) `split_by` turns one pipeline into two. The predicate gets each item as a `Result`, errors included. Items for which it returns true go to the first pipeline, the rest to the second. Here, errors go to the code branch. `merge` joins two pipelines with the same node type into one stream. Node cache registrations and recorded failures of both sides are combined, so a `filter_cached` inside either branch still marks its sources and failures in the other branch keep them out of the cache. Each branch of `split_by` gets its own statistics collector, seeded with a copy of what was collected before the split, so a branch you run on its own reports only its own nodes. A collector handle taken before the split sees neither branch. `merge` keeps the collector of the pipeline you call it on, so `run` after the merge returns one `PipelineStats` for both branches. ### Split gotchas [Section titled “Split gotchas”](#split-gotchas) * `split_by` is not lazy. It spawns a task that starts reading the stream at once and sends each item into one of two channels that hold 1000 items each. * When one branch is not read, its channel fills up and the split stops, which also stalls the other branch. Run both branches together: merge them, or run them at the same time with `tokio::join!`. Running one to completion before starting the other can hang once the waiting branch has 1000 items. * If a branch is dropped, the split task panics on its next send to it. The panic stays inside that task, and the other branch then ends early without an error. * `merge` keeps the storage setup of the pipeline you call it on and drops that of the pipeline you pass in. Add storage after the merge, as above, or to the branch you call `merge` on. ## Errors [Section titled “Errors”](#errors) ```rust indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .filter(|result| match result { Ok(node) => !node.chunk.trim().is_empty(), // Keep errors, so `log_errors` sees them Err(_) => true, }) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then(MetadataQAText::new(openai.clone())) // Drop failed nodes before the batch step, so an error does not take its batch with it .log_errors() .filter_errors() .then_in_batch(Embed::new(openai).with_batch_size(64)) .log_nodes() .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/indexing/streams.rs#L64-L79`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/streams.rs#L64-L79) `run` stops at the first error a step returns and returns that error, and a node cache marks nothing from a failed run. The steps above drop failed nodes so the run continues. The returned error is the one the step failed with, even when a batch step follows it. A `LanguageModelError` from an LLM transformer can be recovered with `downcast_ref` or found in `chain()`, see [Errors, backoff and usage](/models/errors-backoff-and-usage/). * `filter` keeps an item when the closure returns true. It sees nodes and errors, so it can drop specific errors as well as nodes. This one drops empty files and keeps every error. * `log_errors` logs each error at ERROR level and passes it on. * `filter_errors` drops errors without logging them. * `log_nodes` logs each node at DEBUG level. `log_all` is `log_errors` followed by `log_nodes`. Order matters. `filter_errors` before `log_errors` leaves nothing to log. Each pair covers only the steps before it, and errors from steps added after it still abort the run. Add another pair after `then_store_with` if a failed write should not stop the run. ### Errors before a batch step [Section titled “Errors before a batch step”](#errors-before-a-batch-step) A batch step collects nodes until it has a full batch. Those are `then_in_batch`, and `then_store_with` when the storage has a batch size. When an error reaches it, the step passes the error on and drops the nodes it has collected so far, without a trace. A `filter_errors` after the batch step keeps the run going, but those good nodes are already gone. With a batch size of 64, one failed LLM call can drop up to 63 good chunks. Those dropped nodes are not recorded as failed. When other chunks of the same source complete, the node cache marks that source and the next run skips it. Put a `filter_errors` pair before every batch step, as the snippet does before `then_in_batch`. The source of a node that fails in a transformer stays out of the node cache, so the next run retries it. [Caching](/indexing/caching/) covers when sources are marked. ## Statistics [Section titled “Statistics”](#statistics) `run` returns the `PipelineStats` of the run, with node counts, token usage and timing, as `let stats = pipeline.run().await?;`. It also logs them as the INFO event `Pipeline completed`. `run` takes the pipeline by value, and `stats()` before `run` returns the empty initial statistics. To watch a run while it is going, clone the handle from `stats_collector()` before you call `run`. [Pipeline statistics](/observability/metrics-and-statistics/#pipeline-statistics) shows both. ## Streams [Section titled “Streams”](#streams) Loaders, chunkers, batch transformers and batch storage return their nodes as an `IndexingStream`. `IndexingStream::iter` takes any iterator of results, `IndexingStream::from_nodes` a `Vec` of nodes, and `IndexingStream::empty` makes an empty stream. Use them to return nodes from your own chunkers and batch transformers. When you build a pipeline with `Pipeline::from_stream`, a `Result>` that is an error and an `anyhow::Error` each become a stream with that one error item. A Tokio channel’s capacity applies backpressure: `send` waits while the pipeline is behind. [Loading](/indexing/loading/#nodes-in-memory) shows `from_stream` with a vector and a channel. ## Reference [Section titled “Reference”](#reference) * [`Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) (`with_concurrency`, `throttle`, `then`, `then_chunk`, `then_in_batch`, `then_store_with`, `split_by`, `merge`, `filter`, `filter_cached`, `filter_errors`, `log_all`, `log_errors`, `log_nodes`, `from_stream`, `run`, `stats`, `stats_collector`) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`Transformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Transformer.html) (`concurrency`) * [`BatchableTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.BatchableTransformer.html) (`concurrency`) * [`ChunkerTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChunkerTransformer.html) (`concurrency`) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`MemoryStorage`](https://docs.rs/swiftide/0.34/swiftide/indexing/persist/struct.MemoryStorage.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OpenAI.html) ## Next [Section titled “Next”](#next) [Code-aware indexing](/indexing/code-aware-indexing/) # Transformers and metadata > Add metadata to nodes with closures, the built-in LLM metadata transformers and their prompts, and your own transformers and batch transformers. You add metadata to nodes with closures, with the built-in transformers that ask a large language model (LLM), or with a transformer you write yourself. Metadata travels with each node to the embedder, so a retriever can match on it and an answer can cite it. ## Closures [Section titled “Closures”](#closures) The [ask-your-codebase](/showcases/ask-your-codebase/) showcase stores each file’s path, relative to the codebase root, on every node. examples/ask-your-codebase/src/index.rs ```rust /// Stores the file path, relative to the codebase root, as metadata on the node. /// /// The vector store keeps this field next to each chunk, so answers can cite their sources. fn record_relative_path( codebase: Codebase, ) -> impl Fn(TextNode) -> Result + Clone + Send + Sync + 'static { move |mut node| { let path = codebase .relative_path(&node.path) .to_string_lossy() .into_owned(); node.metadata.insert(PATH_FIELD, path); Ok(node) } } ``` Source: [`examples/ask-your-codebase/src/index.rs#L127-L141`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L127-L141) A closure from `TextNode` to `Result` is a transformer if it is `Clone`, `Send`, `Sync` and `'static`, so it cannot borrow local data. You add it with `.then(...)`. The pipeline runs it on many nodes at once and does not keep their order. `node.metadata` is a `Metadata` map. `insert` takes a key that converts into a `String` and a value that converts into a `serde_json::Value`. LanceDB and PgVector store a field only after you declare it with `with_metadata`, which is why the showcase calls `.with_metadata(PATH_FIELD)`. ## Built-in metadata transformers [Section titled “Built-in metadata transformers”](#built-in-metadata-transformers) Swiftide ships six transformers that prompt the LLM per node and store the reply as metadata: | Transformer | Metadata field | Feature flag | | --------------------- | ------------------------------ | ------------- | | `MetadataQAText` | `Questions and Answers (text)` | none | | `MetadataQACode` | `Questions and Answers (code)` | `tree-sitter` | | `MetadataSummary` | `Summary` | none | | `MetadataTitle` | `Title` | none | | `MetadataKeywords` | `Keywords` | none | | `CompressCodeOutline` | `Outline` | `tree-sitter` | Each module exports the field name as a `NAME` constant, such as `metadata_title::NAME`. `CompressCodeOutline` skips nodes without an `Outline` field; [Code-aware indexing](/indexing/code-aware-indexing/) shows how to fill it. The next example needs the `openai` and `qdrant` feature flags. examples/index_markdown_lots_of_metadata.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("README.md").with_extensions(&["md"])) .with_concurrency(1) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .then(MetadataQAText::new(openai_client.clone())) .then(MetadataSummary::new(openai_client.clone())) .then(MetadataTitle::new(openai_client.clone())) .then(MetadataKeywords::new(openai_client.clone())) .then_in_batch(Embed::new(openai_client.clone())) .log_all() .filter_errors() .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples") .build()?, ) .run() .await?; ``` Source: [`examples/index_markdown_lots_of_metadata.rs#L39-L57`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_markdown_lots_of_metadata.rs#L39-L57) `new(client)` uses the default prompt. `from_client(client)` returns the builder, where you set `prompt_template`, `concurrency` and, for the question transformers, `num_questions` (default 5). `builder()` and `Default` start without a client. After the chunker, each chunk gets its own prompt and reply. Before it, one prompt per file is enough, and chunking copies the reply to every chunk of that file. `concurrency(n)` on the builder, or `with_concurrency(n)`, limits the requests that step sends at once. [Concurrency](/indexing/streams-and-concurrency/#concurrency) explains how this interacts with the pipeline value. By default the embedder sees every metadata field as a `key: value` line in front of the chunk. [Embed mode](/indexing/embedding/#embed-mode) explains how to embed fields separately. ## Prompts [Section titled “Prompts”](#prompts) The default prompts show the expected answer inside a code fence, and models tend to copy that fence into the metadata. The [docs-qa](/showcases/docs-qa/) showcase replaces the `MetadataTitle` and `MetadataKeywords` prompts: examples/docs-qa/src/index.rs ```rust // Swiftide's default metadata prompts show the expected format inside a code fence, and models // tend to copy the fence into the metadata. These ask for plain text and use the page title. // A code-only chunk is titled by the API it shows, because the reranker scores the title too. const TITLE_PROMPT: &str = "Write a short, descriptive title (at most 10 words) for this \ excerpt from the \"{{ node.metadata.Page }}\" documentation. If the excerpt is mostly code, \ name the API it demonstrates (a macro, trait or builder method), not what the example does. \ Reply with the title only: no quotes, no markdown, no code fences. {{ node.chunk }}"; const KEYWORDS_PROMPT: &str = "List up to 8 keywords for this excerpt from the \ \"{{ node.metadata.Page }}\" documentation: names of types, crates, features, tools, commands \ and concepts that literally appear in it. Reply with one comma-separated line only: no \ numbering, no markdown, no code fences. {{ node.chunk }}"; ``` Source: [`examples/docs-qa/src/index.rs#L24-L39`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/index.rs#L24-L39) The pipeline passes them to the builders with `prompt_template`: examples/docs-qa/src/index.rs ```rust indexing::Pipeline::from_loader(loader) .then_chunk(ChunkMarkdown::from_chunk_range( MIN_CHUNK_CHARS..MAX_CHUNK_CHARS, )) // A short title and keywords per chunk; both end up in the embedded text and help the // sparse (keyword) side of hybrid search in particular. .then( MetadataTitle::from_client(llm.clone()) .prompt_template(TITLE_PROMPT) .build()?, ) .then( MetadataKeywords::from_client(llm.clone()) .prompt_template(KEYWORDS_PROMPT) .build()?, ) .then_in_batch(Embed::new(embedders.dense.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_in_batch(SparseEmbed::new(embedders.sparse.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_store_with(qdrant.clone()) .run() .await ``` Source: [`examples/docs-qa/src/index.rs#L50-L70`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/index.rs#L50-L70) `prompt_template` takes anything that converts into a `Prompt`, such as a `&'static str` holding a Tera template. Each transformer adds the node as `node`, so `{{ node.chunk }}` is the chunk and `{{ node.metadata.Page }}` is a field that the docs-qa loader set. The question transformers also add `{{ questions }}`, and `MetadataQACode` adds `{{ outline }}` when the node has an `Outline` field. In the captured docs-qa run, four pages became 43 chunks, and the title and keywords steps made 86 calls to `claude-haiku-5-5` through an OpenAI-compatible gateway to Amazon Web Services (AWS) Bedrock. The whole run, including local embedding and the writes to Qdrant, took: ```text Indexed 43 chunks into `docs-qa` in 71.7s ``` The ask-your-codebase prompt asks for questions a developer would ask: examples/ask-your-codebase/src/index.rs ````rust /// Asks for the questions a developer would ask that a chunk answers. /// /// The default `MetadataQACode` prompt asks generic questions ("What does this code do?"). /// Questions about behaviour, phrased the way a developer would ask them, match real questions /// much better. const QUESTIONS_PROMPT: &str = r" # Task List the questions a developer could ask about this codebase that the code below answers, each with a short answer. They are stored with the code to help a search find it. Focus on behaviour: when and in which order things happen, what starts or ends a process, which conditions and configuration change the outcome, what happens on errors, and which extension points (traits, hooks, builders, options) exist and when they apply. # Constraints - Write {{ questions }} questions and answers. - Phrase questions the way a developer would ask them, e.g. 'When does the agent stop?'. - Name the relevant types, functions and variants in the answers. - Only use what can be derived from the code. - Respond only in this format: Q1: A1: # Code ``` {{ node.chunk }} ``` "; ```` Source: [`examples/ask-your-codebase/src/index.rs#L39-L70`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L39-L70) See [Prompts and templates](/models/prompts-and-templates/) for Tera syntax. ## Default LLM client [Section titled “Default LLM client”](#default-llm-client) You can set one client on the pipeline instead of on each transformer. This example needs the `openai` feature flag. ```rust Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) // Set the client before the steps that use it .with_default_llm_client(openai) .then_chunk(ChunkMarkdown::from_chunk_range(100..1500)) .then(MetadataTitle::default()) .then(MetadataSummary::default()) .then( MetadataQAText::builder() .num_questions(3_usize) .concurrency(4_usize) .build()?, ) .then_store_with(MemoryStorage::default()) .run() .await ``` Source: [`snippets/src/indexing/transformers.rs#L21-L35`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/transformers.rs#L21-L35) `then` hands the pipeline defaults to the transformer when you add the step, so call `with_default_llm_client` first. A client on the transformer takes precedence. Without either, each node fails with `No client provided`. ## Custom transformers [Section titled “Custom transformers”](#custom-transformers) A transformer is a type that implements `Transformer`. ```rust /// Stores the number of words in a chunk as the `Words` metadata field. #[derive(Clone)] pub struct WordCount; #[async_trait] impl Transformer for WordCount { type Input = String; type Output = String; async fn transform_node(&self, mut node: TextNode) -> anyhow::Result { let words = node.chunk.split_whitespace().count(); node.metadata.insert("Words", words); Ok(node) } } // `then` hands pipeline defaults to every transformer; this one has no use for them. impl WithIndexingDefaults for WordCount {} ``` Source: [`snippets/src/indexing/transformers.rs#L40-L57`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/transformers.rs#L40-L57) `transform_node` receives one node and returns it, changed, or an error. The type must be `Clone`, `Send`, `Sync` and `'static`. The trait is an `async_trait`, so add the `async-trait` crate. Override `concurrency` to give the step its own limit. `then` also requires `WithIndexingDefaults`; an empty `impl` is enough. ### Transformer macro [Section titled “Transformer macro”](#transformer-macro) The built-in LLM transformers use `#[indexing_transformer]`. This one asks who a chunk is for: ```rust /// The macro adds items next to the struct, so each annotated struct gets its own module. pub mod metadata_audience { use async_trait::async_trait; use swiftide::{indexing::TextNode, indexing_transformer, traits::Transformer}; /// Asks the LLM who a chunk is written for and stores the reply as `Audience`. #[indexing_transformer( metadata_field_name = "Audience", default_prompt_file = "prompts/metadata_audience.prompt.md" )] pub struct MetadataAudience { /// Product name passed to the prompt as `{{ product }}`. #[builder(default = "\"swiftide\".to_string()")] product: String, } #[async_trait] impl Transformer for MetadataAudience { type Input = String; type Output = String; async fn transform_node(&self, mut node: TextNode) -> anyhow::Result { // The macro renders nothing for you: add the node and your own values here. let prompt = self .prompt_template .clone() .with_node(&node) .with_context_value("product", self.product.as_str()); // `prompt` uses this transformer's client, or the pipeline default. let audience = self.prompt(prompt).await?; node.metadata.insert(NAME, audience); Ok(node) } // The macro stores `with_concurrency(n)` but does not apply it. fn concurrency(&self) -> Option { self.concurrency } } } ``` Source: [`snippets/src/indexing/transformers.rs#L103-L143`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/transformers.rs#L103-L143) `metadata_field_name` becomes the `NAME` constant. `default_prompt_file` goes to `include_str!`, relative to the source file. The macro adds `client`, `prompt_template` and `concurrency` fields, and generates `builder()`, `new`, `from_client`, `with_concurrency`, `Default` and the `WithIndexingDefaults` impl. `derive(skip_debug, skip_clone, skip_default)` turns off generated derives. You still implement `Transformer`. Build the prompt with `with_node(&node)` before you pass it to the generated `self.prompt(...)`, which uses the transformer’s client or the pipeline default. Override `concurrency()` as the example does, or `with_concurrency` has no effect. The macro adds a `hidden` module, `NAME` and `default_prompt()` next to the struct, so give each annotated struct its own module. `new` and `Default` unwrap the builder, so give each of your own fields `#[builder(default = ...)]`. Your crate needs `swiftide-core`, `anyhow` and `derive_builder` as direct dependencies; [Crates](/reference/crates/) has the `cargo add` line. ## Batch transformers [Section titled “Batch transformers”](#batch-transformers) A batch transformer takes a `Vec` of nodes and returns a stream of any length. This one drops repeated chunks: ```rust /// Drops chunks that repeat an earlier chunk in the same batch, such as copied licence headers. #[derive(Clone)] pub struct DropRepeatedChunks { batch_size: usize, } #[async_trait] impl BatchableTransformer for DropRepeatedChunks { type Input = String; type Output = String; async fn batch_transform(&self, nodes: Vec) -> IndexingStream { let mut seen = HashSet::new(); let unique: Vec = nodes .into_iter() .filter(|node| seen.insert(node.chunk.trim().to_owned())) .collect(); unique.into() } fn batch_size(&self) -> Option { Some(self.batch_size) } } impl WithBatchIndexingDefaults for DropRepeatedChunks {} ``` Source: [`snippets/src/indexing/transformers.rs#L61-L86`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/transformers.rs#L61-L86) You add it with `then_in_batch`: ```rust Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..1500)) .then_in_batch(DropRepeatedChunks { batch_size: 64 }) .then(WordCount) .then_store_with(MemoryStorage::default()) .run() .await ``` Source: [`snippets/src/indexing/transformers.rs#L92-L98`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/indexing/transformers.rs#L92-L98) The pipeline groups nodes into batches of `batch_size()`, or 256 when it returns `None`. `into()` turns a `Vec` into an `IndexingStream`. `then_in_batch` requires `WithBatchIndexingDefaults`, again with an empty `impl`. A closure from `Vec` to `IndexingStream` is a batch transformer too, with the pipeline batch size. `Embed` and `SparseEmbed` are batch transformers. Batches are formed in arrival order, so the example only catches repeats that land in the same batch. ## Node types [Section titled “Node types”](#node-types) `Transformer` and `BatchableTransformer` declare `Input` and `Output` types, both bound by `Chunk`. `then` returns a pipeline of the transformer’s `Output`, so a different `Output` changes the node type for every step after it. Closures and most built-in transformers, chunkers and storage backends take `String` nodes, so add a transformer that changes the node type after every step that needs `String`. ## Reference [Section titled “Reference”](#reference) * [`Pipeline::then`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then) * [`Pipeline::then_in_batch`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_in_batch) * [`Pipeline::with_default_llm_client`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.with_default_llm_client) * [`Pipeline::with_concurrency`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.with_concurrency) * [`Transformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Transformer.html) * [`BatchableTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.BatchableTransformer.html) * [`WithIndexingDefaults`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.WithIndexingDefaults.html) * [`WithBatchIndexingDefaults`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.WithBatchIndexingDefaults.html) * [`Node`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html) and [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`Chunk`](https://docs.rs/swiftide/0.34/swiftide/indexing/trait.Chunk.html) * [`Metadata`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Metadata.html) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`MetadataSummary`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataSummary.html) * [`MetadataTitle`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataTitle.html) * [`MetadataKeywords`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataKeywords.html) * [`CompressCodeOutline`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.CompressCodeOutline.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) and [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`indexing_transformer`](https://docs.rs/swiftide/0.34/swiftide/attr.indexing_transformer.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) ## Next [Section titled “Next”](#next) [Embedding](/indexing/embedding/) # Anthropic > Build an Anthropic client for Claude prompts, chat completions, streaming and agents with tools; it has no embeddings or structured output. You end up with an `Anthropic` client that sends prompts and chat completions to Claude, streams responses and runs agents with tools. Enable the `anthropic` feature flag for the client and `swiftide-agents` for the agent; the `all` bundle leaves both out: ```sh cargo add swiftide@0.34 --features anthropic,swiftide-agents cargo add tokio --features full cargo add anyhow futures-util ``` The snippets use `tokio` and `anyhow` around the calls. The streaming snippet needs `futures_util::StreamExt` for `stream.next()`, which Swiftide does not re-export. ## Client [Section titled “Client”](#client) ```rust let anthropic = Anthropic::builder() .default_prompt_model("claude-sonnet-4-6") .on_usage(|usage| { println!( "{} prompt + {} completion tokens", usage.prompt_tokens, usage.completion_tokens ); Ok(()) }) .build()?; let answer = anthropic .prompt("Explain the borrow checker in one sentence.".into()) .await?; ``` Source: [`snippets/src/integrations/anthropic.rs#L14-L27`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/anthropic.rs#L14-L27) `Anthropic::builder()` returns an `AnthropicBuilder`. `default_prompt_model` sets the Claude model for every call. `build()` creates the underlying `async_anthropic::Client`, which reads your key from the `ANTHROPIC_API_KEY` environment variable. A missing key only logs a warning. The first request then fails as unauthorized. The client implements `SimplePrompt` and `ChatCompletion`. You can pass it to indexing transformers that prompt a large language model (LLM), to query pipelines and to agents. `on_usage` receives the token counts of each call. On `prompt` and `complete` the client awaits it, and an `Err` from the callback fails the call. Use `on_usage_async` for a callback that awaits. [Usage callbacks](/models/errors-backoff-and-usage/#usage-callbacks) compares this with the other clients. Swiftide builds the HTTP (Hypertext Transfer Protocol) client with rustls, even when another crate turns on native TLS (Transport Layer Security) for reqwest. It uses the rustls crypto provider your application installed, or else ring, so it needs no setup; see [TLS](/integrations/overview/#tls). ## Models [Section titled “Models”](#models) Always set `default_prompt_model`. Without it the client asks for `claude-3-5-sonnet-20241022`, which Anthropic has retired, so every call fails. `Options` holds only `prompt_model`. The client sends no temperature, stop sequences or thinking settings. Every request asks for at most 2048 output tokens, the default of `async-anthropic` 0.6, and Swiftide has no setting to raise it. A longer answer is cut off, and `ChatCompletionResponse` has no field that tells you it was truncated. `async-anthropic` 0.6 parses only text, tool use and tool result content blocks. A response that holds a thinking block fails to parse. If your model returns thinking blocks without being asked, every call fails. The snippet uses `claude-sonnet-4-6`, which does not. ## Agents with tools [Section titled “Agents with tools”](#agents-with-tools) Define a tool with the `#[swiftide::tool]` macro: ```rust #[swiftide::tool( description = "Count the words in a piece of text", param(name = "text", description = "The text to count") )] async fn count_words(_context: &dyn AgentContext, text: &str) -> Result { Ok(ToolOutput::text( text.split_whitespace().count().to_string(), )) } ``` Source: [`snippets/src/integrations/anthropic.rs#L34-L42`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/anthropic.rs#L34-L42) Then hand the client to the agent: ```rust let mut agent = Agent::builder() .llm(&anthropic) .tools(vec![count_words()]) .limit(5) .build()?; agent .query("How many words are in 'the quick brown fox'? Use the tool.") .await?; ``` Source: [`snippets/src/integrations/anthropic.rs#L48-L56`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/anthropic.rs#L48-L56) `AgentBuilder::llm` clones the client into the agent. Each request carries the tool specs as Anthropic tools with the tool choice set to auto, so the model decides when to call one. The default `stop` tool comes along with your own tools. The client turns the message history into Anthropic messages: * The first system message becomes the request’s `system` field. Later system messages and summaries go out as user messages. * Tool outputs that follow each other go out together in one user message. Swiftide groups them since 0.33. * A tool output without content goes out as `Success`. * `ChatMessage::Reasoning` entries are left out. * A message with content parts may hold text parts only. Any other part, such as an image, returns an error. See [Images and multimodal input](/models/images/). ## Streaming [Section titled “Streaming”](#streaming) ```rust let request = ChatCompletionRequest::builder() .messages(vec![ ChatMessage::new_system("You answer in plain words."), ChatMessage::new_user("Explain lifetimes in Rust in three sentences."), ]) .build()?; let mut stream = anthropic.complete_stream(&request).await; let mut printed = 0; let mut last = None; while let Some(response) = stream.next().await { let response = response?; let text = response.message().unwrap_or_default(); print!("{}", &text[printed..]); printed = text.len(); last = Some(response); } if let Some(usage) = last.and_then(|response| response.usage) { println!("\n{} tokens", usage.total_tokens); } ``` Source: [`snippets/src/integrations/anthropic.rs#L64-L85`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/anthropic.rs#L64-L85) `complete_stream` returns a `ChatCompletionStream`. Every item is the whole response so far, and the last item is the complete response with `usage` set. Treat streamed `usage` as an estimate. The client adds up the counts from the message start and message delta events. Anthropic reports cumulative counts in the message delta event, so the totals can come out higher than the non-streaming counts for the same request. The client has no `stream_full` setting, so you always receive the accumulated response. `delta.message_chunk` holds the latest text, but it keeps that value on events that carry no text. Print the part of `message` you have not printed yet, as the snippet does, rather than printing every delta. On streams the client spawns the usage callback and only logs an `Err` from it. An agent streams when you add an `on_stream` hook. ## Limitations [Section titled “Limitations”](#limitations) * No `EmbeddingModel`. Embed with [FastEmbed](/integrations/fastembed/) or [OpenAI](/integrations/openai/#embeddings) and keep Anthropic for prompts and agents. * No `StructuredPrompt`. See [Structured output](/models/structured-output/) for clients that have it. * No `SparseEmbeddingModel`. To reach Claude through AWS (Amazon Web Services) with structured output and reasoning, use [AWS Bedrock](/integrations/aws-bedrock/). ## Errors and retries [Section titled “Errors and retries”](#errors-and-retries) `async-anthropic` retries a request that gets HTTP 429 (rate limit) or 529 (overloaded) by itself. It waits 15 seconds first, doubles the wait and gives up after 120 seconds. Streaming requests get no such retry. After that Swiftide marks the error permanent, so `LanguageModelWithBackOff` does not retry it again. On `prompt`, a network error or a response body that fails to parse comes back as a `TransientError`. `async-anthropic` reports both as a network error. A response with a thinking block therefore gets retried by `LanguageModelWithBackOff`, and it fails the same way each time. On `complete` and `complete_stream` every error is permanent. See [Backoff](/models/errors-backoff-and-usage/#backoff). ## Custom client [Section titled “Custom client”](#custom-client) `AnthropicBuilder::client` takes your own `async_anthropic::Client`. Swiftide does not re-export the crate, so add it at the version Swiftide uses: ```sh cargo add async-anthropic@0.6 --no-default-features cargo add backoff@0.4 ``` Start from `async_anthropic::Client::from_api_key` to pass a key from your own configuration. Call `async_anthropic::Client::with_backoff` on it to change the retry timing. It takes a `backoff::ExponentialBackoff`, and `async-anthropic` does not re-export the `backoff` crate, hence the second line. `async_anthropic::Client::builder()` leaves the base URL and API version empty, so prefer those two methods. The client does not read a base URL from the environment. ## Reference [Section titled “Reference”](#reference) * [`Anthropic`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Anthropic.html) * [`AnthropicBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html) * [`AnthropicBuilder::client`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html#method.client) * [`AnthropicBuilder::default_prompt_model`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html#method.default_prompt_model) * [`AnthropicBuilder::on_usage`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html#method.on_usage) * [`AnthropicBuilder::on_usage_async`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html#method.on_usage_async) * [`Options` (Anthropic)](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Options.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`ChatCompletionStream`](https://docs.rs/swiftide/0.34/swiftide/traits/type.ChatCompletionStream.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletionResponseDelta`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponseDelta.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`Usage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.Usage.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder::llm`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.llm) * [`tool` macro](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) ## Next [Section titled “Next”](#next) [AWS Bedrock](/integrations/aws-bedrock/) # AWS Bedrock > Call Claude and other AWS Bedrock models through the Converse API with AwsBedrock, for prompts in indexing pipelines, agents with tools and structured output. This page leaves you with an `AwsBedrock` client that sends requests to Amazon Web Services (AWS) Bedrock through the Converse application programming interface (API), and uses it in an indexing pipeline and in an agent with tools. ```sh cargo add swiftide@0.34 --features aws-bedrock cargo add tokio --features macros,rt-multi-thread ``` examples/aws_bedrock.rs ```rust let aws_bedrock = integrations::aws_bedrock_v2::AwsBedrock::builder() .default_prompt_model("global.anthropic.claude-haiku-4-5-20251001-v1:0") .build()?; let memory_storage = MemoryStorage::default(); ``` Source: [`examples/aws_bedrock.rs#L26-L30`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock.rs#L26-L30) ## Client [Section titled “Client”](#client) `AwsBedrock` lives in `swiftide::integrations::aws_bedrock_v2` behind the `aws-bedrock` feature flag. Since 0.33 it uses Bedrock’s Converse and ConverseStream APIs, so one client covers every model family that Converse supports. [Upgrading](/reference/upgrading/) covers the move from the old module. `default_prompt_model` takes a model ID or an Amazon Resource Name (ARN). The `global.` prefix in the example is a cross-region inference profile ID, which Bedrock accepts in place of a model ID. Without a model, the first call fails with a permanent `Model not set` error. Building the client does not check it. The client implements `ChatCompletion` (`complete` and `complete_stream`), `SimplePrompt` and `DynStructuredPrompt`. It does not implement `EmbeddingModel`, so pair it with [FastEmbed](/integrations/fastembed/) or another embedder for vectors. ## Credentials and region [Section titled “Credentials and region”](#credentials-and-region) Without `.client(..)`, the builder loads the AWS configuration with `aws_config::load_from_env`. That is the AWS SDK (software development kit) credential chain: environment variables, shared profiles, single sign-on (SSO) and IAM (Identity and Access Management) roles. Credentials from `aws login` (AWS Console sign-in) are not supported; use a profile, SSO or environment variables, or pass your own client. The region comes from the same chain, for example `AWS_REGION` or your profile. Your account also needs access to the model you name. The builder runs that loading step inside `tokio::task::block_in_place` when you call `build`. So `build` needs a multi-threaded Tokio runtime. `#[tokio::main]` gives you one. `#[tokio::test]` defaults to a current-thread runtime, where `build` panics; use `#[tokio::test(flavor = "multi_thread")]` there. To control the configuration yourself, build an `aws_sdk_bedrockruntime::Client` and pass it to `AwsBedrockBuilder::client`. Swiftide does not re-export the SDK, so add `aws-config` and `aws-sdk-bedrockruntime` to your own crate at the major versions Swiftide uses (1.x). A client you pass in also skips the blocking load, so `build` also works on a current-thread runtime. ## Pipeline summaries [Section titled “Pipeline summaries”](#pipeline-summaries) examples/aws_bedrock.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("./README.md")) .log_nodes() .then_chunk(transformers::ChunkMarkdown::from_chunk_range(100..512)) .then(transformers::MetadataSummary::new(aws_bedrock.clone())) .then_store_with(memory_storage.clone()) .log_all() .run() .await?; ``` Source: [`examples/aws_bedrock.rs#L34-L41`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock.rs#L34-L41) `MetadataSummary` takes any `SimplePrompt`, so `AwsBedrock` works where other examples use `OpenAI`. Each chunk costs one Converse call, so wrap the client in `LanguageModelWithBackOff` for large inputs. `from_chunk_range(100..512)` drops chunks shorter than 100 characters, so short README sections never reach the model or the storage. The upper bound of the range is exclusive. ## Agents with Claude [Section titled “Agents with Claude”](#agents-with-claude) An agent needs the `swiftide-agents` feature flag, and tool arguments need `schemars` and `serde` in your crate: ```sh cargo add swiftide@0.34 --features aws-bedrock,swiftide-agents cargo add schemars@1 serde anyhow tokio --features serde/derive,tokio/macros,tokio/rt-multi-thread ``` Define the tools: examples/aws_bedrock_agent.rs ```rust #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] struct FormatTimestampRequest { /// Prefix to prepend to the timestamp. prefix: String, /// Timestamp to format. timestamp: String, } #[swiftide::tool(description = "Get the current UTC date and time in RFC3339 format")] async fn current_utc_time(context: &dyn AgentContext) -> Result { let command_output = context .executor() .exec_cmd(&Command::shell("date -u +\"%Y-%m-%dT%H:%M:%SZ\"")) .await?; Ok(command_output.into()) } #[swiftide::tool( description = "Format a timestamp with a caller-provided prefix", param(name = "request", description = "Timestamp formatting input") )] async fn format_timestamp( _context: &dyn AgentContext, request: FormatTimestampRequest, ) -> Result { Ok(ToolOutput::text(format!( "{}{}", request.prefix, request.timestamp ))) } ``` Source: [`examples/aws_bedrock_agent.rs#L23-L53`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L23-L53) The `use` lines sit above the excerpt in the linked example source. They bring in `anyhow::Result`, `schemars::JsonSchema`, serde’s `Serialize` and `Deserialize`, and from Swiftide `agents`, `ToolOutput`, `ToolError`, `AgentContext`, `Command` and `aws_bedrock_v2::AwsBedrock`. Build the client and the agent, then run it: examples/aws_bedrock_agent.rs ```rust let bedrock = AwsBedrock::builder() .default_prompt_model("global.anthropic.claude-sonnet-4-6") .build()?; ``` Source: [`examples/aws_bedrock_agent.rs#L61-L63`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L61-L63) examples/aws_bedrock_agent.rs ```rust let mut agent = agents::Agent::builder() .llm(&bedrock) .tools(vec![current_utc_time(), format_timestamp()]) .on_new_message(|_, msg| { let rendered = msg.to_string(); Box::pin(async move { println!("{rendered}"); Ok(()) }) }) .limit(6) .build()?; ``` Source: [`examples/aws_bedrock_agent.rs#L67-L78`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L67-L78) examples/aws_bedrock_agent.rs ```rust agent .query( "Call current_utc_time once. Then call format_timestamp with prefix \"UTC now: \" and \ that timestamp. After that, report the formatted result and stop.", ) .await?; ``` Source: [`examples/aws_bedrock_agent.rs#L82-L87`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L82-L87) `current_utc_time` runs `date` through the tool executor of the agent’s context, which is local by default. `format_timestamp` takes a struct argument, and its doc comments become field descriptions in the tool’s JavaScript Object Notation (JSON) Schema. The loop counter behind `limit(6)` resets when the agent stops, so a second `query` gets six loops again. On the way to Bedrock, Swiftide turns the request into Converse messages: * Every tool is sent with automatic tool choice and strict schema validation. Set `tool_strict` to `false` to turn validation off. * Every system message becomes a Converse system block. * Adjacent tool outputs go back together in one user message. * Each `ChatMessage::Reasoning` in the message history becomes its own assistant message, sent before the assistant text or tool use it belongs to. Reasoning without both text and a signature, such as redacted reasoning, is dropped. On the way back, the client returns Claude’s reasoning in the `reasoning` field of `ChatCompletionResponse`. The agent then adds it to the message history as `ChatMessage::Reasoning`. ## Options [Section titled “Options”](#options) Set defaults with `AwsBedrockBuilder::default_options`, or change them later with `options_mut`. [AWS Bedrock settings](/models/overview/#aws-bedrock-settings) lists the fields, the merge rules and the `reasoning_effort` beta header. A few details only matter here: | Field | Effect | | ------------------------------------------------------ | -------------------------------------------------------------- | | `prompt_model` | model ID or ARN; `default_prompt_model` sets it | | `max_tokens`, `temperature`, `top_p`, `stop_sequences` | unset fields are not sent | | `additional_model_response_field_paths` | JSON Pointer paths of model-specific response fields to return | `additional_model_request_fields` takes an `aws_smithy_types::Document`. Neither Swiftide nor the SDK re-exports that type, so add `aws-smithy-types` (1.x) to your crate to set it. `ReasoningEffort::Max` is meant for Claude Opus 4.6 with adaptive thinking. Set `thinking.type` to `adaptive` through `additional_model_request_fields` to use it. ### Reasoning and beta headers [Section titled “Reasoning and beta headers”](#reasoning-and-beta-headers) The swiftide examples do not set these fields, so this section names them without code. Build the value with `Options::builder()` and pass the result of `build()` to `default_options`. The setters take the value itself, not an `Option`: * `reasoning_effort` takes `swiftide::integrations::aws_bedrock_v2::ReasoningEffort` (`Low`, `Medium`, `High` or `Max`). Swiftide writes it to `output_config.effort` in the model request fields. * `tool_strict` takes a `bool`. `false` sends tools without strict schema validation. * `anthropic_beta` takes a list of beta header names. Swiftide adds them to the `anthropic_beta` array in the model request fields, without duplicates. * `additional_model_request_fields` takes the `Document` that the two fields above are merged into. See [`OptionsBuilder` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.OptionsBuilder.html) for every setter. ## Structured prompts [Section titled “Structured prompts”](#structured-prompts) `AwsBedrock` implements `DynStructuredPrompt`, so `structured_prompt::` from `StructuredPrompt` works through the blanket implementation. Swiftide sends the schema as a Converse output configuration with a JSON Schema text format, then parses the reply into JSON. Whether a model accepts that format is up to Bedrock. [Structured output](/models/structured-output/) shows the call. ## Errors and usage [Section titled “Errors and usage”](#errors-and-usage) Throttling, unavailable services, models that are not ready, model timeouts, internal server errors, network failures, timeouts and unreadable responses are transient. For `complete_stream`, a model stream error is transient too. An error in the middle of a stream is transient only for throttling, unavailable services, internal server errors and model stream errors. Everything else is permanent. A response that stops because the context window is full, with no text, tool calls or reasoning, returns `ContextLengthExceeded`. Wrap the client in `LanguageModelWithBackOff` to retry transient errors; see [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). The wrapper implements `SimplePrompt` and `ChatCompletion` but not `DynStructuredPrompt`, so a wrapped client loses `structured_prompt`. `on_usage` and `on_usage_async` receive the token `Usage` of each call. Its `cached_tokens` holds the cache-read input tokens. When there are none, it holds the cache-write tokens instead, so a first call that writes the prompt cache also reports cached input. Both callbacks run before the call returns; an `Err` from either fails the call. ## Reference [Section titled “Reference”](#reference) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`AwsBedrockBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrockBuilder.html) * [`Options` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.Options.html) * [`ReasoningEffort` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/enum.ReasoningEffort.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`DynStructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.DynStructuredPrompt.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`MetadataSummary`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataSummary.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`tool`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) ## Next [Section titled “Next”](#next) [FastEmbed](/integrations/fastembed/) # FastEmbed > Embed chunks and queries with dense and sparse FastEmbed models, and rerank retrieved documents, on your own CPU without an API key. You embed chunks and queries with local dense and sparse models, pick a model that fits your data, and rerank retrieved documents, all on your CPU without an API key. Turn on the `fastembed` feature flag: ```sh cargo add swiftide@0.34 --features fastembed ``` ## Default models [Section titled “Default models”](#default-models) The hybrid search example builds one dense and one sparse embedder with the default models: examples/hybrid_search.rs ```rust // Ensure all batching is consistent let batch_size = 64; let fastembed_sparse = FastEmbed::try_default_sparse().unwrap().to_owned(); let fastembed = FastEmbed::try_default().unwrap().to_owned(); // Set up openai with the mini model, which is great for indexing let openai = openai::OpenAI::builder() .default_prompt_model("gpt-4o-mini") .build() .unwrap(); // Set up qdrant and use the combined fields (metadata + chunks) for both sparse and dense // vectors let qdrant = Qdrant::builder() .batch_size(batch_size) .vector_size(384) .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name("swiftide-hybrid-example") .build()?; ``` Source: [`examples/hybrid_search.rs#L23-L43`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs#L23-L43) The excerpt also builds an OpenAI client and Qdrant storage. To compile it as is, add the `openai` and `qdrant` feature flags next to `fastembed`. `FastEmbed::try_default()` loads BGE (BAAI General Embedding) small en v1.5, a dense model with 384 dimensions. `FastEmbed::try_default_sparse()` loads SPLADE++ (Sparse Lexical and Expansion model, `prithivida/Splade_PP_en_v1`), a sparse model. Both return a `Result`, because loading the model can fail. `FastEmbed` implements `EmbeddingModel` and `SparseEmbeddingModel`. You pass the dense instance to the `Embed` batch transformer in the indexing pipeline and to `query_transformers::Embed` in the query pipeline. The sparse instance goes to `SparseEmbed` on both sides. Each instance holds one model. Passing a sparse instance to `Embed` fails with the permanent error “Expected dense model, got sparse”, and a dense instance in `SparseEmbed` fails with “Expected sparse model, got dense”. Your storage’s vector size must match the model: 384 for BGE small. ## Model choice [Section titled “Model choice”](#model-choice) The ask-your-codebase showcase indexes source code, so it picks a model trained on code: examples/ask-your-codebase/src/embedding.rs ```rust /// A code-aware embedding model that runs on the CPU and needs no API key. /// /// The model (~600MB) is downloaded on first use. pub fn local_model() -> Result { let model = TextEmbedding::try_new( InitOptions::new(EmbeddingModel::JinaEmbeddingsV2BaseCode).with_max_length(MAX_TOKENS), )?; FastEmbed::builder() .embedding_model(model) .batch_size(BATCH_SIZE) .build() } ``` Source: [`examples/ask-your-codebase/src/embedding.rs#L18-L30`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/embedding.rs#L18-L30) `jina-embeddings-v2-base-code` produces 768-dimensional vectors. `fastembed::TextInitOptions::with_max_length` raises the token limit to 2048 for each embedded text. In ask-your-codebase that text is a chunk of up to 4096 bytes plus its generated questions and answers. FastEmbed truncates longer texts without an error. The default limit is 512 tokens, or less if the model’s own limit is lower. Swiftide does not re-export the `fastembed` crate. To name a model, add it yourself at the major version Swiftide uses: ```sh cargo add fastembed@5 --no-default-features ``` `FastEmbedBuilder::embedding_model` accepts a `fastembed::TextEmbedding` for dense vectors or a `fastembed::SparseTextEmbedding` for sparse ones. The fastembed documentation lists the dense models, with their dimensions, under `fastembed::EmbeddingModel`. The excerpt uses `fastembed::InitOptions`, which fastembed 5 keeps as a deprecated alias of `fastembed::TextInitOptions`. Use `fastembed::TextInitOptions` in new code to avoid the deprecation warning. Its options also set the cache directory (`fastembed::TextInitOptions::with_cache_dir`) and cap the ONNX (Open Neural Network Exchange) Runtime threads (`fastembed::TextInitOptions::with_intra_threads`), which default to every core. ## Batch size [Section titled “Batch size”](#batch-size) How the pipeline batch size and the storage batch size interact is covered in [Embedding](/indexing/embedding/#batch-sizes). The FastEmbed side has one cost to keep in mind: FastEmbed pads every text in a batch to the longest one, so long texts in a large batch cost memory. The docs-qa showcase sets 32 on both embedding steps and leaves FastEmbed at its default of 256. FastEmbed never sees more texts than the pipeline hands over, so its own batch size only matters when it is smaller: examples/docs-qa/src/index.rs ```rust indexing::Pipeline::from_loader(loader) .then_chunk(ChunkMarkdown::from_chunk_range( MIN_CHUNK_CHARS..MAX_CHUNK_CHARS, )) // A short title and keywords per chunk; both end up in the embedded text and help the // sparse (keyword) side of hybrid search in particular. .then( MetadataTitle::from_client(llm.clone()) .prompt_template(TITLE_PROMPT) .build()?, ) .then( MetadataKeywords::from_client(llm.clone()) .prompt_template(KEYWORDS_PROMPT) .build()?, ) .then_in_batch(Embed::new(embedders.dense.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_in_batch(SparseEmbed::new(embedders.sparse.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_store_with(qdrant.clone()) .run() .await ``` Source: [`examples/docs-qa/src/index.rs#L50-L70`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/index.rs#L50-L70) ask-your-codebase sets 32 on both the pipeline and the `FastEmbed` builder, because its embedded texts run up to 2048 tokens. To limit memory, lower the pipeline batch size first. ## Reranking [Section titled “Reranking”](#reranking) `Rerank` is a response transformer that scores retrieved documents with a local cross-encoder and keeps the best ones. It implements `TransformResponse`. docs-qa builds it next to its answer generator: examples/docs-qa/src/query.rs ```rust let reranker = Rerank::builder() .top_k(CONTEXT_CHUNKS) .document_template(Some(RERANK_TEMPLATE.to_string())) .build()?; let answerer = answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(DOCUMENT_TEMPLATE) .build()?; ``` Source: [`examples/docs-qa/src/query.rs#L73-L82`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L73-L82) The default model is BGE reranker base. To use another one, pass a `fastembed::TextRerank` wrapped in `Arc>` to `Rerank::builder().model(...)`. `model_batch_size` sets the reranker’s batch size, which otherwise uses FastEmbed’s default of 256. For where `Rerank` sits in the query pipeline and how to write a rerank template, see [Reranking and response transformers](/querying/reranking-and-response-transformers/). ## First run and model cache [Section titled “First run and model cache”](#first-run-and-model-cache) The first time a model loads, FastEmbed downloads it from Hugging Face into `./.fastembed_cache` in the working directory. Later runs load it from there. Set `FASTEMBED_CACHE_DIR` to share one cache between projects. If `HF_HOME` is set, it takes precedence over both, and the models land there. The three docs-qa models (dense, sparse and reranker) take about 2.3 GB. The ask-your-codebase model takes about 600 MB. The ONNX Runtime library is downloaded once at build time, because the `fastembed` feature flag turns on fastembed’s `ort-download-binaries` feature. The first build therefore needs network access, and so does the first run. ## Gotchas [Section titled “Gotchas”](#gotchas) * A `FastEmbed` keeps its model behind an `Arc>`. Clones share the same model, so calls on one instance and its clones run one at a time. A higher `with_concurrency` on `Embed` does not speed up embedding. * The model runs inside the async call, not on a blocking thread. A large batch occupies one Tokio worker thread until it finishes. * Indexing and querying must use the same model. See [Embedding](/indexing/embedding/) for what to do when you change it. * `Rerank::default()`, and `Rerank::builder().build()` without a model, panic when the default reranker cannot load, for example on a first run without network access. `FastEmbed::try_default()` returns an error instead. ## Reference [Section titled “Reference”](#reference) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`FastEmbedBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbedBuilder.html) * [`Rerank`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.Rerank.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html) * [`transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`transformers::SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`query_transformers::SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.SparseEmbed.html) * [fastembed crate documentation](https://docs.rs/fastembed/5) * [Embedding](/indexing/embedding/) * [Reranking and response transformers](/querying/reranking-and-response-transformers/) * [Feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [Qdrant](/integrations/qdrant/) # LanceDB > Store nodes in an embedded LanceDB table in a local directory and retrieve them with a similarity search, a filter or your own LanceDB query. You end up with a vector store in a directory next to your code, with no database server to run. Swiftide persists nodes into a LanceDB table and retrieves them in a query pipeline. The integration sits behind the `lancedb` feature flag. The code on this page also uses `redb` for the node cache, `fastembed` for the embedder and `tree-sitter` for `MetadataQACode`: ```sh cargo add swiftide@0.34 --features lancedb,redb,fastembed,tree-sitter ``` ## Builder [Section titled “Builder”](#builder) examples/ask-your-codebase/src/codebase.rs ```rust /// Remembers which files were fully indexed, so unchanged files are skipped next time. pub fn node_cache(&self) -> Result { std::fs::create_dir_all(&self.data_dir)?; Redb::builder() .database_path(self.data_dir.join("cache.redb")) .table_name("indexed_files") .build() } /// Embedded vector store with one vector per chunk and the source path next to it. pub fn vector_store(&self) -> Result { let uri = self.lancedb_dir(); LanceDB::builder() .uri(uri.to_str().context("Index path is not valid UTF-8")?) .table_name("code_chunks") .vector_size(embedding::DIMENSIONS) .with_vector(EmbeddedField::Combined) .with_metadata(PATH_FIELD) .build() } ``` Source: [`examples/ask-your-codebase/src/codebase.rs#L56-L77`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/codebase.rs#L56-L77) The ask-your-codebase showcase keeps a `Redb` node cache and a `LanceDB` storage in the same directory. `uri` points at the LanceDB directory, as a Uniform Resource Identifier (URI). You need it unless you pass your own `connection_pool`. Without either, `build` returns the error `URI should be set`. `table_name` defaults to `swiftide`. `vector_size` is the size of every vector column and must match the embedder, here 768 for a FastEmbed code model. `with_vector` adds a vector column for an embedded field. `with_metadata` adds a column for a metadata field. The storage opens a pool of up to 10 connections, which `pool_size` changes. The table has these columns: | Column | Type | Content | | ----------------------- | -------------- | ----------------------------------------------------- | | `id` | 16 bytes | The node id, derived from the file path and the chunk | | `chunk` | text | The chunk | | one per `with_metadata` | text, nullable | The metadata value | | `vector_` | list of `f32` | The vector, for example `vector_combined` | Column names are lowercased, and every character that is not a letter or a digit becomes `_`. The field `MetadataQACode` writes, `Questions and Answers (code)`, becomes the column `questions_and_answers__code_`. ## Indexing pipeline [Section titled “Indexing pipeline”](#indexing-pipeline) examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` Source: [`examples/ask-your-codebase/src/index.rs#L89-L115`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L89-L115) `then_store_with` hands the nodes to `LanceDB`, which implements the `Persist` trait. When the pipeline runs, it calls `setup` once. `setup` creates an empty table with the configured columns if the table does not exist, and opens the existing table otherwise. The storage writes 256 nodes per batch, which `batch_size` changes on the builder. Each batch is a merge insert on `id`. Ids come from the path and the chunk, so changed chunks add rows. See [Storage](/indexing/storage/) for how to clean them up. ## Metadata columns [Section titled “Metadata columns”](#metadata-columns) examples/ask-your-codebase/src/index.rs ```rust /// Stores the file path, relative to the codebase root, as metadata on the node. /// /// The vector store keeps this field next to each chunk, so answers can cite their sources. fn record_relative_path( codebase: Codebase, ) -> impl Fn(TextNode) -> Result + Clone + Send + Sync + 'static { move |mut node| { let path = codebase .relative_path(&node.path) .to_string_lossy() .into_owned(); node.metadata.insert(PATH_FIELD, path); Ok(node) } } ``` Source: [`examples/ask-your-codebase/src/index.rs#L127-L141`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L127-L141) The closure transformer writes a `path` field into the node metadata, and `with_metadata(PATH_FIELD)` gives it a column. Only string values are stored. A number, a list or an object in that field is written as null, as is a field the node does not have. Metadata without a column is not stored at all. In the showcase, the questions from `MetadataQACode` are embedded with the chunk but have no column, so they never reach the table. ## Similarity search [Section titled “Similarity search”](#similarity-search) examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` Source: [`examples/ask-your-codebase/src/query.rs#L36-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L36-L58) `LanceDB` implements the `Retrieve` trait for `SimilaritySingleEmbedding`. The search runs a nearest-neighbour query on the single vector column and returns `top_k` rows, 10 by default. `SimilaritySingleEmbedding::default()` is the search without a filter, and `with_top_k` chains onto it. The query needs an embedding, so an `Embed` query transformer runs before `then_retrieve`. Use the same model that embedded the chunks. A query without an embedding fails with `No embedding for query`. examples/ask-your-codebase/src/query.rs ```rust impl From> for Answer { /// Keeps the distinct file paths of the retrieved chunks, most relevant first. fn from(query: Query) -> Self { let mut sources: Vec = Vec::new(); for document in query.documents() { let Some(path) = document.metadata().get(PATH_FIELD).and_then(|v| v.as_str()) else { continue; }; if !sources.iter().any(|source| source == path) { sources.push(path.to_owned()); } } Self { text: query.answer().to_owned(), sources, } } } ``` Source: [`examples/ask-your-codebase/src/query.rs#L123-L142`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L123-L142) Each retrieved document takes its content from the `chunk` column. Every other text column becomes a metadata entry, keyed by the column name. `path` keeps its name. Vector and id columns are left out. A metadata column that holds null for a row comes back as an empty string, not as a missing key. The showcase’s `ask` run, answered by `claude-sonnet-5-5`, listed these `path` values: ```text Sources: - src/agent.rs - src/tools/control.rs - src/lib.rs - src/state.rs - src/errors.rs - src/task_agent.rs - src/hooks.rs - src/tasks.rs - src/default_context.rs ``` ## Filters and custom queries [Section titled “Filters and custom queries”](#filters-and-custom-queries) `SimilaritySingleEmbedding::from_filter` takes a `String` for LanceDB. Swiftide passes it unchanged to LanceDB’s `only_if`, as a Structured Query Language (SQL) filter on the column names. For example, `SimilaritySingleEmbedding::from_filter("path = 'src/agent.rs'".to_string())` searches only the chunks of that file. For anything else, use `CustomStrategy` with a LanceDB query. `LanceDB` accepts any query that implements LanceDB’s `ExecutableQuery` trait, and turns the rows into documents as above. With `CustomStrategy::from_async_query`, the closure returns a future that awaits `get_connection` or `open_table` and builds the query. `CustomStrategy::from_query` takes a synchronous closure that cannot await, so open the `lancedb::Table` beforehand and capture it. Swiftide does not re-export the `lancedb` crate. Add it at the version Swiftide uses to name its types: ```sh cargo add lancedb@0.33 --no-default-features ``` Swiftide’s integration test [`swiftide/tests/lancedb.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide/tests/lancedb.rs) builds a custom vector search with `from_async_query`. ## Remote storage [Section titled “Remote storage”](#remote-storage) `api_key`, `region` and `storage_options` on the builder go to every LanceDB connection. Swiftide enables the `remote` feature of the `lancedb` crate, so a `db://` URI plus `api_key` and `region` connects to LanceDB Cloud. Object stores such as Amazon Simple Storage Service (S3) need a storage feature of the `lancedb` crate, for example `aws`, that Swiftide does not enable. Add `lancedb` with that feature to your own `Cargo.toml`, and Cargo combines it with Swiftide’s. ## Gotchas [Section titled “Gotchas”](#gotchas) * A vector column without a size panics in `build` with `Vector size should be set either in the field or in the LanceDB builder`. Set `vector_size`. * `setup` never changes an existing table. After a change to the vector size, the columns or the embedding model, use a new table name or delete the directory. In the showcase, delete `.ask-your-codebase/` to start over. * `SimilaritySingleEmbedding` needs exactly one vector column. With zero or several, retrieval fails with `Zero or multiple vector fields configured in schema`. There is no `HybridSearch` support. * LanceDB creates no vector index for you. For large tables, create one through `get_connection`. * LanceDB’s dependencies compile Protocol Buffers definitions at build time, so the build needs `protoc` on your `PATH`. Install it with your package manager, for example `brew install protobuf` or `apt install protobuf-compiler`. ## Reference [Section titled “Reference”](#reference) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`LanceDBBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDBBuilder.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) * [`Pipeline::filter_cached`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.filter_cached) * [`Pipeline::then_store_with`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html#method.then_store_with) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) ## Next [Section titled “Next”](#next) [Tree-sitter](/integrations/tree-sitter/) # OpenAI and Azure > Build an OpenAI or Azure OpenAI client for prompts, chat completions, agents, embeddings and the Responses API, and point it at any OpenAI-compatible base URL. Here you build an `OpenAI` client, or the same client for an Azure OpenAI deployment, and pass it to indexing pipelines, query pipelines and agents. The integration sits behind the `openai` feature flag: ```sh cargo add swiftide@0.34 --features openai ``` ## Client [Section titled “Client”](#client) ```rust let openai = swiftide::integrations::openai::OpenAI::builder() .default_prompt_model("gpt-4o-mini") .default_embed_model("text-embedding-3-small") .build()?; ``` Source: [`examples/agent_can_fail_custom_schema.rs#L64-L67`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agent_can_fail_custom_schema.rs#L64-L67) `default_prompt_model` sets the model for `prompt`, `structured_prompt`, `complete` and `complete_stream`. `default_embed_model` sets the model for `embed`. The client has no default model. A call without one fails with a permanent error that says `Model not set`. Unless you call `api_base`, `api_key` or `client(..)`, the builder creates an `async_openai::Client` with `OpenAIConfig::default()`. That configuration reads `OPENAI_API_KEY` and `OPENAI_BASE_URL` from the environment, and `OPENAI_ORG_ID` and `OPENAI_PROJECT_ID` when they are set. The base URL defaults to `https://api.openai.com/v1`. `OpenAI` implements `SimplePrompt`, `ChatCompletion`, `EmbeddingModel` and `StructuredPrompt`. So one client serves a large language model (LLM) transformer, the `Embed` batch transformer, an answer generator and `Agent::builder().llm(&openai)`. Agents also need the `swiftide-agents` feature flag. Cloning the client is cheap: the clones share one HTTP client. ## Custom base URL [Section titled “Custom base URL”](#custom-base-url) examples/ask-your-codebase/src/llm.rs ```rust impl LlmEndpoint { /// Builds a client for this endpoint. /// /// Rate limits and transient errors are retried with exponential backoff, which matters when /// indexing fires many prompts concurrently. After `retry_budget` the error is returned, so /// an unreachable or broken endpoint fails instead of hanging. pub fn client(&self, retry_budget: Duration) -> Result> { let openai = OpenAI::builder() .api_base(&self.base_url) .api_key(&self.api_key) .default_prompt_model(&self.model) .use_responses_api(matches!(self.api, OpenAiApi::Responses)) .build()?; let backoff = BackoffConfiguration { max_elapsed_time_sec: retry_budget.as_secs(), ..BackoffConfiguration::default() }; Ok(Arc::new(LanguageModelWithBackOff::new(openai, backoff))) } } ``` Source: [`examples/ask-your-codebase/src/llm.rs#L79-L100`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/llm.rs#L79-L100) The ask-your-codebase showcase sends its prompts to any OpenAI-compatible endpoint. `LlmEndpoint` holds the base URL, key and model from the command line. `api_base` and `api_key` on the builder override the base URL and key from the environment. Everything else still comes from `OpenAIConfig::default()`, so `OPENAI_ORG_ID` and `OPENAI_PROJECT_ID` are sent as headers when they are set, also to a third-party gateway. The showcase picks the Responses API or Chat Completions from a command-line flag. The showcase wraps the client in `LanguageModelWithBackOff`. Transient errors, such as rate limits and outages, are retried with exponential backoff until `max_elapsed_time_sec` runs out. Permanent errors, including quota and billing errors that arrive as HTTP 429, return at once. `LlmEndpoint::client` returns `Arc`, so the rest of the showcase is provider-independent. [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/) covers the settings. For anything the two setters do not cover, such as a custom HTTP client or another configuration type, pass a whole `async_openai::Client` to `client(..)`. It replaces what `api_base` and `api_key` set, and whichever you call last wins. Swiftide re-exports `OpenAIConfig`, `AzureConfig` and `ReasoningEffort` from `async-openai` at `swiftide::integrations::openai`. It does not re-export `async_openai::Client`, so add the crate at the version Swiftide uses: ```sh cargo add async-openai@0.41 --no-default-features ``` ## Azure OpenAI [Section titled “Azure OpenAI”](#azure-openai) Azure OpenAI uses its own URL scheme and authentication header, so it needs `AzureConfig` instead of `OpenAIConfig`: ```rust let config = AzureConfig::new() .with_api_base("https://my-resource.openai.azure.com") .with_api_version("2024-10-21") .with_deployment_id("gpt-4o-mini") .with_api_key(std::env::var("AZURE_OPENAI_API_KEY")?); let azure: GenericOpenAI = GenericOpenAIBuilder::default() .client(async_openai::Client::with_config(config)) .default_prompt_model("gpt-4o-mini") .build()?; let answer = azure.prompt("Name one Rust web framework.".into()).await?; ``` Source: [`snippets/src/integrations/openai.rs#L17-L28`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/openai.rs#L17-L28) `OpenAI` is a type alias for `GenericOpenAI`. For Azure you build a `GenericOpenAI` through `GenericOpenAIBuilder` and pass the client with `client(..)`. `api_base` and `api_key` exist only on the `OpenAI` builder. The Azure client implements the same model traits. `AzureConfig::with_api_base` takes the resource URL without a path. Requests go to `{api_base}/openai/deployments/{deployment_id}/...`, with `api-version` as a query parameter and the key in an `api-key` header. Without `with_api_key`, `AzureConfig` falls back to `OPENAI_API_KEY`. A deployment serves one model, so build one client per deployment, for example one for chat and one for embeddings. Set `default_prompt_model` or `default_embed_model` anyway: Swiftide refuses to send a request without a model, even though Azure routes by deployment. `GenericOpenAI` is a different type from `OpenAI`. A field typed `OpenAI` does not accept it. Store the client as `Arc` or another trait object when the provider varies. See [Clients as trait objects](/models/overview/#clients-as-trait-objects). ## Embeddings [Section titled “Embeddings”](#embeddings) ```rust let openai = OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_options(Options::builder().dimensions(512)) .build()?; let vectors = openai .embed(vec!["What is a borrow checker?".into()]) .await?; println!("{} dimensions", vectors[0].len()); indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then_in_batch(Embed::new(openai.clone()).with_batch_size(64)) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/integrations/openai.rs#L37-L52`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/openai.rs#L37-L52) `embed` sends a list of texts in one request and returns one vector per text. Embedding requests carry two settings from `Options`: `dimensions` and `user`. `dimensions` works with text-embedding-3 and later models. The vector size you get must match the vector size of your storage. In an indexing pipeline, the `Embed` batch transformer calls `embed` once per batch. `with_batch_size(64)` sets the batch size. Without it, the pipeline default of 256 applies. Use the same model and dimensions for the `Embed` query transformer, so queries and chunks land in the same vector space. ## Request options [Section titled “Request options”](#request-options) [Request settings](/models/overview/#request-settings) covers `default_options` and which call sends which field. Two details are specific to this client. `parallel_tool_calls` is unset by default, although the builder method’s documentation says it defaults to `true`. Once you set it, the client sends it on every chat request, including `prompt`, `structured_prompt` and `complete` calls without tools. OpenAI rejects `parallel_tool_calls` on a request without tools. Give your agents a separate client when you set it, and keep it off the client that serves transformers and answer generators. `extra_body` merges extra fields into the top level of the request body. Only `complete` and `complete_stream` send it, on the Chat Completions API and the Responses API alike. `prompt`, `structured_prompt` and `embed` do not. ## Responses API [Section titled “Responses API”](#responses-api) ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-4.1-mini") .default_options(Options::builder().temperature(0.2)) .use_responses_api(true) .build()?; ``` Source: [`examples/responses_api.rs#L49-L53`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api.rs#L49-L53) `use_responses_api(true)` sends `complete`, `complete_stream`, `prompt` and `structured_prompt` to the OpenAI Responses API instead of the Chat Completions API. `embed` is unaffected. The flag is off by default. With `reasoning_effort` set, the client asks for reasoning summaries and encrypted reasoning content, and sends the encrypted content back on the next turn. Summaries are not sent back. `reasoning_features(false)` turns that off. The Responses API ignores `seed` and `presence_penalty` and logs a warning. [Responses API and reasoning](/models/overview/#responses-api-and-reasoning) shows an agent that keeps its reasoning in the message history. ## Streaming and usage [Section titled “Streaming and usage”](#streaming-and-usage) `stream_full(false)` makes `complete_stream` send only the delta in every item but the last. See [Deltas only](/models/overview/#deltas-only). `on_usage` and `on_usage_async` report the token `Usage` of every request, embeddings included. The `on_usage_async` callback returns a pinned, boxed future. See [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). ## Gotchas [Section titled “Gotchas”](#gotchas) * Some OpenAI-compatible gateways cap unset `max_completion_tokens` at 4096 and truncate silently; see [OpenAI-compatible providers](/integrations/openai-compatible/#gotchas). * A gateway’s Responses API has to match OpenAI’s response schema exactly. The captured docs-qa run uses Chat Completions, its default, because its gateway returns `"input_tokens_details": null`, which `async-openai` rejects. ## Reference [Section titled “Reference”](#reference) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`OpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAIBuilder.html) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) * [`OpenAIConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OpenAIConfig.html) * [`AzureConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.AzureConfig.html) * [`Options`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`OptionsBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OptionsBuilder.html) * [`ReasoningEffort`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/enum.ReasoningEffort.html) * [`GenericOpenAIBuilder::use_responses_api`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.use_responses_api) * [`GenericOpenAIBuilder::stream_full`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.stream_full) * [`GenericOpenAIBuilder::on_usage`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.on_usage) * [`GenericOpenAIBuilder::parallel_tool_calls`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.parallel_tool_calls) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`BackoffConfiguration`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.BackoffConfiguration.html) * [`Usage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.Usage.html) ## Next [Section titled “Next”](#next) [OpenAI-compatible providers](/integrations/openai-compatible/) # OpenAI-compatible providers > Build a client for Ollama, Groq, Gemini, OpenRouter, Dashscope or any other server that speaks the OpenAI API, with the feature flag, base URL and key variable each one uses. Swiftide talks to Ollama, Groq, Gemini, OpenRouter and Dashscope through the same OpenAI client, each with its own base Uniform Resource Locator (URL) and key. You end up with a working client for each, local embeddings through Ollama, and a client for any other gateway that speaks the OpenAI application programming interface (API). | Client | Feature flag | Base URL | Key variable | | ------------ | ------------- | --------------------------------------------------------- | ---------------------------------------- | | `Ollama` | `ollama` | `http://localhost:11434/v1` | none | | `Groq` | `groq` | `https://api.groq.com/openai/v1` | `GROQ_API_KEY` | | `Gemini` | `gemini` | `https://generativelanguage.googleapis.com/v1beta/openai` | `GEMINI_API_KEY` | | `OpenRouter` | `open-router` | `https://openrouter.ai/api/v1` | `OPENROUTER_API_KEY` | | `Dashscope` | `dashscope` | `https://dashscope.aliyuncs.com/compatible-mode/v1` | `QWEN_API_KEY`, then `DASHSCOPE_API_KEY` | Each of these feature flags also turns on `openai`: ```sh cargo add swiftide@0.34 --features ollama ``` ## Ollama [Section titled “Ollama”](#ollama) examples/index_ollama.rs ```rust let ollama_client = integrations::ollama::Ollama::default() .with_default_prompt_model("llama3.1") .to_owned(); let fastembed = integrations::fastembed::FastEmbed::try_default()?; let memory_store = MemoryStorage::default(); ``` Source: [`examples/index_ollama.rs#L30-L35`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_ollama.rs#L30-L35) Every client on this page is `GenericOpenAI` with a provider-specific configuration type. It shares the `Options` and the model traits of [`OpenAI`](/integrations/openai/). `Ollama::default()` builds a client without a model. `with_default_prompt_model` takes `&mut self` and returns `&mut Self`, so `.to_owned()` turns the result back into a client you own. The builder, `Ollama::builder()`, avoids that step. The example uses Ollama only for the `MetadataQAText` transformer and embeds the chunks with FastEmbed, so it also needs the `fastembed` feature flag. ### Local embeddings [Section titled “Local embeddings”](#local-embeddings) Ollama also serves embeddings. Set an embed model and use the same client on both sides: ```rust let ollama = Ollama::builder() .default_prompt_model("llama3.2") .default_embed_model("nomic-embed-text") .build()?; // nomic-embed-text returns 768 dimensions. let qdrant = Qdrant::builder() .vector_size(768) .collection_name("local-docs") .build()?; indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then_in_batch(Embed::new(ollama.clone()).with_batch_size(32)) .then_store_with(qdrant.clone()) .run() .await?; let answered = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(ollama.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(ollama)) .query("How do I configure the loader?") .await?; ``` Source: [`snippets/src/integrations/openai_compatible.rs#L22-L45`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/openai_compatible.rs#L22-L45) The `Embed` batch transformer embeds the chunks and the `Embed` query transformer embeds the question. Both must use the same model, or the vectors do not match. The storage vector size must match the model’s output too. nomic-embed-text returns 768 dimensions. The snippet stores the vectors in Qdrant, so it also needs the `qdrant` feature flag and a running Qdrant server. [Qdrant](/integrations/qdrant/) covers that part. Pull the models before you run it: ```sh ollama pull nomic-embed-text ollama pull llama3.2 ``` The Ollama example above uses `llama3.1`. Pull that one to run it. ### Remote Ollama [Section titled “Remote Ollama”](#remote-ollama) `OllamaConfig` is public, so you can point the client at another machine: ```rust let mut config = OllamaConfig::default(); config.with_api_base("http://gpu-box:11434/v1"); let ollama = Ollama::builder() .client(async_openai::Client::with_config(config)) .default_prompt_model("llama3.2") .build()?; let answer = ollama.prompt("Name one Rust web framework.".into()).await?; ``` Source: [`snippets/src/integrations/openai_compatible.rs#L54-L62`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/openai_compatible.rs#L54-L62) `OllamaConfig::with_api_base` takes `&mut self`, so build the config in a mutable variable first. `async_openai::Client` is not re-exported. Add the crate at the version Swiftide uses: ```sh cargo add async-openai@0.41 --no-default-features ``` `OllamaConfig` sends no `Authorization` header and reads no environment variable. For an Ollama server behind a proxy that checks a bearer token, use `OpenAI` with `api_base` and `api_key` instead. See [Other gateways](#other-gateways). ## Groq [Section titled “Groq”](#groq) examples/index_groq.rs ```rust let groq_client = integrations::groq::Groq::builder() .default_prompt_model("llama3-8b-8192") .to_owned() .build()?; let fastembed = integrations::fastembed::FastEmbed::try_default()?; let memory_store = MemoryStorage::default(); ``` Source: [`examples/index_groq.rs#L29-L35`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_groq.rs#L29-L35) `Groq::builder()` gives a builder with the Groq base URL and the key from `GROQ_API_KEY`. The `.to_owned()` in the middle clones the builder and changes nothing. Like the Ollama example, this one embeds with FastEmbed and needs the `fastembed` feature flag. Groq has retired `llama3-8b-8192`, the model in the example. Pick a current model from Groq’s model list. ## Gemini [Section titled “Gemini”](#gemini) examples/hello_agents.rs ```rust let openai = swiftide::integrations::gemini::Gemini::builder() .default_embed_model("gemini-embedding-exp-03-07") .default_prompt_model("gemini-2.0-flash") .build()?; ``` Source: [`examples/hello_agents.rs#L109-L112`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hello_agents.rs#L109-L112) The variable is named `openai`, but its type is `Gemini`. Later in the example it goes to `Agent::builder().llm(&openai)`, which works because `Gemini` implements `ChatCompletion`. The agent needs the `swiftide-agents` feature flag as well. The key comes from `GEMINI_API_KEY`. The embed model in the example, `gemini-embedding-exp-03-07`, is experimental. Check Google’s model list for a current embedding model before you copy it. ## OpenRouter [Section titled “OpenRouter”](#openrouter) ```rust let openrouter = OpenRouter::builder() .default_prompt_model("anthropic/claude-sonnet-4.5") .build()?; let config = OpenRouterConfig::builder() .api_key(std::env::var("TEAM_OPENROUTER_KEY")?) .site_url("https://example.com/my-app") .site_name("My App") .build()?; let team_openrouter = OpenRouter::builder() .client(async_openai::Client::with_config(config)) .default_prompt_model("openai/gpt-5-mini") .build()?; ``` Source: [`snippets/src/integrations/openai_compatible.rs#L71-L84`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/openai_compatible.rs#L71-L84) The first client reads `OPENROUTER_API_KEY`. The second takes the key from an `OpenRouterConfig`, built with `OpenRouterConfig::builder()`. That builder requires `api_key`, `site_url` and `site_name`. Leave one out and `build()` returns an error at runtime. `site_url` sets the `HTTP-Referer` header, which OpenRouter uses for its app rankings. At this release the `X-Title` header also takes the `site_url` value, so `site_name` is required but not sent. `OpenRouterConfig::default()` leaves both unset. It then sends `https://github.com/bosun-ai/swiftide` as `HTTP-Referer` and `Swiftide` as `X-Title`. ## Dashscope [Section titled “Dashscope”](#dashscope) examples/dashscope.rs ```rust let client = DashscopeBuilder::default() .default_embed_model("text-embedding-v2") .default_prompt_model("qwen-long") .build()?; let tempdir = TempDir::new().unwrap(); let lancedb = LanceDB::builder() .uri(tempdir.child("lancedb").to_str().unwrap()) .vector_size(1536) .with_vector(EmbeddedField::Combined) .with_metadata(metadata_qa_text::NAME) .table_name("swiftide_test") .build() .unwrap(); ``` Source: [`examples/dashscope.rs#L22-L34`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/dashscope.rs#L22-L34) `DashscopeBuilder::default()` is the same as `Dashscope::builder()`. The client prompts with `qwen-long` and embeds with `text-embedding-v2`. The example stores the vectors in LanceDB with a vector size of 1536, so it also needs the `lancedb` feature flag. [LanceDB](/integrations/lancedb/) covers that part. ## Other gateways [Section titled “Other gateways”](#other-gateways) The configuration types of `Groq`, `Gemini` and `Dashscope` are private. To use another key variable or another base URL with them, use `OpenAI` with `api_base` and `api_key`. ```rust let groq = OpenAI::builder() .api_base("https://api.groq.com/openai/v1") .api_key(std::env::var("TEAM_GROQ_KEY")?) .default_prompt_model("llama-3.3-70b-versatile") .build()?; ``` Source: [`snippets/src/integrations/openai_compatible.rs#L93-L97`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/openai_compatible.rs#L93-L97) For vLLM, LiteLLM or a company gateway, see [Custom base URL](/integrations/openai/#custom-base-url). ## Gotchas [Section titled “Gotchas”](#gotchas) * None of the clients has a default model. Set one with `default_prompt_model` or `default_embed_model`, or calls fail with `Model not set`. * A client reads its key when you build it. For `Groq`, `Gemini` and `Dashscope`, a missing key becomes an empty one, and the provider rejects the first request. `OpenRouter` panics on the first request instead. * Each alias is its own type. To pick the provider at runtime, see [Clients as trait objects](/models/overview/#clients-as-trait-objects). * The endpoint decides what works. Every client implements `EmbeddingModel` and `StructuredPrompt`, but a provider without embeddings or structured output returns an error at runtime. * Every client uses the Chat Completions API by default. `use_responses_api(true)` exists on every builder, but only some providers serve the Responses API. * For request fields that only one provider knows, set `extra_body` in `Options`. Only `complete` and `complete_stream` send it. `prompt`, `structured_prompt` and `embed` do not. See [Extra request fields](/models/overview/#extra-request-fields). * Some gateways cap the output at 4096 tokens when `max_completion_tokens` is unset. Set it in `Options`, because a cut-off answer looks complete. ## Reference [Section titled “Reference”](#reference) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`GenericOpenAI::with_default_prompt_model`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html#method.with_default_prompt_model) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) * [`Options`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`Ollama`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/type.Ollama.html) * [`OllamaConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/config/struct.OllamaConfig.html) * [`OllamaConfig::with_api_base`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/config/struct.OllamaConfig.html#method.with_api_base) * [`Groq`](https://docs.rs/swiftide/0.34/swiftide/integrations/groq/type.Groq.html) * [`Gemini`](https://docs.rs/swiftide/0.34/swiftide/integrations/gemini/type.Gemini.html) * [`OpenRouter`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/type.OpenRouter.html) * [`OpenRouterConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/config/struct.OpenRouterConfig.html) * [`OpenRouterConfigBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/config/struct.OpenRouterConfigBuilder.html) * [`Dashscope`](https://docs.rs/swiftide/0.34/swiftide/integrations/dashscope/type.Dashscope.html) * [`DashscopeBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/dashscope/type.DashscopeBuilder.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`Embed` (indexing)](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`Embed` (query)](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) ## Next [Section titled “Next”](#next) [Anthropic](/integrations/anthropic/) # Integrations > Lists every Swiftide integration with the feature flag that enables it, the traits it implements, its main types, examples and guide. Each Swiftide integration sits behind its own feature flag. The matrix below shows which flag to add, what the integration plugs into, and where its guide is. An integration is a model provider, storage backend, node cache, loader, transformer, agent tool, token estimator or observability layer that lives behind a feature flag of the `swiftide` crate. ## Feature flags [Section titled “Feature flags”](#feature-flags) Name the feature flags of the integrations you use when you add `swiftide`: ```sh cargo add swiftide@0.34 --features qdrant,fastembed ``` Feature flags are additive. Each one compiles its integration and the flags it depends on. Without its flag, an integration’s module does not exist: `swiftide::integrations::qdrant` needs `qdrant`. Only `macros` is on by default. Most integrations live under `swiftide::integrations`, which re-exports the `swiftide-integrations` crate with the same feature names. A few sit elsewhere. MCP (Model Context Protocol) tools are in `swiftide::agents::tools::mcp`, Langfuse is under `swiftide::langfuse`, and the tree-sitter transformers are also re-exported under `swiftide::indexing::transformers`. Some flags turn on other flags: * `groq`, `gemini`, `ollama`, `open-router` and `dashscope` each turn on `openai`. Their clients build on the OpenAI client. * `mcp` and `langfuse` each turn on `swiftide-agents`, which turns on `swiftide-tasks`. You do not need to add `swiftide-agents` next to them. Some flags compile native code or large SDKs (software development kits) and make builds slower: * `fastembed` downloads ONNX (Open Neural Network Exchange) Runtime binaries at build time. * `lancedb` compiles an embedded database. It also needs a `protoc` binary on your `PATH` at build time. * `duckdb` links a DuckDB library that is already on your system. Point `DUCKDB_LIB_DIR` at it, make it findable through pkg-config, or set `DUCKDB_DOWNLOAD_LIB=1` to download it during the build. * `kafka` builds librdkafka from source with CMake. * `fluvio` compiles the Fluvio client. * `aws-bedrock` compiles the AWS (Amazon Web Services) SDK. Turn these on only in the crates that need them. ## The all feature [Section titled “The all feature”](#the-all-feature) The `all` flag turns on ten common integrations, but not `anthropic`, `lancedb`, `mcp` or `swiftide-agents`. It also includes `fastembed` and `aws-bedrock`, so it builds slowly. [The all bundle](/reference/feature-flags/#the-all-bundle) lists exactly what it includes. ## TLS [Section titled “TLS”](#tls) The HTTP (Hypertext Transfer Protocol) clients use rustls for TLS (Transport Layer Security) and need no setup: each uses the rustls crypto provider your application installed, or else its own. The exception is Redis over `rediss://`, where Swiftide installs aws-lc-rs as the process-wide provider before connecting if none is installed, so install yours before the first Redis call to use another. The `fastembed` model download goes through native TLS, which means OpenSSL on Linux. ## Matrix columns [Section titled “Matrix columns”](#matrix-columns) Every flag, type and example in the matrix matches Swiftide 0.34. * **Integration** names the provider or backend, with a short note. * **Feature flag** is the flag to pass to `cargo add`. * **Provides** says where the integration plugs in: * **LLM**: a large language model (LLM) client for prompts, chat completions and agents (`SimplePrompt`, `ChatCompletion`). * **Embeddings** and **Sparse embeddings**: an embedder for dense or sparse vectors (`EmbeddingModel`, `SparseEmbeddingModel`). * **Structured output**: typed responses through `StructuredPrompt`. * **Storage**: a backend that the indexing pipeline stores nodes in (`Persist`). * **Retrieval**: a retriever for the query pipeline (`Retrieve`). The storage guide for each backend lists the search strategies it supports. * **Node cache**: a node cache that skips nodes an indexing pipeline has already processed (`NodeCache`). * **Loader**: a loader that produces nodes (`Loader`). * **Transformers**: transformers and chunkers for the indexing pipeline. * **Reranking**: a response transformer that reorders retrieved documents (`TransformResponse`). * **Agent tooling**: tools for agents, or token estimation. * **Observability**: traces or metrics. * **Main types** are the types you build. Each links to docs.rs for the pinned release. * **Examples** link to example files in the Swiftide repository at the pinned commit. * **Guide** is the page on this site that shows the integration in use. | Integration | Feature flag | Provides | Main types | Examples | Guide | | -------------------------------------------------------------------------------------------------------- | ------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | | OpenAI: Chat Completions and the Responses API; Azure OpenAI through `GenericOpenAI` with `AzureConfig`. | `openai` | LLM, Embeddings, Structured output | [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html), [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) | [`responses_api.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api.rs), [`responses_api_reasoning.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api_reasoning.rs), [`structured_prompt.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/structured_prompt.rs), [`describe_image.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/describe_image.rs) | [integrations/openai](/integrations/openai/) | | Anthropic | `anthropic` | LLM | [`Anthropic`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Anthropic.html) | [`streaming_agents.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/streaming_agents.rs) | [integrations/anthropic](/integrations/anthropic/) | | AWS Bedrock: Uses the Bedrock Converse API. | `aws-bedrock` | LLM, Structured output | [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) | [`aws_bedrock.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock.rs), [`aws_bedrock_agent.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs) | [integrations/aws-bedrock](/integrations/aws-bedrock/) | | Groq: OpenAI-compatible client preconfigured for Groq. | `groq` | LLM, Structured output | [`Groq`](https://docs.rs/swiftide/0.34/swiftide/integrations/groq/type.Groq.html) | [`index_groq.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_groq.rs) | [integrations/openai-compatible](/integrations/openai-compatible/) | | Google Gemini: OpenAI-compatible client preconfigured for the Gemini API. | `gemini` | LLM, Embeddings, Structured output | [`Gemini`](https://docs.rs/swiftide/0.34/swiftide/integrations/gemini/type.Gemini.html) | [`hello_agents.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hello_agents.rs) | [integrations/openai-compatible](/integrations/openai-compatible/) | | Ollama: OpenAI-compatible client preconfigured for a local Ollama server. | `ollama` | LLM, Embeddings, Structured output | [`Ollama`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/type.Ollama.html) | [`index_ollama.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_ollama.rs), [`index_md_into_pgvector.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_md_into_pgvector.rs) | [integrations/openai-compatible](/integrations/openai-compatible/) | | OpenRouter: OpenAI-compatible client preconfigured for OpenRouter. | `open-router` | LLM, Structured output | [`OpenRouter`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/type.OpenRouter.html) | - | [integrations/openai-compatible](/integrations/openai-compatible/) | | Dashscope: OpenAI-compatible client preconfigured for Alibaba Dashscope. | `dashscope` | LLM, Embeddings, Structured output | [`Dashscope`](https://docs.rs/swiftide/0.34/swiftide/integrations/dashscope/type.Dashscope.html) | [`dashscope.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/dashscope.rs) | [integrations/openai-compatible](/integrations/openai-compatible/) | | FastEmbed: Runs dense, sparse and reranking models locally with ONNX Runtime. | `fastembed` | Embeddings, Sparse embeddings, Reranking | [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html), [`Rerank`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.Rerank.html) | [`fastembed.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/fastembed.rs), [`hybrid_search.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs), [`reranking.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs) | [integrations/fastembed](/integrations/fastembed/) | | Qdrant: Similarity and hybrid (dense + sparse) search, named vectors, filters. | `qdrant` | Storage, Retrieval | [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) | [`index_codebase.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase.rs), [`hybrid_search.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs), [`store_multiple_vectors.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/store_multiple_vectors.rs) | [integrations/qdrant](/integrations/qdrant/) | | PgVector | `pgvector` | Storage, Retrieval | [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html) | [`index_md_into_pgvector.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_md_into_pgvector.rs) | [integrations/pgvector](/integrations/pgvector/) | | LanceDB: Embedded, no server required; also supports custom queries. | `lancedb` | Storage, Retrieval | [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) | [`lancedb.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/lancedb.rs), [`dashscope.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/dashscope.rs) | [integrations/lancedb](/integrations/lancedb/) | | DuckDB: Embedded; similarity and hybrid search. | `duckdb` | Storage, Retrieval, Node cache | [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) | - | [indexing/storage](/indexing/storage/) | | Redis: Also stores agent message history. | `redis` | Storage, Node cache | [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) | [`index_into_redis.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_into_redis.rs), [`index_codebase.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase.rs) | [indexing/caching](/indexing/caching/) | | Redb: Embedded node cache, no server required. | `redb` | Node cache | [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) | - | [indexing/caching](/indexing/caching/) | | Kafka | `kafka` | Loader, Storage | [`Kafka`](https://docs.rs/swiftide/0.34/swiftide/integrations/kafka/struct.Kafka.html) | [`kafka.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/kafka.rs) | [indexing/loading](/indexing/loading/) | | Fluvio | `fluvio` | Loader | [`Fluvio`](https://docs.rs/swiftide/0.34/swiftide/integrations/fluvio/struct.Fluvio.html) | [`fluvio.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/fluvio.rs) | [indexing/loading](/indexing/loading/) | | Parquet | `parquet` | Loader | [`Parquet`](https://docs.rs/swiftide/0.34/swiftide/integrations/parquet/struct.Parquet.html) | - | [indexing/loading](/indexing/loading/) | | Scraping (spider) | `scraping` | Loader, Transformers | [`ScrapingLoader`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.ScrapingLoader.html), [`HtmlToMarkdownTransformer`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.HtmlToMarkdownTransformer.html) | [`scraping_index_to_markdown.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/scraping_index_to_markdown.rs) | [indexing/loading](/indexing/loading/) | | Tree-sitter: Code-aware chunking, outlines and metadata; Perl behind `tree-sitter-perl`. | `tree-sitter` | Transformers | [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html), [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html), [`OutlineCodeTreeSitter`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.OutlineCodeTreeSitter.html), [`MetadataRefsDefsCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataRefsDefsCode.html) | [`index_codebase.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase.rs), [`index_codebase_reduced_context.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase_reduced_context.rs) | [integrations/tree-sitter](/integrations/tree-sitter/) | | MCP: Exposes the tools of any MCP server (via rmcp) to an agent. | `mcp` | Agent tooling | [`McpToolbox`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/mcp/struct.McpToolbox.html) | [`agents_mcp_tools.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_mcp_tools.rs) | [agents/mcp](/agents/mcp/) | | tiktoken: Token estimation with OpenAI tokenizers. | `tiktoken` | Agent tooling | [`TikToken`](https://docs.rs/swiftide/0.34/swiftide/integrations/tiktoken/struct.TikToken.html) | - | [models/errors-backoff-and-usage](/models/errors-backoff-and-usage/) | | Langfuse: A tracing layer that sends agent and pipeline spans to Langfuse. | `langfuse` | Observability | [`LangfuseLayer`](https://docs.rs/swiftide/0.34/swiftide/langfuse/struct.LangfuseLayer.html) | [`langfuse.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/langfuse.rs) | [observability/langfuse](/observability/langfuse/) | | Metrics: Emits usage and pipeline metrics through the `metrics` crate facade. | `metrics` | Observability | - | [`usage_metrics.rs`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/usage_metrics.rs) | [observability/metrics-and-statistics](/observability/metrics-and-statistics/) | ## Guides [Section titled “Guides”](#guides) This section has a page for each of these integrations: * [OpenAI and Azure](/integrations/openai/): Chat Completions, the Responses API (application programming interface), and Azure OpenAI. * [OpenAI-compatible providers](/integrations/openai-compatible/): Groq, Gemini, Ollama, OpenRouter, Dashscope and any OpenAI-compatible gateway. * [Anthropic](/integrations/anthropic/): prompts, chat completions, streaming and agents with Claude. * [AWS Bedrock](/integrations/aws-bedrock/): Bedrock models through the Converse API. * [FastEmbed](/integrations/fastembed/): dense and sparse embeddings and reranking on your CPU (central processing unit). * [Qdrant](/integrations/qdrant/): storage with similarity and hybrid search. * [PgVector](/integrations/pgvector/): storage in Postgres with the pgvector extension. * [LanceDB](/integrations/lancedb/): an embedded vector store in a local directory. * [Tree-sitter](/integrations/tree-sitter/): chunking, outlines and metadata for source code. The other integrations are covered on the page for the feature they serve. The Guide column in the matrix links to each one: DuckDB, Redis and Redb with storage and caching, Kafka, Fluvio, Parquet and scraping with loading, MCP with agents, tiktoken with model usage, and Langfuse and metrics with observability. ## Reference [Section titled “Reference”](#reference) * [`swiftide::integrations`](https://docs.rs/swiftide/0.34/swiftide/integrations/index.html) * [`swiftide` crate root, Cargo Features section](https://docs.rs/swiftide/0.34/swiftide/index.html#cargo-features) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html), [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) and [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) and [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html), [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) and [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html) * [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html) and [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html) * [`swiftide::agents::tools::mcp`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/mcp/index.html) * [`swiftide::langfuse`](https://docs.rs/swiftide/0.34/swiftide/langfuse/index.html) * [`swiftide::indexing::transformers`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/index.html) * [Feature flags](/reference/feature-flags/) * [Crates](/reference/crates/) ## Next [Section titled “Next”](#next) [OpenAI and Azure](/integrations/openai/) # PgVector > Store nodes in Postgres with the pgvector extension, search them by cosine distance, filter on a metadata column or run your own SQL. Your indexing pipeline writes nodes into a Postgres table, and your query pipeline finds them by cosine distance. The integration sits behind the `pgvector` feature flag. The excerpts on this page also embed with FastEmbed, prompt through Ollama or use OpenAI, so add the feature flags for the clients you use: ```sh cargo add swiftide@0.34 --features pgvector,fastembed,ollama,openai ``` ## Running Postgres [Section titled “Running Postgres”](#running-postgres) Any Postgres with the pgvector extension works. Swiftide’s own tests use the `pgvector/pgvector:pg17` image: ```sh docker run -d --name swiftide-pgvector -p 5432:5432 \ -e POSTGRES_USER=myuser -e POSTGRES_PASSWORD=mypassword -e POSTGRES_DB=mydatabase \ pgvector/pgvector:pg17 ``` The connection URL for this container is `postgresql://myuser:mypassword@localhost:5432/mydatabase`. ## Builder [Section titled “Builder”](#builder) examples/index_md_into_pgvector.rs ```rust let (_pgv_db_container, pgv_db_url) = swiftide_test_utils::start_postgres().await; tracing::info!("pgv_db_url :: {:#?}", pgv_db_url); let llm_client = integrations::ollama::Ollama::default() .with_default_prompt_model("llama3.2:latest") .to_owned(); let fastembed = integrations::fastembed::FastEmbed::try_default().expect("Could not create FastEmbed"); // Configure Pgvector with a default vector size, a single embedding // and in addition to embedding the text metadata, also store it in a field let pgv_storage = PgVector::builder() .db_url(pgv_db_url) .vector_size(384) .with_vector(EmbeddedField::Combined) .with_metadata(METADATA_QA_TEXT_NAME) .table_name("swiftide_pgvector_test".to_string()) .build() .unwrap(); ``` Source: [`examples/index_md_into_pgvector.rs#L67-L87`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_md_into_pgvector.rs#L67-L87) `start_postgres` is a test helper from `swiftide-test-utils`. That crate is not published, so you cannot add it as a dependency. It starts the same container through testcontainers and returns its URL. In your application, pass your own URL to `db_url`. The example uses the `fastembed` and `ollama` feature flags. `MetadataQAText` calls `llama3.2` through a local Ollama, so pull that model first. `db_url` and `vector_size` are required, and `build()` returns an error without them. `with_vector` is required in practice: without a vector field, setup fails with `No vector field found in configuration`. `vector_size` must match your embedder: FastEmbed’s default model returns 384 dimensions. `with_metadata` adds a metadata column. The example stores the questions and answers that `MetadataQAText` generates, under the field `Questions and Answers (text)`. | Setting | Default | | --------------------- | --------------------- | | `table_name` | `swiftide_pgv_store` | | `batch_size` | 50 nodes per insert | | `db_max_connections` | 10 | | `db_max_retry` | 3 connection attempts | | `db_conn_retry_delay` | 3 seconds | `build()` does not connect. The connection pool opens on first use, retrying as configured above. Cloning a `PgVector` is cheap, and the clones share that pool. In an application, read the URL from your configuration: ```rust let pgvector = PgVector::builder() .db_url(std::env::var("DATABASE_URL")?) .vector_size(1536) .with_vector(EmbeddedField::Combined) .with_metadata("path") .table_name("docs_chunks") .build()?; indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) // Every node needs each `with_metadata` field, or storing it fails .then(|mut node: TextNode| -> anyhow::Result { let path = node.path.to_string_lossy().into_owned(); node.metadata.insert("path", path); Ok(node) }) .then_in_batch(Embed::new(openai.clone())) .then_store_with(pgvector.clone()) .run() .await?; ``` Source: [`snippets/src/integrations/pgvector.rs#L16-L35`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/pgvector.rs#L16-L35) `1536` matches OpenAI’s `text-embedding-3-small`, so build the client with `.default_embed_model("text-embedding-3-small")`. The closure transformer copies the file path into a `path` metadata field, which the [filter](#filters) below uses. ## Table layout [Section titled “Table layout”](#table-layout) When the indexing pipeline starts, the persist setup runs `CREATE EXTENSION IF NOT EXISTS vector`, creates the table if it is missing, and adds a Hierarchical Navigable Small World (HNSW) index. For the example above, that comes down to: ```sql CREATE TABLE IF NOT EXISTS swiftide_pgvector_test ( id UUID NOT NULL, chunk TEXT NOT NULL, vector_combined VECTOR(384), meta_questions_and_answers JSONB, PRIMARY KEY (id) ); CREATE INDEX IF NOT EXISTS swiftide_pgvector_test_embedding_idx ON swiftide_pgvector_test USING hnsw (vector_combined vector_cosine_ops); ``` * `id` is the node id, a universally unique identifier (UUID) derived from the node’s path and chunk. * `chunk` holds the chunk text. * Each `with_vector` field becomes a `vector_` column: `EmbeddedField::Combined` becomes `vector_combined`, `EmbeddedField::Chunk` becomes `vector_chunk`. * Each `with_metadata` field becomes a `meta_` JSONB column. It holds an object with the original field name as key, here `{"Questions and Answers (text)": "..."}`. Column names are normalized. Swiftide keeps the text before the first `(`, `[`, `{` or `<`, takes up to three words, joins them with underscores, lowercases them and drops everything except letters, digits and underscores. The table name must start with a letter or underscore, contain only letters, digits, `_` or `$`, and not be a reserved word such as `table`. It can be at most 49 bytes, because setup appends `_embedding_idx` to name the index and Postgres identifiers stop at 63 bytes. Otherwise setup fails with `Invalid table or field name`. The table is created only when it is missing. An existing table is kept as it is, so a new vector size or a new column means dropping the table first. The example drops it before every run for that reason. `get_pool()` returns the `sqlx` pool for your own Structured Query Language (SQL), such as dropping the table or deleting stale rows, and `get_table_name()` returns the table name. ## Storage [Section titled “Storage”](#storage) examples/index_md_into_pgvector.rs ```rust tracing::info!("Starting indexing pipeline"); indexing::Pipeline::from_loader(FileLoader::new(test_dataset_path).with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(10..2048)) .then(MetadataQAText::new(llm_client.clone())) .then_in_batch(Embed::new(fastembed.clone()).with_batch_size(100)) .then_store_with(pgv_storage.clone()) .run() .await?; ``` Source: [`examples/index_md_into_pgvector.rs#L103-L111`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_md_into_pgvector.rs#L103-L111) `then_store_with` hands nodes to PgVector in batches of `batch_size`. Each batch is one `INSERT ... SELECT FROM UNNEST(...)` statement inside a transaction, with `ON CONFLICT (id) DO UPDATE`. Indexing the same chunk again overwrites its row. A changed chunk gets a new id, so its old row stays until you delete it. The default embed mode produces one `Combined` vector per node, which matches `with_vector(EmbeddedField::Combined)`. ## Retrieval [Section titled “Retrieval”](#retrieval) examples/index_md_into_pgvector.rs ```rust async fn ask_query( llm_client: impl SimplePrompt + Clone + 'static, embed: FastEmbed, vector_store: PgVector, questions: Vec, ) -> Result, Box> { // By default the search strategy is SimilaritySingleEmbedding // which takes the latest query, embeds it, and does a similarity search // // Pgvector will return an error if multiple embeddings are set // // The pipeline generates subquestions to increase semantic coverage, embeds these in a single // embedding, retrieves the default top_k documents, summarizes them and uses that as context // for the final answer. let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( llm_client.clone(), )) .then_transform_query(query_transformers::Embed::from_client(embed)) .then_retrieve(vector_store.clone()) .then_transform_response(response_transformers::Summary::from_client( llm_client.clone(), )) .then_answer(answers::Simple::from_client(llm_client.clone())); let results: Vec = pipeline .query_all(questions) .await? .iter() .map(|result| result.answer().to_string()) .collect(); Ok(results) } ``` Source: [`examples/index_md_into_pgvector.rs#L17-L50`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_md_into_pgvector.rs#L17-L50) `query::Pipeline::default()` uses `SimilaritySingleEmbedding` with a `top_k` of 10. Embed the query with the same model you indexed with. PgVector selects `id`, `chunk` and every metadata column, orders by cosine distance (`<=>`) and returns `top_k` documents. Each document carries its metadata under the original field names. Retrieval needs exactly one vector column. With more than one, it fails with `Search strategy for multiple vector fields in the schema is not yet implemented`. PgVector has no hybrid search. ## Filters [Section titled “Filters”](#filters) `SimilaritySingleEmbedding` takes a filter of the form `key = value`: ```rust // `FileLoader::new("docs")` stores paths with the `docs/` prefix, and the match is exact let search = SimilaritySingleEmbedding::from_filter("path = docs/README.md".to_string()).with_top_k(5); let pipeline = query::Pipeline::from_search_strategy(search) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(pgvector) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/integrations/pgvector.rs#L43-L50`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/pgvector.rs#L43-L50) PgVector splits the filter at the first `=`, trims whitespace and strips double quotes around the value. It then adds `WHERE meta_ ->> $1 = $2`, binding the original key and the value. A filter is one exact text equality on a `with_metadata` field. The key must match the original field name, and the column must exist. The value must match the stored text exactly: `FileLoader::new("docs")` stores `docs/README.md`, not `README.md`. Nodes do not carry a `path` metadata field on their own; add one before storing, as in the [own database excerpt](#builder). ## Custom SQL [Section titled “Custom SQL”](#custom-sql) For anything a filter cannot express, `CustomStrategy>` runs your own query: ```rust use sqlx::{Postgres, QueryBuilder}; // The default table is `swiftide_pgv_store`; `meta_path` exists only with `.with_metadata("path")` let table = pgvector.get_table_name().to_owned(); let keyword_search = CustomStrategy::from_query(move |query: &Query| { // The retriever reads `id`, `chunk` and every `meta_` column from each row let mut sql = QueryBuilder::::new(format!("SELECT id, chunk, meta_path FROM {table}")); sql.push(" WHERE to_tsvector('english', chunk) @@ plainto_tsquery('english', ") .push_bind(query.current().to_string()) .push(") LIMIT 10"); Ok(sql) }); let pipeline = query::Pipeline::from_search_strategy(keyword_search) .then_retrieve(pgvector) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/querying/retrieval.rs#L62-L79`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/retrieval.rs#L62-L79) Query the PgVector table: take its name from `get_table_name()` rather than writing it by hand. PgVector maps each row by column name. `id` and `chunk` are required, and every column that starts with `meta_` becomes metadata. A custom strategy does not read the query embedding, so this pipeline needs no `Embed` step. See [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) for how `CustomStrategy` builds the query and which `sqlx` version to add. ## Gotchas [Section titled “Gotchas”](#gotchas) * Every node must carry every `with_metadata` field. A node without one fails the batch with `Missing metadata field`. * A node without the vector named in `with_vector` is stored as an empty vector, and Postgres rejects the batch with a dimension error. Keep the embed mode and `with_vector` in sync: `EmbedMode::PerField` produces no `Combined` vector. * Only the first vector column gets the HNSW index. * `SimilaritySingleEmbedding` fails with `Missing embedding in query state` when the query pipeline has no `Embed` query transformer. ## Reference [Section titled “Reference”](#reference) * [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html) * [`PgVectorBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVectorBuilder.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) ## Next [Section titled “Next”](#next) [LanceDB](/integrations/lancedb/) # Qdrant > Store indexed chunks in a Qdrant collection with dense and sparse vectors, then retrieve them with similarity or hybrid search and payload filters. With the `qdrant` feature flag, an indexing pipeline writes nodes into a Qdrant collection, and a query pipeline searches that collection by similarity or with hybrid search. ```sh cargo add swiftide@0.34 --features qdrant ``` Hybrid search also needs `fastembed` for sparse embeddings, and the filter snippet needs `openai`. ## Local instance and connection [Section titled “Local instance and connection”](#local-instance-and-connection) Start Qdrant in Docker: ```sh docker run -d -p 6333:6333 -p 6334:6334 qdrant/qdrant ``` Swiftide talks to Qdrant over gRPC Remote Procedure Call (gRPC) on port 6334, not over the Hypertext Transfer Protocol (HTTP) API on 6333. ## Builder [Section titled “Builder”](#builder) examples/docs-qa/src/store.rs ```rust let qdrant = Qdrant::builder() .batch_size(BATCH_SIZE) .vector_size(DENSE_VECTOR_SIZE) // One dense and one sparse vector, both over the chunk *and* its metadata .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name(&collection.0) .build()?; ``` Source: [`examples/docs-qa/src/store.rs#L38-L45`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/store.rs#L38-L45) `Qdrant::builder()` reads the address from `QDRANT_URL`, falls back to `http://localhost:6334`, and sends `QDRANT_API_KEY` when set. `Qdrant::try_from_url(url)` returns `Result`, so add `?` before calling builder methods. The builder’s `client` method takes your own `qdrant_client::Qdrant`. `vector_size` is required and must match your embedder. docs-qa uses 384, the size of FastEmbed’s default dense model. The other settings have defaults: | Setting | Default | | ----------------- | ---------------------------------------------- | | `collection_name` | `swiftide` | | `vector_distance` | `Distance::Cosine` | | `batch_size` | 50 nodes per upsert | | vectors | one dense vector for `EmbeddedField::Combined` | `with_vector` adds a dense vector and `with_sparse_vector` adds a sparse one. The first `with_vector` call replaces the default, so list `EmbeddedField::Combined` yourself when you still need it. Both take an `EmbeddedField`, which names the text the vector was computed from. `Combined` is the chunk together with its metadata. ## Stored points [Section titled “Stored points”](#stored-points) `Qdrant` implements `Persist`, so you hand it to `then_store_with`: examples/docs-qa/src/index.rs ```rust indexing::Pipeline::from_loader(loader) .then_chunk(ChunkMarkdown::from_chunk_range( MIN_CHUNK_CHARS..MAX_CHUNK_CHARS, )) // A short title and keywords per chunk; both end up in the embedded text and help the // sparse (keyword) side of hybrid search in particular. .then( MetadataTitle::from_client(llm.clone()) .prompt_template(TITLE_PROMPT) .build()?, ) .then( MetadataKeywords::from_client(llm.clone()) .prompt_template(KEYWORDS_PROMPT) .build()?, ) .then_in_batch(Embed::new(embedders.dense.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_in_batch(SparseEmbed::new(embedders.sparse.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_store_with(qdrant.clone()) .run() .await ``` Source: [`examples/docs-qa/src/index.rs#L50-L70`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/index.rs#L50-L70) Before the first node, `setup` creates the collection if it is missing. An existing collection is kept as it is. The pipeline then upserts nodes in batches. Every point carries this payload: * each metadata field of the node, under its own name * `path`, the node’s source path * `content`, the chunk text * `last_updated_at`, the time of the upsert The point ID is a universally unique identifier (UUID) computed from the path and the chunk, so indexing the same chunk again overwrites its point. Debug builds wait for each upsert to apply; release builds do not. ### Captured output [Section titled “Captured output”](#captured-output) A debug build of docs-qa indexed four markdown pages into 43 chunks, with metadata from `claude-haiku-5-5` and embeddings from local FastEmbed models: ```text Indexed 43 chunks into `docs-qa` in 71.7s ``` ## Named vectors [Section titled “Named vectors”](#named-vectors) A point can hold one dense vector per embedded field: examples/store_multiple_vectors.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("LICENSE")) .with_concurrency(1) .with_embed_mode(EmbedMode::PerField) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .then(MetadataQAText::new(openai_client.clone())) .then(MetadataSummary::new(openai_client.clone())) .then(MetadataTitle::new(openai_client.clone())) .then(MetadataKeywords::new(openai_client.clone())) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .log_all() .filter_errors() .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-multi-vectors") .with_vector(EmbeddedField::Chunk) .with_vector(EmbeddedField::Metadata(metadata_qa_text::NAME.into())) .with_vector(EmbeddedField::Metadata(metadata_summary::NAME.into())) .with_vector( VectorConfig::builder() .embedded_field(EmbeddedField::Metadata(metadata_title::NAME.into())) .distance(Distance::Manhattan) .build()?, ) .with_vector(EmbeddedField::Metadata(metadata_keywords::NAME.into())) .build()?, ) .run() .await?; ``` Source: [`examples/store_multiple_vectors.rs#L45-L74`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/store_multiple_vectors.rs#L45-L74) `EmbedMode::PerField` embeds the chunk and every metadata field separately. `EmbedMode::Both` adds the combined text. When the collection has named vectors, Qdrant stores only the ones you configured and skips the rest. The embed mode and the configured vectors must agree. A node with exactly one dense vector and no sparse vector is always sent as the unnamed vector. With several configured vectors, use `EmbedMode::PerField` or `Both`. With one configured vector, keep the default embed mode. Sparse vectors are only stored for fields that also have a configured dense vector. `VectorConfig::builder()` overrides the size or distance of one vector, here `Distance::Manhattan` for the title. With one dense vector and no sparse vector, the collection uses Qdrant’s unnamed vector. Otherwise vectors are named after their field: `Combined`, `Chunk` or `Metadata: `. Sparse vectors get a `_sparse` suffix, such as `Combined_sparse`. ## Similarity search [Section titled “Similarity search”](#similarity-search) A query pipeline searches Qdrant through `then_retrieve`. This one narrows the search with a payload filter: ```rust // Payload keys are the node's metadata plus `path`, `content` and `last_updated_at` let guides = Filter { must: vec![Condition::matches("path", ["README.md", "CONTRIBUTING.md"])], must_not: vec![Condition::matches("deprecated", true)], ..Default::default() }; let similarity = SimilaritySingleEmbedding::from_filter(guides.clone()).with_top_k(5); let pipeline = query::Pipeline::from_search_strategy(similarity) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); // On a hybrid search, the filter applies to the dense and the sparse candidates let hybrid = HybridSearch::from_filter(guides).with_top_n(20); ``` Source: [`snippets/src/integrations/qdrant.rs#L20-L35`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/qdrant.rs#L20-L35) `SimilaritySingleEmbedding` returns the 10 nearest chunks unless you set `with_top_k`. Only `then_retrieve(qdrant)` is Qdrant-specific. The query must already carry a dense embedding from `query_transformers::Embed`, made with the model you indexed with. Swiftide picks the vector to search from the configuration of the `Qdrant` value in the query pipeline, not from the collection. With more than one vector or any sparse vector configured, it searches the vector named `Combined`. Build the retriever with the same vector configuration as the indexing side; docs-qa calls the same `store::connect` for both. The collection in the previous section has no `Combined` vector, so it cannot serve this search. ## Hybrid search [Section titled “Hybrid search”](#hybrid-search) Hybrid search adds a sparse, keyword-like search to the dense one: examples/docs-qa/src/query.rs ```rust query::Pipeline::from_search_strategy( HybridSearch::default() .with_top_n(CANDIDATES) .with_top_k(CANDIDATES), ) ``` Source: [`examples/docs-qa/src/query.rs#L108-L112`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L108-L112) The query then needs both embeddings before retrieval: examples/docs-qa/src/query.rs ```rust pipeline // Hybrid search needs both a dense and a sparse embedding of the question .then_transform_query(query_transformers::Embed::from_client( self.embedders.dense.clone(), )) .then_transform_query(query_transformers::SparseEmbed::from_client( self.embedders.sparse.clone(), )) .then_retrieve(self.qdrant.clone()) // A local cross-encoder reorders the fused candidates; only the best few remain .then_transform_response(self.reranker.clone()) .then_answer(self.answerer.clone()) ``` Source: [`examples/docs-qa/src/query.rs#L121-L134`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L121-L134) Qdrant fetches `top_n` candidates from the dense and from the sparse vector and fuses them with Reciprocal Rank Fusion (RRF); see [search strategies](/querying/retrieval-and-search-strategies/) for `top_n` and `top_k`. Both vector fields default to `Combined`; `with_dense_vector_field` and `with_sparse_vector_field` select other named vectors. The collection needs a sparse vector for the same field. Without a sparse embedding on the query, retrieval fails with “No sparse embedding for query”. ## Payload filters [Section titled “Payload filters”](#payload-filters) Both search strategies take a Qdrant `Filter`, as in the similarity snippet. Swiftide re-exports the client crate at `swiftide::integrations::qdrant::qdrant_client`. Conditions match payload keys, including any metadata field. An array of strings matches any of its values as keywords. An owned `String` that contains whitespace becomes a full-text match instead of a keyword match. A full-text match needs a full-text payload index on that field, which Swiftide does not create. Without one, Qdrant matches the text as an exact substring. Qdrant implements hybrid search only for `HybridSearch`. Write that type even when you set no filter. For `SimilaritySingleEmbedding`, Qdrant accepts both `Filter` and no filter. `SimilaritySingleEmbedding::default()` is the search without a filter and needs no type annotation. Before 0.34, it needed one. ## Gotchas [Section titled “Gotchas”](#gotchas) * Retrieved content is `Debug`-quoted, with escaped newlines. docs-qa wraps the retriever in [`CleanQdrant`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/retrieval.rs) (the `self.qdrant` in the query pipeline snippet above), which restores the text before reranking and answering. * A changed vector size, distance or set of vectors needs a new collection. * A changed chunk gets a new point ID, and the old point stays. docs-qa’s `--recreate` flag drops the collection first. * Retrieved documents keep every payload key except `content` as metadata. * `client()` returns a shared reference to the inner client, `&Arc`. docs-qa uses it for a health check, `collection_exists` and `delete_collection`. * If Qdrant is down, `run` fails during setup. See [unreachable stores](/reference/troubleshooting-and-faq/#unreachable-stores). ## Reference [Section titled “Reference”](#reference) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`QdrantBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.QdrantBuilder.html) * [`VectorConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.VectorConfig.html) * [`SparseVectorConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.SparseVectorConfig.html) * [`Distance`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/type.Distance.html) * [`qdrant_client`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/qdrant_client/index.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`EmbedMode`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbedMode.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html) * [docs-qa source](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa) ## Next [Section titled “Next”](#next) [PgVector](/integrations/pgvector/) # Tree-sitter > Split source code along syntax boundaries with ChunkCode and add outlines, questions, references and definitions as metadata, behind the tree-sitter feature flag. With this integration your indexing pipeline splits source files along functions, classes and other syntax boundaries instead of at a fixed length, and adds code-aware metadata to every chunk. Tree-sitter is a parser library with a grammar per language. Swiftide ships the grammars and a set of transformers that use them. ## Feature flag [Section titled “Feature flag”](#feature-flag) ```sh cargo add swiftide@0.34 --features tree-sitter ``` The `tree-sitter` feature flag adds the grammars and the transformers `ChunkCode`, `MetadataQACode`, `MetadataRefsDefsCode`, `OutlineCodeTreeSitter` and `CompressCodeOutline`. They are re-exported at `swiftide::indexing::transformers`, next to the other transformers. `SupportedLanguages` sits at `swiftide::integrations::treesitter`. The `tree-sitter-perl` feature flag adds Perl and turns on `tree-sitter` too. ## Supported languages [Section titled “Supported languages”](#supported-languages) ```rust // Case-insensitive: "python", "Python" and "PYTHON" all parse let language: SupportedLanguages = "python".parse()?; indexing::Pipeline::from_loader( FileLoader::new("src").with_extensions(language.file_extensions()), ) .then_chunk(ChunkCode::try_for_language(language)?) .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`snippets/src/integrations/tree_sitter.rs#L20-L29`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/integrations/tree_sitter.rs#L20-L29) `SupportedLanguages` implements `FromStr` and ignores case. `ChunkCode`, `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` take a `SupportedLanguages` value or a string such as `"rust"`, and return an error that says `Treesitter language not supported` for a name they do not know. `MetadataQACode` and `CompressCodeOutline` take no language, only an LLM client. `file_extensions()` gives the extensions to pass to `FileLoader`. | Language | Names | Extensions | Outline | References and definitions | | ---------- | ------------------------- | ---------------- | ------- | -------------------------- | | Rust | `rust` | rs | yes | yes | | TypeScript | `typescript` | ts, tsx, js, jsx | yes | yes | | JavaScript | `javascript` | js, jsx | yes | yes | | Python | `python` | py | yes | yes | | Ruby | `ruby` | rb | yes | yes | | Java | `java` | java | yes | yes | | C# | `csharp`, `c-sharp`, `c#` | cs, csx | yes | yes | | Go | `go` | go | no | yes | | Solidity | `solidity` | sol | no | yes | | C | `c` | c, h, o | no | no | | C++ | `cpp`, `c++` | c, h, o, cc, cpp | no | no | | Elixir | `elixir` | ex, exs | no | no | | HTML | `html` | html, htm, xhtml | no | no | | PHP | `php` | php | no | no | | Perl | `perl` | pm, pl, t | no | no | Every language works with `ChunkCode`. The last two columns say which languages `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` handle. ## Chunk size and splitting [Section titled “Chunk size and splitting”](#chunk-size-and-splitting) ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .filter_cached(Redis::try_from_url(redis_url, "swiftide-examples")?) .then(MetadataQACode::new(openai_client.clone())) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples") .build()?, ) .run() .await?; ``` Source: [`examples/index_codebase.rs#L52-L68`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_codebase.rs#L52-L68) The example also uses the `redis`, `qdrant` and `openai` feature flags, and needs a running Redis server for the node cache. See [feature flags](/reference/feature-flags/). `ChunkCode` is a chunker. It parses each file and walks the syntax tree, adding top-level items to a chunk. It closes the chunk when the next item would bring it to the maximum size or past it. An item larger than the maximum is split along its own children, one level down at a time. The chunk size is a `ChunkSize`. A number, such as `1000`, is a maximum in bytes. A range, such as `10..2048`, sets a minimum and a maximum: a chunk no larger than the minimum is dropped. `ChunkCode::try_for_language` uses a maximum of 1500 bytes. `with_concurrency` sets how many files are chunked at once. ## Several languages in one pipeline [Section titled “Several languages in one pipeline”](#several-languages-in-one-pipeline) `ChunkCode` handles one language. To index several, pick a `ChunkCode` per file extension, as the ask-your-codebase showcase does with its `ChunkByLanguage` chunker (see [Chunking](/indexing/chunking/)). The first language that lists an extension wins, so the order of the list matters where extensions overlap. ## Questions per chunk [Section titled “Questions per chunk”](#questions-per-chunk) examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` Source: [`examples/ask-your-codebase/src/index.rs#L89-L115`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L89-L115) The showcase builds swiftide with the `tree-sitter`, `fastembed`, `lancedb`, `redb` and `openai` feature flags. `MetadataQACode` asks a large language model (LLM) which questions the code answers and stores the reply under the metadata key `Questions and Answers (code)`. A question at query time then finds the chunk more easily, because the questions are embedded with the code. Run it before chunking to send one prompt per file, which every chunk of that file then inherits. Run it after chunking to get questions per chunk, at one prompt per chunk. It writes 5 questions by default. `MetadataQACode::new(client)` builds it with the defaults, and `from_client(client)` returns a builder for `prompt_template`, `num_questions` and `concurrency`. The showcase runs it after chunking, so each chunk gets its own questions. It also replaces the default prompt with one that asks about behaviour, defined at the top of the same file. When a node has `Outline` metadata, the default prompt includes the outline. The showcase’s run, with `claude-haiku-5-5` writing the questions and FastEmbed embedding on the CPU, indexed the `swiftide-agents` crate like this: ```text Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s ``` ## Outlines, references and definitions [Section titled “Outlines, references and definitions”](#outlines-references-and-definitions) `OutlineCodeTreeSitter`, `CompressCodeOutline` and `MetadataRefsDefsCode` give each chunk an outline of its file and the symbols it defines and uses. For pipelines that add them to every chunk, see [Code-aware indexing](/indexing/code-aware-indexing/). ## Gotchas [Section titled “Gotchas”](#gotchas) * `OutlineCodeTreeSitter` and `MetadataRefsDefsCode` build for every language, but panic on a node in a language they do not handle. `OutlineCodeTreeSitter` skips nodes below its minimum file size first. The panic happens inside a spawned task, so `run()` fails with a join error instead of crashing the process. Check the table before you use them. * When a file has a syntax error, `ChunkCode` logs a warning and returns the whole file as one chunk, whatever its size. `OutlineCodeTreeSitter` returns an error for that node, and the first error stops the whole pipeline: `run()` fails. Add `filter_errors()` after it when your codebase may contain files that do not parse. See [Streams and concurrency](/indexing/streams-and-concurrency/). * `SupportedLanguages::Typescript` lists `tsx`, `js` and `jsx` in `file_extensions()`, but uses the plain TypeScript grammar, which cannot parse JSX. A `.tsx` or `.jsx` file that contains JSX has a syntax error, with the consequences above. Put JavaScript before TypeScript when you pick a language by extension, so `js` and `jsx` go to the JavaScript grammar. * The C and C++ lists both include `c`, `h` and `o`. `o` is the extension of compiled object files, so a `FileLoader` built from them picks up build output. * `MetadataQACode` and `CompressCodeOutline` send one prompt per node. On a large codebase that is many requests; lower the concurrency or wrap the client in `LanguageModelWithBackOff` to stay within rate limits. ## Reference [Section titled “Reference”](#reference) * [`SupportedLanguages`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.SupportedLanguages.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/chunk_code/struct.ChunkCode.html) * [`ChunkSize`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.ChunkSize.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/metadata_qa_code/struct.MetadataQACode.html) * [`MetadataRefsDefsCode`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/metadata_refs_defs_code/struct.MetadataRefsDefsCode.html) * [`OutlineCodeTreeSitter`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/outline_code_tree_sitter/struct.OutlineCodeTreeSitter.html) * [`CompressCodeOutline`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/compress_code_outline/struct.CompressCodeOutline.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [Code-aware indexing](/indexing/code-aware-indexing/) ## Next [Section titled “Next”](#next) [Ask your codebase](/showcases/ask-your-codebase/) # Errors, backoff, usage and tokens > Tell transient from permanent model errors, retry with LanguageModelWithBackOff, record token usage with on_usage, and estimate tokens with CharEstimator or tiktoken. By the end of this page your OpenAI-compatible or `AwsBedrock` client retries rate limits on its own, reports the tokens each call used, and you can estimate the size of a prompt before you send it. ```rust match openai.prompt("Summarise the attached log.".into()).await { Ok(answer) => println!("{answer}"), Err(LanguageModelError::ContextLengthExceeded(e)) => { eprintln!("prompt too long, shorten it: {e}"); } Err(LanguageModelError::TransientError(e)) => { eprintln!("rate limit or outage, try again later: {e}"); } Err(error @ LanguageModelError::PermanentError(_)) => return Err(error.into()), } ``` Source: [`snippets/src/models/errors_backoff_and_usage.rs#L20-L29`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/errors_backoff_and_usage.rs#L20-L29) A failed model call returns a `LanguageModelError`. Each of its three variants wraps the underlying error: * `ContextLengthExceeded`: the input is longer than the model accepts. Shorten it before you send it again. * `TransientError`: the call may succeed if you try again later. * `PermanentError`: the call fails the same way every time. The OpenAI-compatible clients sort provider errors into these variants for you: * The identifier `context_length_exceeded` becomes `ContextLengthExceeded`. * Network errors and Hypertext Transfer Protocol (HTTP) status 408, 429 and 5xx are transient. * OpenRouter’s rate limit, overload, unavailable, server and timeout error types are transient. * Quota and billing errors are permanent, even when they arrive with status 429. * An error inside a successful response is read and sorted the same way. * A response that fails to parse and holds no such error is transient. * Invalid arguments and broken streams are permanent. `AwsBedrock` treats throttling, timeouts, unavailable services and network errors as transient. `Anthropic` treats only network errors on `prompt` as transient. Every other `Anthropic` error is permanent, including status 429. To build a variant yourself, call `LanguageModelError::transient`, `permanent` or `context_length_exceeded`. An `anyhow::Error` converts into a `PermanentError`. To return a variant with `?` or `.into()`, bind the whole `LanguageModelError` (`error @ ...`). The `Box` inside a variant does not convert into `anyhow::Error`. ### Errors from a pipeline [Section titled “Errors from a pipeline”](#errors-from-a-pipeline) When a step of an indexing pipeline fails on a model call, `run` returns an `anyhow::Error` with the `LanguageModelError` still inside. So you can tell a rate limit from a broken request. ```rust let result = indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then(MetadataQAText::new(openai.clone())) .then_in_batch(Embed::new(openai)) .then_store_with(MemoryStorage::default()) .run() .await; if let Err(error) = result { let model_error = error .chain() .find_map(|cause| cause.downcast_ref::()); match model_error { Some(LanguageModelError::TransientError(_)) => { eprintln!("the provider is overloaded, run the pipeline again later: {error}"); } _ => return Err(error), } } ``` Source: [`snippets/src/models/errors_backoff_and_usage.rs#L37-L55`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/errors_backoff_and_usage.rs#L37-L55) `downcast_ref::()` on the error works too. Searching `chain()` also finds the model error when a step added context around it. ## Backoff [Section titled “Backoff”](#backoff) ```rust let openai = LanguageModelWithBackOff::new( openai, BackoffConfiguration { initial_interval_sec: 2, multiplier: 2.0, randomization_factor: 0.5, max_elapsed_time_sec: 120, }, ); let answer = openai.prompt("Name one Rust web framework.".into()).await?; ``` Source: [`snippets/src/models/errors_backoff_and_usage.rs#L63-L73`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/errors_backoff_and_usage.rs#L63-L73) `LanguageModelWithBackOff` wraps a client and retries calls that fail with a `TransientError`. On `prompt` and `complete`, a `ContextLengthExceeded` or `PermanentError` returns at once. The decorated client implements `SimplePrompt`, `ChatCompletion` and `EmbeddingModel` when the inner client does. You can pass it where one of those traits is expected, including `Agent::builder().llm(&openai)`. It never retries an `Anthropic` rate limit, because `Anthropic` marks it permanent. `BackoffConfiguration` has four fields: * `initial_interval_sec`: the wait before the first retry, in seconds. The default is 1. * `multiplier`: the factor the wait grows by after each attempt. The default is 2.0. * `randomization_factor`: random jitter on each wait. The default is 0.5, plus or minus half the interval. * `max_elapsed_time_sec`: the time budget in seconds, including time spent inside calls. The decorator returns the last error as soon as the next wait would pass it, often before the full budget. The default is 60. `Default::default()` gives you these defaults. Every failed attempt that the decorator retries logs a WARN event, `Language model request failed, retrying`, with the attempt number, the wait and the error with its causes. When the decorator gives up, it logs the final error once at ERROR, also for a permanent error that it never retried. Only `prompt` and `complete` log these events. For `complete_stream` the decorator does not send the request again. Your stream still receives every error. After each one, transient or permanent, the decorator waits for the next interval before it reads on. Once `max_elapsed_time_sec` is used up, the stream ends after the last error. The budget restarts after every successful item. OpenAI-compatible streams end after their first error, so the wait only delays that end. Two kinds of call get no retry. `EmbeddingModel` calls the inner client once. `StructuredPrompt` and methods of the client itself, such as `estimate_tokens`, do not exist on the decorator, so call them on the inner client. ## Usage callbacks [Section titled “Usage callbacks”](#usage-callbacks) ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .on_usage(|usage| { println!( "{} tokens in, {} out", usage.prompt_tokens, usage.completion_tokens ); Ok(()) }) .build()?; ``` Source: [`snippets/src/models/errors_backoff_and_usage.rs#L82-L91`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/errors_backoff_and_usage.rs#L82-L91) A usage callback receives the `Usage` of each call: `prompt_tokens`, `completion_tokens` and `total_tokens`. Set it with `on_usage` on the builder of an OpenAI-compatible client (`openai` feature flag), `Anthropic` (`anthropic`) or `AwsBedrock` (`aws-bedrock`). On OpenAI-compatible clients, embedding calls report usage too. Use `on_usage_async` when the callback needs to await: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .on_usage_async(|usage| { Box::pin(async move { // Write to a database or a metrics service here. println!("{} tokens", usage.total_tokens); Ok(()) }) }) .build()?; ``` Source: [`snippets/src/models/errors_backoff_and_usage.rs#L99-L108`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/errors_backoff_and_usage.rs#L99-L108) The closure gets a borrowed `Usage` and returns a boxed future that may borrow it. Copy the numbers out first only if you hand them to a task that must be `'static`, such as one you spawn. The providers run the callback differently: * OpenAI-compatible clients spawn it. An `Err` is logged and the call still succeeds. * `AwsBedrock`, and `Anthropic` on calls that do not stream, await it. An `Err` from the callback fails the call. * `Anthropic` streaming calls spawn it and only log an `Err`. With the `metrics` feature flag, clients also emit usage counters. See [Metrics and statistics](/observability/metrics-and-statistics/). ## Normalized usage [Section titled “Normalized usage”](#normalized-usage) ```rust let request = ChatCompletionRequest::from(vec![ChatMessage::new_user("Hello")]); let response = openai.complete(&request).await?; if let Some(usage) = response.usage.as_ref().map(Usage::normalized) { let cached = usage .details .as_ref() .and_then(|details| details.input.cached_tokens) .unwrap_or(0); println!("{} tokens, {cached} from cache", usage.total_tokens); } ``` Source: [`snippets/src/models/errors_backoff_and_usage.rs#L116-L126`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/errors_backoff_and_usage.rs#L116-L126) `Usage` has an optional `details` field with the provider’s own breakdown. `Usage::normalized` returns a `NormalizedUsage` with one shape for every provider: cached and audio tokens on the input side, and reasoning, audio and prediction tokens on the output side. Its `details` is `None` when the provider reported none of them. OpenAI fills the details from Responses application programming interface (API) calls and from Chat Completions calls that do not stream. `AwsBedrock` reports cached input tokens. Streamed Chat Completions, embeddings and `Anthropic` report the totals only. ## Token estimation [Section titled “Token estimation”](#token-estimation) ```rust let history = vec![ ChatMessage::new_system("You are a helpful assistant."), ChatMessage::new_user("What is a borrow checker?"), ]; let rough = CharEstimator.estimate(history.as_slice()).await?; ``` Source: [`snippets/src/models/errors_backoff_and_usage.rs#L134-L139`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/errors_backoff_and_usage.rs#L134-L139) `CharEstimator` needs no feature flag. It divides the number of characters by 4, then adds 4 tokens per message and 3 per list of messages. `estimate` comes from the `EstimateTokens` trait. It accepts any `Estimatable`: a `&str`, a `String`, a `&Prompt` (which it renders first), a `&ChatMessage` or a `&[ChatMessage]`. For a `Vec`, pass a slice. For estimates closer to what OpenAI models see, enable the `tiktoken` feature flag: ```sh cargo add swiftide@0.34 --features openai,tiktoken ``` ```rust let tokenizer = TikToken::try_from_model("gpt-4o")?; let prompt = Prompt::from("Explain lifetimes in one paragraph."); let from_tokenizer = tokenizer.estimate(&prompt).await?; // The client's tokenizer is gpt-4o unless you set `tiktoken` on its builder. let from_client = openai.estimate_tokens(&prompt).await?; ``` Source: [`snippets/src/models/errors_backoff_and_usage.rs#L147-L152`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/errors_backoff_and_usage.rs#L147-L152) `TikToken` encodes the text with the Byte Pair Encoding (BPE) tokenizer of the model you name in `try_from_model`. `TikToken::default()` uses the `gpt-4o` tokenizer. With the feature on, an OpenAI client also has `estimate_tokens`, which uses the client’s `TikToken`. That is `gpt-4o` whatever the prompt model, unless you set `tiktoken` on the builder. Both return estimates, not exact counts. They skip what they cannot read as text: image, document, audio and video parts of a message, and the text of reasoning messages. Each message still adds its 4 tokens, so an image-only or reasoning message counts as 4. ## Reference [Section titled “Reference”](#reference) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) (`llm`) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`BackoffConfiguration`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.BackoffConfiguration.html) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) (`on_usage`, `on_usage_async`) * [`AnthropicBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html) (`on_usage`, `on_usage_async`) * [`AwsBedrockBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrockBuilder.html) (`on_usage`, `on_usage_async`) * [`Usage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.Usage.html) * [`NormalizedUsage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.NormalizedUsage.html) * [`CharEstimator`](https://docs.rs/swiftide/0.34/swiftide/struct.CharEstimator.html) * [`EstimateTokens`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EstimateTokens.html) * [`Estimatable`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Estimatable.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`TikToken`](https://docs.rs/swiftide/0.34/swiftide/integrations/tiktoken/struct.TikToken.html) ## Next [Section titled “Next”](#next) [Tracing](/observability/tracing/) # Images and multimodal input > Send images, and on some providers documents, audio and video, to a chat model with ChatMessage::new_user_with_parts and content parts. By the end of this page you send an image and a question to a chat model in one request and print its answer. The example uses the OpenAI client, which sits behind the `openai` feature flag: ```sh cargo add swiftide@0.34 --features openai ``` ```rust let bytes = std::fs::read("diagram.png")?; let message = ChatMessage::new_user_with_parts(vec![ ChatMessageContentPart::text("Describe this diagram in one sentence."), ChatMessageContentPart::image(ChatMessageContentSource::bytes(bytes, Some("image/png"))), ]); let request = ChatCompletionRequest::builder() .messages(vec![message]) .build()?; let response = openai.complete(&request).await?; println!("{}", response.message().unwrap_or_default()); ``` Source: [`snippets/src/models/images.rs#L14-L26`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/images.rs#L14-L26) ## Content parts [Section titled “Content parts”](#content-parts) `openai` is an `OpenAI` client built as in [Models](/models/overview/). The prompt model you set with `default_prompt_model` must accept images, for example `gpt-4o-mini`. A client without a model fails with `Model not set`. `ChatMessage::new_user_with_parts` builds a `ChatMessage::UserWithParts` message. It holds a list of content parts (`ChatMessageContentPart`) instead of one string. Here the list has a text part and an image part. The client sends them in that order. Swiftide 0.33 added `UserWithParts`. It is a breaking change: if you match on `ChatMessage`, you need an arm for the new variant. Each media part points at its data through a source (`ChatMessageContentSource`). `ChatMessageContentSource::bytes` takes the raw file and an optional media type. The OpenAI client encodes the bytes as base64 and sends them as a data Uniform Resource Locator (URL), `data:image/png;base64,...`. You do not need a base64 crate yourself. `complete` is the method of the `ChatCompletion` trait, so import `swiftide::traits::ChatCompletion`. `SimplePrompt` only takes a text prompt, so it cannot carry images. ## Image sources [Section titled “Image sources”](#image-sources) A source can be a URL, a data URL, bytes, an Amazon Web Services (AWS) Simple Storage Service (S3) object or a provider file id: ```rust let by_url = ChatMessageContentPart::image("https://example.com/chart.png"); let by_data_url = ChatMessageContentPart::image("data:image/png;base64,iVBORw0KGgo="); let by_file_id = ChatMessageContentPart::image(ChatMessageContentSource::file_id("file-abc123")); let in_s3 = ChatMessageContentPart::image(ChatMessageContentSource::s3( "s3://my-bucket/chart.png", None::, )); let raw_bytes = ChatMessageContentPart::image_with_format(vec![0x89, 0x50, 0x4e, 0x47], "png"); ``` Source: [`snippets/src/models/images.rs#L34-L42`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/images.rs#L34-L42) * A `&str` or `String` becomes a URL source. That covers both `https://` URLs and data URLs. * `ChatMessageContentSource::file_id` refers to a file you uploaded to the provider earlier. * `ChatMessageContentSource::s3` points at an object in S3. The second argument is the bucket owner. Without an owner, write `None::`, because a bare `None` leaves its type unknown and does not compile. * A `Vec` or `&[u8]` becomes a bytes source without a media type. * `ChatMessageContentPart::image_with_format` sets the format, such as `png`, explicitly. AWS Bedrock reads it first and otherwise infers the format from the media type or the file extension. The OpenAI clients ignore it. ## Other content parts [Section titled “Other content parts”](#other-content-parts) Besides text and images, a message can carry document, audio and video parts. The helpers are `ChatMessageContentPart::document`, `document_with_name`, `audio` and `video`. A document with a file name looks like this: ```rust let message = ChatMessage::new_user_with_parts(vec![ ChatMessageContentPart::text("List the action items in this report."), ChatMessageContentPart::document_with_name( ChatMessageContentSource::bytes(pdf, Some("application/pdf")), "report.pdf", ), ]); ``` Source: [`snippets/src/models/images.rs#L50-L56`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/images.rs#L50-L56) AWS Bedrock sends the name as the document name. The OpenAI Responses application programming interface (API) sends it as the file name of a bytes document. Few providers accept these parts, so check the table below before you use them. The `audio`, `document` and `video` helpers leave the format unset, and only images have a `_with_format` helper. To set a format on another part, build the variant directly: ```rust let audio = ChatMessageContentPart::Audio { source: ChatMessageContentSource::bytes(recording, Some("audio/wav")), format: Some("wav".into()), }; ``` Source: [`snippets/src/models/images.rs#L64-L67`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/images.rs#L64-L67) ## Provider support [Section titled “Provider support”](#provider-support) Every client converts content parts to its provider’s format before it sends the request. A part or source the client cannot convert fails `complete` with `LanguageModelError::PermanentError`, before anything reaches the provider. `complete_stream` returns a stream instead, and the error is its first item. | Client | Image | Document | Audio | Video | | ------------------------------------------------- | --------------------------------------- | --------------------- | --------------------- | --------------------- | | OpenAI, Chat Completions (default) | URL, data URL, bytes | no | bytes, `mp3` or `wav` | no | | OpenAI, Responses API (`use_responses_api(true)`) | URL, data URL, bytes, file id | URL, bytes, file id | no | no | | AWS Bedrock (`aws-bedrock`) | bytes, base64 data URL, S3, `s3://` URL | same sources as image | same sources as image | same sources as image | | Anthropic | no | no | no | no | * Ollama, Groq, Gemini, OpenRouter and Dashscope are [OpenAI-compatible clients](/models/overview/#providers) and share the OpenAI conversion. Whether the endpoint accepts the image depends on the provider and the model you pick. * On OpenAI Chat Completions, audio must be a bytes source in `mp3` or `wav`. Without a format it is sent as `mp3`, and any other format fails. The media type does not change this. * AWS Bedrock accepts only base64 data URLs and `s3://` URLs. It rejects every other URL, such as `https://`, `http://` or `file://`, and it rejects file ids. * AWS Bedrock needs a format for images, audio and video. Pass a media type, use a file extension in the URL, or set the format explicitly. Otherwise the call fails with `Bedrock image format is required`. * An AWS Bedrock document without a resolvable format does not fail. It is sent as `txt`, so a PDF in bytes without a media type goes out as a text document. * An AWS Bedrock document part needs a non-empty text part in the same message. Bedrock drops empty text parts, so `text("")` does not count. * The Anthropic client (`anthropic` feature flag) accepts `UserWithParts` messages with text parts only. Any other part returns an error. ## Gotchas [Section titled “Gotchas”](#gotchas) * Token estimates ignore every part except text. See [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). * `ChatMessage::as_ref` returns an empty string for a `UserWithParts` message, unless the message has exactly one text part and nothing else. Code that logs or searches message text through `as_ref` skips these messages. * The `Debug` output of a source cuts data URLs to their first 32 characters of data, and bytes sources show only their length. Debug logs of the message history stay short. * For image parts without a media type, the OpenAI clients label bytes as `application/octet-stream`. Pass the real media type to `ChatMessageContentSource::bytes`. Documents on the Responses API and audio on Chat Completions do not send the media type, so there it has no effect. ## Reference [Section titled “Reference”](#reference) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ChatMessageContentPart`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessageContentPart.html) * [`ChatMessageContentSource`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessageContentSource.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) ## Next [Section titled “Next”](#next) [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/) # Models > The model traits Swiftide uses, which providers implement each one, and how to build a client for OpenAI, Anthropic, Ollama or any OpenAI-compatible endpoint. By the end of this page you have a client for your model provider, and you know which trait each part of Swiftide expects from it. ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .default_embed_model("text-embedding-3-small") .build()?; let anthropic = Anthropic::builder() .default_prompt_model("claude-sonnet-4-5") .build()?; let ollama = Ollama::builder() .default_prompt_model("llama3.2") .default_embed_model("nomic-embed-text") .build()?; ``` Source: [`snippets/src/models/overview.rs#L19-L31`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/overview.rs#L19-L31) Every provider has a client, and every client has a builder. `default_prompt_model` sets the model for prompts and chat completions. `default_embed_model` sets the model for embeddings. `OpenAI` reads its key from `OPENAI_API_KEY`. `Anthropic` reads `ANTHROPIC_API_KEY`. Ollama runs on your machine and needs no key. OpenAI and the other OpenAI-compatible clients have no default model. A call without one fails with a permanent error that says `Model not set`. `Anthropic` falls back to `claude-3-5-sonnet-20241022`, a retired model, so always set `default_prompt_model`. Each provider sits behind its own feature flag. Turn on the ones you use: ```sh cargo add swiftide@0.34 --features openai,anthropic,ollama ``` ## Model traits [Section titled “Model traits”](#model-traits) Swiftide never calls a provider by name. Each part of it asks for a trait, and you pass any client that implements that trait. The traits live in `swiftide::traits`. Import a trait to call its method on a client. | Trait | Method | Used by | | ---------------------- | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SimplePrompt` | `prompt`: one prompt in, text out | Large Language Model (LLM) transformers such as `MetadataQAText`, the `GenerateSubquestions` query transformer, the `answers::Simple` answer generator, and `Pipeline::with_default_llm_client` | | `ChatCompletion` | `complete`: messages and tools in, a response out | agents, through `Agent::builder().llm(..)`, and task nodes in `swiftide::tasks` | | `EmbeddingModel` | `embed`: a list of texts in, dense vectors out | the `Embed` batch transformer and the `Embed` query transformer | | `SparseEmbeddingModel` | `sparse_embed`: a list of texts in, sparse vectors out | the `SparseEmbed` batch transformer and the `SparseEmbed` query transformer | | `StructuredPrompt` | `structured_prompt::`: one prompt in, a Rust value out | your own code, for typed responses | Failed calls return a `LanguageModelError`. `complete_stream` is the exception: it returns a stream, and errors arrive as items of that stream. See [Streaming](/models/overview/#streaming). ## Providers [Section titled “Providers”](#providers) | Client (feature flag) | `SimplePrompt` | `ChatCompletion` | `EmbeddingModel` | `SparseEmbeddingModel` | `StructuredPrompt` | | ------------------------------------------------------------------------------------------------------------------ | -------------- | ---------------- | ---------------- | ---------------------- | ------------------ | | `OpenAI` (`openai`) | yes | yes | yes | no | yes | | `Ollama` (`ollama`), `Groq` (`groq`), `Gemini` (`gemini`), `OpenRouter` (`open-router`), `Dashscope` (`dashscope`) | yes | yes | yes | no | yes | | `Anthropic` (`anthropic`) | yes | yes | no | no | no | | `AwsBedrock` (`aws-bedrock`) | yes | yes | no | no | yes | | `FastEmbed` (`fastembed`) | no | no | yes | yes | no | `OpenAI` is `GenericOpenAI`. `Ollama`, `Groq`, `Gemini`, `OpenRouter` and `Dashscope` are `GenericOpenAI` with their own configuration type. So the aliases implement every trait `OpenAI` does, and you can pass them anywhere a model trait is expected. They are still different types. A field typed `OpenAI` does not accept an `Ollama`. The endpoint and the model decide what works at runtime. Not every provider serves embeddings or structured output. `AwsBedrock` talks to Amazon Web Services (AWS) Bedrock through the Converse application programming interface (API), in the `aws_bedrock_v2` module. `FastEmbed` runs embedding models locally. Setup for both is on [AWS Bedrock](/integrations/aws-bedrock/) and [FastEmbed](/integrations/fastembed/). ## Clients as trait objects [Section titled “Clients as trait objects”](#clients-as-trait-objects) ```rust let prompt: Arc = Arc::new(openai.clone()); let embed: Arc = Arc::new(openai.clone()); let answer = prompt.prompt("Name one Rust web framework.".into()).await?; let vectors = embed.embed(vec![answer]).await?; let agent = Agent::builder().llm(&openai).build()?; ``` Source: [`snippets/src/models/overview.rs#L40-L46`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/overview.rs#L40-L46) Store a client as `Arc` or `Arc` when you pick the provider at runtime. Cloning a client is cheap. Every built-in client keeps its connection or model behind an `Arc`, so clones share it. `Agent::builder().llm(&openai)` takes a reference. It clones the client into a `Box`, so you can keep using `openai` afterwards. `Agent` needs the `swiftide-agents` feature flag on top of the provider flags. The `structured_prompt::` method is generic, so `StructuredPrompt` cannot be a trait object. [Structured output](/models/structured-output/) covers `DynStructuredPrompt`, the variant that can. To retry rate limits and server errors, wrap the client in `LanguageModelWithBackOff`. See [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). ## OpenAI-compatible endpoints [Section titled “OpenAI-compatible endpoints”](#openai-compatible-endpoints) A server that speaks the OpenAI API works with the `OpenAI` client. Point the client at it with `api_base` and `api_key` on the builder. ```rust let local = OpenAI::builder() .api_base("http://localhost:8000/v1") .api_key("not-needed-locally") .default_prompt_model("qwen2.5-coder") .build()?; ``` Source: [`snippets/src/models/overview.rs#L55-L59`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/overview.rs#L55-L59) With a single endpoint you can skip the code. The default `OpenAI` client reads `OPENAI_BASE_URL` and `OPENAI_API_KEY` from the environment when you build it. For a custom HTTP (Hypertext Transfer Protocol) client or another configuration type, pass an `async_openai::Client` to `client(..)` instead. Azure OpenAI needs that route with `GenericOpenAI`; [Azure OpenAI](/integrations/openai/#azure-openai) shows it. The hosted-provider clients have the base URL already set and read their key from the environment: | Client | Key variable | | ------------ | ---------------------------------------- | | `Ollama` | none | | `Groq` | `GROQ_API_KEY` | | `Gemini` | `GEMINI_API_KEY` | | `OpenRouter` | `OPENROUTER_API_KEY` | | `Dashscope` | `QWEN_API_KEY`, then `DASHSCOPE_API_KEY` | `OpenRouter` without a key does not return an error. It panics on the first request. More on these clients is on [OpenAI-compatible providers](/integrations/openai-compatible/). ## Request settings [Section titled “Request settings”](#request-settings) Each OpenAI-compatible client holds one `Options` value. Every request it sends carries those settings: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .default_embed_model("text-embedding-3-small") .default_options( Options::builder() .max_completion_tokens(2_048) .reasoning_effort(ReasoningEffort::Low) .seed(42) .metadata(serde_json::json!({ "team": "docs" })) .dimensions(512), ) .parallel_tool_calls(Some(false)) .for_end_user("user-1234") .build()?; ``` Source: [`snippets/src/models/overview.rs#L106-L119`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/overview.rs#L106-L119) `default_options` merges into the options the builder already has: a field you set replaces the earlier value. `for_end_user` and `parallel_tool_calls` are builder shortcuts for the `user` and `parallel_tool_calls` fields. Apart from `reasoning_features`, every field starts unset, and Swiftide only sends the fields you set. That includes `parallel_tool_calls`: set it to `Some(false)` to stop the model from asking for several tools in one turn. Some reasoning models reject the field, so leave it at `None` for them. | Field | Sent with | | -------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ | | `temperature`, `max_completion_tokens`, `seed`, `presence_penalty`, `metadata`, `reasoning_effort` | `complete`, `prompt` and `structured_prompt` | | `user` | `complete`, `prompt`, `structured_prompt` and `embed`; not sent on the Responses API | | `dimensions` | `embed` only; text-embedding-3 and later models | | `reasoning_features` | the Responses API only, and only when `reasoning_effort` is set | `ReasoningEffort` is re-exported from `async-openai` at `swiftide::integrations::openai`. The Responses API (`use_responses_api(true)`) ignores `seed` and `presence_penalty` and logs a warning. It accepts `metadata` only as a flat map of string values. `reasoning_features` defaults to `true` on `OpenAI`. It asks for reasoning summaries and encrypted reasoning content. Turn it off with `reasoning_features(false)`. To change a built client, clone it and edit the copy through `options_mut`. Each clone has its own `Options`: ```rust let mut creative = openai.clone(); creative.options_mut().temperature = Some(1.2); creative.options_mut().presence_penalty = Some(0.5); ``` Source: [`snippets/src/models/overview.rs#L128-L130`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/overview.rs#L128-L130) ### Extra request fields [Section titled “Extra request fields”](#extra-request-fields) Some providers accept request fields that the OpenAI API does not have. `Options` has `extra_body`, a JavaScript Object Notation (JSON) map that Swiftide merges into the top level of each request: ```rust let mut extra = serde_json::Map::new(); extra.insert("top_k".into(), serde_json::json!(20)); let ollama = Ollama::builder() .default_prompt_model("llama3.2") .default_options(Options::builder().extra_body(extra)) .build()?; ``` Source: [`snippets/src/models/overview.rs#L68-L74`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/overview.rs#L68-L74) Only `ChatCompletion` requests carry `extra_body`. `prompt`, `structured_prompt` and `embed` calls do not. So LLM transformers such as `MetadataQAText` and the `answers::Simple` answer generator send the request without the extra fields. ### AWS Bedrock settings [Section titled “AWS Bedrock settings”](#aws-bedrock-settings) `AwsBedrock` has its own `Options` in `swiftide::integrations::aws_bedrock_v2`. Set them with `AwsBedrockBuilder::default_options`, which merges the same way, or change them later with `options_mut`. | Field | Effect | | ------------------------------------------------------ | -------------------------------------------------------------------------- | | `max_tokens`, `temperature`, `top_p`, `stop_sequences` | the Converse inference configuration | | `tool_strict` | strict schema validation for tool calls; `true` when unset | | `anthropic_beta` | Anthropic beta headers, sent in the model request fields | | `reasoning_effort` | `output_config.effort` for Claude models: `Low`, `Medium`, `High` or `Max` | | `additional_model_request_fields` | any other model request fields, as an `aws_smithy_types::Document` | `reasoning_effort` here is the Bedrock module’s own `ReasoningEffort` enum, not the OpenAI one. For a model ID that contains `claude-opus-4-5`, Swiftide adds the `effort-2025-11-24` beta header itself. Behind an inference profile or Amazon Resource Name (ARN), add it to `anthropic_beta` yourself. With `anthropic_beta` or `reasoning_effort` set, `additional_model_request_fields` must be a JSON object, or the request fails with a permanent error. ## Responses API and reasoning [Section titled “Responses API and reasoning”](#responses-api-and-reasoning) `OpenAI` uses the Chat Completions API by default. `use_responses_api(true)` switches `complete`, `complete_stream`, `prompt` and `structured_prompt` to the OpenAI Responses API. For `OpenAI`, only that API returns the reasoning of a reasoning model: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .use_responses_api(true) .default_options(Options::builder().reasoning_effort(ReasoningEffort::Medium)) .build()?; let mut agent = Agent::builder().llm(&openai).build()?; agent .query("Which crate in this workspace owns the agent loop?") .await?; for message in agent.history().await? { if let ChatMessage::Reasoning(item) = message { println!("reasoning summary: {:?}", item.summary); } } ``` Source: [`snippets/src/models/overview.rs#L83-L98`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/overview.rs#L83-L98) With `reasoning_effort` set, reasoning arrives in the `reasoning` field of `ChatCompletionResponse` as `ReasoningItem` values. An agent adds each one to its message history as a `ChatMessage::Reasoning`, even when the response holds nothing else. On the next request the client sends the items back, so the model continues from its earlier reasoning. These requests set `store` to false, so the encrypted reasoning lives in your message history, not at OpenAI. The client sends back only items with `encrypted_content`. With `reasoning_features(false)` there is none, and nothing goes back. OpenAI returns summaries only to organizations verified for reasoning access. The Responses API does not send `extra_body`. `AwsBedrock` returns signed Claude reasoning as `ReasoningItem` values too, and replays them on the next turn. The `Anthropic` client leaves `ChatMessage::Reasoning` out of its requests. ## Streaming [Section titled “Streaming”](#streaming) `complete_stream` sends a chat completion request and returns a `ChatCompletionStream` instead of one response. Each item is a `Result`. You read it with `StreamExt::next` from the `futures-util` crate. Swiftide does not re-export it: ```sh cargo add futures-util@0.3 ``` ```rust let request = ChatCompletionRequest::builder() .messages(vec![ChatMessage::new_user( "Explain borrowing in Rust in three sentences.", )]) .build()?; let mut stream: ChatCompletionStream = openai.complete_stream(&request).await; let mut printed = 0; let mut last = None; while let Some(response) = stream.next().await { let response = response?; let text = response.message().unwrap_or_default(); print!("{}", &text[printed..]); printed = text.len(); last = Some(response); } if let Some(usage) = last.and_then(|response| response.usage) { println!("\n{} tokens", usage.total_tokens); } ``` Source: [`snippets/src/models/streaming.rs#L13-L33`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/streaming.rs#L13-L33) Every item is a whole `ChatCompletionResponse`, not a fragment. `message` holds all the text so far. `tool_calls` holds the tool calls rebuilt from the fragments so far. `delta` holds the latest change: `delta.message_chunk` is the new text, and `delta.tool_calls_chunk` maps each tool call index to its fragments so far. The snippet prints the part of `message` it has not printed yet, which works the same for every client. The last item is the complete response. Token usage comes in at the end of the stream, so read `usage` from the last item. A request that fails before the stream starts gives a stream with a single `Err` item. A client without its own streaming keeps the default `complete_stream`. It calls `complete` and returns a stream with that one result. An agent that streams calls `complete_stream` for you. You read its chunks in the `on_stream` hook. See [Streaming](/agents/streaming/). ### Deltas only [Section titled “Deltas only”](#deltas-only) By default every item clones the full response, which grows with the answer. The OpenAI-compatible clients can send only the delta. Turn off `stream_full` on the builder: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .stream_full(false) .build()?; let mut stream = openai.complete_stream(&request).await; let mut complete = None; while let Some(response) = stream.next().await { let response = response?; if response.message.is_some() { complete = Some(response); continue; } if let Some(chunk) = response .delta .as_ref() .and_then(|d| d.message_chunk.as_deref()) { print!("{chunk}"); } } ``` Source: [`snippets/src/models/streaming.rs#L41-L62`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/streaming.rs#L41-L62) With `stream_full(false)`, every item but the last carries only `id` and `delta`. `message`, `tool_calls` and `usage` are `None`. The last item is still the complete response, with `message`, `tool_calls` and `usage` set. The snippet uses `message` to tell the last item apart and prints `delta.message_chunk` for the others. `stream_full` defaults to `true`. It is a setting of `GenericOpenAI`, so it exists on `OpenAI`, `Ollama`, `Groq`, `Gemini`, `OpenRouter` and `Dashscope`, on the Chat Completions API and the Responses API alike. `Anthropic` and `AwsBedrock` have no such setting and always send the accumulated response. ## Reference [Section titled “Reference”](#reference) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`DynStructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.DynStructuredPrompt.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`OpenAIConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OpenAIConfig.html) * [`AzureConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.AzureConfig.html) * [`Options`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`OptionsBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OptionsBuilder.html) * [`GenericOpenAIBuilder::for_end_user`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.for_end_user) * [`GenericOpenAIBuilder::parallel_tool_calls`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.parallel_tool_calls) * [`GenericOpenAI::options_mut`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html#method.options_mut) * [`ReasoningEffort` (OpenAI)](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/enum.ReasoningEffort.html) * [`GenericOpenAIBuilder::use_responses_api`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.use_responses_api) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ReasoningItem`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ReasoningItem.html) * [`Agent::history`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.history) * [`Anthropic`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.Anthropic.html) * [`Ollama`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/type.Ollama.html) * [`Groq`](https://docs.rs/swiftide/0.34/swiftide/integrations/groq/type.Groq.html) * [`Gemini`](https://docs.rs/swiftide/0.34/swiftide/integrations/gemini/type.Gemini.html) * [`OpenRouter`](https://docs.rs/swiftide/0.34/swiftide/integrations/open_router/type.OpenRouter.html) * [`Dashscope`](https://docs.rs/swiftide/0.34/swiftide/integrations/dashscope/type.Dashscope.html) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`Options` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.Options.html) * [`ReasoningEffort` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/enum.ReasoningEffort.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder::llm`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.llm) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`ChatCompletion::complete_stream`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html#method.complete_stream) * [`ChatCompletionStream`](https://docs.rs/swiftide/0.34/swiftide/traits/type.ChatCompletionStream.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletionResponseDelta`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponseDelta.html) * [`GenericOpenAIBuilder::stream_full`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.stream_full) ## Next [Section titled “Next”](#next) [Prompts and templates](/models/prompts-and-templates/) # Prompts and templates > Build prompts with Prompt::from and Tera templates, register named templates, and replace the default prompts of transformers, answer generators and agents. By the end of this page you can fill a template with values, reuse named templates, and replace the built-in prompts that Swiftide sends to a Large Language Model (LLM). `Prompt` is always available. The snippets also use the OpenAI client, an agent and the tree-sitter transformers, and each sits behind a feature flag that is off by default: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents,tree-sitter ``` [Feature flags](/reference/feature-flags/) lists the rest. ```rust let prompt = Prompt::from("Explain {{ topic }} to a {{ audience }} in two sentences.") .with_context_value("topic", "ownership") .with_context_value("audience", "Python developer"); let rendered = prompt.render()?; let answer = openai.prompt(prompt).await?; ``` Source: [`snippets/src/models/prompts_and_templates.rs#L11-L16`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/prompts_and_templates.rs#L11-L16) A `Prompt` is the text you send to a model, plus the values to fill into it. `Prompt::from` takes a `&'static str` or a `String`. The text is a [Tera](https://keats.github.io/tera/docs/) template, so `{{ topic }}` is a placeholder. `with_context_value` adds one value to the template context. It accepts anything that converts into a `serde_json::Value`, including nested objects from `serde_json::json!`. `render` returns the final text, or an error if the template does not render. The snippet calls it only to show the result. `SimplePrompt::prompt` takes the `Prompt` by value and renders it inside the client. ## Rendering [Section titled “Rendering”](#rendering) A one-off prompt from `Prompt::from` without template context is sent verbatim. `Prompt::from("hello {{ name }}")` renders to `hello {{ name }}`, braces included. A named template always renders through Tera, so a placeholder without a value fails there. Once you set any value, Tera renders the whole text. Autoescaping is off for one-off templates, so characters such as `<` and `&` reach the model unchanged. A render failure is an `anyhow::Error` from `render`. Clients return it as a `LanguageModelError::PermanentError`, so a retry does not help. `with_context_value` consumes the prompt and returns it. When you hold a `&mut Prompt`, use `insert_context_value` instead. A struct does not convert into a `tera::Context`. Pass it through `tera::Context::from_serialize(&value)?` and hand the result to `with_context`. ## Template syntax [Section titled “Template syntax”](#template-syntax) Tera supports loops, conditions and its built-in filters in any prompt. ```rust let prompt = Prompt::from("Review these files:\n{% for file in files %}- {{ file }}\n{% endfor %}") .with_context_value("files", vec!["src/lib.rs", "src/main.rs"]); ``` Source: [`snippets/src/models/prompts_and_templates.rs#L25-L27`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/prompts_and_templates.rs#L25-L27) The Tera documentation lists the full syntax. Swiftide builds Tera without its `builtins` feature. The `date`, `slugify`, `filesizeformat`, `urlencode` and `urlencode_strict` filters are missing, as are the `now` and `get_random` functions. Enable `builtins` on your own `tera` dependency if you need them. ## Named templates [Section titled “Named templates”](#named-templates) A long prompt that you use in several places belongs in a named template. You compile it once and load it by name. ```rust let mut templates = tera::Tera::default(); templates.add_raw_template( "review", "Review this {{ language }} diff and list risks:\n{{ diff }}", )?; Prompt::extend(&templates)?; let prompt = Prompt::from_compiled_template("review") .with_context_value("language", "Rust") .with_context_value("diff", "- let x = 1;\n+ let x = 2;"); ``` Source: [`snippets/src/models/prompts_and_templates.rs#L36-L45`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/prompts_and_templates.rs#L36-L45) `Prompt::extend` merges your Tera instance into one repository that the whole process shares. Its templates, filters and functions carry over. `Prompt::from_compiled_template` stores only the name. The lookup happens when the prompt renders, so an unknown name fails at that point. Swiftide does not re-export Tera. Add it to your crate to build the instance: ```sh cargo add tera@1.20 --no-default-features ``` Tera keeps the first template it sees under a name. Calling `Prompt::extend` again with a template of the same name does not replace the registered one. Tera also autoescapes named templates whose name ends in `.html`, `.htm` or `.xml`, so give prompts another suffix. ## Transformer prompts [Section titled “Transformer prompts”](#transformer-prompts) Every indexing transformer that prompts a model has a `prompt_template` field. You replace its default prompt on the builder. ```rust let qa = MetadataQAText::builder() .client(openai) .num_questions(3_usize) .prompt_template( "Write {{ questions }} questions a support engineer would ask about:\n\ {{ node.chunk }}", ) .build()?; ``` Source: [`snippets/src/models/prompts_and_templates.rs#L54-L61`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/prompts_and_templates.rs#L54-L61) The transformer fills the template context before each call. It adds the current node as `node`, so `{{ node.chunk }}` is the chunk text. It also adds its own values. `MetadataQAText` adds `questions`, which is `num_questions` and defaults to 5. The same field exists on `MetadataSummary`, `MetadataTitle` and `MetadataKeywords`, and on `MetadataQACode` and `CompressCodeOutline` behind the `tree-sitter` feature flag. Their default templates are `.prompt.md` files in the Swiftide source. Read the default for a transformer to see which variables it uses. [Transformers and metadata](/indexing/transformers-and-metadata/) covers the transformers themselves. ## Answer prompts [Section titled “Answer prompts”](#answer-prompts) The answer generator `answers::Simple` takes a `prompt_template` too. ```rust let answer = answers::Simple::builder() .client(openai) .prompt_template(Prompt::from( "Answer {{ question }} using only:\n{{ documents }}", )) .build()?; ``` Source: [`snippets/src/models/prompts_and_templates.rs#L70-L75`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/prompts_and_templates.rs#L70-L75) Its template context has four variables: * `question`: the question the user asked. * `original`: the same as `question`. * `current`: the query text after the query transformers ran. After a response transformer such as `Summary`, it holds that transformer’s output instead. * `documents`: the retrieved documents, joined with `---` lines. After a response transformer, it holds the same text as `current`. The optional `document_template` renders each document on its own before the join, with its `content` and `metadata` in the template context. The answer generator skips it when a response transformer ran. The query transformer `GenerateSubquestions` and the response transformer `Summary` also take a `prompt_template`. [Answering](/querying/answering/) shows them in a query pipeline. ## Agent system prompts [Section titled “Agent system prompts”](#agent-system-prompts) An agent’s system prompt is a `Prompt`, so it takes template values like any other prompt. ```rust let agent = Agent::builder() .llm(&openai) .system_prompt( Prompt::from("You review {{ language }} code. Be brief.") .with_context_value("language", "Rust"), ) .build()?; ``` Source: [`snippets/src/models/prompts_and_templates.rs#L84-L90`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/prompts_and_templates.rs#L84-L90) `system_prompt` accepts anything that converts into a `Prompt`, so a plain string works too. An agent has no system prompt unless you set one. The system prompt renders once, when the agent first runs, after the `BeforeAll` hooks. A `BeforeAll` hook can call `system_prompt_mut` and `insert_context_value` to fill a value just in time. Later hooks run after the render, so a change there does not reach the system message. If rendering fails, the run returns `AgentError::FailedToRenderSystemPrompt`. [System prompts](/agents/system-prompts/) covers the agent side. ## Gotchas [Section titled “Gotchas”](#gotchas) * A borrowed `&str` that is not `'static` does not convert into a `Prompt`. Turn it into a `String` with `to_string` first. * `format!` treats `{{` and `}}` as escaped braces. `format!("Explain {{ topic }}")` yields `Explain { topic }`, which is no longer a placeholder. Write `{{{{ topic }}}}` inside `format!`. * Filters and functions registered through `Prompt::extend` work only in named templates. A one-off prompt from `Prompt::from` renders with a fresh Tera instance and does not see them. * `answers::Simple::builder().prompt_template` takes a `Prompt`, not a string. Wrap the text in `Prompt::from`. * The prompt module rustdoc mentions a Template type. It does not exist at this version, so use `Prompt`. ## Reference [Section titled “Reference”](#reference) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`CompressCodeOutline`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.CompressCodeOutline.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`response_transformers::Summary`](https://docs.rs/swiftide/0.34/swiftide/query/response_transformers/struct.Summary.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) ## Next [Section titled “Next”](#next) [Structured output](/models/structured-output/) # Structured output > Ask a model for a Rust type with structured_prompt::, where T derives schemars JsonSchema, and use DynStructuredPrompt when you need a trait object. By the end of this page, a model call returns your own Rust struct. The provider shapes its answer to the struct’s JavaScript Object Notation (JSON) Schema. Bring the types and traits into scope: ```rust use schemars::{JsonSchema, schema_for}; use serde::Deserialize; use swiftide::{ agents::{Agent, StopReason, tools::control::StopWithArgs}, integrations::openai::OpenAI, traits::{DynStructuredPrompt, StructuredPrompt as _}, }; ``` Source: [`snippets/src/models/structured_output.rs#L4-L10`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/structured_output.rs#L4-L10) Describe the answer as a type: ```rust /// What a model reads out of a changelog. #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct ReleaseSummary { /// One line that names the release theme. pub title: String, /// Changes a user notices. pub highlights: Vec, /// True when upgrading needs code changes. pub breaking: bool, } ``` Source: [`snippets/src/models/structured_output.rs#L14-L24`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/structured_output.rs#L14-L24) Then ask the model for it: ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; let summary = openai .structured_prompt::( format!("Summarise this changelog:\n{changelog}").into(), ) .await?; println!("{}: {} highlights", summary.title, summary.highlights.len()); if summary.breaking { println!("This release needs code changes."); } ``` Source: [`snippets/src/models/structured_output.rs#L30-L43`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/structured_output.rs#L30-L43) ## Schema and deserialization [Section titled “Schema and deserialization”](#schema-and-deserialization) The doc comments on the fields become descriptions in the schema, so the model reads them as instructions. `structured_prompt::` is a method of the `StructuredPrompt` trait, so the trait must be in scope. The method takes a `Prompt`, so `.into()` turns a `String` or a `&'static str` into one. Templates and template context work as on [Prompts and templates](/models/prompts-and-templates/). Swiftide builds the schema with `schemars::schema_for!(T)` and sends it with the rendered prompt. It then deserializes the JSON the model returns into `T`. When the JSON does not fit `T`, the call returns `LanguageModelError::PermanentError`. You need schemars, serde and serde_json in your own crate: ```sh cargo add swiftide@0.34 --features openai cargo add schemars@1 serde serde_json --features serde/derive ``` `structured_prompt` itself needs no extra feature flag. The `json-schema` feature only derives `JsonSchema` on some of Swiftide’s own types. ### Schemas for Swiftide types [Section titled “Schemas for Swiftide types”](#schemas-for-swiftide-types) With `json-schema`, `ToolCall`, `ToolSpec`, `ToolFeedback`, `StopReason` and the agent `State` derive `JsonSchema`. Your own types can then hold them as fields and still derive a schema, for example the request body of an approval endpoint: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents,json-schema ``` ```rust /// The body your approval endpoint accepts; the `json-schema` feature flag lets it derive /// `JsonSchema` with Swiftide types as fields. #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct ApprovalDecision { /// The call the agent stopped on, as `StopReason::FeedbackRequired` reported it. pub tool_call: swiftide::chat_completion::ToolCall, /// The person's decision. pub feedback: swiftide::traits::ToolFeedback, } /// Prints the JSON Schema of the approval body, for example for an OpenAPI document. pub fn approval_schema() -> anyhow::Result<()> { let schema = schema_for!(ApprovalDecision); println!("{}", serde_json::to_string_pretty(&schema)?); Ok(()) } ``` Source: [`snippets/src/models/structured_output.rs#L88-L104`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/structured_output.rs#L88-L104) `StopReason` and `State` live in `swiftide::agents`, so they also need the `swiftide-agents` feature flag. The `json-schema` feature adds no API of its own. ## Schema rules [Section titled “Schema rules”](#schema-rules) OpenAI clients send the schema in strict mode. This holds for Chat Completions and for the Responses application programming interface (API). Strict mode has two rules for every object in the schema. The first rule is `additionalProperties: false`. Add `#[serde(deny_unknown_fields)]` to the struct, and to every struct it contains. Schemars then writes `additionalProperties: false` for that object. Without it, OpenAI rejects the request with an error about `additionalProperties`. The second rule is that every field is listed in `required`. Schemars leaves `Option` fields and `#[serde(default)]` fields out of `required`, so OpenAI rejects them. Use a required field with a sentinel value, or an enum, instead. Swiftide passes the schemars schema through as it is. It does not rewrite the schema to fit a provider, so the provider’s rules on schemas apply to your type directly. ## Trait objects [Section titled “Trait objects”](#trait-objects) `StructuredPrompt` is generic over `T`, so it is not dyn safe. You cannot store a `Box`. Use `DynStructuredPrompt` instead: ```rust let llm: Box = Box::new(openai); let value = llm .structured_prompt_dyn( "Summarise the 0.33 release.".into(), schema_for!(ReleaseSummary), ) .await?; let summary: ReleaseSummary = serde_json::from_value(value)?; ``` Source: [`snippets/src/models/structured_output.rs#L51-L59`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/structured_output.rs#L51-L59) `structured_prompt_dyn` takes a `schemars::Schema` and returns a `serde_json::Value`. You deserialize the value yourself. Every `DynStructuredPrompt` gets `StructuredPrompt` through a blanket implementation. To add structured output to your own client, implement `DynStructuredPrompt` only. ## Provider support [Section titled “Provider support”](#provider-support) | Client | Feature flag | How the schema is sent | | ----------------------------------------------------- | ------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | `OpenAI` | `openai` | Chat Completions `response_format` with a JSON Schema. With `use_responses_api(true)`, the Responses API text format. | | `Ollama`, `Groq`, `Gemini`, `OpenRouter`, `Dashscope` | `ollama`, `groq`, `gemini`, `open-router`, `dashscope` | Same as `OpenAI`. | | `AwsBedrock` | `aws-bedrock` | Converse `output_config` with a JSON Schema. | | `Anthropic` | `anthropic` | Not supported. | | `FastEmbed` | `fastembed` | Not supported. | Whether a JSON Schema response format works on the `GenericOpenAI` aliases depends on the endpoint and the model. A structured call is not retried. [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/) explains why. ## Agents [Section titled “Agents”](#agents) An agent talks to its model through chat completion, not through `structured_prompt`. To get a typed final answer from an agent, replace the default stop tool with `StopWithArgs`. Agents live behind the `swiftide-agents` feature flag: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents ``` ```rust let mut agent = Agent::builder() .llm(&openai) .without_default_stop_tool() .tools([StopWithArgs::with_parameters_schema(schema_for!( ReleaseSummary ))]) .build()?; agent.query("Summarise the 0.33 release.").await?; let summary: Option = match agent.stop_reason() { Some(StopReason::RequestedByTool(_, Some(payload))) => { Some(serde_json::from_value(payload.clone())?) } _ => None, }; ``` Source: [`snippets/src/models/structured_output.rs#L67-L82`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/models/structured_output.rs#L67-L82) The model ends the run by calling `stop` with arguments that follow the `ReleaseSummary` schema. The stop reason is then `StopReason::RequestedByTool(_, Some(payload))`, where `payload` is the arguments as a `serde_json::Value`. `serde_json::from_value` turns it into your type. The stop tool’s schema follows different rules than `structured_prompt`. Swiftide normalizes tool schemas for OpenAI: it fills in `required` and adds `additionalProperties: false`. [Stopping and structured results](/agents/stopping-and-structured-results/) covers stop tools. Tool arguments use schemars schemas too, so the same derive works for a tool’s arguments; see [Tools](/agents/tools/). ## Reference [Section titled “Reference”](#reference) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`DynStructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.DynStructuredPrompt.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) ## Next [Section titled “Next”](#next) [Images and multimodal input](/models/images/) # Langfuse > Send agent runs, tool calls and LLM generations with their input, output and token usage to Langfuse with LangfuseLayer. Turn on the `langfuse` feature flag and add one tracing layer, and your agent runs, tool calls and OpenAI-compatible or Bedrock model calls show up in [Langfuse](https://langfuse.com/), the model calls with their input, output and token usage. ## Installation [Section titled “Installation”](#installation) ```sh cargo add swiftide@0.34 --features langfuse,openai cargo add tracing anyhow cargo add tracing-subscriber --features env-filter cargo add tokio --features macros,rt-multi-thread ``` The `langfuse` feature flag adds `swiftide::langfuse`. `LangfuseLayer::default()` reads its settings from the environment: | Variable | Also read | Default | | --------------------- | ---------------------------------- | ----------------------- | | `LANGFUSE_PUBLIC_KEY` | `LANGFUSE_INIT_PROJECT_PUBLIC_KEY` | none, required | | `LANGFUSE_SECRET_KEY` | `LANGFUSE_INIT_PROJECT_SECRET_KEY` | none, required | | `LANGFUSE_URL` | | `http://localhost:3000` | Langfuse’s own Docker setup uses the second names. ## Local Langfuse [Section titled “Local Langfuse”](#local-langfuse) Pick the keys first; the container creates its project with them: ```sh export LANGFUSE_PUBLIC_KEY=pk-lf-local LANGFUSE_SECRET_KEY=sk-lf-local OPENAI_API_KEY=sk-... ``` The release-notes-agent README starts Langfuse v2 like this; the last `-e` line adds a login user: ```sh docker network create langfuse docker run -d --name langfuse-db --network langfuse \ -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres postgres:16 docker run -d --name langfuse --network langfuse -p 3000:3000 \ -e DATABASE_URL=postgresql://postgres:postgres@langfuse-db:5432/postgres \ -e NEXTAUTH_SECRET=secret -e SALT=salt -e NEXTAUTH_URL=http://localhost:3000 \ -e LANGFUSE_INIT_ORG_ID=local -e LANGFUSE_INIT_PROJECT_ID=release-notes \ -e LANGFUSE_INIT_PROJECT_PUBLIC_KEY="$LANGFUSE_PUBLIC_KEY" \ -e LANGFUSE_INIT_PROJECT_SECRET_KEY="$LANGFUSE_SECRET_KEY" \ -e LANGFUSE_INIT_USER_EMAIL=me@example.com -e LANGFUSE_INIT_USER_PASSWORD=password \ langfuse/langfuse:2 ``` Open `http://localhost:3000`, sign in, and pick the `release-notes` project to see its traces. ## Layer setup [Section titled “Layer setup”](#layer-setup) The docs-qa showcase adds the layer when you pass `--langfuse`: examples/docs-qa/src/telemetry.rs ```rust let terminal = tracing_subscriber::fmt::layer() .compact() .with_writer(std::io::stderr) .with_filter(log_filter()); // Reads LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_URL let langfuse = send_to_langfuse.then(LangfuseLayer::default); tracing_subscriber::registry() .with(terminal) // Langfuse data is emitted at `debug`, so its layer gets its own, wider filter .with( langfuse .clone() .map(|layer| layer.with_filter(LevelFilter::DEBUG)), ) .try_init()?; ``` Source: [`examples/docs-qa/src/telemetry.rs#L30-L46`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/telemetry.rs#L30-L46) `LangfuseLayer::default()` panics on a missing or empty key, so docs-qa checks the keys first. The layer also starts a Tokio task that sends a batch every 5 seconds. Swiftide records all Langfuse data at the `debug` level, so the Langfuse layer gets its own `LevelFilter::DEBUG` while the terminal layer stays quieter. Running docs-qa also needs Qdrant and an index; its [README](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa) lists the steps. ## Target filter [Section titled “Target filter”](#target-filter) The release-notes-agent showcase adds the layer behind `--trace`, with a `Targets` filter that passes only Swiftide’s spans, shown in [Filters per layer](/observability/tracing/#filters-per-layer). Add `--trace` to any of its run commands: ```sh cargo run -p release-notes-agent -- --version 0.34.0 --trace ``` Its key check ignores the `LANGFUSE_INIT_PROJECT_*` names and lets an empty key through to the panic. Its README states that `--trace` was never run against a live Langfuse. ## Final flush [Section titled “Final flush”](#final-flush) ```rust // Keep a clone: the one inside the registry lives as long as the process. let langfuse = LangfuseLayer::default(); tracing_subscriber::registry() .with(langfuse.clone().with_filter(LevelFilter::DEBUG)) .try_init()?; let result = agent .query("Which crates changed since the last release?") .await; // Batches go out every 5 seconds; send the rest before the process exits. // A failed flush must not hide why the run failed. if let Err(error) = langfuse.flush().await { eprintln!("Could not send traces to Langfuse: {error:#}"); } result?; ``` Source: [`snippets/src/observability/langfuse.rs#L14-L30`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/observability/langfuse.rs#L14-L30) A layer installed as the global subscriber is never dropped, so nothing sends the last partial batch. Events still buffered when `main` returns are lost. Clones of `LangfuseLayer` share one buffer, so keep a clone and call `flush` on it. `flush` sends once. The layer queues events from spawned Tokio tasks, so spans that closed just before `flush` may miss it, and events requeued for retry are lost. ## Reported data [Section titled “Reported data”](#reported-data) Each span that passes the layer’s filter becomes an observation. Its name is the `otel.name` field when the span sets it at creation, such as `tool.git_diff` or `hook.BeforeTool`, and the span name otherwise. Agent steps record `otel.name` later, so they appear as `step`, with `step-N` in the metadata. Fields that start with `langfuse.` fill Langfuse’s type, model, input, output, usage and status message; the rest go into metadata. Events add their fields to their span’s observation; events outside a span are ignored. ### Agents [Section titled “Agents”](#agents) `agent.query` and `agent.query_once` get the rendered query as input. The [stop reason](/agents/stopping-and-structured-results/), as JavaScript Object Notation (JSON), becomes the output of the span current at the stop: `agent.query` or `agent.run` when the model has nothing more to say, a `step` observation when a tool such as `stop` ends the run. ### Tool calls [Section titled “Tool calls”](#tool-calls) Each tool call is a `tool.` observation with the arguments as input. The output is recorded after the `AfterTool` [hooks](/agents/hooks/) ran, so a hook that shortens it also shortens what Langfuse stores. ### Model calls [Section titled “Model calls”](#model-calls) The OpenAI-compatible clients (`OpenAI`, `Groq`, `Gemini`, `Ollama`, `OpenRouter`, `Dashscope`) and `AwsBedrock` mark prompts, completions, structured prompts and streams as GENERATION observations with model, request, response and token usage. Each needs its own feature flag: `openai`, `groq`, `gemini`, `ollama`, `open-router`, `dashscope` or `aws-bedrock`. A failed `AwsBedrock` call records the error as the status message. OpenAI-compatible embedding calls open no span. They set model, usage, and an empty input and output on the span they run in, such as `transformers.embed`. These replace any input and output the span had. `Anthropic` records no Langfuse fields, so its calls appear as plain spans. ### Pipelines [Section titled “Pipelines”](#pipelines) Indexing pipeline step spans are TRACE, so a DEBUG filter drops them; transformer and query pipeline spans are INFO and appear. [Tracing](/observability/tracing/#spans) lists them. ## Minimal example [Section titled “Minimal example”](#minimal-example) examples/langfuse.rs ```rust let fmt_layer = tracing_subscriber::fmt::layer() .compact() .with_target(false) .boxed(); let langfuse_layer = swiftide::langfuse::LangfuseLayer::default() .with_filter(LevelFilter::DEBUG) .boxed(); let registry = tracing_subscriber::registry() .with(EnvFilter::from_default_env()) .with(vec![fmt_layer, langfuse_layer]); registry.init(); ``` Source: [`examples/langfuse.rs#L28-L41`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/langfuse.rs#L28-L41) examples/langfuse.rs ```rust let openai = swiftide::integrations::openai::OpenAI::builder() .default_prompt_model("gpt-5") .build() .unwrap(); let paris = openai .prompt("What is the capital of France?".into()) .await?; println!("The capital of France is {paris}"); ``` Source: [`examples/langfuse.rs#L54-L63`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/langfuse.rs#L54-L63) Here one `EnvFilter` covers both layers, and without `RUST_LOG` nothing reaches Langfuse: ```sh RUST_LOG=swiftide=debug,langfuse=debug cargo run --example langfuse ``` `RUST_LOG=debug` would also send the HTTP client’s spans. The GENERATION observation sits under `prompt_openai`. The example never calls `flush`; add it when events do not arrive. ## Custom configuration [Section titled “Custom configuration”](#custom-configuration) ```rust let langfuse = LangfuseLayer::from_config(Configuration { base_path: "http://langfuse.internal:3000".to_string(), user_agent: Some("my-app".to_string()), basic_auth: Some((public_key, Some(secret_key))), ..Default::default() }); ``` Source: [`snippets/src/observability/langfuse.rs#L38-L43`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/observability/langfuse.rs#L38-L43) `LangfuseLayer::from_config` takes a `Configuration` instead of the environment. It starts its own batch sender. Omitted fields come from `Configuration::default()`, whose base path is `http://localhost`. `LangfuseLayer::from_batch_manager` builds a layer on a `LangfuseBatchManager` you started with `spawn`. Layers on one manager share its sender; each has its own trace. ## Gotchas [Section titled “Gotchas”](#gotchas) * **One trace per layer.** The first span creates a trace named with the current Unix timestamp, and every later span joins it. A long-running server puts all requests in one trace; wrap each request in a span of your own to group its observations; they still share the one trace. * **Retries.** `LanguageModelWithBackOff` calls the inner client again per attempt, so each attempt is its own GENERATION observation. * **Rejected events.** When Langfuse rejects single events in a batch with 429 or 5xx, they are sent at most three times. Other rejected events are dropped with the WARN event “Dropping Langfuse events that will not be retried”. When the whole request fails (wrong keys, network error, Langfuse down, a batch over 3.5 MB), the whole batch is lost: the sender logs an ERROR, or `flush` returns the error. * **Tokio.** `default`, `from_config` and every new span spawn Tokio tasks, and panic outside a runtime. ## Reference [Section titled “Reference”](#reference) * [`LangfuseLayer`](https://docs.rs/swiftide/0.34/swiftide/langfuse/struct.LangfuseLayer.html) * [`Configuration`](https://docs.rs/swiftide/0.34/swiftide/langfuse/struct.Configuration.html) * [`LangfuseBatchManager`](https://docs.rs/swiftide/0.34/swiftide/langfuse/struct.LangfuseBatchManager.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`Targets`](https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/targets/struct.Targets.html) and [`EnvFilter`](https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html) * [Feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [Metrics and pipeline statistics](/observability/metrics-and-statistics/) # Metrics and pipeline statistics > Count token usage per model with the metrics feature flag and a metrics-rs recorder, and report what an indexing run processed from its statistics or from your own counters. Your metrics backend gets a token counter per model and per label you choose, and each successful indexing run reports how many nodes it handled and how long it took. ## Usage metrics [Section titled “Usage metrics”](#usage-metrics) Add the `metrics` feature flag next to the integration you use, plus the `metrics` crate for the recorder: ```sh cargo add swiftide@0.34 --features metrics,openai cargo add metrics@0.24 ``` examples/usage_metrics.rs ```rust let metric_metadata = HashMap::from([("example".to_string(), "metadata".to_string())]); let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-4.1-nano") // Metadata will be added to every metric .metric_metadata(metric_metadata) .build()?; ``` Source: [`examples/usage_metrics.rs#L30-L36`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/usage_metrics.rs#L30-L36) With the `metrics` feature flag, a model client counts the tokens of every call whose response reports usage. `metric_metadata` takes a `HashMap`. Each entry becomes a label on every counter this client emits. Swiftide always adds a `model` label with the model of the call. Two clients with different metadata, such as one per tenant or per pipeline, show up as separate series. ### Recorder [Section titled “Recorder”](#recorder) examples/usage_metrics.rs ```rust #[derive(Clone, Debug)] struct PrintHandle(Key); impl CounterFn for PrintHandle { fn increment(&self, value: u64) { println!("counter increment for '{}': {}", self.0, value); } fn absolute(&self, value: u64) { println!("counter absolute for '{}': {}", self.0, value); } } impl GaugeFn for PrintHandle { fn increment(&self, value: f64) { println!("gauge increment for '{}': {}", self.0, value); } fn decrement(&self, value: f64) { println!("gauge decrement for '{}': {}", self.0, value); } fn set(&self, value: f64) { println!("gauge set for '{}': {}", self.0, value); } } impl HistogramFn for PrintHandle { fn record(&self, value: f64) { println!("histogram record for '{}': {}", self.0, value); } } #[derive(Debug)] struct PrintRecorder; impl Recorder for PrintRecorder { fn describe_counter(&self, key_name: KeyName, unit: Option, description: SharedString) { println!( "(counter) registered key {} with unit {:?} and description {:?}", key_name.as_str(), unit, description ); } fn describe_gauge(&self, key_name: KeyName, unit: Option, description: SharedString) { println!( "(gauge) registered key {} with unit {:?} and description {:?}", key_name.as_str(), unit, description ); } fn describe_histogram(&self, key_name: KeyName, unit: Option, description: SharedString) { println!( "(histogram) registered key {} with unit {:?} and description {:?}", key_name.as_str(), unit, description ); } fn register_counter(&self, key: &Key, _metadata: &Metadata<'_>) -> Counter { Counter::from_arc(Arc::new(PrintHandle(key.clone()))) } fn register_gauge(&self, key: &Key, _metadata: &Metadata<'_>) -> Gauge { Gauge::from_arc(Arc::new(PrintHandle(key.clone()))) } fn register_histogram(&self, key: &Key, _metadata: &Metadata<'_>) -> Histogram { Histogram::from_arc(Arc::new(PrintHandle(key.clone()))) } } fn init_print_logger() { metrics::set_global_recorder(PrintRecorder).unwrap() } ``` Source: [`examples/usage_metrics.rs#L91-L170`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/usage_metrics.rs#L91-L170) Swiftide only calls the metrics-rs facade. It never installs a recorder, so your app does that once at startup with `metrics::set_global_recorder`. Without a recorder the counters do nothing. The example’s recorder prints every call. An exporter crate such as `metrics-exporter-prometheus` installs a recorder the same way. It then serves the values to your backend. Install the recorder before the first model call. Swiftide sends the counter descriptions once per process, on the first usage it emits. A recorder installed later never receives them. ### Counters [Section titled “Counters”](#counters) | Counter | Description | | ---------------------------------- | ------------------------------ | | `swiftide.usage.prompt_tokens` | token usage for the prompt | | `swiftide.usage.completion_tokens` | token usage for the completion | | `swiftide.usage.total_tokens` | total token usage | These clients emit them: * OpenAI-compatible clients (`OpenAI`, `Groq`, `Gemini`, `Ollama`, `OpenRouter`, `Dashscope`): chat completions, streaming, `prompt`, structured prompts, embeddings and the Responses API. * `Anthropic`: prompts, completions and streaming. * `AwsBedrock`. Local embedders such as FastEmbed emit nothing. These three counters are all the `metrics` feature flag emits. It adds no pipeline or agent timing metrics. examples/usage_metrics.rs ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(10..512)) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples-metrics") .build()?, ) .run() .await?; ``` Source: [`examples/usage_metrics.rs#L40-L51`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/usage_metrics.rs#L40-L51) In this pipeline, the embedder sends one request per batch of 10 chunks, and each request adds to all three counters. Run it from a clone of the Swiftide repository, with Qdrant listening on `localhost:6334` and `OPENAI_API_KEY` set: ```sh cargo run -p swiftide-examples --example usage-metrics ``` The examples package is not a default workspace member, so `-p swiftide-examples` is required from the workspace root. That package enables many integrations, including LanceDB, Kafka, Fluvio and DuckDB, so the first build takes a while. The example keeps the output of one run in a comment at the end of `main`. These are its first lines, one per `println!`, for embeddings with `text-embedding-3-small`: ```text (counter) registered key swiftide.usage.prompt_tokens with unit None and description "token usage for the prompt" (counter) registered key swiftide.usage.completion_tokens with unit None and description "token usage for the completion" (counter) registered key swiftide.usage.total_tokens with unit None and description "total token usage" counter increment for 'Key(swiftide.usage.prompt_tokens, [example = metadata, model = text-embedding-3-small])': 356 counter increment for 'Key(swiftide.usage.completion_tokens, [example = metadata, model = text-embedding-3-small])': 0 counter increment for 'Key(swiftide.usage.total_tokens, [example = metadata, model = text-embedding-3-small])': 356 ``` Embeddings have no completion, so `completion_tokens` grows by 0. ### Metrics gotchas [Section titled “Metrics gotchas”](#metrics-gotchas) * `AnthropicBuilder::metric_metadata` takes an `Option>`, so wrap the map in `Some`. The OpenAI-compatible and `AwsBedrock` builders take the map itself. * The counters only go up. Compute rates and per-period totals in your backend. * To act on the usage of one call in your own code, set an `on_usage` callback instead. See [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). ## Pipeline statistics [Section titled “Pipeline statistics”](#pipeline-statistics) Every indexing pipeline carries a `StatsCollector`. `Pipeline::run` returns a `PipelineStats` snapshot of it at the end of the run. ```rust let pipeline = indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then_store_with(MemoryStorage::default()); let stats = pipeline.run().await?; println!( "Stored {} chunks in {:?} ({:.1} chunks/s)", stats.nodes_stored, stats.duration().unwrap_or_default(), stats.nodes_per_second().unwrap_or_default(), ); ``` Source: [`snippets/src/observability/metrics_and_statistics.rs#L40-L51`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/observability/metrics_and_statistics.rs#L40-L51) `PipelineStats` holds node counts, per-model `token_usage` and timing, with helpers such as `duration` and `nodes_per_second`. `run` also logs one INFO event at the end of a successful run, `Pipeline completed`, with the fields `nodes_processed`, `nodes_stored`, `total_tokens`, `total_requests`, `elapsed_secs` and `nodes_per_sec`. Enable INFO for the `swiftide_indexing` target to see it, for example with `RUST_LOG=swiftide_indexing=info`. [Tracing](/observability/tracing/) shows the subscriber setup. ### Watch a run in progress [Section titled “Watch a run in progress”](#watch-a-run-in-progress) `run` takes the pipeline by value. To read the numbers while it runs, clone the collector handle first. `Pipeline::stats_collector` returns the `Arc` the pipeline shares with every step. ```rust let pipeline = indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then_store_with(MemoryStorage::default()); // Take the handle before `run` consumes the pipeline let collector = Arc::clone(pipeline.stats_collector()); let running = tokio::spawn(pipeline.run()); while !running.is_finished() { println!("Stored {} chunks so far", collector.get_stats().nodes_stored); tokio::time::sleep(std::time::Duration::from_secs(5)).await; } let stats = running.await??; println!("Stored {} chunks", stats.nodes_stored); ``` Source: [`snippets/src/observability/metrics_and_statistics.rs#L59-L74`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/observability/metrics_and_statistics.rs#L59-L74) `nodes_stored` goes up as nodes reach the end of the pipeline. `nodes_processed` is set once, when the stream is done. `Pipeline::stats` returns the same snapshot as `collector.get_stats()`; before the run every count is 0. `split_by` gives each branch its own collector, seeded with the counts so far, so a handle taken before the split sees neither branch. `merge` keeps the collector of the pipeline you call it on. ### Statistics gotchas [Section titled “Statistics gotchas”](#statistics-gotchas) * `nodes_processed` and `nodes_stored` both count the nodes that reach the end of the pipeline. After a run they are the same number, even when the last step is not storage. * Nothing in Swiftide records token usage, failed nodes or transformations into the collector during a run. `total_tokens` and `total_requests` are always 0, and so are `nodes_failed` and `transformations_applied`. To record them yourself, call `record_token_usage`, `increment_nodes_failed` or `increment_transformations` on the shared handle. * The returned stats, the event and the `total_nodes` span field only exist when the run succeeds. If a node error reaches the end of the stream, `run` returns the error instead. * For token totals per pipeline, give the client of each pipeline its own `metric_metadata` label. The counters add up across all runs of the process, so take per-run numbers as differences in your backend. ## Node counters [Section titled “Node counters”](#node-counters) The ask-your-codebase showcase takes the chunk count from the stats that `run` returns, and counts files with transformers of its own. examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` Source: [`examples/ask-your-codebase/src/index.rs#L89-L115`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L89-L115) `files_found.count()` sees every source file that passes the test-file filter. `files_changed.count()` sits after `filter_cached`, so it only sees files the [node cache](/indexing/caching/) did not skip. `stats.nodes_stored` is the number of chunks that reached storage. Each `Tally` wraps an `Arc`, and the showcase measures the elapsed time with its own `Instant`, so the time includes loading the models. The same pattern as a closure [transformer](/indexing/transformers-and-metadata/): ```rust let chunks_stored = Arc::new(AtomicU64::new(0)); let count_chunk = { let chunks_stored = Arc::clone(&chunks_stored); move |node: TextNode| -> anyhow::Result { chunks_stored.fetch_add(1, Ordering::Relaxed); Ok(node) } }; indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then_store_with(MemoryStorage::default()) .then(count_chunk) .run() .await?; let chunks_stored = chunks_stored.load(Ordering::Relaxed); println!("Stored {chunks_stored} chunks"); ``` Source: [`snippets/src/observability/metrics_and_statistics.rs#L15-L32`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/observability/metrics_and_statistics.rs#L15-L32) A closure from `TextNode` to `anyhow::Result` is a transformer. This one adds 1 to a shared counter and returns the node unchanged. Steps run nodes concurrently, so the counter is an atomic behind an `Arc`, and the closure holds its own clone of it. The showcase prints these lines from its counters and the run’s stats. It indexed the `swiftide-agents` crate with `claude-haiku-5-5` through an OpenAI-compatible gateway, and FastEmbed embeddings on the CPU: ```text Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s ``` On the second run the Redb node cache skipped every file: ```text Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s ``` The docs-qa showcase takes another route: after the run, it asks Qdrant how many points the collection holds. That counts the whole collection, not only this run. The number matches the run here because the captured run used `--recreate`, which drops the collection first. With `claude-haiku-5-5` writing chunk titles and keywords, it printed: ```text Indexed 43 chunks into `docs-qa` in 71.7s ``` ## Reference [Section titled “Reference”](#reference) * [`Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) (`stats`, `stats_collector`, `run`) * [`PipelineStats`](https://docs.rs/swiftide/0.34/swiftide/indexing/statistics/struct.PipelineStats.html) * [`StatsCollector`](https://docs.rs/swiftide/0.34/swiftide/indexing/statistics/struct.StatsCollector.html) * [`ModelUsage`](https://docs.rs/swiftide/0.34/swiftide/indexing/statistics/struct.ModelUsage.html) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) (`metric_metadata`) * [`AnthropicBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html) * [`AwsBedrockBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrockBuilder.html) * [Feature flags](/reference/feature-flags/) * [metrics-rs](https://docs.rs/metrics/0.24) ## Next [Section titled “Next”](#next) [Integrations overview](/integrations/overview/) # Tracing > Log and trace Swiftide agents, tasks and pipelines with tracing-subscriber, read the logs of retried model calls, and export spans to OpenTelemetry. Here you set up a subscriber that prints Swiftide’s logs and spans, keeps failed model calls from showing up twice, and sends the spans to an OpenTelemetry (OTel) collector. ```sh cargo add tracing cargo add tracing-subscriber --features env-filter ``` ```rust /// Prints Swiftide's logs to stderr. `RUST_LOG` replaces the default filter. pub fn init_logging() -> Result<()> { let filter = EnvFilter::try_from_default_env() // Hide the clients' own WARN line for every failed model call and the HTTP client's warnings. .unwrap_or_else(|_| EnvFilter::new("info,swiftide_integrations=off,async_openai=error")); tracing_subscriber::registry() .with(fmt::layer().with_writer(std::io::stderr)) .with(filter) .try_init()?; Ok(()) } ``` Source: [`snippets/src/observability/tracing.rs#L18-L30`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/observability/tracing.rs#L18-L30) Swiftide emits spans and events through the `tracing` crate. Your app installs a subscriber once at startup to decide where they go. Without one, events fall back to the `log` facade. `EnvFilter` reads the `RUST_LOG` environment variable. When `RUST_LOG` is unset, the fallback directive applies: `info` for everything, nothing from `swiftide_integrations`, and only errors from `async_openai`. [Retry errors](#retry-errors) explains the last two. Agent, tool and hook spans need the `swiftide-agents` feature flag. Task spans need `swiftide-tasks`, which `swiftide-agents` turns on. See [Feature flags](/reference/feature-flags/). ## Filters per layer [Section titled “Filters per layer”](#filters-per-layer) A filter added with `with_filter` applies to one layer, as `logs` and `spans` do in [OpenTelemetry export](#opentelemetry-export). A filter added with `.with(filter)` on the registry applies to every layer. [Langfuse](/observability/langfuse/) uses the same split. The showcases pick their own terminal defaults: * release-notes-agent: `warn,swiftide_agents=off`. The command-line interface (CLI) already prints the stops, approvals and errors the agent logs. * ask-your-codebase: `error,swiftide_integrations=off`. * docs-qa: `warn,swiftide_query=error,async_openai=error,swiftide_integrations=off`. * workspace-architect: `off`. It reads its own variable, `WORKSPACE_ARCHITECT_LOG`, instead of `RUST_LOG`. ## Spans [Section titled “Spans”](#spans) Each span’s target is its module path, such as `swiftide_agents::agent`. OTel tools show `otel.name` instead of the span name where it is set. ### Agents [Section titled “Agents”](#agents) | Span | Level | Fields and `otel.name` | | ---------------------------------------------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | `agent.query`, `agent.query_once`, `agent.run`, `agent.run_once` | INFO | A returned error is logged at ERROR | | `step` | INFO | `step_count`; `otel.name` is `step-0`, `step-1`, … | | `add_message` | INFO | `message`: the full text of the message | | `tool` | INFO | `otel.name` is `tool.` | | `hook` | INFO, TRACE for `OnStream` | `otel.name` is `hook.`, such as `hook.BeforeTool` | | `exec_cmd_streaming` (`LocalExecutor`) | INFO | `cmd` | | `complete`, `prompt`, `complete_stream` | INFO | Model calls; OpenAI-compatible and AWS Bedrock streams add a `stream` span, the OpenAI Responses API a `responses_stream` span | Only registered [hooks](/agents/hooks/) get a `hook` span. The tools of one step run in parallel. ### Tasks [Section titled “Tasks”](#tasks) [`Task::run`](/tasks/overview/) opens `task.run` and `Task::resume` opens `task.resume`; both log a returned error. Each task node runs in an INFO span `task.step` with the fields `node` (the node index) and `branch` (the branch id). An INFO event `task.step.done` follows each one. ### Indexing pipelines [Section titled “Indexing pipelines”](#indexing-pipelines) On an [indexing pipeline](/indexing/overview/), `Pipeline::run` opens the INFO span `indexing_pipeline.run` and records `total_nodes` on it when the run succeeds. The steps `then`, `then_chunk`, `then_expand`, `then_store_with` and `filter_cached` wrap each node in a TRACE span. `then_in_batch` and `then_store_with_batched` wrap each batch. Their `otel.name` adds the step’s name, such as `then_in_batch.Embed`. `split_by` opens one TRACE span for the whole split when you build the pipeline, so it sits outside `indexing_pipeline.run`. The built-in transformers have their own INFO spans, such as `transformers.embed` and `transformers.metadata_qa_text`. ### Query pipelines [Section titled “Query pipelines”](#query-pipelines) On a [query pipeline](/querying/overview/), `query`, `query_mut` and `query_all` open `query_pipeline.query`, `query_pipeline.query_mut` and `query_pipeline.query_all`. Each step runs in an INFO span `then_transform_query`, `then_retrieve`, `then_transform_response` or `then_answer`. Each records the whole query as the field `query`. ## Trace of one agent run [Section titled “Trace of one agent run”](#trace-of-one-agent-run) The release-notes-agent streams its replies and registers `before_tool` and `after_tool` hooks. Its first step produces this span tree (reconstructed from the source, not a capture): ```text agent.query ├── step otel.name = step-0 │ ├── complete_stream │ │ └── stream │ ├── add_message message = the reply and its tool calls │ ├── hook otel.name = hook.BeforeTool │ ├── tool otel.name = tool.git_log_since │ │ └── exec_cmd_streaming cmd = the git command │ ├── hook otel.name = hook.AfterTool │ └── add_message message = the tool output ├── step otel.name = step-1 └── ... ``` ![Agent loop](/diagrams/diagram-agent-loop.svg) Each turn of the [agent loop](/agents/overview/) is one `step` span. The counter starts again at `step-0` on every `query` or `run` call. Tool calls left pending after an approval or feedback stop run before the loop, so their `tool` and `hook` spans sit directly under `agent.query` or `agent.run`. With `--responses-api`, `responses_stream` replaces `stream`. ## Parent spans [Section titled “Parent spans”](#parent-spans) Run a request inside your own span to group its work: ```rust /// Answers one question. Every span the agent opens nests under `answer_question`. pub async fn answer_question(agent: &mut Agent, user_id: &str, question: String) -> Result<()> { agent .query(question) .instrument(tracing::info_span!("answer_question", user = %user_id)) .await?; Ok(()) } ``` Source: [`snippets/src/observability/tracing.rs#L34-L42`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/observability/tracing.rs#L34-L42) Every Swiftide span the query opens becomes a child of `answer_question`. The same works for `Pipeline::run` and `Task::run`. ## Retry errors [Section titled “Retry errors”](#retry-errors) Retries are opt-in: you wrap the client in `LanguageModelWithBackOff` yourself. Each failed attempt it retries logs a WARN event, `Language model request failed, retrying`, with the attempt number, the wait and the error with its causes. When it gives up, it logs the final error once at ERROR, as `Language model request failed, giving up`. Both events have the target `swiftide_core::indexing_decorators`. The OpenAI-compatible, `Anthropic` and `AwsBedrock` clients instrument `prompt`, `complete` and, where they have it, `structured_prompt` with `err(level = "warn")`. So every failed call also logs its error at WARN under `swiftide_integrations`, with or without retries. `async_openai`, the HTTP client underneath, logs server errors at WARN. Hide the duplicates with the directives the showcases use: * `swiftide_integrations=off` drops the clients’ per-call warnings. * `async_openai=error` drops the client’s warnings. * `swiftide_agents=off` drops every agent event: returned errors, failed hooks, tool retries and `Agent loop limit reached`. `off` removes a target’s spans too. Put these directives on the terminal layer with `with_filter`, as the OpenTelemetry snippet does. Otherwise the exporter loses the model-call spans and, with `swiftide_agents=off`, every agent span. The final error still reaches your code, and the ERROR from `LanguageModelWithBackOff` stays visible. [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/) covers when a call is retried. ## Noisy events [Section titled “Noisy events”](#noisy-events) * The query pipeline logs `Sending query` and `Answered query in N seconds` at WARN; `query_all` logs `Sending queries` and `Answered all queries in N seconds`. `swiftide_query=error` hides them. * `add_message` and the `then_*` query spans carry whole messages and queries as fields. An exporter sends them with every span. * A successful indexing run ends with an INFO event `Pipeline completed`; [Metrics and statistics](/observability/metrics-and-statistics/) lists its fields. * A failed agent step logs its error at ERROR three times: in `step`, as `Agent stopped with error`, and in `agent.query`. ## OpenTelemetry export [Section titled “OpenTelemetry export”](#opentelemetry-export) Swiftide has no OTel dependency. To export spans, add a `tracing-opentelemetry` layer and an OpenTelemetry Protocol (OTLP) exporter: ```sh cargo add opentelemetry@0.32 opentelemetry_sdk@0.32 tracing-opentelemetry@0.33 cargo add opentelemetry-otlp@0.32 --no-default-features --features trace,http-proto,reqwest-blocking-client ``` ```rust /// Logs to stderr and exports spans over OTLP/HTTP (default `http://localhost:4318`, or /// `OTEL_EXPORTER_OTLP_ENDPOINT`). pub fn init_tracing() -> Result { let exporter = opentelemetry_otlp::SpanExporter::builder() .with_http() .build()?; let provider = SdkTracerProvider::builder() .with_resource(Resource::builder().with_service_name("my-app").build()) .with_batch_exporter(exporter) .build(); let logs = fmt::layer() .with_writer(std::io::stderr) .with_filter(EnvFilter::new("warn,swiftide_integrations=off")); // Swiftide's spans and your own; INFO leaves out the per-node indexing spans (TRACE). let spans = tracing_opentelemetry::layer() .with_tracer(provider.tracer("my-app")) .with_filter( Targets::new() .with_target("swiftide", LevelFilter::INFO) .with_target(env!("CARGO_CRATE_NAME"), LevelFilter::INFO), ); tracing_subscriber::registry() .with(logs) .with(spans) .try_init()?; Ok(provider) } pub async fn run_app(agent: &mut Agent) -> Result<()> { let provider = init_tracing()?; let result = answer_question( agent, "user-1", "What changed since the last release?".into(), ) .await; // Sends the spans that are still buffered. provider.shutdown()?; result } ``` Source: [`snippets/src/observability/tracing.rs#L46-L91`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/observability/tracing.rs#L46-L91) The exporter posts spans to `http://localhost:4318/v1/traces`. `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` overrides that URL as is. `OTEL_EXPORTER_OTLP_ENDPOINT` takes a base URL and gets `/v1/traces` appended. `tracing-opentelemetry` names each exported span after its `otel.name` field, so a step shows up as `step-0` and a tool call as `tool.git_log_since`. `provider.shutdown()` sends buffered spans before the process exits. ## Gotchas [Section titled “Gotchas”](#gotchas) * Indexing step spans are TRACE. Enable them only to follow single nodes or batches. * Swiftide instruments indexing and query pipeline steps, tools and hooks with `.or_current()`. The agent `step` and `task.step` spans do not use it. When such a span is filtered out, the spans inside it attach to the enclosing span. * A hook that returns an error logs `Error in hook` at ERROR, and the agent carries on. `swiftide_agents=off` hides it. * An `https` collector needs Transport Layer Security (TLS) in the exporter’s HTTP client. Enable a TLS feature of `opentelemetry-otlp`, such as `reqwest-rustls`. ## Reference [Section titled “Reference”](#reference) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`Hook`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/enum.Hook.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`Pipeline` (indexing)](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`Pipeline` (query)](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [tracing-subscriber](https://docs.rs/tracing-subscriber/0.3) * [tracing-opentelemetry](https://docs.rs/tracing-opentelemetry/0.33) * [opentelemetry-otlp](https://docs.rs/opentelemetry-otlp/0.32) * [opentelemetry_sdk](https://docs.rs/opentelemetry_sdk/0.32) ## Next [Section titled “Next”](#next) [Langfuse](/observability/langfuse/) # Generating answers > Turn retrieved documents into an answer with answers::Simple, a custom prompt and a document template that lets the answer cite the file each statement comes from. The last step of a query pipeline sends the question and the retrieved documents to a large language model (LLM). Here you set that step up so every answer names the file it came from, the way the [ask-your-codebase](/showcases/ask-your-codebase/) showcase does. `answers::Simple` needs no feature flag. The snippets below use the OpenAI client and Qdrant, which sit behind the `openai` and `qdrant` feature flags: ```sh cargo add swiftide@0.34 --features openai,qdrant ``` examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` Source: [`examples/ask-your-codebase/src/query.rs#L36-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L36-L58) ## The answer generator [Section titled “The answer generator”](#the-answer-generator) `then_answer` takes an answer generator and moves the query from `Retrieved` to `Answered`. Swiftide ships one: `answers::Simple`. It renders a prompt, sends it through any client that implements `SimplePrompt`, and stores the reply as the answer. `answers::Simple::from_client(llm)` builds it with the default prompt. The builder lets you replace two templates: * `prompt_template` is the whole prompt. * `document_template` renders each retrieved document before they are joined. It is optional. Without a custom prompt, the model receives this text, shown here with `What is love?` as the question and `My context` as the documents: ```text Answer the following question based on the context provided: What is love? ## Constraints * Do not include any information that is not in the provided context. * If the question cannot be answered by the provided context, state that it cannot be answered. * Answer the question completely and format it as markdown. ## Context --- My context --- ``` The prompt template sees four variables: * `question`: the question the user asked, from `Query::original`. * `original`: the same text as `question`. * `current`: the current query, from `Query::current`. * `documents`: the retrieved documents as one string, or the text of a response transformer that changed `current` (see below). Prompts are Tera templates; [Prompts and templates](/models/prompts-and-templates/) covers the syntax. `original` and `current` need 0.33 or later; see [Upgrading](/reference/upgrading/). ## Source citations [Section titled “Source citations”](#source-citations) The answer can only name a file if the prompt contains the file name. The document template puts it there. examples/ask-your-codebase/src/query.rs ````rust /// Widens the question into the follow-up questions its answer involves. /// /// They are phrased like the questions generated for each chunk while indexing, so the two /// embed close together. const SUBQUESTIONS_PROMPT: &str = r" Your job is to help a search find the code that answers a developer's question about a codebase. Question: {{ question }} Write {{ num_questions }} short follow-up questions that the code answering this question would also answer. Think about the mechanisms involved: the main loop, the types and enum variants, the conditions that are checked, the functions that are called, and the configuration that changes the outcome. ## Constraints * Phrase them the way a developer would ask about this codebase, e.g. 'When does the agent stop?'. * Do not guess names of types or functions. * Respond with the original question and the follow-up questions only, one per line, prefixed with '- '. "; /// Renders a retrieved chunk under its file path, so the answer can cite it. /// /// `metadata.path` is the `PATH_FIELD` metadata; a test keeps the two in step. const CHUNK_TEMPLATE: &str = r"### {{ metadata.path }} ``` {{ content }} ``` "; /// Answers from the retrieved code and cites files. const ANSWER_PROMPT: &str = r" Answer a developer's question about a codebase, using snippets of its source code. Question: {{ question }} ## Constraints * Only use the snippets below. If they do not answer the question, say briefly what is missing. * Be concrete: name the types, functions and variants involved and explain when they apply. * Describe behaviour: what calls what, in which order, and under which conditions. * Cite the file each statement comes from in backticks, e.g. (`src/lib.rs`). * Use concise markdown. Add a short code example only if it shows how to use the API. ## Snippets {{ documents }} "; ```` Source: [`examples/ask-your-codebase/src/query.rs#L65-L112`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L65-L112) `CHUNK_TEMPLATE` renders one document. Its template context is the serialized `Document`, so `content` is the chunk text and `metadata` holds the metadata fields your storage kept. ask-your-codebase writes more metadata while indexing, but its LanceDB table keeps only `path`, so `{{ metadata.path }}` becomes a heading above each chunk. `answers::Simple` joins the rendered documents with `---` lines and inserts the result as `{{ documents }}`. `ANSWER_PROMPT` then asks for a citation after each statement. With `claude-sonnet-5-5`, the answers follow it: ```text $ ./target/debug/ask-your-codebase ask --model claude-sonnet-5-5 --path swiftide-agents "How does an agent decide to stop?" # How an agent decides to stop An agent stops when something calls `Agent::stop` (or `stop_with_error`), which moves the state to `State::Stopped(StopReason)`. The main loop in `run_agent` checks that state after each step. `Agent::stop` is a no-op if the agent is already stopped (`src/agent.rs`). ## The stop mechanism - `Agent::stop(reason)` fires the `OnStop` hooks with the reason, then sets `self.state = State::Stopped(reason)`. It only takes effect at the end of the current loop iteration (`src/agent.rs`). ... Sources: - src/agent.rs - src/tools/control.rs - src/lib.rs ... ``` The `Sources:` list does not come from the model. The answered query still holds the documents it was answered from, so the showcase reads their paths back: examples/ask-your-codebase/src/query.rs ```rust impl From> for Answer { /// Keeps the distinct file paths of the retrieved chunks, most relevant first. fn from(query: Query) -> Self { let mut sources: Vec = Vec::new(); for document in query.documents() { let Some(path) = document.metadata().get(PATH_FIELD).and_then(|v| v.as_str()) else { continue; }; if !sources.iter().any(|source| source == path) { sources.push(path.to_owned()); } } Self { text: query.answer().to_owned(), sources, } } } ``` Source: [`examples/ask-your-codebase/src/query.rs#L123-L142`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L123-L142) `answer` returns the text and `documents` returns the documents the answer was built from. A response transformer can reorder or trim that list first: the [docs-qa](/showcases/docs-qa/) showcase reranks its documents, so its numbered list of page and section titles comes best first. ## Original and current question [Section titled “Original and current question”](#original-and-current-question) A query transformer such as `GenerateSubquestions` rewrites the current query. `question` in the prompt stays the user’s question, so the model answers what was asked, not the list of subquestions. Use `current` when the model should also see what the search looked for: ```rust let answerer = answers::Simple::builder() .client(openai) .prompt_template(Prompt::from( "Answer this question: {{ original }}\n\ The search looked for: {{ current }}\n\ Cite the file of every statement in backticks.\n\n\ {{ documents }}", )) .document_template("File: {{ metadata.path }}\n{{ content }}") .build()?; ``` Source: [`snippets/src/querying/answering.rs#L14-L23`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/answering.rs#L14-L23) `answers::Simple` decides what goes into `documents` like this: * If a response transformer changed `current` after retrieval, `documents` is that text and the document template is skipped. `response_transformers::Summary` works this way; see [Reranking and response transformers](/querying/reranking-and-response-transformers/). * Otherwise each document is rendered with the document template, or as its plain content without one. The check compares `current` with the query recorded by the `Retrieved` event, so a rewritten query from before retrieval does not count as a change. A reranker that only reorders documents leaves `current` alone, so the document template still applies. [Transforming queries](/querying/transforming-queries/) explains the transformation history. ## Custom answer generators [Section titled “Custom answer generators”](#custom-answer-generators) Implement the `Answer` trait to control the last step yourself. This one skips the LLM call when the retriever found nothing: ```rust /// Answers with the LLM only when the retriever found something. #[derive(Clone)] pub struct AnswerWhenFound { inner: answers::Simple, } #[async_trait] impl Answer for AnswerWhenFound { async fn answer( &self, query: Query, ) -> anyhow::Result> { if query.documents().is_empty() { return Ok(query.answered("No matching documents were found.")); } self.inner.answer(query).await } } ``` Source: [`snippets/src/querying/answering.rs#L30-L47`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/answering.rs#L30-L47) `Query::answered` stores the text and moves the query to `Answered`. The trait requires `Clone`, so derive it. Every other case goes to the wrapped `answers::Simple`. The generator goes where `answers::Simple` went: ```rust let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(AnswerWhenFound { inner: answers::Simple::from_client(openai), }); let answered = pipeline.query("How do I configure retries?").await?; println!("{}", answered.answer()); ``` Source: [`snippets/src/querying/answering.rs#L53-L61`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/answering.rs#L53-L61) `then_answer` also accepts a synchronous `Fn(Query) -> Result> + Send + Sync + Clone` closure, for example a fixed answer in a test. A closure that captures a value which is not `Clone` does not compile. ## Gotchas [Section titled “Gotchas”](#gotchas) * `prompt_template` takes a `Prompt`, so write `.into()` or `Prompt::from`. `document_template` converts for you, so a `&'static str` works directly. * A document template that reads a missing metadata key fails to render, and the query returns that error. The [docs-qa](/showcases/docs-qa/) templates guard each key with Tera’s `default` filter, as in `{{ metadata.Title | default(value="") }}`. * After answering, `current` holds the answer; see [Transforming queries](/querying/transforming-queries/) for reading the rewritten query from `history`. * The model can only cite files among the retrieved documents. A chunk outside the top results is invisible to it. Raise `top_k` for broad questions; see [Retrieval and search strategies](/querying/retrieval-and-search-strategies/). * `answers::Simple` makes one LLM call and returns its error. Wrap the client in `LanguageModelWithBackOff` to retry transient failures, as docs-qa does; see [Errors, backoff and usage](/models/errors-backoff-and-usage/). ## Reference [Section titled “Reference”](#reference) * [`query::Pipeline::then_answer`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html#method.then_answer) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`Answer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Answer.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [`Document`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Document.html) * [`query_transformers::GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`TransformationEvent`](https://docs.rs/swiftide/0.34/swiftide/query/enum.TransformationEvent.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`response_transformers::Summary`](https://docs.rs/swiftide/0.34/swiftide/query/response_transformers/struct.Summary.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) ## Next [Section titled “Next”](#next) [Evaluating with Ragas](/querying/evaluation/) # Evaluating with Ragas > Record each question's retrieved documents and answer with the Ragas evaluator, and export them with ground truths as a JSON dataset for the Python ragas toolkit. You end with a JSON file that holds every question, the answer your query pipeline gave, the documents it retrieved and the answer you expected. You can score that file and compare it between runs. The scores at the end come from the evaluation in the [docs-qa](/showcases/docs-qa/) showcase. ## Evaluation run [Section titled “Evaluation run”](#evaluation-run) `Ragas` needs no feature flag. The snippets below use the OpenAI client and Qdrant, which sit behind the `openai` and `qdrant` feature flags: ```sh cargo add swiftide@0.34 --features openai,qdrant ``` ```rust let ragas = Ragas::from_prepared_questions(vec![ ( "What does a retriever return?".to_string(), "A retriever returns documents that match the query.".to_string(), ), ( "Which step turns retrieved documents into an answer?".to_string(), "The answer generator, for example answers::Simple.".to_string(), ), ]); let pipeline = query::Pipeline::default() // Attach the evaluator while the pipeline is still pending .evaluate_with(ragas.clone()) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); pipeline.query_all(ragas.questions().await).await?; std::fs::write("ragas.json", ragas.to_json().await)?; ``` Source: [`snippets/src/querying/evaluation.rs#L19-L39`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/evaluation.rs#L19-L39) `Ragas` is an evaluator: a type that implements `EvaluateQuery`. It holds the dataset, and its clones share that dataset. Pass one clone to the pipeline with `evaluate_with` and keep the other to read the results. `ragas.questions()` returns one pending query per question in the dataset, and `query_all` runs them all. The pipeline calls the evaluator twice per question. Right after `then_retrieve`, `Ragas` stores the content of the retrieved documents as the contexts of that question. After `then_answer`, it stores the answer. `to_json` then writes the dataset. ### Evaluation in docs-qa [Section titled “Evaluation in docs-qa”](#evaluation-in-docs-qa) The [docs-qa](/showcases/docs-qa/) showcase runs the same steps on its own question set: examples/docs-qa/src/eval.rs ```rust let ragas = Ragas::from_prepared_questions(&questions); let answered = qa .evaluated_pipeline(ragas.clone()) .query_all(ragas.questions().await) .await?; let ragas_json = ragas.to_json().await; ``` Source: [`examples/docs-qa/src/eval.rs#L89-L96`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/eval.rs#L89-L96) `evaluated_pipeline` builds the same pipeline as the `ask` command and adds `evaluate_with(ragas)` to it. docs-qa also enables the `fastembed` feature flag for its hybrid search. ## Questions and ground truths [Section titled “Questions and ground truths”](#questions-and-ground-truths) `Ragas::from_prepared_questions` takes anything that converts into an `EvaluationDataSet`. An `EvaluationDataSet` converts from: * `Vec` or `&[String]`: questions only, with an empty ground truth. * `Vec<(String, String)>`: pairs of question and ground truth, as in the first snippet. * a JSON string, through `parse`, in the format that `to_json` writes. docs-qa passes `&QuestionSet`, its own YAML-backed type, which implements `From<&QuestionSet> for EvaluationDataSet`. Implement `From` for your own type in the same way. A ground truth is the answer you expect, written in your own words. Scores that compare the answer with the ground truth need it, so write one for every question you can. ### Baseline ground truths [Section titled “Baseline ground truths”](#baseline-ground-truths) Writing ground truths by hand takes time. You can let a first run write them and review them afterwards. ```rust // Runs the pipeline once over every question in the dataset let evaluate = async |ragas: &Ragas| -> anyhow::Result<()> { query::Pipeline::default() .evaluate_with(ragas.clone()) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant.clone()) .then_answer(answers::Simple::from_client(openai.clone())) .query_all(ragas.questions().await) .await?; Ok(()) }; // First run: questions only, then keep this run's answers as the ground truth let questions = vec![ "What does a retriever return?".to_string(), "Which step turns retrieved documents into an answer?".to_string(), ]; let ragas = Ragas::from_prepared_questions(questions); evaluate(&ragas).await?; ragas.record_answers_as_ground_truth().await; std::fs::write("baseline.json", ragas.to_json().await)?; // Later runs: load the reviewed baseline and evaluate against it let dataset: EvaluationDataSet = std::fs::read_to_string("baseline.json")?.parse()?; let ragas = Ragas::from_prepared_questions(dataset); evaluate(&ragas).await?; ``` Source: [`snippets/src/querying/evaluation.rs#L47-L72`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/evaluation.rs#L47-L72) The `evaluate` closure builds a pipeline with `evaluate_with(ragas.clone())` and runs `query_all` over `ragas.questions()`, so each call fills the dataset with answers and contexts. `record_answers_as_ground_truth` then copies each recorded answer into its ground truth. Read the file and fix wrong answers before you treat it as the baseline. Parsing an export keeps its ground truths. The next run overwrites the answers and contexts it loaded. ## The dataset [Section titled “The dataset”](#the-dataset) The export is a pretty-printed JSON array with one object per question. This is the output that the unit test for `to_json` expects for one question with a ground truth that has not run yet: ```json [ { "answer": "", "contexts": [], "ground_truth": "A programming language", "question": "What is Rust?" } ] ``` The keys are in alphabetical order, unless a crate in your build enables the serde_json feature that preserves insertion order. The rows are in hash map order, which changes from run to run. docs-qa sorts the rows back into the order of its questions file, so the exports of two runs can be diffed. `contexts` holds every retrieved document, recorded before any response transformer runs. docs-qa retrieves 20 candidates and reranks them down to 10, so its `ragas.json` has 20 contexts per question, while the answer generator saw 10. Ragas metrics computed from `contexts` therefore score the retriever’s 20 candidates, not the 10 documents the answer generator saw. ## Scores [Section titled “Scores”](#scores) `Ragas` computes no scores. The JSON is input for the Python [ragas](https://docs.ragas.io) toolkit, which calculates metrics such as faithfulness and context recall. docs-qa shows numbers without Python. Its judge asks a large language model (LLM) to split the ground truth and the answer into claims and to check each one, using [structured output](/models/structured-output/). Its scores are ratios modelled on the ragas metrics of the same names. In this run `claude-sonnet-5-5` both answered and judged: ```sh ./target/debug/docs-qa --model claude-sonnet-5-5 eval --judge-model claude-sonnet-5-5 \ --out examples/docs-qa/ragas.json ``` ```text # question ctx faithful answer context recall recall 1 Which crate features and environment variable do I … 10 1.00 1.00 1.00 2 What does the agents-contrib-context feature provid… 10 1.00 1.00 1.00 3 What are the default limits of the read_file tool? 10 1.00 1.00 1.00 4 Which storage and retrieval backends does Swiftide … 10 1.00 1.00 1.00 5 What should I run before opening a pull request? 10 1.00 1.00 1.00 6 What is a task in Swiftide? 10 1.00 1.00 1.00 7 When does ContextCompression summarize the conversa… 10 1.00 1.00 1.00 8 Which commit message convention does the project fo… 10 1.00 1.00 1.00 9 How do I add a custom tool to an agent with the too… 10 1.00 1.00 1.00 mean 1.00 1.00 1.00 RAGAS dataset written to examples/docs-qa/ragas.json ``` * `ctx` is the number of chunks the LLM saw after reranking. * `faithful` is the share of the answer’s claims that those chunks support. * `answer recall` is the share of the ground truth’s claims that the answer states. * `context recall` is the share of the ground truth’s claims that those chunks support. Before retrieval was tuned, two questions scored lower. One fix added page and title to the rerank template; see [Rerank templates](/querying/reranking-and-response-transformers/#rerank-templates). The other changed the indexing prompt so code-only chunks get titles that name the API they show; see the [docs-qa showcase](/showcases/docs-qa/). ## Custom evaluators [Section titled “Custom evaluators”](#custom-evaluators) Any type that implements `EvaluateQuery` can take the place of `Ragas`. The trait has one async method, `evaluate`, which receives a `QueryEvaluation`: `RetrieveDocuments` with the query after retrieval, or `AnswerQuery` with the answered query. The trait requires `Clone`, so keep shared state behind an `Arc`. This evaluator collects the questions for which the retriever found no documents: ```rust /// Flags questions for which the retriever found nothing. #[derive(Clone, Default)] pub struct EmptyRetrievals { questions: Arc>>, } #[async_trait] impl EvaluateQuery for EmptyRetrievals { async fn evaluate(&self, evaluation: QueryEvaluation) -> anyhow::Result<()> { if let QueryEvaluation::RetrieveDocuments(query) = evaluation && query.documents().is_empty() { self.questions .lock() .unwrap() .push(query.original().to_string()); } Ok(()) } } ``` Source: [`snippets/src/querying/evaluation.rs#L80-L99`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/evaluation.rs#L80-L99) ```rust let empty = EmptyRetrievals::default(); query::Pipeline::default() .evaluate_with(empty.clone()) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)) .query("What does a retriever return?") .await?; for question in empty.questions.lock().unwrap().iter() { println!("no documents for: {question}"); } ``` Source: [`snippets/src/querying/evaluation.rs#L105-L117`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/evaluation.rs#L105-L117) An error from `evaluate` fails the query, so return `Ok` for anything that should only be recorded. ## Gotchas [Section titled “Gotchas”](#gotchas) * `evaluate_with` exists only on a pending pipeline. Call it before `then_retrieve`; its position among query transformers does not matter. * `Ragas` matches each query to its row by `original()`. A query whose question is not in the dataset fails with `Question not found`, and so does `query_all`. Query transformers change `current()`, not `original()`, so they do not affect the match. * Rows are keyed by question text, so each question must be unique. docs-qa rejects a questions file with duplicates. * `query_all` returns answers in the order they finish; see [Query methods](/querying/overview/#query-methods). docs-qa matches each answer back to its question with `original()`. * Without the docs-qa workaround in [Retrieval and search strategies](/querying/retrieval-and-search-strategies/), Qdrant contexts in the export arrive `Debug`-quoted in 0.33. * Nine questions and one LLM judge make a regression signal for one corpus, not a benchmark. Keep the question set and the judge model fixed when you compare runs. ## Reference [Section titled “Reference”](#reference) * [`Ragas`](https://docs.rs/swiftide/0.34/swiftide/query/evaluators/ragas/struct.Ragas.html) * [`EvaluationDataSet`](https://docs.rs/swiftide/0.34/swiftide/query/evaluators/ragas/struct.EvaluationDataSet.html) * [`EvaluateQuery`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EvaluateQuery.html) * [`QueryEvaluation`](https://docs.rs/swiftide/0.34/swiftide/query/enum.QueryEvaluation.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [docs-qa source on GitHub](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa) ## Next [Section titled “Next”](#next) [Models](/models/overview/) # How querying works > A query pipeline takes a question from Pending to Retrieved to Answered through query transformers, a retriever, response transformers and an answer generator. You end up with a query pipeline that answers questions about your own code and names the files each answer came from. It is the query side of the [ask-your-codebase](/showcases/ask-your-codebase/) showcase, a retrieval augmented generation (RAG) command line tool. The pipeline reads the `code_chunks` LanceDB table that the showcase’s [indexing pipeline](/indexing/overview/) fills with the same embedding model, so run `ask-your-codebase index` first. The full program needs the `openai`, `fastembed` and `lancedb` feature flags. The helpers and prompt constants live in the showcase. ## The ask-your-codebase pipeline [Section titled “The ask-your-codebase pipeline”](#the-ask-your-codebase-pipeline) examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` Source: [`examples/ask-your-codebase/src/query.rs#L36-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L36-L58) The pipeline runs four steps for each question: 1. `GenerateSubquestions` asks the large language model (LLM) to rewrite the question into several related questions. Their text becomes the current query. 2. `Embed` embeds the current query with the local FastEmbed model that also embedded the chunks during indexing. 3. `then_retrieve` hands the query to the LanceDB storage, the retriever. It returns the `top_k` most similar chunks as documents. 4. `answers::Simple`, the answer generator, renders the documents into a prompt and asks the LLM for the answer. `SimilaritySingleEmbedding` is the search strategy. [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) explains the `let mut` binding and its type annotation. This is the start of a real run against the `swiftide-agents` crate. The answer came from `claude-sonnet-5-5`, and the embeddings from `jina-embeddings-v2-base-code` running locally: ```text $ ./target/debug/ask-your-codebase ask --model claude-sonnet-5-5 --path swiftide-agents "How does an agent decide to stop?" # How an agent decides to stop An agent stops when something calls `Agent::stop` (or `stop_with_error`), which moves the state to `State::Stopped(StopReason)`. The main loop in `run_agent` checks that state after each step. `Agent::stop` is a no-op if the agent is already stopped (`src/agent.rs`). ... Sources: - src/agent.rs - src/tools/control.rs - src/lib.rs - src/state.rs - src/errors.rs - src/task_agent.rs - src/hooks.rs - src/tasks.rs - src/default_context.rs ``` The sources list comes from the `path` metadata of the documents in the answered query. [Generating answers](/querying/answering/) shows the showcase’s `sources` code and the document template that puts the file path in front of each chunk. ## Query states [Section titled “Query states”](#query-states) ![Indexing and query pipelines](/diagrams/diagram-pipeline.svg) A question travels through the pipeline as a `Query`. The state is a type parameter, and it moves from `states::Pending` to `states::Retrieved` to `states::Answered`. A query keeps the original question in `original()` and the text of the latest step in `current()`. A query transformer sets `current()` to the rewritten question. A response transformer such as `Summary` can replace it with a summary, and `answers::Simple` then uses that text in place of the documents. The retriever adds documents, which you read with `documents()`. Once answered, `current()` holds the answer, and `answer()` returns it. The pipeline carries the same state, with different methods per state: | Pipeline state | Methods | | ------------------- | -------------------------------------------------------- | | `states::Pending` | `then_transform_query`, `then_retrieve`, `evaluate_with` | | `states::Retrieved` | `then_transform_response`, `then_answer` | | `states::Answered` | `query`, `query_mut`, `query_all` | The compiler enforces this order. A pipeline without `then_retrieve` has no `then_answer`, and a pipeline without `then_answer` has no `query`. ## Pipeline steps [Section titled “Pipeline steps”](#pipeline-steps) Each step method takes one trait: | Step | Trait | Called | | ------------------------- | ---------------------------------- | ------------------------------------- | | `then_transform_query` | `TransformQuery` | any number of times, before retrieval | | `then_retrieve` | `Retrieve` for the search strategy | once | | `then_transform_response` | `TransformResponse` | any number of times, after retrieval | | `then_answer` | `Answer` | once | Every step accepts a struct that implements the trait, a `Box` or `Arc` of the trait, or a closure. Closures are synchronous and must be `Clone + Send + Sync`. Each step runs every query on its own Tokio task with `tokio::spawn`, so the pipeline needs a Tokio runtime. ### Closure steps [Section titled “Closure steps”](#closure-steps) ```rust let pipeline = query::Pipeline::default() .then_transform_query(|mut query: Query| { let lowercased = query.current().to_lowercase(); query.transformed_query(lowercased); Ok(query) }) .then_retrieve( |_: &SimilaritySingleEmbedding, query: Query| { let documents = vec![Document::from( "Swiftide is a Rust library for LLM applications.", )]; Ok(query.retrieved_documents(documents)) }, ) .then_transform_response(Ok) .then_answer(|query: Query| { let answer = format!("Found {} document(s)", query.documents().len()); Ok(query.answered(answer)) }); let answered = pipeline.query("What is Swiftide?").await?; println!("{}", answered.answer()); ``` Source: [`snippets/src/querying/overview.rs#L14-L35`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/overview.rs#L14-L35) This pipeline needs no model and no storage, which makes it a starting point for tests. `retrieved_documents` and `answered` are the only ways to move a `Query` to the next state, and `transformed_query` records each rewrite in `history()`. ## Search strategy [Section titled “Search strategy”](#search-strategy) `query::Pipeline::default()` uses `SimilaritySingleEmbedding` with a `top_k` of 10 and no filter. Put an `Embed` query transformer before `then_retrieve`. The retriever searches with the embedding it sets. `query::Pipeline::from_search_strategy` starts a pipeline with any other strategy: `SimilaritySingleEmbedding` with a different `top_k` or a filter, `HybridSearch` or `CustomStrategy`. The retriever must implement `Retrieve` for that strategy. If it does not, the pipeline does not compile. [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) lists which storage supports which strategy. ## Query methods [Section titled “Query methods”](#query-methods) ```rust let mut pipeline = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant.clone()) .then_answer(answers::Simple::from_client(openai.clone())); let first = pipeline.query_mut("What is a query pipeline?").await?; let second = pipeline.query_mut("What does a retriever return?").await?; println!("{}\n\n{}", first.answer(), second.answer()); let answered = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)) .query_all(vec!["What is a node?", "What is a chunk?"]) .await?; for query in answered { println!("{}: {}", query.original(), query.answer()); } ``` Source: [`snippets/src/querying/overview.rs#L43-L61`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/overview.rs#L43-L61) `query` consumes the pipeline. `query_mut` borrows the pipeline mutably, so you can call it again for the next question. `query_all` takes a `Vec` of questions, runs them through the steps concurrently and returns every answered query. `query_all` returns results in the order they finish, not in the order you passed them. Match each result to its question with `original()`. An empty `Vec` never returns, because the pipeline waits for an answer that does not come. The first error ends the call, and the answers that already finished are lost. Every query method accepts anything that implements `AsRef`, such as `&str` or `String`. ## Gotchas [Section titled “Gotchas”](#gotchas) * The pipeline steps are lazy. No step runs until `query`, and errors from a model or a storage show up as the error of `query`. Code that builds a step can still do work: in the showcase, `embedding::local_model()` loads the FastEmbed model, and downloads it on first use, before the pipeline exists. * `query` and `query_mut` log `Answered query in N seconds` for every question, and `query_mut` also logs `Sending query`. `query_all` logs `Sending queries` and `Answered all queries in N seconds` once per batch. All are at warn level, so filter the `swiftide_query` target in your tracing subscriber if that is too noisy. * Query transformers, response transformers and answer generators must be owned and `'static`. Pass clients by value and `clone()` the ones you use in more than one step. `then_retrieve` also accepts a reference, such as `&qdrant`, and clones it. * A closure that captures a value without `Clone`, such as a lock guard, does not implement the step trait and fails with a trait-bound error. * Each step runs up to one query per logical central processing unit (CPU) at the same time. The pipeline has no setting to change that. * The source of the query pipeline marks it as being in an early stage. Expect its application programming interface (API) to change between releases, and check [Upgrading](/reference/upgrading/) when you bump Swiftide. * The query pipeline needs no feature flag. The models and retrievers do, for example `openai`, `fastembed`, `lancedb` or `qdrant`. See [Feature flags](/reference/feature-flags/). ## Reference [Section titled “Reference”](#reference) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [`states`](https://docs.rs/swiftide/0.34/swiftide/query/states/index.html) * [`Document`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Document.html) * [`TransformQuery`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformQuery.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html) * [`Answer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Answer.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html) * [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html) * [`EvaluateQuery`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EvaluateQuery.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) ## Next [Section titled “Next”](#next) [Transforming queries](/querying/transforming-queries/) # Reranking and response transformers > Reorder and trim retrieved documents with the FastEmbed Rerank cross-encoder, condense them into an LLM Summary that the answer generator uses instead, or filter them with your own TransformResponse. Your query pipeline fetches twenty candidate documents, keeps the ten a local reranker scores highest, and answers from those. You can also summarize the documents with a large language model (LLM) or filter them with your own response transformer. ## FastEmbed reranker [Section titled “FastEmbed reranker”](#fastembed-reranker) The docs-qa showcase builds its reranker next to its answer generator: examples/docs-qa/src/query.rs ```rust let reranker = Rerank::builder() .top_k(CONTEXT_CHUNKS) .document_template(Some(RERANK_TEMPLATE.to_string())) .build()?; let answerer = answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(DOCUMENT_TEMPLATE) .build()?; ``` Source: [`examples/docs-qa/src/query.rs#L73-L82`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L73-L82) It then places the reranker between retrieval and the answer: examples/docs-qa/src/query.rs ```rust pipeline // Hybrid search needs both a dense and a sparse embedding of the question .then_transform_query(query_transformers::Embed::from_client( self.embedders.dense.clone(), )) .then_transform_query(query_transformers::SparseEmbed::from_client( self.embedders.sparse.clone(), )) .then_retrieve(self.qdrant.clone()) // A local cross-encoder reorders the fused candidates; only the best few remain .then_transform_response(self.reranker.clone()) .then_answer(self.answerer.clone()) ``` Source: [`examples/docs-qa/src/query.rs#L121-L134`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L121-L134) `Rerank` is a response transformer. A response transformer runs after `then_retrieve` and before `then_answer`. It takes the query in the `Retrieved` state and returns it in the same state. `Rerank` runs a cross-encoder: a model that reads the question and one document together and returns a relevance score. That is slower than comparing embeddings, so you run it only on the candidates the retriever already found. The builder has four fields: * `top_k`: how many documents to keep, as a `usize`. The default is 10. * `document_template`: an optional Tera template that renders each document before scoring. The setter takes an `Option`, so pass `Some(..)`. * `model`: the FastEmbed `TextRerank` model, wrapped in `Arc>`. The default is BGE reranker base. * `model_batch_size`: an optional batch size. The default is FastEmbed’s own. `Rerank` scores every document against `query.original()`, the question as the user asked it. Query transformers such as `GenerateSubquestions` replace `current()`, which the retriever searches with and which `answers::Simple` also sees. The reranker ignores `current()`. It replaces the query’s documents with the `top_k` highest-scoring ones, best first. `Rerank` lives in `swiftide::integrations::fastembed` behind the `fastembed` feature flag: ```sh cargo add swiftide@0.34 --features fastembed ``` This is a docs-qa answer, written by `claude-sonnet-5-5` and reranked locally with BGE reranker base. The sources list the pages and sections of the kept documents, best first: ```text Swiftide supports OpenAI and Azure OpenAI, Anthropic, Gemini, OpenRouter, AWS Bedrock Converse API, Groq, Ollama, and Dashscope. Integrations are feature-gated, so you enable a provider by adding its Cargo feature to `swiftide`. ... Sources: [1] Swiftide › Swiftide capabilities for agents, typed task graphs, pipelines, and tracing (README.md) [2] Swiftide › Swiftide: Rust framework for LLM agents, task graphs, and RAG (README.md) [3] Swiftide › Swiftide Integrations Overview by Feature-Gated Category (README.md) ... [9] Swiftide › Swiftide agent tool defined with the swiftide::tool macro (README.md) [10] Swiftide › Installing Swiftide and Setting Up OpenAI Integration (README.md) ``` The `openai` feature flag in that answer comes from the tenth source, the Quick Start chunk. With a `top_k` below 10 it would not have reached the model. Check a few answers before you lower `top_k`. ### Rerank templates [Section titled “Rerank templates”](#rerank-templates) examples/docs-qa/src/query.rs ```rust // The answer prompt receives the question and the rendered documents; each document is rendered // with its page title and the chunk title written during indexing. const ANSWER_PROMPT: &str = "You answer questions about Swiftide, a Rust library for building \ LLM applications, using excerpts from its documentation. Question: {{ question }} ## Instructions * Use only the documentation excerpts below. If they do not answer the question, say so plainly. * Start with the direct answer in one or two sentences, then add supporting details. * Only when the question asks how to do something, include the feature flags, commands or a \ short Rust snippet from the excerpts. Never invent APIs. * Format the answer as markdown and keep it under 250 words. Do not refer to \"the excerpts\". ## Documentation excerpts {{ documents }} "; const DOCUMENT_TEMPLATE: &str = "### {{ metadata.Page | default(value=\"\") }}: \ {{ metadata.Title | default(value=\"\") }} {{ content }}"; // The reranker scores the same title-first rendering. A chunk that is mostly code says little // about what it shows, so its page and the title written during indexing go first. const RERANK_TEMPLATE: &str = "{{ metadata.Page | default(value=\"\") }}: \ {{ metadata.Title | default(value=\"\") }} {{ content }}"; ``` Source: [`examples/docs-qa/src/query.rs#L30-L57`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L30-L57) `RERANK_TEMPLATE` puts the page name and the chunk title, both written during indexing, before the content. A chunk that is mostly code says little about its topic, so the extra context helps it score fairly. The template only changes what the model scores. The documents stay as they were, and the answer generator renders them with its own `document_template`. In docs-qa this template fixed one evaluation result. For “What is a task in Swiftide?”, the chunk that lists what a task supports ranked 14th when the reranker scored only chunk text, so it was dropped. The answer missed two features, with answer recall 0.71 and context recall 0.57. With page and title in the template, the chunk made the kept ten and both scores reached 1.00. [Evaluating with Ragas](/querying/evaluation/#scores) explains these scores. Model download, cache location and load failures are covered in [FastEmbed](/integrations/fastembed/). ### Minimal pipeline [Section titled “Minimal pipeline”](#minimal-pipeline) The single-file example builds its reranker in its setup: ```rust let openai_client = integrations::openai::OpenAI::builder() .default_prompt_model("gpt-4o") .build()?; let fastembed = fastembed::FastEmbed::builder().batch_size(10).build()?; let reranker = fastembed::Rerank::builder().top_k(5).build()?; let qdrant = Qdrant::builder() .batch_size(50) .vector_size(384) .collection_name("swiftide-reranking") .build()?; ``` Source: [`examples/reranking.rs#L25-L36`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs#L25-L36) It then generates subquestions for retrieval, while the reranker still scores against the original question: ```rust // By default the search strategy is SimilaritySingleEmbedding // which takes the latest query, embeds it, and does a similarity search let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( openai_client.clone(), )) .then_transform_query(query_transformers::Embed::from_client(fastembed.clone())) .then_retrieve(qdrant.clone()) .then_transform_response(reranker) .then_answer(answers::Simple::from_client(openai_client.clone())); ``` Source: [`examples/reranking.rs#L49-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs#L49-L58) ## Summaries [Section titled “Summaries”](#summaries) ```rust // By default the search strategy is SimilaritySingleEmbedding // which takes the latest query, embeds it, and does a similarity search let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( openai_client.clone(), )) .then_transform_query(query_transformers::Embed::from_client( openai_client.clone(), )) .then_retrieve(qdrant.clone()) .then_transform_response(response_transformers::Summary::from_client( openai_client.clone(), )) .then_answer(answers::Simple::from_client(openai_client.clone())); ``` Source: [`examples/query_pipeline.rs#L39-L52`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/query_pipeline.rs#L39-L52) `response_transformers::Summary` sends the retrieved documents to an LLM and asks for one summary of them. It stores the reply with `transformed_response`, which records a transformation event and replaces `query.current()`. It leaves `documents()` untouched. `answers::Simple` then answers from the summary instead of the rendered documents; see [Generating answers](/querying/answering/). The documents stay on the query, so you can still list sources. To change the instruction, pass your own `prompt_template`: ```rust let summary = response_transformers::Summary::builder() .client(openai) .prompt_template(Prompt::from( "Summarize these excerpts as one markdown list. Keep every type name, \ feature flag and command exactly as written.\n\ {% for document in documents %}\n\ ---\n\ {{ document.metadata.path | default(value=\"unknown\") }}:\n\ {{ document.content }}\n\ {% endfor %}", )) .build()?; ``` Source: [`snippets/src/querying/reranking.rs#L75-L86`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/reranking.rs#L75-L86) The template sees only `documents`, a list of documents with `content` and `metadata`. The question is not in its context. You can chain response transformers. Put `Rerank` before `Summary` and the LLM summarizes only the kept documents. ## Custom response transformers [Section titled “Custom response transformers”](#custom-response-transformers) ```rust /// Keeps the first document of each file, in retrieval order, for at most `max` files. #[derive(Clone)] pub struct OnePerFile { pub max: usize, } #[async_trait] impl TransformResponse for OnePerFile { async fn transform_response( &self, mut query: Query, ) -> Result> { let mut seen_paths = HashSet::new(); let documents = query.documents_mut(); documents.retain(|document| { let path = document .metadata() .get("path") .and_then(|value| value.as_str()) .unwrap_or_default(); seen_paths.insert(path.to_string()) }); documents.truncate(self.max); Ok(query) } } ``` Source: [`snippets/src/querying/reranking.rs#L15-L42`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/reranking.rs#L15-L42) `TransformResponse` has one async method, `transform_response`. Implement it with `#[async_trait]`, from the `async-trait` crate or `swiftide::reexports::async_trait`. The trait requires `Clone` through `DynClone`, so derive it. This transformer edits the list in place through `documents_mut`. It keeps the first document per `path` metadata value, in the order the documents arrive, and caps the list at `max`. Documents without a `path` share one empty key, so only the first of them is kept. It does not call `transformed_response`, so `answers::Simple` still renders the documents one by one. Call `transformed_response` only when your transformer produces text for the answer to use in place of the documents, as `Summary` does. Place it after `then_retrieve`: ```rust let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_transform_response(OnePerFile { max: 8 }) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/querying/reranking.rs#L48-L52`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/reranking.rs#L48-L52) For a small change, a closure works too. It must be synchronous, take a `Query` and return `anyhow::Result>`. It must also be `Clone + Send + Sync`, so capture only values that are themselves `Clone`: ```rust let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_transform_response(|mut query: Query| { query.documents_mut().truncate(5); Ok(query) }) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/querying/reranking.rs#L60-L67`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/reranking.rs#L60-L67) ## Gotchas [Section titled “Gotchas”](#gotchas) * In 0.33 the plain `Qdrant` retriever returns `Debug`-quoted content, so `Rerank` scores the quoted text. See the gotcha in [Retrieval and search strategies](/querying/retrieval-and-search-strategies/#gotchas) for the docs-qa workaround. * `Rerank` changes neither `current()` nor the transformation history. The `Retrieved` events in `history()` hold copies of the documents as the retriever returned them. Comparing them with `documents()` shows what was dropped only if no wrapper edited the documents after retrieval and the query retrieved once. * The `Ragas` evaluator records contexts before `Rerank` runs, so its dataset holds the retrieved candidates, not the kept documents; see [The dataset](/querying/evaluation/#the-dataset). A custom evaluator can read the reranked documents at its second call, after the answer. * To use another rerank model, build a `TextRerank` yourself, wrap it in `Arc::new(tokio::sync::Mutex::new(..))` and pass it to `model`. That needs direct `fastembed` and `tokio` dependencies, because Swiftide does not re-export `fastembed`. * Clones of a `Rerank` share one model behind a Tokio mutex. Scoring is synchronous CPU work, so concurrent queries, such as those from `query_all`, take turns at the rerank step. * If retrieval returns `top_k` documents or fewer, `Rerank` only reorders them. Retrieve more candidates than you keep. ## Reference [Section titled “Reference”](#reference) * [`Rerank`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.Rerank.html) * [`RerankBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.RerankBuilder.html) * [`response_transformers::Summary`](https://docs.rs/swiftide/0.34/swiftide/query/response_transformers/struct.Summary.html) * [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [`Document`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Document.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * The [docs-qa showcase](/showcases/docs-qa/) and the [FastEmbed integration](/integrations/fastembed/) ## Next [Section titled “Next”](#next) [Generating answers](/querying/answering/) # Retrieval and search strategies > Pick a search strategy (similarity, hybrid or custom), set top_k and filters, and pass it to a Qdrant, PgVector, LanceDB or DuckDB storage that acts as the retriever. A search strategy decides how many documents come back and how a retriever finds them. You pick a strategy, set `top_k` and a filter, and hand it to your storage, which acts as the retriever: Qdrant, PgVector, LanceDB or DuckDB. ## Similarity search [Section titled “Similarity search”](#similarity-search) examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` Source: [`examples/ask-your-codebase/src/query.rs#L36-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L36-L58) `SimilaritySingleEmbedding` searches with the dense embedding that the `Embed` query transformer put on the query. It returns `top_k` documents. The default is 10. ask-your-codebase searches a LanceDB table and sets `top_k` from its `--top-k` flag, which defaults to 20. `with_top_k` takes the strategy by value and returns it, so it chains onto `default()` or `from_filter`. `SimilaritySingleEmbedding::default()` exists only for the unfiltered strategy, so its filter type is inferred as `()`, even on stores that also retrieve with a typed filter. For a filtered search, start from `from_filter` or call `with_filter`, as in [Filters](#filters). ## Hybrid search [Section titled “Hybrid search”](#hybrid-search) examples/docs-qa/src/query.rs ```rust query::Pipeline::from_search_strategy( HybridSearch::default() .with_top_n(CANDIDATES) .with_top_k(CANDIDATES), ) ``` Source: [`examples/docs-qa/src/query.rs#L108-L112`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L108-L112) `HybridSearch` combines a keyword or sparse search with a dense search. `top_n` caps the candidates of each search. `top_k` caps the documents that come back. Both default to 10. docs-qa fetches 20 candidates and later reranks them down to 10. `with_top_n`, `with_top_k`, `with_dense_vector_field` and `with_sparse_vector_field` take the strategy by value and return it, so they chain without `.to_owned()`. `HybridSearch` implements `Default` for every filter type. When nothing else fixes the filter type, name it, as docs-qa does with `HybridSearch` in its return type, or the compiler stops with E0283. On Qdrant, the query needs a dense and a sparse embedding, so the pipeline runs both query transformers before it retrieves. The example creates two FastEmbed models in its setup: `fastembed` for dense and `fastembed_sparse` for sparse embeddings. ```rust // Use sophisticated model for our query let openai = openai::OpenAI::builder() .default_prompt_model("gpt-4o") .build() .unwrap(); let query_pipeline = query::Pipeline::from_search_strategy( // Return a large amount of documents because we have a large context window // By default it uses the Combined fields, no need to configure HybridSearch::default().with_top_n(20).with_top_k(20), ) // Generate subquestions on the initial query to increase our query coverage .then_transform_query(query_transformers::GenerateSubquestions::from_client( openai.clone(), )) // Generate the same embeddings we used for indexing .then_transform_query(query_transformers::Embed::from_client(fastembed.clone())) .then_transform_query(query_transformers::SparseEmbed::from_client( fastembed_sparse.clone(), )) .then_retrieve(qdrant.clone()) // Answer with Simple, which either takes the documents as is (in this case), or any // transformations applied after querying .then_answer(answers::Simple::from_client(openai.clone())); ``` Source: [`examples/hybrid_search.rs#L65-L88`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs#L65-L88) FastEmbed is the only sparse embedding model in Swiftide, so Qdrant hybrid search needs the `qdrant` and `fastembed` feature flags. The chunks need sparse embeddings too. Run the `SparseEmbed` transformer in the indexing pipeline next to `Embed`: ```rust indexing::Pipeline::from_loader(FileLoader::new("swiftide-core/").with_extensions(&["rs"])) // Chunk fairly large as the context window is big .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) // Generate metadata on the code chunks to increase our chances of finding the right code .then(MetadataQACode::from_client(openai.clone()).build().unwrap()) .then_in_batch( transformers::SparseEmbed::new(fastembed_sparse.clone()).with_batch_size(batch_size), ) .then_in_batch(transformers::Embed::new(fastembed.clone()).with_batch_size(batch_size)) .then_store_with(qdrant.clone()) .run() .await?; ``` Source: [`examples/hybrid_search.rs#L47-L61`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs#L47-L61) Qdrant runs a sparse and a dense search, each limited to `top_n`. It merges them with reciprocal rank fusion (RRF) and keeps `top_k`. The collection needs a sparse vector next to the dense one; see [Qdrant](/integrations/qdrant/). DuckDB runs hybrid search without a sparse embedding. It scores the chunk text with BM25 full-text search over `query.current()` and the dense embedding with cosine similarity. Each score is normalized to its maximum, and the final score is 0.8 times the dense score plus 0.2 times the full-text score. After `GenerateSubquestions`, the keyword text is the whole list of generated questions. ## Filters [Section titled “Filters”](#filters) A filter narrows the search to documents with matching metadata. Each storage takes its own filter type. Qdrant takes a `Filter` from its client, which Swiftide re-exports at `swiftide::integrations::qdrant::qdrant_client`: ```rust let only_readme = Filter::must([Condition::matches("path", "README.md".to_string())]); let pipeline = query::Pipeline::from_search_strategy(SimilaritySingleEmbedding::from_filter( only_readme.clone(), )) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); // The same filter on a hybrid search applies to the dense and the sparse search let _hybrid = HybridSearch::from_filter(only_readme).with_top_n(20); ``` Source: [`snippets/src/querying/retrieval.rs#L24-L34`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/retrieval.rs#L24-L34) PgVector and LanceDB take a `String`. PgVector expects `key = value` and compares it with the metadata column `meta_`, so the key must be a metadata field on the table. Swiftide does not put the node path into metadata. To filter on `path`, insert it into each node’s metadata while you index (see [Transformers and metadata](/indexing/transformers-and-metadata/)), as ask-your-codebase does. Without it, PgVector refuses to store the node with “Missing metadata field”. ```rust // `path` must be a metadata field on the table: `PgVector::builder().with_metadata("path")`. // Swiftide does not add it for you; insert `path` into each node's metadata while indexing. let search = SimilaritySingleEmbedding::from_filter("path = README.md".to_string()).with_top_k(5); let pipeline = query::Pipeline::from_search_strategy(search) .then_transform_query(query_transformers::Embed::from_client(openai.clone())) .then_retrieve(pgvector) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/querying/retrieval.rs#L44-L52`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/retrieval.rs#L44-L52) LanceDB passes the string to its `only_if` method, which takes a SQL predicate on the table’s columns, such as `path = 'README.md'`. That column has a value only if you put `path` in the node metadata and configured it with `with_metadata("path")`. DuckDB has no filter. ## Custom strategies [Section titled “Custom strategies”](#custom-strategies) `CustomStrategy` hands the query to your closure, and the closure builds the query the storage runs. `from_query` takes a synchronous closure, `from_async_query` an async one. PgVector runs a `sqlx::QueryBuilder`, LanceDB a LanceDB query, and DuckDB a SQL string. ```rust use sqlx::{Postgres, QueryBuilder}; // The default table is `swiftide_pgv_store`; `meta_path` exists only with `.with_metadata("path")` let table = pgvector.get_table_name().to_owned(); let keyword_search = CustomStrategy::from_query(move |query: &Query| { // The retriever reads `id`, `chunk` and every `meta_` column from each row let mut sql = QueryBuilder::::new(format!("SELECT id, chunk, meta_path FROM {table}")); sql.push(" WHERE to_tsvector('english', chunk) @@ plainto_tsquery('english', ") .push_bind(query.current().to_string()) .push(") LIMIT 10"); Ok(sql) }); let pipeline = query::Pipeline::from_search_strategy(keyword_search) .then_retrieve(pgvector) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/querying/retrieval.rs#L62-L79`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/retrieval.rs#L62-L79) PgVector maps each row by column name. Select `id`, `chunk` and the `meta_` columns you need. `get_table_name()` returns the table, `swiftide_pgv_store` unless you set another. Swiftide does not re-export sqlx, so add it yourself: ```sh cargo add sqlx@0.9 --no-default-features --features postgres ``` A LanceDB custom strategy returns a query that implements `lancedb::query::ExecutableQuery`. Swiftide does not re-export lancedb either. Enable the `lancedb` feature flag and add the crate at the version Swiftide uses: ```sh cargo add lancedb@0.33 ``` DuckDB reads the first three columns of each row by position: the id, the chunk and the path. ## Retrievers per storage [Section titled “Retrievers per storage”](#retrievers-per-storage) | Storage (feature flag) | Strategies | Filter | Limits | | ----------------------- | --------------------------------------------------------------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------- | | `Qdrant` (`qdrant`) | `SimilaritySingleEmbedding`, `HybridSearch` | Qdrant `Filter` | Similarity search uses the combined vector when the collection has more than one vector or a sparse vector | | `PgVector` (`pgvector`) | `SimilaritySingleEmbedding`, `CustomStrategy>` | `String`, `key = value` | One vector field; orders by cosine distance | | `LanceDB` (`lancedb`) | `SimilaritySingleEmbedding`, `CustomStrategy` | `String`, SQL predicate | Exactly one vector field; content comes from the `chunk` column | | `Duckdb` (`duckdb`) | `SimilaritySingleEmbedding`, `HybridSearch`, `CustomStrategy` | none | Uses one vector, picked arbitrarily when several are configured; documents carry `id` and `path` metadata only | Setup for each storage is on [Qdrant](/integrations/qdrant/), [PgVector](/integrations/pgvector/) and [LanceDB](/integrations/lancedb/). DuckDB has no integration page; see the `Duckdb` reference below. ## Custom retrievers [Section titled “Custom retrievers”](#custom-retrievers) A retriever implements `Retrieve` for a search strategy `S`. It receives the strategy and the pending query, and returns `query.retrieved_documents(documents)`. `retrieved_documents` adds the documents and records a retrieval in the query’s transformation history. The trait requires `Clone`. ```rust /// Keeps the documents that share a word with the question. No store, no embeddings. #[derive(Clone)] struct KeywordRetriever { documents: Vec, } #[async_trait] impl Retrieve for KeywordRetriever { async fn retrieve( &self, search_strategy: &SimilaritySingleEmbedding, query: Query, ) -> anyhow::Result> { let words: Vec = query .current() .split_whitespace() .map(str::to_lowercase) .collect(); let matches = self .documents .iter() .filter(|document| { let content = document.content().to_lowercase(); words.iter().any(|word| content.contains(word.as_str())) }) .take(usize::try_from(search_strategy.top_k())?) .cloned() .collect(); Ok(query.retrieved_documents(matches)) } } ``` Source: [`snippets/src/querying/retrieval.rs#L87-L119`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/retrieval.rs#L87-L119) ```rust let retriever = KeywordRetriever { documents: vec![ Document::from("Swiftide indexes data with an indexing pipeline."), Document::from("A query pipeline answers questions over indexed data."), ], }; let pipeline = query::Pipeline::default() .then_retrieve(retriever) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/querying/retrieval.rs#L125-L134`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/retrieval.rs#L125-L134) A closure works too: `then_retrieve` accepts any `Fn(&S, Query) -> Result>` that is `Send + Sync + Clone`. The closure is synchronous. ## Gotchas [Section titled “Gotchas”](#gotchas) * Retrieval fails when the query has no embedding. Qdrant and LanceDB report “No embedding for query”, PgVector and DuckDB “Missing embedding in query state”. Qdrant hybrid search also needs a sparse embedding (“No sparse embedding for query”). * `HybridSearchBuilder::default()` sets `top_k` and `top_n` to 0. Start from `HybridSearch::default()`, which sets both to 10. * Qdrant only implements hybrid search for `HybridSearch`. Write that type in signatures, as docs-qa does, even without a filter. * Qdrant returns each document’s content `Debug`-quoted: wrapped in `"` with escaped newlines. docs-qa wraps the retriever in `CleanQdrant` ([Qdrant content workaround](/showcases/docs-qa/#qdrant-content-workaround)), which restores the text before reranking and answering. * DuckDB hybrid search does not rank before it cuts. Each search takes the first `top_n` rows of the table, unranked, and only rows in both lists get scored. On a table larger than `top_n`, most chunks are never considered. Set `top_n` to at least the row count, or use `SimilaritySingleEmbedding`. * A small `top_k` hides relevant chunks. In ask-your-codebase, one answer had to infer a detail because the chunk that held it was not among the 20 retrieved. Raise `top_k` for broad questions, or retrieve more and [rerank](/querying/reranking-and-response-transformers/). ## Reference [Section titled “Reference”](#reference) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html), [`HybridSearchBuilder`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearchBuilder.html) * [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`SearchStrategy`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SearchStrategy.html) * [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html), [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html), [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html), [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html), [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) * Showcases: [ask-your-codebase](/showcases/ask-your-codebase/), [docs-qa](/showcases/docs-qa/) ## Next [Section titled “Next”](#next) [Reranking and response transformers](/querying/reranking-and-response-transformers/) # Transforming queries > Rewrite a question into subquestions with GenerateSubquestions, embed it densely and sparsely with the model your chunks were embedded with, or write your own TransformQuery. A query pipeline can rewrite the user’s question before the retriever runs: a large language model (LLM) widens it into related questions, and an embedding model turns the result into the vectors your chunks were stored with. The query transformers need no feature flag. The snippets use the OpenAI client and Qdrant, which sit behind the `openai` and `qdrant` feature flags: ```sh cargo add swiftide@0.34 --features openai,qdrant cargo add async-trait anyhow ``` The second line is for the custom query transformer further down, which uses the `#[async_trait]` attribute and `anyhow::Result`. ## Subquestions [Section titled “Subquestions”](#subquestions) ```rust let pipeline = query::Pipeline::default() .then_transform_query( GenerateSubquestions::builder() .client(openai.clone()) .num_questions(3) .build()?, ) // Embed the text that the step above produced, with the model the chunks were embedded with .then_transform_query(Embed::from_client(embedder)) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/querying/transforming_queries.rs#L21-L31`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/transforming_queries.rs#L21-L31) A query transformer runs before retrieval. It takes the pending query and returns a new pending query. You add one with `then_transform_query`, as often as you need, and they run in the order you add them. `GenerateSubquestions` sends the current query to an LLM through `SimplePrompt`. It asks for `num_questions` related questions, 5 by default. The reply replaces the current query, so the next step sees one text: the original question and its subquestions as a list. `GenerateSubquestions::from_client` builds it with the default prompt and count. The default prompt is written for code. It asks the model to think about the dependencies, usage and structure of the code. Replace it when you search other content. ### Subquestion prompts [Section titled “Subquestion prompts”](#subquestion-prompts) The ask-your-codebase showcase sets its own prompt on the builder: examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` Source: [`examples/ask-your-codebase/src/query.rs#L36-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L36-L58) `SUBQUESTIONS_PROMPT` is a constant next to the pipeline: examples/ask-your-codebase/src/query.rs ````rust /// Widens the question into the follow-up questions its answer involves. /// /// They are phrased like the questions generated for each chunk while indexing, so the two /// embed close together. const SUBQUESTIONS_PROMPT: &str = r" Your job is to help a search find the code that answers a developer's question about a codebase. Question: {{ question }} Write {{ num_questions }} short follow-up questions that the code answering this question would also answer. Think about the mechanisms involved: the main loop, the types and enum variants, the conditions that are checked, the functions that are called, and the configuration that changes the outcome. ## Constraints * Phrase them the way a developer would ask about this codebase, e.g. 'When does the agent stop?'. * Do not guess names of types or functions. * Respond with the original question and the follow-up questions only, one per line, prefixed with '- '. "; /// Renders a retrieved chunk under its file path, so the answer can cite it. /// /// `metadata.path` is the `PATH_FIELD` metadata; a test keeps the two in step. const CHUNK_TEMPLATE: &str = r"### {{ metadata.path }} ``` {{ content }} ``` "; /// Answers from the retrieved code and cites files. const ANSWER_PROMPT: &str = r" Answer a developer's question about a codebase, using snippets of its source code. Question: {{ question }} ## Constraints * Only use the snippets below. If they do not answer the question, say briefly what is missing. * Be concrete: name the types, functions and variants involved and explain when they apply. * Describe behaviour: what calls what, in which order, and under which conditions. * Cite the file each statement comes from in backticks, e.g. (`src/lib.rs`). * Use concise markdown. Add a short code example only if it shows how to use the API. ## Snippets {{ documents }} "; ```` Source: [`examples/ask-your-codebase/src/query.rs#L65-L112`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L65-L112) The field takes a `Prompt`, so convert a `&'static str` or `String` with `.into()` or `Prompt::from`. A borrowed `&str` built at runtime needs `.to_string()` first. The template receives two values: `question`, the current query, and `num_questions`. The prompt phrases the subquestions like the questions that its indexing pipeline wrote for each chunk. Similar phrasing puts the query vector close to the chunk vectors. Keep the instruction to repeat the original question: the next step embeds only what the model returns. [Prompts and templates](/models/prompts-and-templates/) covers the template syntax. ## Dense and sparse embeddings [Section titled “Dense and sparse embeddings”](#dense-and-sparse-embeddings) examples/docs-qa/src/query.rs ```rust pipeline // Hybrid search needs both a dense and a sparse embedding of the question .then_transform_query(query_transformers::Embed::from_client( self.embedders.dense.clone(), )) .then_transform_query(query_transformers::SparseEmbed::from_client( self.embedders.sparse.clone(), )) .then_retrieve(self.qdrant.clone()) // A local cross-encoder reorders the fused candidates; only the best few remain .then_transform_response(self.reranker.clone()) .then_answer(self.answerer.clone()) ``` Source: [`examples/docs-qa/src/query.rs#L121-L134`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L121-L134) `query_transformers::Embed` embeds the current query with an `EmbeddingModel` and stores the vector in `query.embedding`. `query_transformers::SparseEmbed` does the same with a `SparseEmbeddingModel` and fills `query.sparse_embedding`. At this release only `FastEmbed`, behind the `fastembed` feature flag, implements `SparseEmbeddingModel`. Use the models that your [indexing pipeline embedded the chunks with](/indexing/embedding/). Vectors from different models do not compare. The docs-qa showcase creates its dense and sparse `FastEmbed` models together, and loads the same two models for its indexing and its query pipeline. Similarity search needs `embedding`. Hybrid search on Qdrant needs both, and fails with `No sparse embedding for query` when the sparse one is missing. [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) covers both strategies. An embedding step reads the current query at the moment it runs. Put it after every step that rewrites the text. If `Embed` runs before `GenerateSubquestions`, the vector search never sees the subquestions. ## Custom query transformers [Section titled “Custom query transformers”](#custom-query-transformers) ```rust /// Puts the instruction that BGE embedding models expect for search queries in front of the /// question. #[derive(Clone)] pub struct BgeQueryInstruction; #[async_trait] impl TransformQuery for BgeQueryInstruction { async fn transform_query( &self, mut query: Query, ) -> anyhow::Result> { let instructed = format!( "Represent this sentence for searching relevant passages: {}", query.current() ); query.transformed_query(instructed); Ok(query) } } ``` Source: [`snippets/src/querying/transforming_queries.rs#L53-L72`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/transforming_queries.rs#L53-L72) `TransformQuery` has one async method, `transform_query`. It takes the `Query` by value and returns it. Implement it with the `#[async_trait]` attribute from the `async-trait` crate. The trait requires `Send`, `Sync` and `DynClone`, so derive `Clone`. The current query has no setter of its own. `transformed_query` replaces it and records a `Transformed` event with the text before and after. The original question never changes. The fields `embedding`, `sparse_embedding` and `documents` are public, so a transformer that computes a vector sets it directly, as `Embed` does. This example prepares a question for an embedding model of the BGE (BAAI General Embedding) family, which suggests an instruction in front of short search queries. The chunks are embedded without it. ```rust let pipeline = query::Pipeline::default() // A closure is enough for a step without state .then_transform_query(|mut query: Query| { let trimmed = query.current().trim().to_string(); query.transformed_query(trimmed); Ok(query) }) .then_transform_query(BgeQueryInstruction) .then_transform_query(Embed::from_client(embedder)) .then_retrieve(qdrant) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/querying/transforming_queries.rs#L82-L92`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/transforming_queries.rs#L82-L92) A closure is a query transformer too. It must be `Fn(Query) -> Result>` and `Clone`, `Send` and `Sync`. Closures are synchronous, so use a struct for a step that awaits. The pipeline logs `Transformed query` at debug level after each step, with the transformer’s `name()` and the new current query. `name()` defaults to the last path segment of the type name, so a closure logs a generic closure name instead of a useful one. Override `name()` on a struct to get a clearer log line. ## Transformed query after retrieval [Section titled “Transformed query after retrieval”](#transformed-query-after-retrieval) ```rust let answered = pipeline.query("How does an agent decide to stop?").await?; for event in answered.history() { if let Some(documents) = event.documents() { println!( "retrieved {} documents for:\n{}", documents.len(), event.before() ); } else { println!("rewrote:\n{}\ninto:\n{}", event.before(), event.after()); } } ``` Source: [`snippets/src/querying/transforming_queries.rs#L35-L47`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/querying/transforming_queries.rs#L35-L47) Since 0.33 the current query keeps the transformed text after retrieval. The `Retrieved` event records the current query at the moment of retrieval, as both `before()` and `after()`, and the documents it returned. Before 0.33, retrieval cleared the current query. That text is not always what the search used. A vector retriever such as Qdrant searches with `query.embedding`, which `Embed` computed when it ran. A step after `Embed` that rewrites the text changes the event, not the vector. `history()` lists the events in order. Each `transformed_query` call adds a `Transformed` event, and each retrieval adds a `Retrieved` event. `Embed` and `SparseEmbed` add nothing, because they leave the text alone. Once the query is answered, `current()` returns the answer. Read the transformed query from the history instead. [Generating answers](/querying/answering/) shows how the answer generator uses the original and the transformed query. ## Gotchas [Section titled “Gotchas”](#gotchas) * Two types are called `Embed`. `query_transformers::Embed` belongs in a query pipeline, `indexing::transformers::Embed` in an indexing pipeline. The same goes for `SparseEmbed`. Import them by module path. * After `GenerateSubquestions`, `Embed` turns the whole list into one vector. The pipeline does not run one search per subquestion. * DuckDB hybrid search, behind the `duckdb` feature flag, uses the subquestion list as its keyword text; see [Retrieval and search strategies](/querying/retrieval-and-search-strategies/). * The FastEmbed [`Rerank` response transformer](/querying/reranking-and-response-transformers/) scores documents against `original()`, and the [`Ragas` evaluator](/querying/evaluation/) matches rows by `original()`. Rewriting the question changes neither. * `Query::new` sets only the original question and leaves the current query empty, so an embedding step would embed an empty string. Pass a `&str` or `String` to `query`, or build the query with `Query::from`, which sets both. ## Reference [Section titled “Reference”](#reference) * [`Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) (`then_transform_query`) * [`TransformQuery`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformQuery.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`query_transformers::SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.SparseEmbed.html) * [`Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [`TransformationEvent`](https://docs.rs/swiftide/0.34/swiftide/query/enum.TransformationEvent.html) * [`states::Pending`](https://docs.rs/swiftide/0.34/swiftide/query/states/struct.Pending.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * Showcases: [ask-your-codebase](/showcases/ask-your-codebase/) and [docs-qa](/showcases/docs-qa/) ## Next [Section titled “Next”](#next) [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) # Changelog > Release-by-release list of Swiftide features, fixes and breaking changes, up to the version these docs describe. This page lists the features, fixes and breaking changes of every Swiftide release up to 0.34, the version these docs describe. One changelog covers every crate in the workspace; see [Versioning](/reference/crates/#versioning). The list below is the repository’s `CHANGELOG.md` at the commit these docs pin. That commit can be ahead of the release. Entries marked **breaking** change the public API. For the code changes each breaking release needs, see [Upgrading](/reference/upgrading/). Next: [Upgrading](/reference/upgrading/) # Crates > The crates in the Swiftide workspace, what each one contains, and when to depend on one directly instead of the swiftide facade. Swiftide is a Cargo workspace: a set of crates that the `swiftide` facade crate re-exports as modules. By the end of this page you know which crate holds which part of the API, which `swiftide::` path reaches it, and which docs.rs page to open. ```rust use swiftide::{ agents::Agent, // swiftide-agents indexing::{self, loaders::FileLoader}, // swiftide-indexing integrations::openai::OpenAI, // swiftide-integrations prompt::Prompt, // swiftide-core query::{self, answers}, // swiftide-query tasks::{NodeError, Task}, // swiftide-tasks traits::{AgentContext, ToolExecutor}, // swiftide-core }; ``` Source: [`snippets/src/reference/crates.rs#L4-L12`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/crates.rs#L4-L12) Your application depends on the `swiftide` crate, and you switch parts on with feature flags. This excerpt needs the `swiftide-agents` and `openai` flags: ```sh cargo add swiftide@0.34 --features swiftide-agents,openai ``` ## The facade [Section titled “The facade”](#the-facade) The `swiftide` crate has almost no code of its own. It maps the other crates onto these modules: * `swiftide::indexing` joins the indexing types of swiftide-core (`Node`, `TextNode`, the indexing traits) with swiftide-indexing. With the `tree-sitter` flag, `swiftide::indexing::transformers` also holds the code transformers. * `swiftide::query` joins the query types of swiftide-core (the query state, the search strategies) with swiftide-query. * `swiftide::traits` collects the traits of swiftide-core in one place: `AgentContext`, `MessageHistory`, the chat completion traits, the indexing and query traits, `ToolExecutor` and the command types. * `swiftide::prompt` and `swiftide::chat_completion` come straight from swiftide-core. * `swiftide::integrations` is swiftide-integrations. Each provider or storage module inside it exists only with its feature flag. * `swiftide::agents` needs the `swiftide-agents` flag, `swiftide::tasks` the `swiftide-tasks` flag, and `swiftide::langfuse` the `langfuse` flag. * `swiftide::agents_contrib` needs the `agents-contrib` flag, and each of its `tools`, `hooks` and `executors` submodules needs a capability flag on top. * The `#[tool]` attribute, the `Tool` derive and `#[indexing_transformer]` sit at the crate root, behind the `macros` flag. The facade always depends on swiftide-core, swiftide-indexing, swiftide-query and swiftide-integrations. Agents, tasks, Langfuse and the macros are optional dependencies. The contrib crate is optional too. [Feature flags](/reference/feature-flags/) lists every flag and what it turns on. ## Workspace crates [Section titled “Workspace crates”](#workspace-crates) | Crate | Contains | Facade path | API docs | | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | swiftide-core | The traits the other crates implement, `Node` and `TextNode`, `Prompt`, chat completion types, `ToolExecutor`, `CommandOutput`, search strategies | `swiftide::traits`, `swiftide::prompt`, `swiftide::chat_completion`; `Node` and `TextNode` through `swiftide::indexing`; search strategies through `swiftide::query::search_strategies` | [swiftide_core](https://docs.rs/swiftide-core/0.34/swiftide_core/) | | swiftide-indexing | The indexing pipeline, loaders, transformers, chunkers, `MemoryStorage` | `swiftide::indexing` | [swiftide_indexing](https://docs.rs/swiftide-indexing/0.34/swiftide_indexing/) | | swiftide-query | The query pipeline, query transformers, response transformers, answer generators, evaluators | `swiftide::query` | [swiftide_query](https://docs.rs/swiftide-query/0.34/swiftide_query/) | | swiftide-integrations | One module per provider or storage, each behind its feature flag | `swiftide::integrations` | [swiftide_integrations](https://docs.rs/swiftide-integrations/0.34/swiftide_integrations/) | | swiftide-agents | `Agent`, `DefaultContext`, hooks, `StopReason`, `LocalExecutor`, Model Context Protocol (MCP) tools (with the `mcp` flag), `TaskAgent` | `swiftide::agents` | [swiftide_agents](https://docs.rs/swiftide-agents/0.34/swiftide_agents/) | | swiftide-tasks | `Task`, `TaskNode`, `Transition`, fan-out and join | `swiftide::tasks` | [swiftide::tasks](https://docs.rs/swiftide/0.34/swiftide/tasks/index.html) | | swiftide-langfuse | `LangfuseLayer`, a tracing layer that reports agent and completion calls to Langfuse | `swiftide::langfuse` | [swiftide_langfuse](https://docs.rs/swiftide-langfuse/0.34/swiftide_langfuse/) | | swiftide-macros | `#[tool]`, the `Tool` derive, `#[indexing_transformer]` | crate root | [swiftide_macros](https://docs.rs/swiftide-macros/0.34/swiftide_macros/) | | swiftide-agents-contrib | `ReadFile`, `ExactReplace`, `ApplyPatch`, `UpdatePlan`, `ContextCompression`, `ToolSummarizer`, `LazyExecutor` | `swiftide::agents_contrib` | [source on GitHub](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib) | swiftide-tasks is new in 0.33. It depends on swiftide-core only. swiftide-agents depends on swiftide-tasks and swiftide-indexing, so the `swiftide-agents` flag also turns on `swiftide-tasks`. The old `swiftide::agents::tasks` module still compiles, but it is deprecated since 0.32.1 and only re-exports the new crate plus `TaskAgent`. swiftide-agents-contrib enables nothing by default, and each capability has its own facade flag. [Feature flags](/reference/feature-flags/#agents-tasks-and-tool-protocols) lists them. swiftide-test-utils also lives in the workspace. It is internal (`publish = false`) and not on crates.io. ## Direct dependencies [Section titled “Direct dependencies”](#direct-dependencies) Applications depend on the facade. A direct dependency on a workspace crate pays off in two cases: * You write a library that only implements Swiftide traits, such as a loader, a transformer, a tool or a tool executor. swiftide-core defines every one of those traits and keeps your dependency tree small. * You write an integration for a provider or storage that Swiftide does not ship. ```sh cargo add swiftide-core@0.34 ``` ### Matching versions [Section titled “Matching versions”](#matching-versions) All crates share one workspace version, 0.34, and each one requires its swiftide dependencies at 0.33. Keep every swiftide crate in your dependency tree on the same minor version. Cargo treats 0.32 and 0.33 as incompatible, so a mix builds two copies of swiftide-core, and a trait from one copy does not accept a type from the other. ### Macros and the facade [Section titled “Macros and the facade”](#macros-and-the-facade) `#[tool]` and the `Tool` derive expand to paths in the `swiftide` crate, such as `swiftide::chat_completion::ToolSpec`. A crate that uses them needs the `swiftide` crate with its `macros` flag, even when the rest of its code imports from swiftide-core. `#[indexing_transformer]` works the other way round. It expands to paths in `swiftide_core`, `anyhow` and `derive_builder`, not `swiftide`. A crate that uses it needs those three crates as direct dependencies, next to the facade: ```sh cargo add swiftide-core@0.34 anyhow derive_builder ``` ### Tasks without agents [Section titled “Tasks without agents”](#tasks-without-agents) The `swiftide-tasks` flag adds `swiftide::tasks` without compiling swiftide-agents: ```sh cargo add swiftide@0.34 --features swiftide-tasks ``` The `mcp`, `langfuse` and `json-schema` flags and every `agents-contrib` flag turn on swiftide-agents. Enable one of them and agents compile again. [Flag relationships](/reference/feature-flags/#flag-relationships) shows which flag turns on which. ## Versioning [Section titled “Versioning”](#versioning) Only the `swiftide` crate gets a git tag (`v0.34`) and a GitHub release, and its changelog covers every crate. Before you move to a new minor version, read [Upgrading](/reference/upgrading/). ## Reference [Section titled “Reference”](#reference) * [`swiftide`](https://docs.rs/swiftide/0.34/swiftide/) * [`swiftide_core`](https://docs.rs/swiftide-core/0.34/swiftide_core/) * [`swiftide_indexing`](https://docs.rs/swiftide-indexing/0.34/swiftide_indexing/) * [`swiftide_query`](https://docs.rs/swiftide-query/0.34/swiftide_query/) * [`swiftide_integrations`](https://docs.rs/swiftide-integrations/0.34/swiftide_integrations/) * [`swiftide_agents`](https://docs.rs/swiftide-agents/0.34/swiftide_agents/) * [`swiftide::tasks`](https://docs.rs/swiftide/0.34/swiftide/tasks/index.html) * [`swiftide_langfuse`](https://docs.rs/swiftide-langfuse/0.34/swiftide_langfuse/) * [`swiftide_macros`](https://docs.rs/swiftide-macros/0.34/swiftide_macros/) * [swiftide-agents-contrib source](https://github.com/bosun-ai/swiftide/tree/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/swiftide-agents-contrib) Next: [Changelog](/reference/changelog/) # Feature flags > Every Cargo feature flag of the swiftide crate, what each one turns on, and which ones the all bundle leaves out. By the end of this page you know which feature flags to pass to `cargo add` for your project. Each integration, and everything beyond the indexing pipeline and the query pipeline, sits behind a feature flag of the `swiftide` crate. You name the ones you need: ```sh cargo add swiftide@0.34 --features openai,qdrant ``` ## Defaults [Section titled “Defaults”](#defaults) Only the `macros` flag is on by default. It provides the `#[tool]` attribute, the `Tool` derive and the `#[indexing_transformer]` attribute. With `default-features = false` these macros are gone, so add `macros` back if you use them: ```sh cargo add swiftide@0.34 --no-default-features --features macros,openai ``` On docs.rs, each item that needs a flag shows an “Available on crate feature … only” badge. The core, indexing, query and integrations crates are always compiled. The integrations crate still compiles no provider or store until you turn on its flag. Without `qdrant`, for example, `swiftide::integrations::qdrant` does not exist. ## The all bundle [Section titled “The all bundle”](#the-all-bundle) The `all` flag turns on exactly these ten flags: `qdrant`, `redis`, `tree-sitter`, `openai`, `fastembed`, `scraping`, `aws-bedrock`, `groq`, `ollama` and `pgvector`. It leaves out everything else: * agents and tasks: `swiftide-agents`, `swiftide-tasks`, `mcp`, and `agents-contrib` with its four sub-flags * the `anthropic` (Anthropic client), `gemini`, `dashscope` and `open-router` providers * the `lancedb` and `duckdb` storage backends, the `redb` node cache, and the `parquet`, `kafka` and `fluvio` loaders * `langfuse`, `metrics`, `json-schema`, `tiktoken`, `test-utils` and `tree-sitter-perl` To build an agent, add `swiftide-agents` next to `all`. `all` also pulls in heavy native dependencies. `fastembed` downloads ONNX (Open Neural Network Exchange) runtime binaries at build time, and `aws-bedrock` compiles the AWS (Amazon Web Services) SDK (software development kit). Name the flags you use instead, so builds stay short. ## Flag relationships [Section titled “Flag relationships”](#flag-relationships) ### Agents, tasks and tool protocols [Section titled “Agents, tasks and tool protocols”](#agents-tasks-and-tool-protocols) * `swiftide-tasks` adds `swiftide::tasks`, the typed task graphs. * `swiftide-agents` adds `swiftide::agents` and turns on `swiftide-tasks`. * `mcp` adds Model Context Protocol (MCP) tool support and turns on `swiftide-agents`. * `langfuse` adds `swiftide::langfuse` for tracing and also turns on `swiftide-agents`. * `json-schema` adds `JsonSchema` derives to core and agent types, and also turns on `swiftide-agents`. * `agents-contrib` adds `swiftide::agents_contrib` and turns on `swiftide-agents`. On its own it enables no capability. * `agents-contrib-context`, `agents-contrib-planning`, `agents-contrib-file-tools` and `agents-contrib-lazy-executor` each turn on `agents-contrib` plus one capability: hooks that summarize tool output and compress the conversation, a plan tool, file tools, or a lazy tool executor. A contrib flag never registers a tool or changes an agent by itself. You still add the tool or hook to your agent. ### OpenAI-compatible providers [Section titled “OpenAI-compatible providers”](#openai-compatible-providers) The `groq`, `gemini`, `ollama`, `open-router` and `dashscope` clients build on the OpenAI client. Each of these flags also compiles `swiftide::integrations::openai`. ## TLS [Section titled “TLS”](#tls) Integrations that make HTTP (Hypertext Transfer Protocol) calls use rustls for TLS (Transport Layer Security) and need no setup, whatever flags you pick. Each HTTP client uses the process-wide rustls crypto provider if your application installed one, and otherwise its own: aws-lc-rs, or ring for `anthropic`. `anthropic` uses rustls even when another crate turns on native TLS. Redis over `rediss://` is the exception. The Redis client needs a process-wide provider, so before it connects Swiftide installs aws-lc-rs if your application has not installed one. To use a different provider, install it before the first Redis call. In the 0.34 release the clients have no provider of their own; see [Crypto provider panic](/reference/troubleshooting-and-faq/#crypto-provider-panic). ## Flag tables [Section titled “Flag tables”](#flag-tables) The “Also enables” column lists only flags of the `swiftide` crate. A flag that turns on a dependency’s feature, such as `json-schema` or `langfuse` turning on `swiftide-agents`, shows an empty cell. ### General [Section titled “General”](#general) | Flag | Description | Also enables | | --------- | ---------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | `default` | By default only macros are enabled | `macros` | | `macros` | Undocumented | | | `all` | Undocumented | `qdrant`, `redis`, `tree-sitter`, `openai`, `fastembed`, `scraping`, `aws-bedrock`, `groq`, `ollama`, `pgvector` | ### Integrations [Section titled “Integrations”](#integrations) | Flag | Description | Also enables | | ------------------ | ------------------------------------------------------------------ | ------------- | | `qdrant` | Enables Qdrant for storage and retrieval | | | `pgvector` | Enables PgVector for storage and retrieval | | | `redis` | Enables Redis as an indexing cache and storage | | | `tree-sitter` | Tree-sitter for various code transformers | | | `tree-sitter-perl` | Adds Perl to the tree-sitter languages | `tree-sitter` | | `openai` | OpenAI | | | `groq` | Groq | | | `gemini` | Google Gemini | | | `dashscope` | Dashscope prompting | | | `open-router` | OpenRouter prompting | | | `ollama` | Ollama prompting | | | `anthropic` | Undocumented | | | `fastembed` | FastEmbed (by qdrant) for fast, local, sparse and dense embeddings | | | `scraping` | Scraping via spider as loader and a html to markdown transformer | | | `aws-bedrock` | AWS Bedrock for prompting | | | `lancedb` | Lancdb for persistance and querying | | | `fluvio` | Fluvio loader | | | `kafka` | Kafka loader | | | `parquet` | Parquet loader | | | `redb` | Redb embeddable nodecache | | | `duckdb` | Duckdb; sqlite fork, support Persist, Retrieve and NodeCache | | ### Other [Section titled “Other”](#other) | Flag | Description | Also enables | | ------------- | ------------------------------------------------- | ----------------- | | `mcp` | MCP tool support for agents (tools only) | `swiftide-agents` | | `metrics` | Metrics for usage, pipeline and agent performance | | | `test-utils` | Various mocking and testing utilities | | | `json-schema` | Json schema for various types | | | `tiktoken` | Estimate token counts using tiktoken | | ### Agent and task orchestration [Section titled “Agent and task orchestration”](#agent-and-task-orchestration) | Flag | Description | Also enables | | ------------------------------ | ---------------------------------------------------------------------------- | ----------------- | | `swiftide-tasks` | Typed task graphs | | | `swiftide-agents` | GenAI agents and tools | `swiftide-tasks` | | `agents-contrib` | Optional agent tools, hooks, and executor helpers (no capabilities enabled). | `swiftide-agents` | | `agents-contrib-context` | Tool output summarization and conversation compression hooks. | `agents-contrib` | | `agents-contrib-planning` | A shared in-memory plan tool. | `agents-contrib` | | `agents-contrib-file-tools` | File reading with line ranges, exact replacement, and file-oriented patches. | `agents-contrib` | | `agents-contrib-lazy-executor` | Lazy initialization of caller-provided executors. | `agents-contrib` | | `langfuse` | Langfuse tracing and observability | | ## Reference [Section titled “Reference”](#reference) * [`swiftide` crate root, Cargo Features section](https://docs.rs/swiftide/0.34/swiftide/index.html#cargo-features) * [`swiftide::agents`](https://docs.rs/swiftide/0.34/swiftide/agents/index.html) * [`swiftide::tasks`](https://docs.rs/swiftide/0.34/swiftide/tasks/index.html) * [`swiftide::integrations`](https://docs.rs/swiftide/0.34/swiftide/integrations/index.html) * [`swiftide::langfuse`](https://docs.rs/swiftide/0.34/swiftide/langfuse/index.html) * [`swiftide::agents_contrib`](https://docs.rs/swiftide/0.34/swiftide/agents_contrib/index.html) Next: [Crates](/reference/crates/) # Glossary > One-line definitions of the core terms used across the Swiftide docs, each linked to the page that explains it. You get a short definition of each core term in the Swiftide docs, grouped by section and linked to the page that explains it. ## Agents [Section titled “Agents”](#agents) * **[Agent](/agents/overview/)**: a loop around a large language model (LLM). Each turn, it sends the message history to the model and runs the tool calls it returns. Build one with `Agent::builder()`. * **[Tool](/agents/tools/)**: a function the model can call by name with JSON arguments. It implements the `Tool` trait. The `#[tool]` macro and the `Tool` derive write it for you. * **[Hook](/agents/hooks/)**: a closure the agent runs at a fixed point in its loop, such as `before_all`, `before_tool`, `after_each` or `on_stop`. Most hooks receive `&mut Agent`. * **[Tool executor](/agents/executors/)**: where tools run commands and read or write files. It implements `ToolExecutor`. The default, `LocalExecutor`, runs commands on your machine. * **[Context](/agents/context-and-history/)**: the state an agent runs in. It implements `AgentContext` and holds the message history and the tool executor. The default is `DefaultContext`. * **[Message history](/agents/context-and-history/)**: the storage that keeps a context’s chat messages. It implements `MessageHistory`. The default keeps messages in memory. `Redis` from the `redis` integration keeps them between runs. * **[Stop reason](/agents/stopping-and-structured-results/)**: why an agent stopped, for example because a tool asked it to or it failed. Read it with `stop_reason()`. Its type is `StopReason`. ## Tasks [Section titled “Tasks”](#tasks) * **[Task](/tasks/overview/)**: a graph of typed steps with one input and one output type. Running a `Task` returns a `TaskRunOutcome`: the output, or paused. * **[Task node](/tasks/nodes-and-transitions/)**: one typed step in a task, with an input, an output and an error type. It implements `TaskNode`. `Task::register_node_fn` turns a closure into a node. * **[Transition](/tasks/nodes-and-transitions/)**: the value that says where a task goes after a node. Register it on that node with `Task::register_transition` and a function from the node’s output to a `Transition`. In that function, `next.transitions_with(value)` moves to node `next`. Pass `task.transitions_to_finish()` as the function to end the task. * **[Fan-out](/tasks/fan-out-and-join/)**: a transition that starts several branches at once. `Transition::fan_out` starts it, `and` adds a branch, and `join_with(join.join())` names the join that waits for every branch. Until then it is a `FanOutTransition`, which cannot run. * **[Join](/tasks/fan-out-and-join/)**: a node that receives the outputs of every fan-out branch together as a `JoinInput`. `NodeId::join` gives the join target. Each branch registers its own transition to that target. ## Indexing [Section titled “Indexing”](#indexing) * **[Indexing pipeline](/indexing/overview/)**: a stream that loads data as nodes, transforms them and stores them. Start one with `Pipeline::from_loader` from `swiftide::indexing` and run it with `run`. * **[Node](/indexing/overview/)**: one unit of data in an indexing pipeline: a path, a chunk, metadata and embeddings. `Node` is generic over its chunk type. `TextNode` is a node whose chunk is a `String`. * **[Chunk](/indexing/chunking/)**: the content a node carries, in its `chunk` field. Any type that is `Clone`, `Send`, `Sync`, `Debug`, `'static` and readable as bytes implements the `Chunk` trait automatically. * **[Loader](/indexing/loading/)**: the source at the start of an indexing pipeline. It implements `Loader` and turns files, pages or messages into nodes. `FileLoader` reads files from disk. * **[Transformer](/indexing/transformers-and-metadata/)**: a step that maps one node to one node, for example to add metadata. It implements `Transformer`. Add it with `then`. * **[Batch transformer](/indexing/transformers-and-metadata/)**: a step that takes many nodes in one call. It implements `BatchableTransformer`. Add it with `then_in_batch`. * **[Chunker](/indexing/chunking/)**: a step that splits one node into many. It implements `ChunkerTransformer`, and `ChunkMarkdown` is one. Add it with `then_chunk`. * **[Embedder](/indexing/embedding/)**: a model that turns text into vectors. Dense models implement `EmbeddingModel` and `Embed` calls them. Sparse models implement `SparseEmbeddingModel` and `SparseEmbed` calls them. * **[Persist](/indexing/storage/)**: the trait a pipeline calls to store nodes. Add a storage with `then_store_with`. * **[Storage](/indexing/storage/)**: the backend that implements `Persist`, such as Qdrant, PgVector, Redis or LanceDB. Their flags are `qdrant`, `pgvector`, `redis` and `lancedb`. * **[Node cache](/indexing/caching/)**: a record of the nodes a pipeline already processed. It implements `NodeCache`. `filter_cached` drops nodes found in the cache. A node enters it only after passing the whole pipeline. ## Querying [Section titled “Querying”](#querying) * **[Query pipeline](/querying/overview/)**: a pipeline that turns a question into an answer in four stages: query transformers (`then_transform_query`), a retriever (`then_retrieve`), response transformers (`then_transform_response`) and an answer generator (`then_answer`). Start one with `Pipeline::from_search_strategy` from `swiftide::query` and ask with `query`. * **[Retriever](/querying/retrieval-and-search-strategies/)**: a store that fetches documents with a search strategy. It implements `Retrieve` for that strategy. Qdrant, PgVector, LanceDB (`lancedb` flag) and DuckDB (`duckdb` flag) are retrievers. * **[Search strategy](/querying/retrieval-and-search-strategies/)**: settings for how a retriever searches, such as result count and filter. It implements `SearchStrategy`. Built-in ones are `SimilaritySingleEmbedding`, `HybridSearch` and `CustomStrategy`. * **[Response transformer](/querying/reranking-and-response-transformers/)**: a step between retrieval and the answer. It implements `TransformResponse`. `Summary` replaces the response with a summary of the documents. The FastEmbed `Rerank` (`fastembed` flag) reorders the documents. * **[Answer generator](/querying/answering/)**: the last step, which writes the answer from the documents. It implements `Answer`. `Simple` from `swiftide::query::answers` asks an LLM. * **[Evaluator](/querying/evaluation/)**: a component that sees the retrieved documents and the final answer of each query. It implements `EvaluateQuery`. Add it with `evaluate_with`. `Ragas` exports a dataset for the Ragas framework. ## Setup [Section titled “Setup”](#setup) * **[Feature flag](/reference/feature-flags/)**: a Cargo feature of `swiftide` that compiles in an optional part. Only `macros` is on by default. Agents need `swiftide-agents`, tasks need `swiftide-tasks`, and each integration needs its own flag. * **[Integration](/integrations/overview/)**: a module under `swiftide::integrations` that wraps an outside library or service. Examples are providers (`openai`), stores (`qdrant`), loaders (`scraping`) and parsers (`treesitter`). Each has its own feature flag. ## Reference [Section titled “Reference”](#reference) * [`swiftide`](https://docs.rs/swiftide/0.34/swiftide/) (crate root, with the list of Cargo features) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html), [`Hook`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/enum.Hook.html), [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html), [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html), [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`Tool`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html), [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html), [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html), [`MessageHistory`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.MessageHistory.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html), [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html), [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html), [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html), [`FanOutTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.FanOutTransition.html), [`NodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html), [`JoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinInput.html) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html), [`Node`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html), [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html), [`Chunk`](https://docs.rs/swiftide/0.34/swiftide/indexing/trait.Chunk.html), [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html), [`Transformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Transformer.html), [`BatchableTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.BatchableTransformer.html), [`ChunkerTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChunkerTransformer.html), [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html), [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html), [`SparseEmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SparseEmbeddingModel.html), [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html), [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html), [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html), [`NodeCache`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.NodeCache.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html), [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html), [`SearchStrategy`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SearchStrategy.html), [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html), [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html), [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html), [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html), [`Summary`](https://docs.rs/swiftide/0.34/swiftide/query/response_transformers/struct.Summary.html), [`Answer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Answer.html), [`Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html), [`EvaluateQuery`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EvaluateQuery.html), [`Ragas`](https://docs.rs/swiftide/0.34/swiftide/query/evaluators/ragas/struct.Ragas.html) * [`swiftide::integrations`](https://docs.rs/swiftide/0.34/swiftide/integrations/index.html) Next: [Contributing and community](/community/contributing/) # Troubleshooting and FAQ > Fixes for the errors people hit most with Swiftide: pipeline failures, the rustls panic on 0.33, missing features, unreachable stores, rejected tool schemas, runaway agents and silent streaming. Each heading on this page names a symptom. Find yours, apply the fix, and your code runs again. Answers to frequently asked questions (FAQ) close the page. ## Pipeline errors [Section titled “Pipeline errors”](#pipeline-errors) ```rust let stats = indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then_in_batch(Embed::new(openai)) .log_errors() .filter_errors() .then_store_with(qdrant) .run() .await?; println!("{} nodes stored", stats.nodes_stored); ``` Source: [`snippets/src/reference/troubleshooting.rs#L18-L26`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/troubleshooting.rs#L18-L26) An indexing pipeline stops at the first node that fails, and `run` returns that error. When it finishes, `run` returns the pipeline statistics, such as `nodes_stored`. `log_errors` logs each error at `ERROR` level and passes it on. `filter_errors` then drops the errors, so the other nodes still reach storage. Keep `log_errors` first: `filter_errors` on its own drops errors without a trace. Both only see errors from the steps before them, so place them after the steps you want to cover. A storage error still stops the pipeline: Qdrant rejects a node without vectors, so embed first. `log_all` also logs every node at `DEBUG` level. `filter` keeps each `Result` its closure returns `true` for, so you can skip one kind of error. The logs need a `tracing` subscriber; see [Tracing](/observability/tracing/). ```rust let result = indexing::Pipeline::from_loader(FileLoader::new("docs").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(100..2048)) .then_in_batch(Embed::new(openai)) .then_store_with(qdrant) .run() .await; if let Err(error) = result { match error.downcast_ref::() { Some(LanguageModelError::ContextLengthExceeded(_)) => { eprintln!("a chunk is too large for the embedding model: {error:#}"); } Some(model_error) => eprintln!("the model failed: {model_error}"), None => return Err(error), } } ``` Source: [`snippets/src/reference/troubleshooting.rs#L34-L49`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/troubleshooting.rs#L34-L49) The error a step fails with reaches `run` unchanged. A failed model call stays a `LanguageModelError`, so `downcast_ref` tells it apart from a storage or loader error, and `chain()` finds it when other code wrapped it in context. `ContextLengthExceeded` means a chunk is too large for the model; lower the chunker’s maximum size. ## Crypto provider panic [Section titled “Crypto provider panic”](#crypto-provider-panic) ```text No rustls crypto provider is configured. When using the `rustls-no-provider` feature you must install a crypto provider before building a Client. ``` Swiftide’s HTTP clients need no TLS (Transport Layer Security) setup. Each one uses the process-wide rustls crypto provider if your application installed one, and otherwise its own. Redis over `rediss://` needs a process-wide provider, and Swiftide installs aws-lc-rs before it connects if none is installed. [TLS](/reference/feature-flags/#tls) has the details. On 0.33, Swiftide builds its clients without a provider, so building a client panics with this message, or with `No provider set` for the Anthropic client. Install a provider with rustls’s `CryptoProvider::install_default` at the start of `main`, or upgrade to 0.34. Another crate in your application that builds its own rustls configuration also relies on the process-wide provider. When the build compiles in both ring and aws-lc-rs, for example with `openai` and `qdrant`, that crate panics until your application installs one. ## Missing features [Section titled “Missing features”](#missing-features) A `use` of `swiftide::agents` or `swiftide::integrations::openai` fails with error E0432, “unresolved import”, when the feature flag behind it is off. Agents, tasks and each integration have their own feature flag: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents ``` [Feature flags](/reference/feature-flags/) lists what each flag turns on. If `#[tool]` is missing after `default-features = false`, add the `macros` flag back; see [Defaults](/reference/feature-flags/#defaults). ## Unreachable stores [Section titled “Unreachable stores”](#unreachable-stores) Each store needs its feature flag: `qdrant`, `pgvector` or `redis`. * **Qdrant**: the client reads `QDRANT_URL` and `QDRANT_API_KEY`, and falls back to `http://localhost:6334`. That is Qdrant’s gRPC (gRPC Remote Procedure Calls) port, not the 6333 HTTP port. Publish both ports: ```sh docker run -p 6333:6333 -p 6334:6334 qdrant/qdrant ``` * **PgVector**: setup runs `CREATE EXTENSION IF NOT EXISTS vector`. The server needs pgvector installed, and your database user needs permission to create the extension. The connection pool tries 3 times, 3 seconds apart, before it gives up. * **Redis**: pass the full URL and a key prefix to `Redis::try_from_url`, for example `Redis::try_from_url("redis://localhost:6379", "swiftide")`. This call does not connect, so a Redis that is down fails at the first store. Qdrant and PgVector set up before the first node, so if either is down, `run` fails before any node is processed. ## Rejected tool schemas [Section titled “Rejected tool schemas”](#rejected-tool-schemas) `ToolSpecBuilder::build` rejects schemas that break the strict rules providers share: maps such as `HashMap`, open objects and `patternProperties`. The `#[tool]` macro unwraps that result, so a bad argument type panics when `tool_spec` is called. The OpenAI client also rejects `oneOf` and array-valued `type`, with “tool schema must be OpenAI compatible”. Enums whose variants carry data or doc comments produce `oneOf`. Model arguments as structs with named fields and unit-only enums. `Option` is fine. See [Tools](/agents/tools/). ## Agent loop limits [Section titled “Agent loop limits”](#agent-loop-limits) Agents need the `swiftide-agents` feature flag. ```rust let mut agent = Agent::builder() .llm(&openai) .on_stream(|_agent, response| { if let Some(chunk) = response .delta .as_ref() .and_then(|d| d.message_chunk.as_deref()) { print!("{chunk}"); } Box::pin(async move { Ok(()) }) }) .limit(10) .build()?; agent.query("Explain Rust lifetimes").await?; if let Some(reason) = agent.stop_reason() { println!("stopped: {reason:?}"); } ``` Source: [`snippets/src/reference/troubleshooting.rs#L57-L76`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/troubleshooting.rs#L57-L76) With the default context, an agent stops when the model answers without calling a tool. Three settings change when it stops: * `with_stop_on_assistant(false)` on the context keeps the loop going until a tool stops it. The default `stop` tool does that. An agent has no loop limit by default, so if you remove that tool with `without_default_stop_tool`, register your own or set `limit`. * `limit(10)` caps the loop at 10 iterations. At the cap Swiftide logs the warning “Agent loop limit reached” and stops with `StopReason::NoNewMessages`. A normal finish has the same stop reason, so watch for the warning. * When the same tool call keeps failing, the agent sends the error back to the model up to `tool_retry_limit` times (default 3). The next failure stops it with `StopReason::ToolCallsOverLimit`, and `query` returns the error. `stop_reason` separates a tool stop and `ToolCallsOverLimit` from the other cases. It does not separate the loop limit from a normal finish. See [Stopping and structured results](/agents/stopping-and-structured-results/). ## Silent streaming [Section titled “Silent streaming”](#silent-streaming) An agent prints nothing until the answer is complete. Register `on_stream`, as in the agent snippet on this page. The hook switches the agent to streaming and runs for every chunk. `streaming(true)` on the builder streams without a hook. In the hook, `delta.message_chunk` holds the new text. The OpenAI client also puts the full message so far in `response.message` on each chunk, because `stream_full` defaults to `true`. The final chunk carries no new text. Before 0.34 it repeated the last text fragment, so the snippet printed it twice. See [Streaming](/agents/streaming/). ## SimilaritySingleEmbedding type annotations [Section titled “SimilaritySingleEmbedding type annotations”](#similaritysingleembedding-type-annotations) ```rust let pipeline = query::Pipeline::from_search_strategy(SimilaritySingleEmbedding::<()>::default()) .then_transform_query(query::query_transformers::Embed::from_client( openai.clone(), )) .then_retrieve(pgvector) .then_answer(answers::Simple::from_client(openai)); ``` Source: [`snippets/src/reference/troubleshooting.rs#L84-L90`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/troubleshooting.rs#L84-L90) `SimilaritySingleEmbedding::default()` exists only for the strategy without a filter, so its filter type is inferred. Build a filtered strategy with `from_filter` or `with_filter`. On 0.33, `default()` fails with error E0283, “type annotations needed”; name the filter type, as in the snippet. `HybridSearch::default()` still needs the annotation on stores that implement the retriever for more than one filter type. The search strategy is generic over its filter type, and Rust does not use the default type parameter to pick an implementation. ## FAQ [Section titled “FAQ”](#faq) ### Swiftide and Rig [Section titled “Swiftide and Rig”](#swiftide-and-rig) Swiftide ships agents, task graphs, and indexing and query pipelines in one crate family. See [What is Swiftide](/start/what-is-swiftide/). ### Python [Section titled “Python”](#python) Swiftide has no Python bindings. Wrap it in your own service to use it from Python. ### Versioning [Section titled “Versioning”](#versioning) Swiftide is pre-1.0. Breaking changes land in minor releases, such as 0.32 to 0.33. Read [Upgrading](/reference/upgrading/) and the [Changelog](/reference/changelog/) before you move to a new release. ### Production use [Section titled “Production use”](#production-use) Swiftide is part of the [bosun.ai](https://bosun.ai) project, and Bosun runs it in production. ## Reference [Section titled “Reference”](#reference) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`ToolSpec`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpec.html) * [`ToolSpecBuilder`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpecBuilder.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html) * [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OpenAI.html) Next: [Glossary](/reference/glossary/) # Upgrading > Code changes needed to move between Swiftide releases from 0.23 to 0.34, newest first. By the end of this page your code compiles against swiftide 0.34. Breaking changes land in minor releases, so upgrade one minor version at a time. Sections run newest first. Start at the section after your current release and work up. Releases without a section, such as 0.25 to 0.26 and 0.28 to 0.30, need no code changes. ```sh cargo add swiftide@0.34 ``` ## 0.34 [Section titled “0.34”](#034) ### Pipeline statistics [Section titled “Pipeline statistics”](#pipeline-statistics) ```rust let stats = indexing::Pipeline::from_stream(vec![TextNode::new("hello")]) .then_store_with(MemoryStorage::default()) .run() .await?; println!("{} nodes stored", stats.nodes_stored); ``` Source: [`snippets/src/reference/upgrading.rs#L27-L31`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L27-L31) `indexing::Pipeline::run` returns `Result` instead of `Result<()>`. `pipeline.run().await?;` keeps compiling. A function that returns `pipeline.run().await` as its `Result<()>` tail expression needs `pipeline.run().await?; Ok(())`. `stats_collector()` returns `&Arc`, shared with the running pipeline. Clone the `Arc` before `run` to watch the pipeline while it runs. Before, cloning the collector made a copy that never changed. Each branch of `split_by` gets its own collector, seeded with the statistics so far. `merge` keeps the collector of the pipeline it is called on, so the merged run reports both branches. See [Metrics and statistics](/observability/metrics-and-statistics/). ### Search strategy setters [Section titled “Search strategy setters”](#search-strategy-setters) ```rust let similarity = SimilaritySingleEmbedding::default().with_top_k(20); let hybrid = HybridSearch::<()>::default().with_top_k(20).with_top_n(5); ``` Source: [`snippets/src/reference/upgrading.rs#L39-L40`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L39-L40) `with_top_k` on `SimilaritySingleEmbedding`, and `with_top_k`, `with_top_n`, `with_dense_vector_field` and `with_sparse_vector_field` on `HybridSearch`, take `self` and return `Self`. Replace `let mut s = Strategy::default(); s.with_top_k(n);` with `let s = Strategy::default().with_top_k(n);`, and drop `.to_owned()` after a setter chain. `SimilaritySingleEmbedding::default()` exists only for the strategy without a filter, so it no longer needs a type annotation. Build a filtered strategy with `from_filter` or `with_filter`. `HybridSearch::default()` still needs the annotation on stores that implement the retriever for more than one filter type. `SearchStrategy` no longer requires `Default`. ### TLS [Section titled “TLS”](#tls) Swiftide’s HTTP clients no longer need a crypto provider from your application. Each one uses a process-wide rustls provider if you installed one, and otherwise its own. You can remove the `rustls` dependency and the `install_default` call from `main`, or keep them to choose the provider. See [TLS](/reference/feature-flags/#tls). ### OpenAI-compatible endpoints [Section titled “OpenAI-compatible endpoints”](#openai-compatible-endpoints) `OpenAI::builder().api_base(url).api_key(key)` points the OpenAI client at another endpoint without `async-openai`’s `OpenAIConfig`. `.client(..)` still takes a full client and replaces both; whichever is called last wins. See [OpenAI-compatible providers](/integrations/openai-compatible/). ## 0.33 [Section titled “0.33”](#033) ### Rust version and dependencies [Section titled “Rust version and dependencies”](#rust-version-and-dependencies) Swiftide now needs Rust 1.97 or newer. `ReasoningItem.id` is now an `Option`. Public types from SQLx, text-splitter, LanceDB and DuckDB follow the new versions of those crates. If you pass these crates’ types in yourself, bump them in your `Cargo.toml`: async-openai 0.41 for `OpenAI::builder().client(..)`, rmcp 2.1 for `McpToolbox::try_from_transport`, and redis 1.0. ### TLS crypto provider [Section titled “TLS crypto provider”](#tls-crypto-provider) In 0.33, Swiftide does not pick a rustls crypto provider. Install one at the start of `main`, before you build the first model client, or the client build panics. See [Crypto provider panic](/reference/troubleshooting-and-faq/#crypto-provider-panic). Since 0.34 the clients no longer need it; see [TLS](#tls). ### System prompts [Section titled “System prompts”](#system-prompts) ```rust let mut agent = Agent::builder() .llm(&openai) .system_prompt("You review pull requests for {{ project }}.") .before_all(|agent: &mut Agent| { Box::pin(async move { if let Some(prompt) = agent.system_prompt_mut() { prompt.insert_context_value("project", "swiftide"); } Ok(()) }) }) .build()?; ``` Source: [`snippets/src/reference/upgrading.rs#L48-L59`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L48-L59) `SystemPrompt` and its builder are gone, with the role, guidelines and constraints fields. The agent’s system prompt is now a `Prompt`. `AgentBuilder::system_prompt` takes anything that converts into a `Prompt`, such as a `&str`. An agent without `.system_prompt(..)` sends no system prompt. Before 0.33 it sent a default one. The `before_all` hook now runs before the agent renders the system prompt. The hook in the snippet uses `system_prompt_mut` to fill the `project` value at run time. See [System prompts](/agents/system-prompts/). ### Hooks and the agent [Section titled “Hooks and the agent”](#hooks-and-the-agent) ```rust let mut agent = Agent::builder() .llm(&openai) .after_each(|agent: &mut Agent| { Box::pin(async move { if agent.history().await?.len() > 40 { agent.stop("history too long").await; } Ok(()) }) }) .build()?; ``` Source: [`snippets/src/reference/upgrading.rs#L68-L78`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L68-L78) Most hooks now receive `&mut Agent` instead of `&Agent`. A hook can call methods that take `&mut self`, such as `stop`. The changed hooks are `before_all`, `on_start`, `before_completion`, `after_completion`, `before_tool`, `after_tool`, `after_each` and `on_stop`. `on_new_message` and `on_stream` still receive `&Agent`. See [Hooks](/agents/hooks/). ### Custom contexts [Section titled “Custom contexts”](#custom-contexts) `AgentContext` has a new required method, `replace_history`, which replaces the whole message history. Implement it if you have your own context. ### Command output [Section titled “Command output”](#command-output) ```rust let cmd = Command::shell("cargo test").with_current_dir("crates/core"); match context.executor().exec_cmd(&cmd).await { Ok(output) => { let stdout: Vec = output.stdout().flat_map(|b| b.iter().copied()).collect(); println!("{}", String::from_utf8_lossy(&stdout)); } Err(CommandError::NonZeroExit(output)) => { for chunk in output.stderr() { eprintln!("{}", String::from_utf8_lossy(chunk)); } } Err(other) => return Err(other.into()), } ``` Source: [`snippets/src/reference/upgrading.rs#L87-L99`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L87-L99) `CommandOutput.output` is gone, and so is `as_ref()` on `CommandOutput`. It keeps stdout and stderr chunks in the order they arrived. Read them with `stdout()` and `stderr()`, or get one string with `to_string_lossy()` or `Display`. `CommandOutput::from` now takes bytes, through `Into`. `AgentContext::exec_cmd` has been deprecated since 0.27. Call `context.executor().exec_cmd(&cmd)` instead. ### Tasks crate [Section titled “Tasks crate”](#tasks-crate) ```rust let mut task = Task::::new(); let double = task.register_node_fn(|n: &i32| Ok::<_, NodeError>(n * 2)); task.starts_with(double); task.register_transition(double, task.transitions_to_finish())?; match task.run(21).await? { TaskRunOutcome::Completed(answer) => println!("{answer}"), TaskRunOutcome::Paused => println!("waiting to resume"), } ``` Source: [`snippets/src/reference/upgrading.rs#L107-L115`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L107-L115) Task graphs moved to the `swiftide-tasks` crate, available as `swiftide::tasks`. Its own `swiftide-tasks` feature flag lets you use tasks without agents. `swiftide::agents::tasks` now re-exports `swiftide::tasks`, but the nested modules are gone. Import from the root: `tasks::task::Task` becomes `tasks::Task`, and `tasks::closures::SyncFn` becomes `tasks::SyncFn`. `TaskAgent` moved from `tasks::impls` to `swiftide::agents::TaskAgent`. `Task::run` and `Task::resume` return a `TaskRunOutcome`: `Completed` with the output, or `Paused`. `transitions_to_finish()` replaces `transitions_to_done()`. `current_nodes()` replaces `current_node()`. `done()`, `current_input()`, `current_output()`, `node_at()` and `node_at_index()` are gone. `TransitionPayload` and `NextNode` are no longer public. In a transition closure, return `Transition::next(&node, value)` or `node.transitions_with(value)` to move on. Return `Transition::pause()` to pause and `Transition::error(err)` to fail. A fan-out ends in a join: `Transition::fan_out(..)` with `join_with(..)`, and the join node receives a `JoinInput`. See [Fan-out and join](/tasks/fan-out-and-join/). ### Tool schemas [Section titled “Tool schemas”](#tool-schemas) `ToolSpecBuilder::build` now validates the parameter schema against the strict tool-schema contract and returns a `ToolSpecBuildError`. A schema that built in 0.32 can fail. See [Rejected tool schemas](/reference/troubleshooting-and-faq/#rejected-tool-schemas). The `#[tool]` macro reads the tool and argument descriptions from the doc comment and its `# Arguments` list, as in the [0.32 snippet](#tool-arguments). ### AWS Bedrock [Section titled “AWS Bedrock”](#aws-bedrock) The Amazon Web Services (AWS) Bedrock integration is now `integrations::aws_bedrock_v2`. Build it with `AwsBedrock::builder().default_prompt_model("model-id")`. The model family builders, such as `build_anthropic_family`, are gone. See [AWS Bedrock](/integrations/aws-bedrock/). ### Chat messages [Section titled “Chat messages”](#chat-messages) `ChatMessage` has two new variants, `UserWithParts` for images and other content parts, and `Reasoning`. A `match` without a wildcard arm needs arms for both. `ChatCompletionRequest` now has a lifetime parameter. See [Images and multimodal input](/models/images/). ## 0.32 [Section titled “0.32”](#032) ### Tool arguments [Section titled “Tool arguments”](#tool-arguments) ```rust /// An inclusive range of lines. #[derive(Debug, Clone, serde::Serialize, serde::Deserialize, schemars::JsonSchema)] pub struct LineRange { /// First line, starting at 1 pub start: u32, /// Last line, inclusive pub end: u32, } /// Reads line ranges from files in the workspace. /// /// # Arguments /// - `paths`: Files to read, relative to the workspace root. /// - `range`: Optional line range to limit the output. #[swiftide::tool] async fn read_lines( context: &dyn AgentContext, paths: Vec, range: Option, ) -> Result { let _ = (context, range); Ok(ToolOutput::text(paths.join("\n"))) } ``` Source: [`snippets/src/reference/upgrading.rs#L121-L143`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L121-L143) `ToolSpec.parameters` and `ParamSpec` are gone. A hand-written tool sets `parameters_schema`, an `Option`, instead. The `#[tool]` macro derives a JSON Schema from the Rust types of the arguments. A struct argument needs `schemars::JsonSchema` next to the serde derives, so add `schemars` 1.x to your dependencies. See [Tools](/agents/tools/). ### Working directories [Section titled “Working directories”](#working-directories) ```rust let repo = executor.scoped("repo"); let output = repo.exec_cmd(&Command::shell("git status --short")).await?; println!("{output}"); ``` Source: [`snippets/src/reference/upgrading.rs#L159-L161`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L159-L161) Its variants are now structs, so patterns like `Command::Shell(script)` no longer compile. Build commands with `Command::shell`, `Command::read_file` and `Command::write_file`, and set a directory with `with_current_dir`. `scoped` on `ExecutorExt` wraps a tool executor in a `ScopedExecutor`. It joins a relative working directory onto the scope and leaves an absolute one unchanged. A custom tool executor reads the directory with `current_dir_path()`. See [Tool executors](/agents/executors/). ### MCP tool names [Section titled “MCP tool names”](#mcp-tool-names) Model Context Protocol (MCP) tools are now named `server:tool`. The prefix is the name the server reports in its handshake, or `mcp` when it reports none. The name you set with `McpToolbox::with_name` does not change it. MCP needs the `mcp` feature flag. See [MCP](/agents/mcp/). ## 0.31 [Section titled “0.31”](#031) ### Generic nodes [Section titled “Generic nodes”](#generic-nodes) ```rust #[derive(Clone)] pub struct Uppercase; #[async_trait] impl Transformer for Uppercase { type Input = String; type Output = String; async fn transform_node(&self, mut node: TextNode) -> anyhow::Result { node.chunk = node.chunk.to_uppercase(); Ok(node) } } ``` Source: [`snippets/src/reference/upgrading.rs#L167-L179`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L167-L179) Indexing pipelines are generic over the content of a node. `Node` replaces `Node`, and `TextNode` is the alias for `Node`. The transformer, batch transformer, chunker and persist traits declare `type Input` and `type Output`. For text, set both to `String`. A loader declares only `Output`, and a node cache only `Input`. ## 0.27 [Section titled “0.27”](#027) ### Tool invocation [Section titled “Tool invocation”](#tool-invocation) ```rust #[derive(Clone)] pub struct Echo; #[async_trait] impl Tool for Echo { async fn invoke( &self, _context: &dyn AgentContext, tool_call: &ToolCall, ) -> Result { Ok(ToolOutput::text(tool_call.args().unwrap_or_default())) } fn name(&self) -> Cow<'_, str> { Cow::Borrowed("echo") } fn tool_spec(&self) -> ToolSpec { ToolSpec::builder() .name("echo") .description("Repeats its arguments") .build() .expect("valid tool spec") } } ``` Source: [`snippets/src/reference/upgrading.rs#L183-L207`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L183-L207) `Tool::invoke` receives the whole `ToolCall` instead of `raw_args: Option<&str>`. The raw arguments are `tool_call.args()`. ### Context and message history [Section titled “Context and message history”](#context-and-message-history) ```rust let mut context = DefaultContext::default(); context.with_existing_messages(saved).await?; let mut agent = Agent::builder().llm(&openai).context(context).build()?; agent.query("Continue where we left off").await?; ``` Source: [`snippets/src/reference/upgrading.rs#L290-L294`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L290-L294) To store messages elsewhere, implement `MessageHistory` and pass it to `DefaultContext::with_message_history`. Loading saved messages is now the async `with_existing_messages`. Redis implements `MessageHistory` behind the `redis` feature flag. See [Context and message history](/agents/context-and-history/). ### Context methods [Section titled “Context methods”](#context-methods) ```rust /// Counts the messages an agent adds and delegates everything else. #[derive(Clone)] pub struct CountingContext { inner: DefaultContext, added: Arc, } #[async_trait] impl AgentContext for CountingContext { async fn next_completion(&self) -> anyhow::Result>> { self.inner.next_completion().await } async fn current_new_messages(&self) -> anyhow::Result> { self.inner.current_new_messages().await } async fn add_messages(&self, items: Vec) -> anyhow::Result<()> { self.added .fetch_add(items.len(), std::sync::atomic::Ordering::Relaxed); self.inner.add_messages(items).await } async fn add_message(&self, item: ChatMessage) -> anyhow::Result<()> { self.added .fetch_add(1, std::sync::atomic::Ordering::Relaxed); self.inner.add_message(item).await } #[allow(deprecated)] async fn exec_cmd(&self, cmd: &Command) -> Result { self.inner.exec_cmd(cmd).await } fn executor(&self) -> &Arc { self.inner.executor() } async fn history(&self) -> anyhow::Result> { self.inner.history().await } async fn replace_history(&self, items: Vec) -> anyhow::Result<()> { self.inner.replace_history(items).await } async fn redrive(&self) -> anyhow::Result<()> { self.inner.redrive().await } async fn has_received_feedback(&self, tool_call: &ToolCall) -> Option { self.inner.has_received_feedback(tool_call).await } async fn feedback_received( &self, tool_call: &ToolCall, feedback: &ToolFeedback, ) -> anyhow::Result<()> { self.inner.feedback_received(tool_call, feedback).await } } ``` Source: [`snippets/src/reference/upgrading.rs#L211-L272`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L211-L272) The context methods that read or change messages now return a `Result`. Add a `?` where you call them, so `context.history().await` becomes `context.history().await?`. This applies to `next_completion`, `current_new_messages`, `add_message`, `add_messages`, `history` and `redrive`. A custom context needs three new methods. `executor` returns the tool executor that runs commands for the agent. `feedback_received` stores the feedback for a tool call, such as a human approval. `has_received_feedback` returns that feedback when the agent calls the tool again. The snippet wraps a `DefaultContext` and passes each call through to it. It targets 0.34, so it also has `replace_history`, which arrived in 0.33. ### Tool output [Section titled “Tool output”](#tool-output) ```rust match output { ToolOutput::Text(text) => text.clone(), ToolOutput::FeedbackRequired(payload) => format!("waiting for approval: {payload:?}"), ToolOutput::Fail(error) => format!("tool failed: {error}"), other => format!("agent stops: {other:?}"), } ``` Source: [`snippets/src/reference/upgrading.rs#L278-L283`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L278-L283) `ToolOutput` has a new `FeedbackRequired` variant. A tool returns it when it needs feedback before it runs. The agent then stops with the new `StopReason::FeedbackRequired`. Both enums are marked non-exhaustive, so your `match` already has a wildcard arm. That arm now catches `FeedbackRequired`, which compiles but can hide a paused agent. Add an arm for it. To ask for approval before a tool runs, wrap the tool in `ApprovalRequired`. See [Human in the loop](/agents/human-in-the-loop/). ## 0.24 [Section titled “0.24”](#024) ### Model errors [Section titled “Model errors”](#model-errors) ```rust #[derive(Debug, Clone)] pub struct CannedModel { pub overloaded: bool, } #[async_trait] impl SimplePrompt for CannedModel { async fn prompt(&self, prompt: Prompt) -> Result { if self.overloaded { return Err(LanguageModelError::transient("model is overloaded")); } // An `anyhow::Error` converts into `LanguageModelError::PermanentError`. let rendered = prompt.render()?; Ok(format!("You asked: {rendered}")) } } ``` Source: [`snippets/src/reference/upgrading.rs#L300-L315`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L300-L315) The model traits now return a `LanguageModelError`. This breaks every model client you wrote yourself. `SimplePrompt::prompt` returns `Result` instead of `anyhow::Result`. `EmbeddingModel::embed` and `SparseEmbeddingModel::sparse_embed` changed the same way. `ChatCompletion::complete` returns `LanguageModelError` in place of `ChatCompletionError`, which is gone. Pick the variant that tells callers what to do next. `LanguageModelError::transient` marks an error worth a retry, such as a rate limit. `permanent` marks one that fails again on retry. `context_length_exceeded` marks a request too long for the model. The `?` operator still works on an `anyhow::Error`, which becomes a `PermanentError`. See [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/). ## 0.23 [Section titled “0.23”](#023) ### Templates [Section titled “Templates”](#templates) ```rust let prompt = Prompt::from("Summarize {{ title }} in one sentence.") .with_context_value("title", "the changelog"); let rendered = prompt.render()?; ``` Source: [`snippets/src/reference/upgrading.rs#L321-L323`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/reference/upgrading.rs#L321-L323) `Template` is gone. `Prompt` holds either a one-off template string or the name of a compiled template. Use `Prompt::from` for a string and `Prompt::from_compiled_template` for a name. `render` is no longer async. See [Prompts and templates](/models/prompts-and-templates/). ### Stopping an agent [Section titled “Stopping an agent”](#stopping-an-agent) `stop` is now async and takes a stop reason, so `agent.stop()` becomes `agent.stop("done").await`. A `&str` converts into `StopReason::Other`. ## Reference [Section titled “Reference”](#reference) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`hooks`](https://docs.rs/swiftide/0.34/swiftide/agents/hooks/index.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`CommandError`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.CommandError.html) * [`Command`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.Command.html) * [`ExecutorExt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ExecutorExt.html) * [`ScopedExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.ScopedExecutor.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`JoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinInput.html) * [`TaskAgent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.TaskAgent.html) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`EmbeddingModel`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.EmbeddingModel.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`Tool`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ToolSpec`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpec.html) * [`Node`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Node.html) * [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`Transformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Transformer.html) * [`MessageHistory`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.MessageHistory.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) * [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) Next: [Troubleshooting and FAQ](/reference/troubleshooting-and-faq/) # Ask your codebase > A CLI that indexes a Rust codebase locally with tree-sitter, FastEmbed and LanceDB, and answers questions about it citing the files it used. This showcase is a command-line tool with two commands. `index` turns a codebase into searchable chunks, and `ask` answers a question from those chunks and lists the files it used. Everything except the large language model (LLM) runs on your machine. You need no Docker and no database server. ## Prerequisites [Section titled “Prerequisites”](#prerequisites) * Rust 1.97.1, which the swiftide workspace pins in `rust-toolchain.toml`. * `OPENAI_API_KEY` for any OpenAI-compatible endpoint. `OPENAI_BASE_URL` is optional and defaults to `https://api.openai.com/v1`. `OPENAI_MODEL` is optional and defaults to `gpt-5.4-mini`. An empty `OPENAI_API_KEY` or `OPENAI_BASE_URL` counts as unset. An empty `OPENAI_MODEL` does not: it sets the model to an empty name, so unset it instead. * About 600MB of disk for the code embedding model. FastEmbed downloads it into `.fastembed_cache/` in the current directory on first use. The crate turns off swiftide’s default features and enables the `tree-sitter`, `fastembed`, `lancedb`, `redb` and `openai` feature flags. To build the same thing in your own crate: ```sh cargo add swiftide@0.34 --no-default-features --features tree-sitter,fastembed,lancedb,redb,openai cargo add fastembed@5 tree-sitter@0.26 --no-default-features ``` The crate also depends on `fastembed` to pick the embedding model and `tree-sitter` to find unit tests, each at the version swiftide uses. TLS needs no setup: swiftide’s HTTP clients bring their own rustls crypto provider ([TLS](/integrations/overview/#tls)). ## Architecture [Section titled “Architecture”](#architecture) The tool runs two pipelines that share one local embedder. The indexing pipeline loads files, chunks them, describes each chunk with the LLM, embeds the result and stores it. The query pipeline embeds your question with the same model, retrieves the closest chunks and asks the LLM to answer from them. ![Indexing and query pipelines](/diagrams/diagram-pipeline.svg) The index lives next to the code, in `/.ask-your-codebase/`. It holds `cache.redb`, the node cache, and `lancedb/`, the vector table. ## Indexing pipeline [Section titled “Indexing pipeline”](#indexing-pipeline) examples/ask-your-codebase/src/index.rs ```rust let stats = indexing::Pipeline::from_loader( FileLoader::new(codebase.root()).with_extensions(&languages.extensions()), ) .filter(test_code::not_test_files(codebase.clone())) // Drop unit tests before the cache, which keys on path and content. Changing the content // after it would make the cache miss on every run. .then(test_code::without_unit_tests) .then(files_found.count()) // Skip files whose path and content are unchanged since the last successful run .filter_cached(codebase.node_cache()?) .then(files_changed.count()) .then(record_relative_path(codebase.clone())) // Split along syntax boundaries (functions, impls, ...) with tree-sitter .then_chunk(ChunkByLanguage::try_new(languages, CHUNK_SIZE)?) // Ask the LLM which questions each chunk answers; stored as metadata and embedded .then( MetadataQACode::from_client(llm) .prompt_template(QUESTIONS_PROMPT) .num_questions(QUESTIONS_PER_CHUNK) .concurrency(concurrent_requests) .build()?, ) // Embed chunk + metadata locally, in batches .then_in_batch(Embed::new(embedding::local_model()?).with_batch_size(embedding::BATCH_SIZE)) .then_store_with(codebase.vector_store()?) .run() .await?; ``` Source: [`examples/ask-your-codebase/src/index.rs#L89-L115`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L89-L115) `FileLoader` respects `.gitignore`. `filter_cached` skips files whose path and content are unchanged since the last successful run. `then_in_batch` sends 32 chunks to the embedder per call. The two `count` closures count files for the summary line. `run` returns the `PipelineStats` of the run, and its `nodes_stored` is the number of chunks in the summary. The order of the test steps matters. The node cache keys on the file path and its content. If unit tests were removed after `filter_cached`, the content would differ on every run and the cache would never hit. ### Test filtering [Section titled “Test filtering”](#test-filtering) examples/ask-your-codebase/src/test_code.rs ```rust /// Drops test files before they are cached or chunked: files in a `tests/` directory, test /// modules in their own file (`src/agent/tests.rs`) and test support modules such as /// `src/test_utils.rs`. /// /// Errors are kept, so they still surface. pub fn not_test_files(codebase: Codebase) -> impl Fn(&Result) -> bool + Send + Sync { move |file| { !file .as_ref() .is_ok_and(|file| is_test_file(codebase.relative_path(&file.path))) } } /// Removes unit tests (`#[cfg(test)] mod tests { .. }`, `#[test] fn ..`) from Rust files. /// /// This runs on whole files, before chunking, so a chunk never mixes implementation and tests. pub fn without_unit_tests(mut file: TextNode) -> Result { if file .path .extension() .is_some_and(|extension| extension == "rs") { file.chunk = strip_test_items(&file.chunk)?; } Ok(file) } ``` Source: [`examples/ask-your-codebase/src/test_code.rs#L16-L41`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/test_code.rs#L16-L41) `Pipeline::filter` takes a closure over `&Result` and keeps the result when the closure returns true. The predicate keeps errors on purpose, so a failed load still surfaces. The second function is an ordinary transformer: any `Fn(TextNode) -> Result` works, and this one rewrites `chunk` for `.rs` files. ### Node cache and storage [Section titled “Node cache and storage”](#node-cache-and-storage) examples/ask-your-codebase/src/codebase.rs ```rust /// Remembers which files were fully indexed, so unchanged files are skipped next time. pub fn node_cache(&self) -> Result { std::fs::create_dir_all(&self.data_dir)?; Redb::builder() .database_path(self.data_dir.join("cache.redb")) .table_name("indexed_files") .build() } /// Embedded vector store with one vector per chunk and the source path next to it. pub fn vector_store(&self) -> Result { let uri = self.lancedb_dir(); LanceDB::builder() .uri(uri.to_str().context("Index path is not valid UTF-8")?) .table_name("code_chunks") .vector_size(embedding::DIMENSIONS) .with_vector(EmbeddedField::Combined) .with_metadata(PATH_FIELD) .build() } ``` Source: [`examples/ask-your-codebase/src/codebase.rs#L56-L77`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/codebase.rs#L56-L77) `Redb` is the node cache, stored in one table of a local database file. `LanceDB` is the storage: one vector column over `EmbeddedField::Combined`, sized at 768 for the embedding model, plus a `path` metadata column. Since 0.33, the pipeline marks files in the node cache only after the whole run finishes without an error. If any file fails, or you interrupt the run, nothing from that run is cached. The next run prompts and embeds every file again, not only the one that failed. ### Chunking per language [Section titled “Chunking per language”](#chunking-per-language) examples/ask-your-codebase/src/languages.rs ```rust /// Splits each file with a tree-sitter `ChunkCode` for the file's language. /// /// `ChunkCode` is built for a single language; this chunker picks the right one per file from /// its extension, so one pipeline can index a polyglot codebase. #[derive(Clone)] pub struct ChunkByLanguage { chunkers: Vec<(SupportedLanguages, ChunkCode)>, } impl ChunkByLanguage { /// Chunks are between `chunk_size.start` and `chunk_size.end` bytes; smaller ones are dropped. pub fn try_new(languages: &SourceLanguages, chunk_size: Range) -> Result { let chunkers = languages .0 .iter() .map(|&language| { let chunker = ChunkCode::try_for_language_and_chunk_size(language, chunk_size.clone())?; Ok((language, chunker)) }) .collect::>()?; Ok(Self { chunkers }) } fn chunker_for(&self, path: &Path) -> Option<&ChunkCode> { let extension = path.extension()?.to_str()?; self.chunkers .iter() .find(|(language, _)| language.file_extensions().contains(&extension)) .map(|(_, chunker)| chunker) } } #[async_trait] impl ChunkerTransformer for ChunkByLanguage { type Input = String; type Output = String; async fn transform_node(&self, node: TextNode) -> IndexingStream { match self.chunker_for(&node.path) { Some(chunker) => chunker.transform_node(node).await, None => anyhow!("No chunker for {}", node.path.display()).into(), } } } ``` Source: [`examples/ask-your-codebase/src/languages.rs#L39-L85`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/languages.rs#L39-L85) `ChunkCode` is built for one language. This chunker implements `ChunkerTransformer`, holds one `ChunkCode` per language and picks one by file extension through `SupportedLanguages::file_extensions`. The chunker aims for 10 to 4096 bytes, which keeps most functions in one piece. Chunks of 10 bytes or less are dropped. When tree-sitter finds a syntax error, the whole file becomes one chunk, whatever its size. ### Question metadata [Section titled “Question metadata”](#question-metadata) examples/ask-your-codebase/src/index.rs ````rust /// Asks for the questions a developer would ask that a chunk answers. /// /// The default `MetadataQACode` prompt asks generic questions ("What does this code do?"). /// Questions about behaviour, phrased the way a developer would ask them, match real questions /// much better. const QUESTIONS_PROMPT: &str = r" # Task List the questions a developer could ask about this codebase that the code below answers, each with a short answer. They are stored with the code to help a search find it. Focus on behaviour: when and in which order things happen, what starts or ends a process, which conditions and configuration change the outcome, what happens on errors, and which extension points (traits, hooks, builders, options) exist and when they apply. # Constraints - Write {{ questions }} questions and answers. - Phrase questions the way a developer would ask them, e.g. 'When does the agent stop?'. - Name the relevant types, functions and variants in the answers. - Only use what can be derived from the code. - Respond only in this format: Q1: A1: # Code ``` {{ node.chunk }} ``` "; ```` Source: [`examples/ask-your-codebase/src/index.rs#L39-L70`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L39-L70) The default `MetadataQACode` prompt asks generic questions. This template asks for questions about behavior, phrased the way a developer asks them. It is a Tera template: `MetadataQACode` fills in `questions` with the count and `node` with the chunk. The answer is stored under the metadata key `Questions and Answers (code)`. It is embedded with the chunk, but LanceDB has no column for it. examples/ask-your-codebase/src/index.rs ```rust /// Stores the file path, relative to the codebase root, as metadata on the node. /// /// The vector store keeps this field next to each chunk, so answers can cite their sources. fn record_relative_path( codebase: Codebase, ) -> impl Fn(TextNode) -> Result + Clone + Send + Sync + 'static { move |mut node| { let path = codebase .relative_path(&node.path) .to_string_lossy() .into_owned(); node.metadata.insert(PATH_FIELD, path); Ok(node) } } ``` Source: [`examples/ask-your-codebase/src/index.rs#L127-L141`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/index.rs#L127-L141) A closure that returns `Result` is a transformer too. This one writes the relative path into the metadata, where the `path` column picks it up. ### Local embeddings [Section titled “Local embeddings”](#local-embeddings) examples/ask-your-codebase/src/embedding.rs ```rust /// A code-aware embedding model that runs on the CPU and needs no API key. /// /// The model (~600MB) is downloaded on first use. pub fn local_model() -> Result { let model = TextEmbedding::try_new( InitOptions::new(EmbeddingModel::JinaEmbeddingsV2BaseCode).with_max_length(MAX_TOKENS), )?; FastEmbed::builder() .embedding_model(model) .batch_size(BATCH_SIZE) .build() } ``` Source: [`examples/ask-your-codebase/src/embedding.rs#L18-L30`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/embedding.rs#L18-L30) `JinaEmbeddingsV2BaseCode` is a code embedding model with 768 dimensions. It runs on the CPU and needs no key. Both pipelines call `local_model`, because query vectors and chunk vectors must come from the same model. After you change the model, delete the index and build it again. ## Query pipeline [Section titled “Query pipeline”](#query-pipeline) examples/ask-your-codebase/src/query.rs ```rust let search = SimilaritySingleEmbedding::default().with_top_k(top_k); let pipeline = query::Pipeline::from_search_strategy(search) // Rephrase the question into several, to widen what the search can match .then_transform_query( GenerateSubquestions::builder() .client(llm.clone()) .prompt_template(SUBQUESTIONS_PROMPT.into()) .build()?, ) // Embed with the same local model that embedded the chunks .then_transform_query(Embed::from_client(embedding::local_model()?)) .then_retrieve(codebase.vector_store()?) // Answer the original question from the retrieved code, each chunk under its file path .then_answer( answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(CHUNK_TEMPLATE) .build()?, ); let answered = pipeline.query(question).await?; ``` Source: [`examples/ask-your-codebase/src/query.rs#L36-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L36-L58) `SimilaritySingleEmbedding` returns 10 chunks by default, and `with_top_k` sets the `--top-k` value instead. `with_top_k` takes the strategy by value and returns it, so it chains onto `default()`. `default()` is only implemented for the unfiltered strategy, so the filter type is inferred as `()`, meaning no filter. `GenerateSubquestions` rewrites the question into several related ones. The `Embed` query transformer embeds the rewritten text, the question and its subquestions together, as one vector with the local model. `query` consumes the pipeline and returns the answered query. ### Prompts and document template [Section titled “Prompts and document template”](#prompts-and-document-template) examples/ask-your-codebase/src/query.rs ````rust /// Widens the question into the follow-up questions its answer involves. /// /// They are phrased like the questions generated for each chunk while indexing, so the two /// embed close together. const SUBQUESTIONS_PROMPT: &str = r" Your job is to help a search find the code that answers a developer's question about a codebase. Question: {{ question }} Write {{ num_questions }} short follow-up questions that the code answering this question would also answer. Think about the mechanisms involved: the main loop, the types and enum variants, the conditions that are checked, the functions that are called, and the configuration that changes the outcome. ## Constraints * Phrase them the way a developer would ask about this codebase, e.g. 'When does the agent stop?'. * Do not guess names of types or functions. * Respond with the original question and the follow-up questions only, one per line, prefixed with '- '. "; /// Renders a retrieved chunk under its file path, so the answer can cite it. /// /// `metadata.path` is the `PATH_FIELD` metadata; a test keeps the two in step. const CHUNK_TEMPLATE: &str = r"### {{ metadata.path }} ``` {{ content }} ``` "; /// Answers from the retrieved code and cites files. const ANSWER_PROMPT: &str = r" Answer a developer's question about a codebase, using snippets of its source code. Question: {{ question }} ## Constraints * Only use the snippets below. If they do not answer the question, say briefly what is missing. * Be concrete: name the types, functions and variants involved and explain when they apply. * Describe behaviour: what calls what, in which order, and under which conditions. * Cite the file each statement comes from in backticks, e.g. (`src/lib.rs`). * Use concise markdown. Add a short code example only if it shows how to use the API. ## Snippets {{ documents }} "; ```` Source: [`examples/ask-your-codebase/src/query.rs#L65-L112`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L65-L112) The subquestion template receives `question` and `num_questions`, which defaults to 5. The answer template receives `question` and `documents`. With `document_template`, each retrieved document is rendered on its own, with `content` and `metadata.path` available, and the results are joined with `---` lines. That is how the answer sees which file each chunk came from. ### Sources [Section titled “Sources”](#sources) examples/ask-your-codebase/src/query.rs ```rust impl From> for Answer { /// Keeps the distinct file paths of the retrieved chunks, most relevant first. fn from(query: Query) -> Self { let mut sources: Vec = Vec::new(); for document in query.documents() { let Some(path) = document.metadata().get(PATH_FIELD).and_then(|v| v.as_str()) else { continue; }; if !sources.iter().any(|source| source == path) { sources.push(path.to_owned()); } } Self { text: query.answer().to_owned(), sources, } } } ``` Source: [`examples/ask-your-codebase/src/query.rs#L123-L142`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/query.rs#L123-L142) `documents` returns the retrieved documents in relevance order. The `path` column comes back as a `path` metadata entry. The tool keeps each path once and prints the list under the answer. ## Model client [Section titled “Model client”](#model-client) examples/ask-your-codebase/src/llm.rs ```rust impl LlmEndpoint { /// Builds a client for this endpoint. /// /// Rate limits and transient errors are retried with exponential backoff, which matters when /// indexing fires many prompts concurrently. After `retry_budget` the error is returned, so /// an unreachable or broken endpoint fails instead of hanging. pub fn client(&self, retry_budget: Duration) -> Result> { let openai = OpenAI::builder() .api_base(&self.base_url) .api_key(&self.api_key) .default_prompt_model(&self.model) .use_responses_api(matches!(self.api, OpenAiApi::Responses)) .build()?; let backoff = BackoffConfiguration { max_elapsed_time_sec: retry_budget.as_secs(), ..BackoffConfiguration::default() }; Ok(Arc::new(LanguageModelWithBackOff::new(openai, backoff))) } } ``` Source: [`examples/ask-your-codebase/src/llm.rs#L79-L100`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/src/llm.rs#L79-L100) The client is always swiftide’s `OpenAI` integration. `api_base` points it at `OPENAI_BASE_URL` and `api_key` sets the key, without building an `async-openai` client yourself. `use_responses_api` switches from Chat Completions to the Responses API when you pass `--responses-api`. `LanguageModelWithBackOff` retries rate limits and transient errors with exponential backoff. `index` keeps retrying a failing prompt for up to 60 seconds, because it sends many prompts at once and expects rate limits. `ask` stops retrying after 30 seconds. ## Commands [Section titled “Commands”](#commands) The crate lives in `examples/ask-your-codebase` of the swiftide repository. Clone the repository and run from its root: ```sh git clone https://github.com/bosun-ai/swiftide && cd swiftide export OPENAI_API_KEY=sk-... cargo run -p ask-your-codebase -- index swiftide-agents cargo run -p ask-your-codebase -- ask --path swiftide-agents "How does an agent decide to stop?" ``` `index` and `ask` are separate runs, so each can use its own model. Through a gateway that serves Anthropic models, a small model writes the indexing prompts and a stronger one writes the answers: ```sh export OPENAI_BASE_URL=http://127.0.0.1:8080/v1 # an OpenAI-compatible gateway that serves these models cargo run -p ask-your-codebase -- index swiftide-agents --model claude-haiku-5-5 cargo run -p ask-your-codebase -- ask --path swiftide-agents --model claude-sonnet-5-5 "How does an agent decide to stop?" ``` | Flag | Applies to | Default | Description | | -------------------- | ---------- | -------------- | ------------------------------------------------------------------- | | `--model ` | both | `gpt-5.4-mini` | Model for chunk metadata or answers. Also read from `OPENAI_MODEL`. | | `--responses-api` | both | off | Send prompts to the Responses API. | | `--lang ` | `index` | `rust` | Comma-separated tree-sitter languages, such as `rust,python`. | | `--concurrency ` | `index` | `16` | Prompts sent at once while indexing. | | `--path ` | `ask` | `.` | Root of the codebase, as given to `index`. | | `--top-k ` | `ask` | `20` | Number of chunks to retrieve. | By default the tool logs only errors and turns off all logging from `swiftide_integrations`, which hides errors from LanceDB, FastEmbed, Redb, tree-sitter and OpenAI. Each retried attempt is logged at WARN, so it stays hidden too. Set `RUST_LOG=info` or `RUST_LOG=debug` to see what the pipelines do. ## Output [Section titled “Output”](#output) The captured run indexed the `swiftide-agents` crate with a debug build. It went through an OpenAI-compatible gateway that forwards to Anthropic models on AWS Bedrock. `claude-haiku-5-5` wrote the per-chunk questions, `claude-sonnet-5-5` wrote the answers, and FastEmbed ran `jina-embeddings-v2-base-code` on the CPU. The first run indexes everything. The second finds nothing changed, so the node cache skips every file and no prompt is sent: ```text $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 34 chunks from 14 new or changed files (14 files found, 0 unchanged) in 91.9s $ ./target/debug/ask-your-codebase index swiftide-agents --model claude-haiku-5-5 Indexing Rust files in swiftide-agents ... Indexed 0 chunks from 0 new or changed files (14 files found, 14 unchanged) in 0.6s ``` The start of one answer, and its sources: ```text $ ./target/debug/ask-your-codebase ask --model claude-sonnet-5-5 --path swiftide-agents "How does an agent decide to stop?" # How an agent decides to stop An agent stops when something calls `Agent::stop` (or `stop_with_error`), which moves the state to `State::Stopped(StopReason)`. The main loop in `run_agent` checks that state after each step. `Agent::stop` is a no-op if the agent is already stopped (`src/agent.rs`). [...] ## Triggers, by `StopReason` variant (`src/state.rs`) | Variant | When it applies | |---|---| | `RequestedByTool(ToolCall, Option)` | A tool returned `ToolOutput::Stop`. | | `FeedbackRequired { tool_call, payload }` | A tool returned `ToolOutput::FeedbackRequired`. | | `AgentFailed(Option)` | A tool returned `ToolOutput::AgentFailed`. | | `ToolCallsOverLimit(ToolCall)` | The same tool call failed too many times. | | `Error` | `step` returned an `Err`. | | `NoNewMessages` | The completion loop ended without another reason. | | `Other(String)` | Anything convertible from `AsRef`, e.g. `agent.stop("done")`. | [...] Sources: - src/agent.rs - src/tools/control.rs - src/lib.rs - src/state.rs - src/errors.rs - src/task_agent.rs - src/hooks.rs - src/tasks.rs - src/default_context.rs ``` The answers in [`OUTPUT.md`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/ask-your-codebase/OUTPUT.md) were checked against the source of `swiftide-agents`. Where a needed chunk was not among the 20 retrieved, the answer marked its claim as an inference. ## Limitations [Section titled “Limitations”](#limitations) * When a file changes, its new chunks are added, but the chunks of the old version stay in LanceDB. Delete `.ask-your-codebase/` to start over. * Files under a `tests` directory, or named `tests`, `test_utils` or `test_helpers`, are dropped in every language. Unit tests inside a file are only removed from `.rs` files. * Retrieval is a single vector search. Raise `--top-k` for broad questions. * A file that produces no chunks is never marked in the node cache, so every run loads it again. It costs no LLM calls. ## Concept pages [Section titled “Concept pages”](#concept-pages) * Loading and caching: [Loading](/indexing/loading/), [Caching](/indexing/caching/) * Chunking: [Code-aware indexing](/indexing/code-aware-indexing/), [Chunking](/indexing/chunking/), [Tree-sitter](/integrations/tree-sitter/) * Metadata and embeddings: [Transformers and metadata](/indexing/transformers-and-metadata/), [Embedding](/indexing/embedding/), [FastEmbed](/integrations/fastembed/) * Storage: [Storage](/indexing/storage/), [LanceDB](/integrations/lancedb/) * Querying: [Transforming queries](/querying/transforming-queries/), [Retrieval and search strategies](/querying/retrieval-and-search-strategies/), [Answering](/querying/answering/) * Models: [OpenAI-compatible endpoints](/integrations/openai-compatible/), [Errors, backoff and usage](/models/errors-backoff-and-usage/) ## Reference [Section titled “Reference”](#reference) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [`ChunkerTransformer`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChunkerTransformer.html) * [`SupportedLanguages`](https://docs.rs/swiftide/0.34/swiftide/integrations/treesitter/enum.SupportedLanguages.html) * [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`Redb`](https://docs.rs/swiftide/0.34/swiftide/integrations/redb/struct.Redb.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`query::Query`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Query.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`BackoffConfiguration`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.BackoffConfiguration.html) ## Next [Section titled “Next”](#next) [Release notes agent](/showcases/release-notes-agent/) # Docs Q&A with evaluation > A retrieval-augmented generation app over markdown docs with a custom loader, dense and sparse embeddings in Qdrant, hybrid search, local reranking, a Ragas dataset export, and claim-based scores from a model judge. This tour builds `docs-qa`, a retrieval-augmented generation (RAG) app with three commands: `index`, `ask` and `eval`. It reads markdown files or an `llms-full.txt`, and every answer lists the pages and sections it came from. The `eval` command prints a score table that you can rerun after each retrieval change. The crate lives in the swiftide repository under `examples/docs-qa`. Run it with `cargo run -p docs-qa` at the repository root. ## Prerequisites [Section titled “Prerequisites”](#prerequisites) * Rust 1.97.1, which the swiftide workspace pins in `rust-toolchain.toml`. * Qdrant in Docker. Swiftide talks to Qdrant over gRPC, so expose port 6334 as well as 6333. `QDRANT_URL` (default `http://localhost:6334`) and `QDRANT_API_KEY` point the client elsewhere. * `OPENAI_API_KEY` for any OpenAI-compatible endpoint. `OPENAI_BASE_URL` defaults to `https://api.openai.com/v1` and `OPENAI_MODEL` to `gpt-5.4-mini`. Embeddings and reranking run locally; only model calls go to this endpoint. * About 2.3GB of disk for the FastEmbed models, downloaded from Hugging Face into `.fastembed_cache` on first use. Set `FASTEMBED_CACHE_DIR` to share one cache. * Optional: `LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY` and `LANGFUSE_URL` for `--langfuse`. The crate turns off default features and enables the feature flags `openai`, `fastembed`, `qdrant` and `langfuse`. The excerpts derive `schemars::JsonSchema`, and the loader fetches an `llms-full.txt` URL with `reqwest`. To build the same thing in your own crate: ```sh cargo add swiftide@0.34 --no-default-features --features openai,fastembed,qdrant,langfuse cargo add schemars@1 --features derive cargo add reqwest@0.13 --no-default-features --features rustls ``` TLS needs no setup, because swiftide’s HTTP clients bring their own rustls crypto provider ([TLS](/integrations/overview/#tls)). ## Architecture [Section titled “Architecture”](#architecture) `index` writes chunks into one Qdrant collection. `ask` and `eval` build the same query pipeline over that collection; `eval` only adds an evaluator. ![Indexing and query pipelines](/diagrams/diagram-pipeline.svg) ## Custom loader [Section titled “Custom loader”](#custom-loader) examples/docs-qa/src/loader.rs ```rust /// Yields one [`TextNode`] per documentation page, with the page title as metadata. #[derive(Debug, Clone)] pub struct DocsLoader { sources: Vec, } impl DocsLoader { pub fn new(sources: Vec) -> Self { Self { sources } } } impl Loader for DocsLoader { type Output = String; fn into_stream(self) -> IndexingStream { stream::iter(self.sources) .then(DocsSource::pages) .map_ok(|pages| stream::iter(pages).map(|page| Ok(page_node(page)))) .try_flatten() .boxed() .into() } // Only needed when the loader is used as a `Box`. fn into_stream_boxed(self: Box) -> IndexingStream { self.into_stream() } } fn page_node(page: Page) -> TextNode { let mut node = TextNode::new(page.markdown); node.path = page.origin.into(); node.metadata.insert(PAGE_TITLE, page.title.as_str()); node } ``` Source: [`examples/docs-qa/src/loader.rs#L74-L109`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/loader.rs#L74-L109) A loader is the start of an indexing pipeline. `Loader` has one associated type, `Output`, and `into_stream` returns an `IndexingStream` of nodes. Here every page becomes one `TextNode`, with its origin as `node.path` and its title under the `Page` metadata key. A futures stream becomes an `IndexingStream` with `.boxed().into()`. `into_stream_boxed` matters only when you use the loader as a `Box`. The default implementation panics, so implement it on the concrete type. `DocsSource` decides how a page is read. `MarkdownDir` walks a directory with `FileLoader` and respects `.gitignore`. `MarkdownFile` reads one file. `LlmsFullFile` and `LlmsFullUrl` split an `llms-full.txt` into one page per top-level heading. ## Hybrid indexing [Section titled “Hybrid indexing”](#hybrid-indexing) examples/docs-qa/src/index.rs ```rust indexing::Pipeline::from_loader(loader) .then_chunk(ChunkMarkdown::from_chunk_range( MIN_CHUNK_CHARS..MAX_CHUNK_CHARS, )) // A short title and keywords per chunk; both end up in the embedded text and help the // sparse (keyword) side of hybrid search in particular. .then( MetadataTitle::from_client(llm.clone()) .prompt_template(TITLE_PROMPT) .build()?, ) .then( MetadataKeywords::from_client(llm.clone()) .prompt_template(KEYWORDS_PROMPT) .build()?, ) .then_in_batch(Embed::new(embedders.dense.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_in_batch(SparseEmbed::new(embedders.sparse.clone()).with_batch_size(EMBED_BATCH_SIZE)) .then_store_with(qdrant.clone()) .run() .await ``` Source: [`examples/docs-qa/src/index.rs#L50-L70`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/index.rs#L50-L70) `ChunkMarkdown::from_chunk_range(120..1500)` splits each page into chunks of at most 1,499 characters, because the range end is exclusive. It drops chunks whose trimmed text is under 120 bytes, which are mostly badges and stray headings. `MetadataTitle` stores a title under `Title` and `MetadataKeywords` stores keywords under `Keywords`, so indexing makes two large language model (LLM) calls per chunk. Two batch transformers follow: `Embed` writes the dense vector and `SparseEmbed` the sparse one, both from local FastEmbed models. `index` returns the `PipelineStats` that `run` produces. The CLI prints the chunk count of the whole collection instead, which also covers earlier runs. The metadata transformers use their own Tera templates: examples/docs-qa/src/index.rs ```rust // Swiftide's default metadata prompts show the expected format inside a code fence, and models // tend to copy the fence into the metadata. These ask for plain text and use the page title. // A code-only chunk is titled by the API it shows, because the reranker scores the title too. const TITLE_PROMPT: &str = "Write a short, descriptive title (at most 10 words) for this \ excerpt from the \"{{ node.metadata.Page }}\" documentation. If the excerpt is mostly code, \ name the API it demonstrates (a macro, trait or builder method), not what the example does. \ Reply with the title only: no quotes, no markdown, no code fences. {{ node.chunk }}"; const KEYWORDS_PROMPT: &str = "List up to 8 keywords for this excerpt from the \ \"{{ node.metadata.Page }}\" documentation: names of types, crates, features, tools, commands \ and concepts that literally appear in it. Reply with one comma-separated line only: no \ numbering, no markdown, no code fences. {{ node.chunk }}"; ``` Source: [`examples/docs-qa/src/index.rs#L24-L39`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/index.rs#L24-L39) The default prompts show the expected format inside a code fence, and models tend to copy the fence. These templates ask for plain text and read the page title from `node.metadata.Page`. A code-only chunk is titled after the API it shows, because the reranker scores the title too. ### Store [Section titled “Store”](#store) examples/docs-qa/src/store.rs ```rust let qdrant = Qdrant::builder() .batch_size(BATCH_SIZE) .vector_size(DENSE_VECTOR_SIZE) // One dense and one sparse vector, both over the chunk *and* its metadata .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name(&collection.0) .build()?; ``` Source: [`examples/docs-qa/src/store.rs#L38-L45`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/store.rs#L38-L45) The collection holds one dense and one sparse vector per chunk. Both use `EmbeddedField::Combined`, which embeds the chunk together with its metadata, so the title and keywords reach the keyword side of the search. The vector size is 384 because the dense model is BGE small, the default of `FastEmbed::try_default`. The sparse model comes from `FastEmbed::try_default_sparse`, which loads SPLADE. ## Hybrid search and reranking [Section titled “Hybrid search and reranking”](#hybrid-search-and-reranking) examples/docs-qa/src/query.rs ```rust query::Pipeline::from_search_strategy( HybridSearch::default() .with_top_n(CANDIDATES) .with_top_k(CANDIDATES), ) ``` Source: [`examples/docs-qa/src/query.rs#L108-L112`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L108-L112) `HybridSearch` defaults to 10 for both `top_n` and `top_k`. `with_top_n` and `with_top_k` take the strategy by value and return it, so they chain onto `default()` and the result goes straight into the pipeline. examples/docs-qa/src/query.rs ```rust pipeline // Hybrid search needs both a dense and a sparse embedding of the question .then_transform_query(query_transformers::Embed::from_client( self.embedders.dense.clone(), )) .then_transform_query(query_transformers::SparseEmbed::from_client( self.embedders.sparse.clone(), )) .then_retrieve(self.qdrant.clone()) // A local cross-encoder reorders the fused candidates; only the best few remain .then_transform_response(self.reranker.clone()) .then_answer(self.answerer.clone()) ``` Source: [`examples/docs-qa/src/query.rs#L121-L134`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L121-L134) Qdrant’s hybrid retriever needs both a dense and a sparse embedding of the question and fails without either. It runs a dense and a sparse search, each limited to `top_n`, fuses them with reciprocal rank fusion (RRF), and returns `top_k` documents: 20 candidates here. The reranker is a response transformer. It runs between retrieval and the answer generator: examples/docs-qa/src/query.rs ```rust let reranker = Rerank::builder() .top_k(CONTEXT_CHUNKS) .document_template(Some(RERANK_TEMPLATE.to_string())) .build()?; let answerer = answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(DOCUMENT_TEMPLATE) .build()?; ``` Source: [`examples/docs-qa/src/query.rs#L73-L82`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L73-L82) `Rerank` scores each candidate against the original question with a local cross-encoder and keeps `top_k` of them (default 10). With a `document_template`, each document is rendered with Tera first, with `metadata` and `content` available. `answers::Simple` renders every kept document with its own `document_template` and joins them with `---` lines into `{{ documents }}`. Its `prompt_template` takes a `Prompt`, hence the `.into()`. examples/docs-qa/src/query.rs ```rust // The answer prompt receives the question and the rendered documents; each document is rendered // with its page title and the chunk title written during indexing. const ANSWER_PROMPT: &str = "You answer questions about Swiftide, a Rust library for building \ LLM applications, using excerpts from its documentation. Question: {{ question }} ## Instructions * Use only the documentation excerpts below. If they do not answer the question, say so plainly. * Start with the direct answer in one or two sentences, then add supporting details. * Only when the question asks how to do something, include the feature flags, commands or a \ short Rust snippet from the excerpts. Never invent APIs. * Format the answer as markdown and keep it under 250 words. Do not refer to \"the excerpts\". ## Documentation excerpts {{ documents }} "; const DOCUMENT_TEMPLATE: &str = "### {{ metadata.Page | default(value=\"\") }}: \ {{ metadata.Title | default(value=\"\") }} {{ content }}"; // The reranker scores the same title-first rendering. A chunk that is mostly code says little // about what it shows, so its page and the title written during indexing go first. const RERANK_TEMPLATE: &str = "{{ metadata.Page | default(value=\"\") }}: \ {{ metadata.Title | default(value=\"\") }} {{ content }}"; ``` Source: [`examples/docs-qa/src/query.rs#L30-L57`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L30-L57) ### Qdrant content workaround [Section titled “Qdrant content workaround”](#qdrant-content-workaround) Qdrant retrieval turns the stored `content` payload into a string with `to_string` on Qdrant’s own `Value` type, which formats strings with `Debug`. Every chunk then comes back wrapped in quotes, with escaped newlines. The example wraps the store in `CleanQdrant` (`src/retrieval.rs`), which implements `Retrieve` for `HybridSearch`, calls Qdrant, and restores the original text. The reranker, the LLM and the evaluator then see clean markdown. ## Evaluation [Section titled “Evaluation”](#evaluation) examples/docs-qa/src/eval.rs ```rust let ragas = Ragas::from_prepared_questions(&questions); let answered = qa .evaluated_pipeline(ragas.clone()) .query_all(ragas.questions().await) .await?; let ragas_json = ragas.to_json().await; ``` Source: [`examples/docs-qa/src/eval.rs#L89-L96`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/eval.rs#L89-L96) Retrieval-Augmented Generation Assessment (Ragas) is a Python toolkit. Swiftide’s `Ragas` evaluator only exports data for it and computes no scores. `Ragas::from_prepared_questions` takes anything that converts into an `EvaluationDataSet`. The example implements that conversion for `&QuestionSet`, which is why the snippet passes `&questions`. `qa.evaluated_pipeline` builds the same query pipeline and attaches the evaluator with `evaluate_with`. `query_all` runs every question, and `to_json` exports the dataset with `question`, `answer`, `contexts` and `ground_truth`. The evaluator records contexts at retrieval, before any response transformer. `ragas.json` therefore holds all 20 candidates per question, not the 10 the reranker kept. ### LLM judge [Section titled “LLM judge”](#llm-judge) To print scores without leaving Rust, a judge splits the ground truth and the answer into claims and checks each one: examples/docs-qa/src/judge.rs ```rust #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct Judgement { reference_claims: Vec, answer_claims: Vec, } #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct ReferenceClaim { // Never read; asking for the claim first makes the model commit to it before judging it. #[expect(dead_code)] claim: String, in_answer: bool, in_context: bool, } #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct AnswerClaim { #[expect(dead_code)] claim: String, supported_by_context: bool, } ``` Source: [`examples/docs-qa/src/judge.rs#L91-L114`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/judge.rs#L91-L114) examples/docs-qa/src/judge.rs ```rust let contexts = answer.contexts().collect::>().join("\n---\n"); let prompt = Prompt::from(JUDGE_PROMPT) .with_context_value("question", answer.question()) .with_context_value("ground_truth", ground_truth) .with_context_value("answer", answer.text()) .with_context_value("contexts", contexts); // The JSON schema of `Judgement` is sent along; the reply is parsed into it let judgement: Judgement = retry_transient(|| self.llm.structured_prompt(prompt.clone())).await?; ``` Source: [`examples/docs-qa/src/judge.rs#L154-L163`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/judge.rs#L154-L163) `structured_prompt` sends the JSON schema of `Judgement` with the prompt and parses the reply into it. The scores are plain ratios over the claims: faithful (answer claims the reranked context supports), answer recall and context recall. `LanguageModelWithBackOff` does not wrap `StructuredPrompt`, so the judge calls the plain `OpenAI` client and retries transient errors itself, up to five attempts. ## Model client [Section titled “Model client”](#model-client) examples/docs-qa/src/models.rs ```rust const DEFAULT_BASE_URL: &str = "https://api.openai.com/v1"; /// A chat model that retries rate limits and other transient errors with exponential backoff. /// Indexing makes two calls per chunk, concurrently, so it will hit rate limits. pub type RetryingModel = LanguageModelWithBackOff; /// An OpenAI-compatible API: `OpenAI` itself, or any gateway or local server that speaks it. #[derive(Clone)] pub struct LlmEndpoint { base_url: String, api_key: String, use_responses_api: bool, } impl LlmEndpoint { /// Reads `OPENAI_BASE_URL` (optional, defaults to `OpenAI`) and `OPENAI_API_KEY` (required). pub fn from_env(use_responses_api: bool) -> Result { let api_key = non_empty_env("OPENAI_API_KEY") .context("Set OPENAI_API_KEY to the API key of your OpenAI-compatible endpoint")?; let base_url = non_empty_env("OPENAI_BASE_URL").unwrap_or_else(|| DEFAULT_BASE_URL.to_string()); Ok(Self { base_url, api_key, use_responses_api, }) } /// A client for `model` on this endpoint. pub fn model(&self, model: &str) -> Result { Ok(OpenAI::builder() .api_base(&self.base_url) .api_key(&self.api_key) .default_prompt_model(model) .use_responses_api(self.use_responses_api) .build()?) } /// The same client, wrapped in Swiftide's backoff decorator. An unreachable endpoint gives /// up after two minutes. pub fn retrying_model(&self, model: &str) -> Result { let backoff = BackoffConfiguration { initial_interval_sec: 5, max_elapsed_time_sec: 120, ..BackoffConfiguration::default() }; Ok(LanguageModelWithBackOff::new(self.model(model)?, backoff)) } } ``` Source: [`examples/docs-qa/src/models.rs#L30-L79`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/models.rs#L30-L79) Answer and metadata calls go through `LanguageModelWithBackOff`, as in [Ask your codebase](/showcases/ask-your-codebase/#model-client). Here it retries with a longer backoff than the defaults (1 s start, 60 s cap). ## Tracing [Section titled “Tracing”](#tracing) examples/docs-qa/src/telemetry.rs ```rust let terminal = tracing_subscriber::fmt::layer() .compact() .with_writer(std::io::stderr) .with_filter(log_filter()); // Reads LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY and LANGFUSE_URL let langfuse = send_to_langfuse.then(LangfuseLayer::default); tracing_subscriber::registry() .with(terminal) // Langfuse data is emitted at `debug`, so its layer gets its own, wider filter .with( langfuse .clone() .map(|layer| layer.with_filter(LevelFilter::DEBUG)), ) .try_init()?; ``` Source: [`examples/docs-qa/src/telemetry.rs#L30-L46`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/telemetry.rs#L30-L46) The Langfuse layer is set up the same way as in the [release notes agent](/showcases/release-notes-agent/#tracing). Langfuse data is emitted at debug level, so its layer gets its own debug-level filter. ## Commands [Section titled “Commands”](#commands) ```sh git clone https://github.com/bosun-ai/swiftide && cd swiftide docker run -d --name docs-qa-qdrant -p 6333:6333 -p 6334:6334 qdrant/qdrant export OPENAI_API_KEY=sk-... cargo run -p docs-qa -- index --recreate \ README.md CONTRIBUTING.md AGENTS.md swiftide-agents-contrib/README.md cargo run -p docs-qa -- ask "How do I give an agent tools to read and edit files?" cargo run -p docs-qa -- eval --out examples/docs-qa/ragas.json ``` | Flag | Where | Default | What it does | | ------------------ | ------- | ------------------------------- | ------------------------------------------------ | | `--model` | global | `gpt-5.4-mini` (`OPENAI_MODEL`) | Answers questions; default for the other models | | `--responses-api` | global | off | Responses API instead of chat completions | | `--collection` | global | `docs-qa` | Qdrant collection | | `--langfuse` | global | off | Send traces to Langfuse (`LANGFUSE_URL`) | | `--recreate` | `index` | off | Drop the collection first | | `--metadata-model` | `index` | `--model` | Writes titles and keywords; pick a cheaper model | | `--questions` | `eval` | bundled `questions.yaml` | Questions with ground truths | | `--out` | `eval` | `ragas.json` | Where the dataset goes | | `--judge-model` | `eval` | `--model` | Grades the answers | ## Output [Section titled “Output”](#output) The captured run used an OpenAI-compatible gateway in front of AWS Bedrock. `claude-haiku-5-5` wrote titles and keywords, and `claude-sonnet-5-5` wrote the answers and acted as judge. FastEmbed ran BGE small en v1.5, SPLADE++ and BGE reranker base locally. ```text Indexed 43 chunks into `docs-qa` in 71.7s ``` The answer below describes the agents-contrib file tools, which shipped in 0.34. The CLI prints the answer followed by its sources. ```text Enable the `agents-contrib-file-tools` feature on `swiftide`, then register `ReadFile`, `ExactReplace`, and `ApplyPatch` on your agent with `add_tool`. ... **Requirement:** the executor must provide a POSIX-like shell with `cat`, `mkdir`, and `rm` commands. Sources: [1] Swiftide › Agent Tools and Hooks: Enabling Contrib File, Planning, and Context Features (README.md) [2] swiftide-agents-contrib › read_file tool: arguments, defaults, and offset/limit behavior (swiftide-agents-contrib/README.md) [3] swiftide-agents-contrib › File Tools: Line-Numbered Reads and Exact-Match Edits (swiftide-agents-contrib/README.md) ... [10] Swiftide › Swiftide capabilities for agents, typed task graphs, pipelines, and tracing (README.md) ``` ```text # question ctx faithful answer context recall recall 1 Which crate features and environment variable do I … 10 1.00 1.00 1.00 2 What does the agents-contrib-context feature provid… 10 1.00 1.00 1.00 3 What are the default limits of the read_file tool? 10 1.00 1.00 1.00 4 Which storage and retrieval backends does Swiftide … 10 1.00 1.00 1.00 5 What should I run before opening a pull request? 10 1.00 1.00 1.00 6 What is a task in Swiftide? 10 1.00 1.00 1.00 7 When does ContextCompression summarize the conversa… 10 1.00 1.00 1.00 8 Which commit message convention does the project fo… 10 1.00 1.00 1.00 9 How do I add a custom tool to an agent with the too… 10 1.00 1.00 1.00 mean 1.00 1.00 1.00 RAGAS dataset written to examples/docs-qa/ragas.json ``` `ctx` is the number of chunks the LLM saw after reranking. ## Gotchas [Section titled “Gotchas”](#gotchas) * A chunk that is mostly code ranks poorly when the reranker scores only its text. Render page and chunk title into the reranker’s `document_template`, and title code-only chunks after the API they show. * The scores come from one judge over nine questions. Treat them as a regression signal for this corpus, not as a benchmark. * `--responses-api` failed against the gateway used for the captured run, which returns a field the `async-openai` client rejects. * Ragas exports rows in hash-map order, so `eval` sorts them back into question order before writing. ## Concept pages [Section titled “Concept pages”](#concept-pages) * Indexing: [Loading](/indexing/loading/), [Chunking](/indexing/chunking/), [Transformers and metadata](/indexing/transformers-and-metadata/), [Embedding](/indexing/embedding/) * Storage: [Storage](/indexing/storage/), [Qdrant](/integrations/qdrant/), [FastEmbed](/integrations/fastembed/) * Querying: [Retrieval and search strategies](/querying/retrieval-and-search-strategies/), [Reranking and response transformers](/querying/reranking-and-response-transformers/), [Answering](/querying/answering/), [Evaluation](/querying/evaluation/) * Models and tracing: [Structured output](/models/structured-output/), [Langfuse](/observability/langfuse/) * Recipe: [Hybrid search and reranking](/cookbook/hybrid-search-and-reranking/) ## Reference [Section titled “Reference”](#reference) * [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`TextNode`](https://docs.rs/swiftide/0.34/swiftide/indexing/type.TextNode.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`IndexingStream`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.IndexingStream.html) * [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html) * [`MetadataTitle`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataTitle.html) * [`MetadataKeywords`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataKeywords.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) and [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) (indexing) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Rerank`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.Rerank.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html) and [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.SparseEmbed.html) (query) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`Ragas`](https://docs.rs/swiftide/0.34/swiftide/query/evaluators/ragas/struct.Ragas.html) and [`EvaluationDataSet`](https://docs.rs/swiftide/0.34/swiftide/query/evaluators/ragas/struct.EvaluationDataSet.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`BackoffConfiguration`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.BackoffConfiguration.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`LangfuseLayer`](https://docs.rs/swiftide/0.34/swiftide/langfuse/struct.LangfuseLayer.html) ## Next [Section titled “Next”](#next) [Agent control recipes](/cookbook/agent-control/) # Release notes agent > An agent that reads git history through scoped tools, streams a release-notes draft, waits for your approval, and ends with structured notes. You end up with a command-line agent that drafts release notes for the commits since your last tag. It streams the draft to your terminal and writes `RELEASE_NOTES.md` only after you answer `y`. The code lives in `examples/release-notes-agent` of the swiftide repository. ## Prerequisites [Section titled “Prerequisites”](#prerequisites) * Rust 1.97.1 and a git repository with at least one tag. * `OPENAI_API_KEY`. Set `OPENAI_BASE_URL` to use any OpenAI-compatible endpoint instead of `https://api.openai.com/v1`. * For `--mcp`: Node.js (`npx`) and `GITHUB_PERSONAL_ACCESS_TOKEN` or `GITHUB_TOKEN`. * For `--trace`: `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY`, and optionally `LANGFUSE_URL`. The CLI also reads variables from a `.env` file, found in the current directory or the nearest parent, or from the file you pass with `--env-file`. TLS needs no setup, because swiftide’s HTTP clients bring their own rustls crypto provider ([TLS](/integrations/overview/#tls)). The crate turns off default features and enables the feature flags `macros`, `swiftide-agents`, `openai`, `mcp` and `langfuse`. The excerpts below also need `async-trait`, because `#[derive(Tool)]` expands to `#[async_trait]`, plus `schemars` and `rmcp`. The model client sets the endpoint with `api_base` and `api_key` on the `OpenAI` builder, which arrived in 0.34. To build the same thing in your own crate: ```sh cargo add swiftide@0.34 --no-default-features --features macros,swiftide-agents,openai,mcp,langfuse cargo add async-trait schemars --features schemars/derive cargo add rmcp --features client,transport-child-process ``` The example itself builds against the swiftide workspace through a path dependency. ## Architecture [Section titled “Architecture”](#architecture) ![Agent loop](/diagrams/diagram-agent-loop.svg) The app has one agent and one tool executor. The agent reads git history with three tools, drafts the notes, and calls `write_changelog`. That tool waits for approval, which the session loop collects outside the agent. A custom `stop` tool ends the run with the notes as structured data. ## Tool executor [Section titled “Tool executor”](#tool-executor) examples/release-notes-agent/src/main.rs ```rust let root = Git::new(&LocalExecutor::new(&cli.repo)) .top_level() .await .with_context(|| format!("{} is not a git repository", cli.repo.display()))?; // Every tool the agent runs, runs with the repository as its working directory. let executor: Arc = Arc::new( LocalExecutor::builder() .workdir(&root) .default_timeout(Some(Duration::from_mins(1))) .build()?, ); ``` Source: [`examples/release-notes-agent/src/main.rs#L93-L103`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/main.rs#L93-L103) `LocalExecutor` runs every command with the repository root as its working directory. A command without its own timeout is stopped after one minute. It does not sandbox anything: absolute paths and `..` still reach outside the repository. The tools therefore parse revisions and paths from the model into value types, `Revision` and `RepoPath`, before they build a command. The output path comes from the CLI, not from the model. ## Tools [Section titled “Tools”](#tools) examples/release-notes-agent/src/tools.rs ```rust /// Lists every commit since a release tag, oldest first: hash, subject, body and the files it /// changed. /// /// # Arguments /// - `tag`: Tag to start from, e.g. `v0.33.0`. Use null for the latest tag. #[swiftide::tool] pub async fn git_log_since( context: &dyn AgentContext, tag: &Option, ) -> Result { let git = Git::new(context.executor().as_ref()); let since = match tag { Some(tag) => tag.parse()?, None => git.latest_tag().await?, }; let log = git.log_since(&since).await?; Ok(ToolOutput::text(clip( log, MAX_LOG_LINES, "Use git_diff on a specific path for details.", ))) } /// Shows the diff of one file or directory over a revision range. /// /// # Arguments /// - `range`: Revision range, e.g. `v0.33.0..HEAD`, or a single commit hash. /// - `path`: File or directory relative to the repository root. #[swiftide::tool] pub async fn git_diff( context: &dyn AgentContext, range: &str, path: &str, ) -> Result { let (range, path): (Revision, RepoPath) = (range.parse()?, path.parse()?); let diff = Git::new(context.executor().as_ref()) .diff(&range, &path) .await?; if diff.trim().is_empty() { return Ok(ToolOutput::text(format!( "No changes to {path} in {range}." ))); } Ok(ToolOutput::text(clip( diff, MAX_DIFF_LINES, "Diff a narrower path to see the rest.", ))) } /// Reads a file from the repository, e.g. to check a public API or the style of the changelog. /// /// # Arguments /// - `path`: File path relative to the repository root. #[swiftide::tool] pub async fn read_file(context: &dyn AgentContext, path: &str) -> Result { let path: RepoPath = path.parse()?; let content = context .executor() .exec_cmd(&Command::read_file(path.as_path())) .await?; Ok(ToolOutput::text(clip( content.into_string_lossy(), MAX_FILE_LINES, "The file is longer than shown.", ))) } ``` Source: [`examples/release-notes-agent/src/tools.rs#L32-L101`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/tools.rs#L32-L101) `#[swiftide::tool]` turns each async function into a tool. It also generates a constructor with the same name that returns a boxed tool, which is why the agent calls `git_log_since()`. The tool description comes from the doc comment, and the parameter descriptions come from its `# Arguments` list. Every tool sends its `Command` through `executor()` on the context, so all commands share the executor above. The git queries run as shell commands. `Command::read_file` and `Command::write_file` go straight to the filesystem. ### Stateful tool [Section titled “Stateful tool”](#stateful-tool) examples/release-notes-agent/src/tools.rs ```rust /// Prepends the approved release notes to the release notes file. /// /// The file is fixed by the CLI (`--output`), never chosen by the model. Wrap this tool in /// `ApprovalRequired` so a human signs off before anything is written. #[derive(Clone, Tool)] #[tool( description = "Prepends the finished release notes to the release notes file. A maintainer \ reviews the notes before they are written.", param( name = "markdown", description = "The complete release notes for this version, as GitHub flavored Markdown" ) )] pub struct WriteChangelog { file: PathBuf, } impl WriteChangelog { pub fn new(file: impl Into) -> Self { Self { file: file.into() } } async fn write_changelog( &self, context: &dyn AgentContext, markdown: &str, ) -> Result { let executor = context.executor(); // A missing file is started; any other failure must not overwrite the existing notes. let existing = if file_exists(executor.as_ref(), &self.file).await? { executor .exec_cmd(&Command::read_file(&self.file)) .await .map(CommandOutput::into_string_lossy)? } else { String::new() }; executor .exec_cmd(&Command::write_file( &self.file, prepend_notes(markdown, &existing), )) .await?; Ok(ToolOutput::text(format!("wrote {}", self.file.display()))) } } ``` Source: [`examples/release-notes-agent/src/tools.rs#L105-L152`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/tools.rs#L105-L152) The `Tool` derive macro builds a tool from a struct. Here the struct holds the output file, which the CLI fixes, so the model cannot choose where to write. The macro calls the method named after the tool, `write_changelog`, and generates the argument type `WriteChangelogArgs`. The session deserializes the pending call into that type to report how many lines the draft has and where it goes. ## System prompt [Section titled “System prompt”](#system-prompt) examples/release-notes-agent/src/agent.rs ```rust /// The system prompt is a Tera template; optional values are left undefined when absent. /// /// GitHub tools are only mentioned when the repository is on GitHub, since the prompt tells /// the agent which repository to query. fn system_prompt(&self, github_tools: bool) -> Prompt { let mut prompt = Prompt::from(SYSTEM_PROMPT) .with_context_value("since", self.since.as_str()) .with_context_value("github_tools", github_tools && self.github.is_some()); if let Some(version) = &self.next_version { prompt.insert_context_value("next_version", version.as_str()); } if let Some(github) = &self.github { prompt.insert_context_value("github", github.to_string()); } prompt } ``` Source: [`examples/release-notes-agent/src/agent.rs#L69-L84`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/agent.rs#L69-L84) The system prompt is a `Prompt`, a Tera template read from `src/system_prompt.md`. Since 0.33, `Prompt` replaces `SystemPrompt`; see [Upgrading](/reference/upgrading/). `with_context_value` fills a template variable. ## Agent [Section titled “Agent”](#agent) examples/release-notes-agent/src/agent.rs ```rust /// Builds the agent. All tools run through `executor`, whose working directory is the repo. pub fn build( llm: &(impl ChatCompletion + Clone + 'static), executor: Arc, task: &ReleaseTask, github_tools: Option, ) -> Result { let mut builder = Agent::builder(); builder .llm(llm) .context(DefaultContext::from_executor(executor)) .system_prompt(task.system_prompt(github_tools.is_some())) // Our own `stop` tool replaces the default one; call this before `tools`. .without_default_stop_tool() .tools([ git_log_since(), git_diff(), read_file(), // Drafts that describe a change twice go back to the model before anyone reviews them. OneEntryPerChange::new(ApprovalRequired::new(WriteChangelog::new(&task.notes_file))) .into(), ReleaseNotes::stop_tool().into(), ]) .limit(STEP_LIMIT); if let Some(github_tools) = github_tools { builder.add_toolbox(github_tools); } progress::report_tool_calls(&mut builder); progress::stream_draft(&mut builder); Ok(builder.build()?) } ``` Source: [`examples/release-notes-agent/src/agent.rs#L104-L136`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/agent.rs#L104-L136) `DefaultContext::from_executor` gives the agent’s context the executor, so tools get it through `executor()`. Call `without_default_stop_tool` before `tools`, because `tools` adds the default `stop` tool at the moment you call it. The `write_changelog` tool sits inside two wrappers. `ApprovalRequired` holds the call until you decide. `OneEntryPerChange`, defined in the example, sends a draft that lists one pull request twice back to the model before you see it. `limit(STEP_LIMIT)` caps one run at 40 completions. ## Hooks and streaming [Section titled “Hooks and streaming”](#hooks-and-streaming) examples/release-notes-agent/src/progress.rs ```rust /// Prints each tool call and its result on one line. pub fn report_tool_calls(builder: &mut AgentBuilder) { builder .before_tool(|_agent, tool_call| { println!(" → {}", describe_call(tool_call)); Box::pin(async { Ok(()) }) }) .after_tool(|_agent, tool_call, result| { println!(" {} {}", tool_call.name(), describe_result(result)); Box::pin(async { Ok(()) }) }); } ``` Source: [`examples/release-notes-agent/src/progress.rs#L16-L27`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/progress.rs#L16-L27) Since 0.33, `before_tool` and `after_tool` receive `&mut Agent`. `after_tool` also gets the tool result as `&mut`, so a hook could rewrite it. examples/release-notes-agent/src/progress.rs ```rust /// Streams the model's text, and the release notes while they are being drafted. pub fn stream_draft(builder: &mut AgentBuilder) { let preview = Arc::new(Mutex::new(DraftPreview::default())); // Registering `on_stream` switches the agent to streaming completions. builder.on_stream(move |_agent, response| { let Some(delta) = &response.delta else { return Box::pin(async { Ok(()) }); }; let mut out = stdout().lock(); if let Some(text) = &delta.message_chunk { let _ = write!(out, "{text}"); } // The notes arrive as the (JSON) argument of `write_changelog`; show them as they // are written. let drafts = delta.tool_calls_chunk.iter().flat_map(HashMap::values); for call in drafts.filter(|call| call.name.as_deref() == Some(WRITE_CHANGELOG)) { if let (Some(id), Some(args)) = (&call.id, &call.arguments) { let mut preview = preview.lock().unwrap_or_else(PoisonError::into_inner); if let Some(text) = preview.advance(id, args) { let _ = write!(out, "{text}"); } } } let _ = out.flush(); Box::pin(async { Ok(()) }) }); } ``` Source: [`examples/release-notes-agent/src/progress.rs#L31-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/progress.rs#L31-L58) Registering `on_stream` turns on streaming completions for the agent. The notes arrive as the JSON argument of `write_changelog`, so the hook reads `tool_calls_chunk` and decodes the argument as it grows. The client in `src/llm.rs` sets `stream_full(false)` on the OpenAI builder. Each chunk then leaves out the accumulated message, tool calls and usage. `message_chunk` holds only the new text, but `tool_calls_chunk` still holds the argument accumulated so far. ## Structured stop [Section titled “Structured stop”](#structured-stop) examples/release-notes-agent/src/notes.rs ```rust /// Structured release notes. The agent hands these over when it calls `stop`. #[derive(Debug, Serialize, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] pub struct ReleaseNotes { /// The version being released, without a leading `v`. pub version: String, /// One or two sentences on what this release is about, written for users. pub summary: String, /// New features and notable improvements. pub highlights: Vec, /// Changes that require users to update their code. Empty if there are none. pub breaking_changes: Vec, /// Bug fixes users may notice. pub fixes: Vec, } #[derive(Debug, Serialize, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] pub struct Item { /// Short, user-facing title. pub title: String, /// What changed and why it matters, in one or two sentences. pub details: String, /// Pull requests or commits as plain text, e.g. `#1207` or `90bcadc7`, not Markdown links. pub references: Vec, } impl ReleaseNotes { /// A `stop` tool whose arguments must match [`ReleaseNotes`]. /// /// With a custom schema the arguments are forwarded as-is, and end up in /// `StopReason::RequestedByTool`. pub fn stop_tool() -> StopWithArgs { StopWithArgs::with_parameters_schema(schema_for!(ReleaseNotes)) } } ``` Source: [`examples/release-notes-agent/src/notes.rs#L12-L47`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/notes.rs#L12-L47) `StopWithArgs::with_parameters_schema` creates a tool named `stop` whose arguments must match the schema of `ReleaseNotes`. With a custom schema the whole argument payload is forwarded, without an `output` wrapper. It arrives in `StopReason::RequestedByTool`, where the session deserializes it into `ReleaseNotes`. `render` in `src/notes.rs` prints that value as the summary at the end of a run. ## Approval [Section titled “Approval”](#approval) examples/release-notes-agent/src/session.rs ```rust /// Asks the agent for release notes and loops through review rounds until they are written or /// rejected. pub async fn draft_release_notes( agent: &mut Agent, task: Prompt, reviewer: &mut impl Reviewer, ) -> Result { agent.query(task).await?; let mut written = false; let mut nudged = false; loop { // `write_changelog` is wrapped in `ApprovalRequired`: the agent stops and waits. let tool_call = match agent.stop_reason() { Some(StopReason::FeedbackRequired { tool_call, .. }) => tool_call.clone(), // The custom `stop` tool carries the structured release notes. Some(StopReason::RequestedByTool(_, Some(notes))) if written => { let notes = ReleaseNotes::deserialize(notes) .context("the agent stopped with malformed release notes")?; return Ok(Outcome::Written { notes: Some(notes) }); } _ if written => return Ok(Outcome::Written { notes: None }), // A turn without a tool call, or the step limit. Some(StopReason::NoNewMessages) => { nudge_once(agent, &mut nudged).await?; continue; } other => bail!("the agent stopped without writing the notes: {other:?}"), }; let proposal: WriteChangelogArgs = serde_json::from_str(tool_call.args().unwrap_or_default())?; match reviewer.review(&proposal)? { Verdict::Approve => { agent .context() .feedback_received(&tool_call, &ToolFeedback::approved()) .await?; // Resumes the pending tool call, which now runs, and continues the loop. agent.run().await?; // The write itself can still fail; the agent then hears why and carries on. written |= succeeded(agent, &tool_call).await?; } Verdict::Revise(feedback) => { agent .context() .feedback_received(&tool_call, &ToolFeedback::refused()) .await?; // The refusal is recorded as the tool's output, followed by the feedback. agent.query(feedback).await?; } Verdict::Reject => return Ok(Outcome::Rejected), } } } ``` Source: [`examples/release-notes-agent/src/session.rs#L41-L96`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/session.rs#L41-L96) `ApprovalRequired` returns `ToolOutput::FeedbackRequired` until the context holds feedback for that tool call. The agent then stops with `StopReason::FeedbackRequired`, which carries the pending tool call. To approve, record `ToolFeedback::approved()` with `feedback_received` and call `run()`. The pending call then runs the inner tool. To ask for changes, record `ToolFeedback::refused()` and send your feedback with `query()`. The model sees “This tool call was refused” as the tool output, followed by your message. The session reports success only when the approved write returned text. A write that fails goes back to the model, and the loop continues. ### Turn limit and nudges [Section titled “Turn limit and nudges”](#turn-limit-and-nudges) The step counter resets on every `query()` and `run()`. When the agent reaches the limit, it stops with `StopReason::NoNewMessages`, the same stop reason as a turn without tool calls. The session tells the two apart from the message history. After a turn without a tool call it asks the model once to continue. At the limit, or after a second empty turn, it gives up. ## MCP tools [Section titled “MCP tools”](#mcp-tools) examples/release-notes-agent/src/mcp.rs ```rust /// Starts `@modelcontextprotocol/server-github` with `npx` and exposes a few of its tools. /// /// Errors explain what is missing, so the caller can carry on without GitHub. pub async fn github_toolbox() -> Result { let token = env::var("GITHUB_PERSONAL_ACCESS_TOKEN") .or_else(|_| env::var("GITHUB_TOKEN")) .context("set GITHUB_PERSONAL_ACCESS_TOKEN (or GITHUB_TOKEN)")?; let server = Command::new("npx").configure(|cmd| { cmd.args(["-y", "@modelcontextprotocol/server-github"]) .env("GITHUB_PERSONAL_ACCESS_TOKEN", token); }); let (transport, _stderr) = TokioChildProcess::builder(server) .stderr(Stdio::null()) .spawn() .context("could not run `npx`; is Node.js installed?")?; let mut toolbox = timeout( Duration::from_mins(1), McpToolbox::try_from_transport(transport), ) .await .context("the GitHub MCP server did not start within a minute")? .context("the GitHub MCP server failed to start")?; toolbox.with_whitelist(GITHUB_TOOLS); // Swiftide names MCP tools `:`; OpenAI rejects ':', see `OpenAiToolNames` below. let toolbox = OpenAiToolNames(toolbox); // Fail early, rather than mid-run, if the server does not offer the tools we expect. let available = toolbox.available_tools().await?; if available.len() != GITHUB_TOOLS.len() { bail!("the GitHub MCP server does not offer {GITHUB_TOOLS:?}"); } Ok(toolbox) } ``` Source: [`examples/release-notes-agent/src/mcp.rs#L18-L52`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/mcp.rs#L18-L52) `--mcp` starts the GitHub Model Context Protocol (MCP) server as a child process. `McpToolbox::try_from_transport` connects to it, and `with_whitelist` keeps two read-only tools, `get_pull_request` and `get_issue`. Swiftide registers MCP tools as `:`, and OpenAI rejects `:` in tool names. The example wraps the toolbox to replace those characters with `_`, so the model sees the two tools with the server name and `_` in front. Without `npx`, a token or a GitHub `origin`, the CLI continues without these tools. ## Tracing [Section titled “Tracing”](#tracing) examples/release-notes-agent/src/telemetry.rs ```rust /// Logs warnings to stderr (override with `RUST_LOG`) and, with `trace`, sends traces to /// Langfuse. pub fn init(trace: bool) -> Result { let logs = tracing_subscriber::fmt::layer() .compact() .with_writer(io::stderr) .with_ansi(io::stderr().is_terminal()) .with_filter( EnvFilter::try_from_default_env() // The agent logs stops and approvals as warnings, and errors it also returns; // the CLI reports both itself. .unwrap_or_else(|_| EnvFilter::new("warn,swiftide_agents=off")), ); let langfuse = if trace { for key in ["LANGFUSE_PUBLIC_KEY", "LANGFUSE_SECRET_KEY"] { ensure!(env::var_os(key).is_some(), "--trace needs {key}"); } // Reads the keys and `LANGFUSE_URL` (default http://localhost:3000) from the env. Some(LangfuseLayer::default()) } else { None }; tracing_subscriber::registry() .with(logs) .with( langfuse .clone() // Langfuse data is logged at debug level; only send Swiftide's spans, not // those of every dependency. .map(|layer| { layer .with_filter(Targets::new().with_target("swiftide", LevelFilter::DEBUG)) }), ) .try_init()?; Ok(Self { langfuse }) } ``` Source: [`examples/release-notes-agent/src/telemetry.rs#L23-L62`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/release-notes-agent/src/telemetry.rs#L23-L62) `LangfuseLayer::default()` reads the keys and `LANGFUSE_URL` (default `http://localhost:3000`) from the environment. It panics when a key is missing or empty. `init` checks that the keys are set first, but an empty value still passes that check. The `Targets` filter matches by prefix, so the layer receives spans and events from every target that starts with `swiftide`, at debug level and above. `main` calls `flush` before the process exits, so buffered events reach Langfuse. ## Commands [Section titled “Commands”](#commands) Clone the repository: ```sh git clone https://github.com/bosun-ai/swiftide cd swiftide ``` From the checkout root, this drafts notes for swiftide itself and writes `RELEASE_NOTES.md` there: ```sh cargo run -p release-notes-agent -- --version 0.34.0 ``` Point it at another OpenAI-compatible endpoint, or at another repository: ```sh OPENAI_BASE_URL=http://127.0.0.1:8080/v1 OPENAI_MODEL=claude-sonnet-5-5 \ cargo run -p release-notes-agent -- --version 0.34.0 cargo run -p release-notes-agent -- --repo ../my-crate --since v1.2.0 --mcp --trace ``` Answer `y` or `yes` to write the notes. Any other answer, including an empty one, asks what should change. An empty answer to that second question stops without writing. `cargo test -p release-notes-agent` runs the revise round and the nudge against a scripted model ([Testing](/agents/testing/)), and checks the step-limit detection on a hand-built message history. | Flag | Default | | | ------------------- | --------------------- | ----------------------------------------------------------------- | | `--repo ` | `.` | The git repository. | | `--since ` | latest tag | The previous release. | | `--version ` | proposed by the agent | The version being released. | | `--output ` | `RELEASE_NOTES.md` | File the notes are prepended to, relative to the repository root. | | `--mcp` | off | Read pull requests through the GitHub MCP server. | | `--trace` | off | Send traces to Langfuse. | | `--model ` | `gpt-5.4` | Model name, also read from `OPENAI_MODEL`. | | `--responses-api` | off | Use the OpenAI Responses API instead of chat completions. | | `--env-file ` | nearest `.env` | Load environment variables from this file. | ## Output [Section titled “Output”](#output) This run used `claude-sonnet-5-5` through an OpenAI-compatible gateway in front of AWS Bedrock, with chat completions. It covers the 7 commits since v0.33.0, released as 0.34.0, with `y` piped into stdin. The tool calls in a group run in parallel, but the agent awaits them in the order they were issued, so their results print in call order. ```text Drafting release notes for the 7 commits since v0.33.0 with claude-sonnet-5-5 at http://127.0.0.1:8080/v1 (chat completions) → git_log_since v0.33.0 git_log_since ✓ 346 lines → git_diff swiftide-integrations/Cargo.toml 3b1ad22e^..3b1ad22e → git_diff …/src/chat_completion/chat_completion_response.rs 90bcadc7^..90bcadc7 ... git_diff ✓ 15 lines git_diff ✓ 72 lines ... ## 0.34.0 This release adds the opt-in `swiftide-agents-contrib` crate and Perl support for tree-sitter. It also fixes duplicated text in OpenAI streaming and resolves security advisories, which removes AWS Console login from the Bedrock client. ... → write_changelog <32 lines> write_changelog ⏸ waiting for approval ── proposed: 32 lines into a new file, RELEASE_NOTES.md Write these release notes? [y/N] y → write_changelog <32 lines> write_changelog ✓ wrote RELEASE_NOTES.md → stop (version 0.34.0: highlights 2, breaking 1, fixes 1) stop ■ done ✔ RELEASE_NOTES.md 0.34.0 — This release adds the opt-in `swiftide-agents-contrib` crate and Perl support for tree-sitter. It also fixes duplicated text in OpenAI streaming and resolves security advisories, which removes AWS Console login from the Bedrock client. highlights (2) • Agent tools, hooks and executors in `swiftide-agents-contrib` [#1207] • Perl support for tree-sitter [#1198] breaking changes (1) • AWS Console login removed from the Bedrock client [#1215] fixes (1) • Repeated text in OpenAI streaming [#1216] ``` Everything these notes describe, `swiftide-agents-contrib`, Perl tree-sitter support, the Bedrock change (#1215) and the streaming fix (#1216), shipped in 0.34. ## Gotchas [Section titled “Gotchas”](#gotchas) * The notes are a draft. Check each entry against `git show` before you approve. * `--responses-api` failed against the gateway used for this run, because the gateway streams Responses events that `async-openai` cannot parse. Use it with OpenAI itself. * `--trace` is untested against a live Langfuse; check that traces arrive before relying on them. ## Concept pages [Section titled “Concept pages”](#concept-pages) * [Tools](/agents/tools/) and [Executors](/agents/executors/) * [System prompts](/agents/system-prompts/) * [Hooks](/agents/hooks/) and [Streaming](/agents/streaming/) * [Stopping and structured results](/agents/stopping-and-structured-results/) and [Structured output](/models/structured-output/) * [Human in the loop](/agents/human-in-the-loop/) * [MCP](/agents/mcp/) * [Langfuse](/observability/langfuse/) ## Reference [Section titled “Reference”](#reference) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`ApprovalRequired`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.ApprovalRequired.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`McpToolbox`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/mcp/struct.McpToolbox.html) * [`ToolFeedback`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.ToolFeedback.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`ToolBox`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolBox.html) * [`Command`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.Command.html) * [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`tool`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) and [`Tool`](https://docs.rs/swiftide/0.34/swiftide/derive.Tool.html) macros * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`LangfuseLayer`](https://docs.rs/swiftide/0.34/swiftide/langfuse/struct.LangfuseLayer.html) ## Next [Section titled “Next”](#next) [Workspace architect](/showcases/workspace-architect/) # Workspace architect > A typed task graph that runs one agent per crate of a Cargo workspace in parallel, joins their reports, drafts an ARCHITECTURE.md and writes it after your approval. This showcase gives you a command-line interface (CLI) that drafts an `ARCHITECTURE.md` for any Cargo workspace. A task graph discovers the crates, runs one agent per crate in parallel, joins their reports, has a model write the document, and pauses until you approve it. Only then does it write the file. The crate lives in the swiftide repository under `examples/workspace-architect`. Clone the repository and run the commands below from its root: ```sh git clone https://github.com/bosun-ai/swiftide cd swiftide ``` ## Prerequisites [Section titled “Prerequisites”](#prerequisites) * Rust 1.97.1, which the swiftide workspace pins in `rust-toolchain.toml`. * `OPENAI_API_KEY`. The CLI checks for it before any agent starts. `OPENAI_BASE_URL` points it at any OpenAI-compatible endpoint, and `OPENAI_MODEL` picks the model (default `gpt-5.4`). The CLI also reads `.env` in the current directory, but not in parent directories. * `cargo`, `sh`, `find`, `sort` and `grep` on your `PATH`. The discover step runs `cargo metadata`. The agent tools run `find` piped into `sort` through a shell, and `grep`. * The feature flags `swiftide-agents`, `swiftide-tasks` and `openai`, plus the default `macros`. `swiftide-agents` already turns on `swiftide-tasks`. * A direct `schemars` dependency with the `derive` feature, for the `JsonSchema` derive on `CrateReport`. * Swiftide 0.34 or newer: the model client sets the endpoint with `api_base` and `api_key` on the `OpenAI` builder, which arrived in 0.34. TLS needs no setup, because swiftide’s HTTP clients bring their own rustls crypto provider ([TLS](/integrations/overview/#tls)). No Docker or database is needed. ## Task graph [Section titled “Task graph”](#task-graph) The module comment in `graph.rs` sketches the graph: ```text discover ──fan-out──▶ analyse × N (parallel) ──▶ join (builds prompt) ──▶ render ──▶ review ──▶ write │ ▲ pause ◀──┘ └── resume ``` The task is a `Task`: a workspace root goes in, the written file comes out. It has six task nodes, registered first: examples/workspace-architect/src/graph.rs ```rust let discover = task.register_node_fn({ let progress = progress.clone(); move |root: &WorkspaceRoot| { let members = workspace::discover(root, settings.selection)?; progress.step("discover", format_args!("{members} in {root}")); Ok::<_, workspace::DiscoveryError>(members) } }); let analyse = task.register_node(CrateAnalyst::new(llm.clone(), progress.clone())); let join = task.register_node_fn({ let progress = progress.clone(); move |summaries: &JoinInput| { progress.joined(summaries.len()); architecture_prompt(summaries) } }); // `Arc` is a task node out of the box: `Prompt` in, `String` out. let render = task.register_node(Arc::new(llm.clone()) as Arc); let review = task.register_node(ApprovalGate::new(settings.pre_approved, progress.clone())); let write = task.register_node_async_fn({ let output = settings.output; let progress = progress.clone(); move |draft: &ArchitectureDraft| { let output = output.clone(); let progress = progress.clone(); Box::pin(async move { let written = draft.write_to(&output).await?; progress.step( "write", format_args!("{} ({} bytes)", written.path.display(), written.bytes), ); Ok::<_, std::io::Error>(written) }) } }); ``` Source: [`examples/workspace-architect/src/graph.rs#L47-L86`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L47-L86) * `discover` and `join` are sync closures registered with `register_node_fn`. A closure takes `&I` and returns `Result`. * `analyse` is a struct that implements `TaskNode`, registered with `register_node`. * `render` is the model client itself, registered as `Arc`. * `review` is another custom `TaskNode`, the approval gate. * `write` is an async closure registered with `register_node_async_fn`. `task.starts_with(discover)` sets the entry node. The excerpts below show most transitions. Three short ones in `graph.rs` sit between the excerpts: `starts_with`, the `render` to `review` transition covered under Join and render, and `task.register_transition(write, task.transitions_to_finish())`, which ends the run. ## Per-crate agents [Section titled “Per-crate agents”](#per-crate-agents) Every fan-out branch evaluates the same `analyse` node with a different crate. Each evaluation builds a fresh agent: examples/workspace-architect/src/analyst.rs ```rust fn agent_for( &self, member: &WorkspaceCrate, analysis: Analysis, ) -> Result { // Every command the tools run executes inside the crate directory. let executor: Arc = Arc::new(LocalExecutor::new(&member.dir)); let tools: [Box; 4] = [ list_files(), read_file(), grep(), summary_stop_tool().into(), ]; Agent::builder() .name(member.name.as_str()) .llm(&self.llm) .context(DefaultContext::from_executor(executor)) .system_prompt(SYSTEM_PROMPT) // Replace the default `stop` tool with one that requires a `CrateReport`. .without_default_stop_tool() .tools(tools) .before_tool(move |_agent, tool_call| { analysis.tool_called(tool_call.name(), tool_call.args().unwrap_or_default()); Box::pin(async { Ok(()) }) }) .limit(MAX_TURNS) .build() } ``` Source: [`examples/workspace-architect/src/analyst.rs#L75-L105`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/analyst.rs#L75-L105) `LocalExecutor::new(&member.dir)` makes the crate directory the working directory for every command the tools run. The context wraps that tool executor with `DefaultContext::from_executor`. Branches share nothing, so they can run at the same time. `without_default_stop_tool()` comes before `tools(..)`, because `tools` merges in the default `stop` tool when you call it. The `before_tool` hook receives `&mut Agent` and the tool call. It counts the call, and with `--verbose` it also prints a `tool` line. `.limit(MAX_TURNS)` caps one query at 30 completions. The three tools are read-only and run their commands through the agent’s executor: examples/workspace-architect/src/tools.rs ```rust #[swiftide::tool( description = "List the files of the crate (relative paths, build output excluded).", param( name = "dir", description = "Directory relative to the crate root, e.g. `.` or `src/tools`" ) )] pub async fn list_files(context: &dyn AgentContext, dir: &str) -> Result { let Some(dir) = CratePath::parse(dir) else { return Ok(CratePath::rejected(dir)); }; let files = run( context, format!( "find {} -type f -not -path '*/target/*' -not -path '*/.git/*' | sort", dir.quoted() ), ) .await?; Ok(first_lines(&files, MAX_LISTED_FILES, "files").into()) } #[swiftide::tool( description = "Read a file of the crate, 250 lines at a time.", param(name = "path", description = "File path relative to the crate root"), param( name = "start_line", description = "1-based line to start from; null to start at the top" ) )] pub async fn read_file( context: &dyn AgentContext, path: &str, start_line: &Option, ) -> Result { let Some(path) = CratePath::parse(path) else { return Ok(CratePath::rejected(path)); }; let content = context .executor() .exec_cmd(&Command::read_file(path.as_path())) .await? .into_string_lossy(); Ok(line_window(&content, start_line.unwrap_or(1)).into()) } #[swiftide::tool( description = "Search the crate's files with an extended regular expression (grep -E).", param(name = "pattern", description = "Regex, e.g. `pub (trait|struct) `"), param( name = "dir", description = "Directory relative to the crate root to search in, e.g. `.` or `src`" ) )] pub async fn grep( context: &dyn AgentContext, pattern: &str, dir: &str, ) -> Result { let Some(dir) = CratePath::parse(dir) else { return Ok(CratePath::rejected(dir)); }; let matches = run( context, format!( "grep -rnIE --exclude-dir=target --exclude-dir=.git -e {} -- {}", shell_quote(pattern), dir.quoted() ), ) .await?; if matches.trim().is_empty() { return Ok("No matches.".into()); } Ok(first_lines(&matches, MAX_GREP_MATCHES, "matches").into()) } ``` Source: [`examples/workspace-architect/src/tools.rs#L25-L103`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/tools.rs#L25-L103) `#[swiftide::tool]` turns each async function into a constructor that returns `Box`. The `description` and `param` attributes set what the model sees. `CratePath::parse` rejects absolute paths and `..` before any command runs. ### Structured report [Section titled “Structured report”](#structured-report) The agent finishes by calling a custom `stop` tool whose arguments must match `CrateReport`: examples/workspace-architect/src/summary.rs ```rust /// What an analyst agent reports about one crate when it calls `stop`. /// /// The doc comments become field descriptions in the JSON schema the model sees, so they double /// as instructions. #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct CrateReport { /// Two to four sentences: the problem this crate solves and the role it plays in the /// workspace. Describe responsibilities, not files. pub purpose: String, /// Four to eight of the most important public items a newcomer must know (traits, structs, /// enums, macros or functions), each formatted as "`Name`: what it is for". Use names exactly /// as they appear in the source. pub key_types: Vec, /// One entry per workspace crate listed as a dependency in the brief. Empty if the brief /// lists none. pub depends_on: Vec, /// One to three sentences on feature flags, extension points (traits meant to be /// implemented) and non-obvious design decisions. pub notes: String, } /// How the documented crate uses one of its workspace dependencies. #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct DependencyUsage { /// The dependency's crate name, exactly as listed in the brief. pub crate_name: String, /// What the documented crate uses it for. pub usage: String, } /// A `stop` tool whose arguments must match [`CrateReport`]. pub fn summary_stop_tool() -> StopWithArgs { StopWithArgs::with_parameters_schema(schema_for!(CrateReport)) } ``` Source: [`examples/workspace-architect/src/summary.rs#L13-L48`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/summary.rs#L13-L48) `StopWithArgs::with_parameters_schema` works as in the [release notes agent](/showcases/release-notes-agent/#structured-stop), here with the schema of `CrateReport`. The doc comments become field descriptions in that schema, so they double as instructions. When the model calls `stop`, the agent stops with `StopReason::RequestedByTool(_, Some(arguments))`, and the arguments are the report. ### Task node [Section titled “Task node”](#task-node) `CrateAnalyst` implements `TaskNode` by running the agent and reading the report from its stop reason: examples/workspace-architect/src/analyst.rs ```rust /// How often an agent is told to fix a missing or malformed report before the crate fails. const MAX_CORRECTIONS: usize = 2; #[async_trait] impl TaskNode for CrateAnalyst { type Input = WorkspaceCrate; type Output = CrateSummary; type Error = AnalystError; async fn evaluate( &self, _node_id: &DynNodeId, member: &WorkspaceCrate, ) -> Result { self.analyse(member).await.map_err(|cause| AnalystError { crate_name: member.name.clone(), cause, }) } } impl CrateAnalyst { async fn analyse(&self, member: &WorkspaceCrate) -> Result { let analysis = self.progress.analysis(&member.name); let mut agent = self.agent_for(member, analysis.clone())?; agent.query(brief(member)).await?; // Models occasionally answer in prose or nest the `stop` arguments; say what went wrong. let mut corrections = 0; let report = loop { match submitted_report(&agent) { Ok(report) => break report, Err(problem) if corrections < MAX_CORRECTIONS => { corrections += 1; analysis.corrected(&problem); agent.query(problem.correction()).await?; } Err(problem) => return Err(problem.into()), } }; analysis.finished(&report); Ok(report.reconciled_with(member)) } } /// The report the agent passed to `stop`, checked against the `CrateReport` schema. fn submitted_report(agent: &Agent) -> Result { match agent.stop_reason() { Some(StopReason::RequestedByTool(_, Some(arguments))) => { Ok(CrateReport::deserialize(arguments)?) } other => Err(ReportProblem::Missing { stop_reason: other.cloned(), }), } } ``` Source: [`examples/workspace-architect/src/analyst.rs#L138-L194`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/analyst.rs#L138-L194) `#[serde(deny_unknown_fields)]` matters here. Some models nest the report under an `arguments` key, and serde rejects that with ``unknown field `arguments` ``. The node sends the error back with `agent.query(..)` and reads the stop reason again, at most `MAX_CORRECTIONS` (2) times. `reconciled_with` then replaces the model’s dependency list with the one from `cargo metadata`, keeping the model’s usage notes. ## Parallel fan-out [Section titled “Parallel fan-out”](#parallel-fan-out) The `discover` transition turns the crate list into one branch per crate: examples/workspace-architect/src/graph.rs ```rust // One branch per crate, all evaluating the same analyst node with a different input. task.register_transition(discover, move |members: WorkspaceMembers| { let mut crates = members.crates.into_iter(); let Some(first) = crates.next() else { return Transition::error("the workspace has no crates to document"); }; crates .fold(Transition::fan_out(&analyse, first), |fan_out, member| { fan_out.and(&analyse, member) }) .concurrency_model(ConcurrencyModel::Parallel) .join_with(join.join()) })?; ``` Source: [`examples/workspace-architect/src/graph.rs#L92-L105`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L92-L105) `Transition::fan_out(&analyse, first)` creates the first branch, and `.and(&analyse, member)` adds one per remaining crate. `fan_out` needs a first target, so an empty crate list fails the run with `Transition::error` and “the workspace has no crates to document”. By default only publishable crates are selected. A workspace whose members all set `publish = false` fails the same way unless you pass `--all-members`. The default concurrency model is `ConcurrencyModel::Sequential`, so `.concurrency_model(ConcurrencyModel::Parallel)` is what runs the branches at the same time. `.join_with(join.join())` turns the `FanOutTransition` into a `Transition` and names the join node that waits for the group. The join line in the captured run, with `claude-sonnet-5-5` as the model, shows the effect: `230s of agent work in 41s wall-clock (5.7x)`. ## Join and render [Section titled “Join and render”](#join-and-render) The graph registers one transition for the `analyse` node. Every fan-out branch evaluates that node, so all branches use the same transition to the join. The join then hands a prompt to `render`: examples/workspace-architect/src/graph.rs ```rust // Each branch hands its `CrateSummary` to the join; the join runs once all have arrived. task.register_transition(analyse, join.join())?; task.register_transition(join, { let progress = progress.clone(); move |prompt| { progress.step("render", "drafting ARCHITECTURE.md"); render.transitions_with(prompt) } })?; ``` Source: [`examples/workspace-architect/src/graph.rs#L109-L117`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L109-L117) Fan-out branches cannot finish the task themselves. They have to reach the join. The join runs once every branch has delivered its `CrateSummary`. `render.transitions_with(prompt)` sends the result on to the render node. The join closure calls `architecture_prompt`: examples/workspace-architect/src/render.rs ```rust /// Tera template for the document. The crate summaries are the model's only source of truth. const ARCHITECTURE_TEMPLATE: &str = r#"You are writing ARCHITECTURE.md for a Rust Cargo workspace. The readers are Rust developers who are new to the codebase and want a mental map before they read code. Below are per-crate summaries written by agents that read each crate's source. They are your only source of truth: do not invent crates, items, flows or relationships that are not supported by them. Crates are listed foundations first. {% for crate in crates %} ### {{ crate.name }} Purpose: {{ crate.report.purpose }} Key items: {% for item in crate.report.key_types %}- {{ item }} {% endfor %}Depends on: {% for dep in crate.dependencies %}- {{ dep.name }}{% if dep.optional %} (optional){% endif %}: {{ dep.usage }} {% else %}- no workspace crates {% endfor %}Notes: {{ crate.report.notes }} {% endfor %} Write the document in GitHub-flavoured Markdown with exactly these sections: # Architecture One short paragraph: what the workspace is for and how it is split into crates. ## Bird's-eye view Two or three paragraphs describing the layers (for example foundation, building blocks, integrations, facade) and which crate a user of the project depends on. ## Crate map A table with the columns `Crate`, `Responsibility` and `Start reading at`. List every crate above, in the same order. `Start reading at` names one to three key items in backticks. ## Dependency graph Copy this diagram verbatim (it was generated from `cargo metadata`; dotted edges are optional dependencies): {{ dependency_graph }} After the diagram, three to five bullets on what the layering means for contributors (which crate to change for what, and what a change ripples into). Stick to what the graph and the summaries show; no generic advice. ## Data flow The two to four most important things a user of the workspace does with it, each under its own `###` heading. For each, numbered steps from input to output that name the crates and items involved, in backticks. Skip internal plumbing. ## Extension points One bullet per kind of extension, phrased as "To add , implement `Trait` (`crate`)" or "use `macro` (`crate`)". Only traits the summaries describe as meant to be implemented. ## Cross-cutting concerns Short, specific bullets on feature flags, observability, error handling and test utilities. Leave out anything you cannot name concretely. Rules: be concrete, precise and terse; prefer short sentences. Put crate and item names in backticks. Refer to an item by its name and, where useful, its crate in parentheses, e.g. `Agent` (`my-crate`); do not make up module paths. Never mention the summaries, the agents or these instructions in the document. Output only the Markdown document, without wrapping it in a code fence."#; /// Builds the prompt for the render node from every branch that reached the join. pub fn architecture_prompt( summaries: &JoinInput, ) -> Result { let crates = serde_json::to_value(summaries.iter().collect::>())?; Ok(Prompt::from(ARCHITECTURE_TEMPLATE) .with_context_value("crates", crates) .with_context_value("dependency_graph", dependency_graph(summaries))) } ``` Source: [`examples/workspace-architect/src/render.rs#L15-L66`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/render.rs#L15-L66) `JoinInput` gives you `iter()`, `len()` and `is_empty()`. The prompt is a Tera template with two context values: `crates`, the serialized summaries, and `dependency_graph`. The Mermaid dependency graph is generated from `cargo metadata`, not by the model, and the template tells the model to copy it verbatim. The `render` to `review` transition wraps the model’s `String` in an `ArchitectureDraft`. `ArchitectureDraft::from_model_output` strips a code fence around the whole document, in case the model adds one. ## Pause and resume [Section titled “Pause and resume”](#pause-and-resume) The review transition either continues to `write` or pauses the task: examples/workspace-architect/src/graph.rs ```rust task.register_transition(review, move |decision| match decision { Review::Approved(draft) => write.transitions_with(draft).into(), Review::AwaitingApproval => Transition::pause(), })?; ``` Source: [`examples/workspace-architect/src/graph.rs#L125-L128`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L125-L128) `Transition::pause()` pauses the current branch. A paused branch keeps its node, and on `resume()` that node evaluates again with the same input. The approval gate is built around that: examples/workspace-architect/src/approval.rs ```rust /// Lets a draft through once it is approved, and otherwise keeps it for review. /// /// The graph pauses the task when the gate answers [`Review::AwaitingApproval`]. A paused branch /// re-runs its node on `resume()`, so after [`ApprovalGate::approve`] the same draft passes /// through. The gate is cheap to evaluate, which is why the pause sits here and not on the /// expensive render node. #[derive(Debug, Clone)] pub struct ApprovalGate { state: Arc>, progress: Progress, } #[derive(Debug, Default)] struct GateState { approved: bool, pending: Option, } impl ApprovalGate { pub fn new(pre_approved: bool, progress: Progress) -> Self { let state = GateState { approved: pre_approved, pending: None, }; Self { state: Arc::new(Mutex::new(state)), progress, } } /// The draft waiting for a decision, if any. pub fn pending_draft(&self) -> Option { self.state().pending.clone() } pub fn approve(&self) { self.state().approved = true; } fn state(&self) -> MutexGuard<'_, GateState> { self.state.lock().unwrap_or_else(PoisonError::into_inner) } } #[async_trait] impl TaskNode for ApprovalGate { type Input = ArchitectureDraft; type Output = Review; type Error = Infallible; async fn evaluate( &self, _node_id: &DynNodeId, draft: &ArchitectureDraft, ) -> Result { let mut state = self.state(); if state.approved { state.pending = None; self.progress.step("review", "draft approved"); return Ok(Review::Approved(draft.clone())); } state.pending = Some(draft.clone()); self.progress.step( "review", format_args!( "draft ready ({} bytes), waiting for approval", draft.as_str().len() ), ); Ok(Review::AwaitingApproval) } } ``` Source: [`examples/workspace-architect/src/approval.rs#L23-L95`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/approval.rs#L23-L95) The first evaluation stores the draft and answers `Review::AwaitingApproval`. After `approve()`, the second evaluation lets the same draft through. Putting the pause on `render` would call the model a second time. The CLI drives the loop: examples/workspace-architect/src/main.rs ```rust let mut outcome = task.run(root).await?; loop { match outcome { TaskRunOutcome::Completed(written) => return Ok(Some(written)), TaskRunOutcome::Paused => { if !approve_paused_draft(&task)? { return Ok(None); } outcome = task.resume().await?; } } } ``` Source: [`examples/workspace-architect/src/main.rs#L125-L136`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/main.rs#L125-L136) `run` returns `TaskRunOutcome::Paused` instead of an output. `current_nodes::()` returns the paused gate as `CurrentNodes::One`, so the CLI can show its draft and call `approve()` before `task.resume()`. ## Commands [Section titled “Commands”](#commands) From the swiftide repository root: ```sh export OPENAI_API_KEY=sk-... cargo run -p workspace-architect ``` To document another workspace without the approval pause: ```sh cargo run -p workspace-architect -- --workspace ../other-repo --yes ``` | Flag | Default | Description | | -------------------- | ----------------- | -------------------------------------------------------------------------------------------- | | `--workspace ` | `.` | Any directory inside the workspace to document | | `--output ` | `ARCHITECTURE.md` | Output file, relative to the workspace root | | `--yes` | off | Write the draft without pausing for approval | | `--all-members` | off | Also document unpublished members such as examples | | `--model ` | `gpt-5.4` | Model for the analysts and the writer (env `OPENAI_MODEL`) | | `--responses-api` | off | Use the OpenAI Responses application programming interface (API) instead of Chat Completions | | `--verbose` | off | Print every tool call and each agent’s raw report | Swiftide’s tracing is off unless `WORKSPACE_ARCHITECT_LOG` is set, for example to `swiftide=debug`. The CLI ignores `RUST_LOG`, so a global cargo config that sets it does not bury the draft. The client sets `max_completion_tokens` to 16,384 through `Options`, because some gateways cap an unset limit at 4,096 tokens and the document would be cut off. The tests run the whole graph offline with a scripted model, including the pause, approval and `resume()`: ```sh cargo test -p workspace-architect ``` ## Output [Section titled “Output”](#output) This run on the swiftide repository used `claude-sonnet-5-5` for all ten analysts and the writer, through an OpenAI-compatible gateway (Bifrost in front of AWS Bedrock) with Chat Completions. The run answered `y` at the prompt and passed `--output target/ARCHITECTURE.md`. Trimmed from `OUTPUT.md`: ```text [ 0.0s] model claude-sonnet-5-5 via Chat Completions at http://127.0.0.1:8080/v1 [ 0.0s] discover 10 crates (7 other members skipped) in /Users/timonv/projects/swiftide-wt-docs-showcases [ 0.0s] analyse started swiftide-core [ 0.0s] analyse started swiftide-langfuse ... [ 14.0s] analyse finished swiftide-langfuse in 14.0s (6 tool calls) [ 14.3s] analyse finished swiftide-macros in 14.3s (6 tool calls) ... [ 33.9s] analyse retry swiftide-core: the `stop` arguments do not match the schema: unknown field `arguments`, expected one of `purpose`, `key_types`, `depends_on`, `notes` [ 40.5s] analyse finished swiftide-core in 40.5s (15 tool calls) [ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x) [ 40.5s] render drafting ARCHITECTURE.md [ 68.8s] review draft ready (11210 bytes), waiting for approval ... [ 68.8s] review draft approved [ 68.8s] write /Users/timonv/projects/swiftide-wt-docs-showcases/target/ARCHITECTURE.md (11210 bytes) Wrote /Users/timonv/projects/swiftide-wt-docs-showcases/target/ARCHITECTURE.md (11210 bytes) in 68.8s. ``` A second run with `--verbose` took 217s of agent work in 33s (6.5x). Writing the document took about 28 seconds in both runs. A few lines of the generated crate map: ```text | Crate | Responsibility | Start reading at | |---|---|---| | `swiftide-core` | Shared traits, data types and errors for indexing, querying, chat completion, tools and agent context. No integrations. | `Node`, `Query`, `ChatCompletion` | ... | `swiftide-tasks` | In-process typed task graph engine with transitions, fan-out/join, pause and resume, and adapters for closures and core traits. | `Task`, `TaskNode`, `Transition` | | `swiftide-agents` | LLM agent loop over chat completions and tool calls, with hooks, a default context, a local executor and optional MCP toolboxes. | `Agent`, `AgentBuilder`, `DefaultContext` | ``` The run documents a swiftide checkout that includes `swiftide-agents-contrib`, which shipped in 0.34. ## Gotchas [Section titled “Gotchas”](#gotchas) * Retries cost time. In the approved run a retry made `swiftide-core` the slowest branch, and the join waits for it. * Each correction is a new `query`, so it gets a fresh turn limit. One crate can use up to 90 completions. * See [Turn limit and nudges](/showcases/release-notes-agent/#turn-limit-and-nudges) in the release notes agent for how a turn limit and a missing tool call share a stop reason. This node treats both as a missing report. * `LocalExecutor` sets a working directory. It does not sandbox. The tools reject absolute paths and `..`, but symlinks inside a crate are followed. Treat the checks as a guard rail. ## Concept pages [Section titled “Concept pages”](#concept-pages) * [Tasks overview](/tasks/overview/) * [Nodes and transitions](/tasks/nodes-and-transitions/) * [Fan-out and join](/tasks/fan-out-and-join/) * [Pause and resume](/tasks/pause-and-resume/) * [Agents as nodes](/tasks/agents-as-nodes/) * [Built-in nodes](/tasks/built-in-nodes/) * [Executors](/agents/executors/) * [Stopping and structured results](/agents/stopping-and-structured-results/) * [Tools](/agents/tools/) ## Reference [Section titled “Reference”](#reference) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`FanOutTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.FanOutTransition.html) * [`ConcurrencyModel`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.ConcurrencyModel.html) * [`JoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinInput.html) * [`JoinTarget`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinTarget.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`CurrentNodes`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.CurrentNodes.html) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`Options`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`#[swiftide::tool]`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) ## Next [Section titled “Next”](#next) [Docs Q\&A with evaluation](/showcases/docs-qa/) # Agents as task nodes > Run an agent as a task node with TaskAgent, or implement TaskNode to turn its stop payload into typed output. You wrap an agent as a task node, first with the `TaskAgent` adapter and then as your own `TaskNode` that hands the next node a typed value read from the agent’s stop payload. `TaskAgent` is the shortest form: ```rust /// Adds an entry to the top of `CHANGELOG.md`. /// /// # Arguments /// - `entry`: The Markdown entry to add, including its heading. #[swiftide::tool] async fn add_changelog_entry( context: &dyn AgentContext, entry: &str, ) -> Result { let executor = context.executor(); let changelog = executor .exec_cmd(&Command::read_file("CHANGELOG.md")) .await? .into_string_lossy(); executor .exec_cmd(&Command::write_file( "CHANGELOG.md", format!("{entry}\n\n{changelog}"), )) .await?; Ok(ToolOutput::text("Entry added.")) } /// An agent wrapped in `TaskAgent` as the only node of a task. pub async fn update_changelog(openai: OpenAI) -> anyhow::Result<()> { let checkout: Arc = Arc::new(LocalExecutor::new("/srv/checkout")); let agent = Agent::builder() .llm(&openai) .context(DefaultContext::from_executor(checkout)) .system_prompt("You keep the changelog of this repository up to date.") .tools([add_changelog_entry()]) .build()?; let mut task = Task::::new(); let changelog = task.register_node(TaskAgent::from(agent)); task.starts_with(changelog); task.register_transition(changelog, task.transitions_to_finish())?; // `TaskAgent` returns `()`: the result is the file the tool wrote. match task.run("Add an entry for the 0.33 release.").await? { TaskRunOutcome::Completed(()) => println!("CHANGELOG.md is updated"), TaskRunOutcome::Paused => unreachable!("no node in this task pauses"), } Ok(()) } ``` Source: [`snippets/src/tasks/agents_as_nodes.rs#L15-L60`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/agents_as_nodes.rs#L15-L60) `TaskAgent::from(agent)` turns the agent into a task node. Its input is a `Prompt`, its output is `()` and its error is `AgentError`. Each evaluation calls `query` on the agent with the prompt. `TaskAgent` gives you no way to read the agent’s answer. It fits agents whose work is a side effect, like the changelog entry that `add_changelog_entry` writes through the tool executor. When the next node needs data from the agent, write your own node. The adapter keeps the agent behind an `Arc>`. Branches that share one `TaskAgent` run one at a time and add to the same [message history](/agents/context-and-history/#message-history). The `swiftide-agents` feature flag turns on `swiftide-tasks` as well: ```sh cargo add swiftide@0.34 --features swiftide-agents,openai ``` The custom node below also needs serde and schemars 1 (the version swiftide re-exports as `swiftide::reexports::schemars`), plus `thiserror` for its error type: ```sh cargo add serde --features derive cargo add schemars@1 --features derive cargo add serde_json thiserror ``` ## Custom agent node [Section titled “Custom agent node”](#custom-agent-node) The workspace-architect showcase fans out one agent per publishable crate of a Cargo workspace, and each branch returns a `CrateSummary`. The node holds the model client, not an agent: examples/workspace-architect/src/analyst.rs ```rust /// A task node that documents one workspace crate with a freshly built agent. /// /// The node is registered once and used by every fan-out branch: each branch evaluates it with a /// different [`WorkspaceCrate`], and every evaluation builds its own agent scoped to that crate's /// directory, so branches share nothing and can run in parallel. #[derive(Clone)] pub struct CrateAnalyst { llm: L, progress: Progress, } impl CrateAnalyst { pub fn new(llm: L, progress: Progress) -> Self { Self { llm, progress } } } ``` Source: [`examples/workspace-architect/src/analyst.rs#L55-L70`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/analyst.rs#L55-L70) The `TaskNode` implementation builds an agent, queries it and reads what the agent passed to `stop`: examples/workspace-architect/src/analyst.rs ```rust /// How often an agent is told to fix a missing or malformed report before the crate fails. const MAX_CORRECTIONS: usize = 2; #[async_trait] impl TaskNode for CrateAnalyst { type Input = WorkspaceCrate; type Output = CrateSummary; type Error = AnalystError; async fn evaluate( &self, _node_id: &DynNodeId, member: &WorkspaceCrate, ) -> Result { self.analyse(member).await.map_err(|cause| AnalystError { crate_name: member.name.clone(), cause, }) } } impl CrateAnalyst { async fn analyse(&self, member: &WorkspaceCrate) -> Result { let analysis = self.progress.analysis(&member.name); let mut agent = self.agent_for(member, analysis.clone())?; agent.query(brief(member)).await?; // Models occasionally answer in prose or nest the `stop` arguments; say what went wrong. let mut corrections = 0; let report = loop { match submitted_report(&agent) { Ok(report) => break report, Err(problem) if corrections < MAX_CORRECTIONS => { corrections += 1; analysis.corrected(&problem); agent.query(problem.correction()).await?; } Err(problem) => return Err(problem.into()), } }; analysis.finished(&report); Ok(report.reconciled_with(member)) } } /// The report the agent passed to `stop`, checked against the `CrateReport` schema. fn submitted_report(agent: &Agent) -> Result { match agent.stop_reason() { Some(StopReason::RequestedByTool(_, Some(arguments))) => { Ok(CrateReport::deserialize(arguments)?) } other => Err(ReportProblem::Missing { stop_reason: other.cloned(), }), } } ``` Source: [`examples/workspace-architect/src/analyst.rs#L138-L194`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/analyst.rs#L138-L194) `evaluate` borrows its input and reaches the node through `&self`. Every branch of the fan-out evaluates the same node instance. That is why the agent is a local variable of `analyse` and not a field. After `query` returns, `agent.stop_reason()` says why the agent stopped. A call to the stop tool with arguments gives `StopReason::RequestedByTool(_, Some(arguments))`. `CrateReport::deserialize` reads the report straight from the borrowed `serde_json::Value`. Any other stop reason, or a payload that does not match, becomes a `ReportProblem`. The node then checks the report against `cargo metadata` and returns a `CrateSummary`. The transitions that follow are typed, so the compiler checks that the next node accepts it. ## Stop schema [Section titled “Stop schema”](#stop-schema) The agent’s stop tool requires a `CrateReport`: examples/workspace-architect/src/summary.rs ```rust /// What an analyst agent reports about one crate when it calls `stop`. /// /// The doc comments become field descriptions in the JSON schema the model sees, so they double /// as instructions. #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct CrateReport { /// Two to four sentences: the problem this crate solves and the role it plays in the /// workspace. Describe responsibilities, not files. pub purpose: String, /// Four to eight of the most important public items a newcomer must know (traits, structs, /// enums, macros or functions), each formatted as "`Name`: what it is for". Use names exactly /// as they appear in the source. pub key_types: Vec, /// One entry per workspace crate listed as a dependency in the brief. Empty if the brief /// lists none. pub depends_on: Vec, /// One to three sentences on feature flags, extension points (traits meant to be /// implemented) and non-obvious design decisions. pub notes: String, } /// How the documented crate uses one of its workspace dependencies. #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] pub struct DependencyUsage { /// The dependency's crate name, exactly as listed in the brief. pub crate_name: String, /// What the documented crate uses it for. pub usage: String, } /// A `stop` tool whose arguments must match [`CrateReport`]. pub fn summary_stop_tool() -> StopWithArgs { StopWithArgs::with_parameters_schema(schema_for!(CrateReport)) } ``` Source: [`examples/workspace-architect/src/summary.rs#L13-L48`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/summary.rs#L13-L48) `summary_stop_tool` replaces the default `stop` tool with one whose arguments must match `CrateReport`. [Stopping and structured results](/agents/stopping-and-structured-results/) explains how the payload reaches `StopReason::RequestedByTool`. ## One agent per input [Section titled “One agent per input”](#one-agent-per-input) `agent_for` builds a fresh agent for every evaluation: examples/workspace-architect/src/analyst.rs ```rust fn agent_for( &self, member: &WorkspaceCrate, analysis: Analysis, ) -> Result { // Every command the tools run executes inside the crate directory. let executor: Arc = Arc::new(LocalExecutor::new(&member.dir)); let tools: [Box; 4] = [ list_files(), read_file(), grep(), summary_stop_tool().into(), ]; Agent::builder() .name(member.name.as_str()) .llm(&self.llm) .context(DefaultContext::from_executor(executor)) .system_prompt(SYSTEM_PROMPT) // Replace the default `stop` tool with one that requires a `CrateReport`. .without_default_stop_tool() .tools(tools) .before_tool(move |_agent, tool_call| { analysis.tool_called(tool_call.name(), tool_call.args().unwrap_or_default()); Box::pin(async { Ok(()) }) }) .limit(MAX_TURNS) .build() } ``` Source: [`examples/workspace-architect/src/analyst.rs#L75-L105`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/analyst.rs#L75-L105) Each agent gets a [`LocalExecutor`](/agents/executors/#local-executor) whose working directory is its crate’s directory, so relative paths in its tools resolve inside that crate. This is not a sandbox. Absolute paths and `..` still reach the rest of the file system. The showcase’s own tools reject absolute paths and `..`, but they follow symlinks. `without_default_stop_tool()` takes effect in the next `.tools(...)` call, which builds the tool set without the default `stop`. `summary_stop_tool()` then adds the one bound to `CrateReport`. The order matters, as the gotchas below explain. The `before_tool` [hook](/agents/hooks/) reports progress. `limit(MAX_TURNS)` caps the completions of a single query at 30. The branches share no agent, executor or message history. That makes it safe to run them under `ConcurrencyModel::Parallel`, as [Fan-out and join](/tasks/fan-out-and-join/#concurrency-model) shows. ## Corrections [Section titled “Corrections”](#corrections) A model sometimes answers in prose or nests its report. The node then queries the same agent again with a correction, up to `MAX_CORRECTIONS` (2) times. For a payload that does not match the schema, the correction quotes the serde error. For a missing stop, it is a fixed reminder to call `stop` now. The agent keeps its message history between the queries, so it sees its own mistake. [Workspace architect](/showcases/workspace-architect/#output) shows the captured retries. ## Shared agent [Section titled “Shared agent”](#shared-agent) `examples/tasks.rs` keeps one agent for the life of the node instead: ```rust /// A domain-specific task step around an `Agent`. /// /// We do not use `TaskAgent` here on purpose. `TaskAgent` is useful when you simply want /// "run this prompt through an agent" behavior. In many applications, however, an agent step has /// its own responsibility and its own input/output shape. That is when implementing `TaskNode` /// directly becomes the clearer API. #[derive(Clone, Debug)] struct BriefingAgent { agent: Arc>, } impl BriefingAgent { fn new(agent: Agent) -> Self { Self { agent: Arc::new(Mutex::new(agent)), } } } /// The structured payload that our custom stop tool expects from the agent. /// /// Returning a typed value like this is the main reason to implement a task node around an agent: /// downstream task steps can now work with domain data instead of unstructured free text. #[derive(Clone, Debug, Deserialize, Serialize, JsonSchema)] #[schemars(deny_unknown_fields)] struct BriefingDecision { audience: String, summary: String, next_steps: Vec, } #[derive(Debug)] enum BriefingAgentError { Agent(AgentError), MissingStructuredStop, InvalidStructuredStop(serde_json::Error), } impl std::fmt::Display for BriefingAgentError { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { match self { Self::Agent(error) => write!(f, "{error}"), Self::MissingStructuredStop => { write!(f, "agent stopped without the structured stop payload") } Self::InvalidStructuredStop(error) => { write!( f, "agent returned an invalid structured stop payload: {error}" ) } } } } impl std::error::Error for BriefingAgentError {} impl From for BriefingAgentError { fn from(value: AgentError) -> Self { Self::Agent(value) } } impl From for BriefingAgentError { fn from(value: serde_json::Error) -> Self { Self::InvalidStructuredStop(value) } } #[swiftide::reexports::async_trait::async_trait] impl TaskNode for BriefingAgent { type Input = String; type Output = BriefingDecision; type Error = BriefingAgentError; async fn evaluate( &self, _node_id: &NodeId< dyn TaskNode, >, brief: &Self::Input, ) -> Result { // The previous node produced a short brief. This node asks the agent to turn that brief // into a structured hand-off plan and to stop with a JSON payload we can deserialize. let handoff_prompt = Prompt::from( "Turn this brief into a structured hand-off plan.\n\ Brief:\n\ {{brief}}\n\n\ Call the `stop` tool once you have produced the final plan.", ) .with_context_value("brief", brief.clone()); let mut agent = self.agent.lock().await; agent.query_once(handoff_prompt).await?; let Some(StopReason::RequestedByTool(_, Some(payload))) = agent.stop_reason() else { return Err(BriefingAgentError::MissingStructuredStop); }; Ok(serde_json::from_value(payload.clone())?) } } ``` Source: [`examples/tasks.rs#L25-L126`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/tasks.rs#L25-L126) `BriefingAgent` locks the agent, calls `query_once` and deserializes the stop payload into a `BriefingDecision`. `query_once` runs one turn of the agent loop instead of looping until the agent stops. If the model calls another tool in that turn, the agent stops with `StopReason::NoNewMessages` and the node returns `MissingStructuredStop`. Use `query` for an agent that needs tools before it stops. The mutex serializes evaluations, and the message history carries over from one evaluation to the next. Use this shape when the node runs once per task, or when later evaluations should see earlier ones. ## Gotchas [Section titled “Gotchas”](#gotchas) * Every call to `query` starts the `limit` count from zero. With corrections, one crate in the showcase can use up to `(1 + MAX_CORRECTIONS) * MAX_TURNS` completions. * An agent that reaches its limit stops with `StopReason::NoNewMessages`, not with a stop payload. The payload itself is an `Option`, so handle `None` too. * Call `without_default_stop_tool()` before `.tools(...)`. Called after it, or without any `.tools(...)` call, the default `stop` stays. Tools are unique by name, so your own `stop` passed later is dropped and the agent stops with `RequestedByTool(_, None)`. * `build()` returns an `AgentBuilderError`, and `query` returns an `AgentError`. Wrap both in one error type that names the input. `AnalystError` prints “analysing `{crate_name}` failed: …”, so a failed branch says which crate it was working on. ## Reference [Section titled “Reference”](#reference) * [`TaskAgent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.TaskAgent.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html), [`Agent::query_once`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.query_once) and [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) and [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`AgentError`](https://docs.rs/swiftide/0.34/swiftide/agents/errors/enum.AgentError.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) and [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) and [`DynNodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/type.DynNodeId.html) * [`ConcurrencyModel`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.ConcurrencyModel.html) ## Next [Section titled “Next”](#next) [Built-in nodes](/tasks/built-in-nodes/) # Built-in nodes > Register a SimplePrompt, ChatCompletion or ToolExecutor trait object, or a plain closure, as a task node without writing a type. You end with a model call, a shell command and a reusable closure each running as a task node, and a rule for when to write your own `TaskNode` instead. Swiftide implements `TaskNode` for `Arc` and `Box` trait objects of `SimplePrompt`, `ChatCompletion` and `ToolExecutor`, and for the `SyncFn` and `AsyncFn` closure wrappers. The snippets use the OpenAI client and the local tool executor, so enable those feature flags next to `swiftide-tasks`: ```sh cargo add swiftide@0.34 --features swiftide-tasks,swiftide-agents,openai ``` ## Prompt nodes [Section titled “Prompt nodes”](#prompt-nodes) A closure builds a `Prompt`, and the model client answers it: ```rust let mut task: Task = Task::new(); let ask = task.register_node_fn(|topic: &String| { Ok::<_, NodeError>( Prompt::from("Explain {{ topic }} in two sentences.") .with_context_value("topic", topic.as_str()), ) }); // The cast turns the concrete client into a node: `Prompt` in, `String` out. let answer = task.register_node(Arc::new(openai) as Arc); task.starts_with(ask); task.register_transition(ask, move |prompt| answer.transitions_with(prompt))?; task.register_transition(answer, task.transitions_to_finish())?; if let TaskRunOutcome::Completed(text) = task.run("a task node").await? { println!("{text}"); } ``` Source: [`snippets/src/tasks/built_in_nodes.rs#L17-L35`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/built_in_nodes.rs#L17-L35) The closure fills a template and returns the `Prompt`. The next node is the model client itself. The cast matters. Swiftide implements `TaskNode` for `Arc` and `Box`, not for each client type. `register_node(openai)` on its own does not compile, because the concrete client is not a node. A `SimplePrompt` node has `Input = Prompt`, `Output = String` and `Error = LanguageModelError`. Each evaluation clones the input `Prompt` and calls `prompt` on the client, which renders the template. See [Prompts and templates](/models/prompts-and-templates/) for building the `Prompt`. One `Arc` can back several nodes, as in the `examples/tasks.rs` excerpt in [Nodes and transitions](/tasks/nodes-and-transitions/#transitions). The workspace-architect showcase uses the same pattern. Its join node builds one `Prompt` from every crate summary, and a `SimplePrompt` node renders it into an `ARCHITECTURE.md` draft. In the approved run from `OUTPUT.md`, the render step took 28 seconds for an 11 KB draft. The model was `claude-sonnet-5-5`, called through Chat Completions on an OpenAI-compatible gateway (Bifrost in front of AWS Bedrock): ```text [ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x) [ 40.5s] render drafting ARCHITECTURE.md [ 68.8s] review draft ready (11210 bytes), waiting for approval ``` ## Chat completion nodes [Section titled “Chat completion nodes”](#chat-completion-nodes) When a step needs a system message, a full message history or tool specs, use a `ChatCompletion` node instead: ```rust let mut task: Task = Task::new(); let ask = task.register_node_fn(|question: &String| { Ok::<_, NodeError>(ChatCompletionRequest::from(vec![ ChatMessage::new_system("Answer in one sentence."), ChatMessage::new_user(question.clone()), ])) }); let chat: Arc = Arc::new(openai); let complete = task.register_node(chat); let answer = task.register_node_fn(|response: &ChatCompletionResponse| { Ok::<_, NodeError>(response.message().unwrap_or_default().to_string()) }); task.starts_with(ask); task.register_transition(ask, move |request| complete.transitions_with(request))?; task.register_transition(complete, move |response| answer.transitions_with(response))?; task.register_transition(answer, task.transitions_to_finish())?; if let TaskRunOutcome::Completed(text) = task.run("What does a task node do?").await? { println!("{text}"); } ``` Source: [`snippets/src/tasks/built_in_nodes.rs#L43-L66`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/built_in_nodes.rs#L43-L66) An `Arc` or `Box` node has `Input = ChatCompletionRequest<'static>`, `Output = ChatCompletionResponse` and `Error = LanguageModelError`. It calls `complete` with the request. Task payloads must be `'static`, so the request has to own its messages. `ChatCompletionRequest::from(Vec)` does that, and so does `ChatCompletionRequest::builder()` with an owned `Vec`. To keep a borrowed request, call `to_owned()` on it first. The response is the whole `ChatCompletionResponse`. A node after it reads `message()`, which returns `Option<&str>`, or `tool_calls()`. The node does not run tools. To act on tool calls in a loop, use an agent. [Models overview](/models/overview/) covers the clients that implement `ChatCompletion`. ## Tool executor nodes [Section titled “Tool executor nodes”](#tool-executor-nodes) A `ToolExecutor` node runs one command. Pair it with a closure that builds the `Command`: ```rust let mut task: Task = Task::new(); let command = task.register_node_fn(|count: &usize| { Ok::<_, NodeError>(Command::shell(format!("git log --oneline -n {count}"))) }); let shell: Arc = Arc::new(LocalExecutor::new(".")); let run = task.register_node(shell); task.starts_with(command); task.register_transition(command, move |cmd| run.transitions_with(cmd))?; task.register_transition(run, task.transitions_to_finish())?; match task.run(5_usize).await { Ok(TaskRunOutcome::Completed(output)) => { for chunk in output.stdout() { print!("{}", String::from_utf8_lossy(chunk)); } } Ok(TaskRunOutcome::Paused) => unreachable!("no node in this task pauses"), Err(TaskError::NodeError(failed)) => { // An exit status, a spawn failure or an I/O error. The text can sit in either stream. if let Some(CommandError::NonZeroExit(output)) = failed.node_error.downcast_ref() { eprintln!("{}", output.to_string_lossy()); } return Err(TaskError::NodeError(failed).into()); } Err(other) => return Err(other.into()), } ``` Source: [`snippets/src/tasks/built_in_nodes.rs#L74-L102`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/built_in_nodes.rs#L74-L102) An `Arc` or `Box` node has `Input = Command`, `Output = CommandOutput` and `Error = CommandError`. It calls `exec_cmd` on the executor. `LocalExecutor` runs commands on the local machine. [Executors](/agents/executors/) covers the executors and their commands. Build the input with `Command::shell`, `Command::read_file` or `Command::write_file`. Chain `with_current_dir` or `with_timeout` to set a directory or a time limit for one command. Since 0.33, `CommandOutput` keeps standard output (stdout) and standard error (stderr) apart. `stdout()` and `stderr()` iterate over the chunks of one stream, in the order the executor read them. `to_string_lossy()` and `into_string_lossy()` merge both streams into one text. A command that exits with a non-zero status does not produce output for the next node. `LocalExecutor` returns `CommandError::NonZeroExit` with the captured output, and the run fails with `TaskError::NodeError`. Its `node_error` field holds the original `CommandError`, so `downcast_ref` gets the output back, as the snippet does. `NonZeroExit` also carries every I/O error, such as a command that fails to start or a missing file for `Command::read_file`. In that case the error text sits in a stdout chunk, not in stderr. The snippet prints both streams with `to_string_lossy()` for that reason. A command that runs past its timeout fails with `CommandError::TimedOut`, which the snippet passes on with the rest. ## Closure adapters [Section titled “Closure adapters”](#closure-adapters) `register_node_fn` wraps a closure in `SyncFn`, and `register_node_async_fn` wraps one in `AsyncFn`. Both types are public. Build one yourself when you want to name the node’s type, store it in a struct, or register the same adapter more than once: ```rust let mut task: Task = Task::new(); let double = SyncFn::new(|n: &i32| Ok::<_, NodeError>(n * 2)); let first = task.register_node(double.clone()); let second = task.register_node(double); task.starts_with(first); task.register_transition(first, move |n| second.transitions_with(n))?; task.register_transition(second, task.transitions_to_finish())?; assert_eq!(task.run(3).await?, TaskRunOutcome::Completed(12)); ``` Source: [`snippets/src/tasks/built_in_nodes.rs#L110-L120`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/built_in_nodes.rs#L110-L120) `SyncFn::new` takes a closure `Fn(&I) -> Result`. `AsyncFn::new` takes a closure that returns `Box::pin(async move { .. })`. Both are `Clone`, and every `register_node` call adds a separate node. The error type defaults to `NodeError`, but a closure still needs its error type written out; see [Closure nodes](/tasks/nodes-and-transitions/#closure-nodes). ## Custom node or built-in [Section titled “Custom node or built-in”](#custom-node-or-built-in) Use a built-in node when the input and output types already fit the step. A prompt in and text out is a `SimplePrompt` node. A command in and its output out is a `ToolExecutor` node. Small conversions between steps are closures. Implement `TaskNode` yourself when the step: * takes or returns your own domain types * calls more than one service, or calls one in a loop * keeps state that the caller reads from outside the task * needs structured output from a model * should fail with an error that names its input The showcase’s `CrateAnalyst` runs an agent and returns a typed `CrateSummary`. Its `ApprovalGate` keeps a draft that the command-line interface (CLI) approves from outside. Both are custom nodes. [Agents as task nodes](/tasks/agents-as-nodes/#custom-agent-node) and [Pause and resume](/tasks/pause-and-resume/#approval-gate) show them. ## Gotchas [Section titled “Gotchas”](#gotchas) * A `SimplePrompt` node clones its `Prompt` on every evaluation. Keep large values out of the template context if the node runs often. * A `ToolExecutor` node runs commands in the executor’s working directory. With `LocalExecutor`, a relative `with_current_dir` path resolves against that directory. * To branch on a failed command instead of failing the run, write your own node that calls the executor and returns an enum. Node outputs must be `Clone`, and `CommandError` is not, so map the error to a type of your own first. ## Reference [Section titled “Reference”](#reference) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [`Command`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.Command.html) * [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`CommandError`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.CommandError.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`SyncFn`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.SyncFn.html) * [`AsyncFn`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.AsyncFn.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) * [`NodeError`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeError.html) ## Next [Section titled “Next”](#next) [How indexing works](/indexing/overview/) # Fan-out and join > Fan a task out into one branch per item, optionally run the branches in parallel, and collect their typed outputs in one join node. You end up with a task that runs one task node per item of a list, all at the same time, and collects every result in a single join node. The task types need the `swiftide-tasks` feature flag ([Tasks overview](/tasks/overview/#feature-flag)). The code comes from the workspace-architect showcase, which analyses each crate of a Cargo workspace with its own agent. examples/workspace-architect/src/graph.rs ```rust // One branch per crate, all evaluating the same analyst node with a different input. task.register_transition(discover, move |members: WorkspaceMembers| { let mut crates = members.crates.into_iter(); let Some(first) = crates.next() else { return Transition::error("the workspace has no crates to document"); }; crates .fold(Transition::fan_out(&analyse, first), |fan_out, member| { fan_out.and(&analyse, member) }) .concurrency_model(ConcurrencyModel::Parallel) .join_with(join.join()) })?; ``` Source: [`examples/workspace-architect/src/graph.rs#L92-L105`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L92-L105) examples/workspace-architect/src/graph.rs ```rust // Each branch hands its `CrateSummary` to the join; the join runs once all have arrived. task.register_transition(analyse, join.join())?; task.register_transition(join, { let progress = progress.clone(); move |prompt| { progress.step("render", "drafting ARCHITECTURE.md"); render.transitions_with(prompt) } })?; ``` Source: [`examples/workspace-architect/src/graph.rs#L109-L117`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L109-L117) The `discover` node returns the workspace members, and its transition folds them into one fan-out with a branch per crate. Every branch evaluates the same `analyse` node with a different `WorkspaceCrate`, in parallel, and `join_with` names the node that waits for the whole group. The `analyse` node registers one transition to the join target, `task.register_transition(analyse, join.join())`, and every branch uses it. Transitions belong to nodes, not branches, and each node has [exactly one](/tasks/nodes-and-transitions/#transitions). The join node is a closure over `&JoinInput`, registered with `task.register_node_fn` in the `nodes` region of `graph.rs`. It runs once every branch has delivered its `CrateSummary`, and its `Prompt` goes to the render node with an ordinary `transitions_with`. [Built-in nodes](/tasks/built-in-nodes/#prompt-nodes) covers the render node. ## Fan-out [Section titled “Fan-out”](#fan-out) `Transition::fan_out(&node, input)` starts a `FanOutTransition` with one branch. `.and(&node, input)` adds another. Branches can target the same node or different nodes. Each input has to convert into the target node’s input type, or the code does not compile. A `FanOutTransition` becomes a `Transition` only when you call `join_with`, so a fan-out without a join does not compile. `fan_out` needs a first target, so handle an empty list first. The showcase returns `Transition::error(..)`, which fails the run with a `TaskError::NodeError`. With branches known up front, write them out. This example starts two branches on two different nodes: ```rust // The fan-out transition defines the branch group and the join that waits for that group. task.register_transition(start, move |input| { Transition::fan_out(&increment, input) .and(&double, input) .join_with(join.join()) })?; // Each branch still declares where its own output should go. Branches can do any amount of work // before eventually transitioning to the join. task.register_transition(increment, join.join())?; task.register_transition(double, join.join())?; task.register_transition(join, task.transitions_to_finish())?; ``` Source: [`examples/tasks_fanout.rs#L27-L38`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/tasks_fanout.rs#L27-L38) A branch can pass through intermediate nodes before it joins. Only the last node on the branch registers the join target. If a branch joins a different node than the one its fan-out named, the run fails with `TaskError::InvalidState`. ## Join node [Section titled “Join node”](#join-node) A join node takes `JoinInput` as its input, where `T` is the payload every branch contributes. `NodeId::join()` exists only on nodes with that input type. `JoinInput` has three methods: * `iter()` yields the payloads in the order the branches were created, not the order they finished. * `len()` returns the number of payloads. * `is_empty()` returns true when there are none. The showcase relies on that order: the prompt lists the crates as `discover` found them, although `swiftide-core` finished last after a retry. A branch whose output does not convert into `T` does not compile. ## Payload mapping [Section titled “Payload mapping”](#payload-mapping) A branch does not have to return the payload type itself. `JoinTarget::map` converts the branch output on its way into the join: ```rust let mut task = Task::::new(); let start = task.register_node_fn(|text: &String| -> Result { Ok(text.trim().to_string()) }); let count = task.register_node_fn(|text: &String| -> Result { Ok(text.split_whitespace().count()) }); let length = task .register_node_fn(|text: &String| -> Result { Ok(text.chars().count()) }); let shout = task .register_node_fn(|text: &String| -> Result { Ok(text.to_uppercase()) }); let join = task.register_node_fn(|parts: &JoinInput| -> Result { Ok(parts.iter().cloned().collect::>().join("; ")) }); task.starts_with(start); task.register_transition(start, move |text: String| { Transition::fan_out(&count, text.clone()) .and(&length, text.clone()) .and(&shout, text) .join_with(join.join()) })?; // `count` returns a usize; `map` turns it into the join's String payload. task.register_transition( count, join.join().map(|words: usize| format!("{words} words")), )?; // An async mapping is registered with `register_transition_async`. task.register_transition_async( length, join.join() .map_async(|chars: usize| async move { format!("{chars} characters") }), )?; // `shout` already returns a String, so it joins as is. task.register_transition(shout, join.join())?; task.register_transition(join, task.transitions_to_finish())?; // "2 words; 11 characters; HELLO TASKS" let TaskRunOutcome::Completed(summary) = task.run(" hello tasks ").await? else { anyhow::bail!("nothing in this task pauses"); }; ``` Source: [`snippets/src/tasks/fan_out_and_join.rs#L8-L50`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/fan_out_and_join.rs#L8-L50) `map` turns the `usize` from `count` into the `String` the join expects. `JoinTarget::map_async` does the same with an async closure and is registered with `Task::register_transition_async`. `shout` already returns a `String`. Any output that converts into the payload type with `Into` needs no mapping. The task returns `"2 words; 11 characters; HELLO TASKS"`, in branch creation order. ## Mixed payloads [Section titled “Mixed payloads”](#mixed-payloads) When branches return different types and the join needs all of them, give the join node an `AnyJoinInput`. Use `join_any()` in place of `join()`, both in the fan-out and for each branch: ```rust let mut task = Task::::new(); let start = task.register_node_fn(|text: &String| -> Result { Ok(text.trim().to_string()) }); let count = task.register_node_fn(|text: &String| -> Result { Ok(text.split_whitespace().count()) }); let shout = task .register_node_fn(|text: &String| -> Result { Ok(text.to_uppercase()) }); let join = task.register_node_fn(|parts: &AnyJoinInput| -> Result { let words: usize = parts.iter::().sum(); let shouted = parts.iter::().cloned().collect::>(); Ok(format!("{} ({words} words)", shouted.join(" "))) }); task.starts_with(start); task.register_transition(start, move |text: String| { Transition::fan_out(&count, text.clone()) .and(&shout, text) .join_with(join.join_any()) })?; task.register_transition(count, join.join_any())?; task.register_transition(shout, join.join_any())?; task.register_transition(join, task.transitions_to_finish())?; // "HELLO TASKS (2 words)" let TaskRunOutcome::Completed(summary) = task.run(" hello tasks ").await? else { anyhow::bail!("nothing in this task pauses"); }; ``` Source: [`snippets/src/tasks/fan_out_and_join.rs#L58-L87`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/fan_out_and_join.rs#L58-L87) `AnyJoinInput::iter::()` downcasts each payload and skips the ones of other types. `iter_any()` yields every payload as a type-erased value. The compiler no longer checks that a branch delivers a type the join reads, so prefer `JoinInput` with `map` where you can. ## Concurrency model [Section titled “Concurrency model”](#concurrency-model) `ConcurrencyModel` has two variants. `Sequential` is the default, and `Parallel` is opt-in. Set the default for the whole task with `with_default_concurrency_model`: ```rust let mut task = Task::new().with_default_concurrency_model(ConcurrencyModel::Parallel); let start = task.register_node_fn(|input: &i32| -> std::result::Result { Ok(*input) }); let increment = task .register_node_fn(|input: &i32| -> std::result::Result { Ok(*input + 1) }); let double = task .register_node_fn(|input: &i32| -> std::result::Result { Ok(*input * 2) }); let join = task.register_node_fn( |input: &JoinInput| -> std::result::Result { Ok(input.iter().copied().sum::()) }, ); task.starts_with(start); ``` Source: [`examples/tasks_fanout.rs#L9-L23`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/tasks_fanout.rs#L9-L23) To override the model from a given step on, call `concurrency_model` on a `Transition`, `MarkedTransition` or `FanOutTransition`. A branch keeps the override for every later step until another transition overrides it, and the branches of a fan-out inherit its model. The join node always runs with the task’s default model. Under `Sequential`, the runtime evaluates one node at a time. Fan-out branches take turns: after a node finishes, the branch’s next step goes to the back of the queue. Under `Parallel`, the branches run concurrently inside the future that `run` returns, as the [overview](/tasks/overview/#crate-contents) explains. Waiting on input/output (I/O), such as model calls, overlaps. Synchronous, compute-only closures do not. This is the showcase’s captured run, with `claude-sonnet-5-5` as the model for all ten analysts. The `model` and `discover` lines before it and the render steps after it are left out: ```text [ 0.0s] analyse started swiftide-core [ 0.0s] analyse started swiftide-langfuse [ 0.0s] analyse started swiftide-macros [ 0.0s] analyse started swiftide-indexing [ 0.0s] analyse started swiftide-integrations [ 0.0s] analyse started swiftide-query [ 0.0s] analyse started swiftide-tasks [ 0.0s] analyse started swiftide-agents [ 0.0s] analyse started swiftide-agents-contrib [ 0.0s] analyse started swiftide [ 14.0s] analyse finished swiftide-langfuse in 14.0s (6 tool calls) [ 14.3s] analyse finished swiftide-macros in 14.3s (6 tool calls) [ 14.7s] analyse finished swiftide in 14.7s (5 tool calls) [ 19.2s] analyse finished swiftide-query in 19.1s (10 tool calls) [ 19.7s] analyse retry swiftide-agents-contrib: the `stop` arguments do not match the schema: unknown field `arguments`, expected one of `purpose`, `key_types`, `depends_on`, `notes` [ 20.2s] analyse finished swiftide-agents in 20.2s (10 tool calls) [ 24.4s] analyse finished swiftide-indexing in 24.4s (10 tool calls) [ 24.9s] analyse finished swiftide-tasks in 24.9s (10 tool calls) [ 27.1s] analyse finished swiftide-agents-contrib in 27.1s (11 tool calls) [ 31.4s] analyse finished swiftide-integrations in 31.4s (13 tool calls) [ 33.9s] analyse retry swiftide-core: the `stop` arguments do not match the schema: unknown field `arguments`, expected one of `purpose`, `key_types`, `depends_on`, `notes` [ 40.5s] analyse finished swiftide-core in 40.5s (15 tool calls) [ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x) ``` All ten branches start at 0.0s, and the fan-out takes as long as its slowest branch. The join line sums the agent time of every branch: 230 seconds of work in 41 seconds. `OUTPUT.md` estimates close to four minutes for the same analyses run one after another. ## Limits [Section titled “Limits”](#limits) * A branch cannot start a second fan-out while it belongs to an active join group. The run fails with `TaskError::InvalidState`. Fan out again from the join node instead. * A branch cannot finish the task on its own. If a branch inside a group registers `task.transitions_to_finish()`, the task output counts as an arrival at the join, unmapped. When that type differs from the join’s payload type, the join fails at runtime with `TaskError::InvalidState`. Route every branch to the join and finish from there. * The first branch that fails fails the run. The other in-flight branches are dropped and never complete. * All branches share one node instance; keep per-branch state inside `evaluate` ([Agents as task nodes](/tasks/agents-as-nodes/#custom-agent-node)). ## Reference [Section titled “Reference”](#reference) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`Transition::error`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html#method.error) * [`FanOutTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.FanOutTransition.html) * [`MarkedTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.MarkedTransition.html) * [`JoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinInput.html) * [`JoinTarget`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinTarget.html) * [`MappedJoinTarget`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.MappedJoinTarget.html) * [`AnyJoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.AnyJoinInput.html) * [`AnyJoinTarget`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.AnyJoinTarget.html) * [`ConcurrencyModel`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.ConcurrencyModel.html) * [`NodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html) * [`NodeId::join_any`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html#method.join_any) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`Task::register_transition_async`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.register_transition_async) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) ## Next [Section titled “Next”](#next) [Pause and resume](/tasks/pause-and-resume/) # Nodes and transitions > Register closures and TaskNode types as nodes and connect them with typed transitions from start to finish. Here you build a linear task from two closures and a node type of your own, run it, and find out which node failed when something goes wrong. Import the task types from `swiftide::tasks`. [Tasks overview](/tasks/overview/#feature-flag) shows the feature flag to turn on. ```rust use std::num::ParseIntError; use swiftide::{ reexports::async_trait::async_trait, tasks::{DynNodeId, NodeError, Task, TaskError, TaskNode, TaskRunOutcome, Transition}, }; ``` Source: [`snippets/src/tasks/nodes_and_transitions.rs#L4-L9`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/nodes_and_transitions.rs#L4-L9) Register the nodes, connect them, and run the task: ```rust let mut task: Task = Task::new(); // A sync closure node. `Ok::<_, NodeError>` names the error type. let trim = task.register_node_fn(|text: &String| Ok::<_, NodeError>(text.trim().to_string())); // A node type that implements `TaskNode`. let count = task.register_node(CountWords); // A closure that returns a pinned, boxed future. Copy what it needs before `async move`. let label = task.register_node_async_fn(|words: &usize| { let words = *words; Box::pin(async move { Ok::<_, NodeError>(format!("{words} words")) }) }); task.starts_with(trim); task.register_transition(trim, move |text: String| { if text.is_empty() { return Transition::error("there is no text to count"); } count.transitions_with(text).into() })?; task.register_transition(count, move |words| label.transitions_with(words))?; task.register_transition(label, task.transitions_to_finish())?; match task.run(" nodes and transitions ").await? { TaskRunOutcome::Completed(summary) => println!("{summary}"), TaskRunOutcome::Paused => println!("paused; call task.resume() to continue"), } ``` Source: [`snippets/src/tasks/nodes_and_transitions.rs#L36-L63`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/nodes_and_transitions.rs#L36-L63) The run prints `3 words`. `Task` fixes the types at both ends. Each node declares its own input and output, so a `String` becomes a `usize` and then a `String` again. `register_node_fn` turns a synchronous closure into a node, and `register_node_async_fn` does the same for a closure that returns a pinned, boxed future (`Box::pin(async move { .. })`). `register_node` takes any value that implements `TaskNode`. All three return a `NodeId`, the typed handle you wire the node with. `NodeId` is `Copy`, so you move it into transition closures without cloning. `id()` returns its number, assigned in registration order from 0. `starts_with` picks the start node, and `register_transition` says where to go after a node. `transitions_with` moves on to another node with a value, and `transitions_to_finish` hands the task its output. Inside a fan-out branch, that output goes to the join instead of ending the task. `run` returns a `TaskRunOutcome`. `Completed` carries the output. `Paused` is covered in [Pause and resume](/tasks/pause-and-resume/). ## Task nodes [Section titled “Task nodes”](#task-nodes) A task node is a type that implements `TaskNode`: ```rust /// Counts the words in a text. #[derive(Clone, Debug)] pub struct CountWords; #[async_trait] impl TaskNode for CountWords { type Input = String; type Output = usize; type Error = std::convert::Infallible; async fn evaluate( &self, _node_id: &DynNodeId, text: &Self::Input, ) -> Result { Ok(text.split_whitespace().count()) } } ``` Source: [`snippets/src/tasks/nodes_and_transitions.rs#L13-L30`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/nodes_and_transitions.rs#L13-L30) `Input` is what the node receives and `Output` is what it returns. `Error` can be any `std::error::Error + Send + Sync + 'static`; `CountWords` cannot fail, so it uses `Infallible`. `evaluate` borrows both the node and the input. The `&DynNodeId` argument is the node’s own id with its concrete type erased, and most nodes ignore it. Write the impl with `#[async_trait]`, which Swiftide re-exports as `swiftide::reexports::async_trait`. `Input` and `Output` must implement `NodeArg`, which covers every type that is `Send + Sync + Debug + Clone + 'static`. A missing derive shows up as an unmet `NodeArg` bound. `register_node` also needs the node itself to be `Clone`. A run does not clone your node. Every evaluation calls the same instance through `&self`, so keep per-run state out of the struct. ## Closure nodes [Section titled “Closure nodes”](#closure-nodes) The workspace-architect showcase registers its nodes in one block. `discover` and `write` are closure nodes: examples/workspace-architect/src/graph.rs ```rust let discover = task.register_node_fn({ let progress = progress.clone(); move |root: &WorkspaceRoot| { let members = workspace::discover(root, settings.selection)?; progress.step("discover", format_args!("{members} in {root}")); Ok::<_, workspace::DiscoveryError>(members) } }); let analyse = task.register_node(CrateAnalyst::new(llm.clone(), progress.clone())); let join = task.register_node_fn({ let progress = progress.clone(); move |summaries: &JoinInput| { progress.joined(summaries.len()); architecture_prompt(summaries) } }); // `Arc` is a task node out of the box: `Prompt` in, `String` out. let render = task.register_node(Arc::new(llm.clone()) as Arc); let review = task.register_node(ApprovalGate::new(settings.pre_approved, progress.clone())); let write = task.register_node_async_fn({ let output = settings.output; let progress = progress.clone(); move |draft: &ArchitectureDraft| { let output = output.clone(); let progress = progress.clone(); Box::pin(async move { let written = draft.write_to(&output).await?; progress.step( "write", format_args!("{} ({} bytes)", written.path.display(), written.bytes), ); Ok::<_, std::io::Error>(written) }) } }); ``` Source: [`examples/workspace-architect/src/graph.rs#L47-L86`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L47-L86) * `discover` is a sync closure. `Ok::<_, workspace::DiscoveryError>(members)` pins its error type, so the `?` inside converts into it. * `write` is a plain closure that returns `Box::pin(async move { .. })`. The other four nodes have their own pages: [Agents as task nodes](/tasks/agents-as-nodes/), [Fan-out and join](/tasks/fan-out-and-join/), [Built-in nodes](/tasks/built-in-nodes/) and [Pause and resume](/tasks/pause-and-resume/). A closure node needs a concrete error type. Name it with `Ok::<_, E>(..)` or a return type such as `-> Result`, unless the closure already returns a typed `Result`, as `str::parse` does. The closure is `Fn` and runs once per evaluation, so each future needs its own copy of the captures. That is why `write` clones `output` and `progress` before `async move`. The future may still borrow the input, here `draft`. For reusable domain logic, implement `TaskNode` instead. ## Transitions [Section titled “Transitions”](#transitions) `examples/tasks.rs` chains a prompt node, an agent node and a second prompt node: ```rust // Tasks are typed from input to output. This task starts with a `Prompt` and finishes with the // final teammate-facing hand-off note. let mut task: Task = Task::new(); // Convenience helper: `Arc` already implements `TaskNode`, so prompt-model // steps can be registered directly without any wrapper code. let prompt_step: Arc = Arc::new(openai.clone()); let brief_prompt_id = task.register_node(prompt_step.clone()); let handoff_prompt_id = task.register_node(prompt_step); // Custom node: the agent step owns the stop-tool contract and returns structured task data. let agent_id = task.register_node(BriefingAgent::new(agent)); task.starts_with(brief_prompt_id); // Step 1: turn the incoming request into a short brief that the agent can work from. task.register_transition(brief_prompt_id, move |brief| { agent_id.transitions_with(brief) })?; // Step 2: use the structured stop payload to drive a final LLM rendering step. task.register_transition(agent_id, move |decision: BriefingDecision| { Transition::next( &handoff_prompt_id, Prompt::from( "Write a teammate-facing hand-off note.\n\ Audience: {{audience}}\n\ Summary: {{summary}}\n\ Next steps:\n\ {% for step in next_steps %}- {{ step }}\n{% endfor %}", ) .with_context_value("audience", decision.audience) .with_context_value("summary", decision.summary) .with_context_value("next_steps", decision.next_steps), ) })?; // Step 3: the rendered hand-off note becomes the task output. task.register_transition(handoff_prompt_id, task.transitions_to_finish())?; ``` Source: [`examples/tasks.rs#L157-L195`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/tasks.rs#L157-L195) A transition closure receives the node’s output by value and returns where to go next. `agent_id.transitions_with(brief)` is shorthand for `Transition::next(&agent_id, brief)`. Both return a `MarkedTransition` that carries the target’s type, so the compiler checks the value against the target’s `Input`. Each node has exactly one transition, and registering a second one returns `TaskError::InvalidState`. To branch, return different transitions from that one closure. When the arms point at nodes of different types, or one arm pauses or fails, convert each `MarkedTransition` into a `Transition` with `.into()`, as the linear task does. `Transition::error` fails the whole run, drops any in-flight fan-out branches and leaves nothing to resume. It accepts anything that converts into `Box`, including a `&str`. `Transition::pause()` pauses the branch. When the decision needs to await something, use `register_transition_async`. Its closure returns a future, such as `move |value| async move { next.transitions_with(value) }`. ## Start and finish [Section titled “Start and finish”](#start-and-finish) The start node’s `Input` must be the task’s `Input`, or `starts_with` does not compile. `run` takes `impl Into`, so the linear task can pass a `&str` to a `Task`. `transitions_to_finish()` returns a closure that takes the task’s `Output`. The finish constructor on `Transition` is private, so the compiler checks the value against that task’s output type. A finish closure taken from a different task still compiles, and `run` then fails with `TaskError::TypeError`. Code from 0.32 called `transitions_to_done` and got an `Option` from `run`. [Upgrading](/reference/upgrading/#tasks-crate) lists the 0.33 changes. ## Errors [Section titled “Errors”](#errors) Match on the result of `run` to tell a failing node from a wiring mistake: ```rust let mut task: Task = Task::new(); let parse = task.register_node_fn(|text: &String| text.trim().parse::()); task.starts_with(parse); task.register_transition(parse, task.transitions_to_finish())?; match task.run("forty-two").await { Ok(TaskRunOutcome::Completed(number)) => println!("parsed {number}"), Ok(TaskRunOutcome::Paused) => println!("paused"), Err(TaskError::NodeError(failed)) => { // `node_error` is the error the node returned, boxed. if let Some(error) = failed.node_error.downcast_ref::() { eprintln!("node {} could not parse the input: {error}", failed.node_id); } } Err(wiring @ (TaskError::NoSteps | TaskError::MissingTransition(_))) => { // `run` checks the graph before the first node runs. panic!("the task is wired wrong: {wiring}"); } Err(other) => return Err(other.into()), } ``` Source: [`snippets/src/tasks/nodes_and_transitions.rs#L72-L92`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/nodes_and_transitions.rs#L72-L92) A failing node makes `run` return `TaskError::NodeError`. The `NodeError` inside carries `node_id`, the number of the failing node, and `node_error`, the node’s own error boxed. Downcast `node_error` to read your error type. The error displays as `Node error in node 0: invalid digit found in string`. A `Transition::error` produces the same variant, with the id of the node whose transition failed. `run` and `resume` check the graph before any node runs. Without a start node you get `TaskError::NoSteps`. When any registered node has no transition, you get `TaskError::MissingTransition`, even when no path reaches that node, with a message such as `Node 3 is missing a transition`. `TaskError::Incomplete` means all work drained without a finish or a pause. ## Reference [Section titled “Reference”](#reference) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) * [`NodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html) * [`DynNodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/type.DynNodeId.html) * [`NodeArg`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.NodeArg.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`MarkedTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.MarkedTransition.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) * [`NodeError`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeError.html) ## Next [Section titled “Next”](#next) [Fan-out and join](/tasks/fan-out-and-join/) # How tasks work > A task is a typed graph of nodes and transitions that runs functions, prompts and agents, with fan-out, join and pause. You will be able to read a task graph, enable the `swiftide-tasks` feature flag, and decide whether a job needs a task or a single agent. A task runs a fixed graph of steps, and each step can be a function, a prompt or an agent. [Your first task](/start/your-first-task/) builds a two-node task step by step if you want a smaller start. ## Showcase graph [Section titled “Showcase graph”](#showcase-graph) The workspace-architect showcase drafts an `ARCHITECTURE.md` for a Cargo workspace. This function builds its whole task: examples/workspace-architect/src/graph.rs ```rust /// Builds the task: `WorkspaceRoot` in, the written ARCHITECTURE.md out. pub fn architect_task( llm: &L, settings: ArchitectSettings, progress: &Progress, ) -> Result, TaskError> where L: ChatCompletion + SimplePrompt + Clone + 'static, { let mut task = Task::new(); let discover = task.register_node_fn({ let progress = progress.clone(); move |root: &WorkspaceRoot| { let members = workspace::discover(root, settings.selection)?; progress.step("discover", format_args!("{members} in {root}")); Ok::<_, workspace::DiscoveryError>(members) } }); let analyse = task.register_node(CrateAnalyst::new(llm.clone(), progress.clone())); let join = task.register_node_fn({ let progress = progress.clone(); move |summaries: &JoinInput| { progress.joined(summaries.len()); architecture_prompt(summaries) } }); // `Arc` is a task node out of the box: `Prompt` in, `String` out. let render = task.register_node(Arc::new(llm.clone()) as Arc); let review = task.register_node(ApprovalGate::new(settings.pre_approved, progress.clone())); let write = task.register_node_async_fn({ let output = settings.output; let progress = progress.clone(); move |draft: &ArchitectureDraft| { let output = output.clone(); let progress = progress.clone(); Box::pin(async move { let written = draft.write_to(&output).await?; progress.step( "write", format_args!("{} ({} bytes)", written.path.display(), written.bytes), ); Ok::<_, std::io::Error>(written) }) } }); task.starts_with(discover); // One branch per crate, all evaluating the same analyst node with a different input. task.register_transition(discover, move |members: WorkspaceMembers| { let mut crates = members.crates.into_iter(); let Some(first) = crates.next() else { return Transition::error("the workspace has no crates to document"); }; crates .fold(Transition::fan_out(&analyse, first), |fan_out, member| { fan_out.and(&analyse, member) }) .concurrency_model(ConcurrencyModel::Parallel) .join_with(join.join()) })?; // Each branch hands its `CrateSummary` to the join; the join runs once all have arrived. task.register_transition(analyse, join.join())?; task.register_transition(join, { let progress = progress.clone(); move |prompt| { progress.step("render", "drafting ARCHITECTURE.md"); render.transitions_with(prompt) } })?; task.register_transition(render, move |markdown: String| { review.transitions_with(ArchitectureDraft::from_model_output(&markdown)) })?; task.register_transition(review, move |decision| match decision { Review::Approved(draft) => write.transitions_with(draft).into(), Review::AwaitingApproval => Transition::pause(), })?; task.register_transition(write, task.transitions_to_finish())?; Ok(task) } ``` Source: [`examples/workspace-architect/src/graph.rs#L35-L134`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L35-L134) The task has the type `Task`. It takes a workspace root and returns the written document. Each `register_node*` call adds a node and returns its typed `NodeId`. Each `register_transition` call says where a node’s output goes next. * **discover** is a closure node. It runs `cargo metadata` and returns the member crates. * **analyse** is a custom task node, `CrateAnalyst`. The fan-out sends every crate to it as its own branch. The fan-out sets `ConcurrencyModel::Parallel`, so the branches run at the same time. Each branch runs a fresh agent that reads one crate and returns a `CrateSummary`. * **join** waits for every branch. It receives a `JoinInput` and builds the prompt for the writer. * **render** is an `Arc` registered as a node. It sends the prompt to the model and returns Markdown. * **review** is `ApprovalGate`. Until someone approves the draft, its transition returns `Transition::pause()` and the run stops. * **write** is an async closure node. It saves the file, and its transition, `task.transitions_to_finish()`, ends the task. In the captured run, ten analyst agents on `claude-sonnet-5-5` finished in the time of the slowest one. The join line from `OUTPUT.md`: ```text [ 40.5s] join 10 summaries; 230s of agent work in 41s wall-clock (5.7x) ``` Run it against the Swiftide workspace from the repository root: ```sh export OPENAI_API_KEY=sk-... cargo run -p workspace-architect ``` The command-line program starts the task with `task.run(WorkspaceRoot)`. That returns `TaskRunOutcome::Completed(WrittenDocument)` or `TaskRunOutcome::Paused`: examples/workspace-architect/src/main.rs ```rust let mut outcome = task.run(root).await?; loop { match outcome { TaskRunOutcome::Completed(written) => return Ok(Some(written)), TaskRunOutcome::Paused => { if !approve_paused_draft(&task)? { return Ok(None); } outcome = task.resume().await?; } } } ``` Source: [`examples/workspace-architect/src/main.rs#L125-L136`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/main.rs#L125-L136) On a pause, the program asks you to approve the draft and then calls `resume()`. ## Task graph [Section titled “Task graph”](#task-graph) ![Task graph](/diagrams/diagram-task-graph.svg) The diagram shows a smaller task with the same parts. A start node fans out to two prompt nodes that run in parallel. A join node collects both outputs, and the task finishes. A task has these parts: * A **task node** implements `TaskNode`. It declares an `Input`, an `Output` and an `Error` type, and its `evaluate` method turns a borrowed input into an output. [Nodes and transitions](/tasks/nodes-and-transitions/#task-nodes) covers the trait. * A **transition** decides where a node’s output goes next. It is a closure that takes the output by value, or a ready-made target such as `join.join()` or `task.transitions_to_finish()`. Each node has exactly one transition. * The **start node** is set with `starts_with`. Its input type is the task’s input type. * A **fan-out** sends one input to each of several branches. Every fan-out names the join node that waits for all of them. By default the branches run one after another. Set `ConcurrencyModel::Parallel` on the fan-out or the task to run them at the same time. * A **join** node receives the branch outputs as one `JoinInput`. [Fan-out and join](/tasks/fan-out-and-join/) covers both. * A **pause** stops the run, and `run` returns `TaskRunOutcome::Paused`. The paused branch keeps the node whose transition paused. `resume()` runs that node again with the same input. This is why the showcase pauses on the cheap review node and not on render. [Pause and resume](/tasks/pause-and-resume/) covers the details. When a branch reaches the finishing transition, `run` or `resume()` returns `TaskRunOutcome::Completed` with the task’s output. [Nodes and transitions](/tasks/nodes-and-transitions/#errors) covers the checks `run` makes before any node runs. ## Crate contents [Section titled “Crate contents”](#crate-contents) The `swiftide-tasks` crate holds the graph and its runtime: * `Task`, `TaskNode`, `NodeId` and `Transition` * fan-out and join, with `JoinInput` and `AnyJoinInput` * `ConcurrencyModel::Sequential` (the default) and `ConcurrencyModel::Parallel` * pause, resume and reset, with `TaskRunOutcome` * `TaskError` and `NodeError` * closure nodes through `register_node_fn` and `register_node_async_fn` * [built-in nodes](/tasks/built-in-nodes/) for `SimplePrompt`, `ChatCompletion` and `ToolExecutor` trait objects The crate’s own code spawns no tasks or threads. Parallel branches run concurrently inside the future that `run` returns. Swiftide itself still depends on tokio. `TaskAgent`, which wraps an agent as a node, lives in swiftide-agents. Import it from `swiftide::agents::TaskAgent`. ## Feature flag [Section titled “Feature flag”](#feature-flag) Turn on the `swiftide-tasks` feature flag: ```sh cargo add swiftide@0.34 --features swiftide-tasks ``` The `swiftide-agents` flag turns it on as well. Import the graph types from `swiftide::tasks`. The old path `swiftide::agents::tasks` still works, but it is deprecated since 0.32.1. [Upgrading](/reference/upgrading/#tasks-crate) lists the 0.33 changes to tasks. A comment in the crate source says the tasks application programming interface (API) “is not stable yet” and “is subject to rapid change”, so check Upgrading when you move to a new minor version. ## Tasks and agents [Section titled “Tasks and agents”](#tasks-and-agents) An [agent](/agents/overview/) runs a loop. After every model response it decides which tool to call next. It stops when a tool or the model asks it to, when it reaches its completion limit, or when it fails. A task fixes the order of steps in code. Every node declares its input and output types. A transition to a node with the wrong input type does not compile, and neither does a fan-out without a join. Control flow lives in Rust closures, not in model output. The model can still make decisions inside a node, but the graph decides what happens with the result. Each node does one job. You can test a closure node without a model, swap the model behind a prompt node, or give an agent node its own tools and limits. Use a task when you know the steps in advance, when some of them need no model, or when you want to fan out work and collect it in one place. Put an agent inside a node for the open-ended part, such as reading one crate, and let the task handle the order, the parallel branches and the approval. [Agents as task nodes](/tasks/agents-as-nodes/) shows how. The [workspace-architect showcase](/showcases/workspace-architect/) walks through the whole program. ## Reference [Section titled “Reference”](#reference) * [`swiftide::tasks`](https://docs.rs/swiftide/0.34/swiftide/tasks/index.html) * [`Task`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) * [`NodeId`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeId.html) * [`Transition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html) * [`JoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.JoinInput.html) * [`AnyJoinInput`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.AnyJoinInput.html) * [`ConcurrencyModel`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.ConcurrencyModel.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) * [`NodeError`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.NodeError.html) * [`TaskAgent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.TaskAgent.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`ToolExecutor`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ToolExecutor.html) * [Feature flags](/reference/feature-flags/) * [Crates](/reference/crates/) ## Next [Section titled “Next”](#next) [Nodes and transitions](/tasks/nodes-and-transitions/) # Pause and resume > Pause a task at an approval gate, inspect the paused node, and resume or rerun it with the same Task value. You end up with a task that stops before a side effect, waits for a person or an outside event, and then continues from the node where it stopped. The workspace-architect showcase uses this to hold its generated `ARCHITECTURE.md` until someone approves it: examples/workspace-architect/src/graph.rs ```rust task.register_transition(review, move |decision| match decision { Review::Approved(draft) => write.transitions_with(draft).into(), Review::AwaitingApproval => Transition::pause(), })?; ``` Source: [`examples/workspace-architect/src/graph.rs#L125-L128`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/graph.rs#L125-L128) The `review` node returns a `Review`. When the draft is approved, the transition sends it on to the `write` node. When it still waits for approval, the transition returns `Transition::pause()`, and `run` comes back with `TaskRunOutcome::Paused` instead of an output. The `.into()` on the first arm is the same `MarkedTransition` conversion described in [Nodes and transitions](/tasks/nodes-and-transitions/#transitions). The types live in `swiftide::tasks`, behind the `swiftide-tasks` [feature flag](/tasks/overview/#feature-flag): ```sh cargo add swiftide@0.34 --features swiftide-tasks ``` A task pause is not the same as an agent pause. To stop an agent until someone approves a single tool call, see [Human in the loop](/agents/human-in-the-loop/). ## Approval gate [Section titled “Approval gate”](#approval-gate) The `review` node is `ApprovalGate`, a task node that remembers the draft and whether someone approved it: examples/workspace-architect/src/approval.rs ```rust /// Lets a draft through once it is approved, and otherwise keeps it for review. /// /// The graph pauses the task when the gate answers [`Review::AwaitingApproval`]. A paused branch /// re-runs its node on `resume()`, so after [`ApprovalGate::approve`] the same draft passes /// through. The gate is cheap to evaluate, which is why the pause sits here and not on the /// expensive render node. #[derive(Debug, Clone)] pub struct ApprovalGate { state: Arc>, progress: Progress, } #[derive(Debug, Default)] struct GateState { approved: bool, pending: Option, } impl ApprovalGate { pub fn new(pre_approved: bool, progress: Progress) -> Self { let state = GateState { approved: pre_approved, pending: None, }; Self { state: Arc::new(Mutex::new(state)), progress, } } /// The draft waiting for a decision, if any. pub fn pending_draft(&self) -> Option { self.state().pending.clone() } pub fn approve(&self) { self.state().approved = true; } fn state(&self) -> MutexGuard<'_, GateState> { self.state.lock().unwrap_or_else(PoisonError::into_inner) } } #[async_trait] impl TaskNode for ApprovalGate { type Input = ArchitectureDraft; type Output = Review; type Error = Infallible; async fn evaluate( &self, _node_id: &DynNodeId, draft: &ArchitectureDraft, ) -> Result { let mut state = self.state(); if state.approved { state.pending = None; self.progress.step("review", "draft approved"); return Ok(Review::Approved(draft.clone())); } state.pending = Some(draft.clone()); self.progress.step( "review", format_args!( "draft ready ({} bytes), waiting for approval", draft.as_str().len() ), ); Ok(Review::AwaitingApproval) } } ``` Source: [`examples/workspace-architect/src/approval.rs#L23-L95`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/approval.rs#L23-L95) A paused branch keeps its current node and that node’s input. On `resume()`, the runtime evaluates the same node again with the same input, and then runs its transition again. The gate relies on this. The first evaluation stores the draft and answers `AwaitingApproval`. After `approve()`, the second evaluation answers `Approved` with the same draft. Put the pause on a cheap node. The showcase could pause right after `render`, but then `resume()` would call the model and draft the document a second time. The gate keeps its state in `Arc>`. The `Mutex` is there because `evaluate` and `approve` take `&self`, so plain fields cannot change. The `Arc` is there because a task node must be `Clone`, and `Mutex` is not. With the `Arc`, clones of the gate share one state. The gate’s error type is `Infallible`, because waiting is an answer, not a failure. ## Resume loop [Section titled “Resume loop”](#resume-loop) The command-line interface (CLI) runs the task and loops until it completes or the user declines the draft: examples/workspace-architect/src/main.rs ```rust let mut outcome = task.run(root).await?; loop { match outcome { TaskRunOutcome::Completed(written) => return Ok(Some(written)), TaskRunOutcome::Paused => { if !approve_paused_draft(&task)? { return Ok(None); } outcome = task.resume().await?; } } } ``` Source: [`examples/workspace-architect/src/main.rs#L125-L136`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/workspace-architect/src/main.rs#L125-L136) On `TaskRunOutcome::Paused`, `approve_paused_draft` (in the same file) finds the gate with `task.current_nodes::()`, shows the draft and asks for confirmation. On yes it calls `gate.approve()` and returns `true`, and the loop calls `task.resume()`. On no it returns `false`, and `draft_architecture` returns `Ok(None)` and drops the paused task. Nothing is written. This is the end of a captured run, with `claude-sonnet-5-5` as the writer. The draft between the review line and the question is trimmed: ```text [ 68.8s] review draft ready (11210 bytes), waiting for approval ... Write this ARCHITECTURE.md? [y/N] [ 68.8s] review draft approved [ 68.8s] write .../target/ARCHITECTURE.md (11210 bytes) Wrote .../target/ARCHITECTURE.md (11210 bytes) in 68.8s. ``` The `--yes` flag creates the gate already approved, so the first evaluation answers `Approved` and the task never pauses. ## Current nodes [Section titled “Current nodes”](#current-nodes) `Task::current_nodes::()` gives you the paused node, so you can read or change its state before you resume. This helper delivers the CI result from the next section through the task: ```rust /// Delivers a CI result through the task, for callers that hold the task instead of the node. pub fn deliver_through_task(task: &Task, result: &str) -> anyhow::Result<()> { let CurrentNodes::One(ci) = task.current_nodes::() else { anyhow::bail!("the task is not waiting on exactly one CI result"); }; ci.deliver(result); Ok(()) } ``` Source: [`snippets/src/tasks/pause_and_resume.rs#L92-L99`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/pause_and_resume.rs#L92-L99) The method returns the current nodes of type `T` as a `CurrentNodes`: * `CurrentNodes::None` when no current node has that type. * `CurrentNodes::One(&T)` for exactly one. * `CurrentNodes::Multiple(Vec<&T>)` for more than one. Paused branches take precedence. The method returns runnable nodes only when no branch is paused. In a parallel fan-out several branches can pause on the same node type, so handle `Multiple` there. The references point at the registered node instances. ## External callbacks [Section titled “External callbacks”](#external-callbacks) The same pattern waits for another system, such as a continuous integration (CI) webhook. The node reads a slot that the webhook handler fills, and the transition pauses while the slot is empty: ```rust /// Waits for a CI result that a webhook delivers from outside the task. #[derive(Debug, Clone, Default)] pub struct AwaitCi { result: Arc>>, } impl AwaitCi { /// Called by the webhook handler. Clones share the slot, so any clone can deliver. pub fn deliver(&self, result: impl Into) { let mut slot = self.result.lock().unwrap_or_else(PoisonError::into_inner); *slot = Some(result.into()); } } #[async_trait] impl TaskNode for AwaitCi { type Input = String; type Output = CiStatus; type Error = Infallible; async fn evaluate( &self, _node_id: &DynNodeId, commit: &String, ) -> Result { let slot = self.result.lock().unwrap_or_else(PoisonError::into_inner); Ok(match slot.as_deref() { Some(result) => CiStatus::Finished(format!("{commit}: {result}")), None => CiStatus::Pending, }) } } pub async fn wait_for_ci() -> anyhow::Result { let ci = AwaitCi::default(); let mut task = Task::::new(); let push = task .register_node_fn(|commit: &String| -> Result { Ok(commit.clone()) }); let wait = task.register_node(ci.clone()); task.starts_with(push); task.register_transition(push, move |commit| wait.transitions_with(commit))?; task.register_transition(wait, { let finish = task.transitions_to_finish(); move |status| match status { CiStatus::Finished(report) => finish(report), CiStatus::Pending => Transition::pause(), } })?; assert_eq!(task.run("4f2a91c").await?, TaskRunOutcome::Paused); // One task value holds one run: a second `run` while paused is refused. assert!(matches!( task.run("77b0e3d").await, Err(TaskError::TaskActive) )); // Later, the webhook handler that holds `ci` receives the result, // and the same task value continues. ci.deliver("all checks passed"); let TaskRunOutcome::Completed(report) = task.resume().await? else { anyhow::bail!("the CI result was delivered, so the task should finish"); }; ``` Source: [`snippets/src/tasks/pause_and_resume.rs#L23-L86`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/pause_and_resume.rs#L23-L86) The excerpt registers `ci.clone()` and keeps `ci` itself. In a server, `ci` goes to the webhook handler, and the comment marks where that handler runs. The clone shares the `Arc`, so `deliver` fills the slot that the registered node reads. A handler that holds the task instead can use `deliver_through_task` from the previous section. Keep the same `Task` value alive between the pause and the resume. The runtime state is not serializable. If the process restarts, the paused run is gone, so store what you need to start again, such as the commit. In a server, keep paused tasks in a map keyed by job. One task value holds one run. While the task has paused or queued work, `run` returns `TaskError::TaskActive`. Build a new task for the next job, for example with a function like the showcase’s `architect_task`. ## Reruns after reset [Section titled “Reruns after reset”](#reruns-after-reset) `reset()` discards the current run and queues a new one at the start node, with the last input passed to `run`. A `resume()` after it runs that queued run: ```rust let mut task = Task::::new(); let double = task.register_node_fn(|n: &i32| -> Result { Ok(n * 2) }); task.starts_with(double); task.register_transition(double, task.transitions_to_finish())?; assert_eq!(task.run(21).await?, TaskRunOutcome::Completed(42)); // A completed run leaves nothing to continue. assert!(matches!(task.resume().await, Err(TaskError::NotResumable))); // `reset` queues a new run at the start node with the last input. task.reset(); // That queued run counts as active work, so `run` with another input is refused. assert!(matches!(task.run(5).await, Err(TaskError::TaskActive))); assert_eq!(task.resume().await?, TaskRunOutcome::Completed(42)); // A clone has the same graph and a fresh runtime of its own. let mut copy = task.clone(); assert_eq!(copy.run(5).await?, TaskRunOutcome::Completed(10)); ``` Source: [`snippets/src/tasks/pause_and_resume.rs#L104-L121`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/tasks/pause_and_resume.rs#L104-L121) * After a completed run there is nothing to continue, so `resume()` returns `TaskError::NotResumable`. * `reset()` also discards a paused run. Use it to start over with the same input instead of continuing. * The queued run counts as active work, so `run(other_input)` after `reset()` returns `TaskError::TaskActive`. To start over with a different input, use `task.clone()`. Calling `starts_with` again also clears the runtime state. * `reset()` before the first `run` has no input to queue. The `resume()` that follows returns `TaskError::NotResumable`. * A failed run leaves no state to resume. `reset()` and `resume()` retry it with the same input. * `task.clone()` copies the graph with a fresh runtime, without the paused run. Node state behind an `Arc` is shared between the copies. ## Pauses in parallel branches [Section titled “Pauses in parallel branches”](#pauses-in-parallel-branches) A pause stops the whole task, not only its branch. When a branch asks to pause, the runtime starts no new branches. Parallel branches that are already running finish their current node and transition, and their next steps wait in the queue. Then `run` returns `Paused`, unless one of those running branches reaches the finish transition. In that case `run` returns `Completed` and the paused branch is discarded. If a running branch fails after another branch asked to pause, `run` returns that error. The paused run is lost, and `resume()` has nothing to continue. `resume()` puts the paused branches back in the queue in the order they were created and continues with all queued work. A join runs only when every branch has arrived, so a paused branch also holds back its join. [Fan-out and join](/tasks/fan-out-and-join/#join-node) covers branches and joins. ## Reference [Section titled “Reference”](#reference) * [`Transition::pause`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Transition.html#method.pause) * [`MarkedTransition`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.MarkedTransition.html) * [`Task::run`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.run) * [`Task::resume`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.resume) * [`Task::reset`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.reset) * [`Task::current_nodes`](https://docs.rs/swiftide/0.34/swiftide/tasks/struct.Task.html#method.current_nodes) * [`CurrentNodes`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.CurrentNodes.html) * [`TaskRunOutcome`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskRunOutcome.html) * [`TaskError`](https://docs.rs/swiftide/0.34/swiftide/tasks/enum.TaskError.html) * [`TaskNode`](https://docs.rs/swiftide/0.34/swiftide/tasks/trait.TaskNode.html) ## Next [Section titled “Next”](#next) [Agents as task nodes](/tasks/agents-as-nodes/) # Agent control recipes > Resume an agent from saved message history, end a run with a typed stop or failure payload, and give a tool a hand-written JSON Schema. Four recipes give you an agent that continues a saved conversation, an agent that ends with a typed report or a typed failure, and a tool whose argument schema you write yourself. ## Setup [Section titled “Setup”](#setup) Add Swiftide with the OpenAI integration and agents, plus the crates the recipes use: ```sh cargo add swiftide@0.34 --features openai,swiftide-agents cargo add schemars@1 serde serde_json --features serde/derive cargo add tokio --features full cargo add anyhow async-trait ``` Every agent below borrows an OpenAI client built like this: ```rust let openai = swiftide::integrations::openai::OpenAI::builder() .default_embed_model("text-embeddings-3-small") .default_prompt_model("gpt-4o-mini") .build()?; ``` Source: [`examples/agents_resume.rs#L10-L13`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_resume.rs#L10-L13) Most recipes are examples in the swiftide repository. To run them, clone it and set an OpenAI key: ```sh git clone https://github.com/bosun-ai/swiftide && cd swiftide export OPENAI_API_KEY=sk-... ``` The examples crate turns on most feature flags, so the first build takes several minutes. Each recipe ends with its run command. ## Resumed agents [Section titled “Resumed agents”](#resumed-agents) A conversation that has to survive a restart needs its message history stored. The first agent answers one query: ```rust let mut first_agent = agents::Agent::builder().llm(&openai).build()?; first_agent.query("Say hello!").await?; ``` Source: [`examples/agents_resume.rs#L17-L19`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_resume.rs#L17-L19) Then the example stores its history and loads it into a new context: ```rust // Let's store the messages in a database, retrieve them back, and start a new agent let stored_history = serde_json::to_string(&first_agent.history().await?)?; let retrieved_history: Vec<_> = serde_json::from_str(&stored_history)?; let restored_context = DefaultContext::default() .with_existing_messages(retrieved_history) .await? .to_owned(); ``` Source: [`examples/agents_resume.rs#L23-L30`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_resume.rs#L23-L30) A second agent continues from that context: ```rust let mut second_agent = agents::Agent::builder() .llm(&openai) .context(restored_context) // We'll use the one from the first agent, alternatively we could also pop it from the // previous history and add a new one here .no_system_prompt() .build()?; second_agent.query("What did you say?").await?; ``` Source: [`examples/agents_resume.rs#L34-L42`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agents_resume.rs#L34-L42) `Agent::history` returns the message history as a `Vec`, and `ChatMessage` round-trips through JSON (JavaScript Object Notation). `DefaultContext::with_existing_messages` replaces the context’s message history with the messages you pass. [Message history](/agents/context-and-history/#message-history) explains where those messages live. A fresh agent appends its system prompt to the context on its first run, after the restored messages. If the stored history already starts with a system message, that adds a second one. The example’s first agent has no system prompt, and `no_system_prompt()` keeps the second agent the same. The comment in the `resume` excerpt dates from before 0.33, when agents had a default system prompt. ```sh cargo run -p swiftide-examples --example agents-resume ``` ### Message history backends [Section titled “Message history backends”](#message-history-backends) `DefaultContext::with_message_history` takes any `MessageHistory` backend, so you do not have to serialize the history yourself. [Redis storage](/agents/context-and-history/#redis-storage) shows the `Redis` backend. ## Typed stop payloads [Section titled “Typed stop payloads”](#typed-stop-payloads) When the caller needs a structured final answer, describe it as a type: ```rust #[derive(Debug, Deserialize, JsonSchema)] #[serde(rename_all = "snake_case")] enum TaskStatus { Succeeded, Failed, Cancelled, } #[derive(Debug, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] struct StopPayload { status: TaskStatus, summary: String, /// Extra context for the reader, or null when there is nothing to add. details: Option, } ``` Source: [`snippets/src/cookbook/agent_control.rs#L17-L32`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/cookbook/agent_control.rs#L17-L32) Turn its schema into the stop tool, register it in place of the default one and read the payload when the agent stops: ```rust let stop_tool = StopWithArgs::with_parameters_schema(schema_for!(StopPayload)); let mut agent = Agent::builder() .llm(&openai) .system_prompt( "Summarize the completed work. When done, call `stop` with the JSON schema. \ Use null for `details` when there is nothing to add.", ) .without_default_stop_tool() .tools([stop_tool]) .on_stop(|_agent, reason, _| { Box::pin(async move { if let StopReason::RequestedByTool(_call, Some(payload)) = reason { let report = StopPayload::deserialize(&payload)?; println!("{:?}: {}", report.status, report.summary); if let Some(details) = report.details { println!("{details}"); } } Ok(()) }) }) .build()?; agent .query_once("You onboarded five merchants today. Write the handoff report and stop.") .await?; ``` Source: [`snippets/src/cookbook/agent_control.rs#L38-L64`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/cookbook/agent_control.rs#L38-L64) `StopWithArgs::with_parameters_schema` builds a tool named `stop` and forwards the whole argument object as the payload. The `on_stop` hook receives `StopReason::RequestedByTool(tool_call, Some(payload))`. A `&serde_json::Value` is a deserializer, so `StopPayload::deserialize(&payload)` turns it into your type. Why `without_default_stop_tool()` comes before `tools`: [Stopping and structured results](/agents/stopping-and-structured-results/#structured-results). The OpenAI client marks every property as required when it sends a tool schema. The model sends `details` even when it is optional, which is why the system prompt asks for `null`. `query_once` runs one completion and the tool calls it returns, then hands control back. Every field needs a schema that is an object. A field of type `serde_json::Value` or `Option` without a description produces the schema `true`, and `tool_spec()` panics with `NodeMustBeObject`. The `stop_with_args_custom_schema.rs` example in the repository declares `details` that way, so at this version it panics before it reaches the model. Use a concrete type, or add `#[schemars(description = "...")]` to the field. ### Payload after the run [Section titled “Payload after the run”](#payload-after-the-run) You can also read the payload after `query_once` returns. `Agent::stop_reason` returns the reason of the last stop: ```rust #[derive(Debug, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] struct HandoffReport { summary: String, next_actions: Vec, } let mut agent = Agent::builder() .llm(&openai) .without_default_stop_tool() .tools([StopWithArgs::with_parameters_schema(schema_for!( HandoffReport ))]) .build()?; agent .query_once("Write the handoff report for today's onboarding and stop.") .await?; if let Some(StopReason::RequestedByTool(_call, Some(payload))) = agent.stop_reason() { let report: HandoffReport = serde_json::from_value(payload.clone())?; println!("{}: {:?}", report.summary, report.next_actions); } ``` Source: [`snippets/src/cookbook/agent_control.rs#L72-L94`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/cookbook/agent_control.rs#L72-L94) ## Typed failure payloads [Section titled “Typed failure payloads”](#typed-failure-payloads) A model that has to give up can hand your code a report to route, for example to an on-call queue. Describe the report: ```rust #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(rename_all = "snake_case")] enum FailureCategory { MissingDependency, PermissionDenied, UnexpectedRegression, } #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(rename_all = "snake_case")] enum RemediationStatus { Planned, Blocked, Complete, } #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[schemars(deny_unknown_fields)] struct FailureReport { category: FailureCategory, summary: String, impact: String, #[serde(default, skip_serializing_if = "Vec::is_empty")] recommended_actions: Vec, #[serde(default, skip_serializing_if = "Option::is_none")] remediation_status: Option, } fn failure_schema() -> Schema { schema_for!(FailureReport) } ``` Source: [`examples/agent_can_fail_custom_schema.rs#L16-L46`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agent_can_fail_custom_schema.rs#L16-L46) `AgentCanFail::with_parameters_schema` turns that schema into the arguments of the `task_failed` tool. The example prints the resulting tool spec: ```rust let schema = failure_schema(); let failure_tool = AgentCanFail::with_parameters_schema(schema.clone()); println!( "task_failed tool schema:\n{}", to_string_pretty(&failure_tool.tool_spec())?, ); ``` Source: [`examples/agent_can_fail_custom_schema.rs#L54-L60`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agent_can_fail_custom_schema.rs#L54-L60) The agent keeps the default stop tool and adds `task_failed`, so the model can either finish or fail. The `on_stop` hook prints the failure payload: ```rust let mut builder = Agent::builder(); builder .llm(&openai) .system_prompt( "You are an incident response coordinator. If the task cannot be completed, call \ `task_failed` with the provided JSON schema. Populate every required field, include \ at least one `recommended_actions` entry, and document the impact. Do not claim \ success while blockers remain unresolved.", ) .tools([failure_tool.clone()]) .on_stop(|_, reason, _| { Box::pin(async move { if let StopReason::AgentFailed(Some(payload)) = reason { let json = to_string_pretty(&payload).unwrap(); println!("agent reported failure:\n{json}"); } Ok(()) }) }); let mut agent = builder.build()?; ``` Source: [`examples/agent_can_fail_custom_schema.rs#L71-L91`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agent_can_fail_custom_schema.rs#L71-L91) ```rust agent .query_once( "You must restore last night's database backup, but the only backup file is corrupted and no redundant copy exists. Report the failure.", ) .await?; ``` Source: [`examples/agent_can_fail_custom_schema.rs#L95-L99`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/agent_can_fail_custom_schema.rs#L95-L99) The run returns `Ok` even when the agent fails, and the payload is JSON text inside a `Value::String` ([Failure reports](/agents/stopping-and-structured-results/#failure-reports) explains both). Decode it after the run: ```rust #[derive(Debug, Deserialize)] struct FailureReport { summary: String, impact: String, } if let Some(StopReason::AgentFailed(Some(Value::String(json)))) = agent.stop_reason() { let report: FailureReport = serde_json::from_str(json)?; println!("failed: {} ({})", report.summary, report.impact); } ``` Source: [`snippets/src/cookbook/agent_control.rs#L102-L111`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/cookbook/agent_control.rs#L102-L111) Your reading type can hold fewer fields than the schema type, because serde ignores unknown fields unless you deny them. ```sh cargo run -p swiftide-examples --example agent-can-fail-custom-schema ``` ## Hand-written tool schemas [Section titled “Hand-written tool schemas”](#hand-written-tool-schemas) `#[swiftide::tool]` cannot express every argument shape, and it gives you no say over the schema title, field descriptions or enum values. A hand-written tool can. Describe the arguments: ```rust #[derive(Clone)] struct WorkflowTool; #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[schemars( title = "WorkflowInstruction", description = "Choose a workflow action and optional payload", deny_unknown_fields )] struct WorkflowInstruction { #[schemars(description = "Which workflow action to execute")] action: WorkflowAction, #[schemars(description = "Optional payload forwarded to the workflow engine")] #[serde(default, skip_serializing_if = "Option::is_none")] payload: Option, } #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] #[serde(rename_all = "lowercase")] enum WorkflowAction { Start, Stop, Status, } ``` Source: [`examples/tool_custom_schema.rs#L11-L34`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/tool_custom_schema.rs#L11-L34) Implement `Tool` and return the schema from `tool_spec`: ```rust #[swiftide::reexports::async_trait::async_trait] impl Tool for WorkflowTool { async fn invoke( &self, _agent_context: &dyn AgentContext, _tool_call: &ToolCall, ) -> Result { Ok(ToolOutput::text( "Workflow execution not implemented in this example", )) } fn name<'tool>(&'tool self) -> Cow<'tool, str> { Cow::Borrowed("workflow_tool") } fn tool_spec(&self) -> ToolSpec { ToolSpec::builder() .name("workflow_tool") .description("Executes a workflow action with strict input choices") .parameters_schema(workflow_schema()) .build() .expect("tool spec should be valid") } } fn workflow_schema() -> Schema { schema_for!(WorkflowInstruction) } ``` Source: [`examples/tool_custom_schema.rs#L38-L66`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/tool_custom_schema.rs#L38-L66) ```rust let tool = WorkflowTool; let spec = tool.tool_spec(); println!( "{}", serde_json::to_string_pretty(&spec).expect("tool spec should serialize"), ); ``` Source: [`examples/tool_custom_schema.rs#L71-L77`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/tool_custom_schema.rs#L71-L77) The `#[schemars(description = ...)]` on `payload` matters here. Without it, the `serde_json::Value` field gets the schema `true` and `ToolSpec::builder().build()` returns an error. [Hand-written tools](/agents/tools/#hand-written-tools) lists the other schema rules and why `name()` must match the spec name. Swiftide has no blanket conversion from a `Tool` into `Box`, so `.tools([WorkflowTool])` does not compile. Register a hand-written tool with `add_tool`, or box it yourself: ```rust #[derive(Clone)] struct CurrentBranch; #[async_trait::async_trait] impl Tool for CurrentBranch { async fn invoke( &self, _context: &dyn AgentContext, _tool_call: &ToolCall, ) -> Result { Ok(ToolOutput::text("main")) } fn name(&self) -> Cow<'_, str> { "current_branch".into() } fn tool_spec(&self) -> ToolSpec { ToolSpec::builder() .name("current_branch") .description("Returns the name of the checked out git branch") .build() .expect("tool spec should be valid") } } let agent = Agent::builder() .llm(&openai) .add_tool(CurrentBranch) .build()?; let same_agent = Agent::builder() .llm(&openai) .tools([CurrentBranch.boxed()]) .build()?; ``` Source: [`snippets/src/cookbook/agent_control.rs#L119-L153`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/cookbook/agent_control.rs#L119-L153) This example needs no API key. It prints the tool spec as JSON: ```sh cargo run -p swiftide-examples --example tool-custom-schema ``` ## Reference [Section titled “Reference”](#reference) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html), [`Agent::history`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.history) and [`Agent::stop_reason`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html#method.stop_reason) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html): [`without_default_stop_tool`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.without_default_stop_tool), [`no_system_prompt`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.no_system_prompt), [`add_tool`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.add_tool) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html): [`with_existing_messages`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.with_existing_messages), [`with_message_history`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html#method.with_message_history) * [`MessageHistory`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.MessageHistory.html) and [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) and [`AgentCanFail`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.AgentCanFail.html) * [`Tool`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html) and [`Tool::boxed`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/trait.Tool.html#method.boxed), [`ToolSpec`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpec.html) and [`ToolSpecBuilder`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolSpecBuilder.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) and [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) ## Next [Section titled “Next”](#next) [OpenAI Responses API and reasoning](/cookbook/responses-api-and-reasoning/) # Agents on AWS Bedrock > Run a Swiftide agent with two tools on a Claude model hosted by Amazon Web Services (AWS) Bedrock, and read why it stopped. The agent on this page runs on a Claude model hosted by Amazon Web Services (AWS) Bedrock, calls two tools and then stops on its own. Client setup, credentials, region and runtime requirements are on [AWS Bedrock](/integrations/aws-bedrock/). This page covers what changes once that client drives an agent. ## Setup [Section titled “Setup”](#setup) Bedrock sits behind the `aws-bedrock` feature flag, and agents behind `swiftide-agents`: ```sh cargo add swiftide@0.34 --features aws-bedrock,swiftide-agents cargo add schemars@1 serde --features serde/derive cargo add tokio --features macros,rt-multi-thread cargo add anyhow ``` The full example lives in the swiftide repository. To run it, clone the repository and start it from the root: ```sh git clone https://github.com/bosun-ai/swiftide && cd swiftide cargo run -p swiftide-examples --example aws-bedrock-agent ``` The `swiftide-examples` package turns on the `all` feature flag. Its first build compiles every integration, including LanceDB, FastEmbed, Kafka and Fluvio. Those need native build tools such as `protoc` and `cmake`. If you only want Bedrock, copy the snippets below into your own crate instead. ## Tools [Section titled “Tools”](#tools) examples/aws_bedrock_agent.rs ```rust #[derive(Debug, Clone, Serialize, Deserialize, JsonSchema)] struct FormatTimestampRequest { /// Prefix to prepend to the timestamp. prefix: String, /// Timestamp to format. timestamp: String, } #[swiftide::tool(description = "Get the current UTC date and time in RFC3339 format")] async fn current_utc_time(context: &dyn AgentContext) -> Result { let command_output = context .executor() .exec_cmd(&Command::shell("date -u +\"%Y-%m-%dT%H:%M:%SZ\"")) .await?; Ok(command_output.into()) } #[swiftide::tool( description = "Format a timestamp with a caller-provided prefix", param(name = "request", description = "Timestamp formatting input") )] async fn format_timestamp( _context: &dyn AgentContext, request: FormatTimestampRequest, ) -> Result { Ok(ToolOutput::text(format!( "{}{}", request.prefix, request.timestamp ))) } ``` Source: [`examples/aws_bedrock_agent.rs#L23-L53`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L23-L53) The tools import `swiftide::chat_completion::{ToolOutput, errors::ToolError}` and `swiftide::traits::{AgentContext, Command}`. The client comes from `swiftide::integrations::aws_bedrock_v2::AwsBedrock`. `#[swiftide::tool]` turns each async function into a tool. It also generates a function with the same name that returns a `Box`. The `description` attribute becomes the tool description the model reads. `current_utc_time` runs a shell command through the tool executor of the agent’s context. The default context uses a `LocalExecutor`, so `date` runs on your machine. `exec_cmd` returns a `CommandOutput`, which keeps stdout and stderr as separate chunks since 0.33. `.into()` turns it into `ToolOutput::Text`, with all output as text in the order it arrived. `format_timestamp` takes one parameter, `request`, of type `FormatTimestampRequest`. The model sends the arguments as a JavaScript Object Notation (JSON) object with a `request` field. That field holds `prefix` and `timestamp`. The struct derives `JsonSchema`, so its doc comments become the field descriptions in the schema. A struct used as a tool parameter needs `Serialize`, `Deserialize`, `JsonSchema` and `Debug`. Bedrock receives every tool spec with strict schema validation turned on. That comes from the `tool_strict` option, which is `true` when you leave it unset. To turn it off, pass `Options::builder().tool_strict(false).build()?` to `AwsBedrockBuilder::default_options`. ## Agent [Section titled “Agent”](#agent) examples/aws_bedrock_agent.rs ```rust let bedrock = AwsBedrock::builder() .default_prompt_model("global.anthropic.claude-sonnet-4-6") .build()?; ``` Source: [`examples/aws_bedrock_agent.rs#L61-L63`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L61-L63) examples/aws_bedrock_agent.rs ```rust let mut agent = agents::Agent::builder() .llm(&bedrock) .tools(vec![current_utc_time(), format_timestamp()]) .on_new_message(|_, msg| { let rendered = msg.to_string(); Box::pin(async move { println!("{rendered}"); Ok(()) }) }) .limit(6) .build()?; ``` Source: [`examples/aws_bedrock_agent.rs#L67-L78`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L67-L78) `llm(&bedrock)` gives the agent the Bedrock client. `tools` registers both tools next to the default `stop` tool. The `on_new_message` hook runs before each message enters the message history. It receives the agent and the message. The future it returns may not borrow the message, so the example renders the message to a string first. The `Display` form of `ChatMessage` shows the role, the text and any tool calls. `limit(6)` caps the agent at six completions. Turns that only run pending tool calls do not count. At the cap, Swiftide logs “Agent loop limit reached” and stops the agent with `StopReason::NoNewMessages`. ## Run [Section titled “Run”](#run) examples/aws_bedrock_agent.rs ```rust agent .query( "Call current_utc_time once. Then call format_timestamp with prefix \"UTC now: \" and \ that timestamp. After that, report the formatted result and stop.", ) .await?; ``` Source: [`examples/aws_bedrock_agent.rs#L82-L87`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/aws_bedrock_agent.rs#L82-L87) `query` adds the prompt as a user message and runs the loop. The prompt asks the model to call `current_utc_time`, then `format_timestamp`, then report the result and stop. The model decides how it finishes, and the two ways lead to different stop reasons: * It answers in text without a tool call. The default context then has no new messages to send, and the agent stops with `StopReason::NoNewMessages`. * It calls the default `stop` tool. The agent stops with `StopReason::RequestedByTool`, which carries the tool call. After `query` returns, `agent.stop_reason()` tells you which one happened. Branch on it when your code needs to know whether the model finished or the loop ran out. ## Variations [Section titled “Variations”](#variations) * To make the model always finish through a tool, keep the `stop` tool and say so in the prompt. To remove it, call `without_default_stop_tool` on the builder before `tools`, or register your own tool named `stop` with `add_tool`. A tool named `stop` passed to `tools` is dropped, because the default one is already in the set. * To keep the agent going after a text answer, give it a `DefaultContext` with `with_stop_on_assistant(false)`. Then only the `stop` tool or the limit ends the run. * A second `query` on the same agent starts with a fresh loop counter, so it gets six completions again. * Bedrock has its own `Options` and its own `ReasoningEffort` with `Low`, `Medium`, `High` and `Max`. They are not the OpenAI types. See [AWS Bedrock settings](/models/overview/#aws-bedrock-settings). ## Gotchas [Section titled “Gotchas”](#gotchas) * The agent needs a model that supports tool use through the Converse API. * `#[swiftide::tool]` reads the parameter names from the function signature. Renaming `request` changes the JSON field the model must send. ## Reference [Section titled “Reference”](#reference) * [AWS Bedrock integration](/integrations/aws-bedrock/) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`AwsBedrockBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrockBuilder.html) * [`Options` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.Options.html) * [`ReasoningEffort` (AWS Bedrock)](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/enum.ReasoningEffort.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html) * [`StopReason`](https://docs.rs/swiftide/0.34/swiftide/agents/enum.StopReason.html) * [`DefaultContext`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.DefaultContext.html) * [`Stop`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.Stop.html) * [`LocalExecutor`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/local_executor/struct.LocalExecutor.html) * [`#[swiftide::tool]`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) * [`AgentContext`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.AgentContext.html) * [`Command`](https://docs.rs/swiftide/0.34/swiftide/traits/enum.Command.html) * [`CommandOutput`](https://docs.rs/swiftide/0.34/swiftide/traits/struct.CommandOutput.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) ## Next [Section titled “Next”](#next) [Structured prompts](/cookbook/structured-prompts/) # Describing images > Send a Portable Network Graphics (PNG) image to an OpenAI model as a base64 data URL in a multi-part user message and print a one-sentence description. This recipe sends the Swiftide logo to `gpt-4o-mini` and prints the one-sentence description the model writes back. It is the `describe_image.rs` example from the swiftide repository: one user message that carries a text part and an image part, sent with a single chat completion call. ## Setup [Section titled “Setup”](#setup) To run the example, clone the swiftide repository, set `OPENAI_API_KEY` and run it from the repository root: ```sh git clone https://github.com/bosun-ai/swiftide && cd swiftide export OPENAI_API_KEY=sk-... cargo run -p swiftide-examples --example describe-image ``` The examples crate turns on every integration, so the first build compiles all of them. It needs `cmake` and the Protocol Buffers compiler (`protoc`) on your machine. To use the code in your own crate, add the OpenAI client, which sits behind the `openai` feature flag, and the `base64` crate to encode the image: ```sh cargo add swiftide@0.34 --features openai cargo add base64@0.22 anyhow cargo add tokio --features macros,rt-multi-thread ``` ## Client [Section titled “Client”](#client) ```rust let openai = swiftide::integrations::openai::OpenAI::builder() .default_prompt_model("gpt-4o-mini") .build()?; ``` Source: [`examples/describe_image.rs#L15-L17`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/describe_image.rs#L15-L17) The prompt model must accept images. `gpt-4o-mini` does. The client reads the API key from `OPENAI_API_KEY`. ## Image data [Section titled “Image data”](#image-data) ```rust let image_path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../images/logo.png"); let image_bytes = std::fs::read(&image_path).with_context(|| format!("Read {image_path:?}"))?; let encoded = general_purpose::STANDARD.encode(&image_bytes); let data_url = format!("data:image/png;base64,{encoded}"); ``` Source: [`examples/describe_image.rs#L21-L24`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/describe_image.rs#L21-L24) `env!("CARGO_MANIFEST_DIR")` is the directory of the examples crate, fixed at compile time. The path climbs one level to the repository root and reads `images/logo.png`. The example then encodes the bytes as base64 and wraps them in a data Uniform Resource Locator (URL) of the form `data:image/png;base64,...`. The snippets leave out the imports. `encode` comes from the `base64::Engine` trait, and `with_context` from `anyhow::Context`, so bring both into scope with `use base64::{Engine as _, engine::general_purpose};` and `use anyhow::Context as _;`. The request types come from `swiftide::chat_completion`, and `complete` from `swiftide::traits::ChatCompletion`. The media type in the data URL tells the model how to decode the bytes. Use the type of the file you actually send, such as `image/jpeg` for a Joint Photographic Experts Group (JPEG) file. ## Request [Section titled “Request”](#request) ```rust let message = ChatMessage::new_user_with_parts(vec![ ChatMessageContentPart::text("Describe this image in one sentence."), ChatMessageContentPart::image(data_url), ]); let request = ChatCompletionRequest::builder() .messages(vec![message]) .build()?; ``` Source: [`examples/describe_image.rs#L28-L35`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/describe_image.rs#L28-L35) The message carries a text part and an image part, in that order. The data URL is a string, so `ChatMessageContentPart::image` takes it as a URL source. [Images and multimodal input](/models/images/#content-parts) explains content parts and sources. The request has no system message here. Add one with `ChatMessage::new_system` in front of the user message when you want to set the tone or length of the answer. ## Response [Section titled “Response”](#response) ```rust let response = openai.complete(&request).await?; println!( "Image description: {}", response.message().unwrap_or("") ); ``` Source: [`examples/describe_image.rs#L39-L43`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/describe_image.rs#L39-L43) `complete` sends one request and returns a `ChatCompletionResponse`. `message()` returns the text of the answer as an `Option<&str>`. ## Variations [Section titled “Variations”](#variations) ### Bytes instead of a data URL [Section titled “Bytes instead of a data URL”](#bytes-instead-of-a-data-url) You can skip the base64 step. Pass the raw bytes and the media type, as `ChatMessageContentSource::bytes(image_bytes, Some("image/png"))`, and the OpenAI client builds the same data URL for you. A plain `Vec` also works, but it has no media type, and the OpenAI clients then label the image `application/octet-stream`. [Images and multimodal input](/models/images/) shows this form in full. ### Images on the web [Section titled “Images on the web”](#images-on-the-web) With the OpenAI client, an `https://` URL works as is: `ChatMessageContentPart::image("https://example.com/chart.png")`. The client passes the URL to the provider unchanged. ### Several images [Section titled “Several images”](#several-images) A message can carry more than one image part. Add each one to the list, and refer to them by position in the text part, for example “Compare the first and the second image”. ### Other providers [Section titled “Other providers”](#other-providers) The Anthropic client rejects image parts, and Amazon Web Services (AWS) Bedrock needs an image format and accepts only `data:` and `s3://` URLs. The [provider support table](/models/images/#provider-support) lists what each client accepts. ## Gotchas [Section titled “Gotchas”](#gotchas) * `CARGO_MANIFEST_DIR` is a compile-time path. A binary you copy to another machine does not find the file there. Read the path from configuration or a command-line argument, or embed the file with `include_bytes!`. * A large image makes a large request. Resize it before you encode it when the model does not need full resolution. ## Reference [Section titled “Reference”](#reference) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`ChatMessageContentPart`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessageContentPart.html) * [`ChatMessageContentSource`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessageContentSource.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) ## Next [Section titled “Next”](#next) [Usage metrics](/cookbook/usage-metrics/) # Hybrid search and reranking > Retrieve with dense and sparse vectors fused in Qdrant, and rerank retrieved documents with a local FastEmbed cross-encoder before answering. You build two query pipelines over Qdrant: one that fuses dense and sparse vector search, and one that reranks retrieved documents with a local cross-encoder before the large language model (LLM) answers. Both are runnable examples in the swiftide repository: `hybrid-search` and `reranking`. ## Setup [Section titled “Setup”](#setup) To run the examples, clone the repository and start Qdrant with its gRPC port 6334 exposed ([Running Qdrant](/integrations/qdrant/#local-instance-and-connection) covers the options). Set `OPENAI_API_KEY`. Run from the repository root, because the examples read `swiftide-core/` and `README.md` by relative path: ```sh git clone https://github.com/bosun-ai/swiftide && cd swiftide docker run -d -p 6333:6333 -p 6334:6334 qdrant/qdrant cargo run -p swiftide-examples --example hybrid-search cargo run -p swiftide-examples --example reranking ``` To use the same code in your own project, add Swiftide with the feature flags the examples use, plus the async runtime and logging they call from `main`: ```sh cargo add swiftide@0.34 --features openai,qdrant,fastembed,tree-sitter cargo add tokio --features full cargo add tracing-subscriber ``` `tree-sitter` is only needed for the code chunker in the hybrid example. ## Hybrid indexing [Section titled “Hybrid indexing”](#hybrid-indexing) examples/hybrid_search.rs ```rust // Ensure all batching is consistent let batch_size = 64; let fastembed_sparse = FastEmbed::try_default_sparse().unwrap().to_owned(); let fastembed = FastEmbed::try_default().unwrap().to_owned(); // Set up openai with the mini model, which is great for indexing let openai = openai::OpenAI::builder() .default_prompt_model("gpt-4o-mini") .build() .unwrap(); // Set up qdrant and use the combined fields (metadata + chunks) for both sparse and dense // vectors let qdrant = Qdrant::builder() .batch_size(batch_size) .vector_size(384) .with_vector(EmbeddedField::Combined) .with_sparse_vector(EmbeddedField::Combined) .collection_name("swiftide-hybrid-example") .build()?; ``` Source: [`examples/hybrid_search.rs#L23-L43`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs#L23-L43) `FastEmbed::try_default()` loads a local dense embedding model with 384 dimensions, which is why the collection uses `vector_size(384)`. `FastEmbed::try_default_sparse()` loads SPLADE, a sparse model that weights individual terms. A dense vector matches on meaning. A sparse vector also rewards exact words, such as a type or function name. `with_vector` and `with_sparse_vector` both name `EmbeddedField::Combined`: the chunk and its metadata embedded as one text. Qdrant stores the two vectors on the same point. examples/hybrid_search.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("swiftide-core/").with_extensions(&["rs"])) // Chunk fairly large as the context window is big .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) // Generate metadata on the code chunks to increase our chances of finding the right code .then(MetadataQACode::from_client(openai.clone()).build().unwrap()) .then_in_batch( transformers::SparseEmbed::new(fastembed_sparse.clone()).with_batch_size(batch_size), ) .then_in_batch(transformers::Embed::new(fastembed.clone()).with_batch_size(batch_size)) .then_store_with(qdrant.clone()) .run() .await?; ``` Source: [`examples/hybrid_search.rs#L47-L61`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs#L47-L61) `ChunkCode` splits the Rust files in `swiftide-core/` with tree-sitter. `MetadataQACode` asks `gpt-4o-mini` for five questions and answers per chunk and stores them under the `Questions and Answers (code)` metadata key. That metadata is part of the combined text, so both vectors include it. `SparseEmbed` and `Embed` are batch transformers. The example uses one `batch_size` of 64 for both of them and for the Qdrant storage. ## Hybrid query [Section titled “Hybrid query”](#hybrid-query) examples/hybrid_search.rs ```rust // Use sophisticated model for our query let openai = openai::OpenAI::builder() .default_prompt_model("gpt-4o") .build() .unwrap(); let query_pipeline = query::Pipeline::from_search_strategy( // Return a large amount of documents because we have a large context window // By default it uses the Combined fields, no need to configure HybridSearch::default().with_top_n(20).with_top_k(20), ) // Generate subquestions on the initial query to increase our query coverage .then_transform_query(query_transformers::GenerateSubquestions::from_client( openai.clone(), )) // Generate the same embeddings we used for indexing .then_transform_query(query_transformers::Embed::from_client(fastembed.clone())) .then_transform_query(query_transformers::SparseEmbed::from_client( fastembed_sparse.clone(), )) .then_retrieve(qdrant.clone()) // Answer with Simple, which either takes the documents as is (in this case), or any // transformations applied after querying .then_answer(answers::Simple::from_client(openai.clone())); ``` Source: [`examples/hybrid_search.rs#L65-L88`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs#L65-L88) `HybridSearch::default()` searches the `Combined` dense and sparse vectors. `with_top_n(20)` limits each sub-search to 20 candidates. `with_top_k(20)` limits the fused result. The Qdrant retriever needs both query embeddings. `query_transformers::Embed` sets the dense one and `query_transformers::SparseEmbed` the sparse one, each with the model used for indexing. If either is missing, retrieval fails with `No embedding for query` or `No sparse embedding for query`. `GenerateSubquestions` runs first. Its default prompt asks `gpt-4o` for the original question plus five extra questions. That list becomes the current query, and both embeddings are made from it. Qdrant fuses the two searches with Reciprocal Rank Fusion (RRF); see [Retrieval and search strategies](/querying/retrieval-and-search-strategies/). examples/hybrid_search.rs ```rust let answer = query_pipeline .query("What are the different pipelines in Swiftide and how do they work? Provide an elaborate answer with examples.") .await .unwrap(); println!("{}", answer.answer()); ``` Source: [`examples/hybrid_search.rs#L92-L97`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/hybrid_search.rs#L92-L97) ### Stores and filters [Section titled “Stores and filters”](#stores-and-filters) `Qdrant` implements `Retrieve` for `HybridSearch` with a Qdrant `Filter` type, so type inference picks that filter type for `HybridSearch::default()`. Pass a filter with `HybridSearch::from_filter` or `with_filter`, and Qdrant applies it to both sub-searches. [Retrieval and search strategies](/querying/retrieval-and-search-strategies/) shows a filter. `Duckdb`, behind the `duckdb` feature flag, also implements `Retrieve` for `HybridSearch`, without a filter. It combines cosine similarity with a full-text keyword score on the query text, so it needs only the dense query embedding. `LanceDB` and `PgVector` do not implement it. ### Existing collections [Section titled “Existing collections”](#existing-collections) Qdrant creates the collection with its named vectors only when the collection does not exist yet. For an existing collection it logs a warning and keeps the old configuration. A collection created without `with_sparse_vector` stays without a sparse vector. Use a new collection name, as the example does with `swiftide-hybrid-example`, or delete the old collection first. ## Reranking [Section titled “Reranking”](#reranking) examples/reranking.rs ```rust let openai_client = integrations::openai::OpenAI::builder() .default_prompt_model("gpt-4o") .build()?; let fastembed = fastembed::FastEmbed::builder().batch_size(10).build()?; let reranker = fastembed::Rerank::builder().top_k(5).build()?; let qdrant = Qdrant::builder() .batch_size(50) .vector_size(384) .collection_name("swiftide-reranking") .build()?; ``` Source: [`examples/reranking.rs#L25-L36`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs#L25-L36) `Rerank` is a response transformer: `top_k(5)` keeps the five best of the retrieved documents. [Reranking and response transformers](/querying/reranking-and-response-transformers/) covers its builder fields and templates. examples/reranking.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("README.md")) .then_chunk(ChunkMarkdown::from_chunk_range(10..2048)) .then_in_batch(Embed::new(fastembed.clone())) .then_store_with(qdrant.clone()) .run() .await?; ``` Source: [`examples/reranking.rs#L40-L45`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs#L40-L45) The indexing pipeline chunks `README.md` as markdown, embeds the chunks with FastEmbed and stores them in Qdrant. With one dense vector and no sparse vector, Qdrant stores the embedding as the point’s single unnamed vector. examples/reranking.rs ```rust // By default the search strategy is SimilaritySingleEmbedding // which takes the latest query, embeds it, and does a similarity search let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( openai_client.clone(), )) .then_transform_query(query_transformers::Embed::from_client(fastembed.clone())) .then_retrieve(qdrant.clone()) .then_transform_response(reranker) .then_answer(answers::Simple::from_client(openai_client.clone())); ``` Source: [`examples/reranking.rs#L49-L58`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs#L49-L58) `query::Pipeline::default()` uses `SimilaritySingleEmbedding`, which retrieves at most 10 documents, its default `top_k`. `then_transform_response(reranker)` runs after retrieval and before the answer. It scores the retrieved documents and passes the best 5 to `answers::Simple`. examples/reranking.rs ```rust let result = pipeline .query("What is swiftide? Please provide an elaborate explanation") .await?; println!("{:?}", result.answer()); ``` Source: [`examples/reranking.rs#L62-L66`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/reranking.rs#L62-L66) ### Gotchas [Section titled “Gotchas”](#gotchas) * Retrieve more documents than you keep. Here retrieval returns up to 10 and `Rerank` keeps 5. If retrieval returns `top_k` documents or fewer, `Rerank` only reorders them. * FastEmbed loads its models when you construct them, in `FastEmbed::try_default()`, `try_default_sparse()` and `Rerank::builder().build()`, and downloads them on the first run. A `Rerank` built without `model` loads the default model with `expect`, so a failed download panics at `build()` instead of returning an error. * The plain `Qdrant` retriever returns each document’s content `Debug`-quoted: wrapped in `"` with escaped newlines. In both examples, `Rerank` scores and `answers::Simple` renders that quoted text. docs-qa works around it with a `CleanQdrant` wrapper; see [Qdrant content workaround](/showcases/docs-qa/#qdrant-content-workaround). ## Combined pipeline [Section titled “Combined pipeline”](#combined-pipeline) The docs-qa showcase puts a reranker behind a hybrid search. Its strategy fetches 20 candidates per sub-search and keeps 20 after fusion: examples/docs-qa/src/query.rs ```rust query::Pipeline::from_search_strategy( HybridSearch::default() .with_top_n(CANDIDATES) .with_top_k(CANDIDATES), ) ``` Source: [`examples/docs-qa/src/query.rs#L108-L112`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L108-L112) Its reranker keeps 10 of the 20 candidates and scores each one with the page name and chunk title in front of the content: examples/docs-qa/src/query.rs ```rust let reranker = Rerank::builder() .top_k(CONTEXT_CHUNKS) .document_template(Some(RERANK_TEMPLATE.to_string())) .build()?; let answerer = answers::Simple::builder() .client(llm) .prompt_template(ANSWER_PROMPT.into()) .document_template(DOCUMENT_TEMPLATE) .build()?; ``` Source: [`examples/docs-qa/src/query.rs#L73-L82`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L73-L82) The query pipeline embeds the question twice, retrieves the fused candidates and reranks them before answering: examples/docs-qa/src/query.rs ```rust pipeline // Hybrid search needs both a dense and a sparse embedding of the question .then_transform_query(query_transformers::Embed::from_client( self.embedders.dense.clone(), )) .then_transform_query(query_transformers::SparseEmbed::from_client( self.embedders.sparse.clone(), )) .then_retrieve(self.qdrant.clone()) // A local cross-encoder reorders the fused candidates; only the best few remain .then_transform_response(self.reranker.clone()) .then_answer(self.answerer.clone()) ``` Source: [`examples/docs-qa/src/query.rs#L121-L134`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/query.rs#L121-L134) `self.qdrant` is the `CleanQdrant` wrapper around `Qdrant`, so the reranker and the answer see the text as it was indexed. The [docs-qa showcase](/showcases/docs-qa/) covers the indexing side and shows captured answers. ## Reference [Section titled “Reference”](#reference) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) and [`QdrantBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.QdrantBuilder.html) * [`Duckdb`](https://docs.rs/swiftide/0.34/swiftide/integrations/duckdb/struct.Duckdb.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Rerank`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.Rerank.html) and [`RerankBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.RerankBuilder.html) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html), [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html) and [`MetadataQACode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQACode.html) * [`transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) and [`transformers::SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`query_transformers::Embed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.Embed.html), [`query_transformers::SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.SparseEmbed.html) and [`GenerateSubquestions`](https://docs.rs/swiftide/0.34/swiftide/query/query_transformers/struct.GenerateSubquestions.html) * [`answers::Simple`](https://docs.rs/swiftide/0.34/swiftide/query/answers/struct.Simple.html) * [`query::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`Retrieve`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Retrieve.html) and [`TransformResponse`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.TransformResponse.html) ## Next [Section titled “Next”](#next) [Rich metadata and multiple vectors](/cookbook/rich-metadata-and-multiple-vectors/) # Other providers and stores > Short indexing recipes for Ollama, Groq, Dashscope and FastEmbed models and for Redis, PgVector and LanceDB storage. Each recipe is a runnable example from the swiftide repository that indexes files into one provider or store; the PgVector, LanceDB and Dashscope recipes also query what they stored. Each section points to the integration page that covers the client options in depth. ## Prerequisites [Section titled “Prerequisites”](#prerequisites) Run an example from the root of the swiftide repository, because the examples read `README.md` and other files by relative path: ```sh # in the swiftide repository cargo run -p swiftide-examples --example ``` To copy a recipe into your own crate, add the `swiftide` line from its section, plus the runtime and logging crates every example uses: ```sh # in your own crate cargo add tokio --features full cargo add tracing-subscriber ``` ## Ollama [Section titled “Ollama”](#ollama) ```sh # in your own crate cargo add swiftide@0.34 --features ollama,fastembed ``` ```sh # in the swiftide repository ollama pull llama3.1 cargo run -p swiftide-examples --example index-ollama ``` examples/index_ollama.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("README.md")) .then_chunk(ChunkMarkdown::from_chunk_range(10..2048)) .then(MetadataQAText::new(ollama_client.clone())) .then_in_batch(Embed::new(fastembed).with_batch_size(10)) .then_store_with(memory_store.clone()) .run() .await?; ``` Source: [`examples/index_ollama.rs#L39-L45`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_ollama.rs#L39-L45) FastEmbed computes the embeddings locally, so only metadata generation calls Ollama. `Ollama::default()` talks to `http://localhost:11434/v1` and sends no application programming interface (API) key. Client options and gotchas: [OpenAI-compatible providers](/integrations/openai-compatible/). ## Groq [Section titled “Groq”](#groq) ```sh # in your own crate cargo add swiftide@0.34 --features groq,fastembed ``` ```sh # in the swiftide repository cargo run -p swiftide-examples --example index-groq ``` examples/index_groq.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("README.md")) .then_chunk(ChunkMarkdown::from_chunk_range(10..2048)) .then(MetadataQAText::new(groq_client.clone())) .then_in_batch(Embed::new(fastembed).with_batch_size(10)) .then_store_with(memory_store.clone()) .run() .await?; ``` Source: [`examples/index_groq.rs#L39-L45`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_groq.rs#L39-L45) The client reads `GROQ_API_KEY`. The example sets the model `llama3-8b-8192`, which Groq has retired. Change it in the example’s setup to a current model, such as `llama-3.1-8b-instant`, before you run it. Client options and gotchas: [OpenAI-compatible providers](/integrations/openai-compatible/). ## Dashscope [Section titled “Dashscope”](#dashscope) ```sh # in your own crate cargo add swiftide@0.34 --features dashscope,lancedb cargo add temp-dir anyhow ``` ```sh # in the swiftide repository cargo run -p swiftide-examples --example dashscope ``` examples/dashscope.rs ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["md"])) .with_default_llm_client(client.clone()) .then_chunk(ChunkMarkdown::from_chunk_range(10..2048)) .then(MetadataQAText::new(client.clone())) .then_in_batch(Embed::new(client.clone()).with_batch_size(10)) .then_store_with(lancedb.clone()) .run() .await?; ``` Source: [`examples/dashscope.rs#L38-L45`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/dashscope.rs#L38-L45) `Dashscope` reads `QWEN_API_KEY`, or `DASHSCOPE_API_KEY` when that is unset. One client sets both an embed model and a prompt model, so it serves as large language model (LLM) and embedder. `with_default_llm_client` gives a client to every transformer added after it that has none of its own, such as `MetadataQAText::default()`. A transformer built with `new(client)` keeps its own client. The loader picks up every markdown file under the current directory. It uses the default filters of the `ignore` crate, so it skips hidden files and directories, such as `.github/`. It also skips what `.ignore` files, the global gitignore and `.git/info/exclude` exclude, and, inside a git repository, what `.gitignore` excludes. examples/dashscope.rs ```rust let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( client.clone(), )) .then_transform_query(query_transformers::Embed::from_client(client.clone())) .then_retrieve(lancedb.clone()) .then_transform_response(response_transformers::Summary::from_client(client.clone())) .then_answer(answers::Simple::from_client(client.clone())); let result = pipeline .query("What is swiftide? Please provide an elaborate explanation") .await?; ``` Source: [`examples/dashscope.rs#L49-L60`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/dashscope.rs#L49-L60) The query pipeline generates subquestions, embeds them, retrieves from LanceDB, summarizes the documents with the `Summary` response transformer, and answers with `answers::Simple`. Client options and gotchas: [OpenAI-compatible providers](/integrations/openai-compatible/) and [LanceDB](/integrations/lancedb/). ## FastEmbed [Section titled “FastEmbed”](#fastembed) ```sh # in your own crate cargo add swiftide@0.34 --features fastembed,qdrant ``` ```sh # in the swiftide repository docker run -p 6334:6334 qdrant/qdrant cargo run -p swiftide-examples --example fastembed ``` examples/fastembed.rs ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .then_in_batch(Embed::new(FastEmbed::builder().batch_size(10).build()?)) .then_store_with( Qdrant::try_from_url(qdrant_url)? .batch_size(50) .vector_size(384) .collection_name("swiftide-examples-fastembed".to_string()) .build()?, ) .run() .await?; ``` Source: [`examples/fastembed.rs#L32-L42`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/fastembed.rs#L32-L42) The recipe needs Qdrant on `QDRANT_URL`, which defaults to the gRPC port at `http://localhost:6334`. It loads, embeds and stores, with no chunker and no metadata. FastEmbed runs embedding models in your process, with no API key. It downloads the default model from Hugging Face on first use. That model produces 384-dimensional vectors, so the Qdrant collection uses `vector_size(384)`. `batch_size(10)` on the FastEmbed builder sets how many texts the model embeds at once (default 256). `batch_size(50)` on the Qdrant builder sets the nodes per write. [Embedding](/indexing/embedding/) explains how these relate to the pipeline’s batch size. Client options and gotchas: [FastEmbed](/integrations/fastembed/) and [Qdrant](/integrations/qdrant/). ## Redis [Section titled “Redis”](#redis) ```sh # in your own crate cargo add swiftide@0.34 --features redis,tree-sitter cargo add serde_json ``` ```sh # in the swiftide repository cargo run -p swiftide-examples --example index-redis ``` examples/index_into_redis.rs ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["rs"])) .then_chunk(ChunkCode::try_for_language_and_chunk_size( "rust", 10..2048, )?) .then_store_with( // By default the value is the full node serialized to JSON. // We can customize this by providing a custom function. Redis::try_build_from_url(&redis_url)? .persist_value_fn(|node| Ok(serde_json::to_string(&node.metadata)?)) .batch_size(50) .build()?, ) .run() .await?; ``` Source: [`examples/index_into_redis.rs#L39-L53`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_into_redis.rs#L39-L53) The recipe needs Redis on `REDIS_URL`, which defaults to `redis://localhost:6379`. It chunks Rust files with `ChunkCode` and stores the chunks in Redis, with no LLM or embedder. Redis is storage only here. It does not implement retrieval, but it also works as a node cache and as a message history backend. Keys, values and the other backends: [Storage](/indexing/storage/). ## PgVector [Section titled “PgVector”](#pgvector) ```sh # in your own crate cargo add swiftide@0.34 --features pgvector,ollama,fastembed ``` ```sh # in the swiftide repository ollama pull llama3.2 cargo run -p swiftide-examples --example index-md-pgvector ``` The example starts a `pgvector/pgvector` container through Docker with a test helper, so Docker must be running. In your application, pass your own database URL to `db_url`. examples/index_md_into_pgvector.rs ```rust tracing::info!("Starting indexing pipeline"); indexing::Pipeline::from_loader(FileLoader::new(test_dataset_path).with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(10..2048)) .then(MetadataQAText::new(llm_client.clone())) .then_in_batch(Embed::new(fastembed.clone()).with_batch_size(100)) .then_store_with(pgv_storage.clone()) .run() .await?; ``` Source: [`examples/index_md_into_pgvector.rs#L103-L111`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_md_into_pgvector.rs#L103-L111) examples/index_md_into_pgvector.rs ```rust async fn ask_query( llm_client: impl SimplePrompt + Clone + 'static, embed: FastEmbed, vector_store: PgVector, questions: Vec, ) -> Result, Box> { // By default the search strategy is SimilaritySingleEmbedding // which takes the latest query, embeds it, and does a similarity search // // Pgvector will return an error if multiple embeddings are set // // The pipeline generates subquestions to increase semantic coverage, embeds these in a single // embedding, retrieves the default top_k documents, summarizes them and uses that as context // for the final answer. let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( llm_client.clone(), )) .then_transform_query(query_transformers::Embed::from_client(embed)) .then_retrieve(vector_store.clone()) .then_transform_response(response_transformers::Summary::from_client( llm_client.clone(), )) .then_answer(answers::Simple::from_client(llm_client.clone())); let results: Vec = pipeline .query_all(questions) .await? .iter() .map(|result| result.answer().to_string()) .collect(); Ok(results) } ``` Source: [`examples/index_md_into_pgvector.rs#L17-L50`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_md_into_pgvector.rs#L17-L50) `query_all` sends several questions through one pipeline and returns an answered query for each. The `SimilaritySingleEmbedding` search strategy needs exactly one vector field; with more, it fails with “Search strategy for multiple vector fields in the schema is not yet implemented”. A `CustomStrategy` does not have this limit. Table setup, columns and indexes: [PgVector](/integrations/pgvector/). ## LanceDB [Section titled “LanceDB”](#lancedb) ```sh # in your own crate cargo add swiftide@0.34 --features lancedb,openai cargo add temp-dir ``` ```sh # in the swiftide repository cargo run -p swiftide-examples --example lancedb ``` examples/lancedb.rs ```rust let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-4o-mini") .build()?; let tempdir = TempDir::new().unwrap(); // Configure lancedb with a default vector size, a single embedding // and in addition to embedding the text metadata, also store it in a field let lancedb = LanceDB::builder() .uri(tempdir.child("lancedb").to_str().unwrap()) .vector_size(1536) .with_vector(EmbeddedField::Combined) .with_metadata(METADATA_QA_TEXT_NAME) .table_name("swiftide_test") .build() .unwrap(); ``` Source: [`examples/lancedb.rs#L20-L36`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/lancedb.rs#L20-L36) examples/lancedb.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("README.md")) .then_chunk(ChunkMarkdown::from_chunk_range(10..2048)) .then(MetadataQAText::new(openai_client.clone())) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .then_store_with(lancedb.clone()) .run() .await?; ``` Source: [`examples/lancedb.rs#L40-L46`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/lancedb.rs#L40-L46) examples/lancedb.rs ```rust // By default the search strategy is SimilaritySingleEmbedding // which takes the latest query, embeds it, and does a similarity search // // LanceDB will return an error if multiple embeddings are set // // The pipeline generates subquestions to increase semantic coverage, embeds these in a single // embedding, retrieves the default top_k documents, summarizes them and uses that as context // for the final answer. let pipeline = query::Pipeline::default() .then_transform_query(query_transformers::GenerateSubquestions::from_client( openai_client.clone(), )) .then_transform_query(query_transformers::Embed::from_client( openai_client.clone(), )) .then_retrieve(lancedb.clone()) .then_transform_response(response_transformers::Summary::from_client( openai_client.clone(), )) .then_answer(answers::Simple::from_client(openai_client.clone())); ``` Source: [`examples/lancedb.rs#L50-L69`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/lancedb.rs#L50-L69) LanceDB is embedded: `uri` points at a local directory, and the table is created on the first run. The example uses a temporary directory, so the data is gone when it exits. The recipe needs `OPENAI_API_KEY`. With `SimilaritySingleEmbedding`, retrieval also needs exactly one vector field, and fails with “Zero or multiple vector fields configured in schema” otherwise. A `CustomStrategy` does not have this limit. Remote storage and vector indexes: [LanceDB](/integrations/lancedb/). ## Reference [Section titled “Reference”](#reference) * [`Ollama`](https://docs.rs/swiftide/0.34/swiftide/integrations/ollama/type.Ollama.html), [`Groq`](https://docs.rs/swiftide/0.34/swiftide/integrations/groq/type.Groq.html), [`Dashscope`](https://docs.rs/swiftide/0.34/swiftide/integrations/dashscope/type.Dashscope.html), [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Redis`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.Redis.html), [`RedisBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/redis/struct.RedisBuilder.html) * [`PgVector`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVector.html), [`PgVectorBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/pgvector/struct.PgVectorBuilder.html) * [`LanceDB`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDB.html), [`LanceDBBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/lancedb/struct.LanceDBBuilder.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html), [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html), [`ChunkCode`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkCode.html) * [indexing `Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html), [query `Pipeline`](https://docs.rs/swiftide/0.34/swiftide/query/struct.Pipeline.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html), [`CustomStrategy`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.CustomStrategy.html), [`Summary`](https://docs.rs/swiftide/0.34/swiftide/query/response_transformers/struct.Summary.html) * [OpenAI-compatible providers](/integrations/openai-compatible/), [FastEmbed](/integrations/fastembed/), [Qdrant](/integrations/qdrant/), [PgVector](/integrations/pgvector/), [LanceDB](/integrations/lancedb/), [Storage](/indexing/storage/) ## Next [Section titled “Next”](#next) [Feature flags](/reference/feature-flags/) # OpenAI Responses API and reasoning > Switch the OpenAI client to the Responses API for prompts, chat, streaming and tool calls, and print reasoning summaries from an agent. This recipe gives you an OpenAI client that sends every kind of request through the OpenAI Responses application programming interface (API): prompts, structured prompts, chat completions, streams and tool calls. A second recipe gives you an agent that prints the reasoning summary of a reasoning model. Both recipes are runnable examples in the swiftide repository: `responses_api.rs` and `responses_api_reasoning.rs`. ## Setup [Section titled “Setup”](#setup) Add Swiftide with the `openai` and `swiftide-agents` feature flags. The first example also streams, defines a tool argument type and parses tool arguments, so it needs `futures-util`, `schemars`, `serde` and `serde_json`. ```sh cargo add swiftide@0.34 --features openai,swiftide-agents cargo add futures-util schemars@1 serde_json cargo add serde --features derive cargo add tokio --features full cargo add anyhow ``` The examples are on Rust edition 2024. The `agent` snippet below uses a let-chain, which does not compile on edition 2021. Set `OPENAI_API_KEY` and run the examples from the root of the swiftide repository: ```sh export OPENAI_API_KEY=sk-... cargo run -p swiftide-examples --example responses-api cargo run -p swiftide-examples --example responses-api-reasoning ``` ## Imports and types [Section titled “Imports and types”](#imports-and-types) The anchored excerpts from `responses_api.rs` leave out its imports and its two argument types. Here they are: ```rust use anyhow::{Context, Result}; use futures_util::StreamExt as _; use schemars::JsonSchema; use serde::{Deserialize, Serialize}; use swiftide::{ chat_completion::{ChatCompletionRequest, ChatMessage, ToolOutput, errors::ToolError}, integrations::openai::{OpenAI, Options}, traits::{AgentContext, ChatCompletion, SimplePrompt, StructuredPrompt}, }; /// The type `structured_prompt` deserializes the model's answer into. #[derive(Debug, Serialize, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct WeatherSummary { description: String, } /// The single parameter of the echo tool. #[derive(Debug, Serialize, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct EchoArgs { message: String, } ``` Source: [`snippets/src/cookbook/responses_api.rs#L6-L28`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/cookbook/responses_api.rs#L6-L28) `ChatCompletion` brings `complete` and `complete_stream` into scope. `StreamExt` gives you `next()` on the stream. `anyhow::Context` gives you `.context(...)` in the tool call code. ## Client [Section titled “Client”](#client) examples/responses_api.rs ```rust let openai = OpenAI::builder() .default_prompt_model("gpt-4.1-mini") .default_options(Options::builder().temperature(0.2)) .use_responses_api(true) .build()?; ``` Source: [`examples/responses_api.rs#L49-L53`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api.rs#L49-L53) `use_responses_api(true)` is the only change from a Chat Completions client. The flag defaults to false. With it set, `complete`, `complete_stream`, `prompt` and `structured_prompt` all go through the Responses API. Embeddings still use the embeddings endpoint. `default_options` takes anything that converts into `Options`, so you can pass an `OptionsBuilder` without calling `build()`. On the Responses path, the client drops some options instead of sending them. It logs a warning for `seed` and `presence_penalty`. It drops `user`, also set by `for_end_user`, without a warning. It skips `metadata` that is not a flat object, with a warning. ## Prompts [Section titled “Prompts”](#prompts) examples/responses_api.rs ```rust let greeting = openai .prompt("Say hello in one short sentence".into()) .await?; println!("Prompt result: {greeting}"); let structured: WeatherSummary = openai .structured_prompt("Summarise today's weather in Amsterdam as JSON".into()) .await?; println!("Structured result: {structured:?}"); ``` Source: [`examples/responses_api.rs#L57-L65`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api.rs#L57-L65) `prompt` returns the model’s text. `structured_prompt` returns a `WeatherSummary`, built from the JavaScript Object Notation (JSON) Schema that schemars derives for the type. The struct carries `#[serde(deny_unknown_fields)]` because OpenAI’s strict mode requires `additionalProperties: false` on every object. The [schema rules](/models/structured-output/#schema-rules) list the other constraints. `SimplePrompt`, `StructuredPrompt` and `ChatCompletion` from `swiftide::traits` must be in scope, and `futures_util::StreamExt` for the stream. ## Chat completion [Section titled “Chat completion”](#chat-completion) examples/responses_api.rs ```rust let chat_request = ChatCompletionRequest::builder() .messages(vec![ ChatMessage::new_system("You are a concise assistant."), ChatMessage::new_user("Share one fun fact about Amsterdam."), ]) .build()?; let completion = openai.complete(&chat_request).await?; println!( "Complete result: {}", completion.message().unwrap_or("") ); ``` Source: [`examples/responses_api.rs#L69-L80`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api.rs#L69-L80) A `ChatCompletionRequest` holds the messages and, optionally, tool specs. `build()` returns a `Result`. `complete` returns a `ChatCompletionResponse`, and `message()` gives you the assistant text if there is any. ## Streaming [Section titled “Streaming”](#streaming) examples/responses_api.rs ```rust let mut stream = openai.complete_stream(&chat_request).await; print!("Streaming result: "); let mut streamed_message = String::new(); while let Some(chunk) = stream.next().await { let chunk = chunk?; if let Some(delta) = chunk .delta .as_ref() .and_then(|delta| delta.message_chunk.as_deref()) { print!("{delta}"); std::io::stdout().flush().ok(); } if let Some(message) = chunk.message() { streamed_message = message.to_string(); } } println!(); if streamed_message.is_empty() { println!("Full streamed result: "); } else { println!("Full streamed result: {streamed_message}"); } ``` Source: [`examples/responses_api.rs#L84-L107`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api.rs#L84-L107) `complete_stream` returns a stream of `ChatCompletionResponse` values. The new text of each chunk is in `delta.message_chunk`. The client’s `stream_full` setting defaults to true, so every chunk also carries the message accumulated so far in `message()`. The example prints the deltas and keeps the last accumulated message. With `stream_full(false)` on the builder, every chunk but the last carries only the delta. The last one still holds the complete response, including usage, so the same logic works. ## Tool calls [Section titled “Tool calls”](#tool-calls) examples/responses_api.rs ```rust /// Minimal echo tool used to demonstrate tool calling with the Responses API. /// The macro implements the `Tool` trait, derives the JSON schema, and generates /// a helper constructor (`echo_tool()`) that returns a boxed tool ready for use. #[swiftide::tool( description = "Echos the provided message back to the caller.", param(name = "payload", description = "Text to echo back") )] async fn echo_tool( _context: &dyn AgentContext, payload: EchoArgs, ) -> Result { Ok(ToolOutput::text(format!("Echo: {}", payload.message))) } ``` Source: [`examples/responses_api.rs#L27-L39`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api.rs#L27-L39) `#[swiftide::tool]` turns the function into a tool. It generates a constructor named after the function, `echo_tool()`, which returns a boxed tool. It also generates an argument struct named after the tool in PascalCase with an `Args` suffix, here `EchoToolArgs`, with one field per parameter. The tool name defaults to the function name, and `name = "..."` in the attribute changes both. The only parameter, `payload`, is the `EchoArgs` struct from the imports excerpt. examples/responses_api.rs ```rust let tool_request = ChatCompletionRequest::builder() .messages(vec![ ChatMessage::new_system( "You are a precise assistant. Use available tools before replying directly.", ), ChatMessage::new_user( "Call the echo tool with the phrase \"Hello Responses API\" and then summarise the result.", ), ]) .tool(echo_tool()) .build()?; let tool_completion = openai.complete(&tool_request).await?; if let Some(tool_call) = tool_completion .tool_calls() .and_then(|calls| calls.first()) .cloned() { println!( "Assistant requested tool `{}` with arguments {}", tool_call.name(), tool_call.args().unwrap_or("") ); let args_json = tool_call .args() .context("echo tool call missing arguments")?; let args: EchoToolArgs = serde_json::from_str(args_json)?; let tool_output = format!("Echo: {}", args.payload.message); let mut follow_up_messages = tool_request.messages().to_vec(); follow_up_messages.push(ChatMessage::new_assistant( None::, Some(vec![tool_call.clone()]), )); follow_up_messages.push(ChatMessage::new_tool_output( tool_call.clone(), ToolOutput::text(tool_output), )); let follow_up_request = ChatCompletionRequest::builder() .messages(follow_up_messages) .tool(echo_tool()) .build()?; let final_completion = openai.complete(&follow_up_request).await?; println!( "Final response after tool call: {}", final_completion.message().unwrap_or("") ); } else { println!( "Assistant responded without tool calls: {}", tool_completion.message().unwrap_or("") ); } ``` Source: [`examples/responses_api.rs#L111-L167`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api.rs#L111-L167) `ChatCompletionRequest::builder().tool(...)` adds only the tool’s spec to the request. Nothing runs the tool for you. The example reads the first tool call, parses its arguments into `EchoToolArgs` and builds the output itself. It then appends the assistant’s tool call with `ChatMessage::new_assistant` and the result with `ChatMessage::new_tool_output`, and sends a follow-up request. An [agent](/agents/overview/) runs this loop for you, as the next recipe shows. ## Reasoning summaries [Section titled “Reasoning summaries”](#reasoning-summaries) examples/responses_api_reasoning.rs ```rust // Reasoning models require the Responses API. Enabling reasoning effort also asks for a // summary and encrypted reasoning content (enabled by default). If your OpenAI org is not // verified for reasoning access, summaries may be absent. Disable with // `reasoning_features(false)` if desired. let openai = OpenAI::builder() .default_prompt_model("o3-mini") .default_options(Options::builder().reasoning_effort(ReasoningEffort::Low)) .use_responses_api(true) .build()?; ``` Source: [`examples/responses_api_reasoning.rs#L34-L42`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api_reasoning.rs#L34-L42) Setting `reasoning_effort` on the options turns on reasoning for the request. With the Responses API, the client then also asks for an automatic reasoning summary and for the encrypted reasoning content. `reasoning_features(false)` on the options turns both off. If your OpenAI organization is not verified for reasoning access, the response may hold no summary. examples/responses_api_reasoning.rs ```rust fn reasoning_summary(reasoning: Option<&[ReasoningItem]>) -> Option { let summary = reasoning .unwrap_or(&[]) .iter() .flat_map(|item| item.summary.iter()) .cloned() .collect::>() .join("\n"); if summary.is_empty() { None } else { Some(summary) } } ``` Source: [`examples/responses_api_reasoning.rs#L10-L24`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api_reasoning.rs#L10-L24) The helper joins the summaries of all `ReasoningItem` values into one string. examples/responses_api_reasoning.rs ```rust let mut agent = Agent::builder() .llm(&openai) .on_new_message(|_, message| { if let ChatMessage::Assistant(content, _) = message && let Some(content) = content.as_deref() { println!("Assistant: {content}"); } Box::pin(async move { Ok(()) }) }) .after_completion(|_, response| { if let Some(summary) = reasoning_summary(response.reasoning.as_deref()) { println!("Reasoning summary:\n{summary}"); } let has_encrypted = response .reasoning .as_ref() .is_some_and(|items| items.iter().any(|item| item.encrypted_content.is_some())); println!("Encrypted reasoning content present: {has_encrypted}"); Box::pin(async move { Ok(()) }) }) .build()?; ``` Source: [`examples/responses_api_reasoning.rs#L46-L68`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api_reasoning.rs#L46-L68) Two hooks print what happens. `after_completion` receives the `ChatCompletionResponse` before the agent takes the reasoning out of it, so `response.reasoning` is still there. `on_new_message` fires for the messages the agent adds itself: reasoning items, assistant messages and tool outputs. The system prompt and your query go into the context directly and skip it. The example prints only assistant messages. examples/responses_api_reasoning.rs ```rust agent .query("Explain why the sky is blue in one short paragraph.") .await?; ``` Source: [`examples/responses_api_reasoning.rs#L72-L74`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/responses_api_reasoning.rs#L72-L74) [Reasoning items](/agents/providers-and-reasoning/#reasoning-items) explains how the agent stores the items and replays them on the next request. ### Variations [Section titled “Variations”](#variations) * `ReasoningEffort` has the variants `None`, `Minimal`, `Low`, `Medium`, `High` and `Xhigh`. Which ones a model accepts depends on the model. * Without an agent, call `complete` on a client with `use_responses_api(true)` and `reasoning_effort` set, and read `response.reasoning` yourself. On the Chat Completions path, `response.reasoning` stays `None`. * The `on_new_message` hook uses a let-chain, which needs `edition = "2024"`. On edition 2021, nest the two `if let` statements. * [Responses API and reasoning](/models/overview/#responses-api-and-reasoning) covers how other providers return reasoning. ## Reference [Section titled “Reference”](#reference) * [`GenericOpenAIBuilder::use_responses_api`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.use_responses_api) * [`GenericOpenAIBuilder::stream_full`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html#method.stream_full) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`Options`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.Options.html) * [`OptionsBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OptionsBuilder.html) * [`OptionsBuilder::reasoning_features`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.OptionsBuilder.html#method.reasoning_features) * [`ReasoningEffort` (OpenAI)](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/enum.ReasoningEffort.html) * [`ChatCompletionRequest`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionRequest.html) * [`ChatCompletionResponse`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponse.html) * [`ChatCompletionResponseDelta`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ChatCompletionResponseDelta.html) * [`ReasoningItem`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ReasoningItem.html) * [`ToolOutput`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ToolOutput.html) * [`ToolCall`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/struct.ToolCall.html) * [`ChatMessage`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/enum.ChatMessage.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`ChatCompletion`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.ChatCompletion.html) * [`#[swiftide::tool]`](https://docs.rs/swiftide/0.34/swiftide/attr.tool.html) * [`Agent`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.Agent.html) * [`AgentBuilder::after_completion`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.after_completion) * [`AgentBuilder::on_new_message`](https://docs.rs/swiftide/0.34/swiftide/agents/struct.AgentBuilder.html#method.on_new_message) ## Next [Section titled “Next”](#next) [Agents on AWS Bedrock](/cookbook/bedrock-agents/) # Rich metadata and multiple vectors > Add questions, a summary, a title and keywords to every chunk with LLM transformers, and store the chunk and each metadata field as separate named vectors in Qdrant. Both recipes fill a Qdrant collection in which every chunk carries four fields that a large language model (LLM) writes: questions and answers, a summary, a title and keywords. One recipe embeds each chunk together with its metadata as a single vector. The other stores the chunk and each metadata field as its own named vector. ## Setup [Section titled “Setup”](#setup) Add Swiftide with the `openai` and `qdrant` feature flags, plus the runtime and logging crates the examples use: ```sh cargo add swiftide@0.34 --features openai,qdrant cargo add tokio --features macros,rt-multi-thread cargo add tracing-subscriber --features env-filter ``` Set `OPENAI_API_KEY` and start Qdrant on `localhost:6334`, or point `QDRANT_URL` somewhere else; [Running Qdrant](/integrations/qdrant/#local-instance-and-connection) has the Docker command. Run the examples from the root of the swiftide repository, because they load `README.md` and `LICENSE` by relative path: ```sh cargo run -p swiftide-examples --example index-markdown-metadata cargo run -p swiftide-examples --example store-multiple-vectors ``` ## Metadata transformers [Section titled “Metadata transformers”](#metadata-transformers) Both recipes use one OpenAI client for prompts and embeddings: examples/index_markdown_lots_of_metadata.rs ```rust let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-4o") .build()?; ``` Source: [`examples/index_markdown_lots_of_metadata.rs#L32-L35`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_markdown_lots_of_metadata.rs#L32-L35) The client sets `gpt-4o` for the metadata prompts and `text-embedding-3-small` for embeddings. That embedding model returns 1536 dimensions, which must match `vector_size(1536)` on the storage. examples/index_markdown_lots_of_metadata.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("README.md").with_extensions(&["md"])) .with_concurrency(1) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .then(MetadataQAText::new(openai_client.clone())) .then(MetadataSummary::new(openai_client.clone())) .then(MetadataTitle::new(openai_client.clone())) .then(MetadataKeywords::new(openai_client.clone())) .then_in_batch(Embed::new(openai_client.clone())) .log_all() .filter_errors() .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples") .build()?, ) .run() .await?; ``` Source: [`examples/index_markdown_lots_of_metadata.rs#L39-L57`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/index_markdown_lots_of_metadata.rs#L39-L57) `ChunkMarkdown::from_chunk_range(20..2048)` splits the file into chunks of fewer than 2048 characters. It drops chunks that are under 20 bytes once whitespace is trimmed. Each metadata transformer then sends one prompt per chunk and writes the answer under a fixed metadata key: | Transformer | Metadata key | | ------------------ | ------------------------------ | | `MetadataQAText` | `Questions and Answers (text)` | | `MetadataSummary` | `Summary` | | `MetadataTitle` | `Title` | | `MetadataKeywords` | `Keywords` | Each module exposes its key as a `NAME` constant, for example `metadata_qa_text::NAME`. The default `MetadataQAText` prompt asks for at most five questions, so the model may return fewer. The prompts are covered in [Transformers and metadata](/indexing/transformers-and-metadata/). Each chunk costs four prompts. `with_concurrency(1)` sends them one node at a time. Raise the pipeline concurrency when your OpenAI rate limit allows it. The pipeline keeps the default embed mode. `Embed` turns each node into one text: the metadata as `key: value` lines, then the chunk. With only the `Combined` field configured, Qdrant stores it as the collection’s single unnamed vector. The metadata goes into the point payload next to `path` and `content`. ### Error handling [Section titled “Error handling”](#error-handling) `log_all` logs errors, and logs every node at `DEBUG` level. Set `RUST_LOG=debug` to see each chunk with its metadata. `filter_errors` drops errors instead of stopping the run, but an error can take healthy nodes with it. `then_in_batch(Embed)` collects nodes into batches before it embeds them. When a failed prompt reaches that step, the batch returns the error together with every node it had collected so far. `filter_errors` drops the whole error. One failed prompt can therefore drop up to 255 good nodes in the first recipe, which uses the default batch size of 256, and up to 9 in the second. A failed embedding request also fails its whole batch. Keep the batch size small when your prompts fail often. ## Named vectors [Section titled “Named vectors”](#named-vectors) The second recipe stores five vectors per chunk instead of one: examples/store_multiple_vectors.rs ```rust indexing::Pipeline::from_loader(FileLoader::new("LICENSE")) .with_concurrency(1) .with_embed_mode(EmbedMode::PerField) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .then(MetadataQAText::new(openai_client.clone())) .then(MetadataSummary::new(openai_client.clone())) .then(MetadataTitle::new(openai_client.clone())) .then(MetadataKeywords::new(openai_client.clone())) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .log_all() .filter_errors() .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-multi-vectors") .with_vector(EmbeddedField::Chunk) .with_vector(EmbeddedField::Metadata(metadata_qa_text::NAME.into())) .with_vector(EmbeddedField::Metadata(metadata_summary::NAME.into())) .with_vector( VectorConfig::builder() .embedded_field(EmbeddedField::Metadata(metadata_title::NAME.into())) .distance(Distance::Manhattan) .build()?, ) .with_vector(EmbeddedField::Metadata(metadata_keywords::NAME.into())) .build()?, ) .run() .await?; ``` Source: [`examples/store_multiple_vectors.rs#L45-L74`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/store_multiple_vectors.rs#L45-L74) `with_embed_mode(EmbedMode::PerField)` right after the loader makes `Embed` send the chunk and each metadata value as separate texts, so `with_batch_size(10)` keeps each request at 50 inputs. Each `with_vector` names one stored field. Qdrant names the vectors `Chunk`, `Metadata: Questions and Answers (text)`, `Metadata: Summary` and so on. The title vector uses `Distance::Manhattan` through `VectorConfig::builder()`. [Embedding](/indexing/embedding/#embed-mode) explains embed modes and named vectors in full. ## Gotchas [Section titled “Gotchas”](#gotchas) Qdrant keeps the vector configuration of an existing collection, as [Existing collections](/cookbook/hybrid-search-and-reranking/#existing-collections) explains. That is why the two examples use different collection names. Delete the collection or pick a new name when you change the vectors. ### Search on named vectors [Section titled “Search on named vectors”](#search-on-named-vectors) A query pipeline with the default search strategy, `SimilaritySingleEmbedding`, searches the vector named `Combined` whenever the `Qdrant` storage is built with more than one `with_vector` or with any `with_sparse_vector`. The check uses the `Qdrant` value in the query pipeline, not the collection. The `store-multiple-vectors` collection has no `Combined` vector, so that search fails. To keep similarity search working, index with `EmbedMode::Both` and add `with_vector(EmbeddedField::Combined)`. Build the query-side `Qdrant` with the same `with_vector` calls. A `Qdrant` with the default single-vector config sends an unnamed search, which fails against a collection of named vectors. `HybridSearch` lets you pick the field: `with_dense_vector_field` and `with_sparse_vector_field` both default to `EmbeddedField::Combined`. Hybrid search also needs a sparse vector for the chosen field, from `SparseEmbed` and `with_sparse_vector`. [Hybrid search and reranking](/cookbook/hybrid-search-and-reranking/) shows that setup. For other searches on one named vector, use the Qdrant client from `Qdrant::client()` directly. ## Reference [Section titled “Reference”](#reference) * [`Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`EmbedMode`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbedMode.html) * [`EmbeddedField`](https://docs.rs/swiftide/0.34/swiftide/indexing/enum.EmbeddedField.html) * [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html) * [`MetadataQAText`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataQAText.html) and [`metadata_qa_text::NAME`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/metadata_qa_text/constant.NAME.html) * [`MetadataSummary`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataSummary.html) and [`metadata_summary::NAME`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/metadata_summary/constant.NAME.html) * [`MetadataTitle`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataTitle.html) and [`metadata_title::NAME`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/metadata_title/constant.NAME.html) * [`MetadataKeywords`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.MetadataKeywords.html) and [`metadata_keywords::NAME`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/metadata_keywords/constant.NAME.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`SparseEmbed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.SparseEmbed.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`QdrantBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.QdrantBuilder.html) * [`VectorConfig`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.VectorConfig.html) * [`Distance`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/type.Distance.html) * [`SimilaritySingleEmbedding`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.SimilaritySingleEmbedding.html) * [`HybridSearch`](https://docs.rs/swiftide/0.34/swiftide/query/search_strategies/struct.HybridSearch.html) ## Next [Section titled “Next”](#next) [Scraping websites to markdown](/cookbook/scraping-to-markdown/) # Scraping websites to markdown > Crawl a website with the spider-based ScrapingLoader, convert each page to markdown with HtmlToMarkdownTransformer and split it into markdown chunks. This recipe crawls one page of a website, turns its HyperText Markup Language (HTML) into markdown and keeps the markdown chunks in memory. It is the `scraping_index_to_markdown.rs` example in the swiftide repository. It needs no application programming interface (API) key and no running services, only network access. ## Setup [Section titled “Setup”](#setup) The loader and the transformer sit behind the `scraping` feature flag. The loader takes a `Website` from the spider crate, so add spider as well. The example prints its output through `tracing`, so add a subscriber too: ```sh cargo add swiftide@0.34 --features scraping cargo add spider@2 cargo add tokio --features macros,rt-multi-thread cargo add tracing-subscriber --features env-filter ``` Run the example from the root of the swiftide repository: ```sh RUST_LOG=swiftide_indexing=debug cargo run -p swiftide-examples --example scraping-index ``` The example logs every node at the `DEBUG` level. Without `RUST_LOG` it prints errors only, so you see nothing on a successful run. In your own binary, call `tracing_subscriber::fmt::init()` at the start of an async `main` that returns `Result<(), Box>`, as the example does. Without a subscriber, `log_all` prints nothing at any level. ## Pipeline [Section titled “Pipeline”](#pipeline) examples/scraping_index_to_markdown.rs ```rust indexing::Pipeline::from_loader(ScrapingLoader::from_spider( Website::new("https://www.bosun.ai/") .with_limit(1) .to_owned(), )) .then(HtmlToMarkdownTransformer::default()) .then_chunk(ChunkMarkdown::from_chunk_range(20..2048)) .log_all() .then_store_with(MemoryStorage::default()) .run() .await?; ``` Source: [`examples/scraping_index_to_markdown.rs#L26-L36`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/scraping_index_to_markdown.rs#L26-L36) The snippet leaves out the imports. These are the ones the example uses: `use spider::website::Website;` and `use swiftide::{indexing, indexing::persist::MemoryStorage, indexing::transformers::ChunkMarkdown, integrations::scraping::{HtmlToMarkdownTransformer, ScrapingLoader}};` ### Loader [Section titled “Loader”](#loader) `Website::new` sets the start address (URL). `with_limit(1)` caps the crawl at one page. Spider’s builder methods return `&mut Website`, and `ScrapingLoader::from_spider` takes the `Website` by value, which is why the example calls `.to_owned()`. `ScrapingLoader::from_url(url)` is the short form. It builds a `Website` with spider’s defaults, so it follows links on the site with no page limit, bounded only by spider’s default depth of 25. Use `from_spider` whenever you want to bound or tune the crawl. `Pipeline::from_loader` turns the loader into a stream right away. At that point the loader subscribes to the crawl and spawns two background Tokio tasks: one that crawls and one that receives pages. The crawl starts when you build the pipeline, not when you call `run`. Each page spider fetches becomes one node. The node’s chunk is the raw HTML of the page, and its path is the page URL. ### Transformer [Section titled “Transformer”](#transformer) `HtmlToMarkdownTransformer` converts the HTML in each node to markdown with the htmd crate. The default skips `script` and `style` tags, so inline scripts and stylesheets do not end up in your chunks. A page that fails to convert becomes an error in the stream. The transformer keeps the node’s path, so every markdown node still carries the URL it came from. ### Chunker [Section titled “Chunker”](#chunker) `ChunkMarkdown::from_chunk_range(20..2048)` splits each page along markdown structure into chunks of fewer than 2048 characters. It drops chunks whose trimmed text is shorter than 20 bytes, which removes most leftover navigation fragments. `ChunkMarkdown::from_max_characters(n)` sets only the upper bound. `then_chunk` turns one node into many. ### Storage [Section titled “Storage”](#storage) `log_all` logs errors and every node that passes. `MemoryStorage` keeps the chunks in a map in memory. Clone the `MemoryStorage` before you pass it to `then_store_with`. The clone shares the same map, so `storage.get_all_values().await` returns the chunks after `run` finishes. Swap in any other storage backend to keep the result. ## Spider Cloud [Section titled “Spider Cloud”](#spider-cloud) Set `SPIDER_CLOUD_API_KEY` and the loader routes the crawl through Spider Cloud, the hosted service of the spider project, with no code change. The loader reads the variables once per process, so set them before the first crawl. See [Spider Cloud](/indexing/loading/#spider-cloud) for the modes and the endpoint override. The loader still crawls some hosts directly, because Spider Cloud cannot reach them. These are `localhost`, names ending in `.localhost` or `.local`, Internet Protocol version 4 (IPv4) loopback, private, link-local and unspecified addresses, and IPv6 loopback and unspecified addresses. Private and link-local IPv6 addresses still go through Spider Cloud, so crawl those without the key set. ## Gotchas [Section titled “Gotchas”](#gotchas) ### Crawl limits [Section titled “Crawl limits”](#crawl-limits) All crawl settings live on spider’s `Website`. Besides `with_limit`, spider offers a depth limit, a delay between requests, robots.txt handling, and allow and block lists of URLs. See the spider documentation for the full list. A limit of 0 means no limit. ### Failed pages [Section titled “Failed pages”](#failed-pages) `run` stops at the first error in the stream. On a larger crawl one malformed page would end the whole run. Add `.filter_errors()` after the transformer to drop failed nodes and keep going. Put `log_all` before it if you want those errors logged. ### Early end of a large crawl [Section titled “Early end of a large crawl”](#early-end-of-a-large-crawl) The receive task reads pages from a spider broadcast channel and forwards them into the pipeline. If the pipeline falls behind while spider keeps fetching, the receive task lags behind the broadcast, stops reading and ends the stream. `run` then returns `Ok` with only part of the site. On a large crawl with slow steps, keep the steps after the loader fast, or check the number of stored chunks against what you expect. ### Tokio runtime [Section titled “Tokio runtime”](#tokio-runtime) The loader calls `tokio::spawn` inside `Pipeline::from_loader`. Building the pipeline outside a Tokio runtime panics, even if you never call `run`. `#[tokio::main]` is enough. ## Embedding the chunks [Section titled “Embedding the chunks”](#embedding-the-chunks) The chunks carry no embeddings. To search them, add an embedder with `then_in_batch(Embed::new(...))` and store the nodes in a vector database such as Qdrant. [Rich metadata and multiple vectors](/cookbook/rich-metadata-and-multiple-vectors/) shows that shape with markdown files. ## Reference [Section titled “Reference”](#reference) * [`ScrapingLoader`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.ScrapingLoader.html) * [`HtmlToMarkdownTransformer`](https://docs.rs/swiftide/0.34/swiftide/integrations/scraping/struct.HtmlToMarkdownTransformer.html) * [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html) * [`MemoryStorage`](https://docs.rs/swiftide/0.34/swiftide/indexing/persist/struct.MemoryStorage.html) * [`indexing::Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html) * [Feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [Streaming loaders](/cookbook/streaming-loaders/) # Streaming loaders > Index messages from a Kafka or Fluvio topic as they arrive, and store the embedded nodes in a second Kafka topic or in Qdrant. These two recipes give you an indexing pipeline that reads a message topic and keeps running as new messages arrive. The first reads a Kafka topic, embeds each message and writes the result to a second Kafka topic. The second reads a Fluvio topic and stores the embeddings in Qdrant. [Loading](/indexing/loading/#kafka-and-fluvio) covers the loaders themselves; this page runs them end to end. ## Kafka [Section titled “Kafka”](#kafka) Add Swiftide with the `kafka` and `fastembed` feature flags, plus Tokio for the async runtime: ```sh cargo add swiftide@0.34 --features kafka,fastembed cargo add tokio --features macros,rt-multi-thread ``` The `kafka` feature flag builds librdkafka from source through rdkafka’s `cmake-build` feature, so the build machine needs CMake and a C compiler. ### Kafka loader and storage [Section titled “Kafka loader and storage”](#kafka-loader-and-storage) examples/kafka.rs ```rust static LOADER_TOPIC: &str = "loader"; static STORAGE_TOPIC: &str = "storage"; let mut client_config = ClientConfig::new(); client_config.set("bootstrap.servers", "localhost:9092"); client_config.set("group.id", "group_id"); client_config.set("auto.offset.reset", "earliest"); let loader = Kafka::builder() .client_config(client_config.clone()) .topic(LOADER_TOPIC) .build() .unwrap(); let storage = Kafka::builder() .client_config(client_config) .topic(STORAGE_TOPIC) .create_topic_if_not_exists(true) .batch_size(2usize) .build() .unwrap(); ``` Source: [`examples/kafka.rs#L27-L47`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/kafka.rs#L27-L47) `Kafka` is both a loader and a storage backend, so the example builds two of them from one rdkafka `ClientConfig`. `auto.offset.reset` set to `earliest` makes a new consumer group start at the oldest message in the topic. The loader reads the topic `loader`. The storage writes to the topic `storage`. Before the first node, the pipeline calls `setup` on every storage. The Kafka `setup` returns an error only when its metadata check reports the topic missing. A metadata request for a named topic almost always returns an entry, so `setup` passes and `create_topic_if_not_exists(true)` rarely does anything. `batch_size` sets how many nodes the storage sends per batch. The default is 32; the example uses 2. ### Kafka pipeline [Section titled “Kafka pipeline”](#kafka-pipeline) examples/kafka.rs ```rust indexing::Pipeline::from_loader(loader) .then_in_batch(Embed::new(FastEmbed::try_default().unwrap()).with_batch_size(10)) .then_store_with(storage) .run() .await?; ``` Source: [`examples/kafka.rs#L51-L55`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/kafka.rs#L51-L55) Each Kafka message with a UTF-8 payload becomes one node, with the payload as its chunk. The loader skips other messages and logs them only at debug level. `Embed` embeds 10 nodes per call with FastEmbed, a local embedder. The Kafka storage produces one message per node on the storage topic. The pipeline runs until you stop the process. ### Storage keys and payloads [Section titled “Storage keys and payloads”](#storage-keys-and-payloads) As storage, `Kafka` writes each node with the key `:` and the node serialized as JavaScript Object Notation (JSON) as the payload. A node from the Kafka loader has an empty path, and its id is derived from the path and the chunk. Two messages with the same payload therefore get the same key. To change either, set `persist_key_fn` or `persist_payload_fn` on the builder. Both take a plain function pointer, `fn(&TextNode) -> Result`, so a closure that captures variables does not fit. ### Kafka run [Section titled “Kafka run”](#kafka-run) Start a broker on `localhost:9092`: ```sh docker run -d --name kafka -p 9092:9092 apache/kafka:3.9.0 ``` To run the unchanged example instead, run it from the root of a swiftide checkout: ```sh cargo run -p swiftide-examples --example kafka ``` In a second terminal, produce messages on `loader`, one per line: ```sh docker exec -it kafka /opt/kafka/bin/kafka-console-producer.sh --bootstrap-server localhost:9092 --topic loader ``` Read the embedded nodes back from `storage`: ```sh docker exec -it kafka /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic storage --from-beginning ``` Nothing appears until you have produced 10 messages, the embed batch size. ## Fluvio [Section titled “Fluvio”](#fluvio) Add Swiftide with the `fluvio`, `fastembed` and `qdrant` feature flags. The example also uses `fluvio::Offset`, which Swiftide does not re-export, so add the fluvio crate at the version Swiftide depends on: ```sh cargo add swiftide@0.34 --features fluvio,fastembed,qdrant cargo add fluvio@0.50.1 --no-default-features cargo add tokio --features macros,rt-multi-thread ``` A different fluvio version gives you a different `Offset` type, and the consumer config does not accept it. ### Fluvio loader [Section titled “Fluvio loader”](#fluvio-loader) examples/fluvio.rs ```rust static TOPIC_NAME: &str = "hello-rust"; static PARTITION_NUM: u32 = 0; let loader = Fluvio::builder() .consumer_config_ext( ConsumerConfigExt::builder() .topic(TOPIC_NAME) .partition(PARTITION_NUM) .offset_start(fluvio::Offset::from_end(1)) .build() .unwrap(), ) .build() .unwrap(); ``` Source: [`examples/fluvio.rs#L36-L49`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/fluvio.rs#L36-L49) The loader reads partition 0 of the topic `hello-rust`. `Offset::from_end(1)` starts one record before the end of the partition. `Fluvio` is a loader only; it has no storage side. Without `fluvio_config`, the loader connects with `Fluvio::connect()`, which uses the current profile in `~/.fluvio/config`. Pass a `FluvioConfig` to `fluvio_config` to connect elsewhere. `Fluvio::from_consumer_config` is a shorter constructor when the current profile is enough. ### Fluvio pipeline [Section titled “Fluvio pipeline”](#fluvio-pipeline) examples/fluvio.rs ```rust indexing::Pipeline::from_loader(loader) .then_in_batch(Embed::new(FastEmbed::try_default().unwrap()).with_batch_size(10)) .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(384) .collection_name("swiftide-examples") .build()?, ) .run() .await?; ``` Source: [`examples/fluvio.rs#L53-L63`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/fluvio.rs#L53-L63) Each record becomes a node whose chunk is the record value. The embeddings go to the Qdrant collection `swiftide-examples`. Its vector size of 384 matches the default FastEmbed model. Qdrant is reached over gRPC at `http://localhost:6334` unless `QDRANT_URL` says otherwise. ### Fluvio run [Section titled “Fluvio run”](#fluvio-run) Start Qdrant and a local Fluvio cluster, and create the topic: ```sh docker run -d -p 6333:6333 -p 6334:6334 qdrant/qdrant fluvio cluster start fluvio topic create hello-rust ``` Run the example from a swiftide checkout, then produce records, one per line, in a second terminal: ```sh cargo run -p swiftide-examples --example fluvio fluvio produce hello-rust ``` ## Gotchas [Section titled “Gotchas”](#gotchas) * **Errors stop the run.** `run` returns at the first error in the stream, such as a failed `recv` or a failed embedding call. To keep a long-running stream alive, put `.log_errors().filter_errors()` directly after `from_loader`. Placed after `then_in_batch` or `then_store_with`, an error drops the nodes already buffered for that batch along with it. * **Partial batches wait.** Batch transformers and batch storage wait for a full batch. On a quiet topic, the last few nodes wait for new messages. * **Order is not kept.** The pipeline stores batches concurrently, so nodes can land in the storage topic in a different order than they were read. * **Metadata is embedded.** The default embed mode embeds the metadata together with the chunk, so the loader’s metadata fields become part of the embedded text. [Rich metadata and multiple vectors](/cookbook/rich-metadata-and-multiple-vectors/) shows the other embed modes. * **Loaders panic when you build the pipeline.** `Pipeline::from_loader` starts the loader, so the Kafka loader panics there when it cannot create the consumer or subscribe, and the Fluvio loader panics there when it cannot connect. For Kafka that means an invalid configuration; an unreachable broker does not panic, and the loader keeps waiting. * **Fluvio needs a multi-thread runtime.** The Fluvio loader connects inside `tokio::task::block_in_place`, which panics on a current-thread Tokio runtime. `#[tokio::main]` uses a multi-thread runtime by default. * **FastEmbed downloads its model.** The first `FastEmbed::try_default()` downloads the embedding model from Hugging Face. ## Reference [Section titled “Reference”](#reference) * [`Kafka`](https://docs.rs/swiftide/0.34/swiftide/integrations/kafka/struct.Kafka.html) * [`KafkaBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/kafka/struct.KafkaBuilder.html) * [`ClientConfig`](https://docs.rs/rdkafka/0.39.0/rdkafka/config/struct.ClientConfig.html) (rdkafka) * [`Fluvio`](https://docs.rs/swiftide/0.34/swiftide/integrations/fluvio/struct.Fluvio.html) * [`FluvioBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/fluvio/struct.FluvioBuilder.html) * [`ConsumerConfigExt`](https://docs.rs/fluvio/0.50.1/fluvio/consumer/struct.ConsumerConfigExt.html) (fluvio) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`FastEmbed`](https://docs.rs/swiftide/0.34/swiftide/integrations/fastembed/struct.FastEmbed.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`Loader`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Loader.html) * [`Persist`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.Persist.html) ## Next [Section titled “Next”](#next) [Other providers and stores](/cookbook/providers-and-stores/) # Structured prompts > Get a typed Rust struct back from one model call with structured_prompt, or a JSON value through DynStructuredPrompt when you hold a trait object. This recipe asks an OpenAI model for three questions about Rust and reads them from a `Vec` inside your own struct. A second call gets the same answer through a trait object. The code comes from the `structured-prompt` example in the swiftide repository. ## Setup [Section titled “Setup”](#setup) Add Swiftide with the `openai` feature flag. Add schemars, serde and serde_json for the response type, and tokio and anyhow for `main`: ```sh cargo add swiftide@0.34 --features openai cargo add schemars@1 serde serde_json --features serde/derive cargo add anyhow cargo add tokio --features macros,rt-multi-thread ``` Set `OPENAI_API_KEY` and run the example from the root of the swiftide repository: ```sh cargo run -p swiftide-examples --example structured-prompt ``` ## Imports and client [Section titled “Imports and client”](#imports-and-client) The example needs no extra setup inside the swiftide repository. Your own binary starts like this: ```rust use anyhow::Result; use schemars::JsonSchema; use serde::Deserialize; use swiftide::{integrations, traits::StructuredPrompt as _}; #[tokio::main] async fn main() -> Result<()> { let client = integrations::openai::OpenAI::builder() .default_prompt_model("gpt-5-mini") .build()?; ``` Source: [`snippets/src/cookbook/structured_prompts.rs#L5-L14`](https://github.com/bosun-ai/swiftide-website/blob/master/snippets/src/cookbook/structured_prompts.rs#L5-L14) `structured_prompt::` is a method of the `StructuredPrompt` trait. The import `StructuredPrompt as _` brings the method into scope without the name. Leave it out and the call fails to compile with `no method named structured_prompt`. `OpenAI` is a type alias of `GenericOpenAI` for the OpenAI endpoint. A structured prompt uses the default prompt model, so set one with `default_prompt_model`. Without it the call fails with `LanguageModelError::PermanentError("Model not set")`. ## Response type [Section titled “Response type”](#response-type) examples/structured_prompt.rs ```rust // Note that deny unknown fields is required. If you get an error on 'additionalProperties' to // be required, and false, this is what is missing. #[derive(Deserialize, JsonSchema, Serialize, Debug)] #[serde(deny_unknown_fields)] struct MyResponse { questions: Vec, } ``` Source: [`examples/structured_prompt.rs#L15-L21`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/structured_prompt.rs#L15-L21) Swiftide turns the struct into a JavaScript Object Notation (JSON) Schema with `schemars::schema_for!` and sends that schema with the prompt. OpenAI checks the schema in strict mode, which is why the struct carries `#[serde(deny_unknown_fields)]`. [Schema rules](/models/structured-output/#schema-rules) lists what strict mode rejects, including `Option` fields. ## Typed call [Section titled “Typed call”](#typed-call) examples/structured_prompt.rs ```rust let response = client .structured_prompt::( "List three interesting questions about the Rust programming language.".into(), ) .await?; println!("Response: {:?}", response.questions); ``` Source: [`examples/structured_prompt.rs#L25-L31`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/structured_prompt.rs#L25-L31) The method takes a `Prompt`. `.into()` turns a `&'static str` or a `String` into one. The client sends the rendered prompt as one user message, with no system message. It then deserializes the JSON the model returns into `MyResponse`. A reply that does not fit the type fails with `LanguageModelError::PermanentError`. Swiftide never retries a `structured_prompt` call, including transient and rate-limit errors. `LanguageModelWithBackOff` does not implement `StructuredPrompt`, so you call the inner client and handle retries yourself. [Errors, backoff, usage and tokens](/models/errors-backoff-and-usage/) covers the decorator. ## Templated prompts [Section titled “Templated prompts”](#templated-prompts) The docs-qa showcase grades answers with a structured prompt. It renders a template and wraps the call in its own retry helper: examples/docs-qa/src/judge.rs ```rust let contexts = answer.contexts().collect::>().join("\n---\n"); let prompt = Prompt::from(JUDGE_PROMPT) .with_context_value("question", answer.question()) .with_context_value("ground_truth", ground_truth) .with_context_value("answer", answer.text()) .with_context_value("contexts", contexts); // The JSON schema of `Judgement` is sent along; the reply is parsed into it let judgement: Judgement = retry_transient(|| self.llm.structured_prompt(prompt.clone())).await?; ``` Source: [`examples/docs-qa/src/judge.rs#L154-L163`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/judge.rs#L154-L163) `Prompt::from` takes the template, and each `with_context_value` fills one variable. [Prompts and templates](/models/prompts-and-templates/) covers the syntax. The prompt is cloned on every attempt because `structured_prompt` takes it by value. The response type nests structs, and each one carries `#[serde(deny_unknown_fields)]`: examples/docs-qa/src/judge.rs ```rust #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct Judgement { reference_claims: Vec, answer_claims: Vec, } #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct ReferenceClaim { // Never read; asking for the claim first makes the model commit to it before judging it. #[expect(dead_code)] claim: String, in_answer: bool, in_context: bool, } #[derive(Debug, Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] struct AnswerClaim { #[expect(dead_code)] claim: String, supported_by_context: bool, } ``` Source: [`examples/docs-qa/src/judge.rs#L91-L114`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/docs-qa/src/judge.rs#L91-L114) Strict mode applies to every object in the schema, not only the outer one. A nested struct without the attribute makes OpenAI reject the request. ## Trait objects [Section titled “Trait objects”](#trait-objects) examples/structured_prompt.rs ```rust // Because we use generics, structured_prompt is not dyn safe. However, there is an // alternative: let client: Box = Box::new(client); let response: serde_json::Value = client .structured_prompt_dyn( "List three interesting questions about the Rust programming language.".into(), schemars::schema_for!(MyResponse), ) .await?; let parsed: MyResponse = serde_json::from_value(response)?; println!("Response: {:?}", parsed); ``` Source: [`examples/structured_prompt.rs#L35-L49`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/structured_prompt.rs#L35-L49) `structured_prompt` is generic over `T`, so `StructuredPrompt` is not dyn safe. You cannot hold a `Box`. `DynStructuredPrompt` is the object-safe variant. Its `structured_prompt_dyn` takes the prompt and a `schemars::Schema`, and returns a `serde_json::Value`. You pass the schema and decode the value yourself. Use the trait object when the client is chosen at runtime, for example OpenAI in one deployment and Amazon Web Services (AWS) Bedrock in another. ## Variations [Section titled “Variations”](#variations) ### Other providers [Section titled “Other providers”](#other-providers) The OpenAI-compatible clients and `AwsBedrock` support structured prompts, each behind its own feature flag, and `Anthropic` does not. [Provider support](/models/structured-output/#provider-support) lists the clients and their flags. ### Responses API [Section titled “Responses API”](#responses-api) With `use_responses_api(true)` on the OpenAI builder, `structured_prompt` goes through the Responses application programming interface (API) instead of Chat Completions. The schema rules stay the same. [OpenAI Responses API and reasoning](/cookbook/responses-api-and-reasoning/) shows that client. ### Agents [Section titled “Agents”](#agents) An agent talks to its model through chat completion, not `structured_prompt`. For a typed final answer, replace the default stop tool with `StopWithArgs`, behind the `swiftide-agents` feature flag; [Agents](/models/structured-output/#agents) shows how. Call `without_default_stop_tool()` or register it with `add_tool`, because `tools()` keeps the default `stop` tool and drops yours. ## Reference [Section titled “Reference”](#reference) * [`StructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.StructuredPrompt.html) * [`DynStructuredPrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.DynStructuredPrompt.html) * [`Prompt`](https://docs.rs/swiftide/0.34/swiftide/prompt/struct.Prompt.html) * [`OpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/type.OpenAI.html) * [`GenericOpenAI`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAI.html) * [`AwsBedrock`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrock.html) * [`LanguageModelError`](https://docs.rs/swiftide/0.34/swiftide/chat_completion/errors/enum.LanguageModelError.html) * [`LanguageModelWithBackOff`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.LanguageModelWithBackOff.html) * [`StopWithArgs`](https://docs.rs/swiftide/0.34/swiftide/agents/tools/control/struct.StopWithArgs.html) ## Next [Section titled “Next”](#next) [Describing images](/cookbook/describe-images/) # Usage metrics > Run the usage_metrics and langfuse examples to print token counters per model from an indexing pipeline and send one model call to Langfuse. After this recipe you have two examples from the swiftide repository running on your machine. `usage_metrics.rs` prints the token counters of an indexing pipeline, labelled with the model and your own metadata. `langfuse.rs` sends one model call, with its input, output and token usage, to Langfuse. [Metrics and pipeline statistics](/observability/metrics-and-statistics/) and [Langfuse](/observability/langfuse/) cover the reference details; this page is the end-to-end run. ## Setup [Section titled “Setup”](#setup) Token counters sit behind the `metrics` feature flag. The recorder comes from the `metrics` crate. In your own crate, add: ```sh cargo add swiftide@0.34 --features openai,qdrant,metrics cargo add metrics@0.24 cargo add tokio --features macros,rt-multi-thread ``` Other clients that emit counters need their own feature flag as well: `anthropic`, `aws-bedrock`, `ollama` or `groq`. To run the example itself, clone the swiftide repository, set `OPENAI_API_KEY`, and start Qdrant on `localhost:6334` (or set `QDRANT_URL`). Then run from the repository root: ```sh export OPENAI_API_KEY=sk-... cargo run -p swiftide-examples --example usage-metrics ``` `swiftide-examples` enables many integrations, including LanceDB, Kafka, Fluvio and DuckDB, so the first build takes a while and needs their native toolchains. ## Recorder [Section titled “Recorder”](#recorder) ```rust #[derive(Clone, Debug)] struct PrintHandle(Key); impl CounterFn for PrintHandle { fn increment(&self, value: u64) { println!("counter increment for '{}': {}", self.0, value); } fn absolute(&self, value: u64) { println!("counter absolute for '{}': {}", self.0, value); } } impl GaugeFn for PrintHandle { fn increment(&self, value: f64) { println!("gauge increment for '{}': {}", self.0, value); } fn decrement(&self, value: f64) { println!("gauge decrement for '{}': {}", self.0, value); } fn set(&self, value: f64) { println!("gauge set for '{}': {}", self.0, value); } } impl HistogramFn for PrintHandle { fn record(&self, value: f64) { println!("histogram record for '{}': {}", self.0, value); } } #[derive(Debug)] struct PrintRecorder; impl Recorder for PrintRecorder { fn describe_counter(&self, key_name: KeyName, unit: Option, description: SharedString) { println!( "(counter) registered key {} with unit {:?} and description {:?}", key_name.as_str(), unit, description ); } fn describe_gauge(&self, key_name: KeyName, unit: Option, description: SharedString) { println!( "(gauge) registered key {} with unit {:?} and description {:?}", key_name.as_str(), unit, description ); } fn describe_histogram(&self, key_name: KeyName, unit: Option, description: SharedString) { println!( "(histogram) registered key {} with unit {:?} and description {:?}", key_name.as_str(), unit, description ); } fn register_counter(&self, key: &Key, _metadata: &Metadata<'_>) -> Counter { Counter::from_arc(Arc::new(PrintHandle(key.clone()))) } fn register_gauge(&self, key: &Key, _metadata: &Metadata<'_>) -> Gauge { Gauge::from_arc(Arc::new(PrintHandle(key.clone()))) } fn register_histogram(&self, key: &Key, _metadata: &Metadata<'_>) -> Histogram { Histogram::from_arc(Arc::new(PrintHandle(key.clone()))) } } fn init_print_logger() { metrics::set_global_recorder(PrintRecorder).unwrap() } ``` Source: [`examples/usage_metrics.rs#L91-L170`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/usage_metrics.rs#L91-L170) Swiftide emits its counters through the `metrics` facade from the metrics-rs project. The facade sends them to whichever `Recorder` you install, and without one every counter goes nowhere. `PrintRecorder` is the basic example from metrics-rs: it prints every increment. The example’s `main` calls `init_print_logger()` before it builds the client, because a counter emitted before the recorder exists is lost. In an application you install an exporter crate in its place, such as a Prometheus exporter. ## Client metadata [Section titled “Client metadata”](#client-metadata) ```rust let metric_metadata = HashMap::from([("example".to_string(), "metadata".to_string())]); let openai_client = integrations::openai::OpenAI::builder() .default_embed_model("text-embedding-3-small") .default_prompt_model("gpt-4.1-nano") // Metadata will be added to every metric .metric_metadata(metric_metadata) .build()?; ``` Source: [`examples/usage_metrics.rs#L30-L36`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/usage_metrics.rs#L30-L36) Each entry in `metric_metadata` becomes a label on every counter this client emits, next to the `model` label Swiftide adds. Give each client a label for the tenant, feature or environment it serves, and your metrics backend can sum per label. The method only exists with the `metrics` feature flag. ## Indexing pipeline [Section titled “Indexing pipeline”](#indexing-pipeline) ```rust indexing::Pipeline::from_loader(FileLoader::new(".").with_extensions(&["md"])) .then_chunk(ChunkMarkdown::from_chunk_range(10..512)) .then_in_batch(Embed::new(openai_client.clone()).with_batch_size(10)) .then_store_with( Qdrant::builder() .batch_size(50) .vector_size(1536) .collection_name("swiftide-examples-metrics") .build()?, ) .run() .await?; ``` Source: [`examples/usage_metrics.rs#L40-L51`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/usage_metrics.rs#L40-L51) Each batch of 10 chunks is one embedding call, and each call increments all three counters once. `vector_size(1536)` must match `text-embedding-3-small`. `FileLoader` walks the directory with the `ignore` crate, so it skips hidden and gitignored paths such as `target/`. ## Output [Section titled “Output”](#output) The example keeps the output of one run, embedding with `text-embedding-3-small`, in a comment at the end of `main`. These are its first lines: ```text (counter) registered key swiftide.usage.prompt_tokens with unit None and description "token usage for the prompt" (counter) registered key swiftide.usage.completion_tokens with unit None and description "token usage for the completion" (counter) registered key swiftide.usage.total_tokens with unit None and description "total token usage" counter increment for 'Key(swiftide.usage.prompt_tokens, [example = metadata, model = text-embedding-3-small])': 356 counter increment for 'Key(swiftide.usage.completion_tokens, [example = metadata, model = text-embedding-3-small])': 0 counter increment for 'Key(swiftide.usage.total_tokens, [example = metadata, model = text-embedding-3-small])': 356 ``` The three `registered key` lines appear once, on the first emit. After that, every embedding call prints three increment lines: prompt, completion and total. Embeddings have no completion, so `completion_tokens` grows by 0. The three counters and the clients that emit them are listed in [Counters](/observability/metrics-and-statistics/#counters). ## Langfuse traces [Section titled “Langfuse traces”](#langfuse-traces) The `langfuse` feature flag sends traces to [Langfuse](https://langfuse.com/) through a `tracing` layer: ```sh cargo add swiftide@0.34 --features openai,langfuse cargo add tracing anyhow cargo add tracing-subscriber --features env-filter ``` Set `LANGFUSE_PUBLIC_KEY` and `LANGFUSE_SECRET_KEY`, or their fallbacks `LANGFUSE_INIT_PROJECT_PUBLIC_KEY` and `LANGFUSE_INIT_PROJECT_SECRET_KEY`. `LangfuseLayer::default()` panics when a key is missing or empty. `LANGFUSE_URL` defaults to `http://localhost:3000`. The example also needs `OPENAI_API_KEY`: ```sh RUST_LOG=swiftide=debug,langfuse=debug cargo run -p swiftide-examples --example langfuse ``` ```rust let fmt_layer = tracing_subscriber::fmt::layer() .compact() .with_target(false) .boxed(); let langfuse_layer = swiftide::langfuse::LangfuseLayer::default() .with_filter(LevelFilter::DEBUG) .boxed(); let registry = tracing_subscriber::registry() .with(EnvFilter::from_default_env()) .with(vec![fmt_layer, langfuse_layer]); registry.init(); ``` Source: [`examples/langfuse.rs#L28-L41`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/langfuse.rs#L28-L41) The `EnvFilter` sits on the registry itself, so it filters for every layer, the Langfuse layer included. Without `RUST_LOG` it lets only errors through, and Langfuse receives no spans. That is why the run command sets `RUST_LOG`. [Layer setup](/observability/langfuse/#layer-setup) explains the layer’s own `LevelFilter::DEBUG` and its batching. ```rust let openai = swiftide::integrations::openai::OpenAI::builder() .default_prompt_model("gpt-5") .build() .unwrap(); let paris = openai .prompt("What is the capital of France?".into()) .await?; println!("The capital of France is {paris}"); ``` Source: [`examples/langfuse.rs#L54-L63`](https://github.com/bosun-ai/swiftide/blob/e377c4a2e76fe2559629ae0efdae3c6e0fb4d829/examples/langfuse.rs#L54-L63) With the `langfuse` feature flag, the client’s `prompt` call opens a span of type `GENERATION`. Langfuse shows that generation with the model, the prompt, the answer and the token usage. [Reported data](/observability/langfuse/#reported-data) covers what each client and agent records. ## Variations [Section titled “Variations”](#variations) ### Usage callbacks [Section titled “Usage callbacks”](#usage-callbacks) When you want the numbers in your own code rather than in a metrics backend, set `on_usage` or `on_usage_async` on the client builder. The callback receives the `Usage` of each call and needs no feature flag. See [Usage callbacks](/models/errors-backoff-and-usage/#usage-callbacks). ### Pipeline statistics [Section titled “Pipeline statistics”](#pipeline-statistics) A successful indexing run logs a `Pipeline completed` event with its node counts. [Pipeline statistics](/observability/metrics-and-statistics/#pipeline-statistics) covers it and its limits. ## Gotchas [Section titled “Gotchas”](#gotchas) * `AnthropicBuilder::metric_metadata` takes an `Option>`, so wrap the map in `Some`. The OpenAI-compatible and `AwsBedrock` builders take the map itself. * `metrics::set_global_recorder` errors on a second call. In tests that each install a recorder, use `metrics::with_local_recorder` instead. * The Langfuse layer does not send its last partial batch on exit. A short program can end before it leaves. Keep a clone of the layer and call `flush` before `main` returns, as [Final flush](/observability/langfuse/#final-flush) shows. ## Reference [Section titled “Reference”](#reference) * [`GenericOpenAIBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/openai/struct.GenericOpenAIBuilder.html) (`metric_metadata`, `on_usage`) * [`AnthropicBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/anthropic/struct.AnthropicBuilder.html) (`metric_metadata`) * [`AwsBedrockBuilder`](https://docs.rs/swiftide/0.34/swiftide/integrations/aws_bedrock_v2/struct.AwsBedrockBuilder.html) (`metric_metadata`) * [`Pipeline`](https://docs.rs/swiftide/0.34/swiftide/indexing/struct.Pipeline.html) * [`FileLoader`](https://docs.rs/swiftide/0.34/swiftide/indexing/loaders/struct.FileLoader.html) * [`ChunkMarkdown`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.ChunkMarkdown.html) * [`Embed`](https://docs.rs/swiftide/0.34/swiftide/indexing/transformers/struct.Embed.html) * [`Qdrant`](https://docs.rs/swiftide/0.34/swiftide/integrations/qdrant/struct.Qdrant.html) * [`SimplePrompt`](https://docs.rs/swiftide/0.34/swiftide/traits/trait.SimplePrompt.html) * [`LangfuseLayer`](https://docs.rs/swiftide/0.34/swiftide/langfuse/struct.LangfuseLayer.html) * [Feature flags](/reference/feature-flags/) ## Next [Section titled “Next”](#next) [Hybrid search and reranking](/cookbook/hybrid-search-and-reranking/)